M5Stickv / UNIT-V Analog Meter Readout

Get an image of an analog meter with a camera and read the figures shown by the meter.

It is easy to do software with software because it changes greatly in the direction of light, and it is easy to do the software.

The lens is often attached to the meter, but the lens will be obstructed by the angle of the light.

The correction of the inclination of the meter was tried by using the character recognition and the like, but finally, it was possible to easily measure the inclination amount by applying a circular circular seal to the meter.

Future issues

It is quite slow when two loops for the loop are rotated by the micropyth.this part is written in C, and it seems to be better to be a module. This time, the parameter adjustment of the filter, etc.carried out many times, and it took a long time to adjust the parameter.

Get the tilt of a color seal

In order to increase versatility, we first examine the tilt of the meter.

At first, I tried to read the meter of the meter and tried to determine the inclination, but it became a specification for the meter used.

As an approach, it is a processing using the image, but it is made to be a specification to use the seal of the color difference and to put it horizontally.

The micropython has a method of retrieving a circle from an image, and if it is processed for each color, it is considered that the central coordinate of each circular seal is acquired, and the inclination of the meter can be obtained.

Something used

Hardware:

Unit V (M5Stack)

Thermal humidity meter

Development environment: maixy IDE

Language development: micropython

An outline of the theory

Examine the inclination of a thermometer

Get image (IMG)

Leave green against img and make it black

Cartoon filter Image ())

Bilateral filter Images ())

High pass filter

Low pass filter

Grayscale

Search for circles from images Image ())

Save the central coordinate of the circle

New image (IMG)

Leave blue against img and turn black

Cartoon filter Image ())

Bilateral filter Images ())

High pass filter

Low pass filter

Grayscale

Find circle from image

Save the central coordinate of the circle

The inclination of the thermometer is calculated by connecting the center of the green circle and the blue circle

Examine the angle of a red needle

Get image (IMG)

Image processing is cut out (img2)

Filter and leave red only

High pass filter

Low pass filter

Cartoon filter

Bilateral filter

Find line segments from images

Calculate the temperature from the difference between the line segment and the thermometer

Program

import sensor, image, time, gc, math

def highPassFilter(img1, limit, rgb):
    for pixel_y in range(img1.height()):
        for pixel_x in range(img1.width()):
            pixel = img1.get_pixel(pixel_x, pixel_y)
            if(limit > pixel[rgb]):
                img1.set_pixel(pixel_x, pixel_y,(0,0,0))
    return img1

def lowPassFilter(img1, limit, rgb):
    rgb1 = (rgb + 1) % 3
    rgb2 = (rgb + 2) % 3
    for pixel_y in range(img1.height()):
        for pixel_x in range(img1.width()):
            pixel = img1.get_pixel(pixel_x, pixel_y)
            if(limit < pixel[rgb1] or limit < pixel[rgb2]):
                img1.set_pixel(pixel_x, pixel_y,(0,0,0))
    return img1

def findAngle(inImg):
    #lines = inImg.find_lines(threshold=1400, x_stride=2, y_stride=2, theta_margin=4, rho_margin=5)
    lines = inImg.find_line_segments(threshold=5000, merge_distance=5, max_theta_difference=2)
    angle = 0
    maxLength = 0
    for item in lines:
        if(item[4] > 4):
            inImg.draw_line(item[0], item[1], item[2], item[3], color=(0,255,0), thickness=1)
            print(item)
            if(maxLength < item[4]):
                maxLength = item[4]
                angle = item.theta()
    return angle

def findCircleXY(img, highpass, lowpass, rgb):
    workImg = img.copy()
    workImg = workImg.cartoon(0.3, 0.2)
    workImg = workImg.bilateral(1, color_sigma=1.0, space_sigma=0.5)
    highPassFilter(workImg, highpass, rgb)

    img.draw_image(workImg,0,0)
    sensor.snapshot()

    print("lowPassFilter")
    lowPassFilter(workImg, lowpass, rgb)
    img.draw_image(workImg,0,0)
    sensor.snapshot()

    workImg = workImg.to_grayscale()
    gc.collect()

    obj = workImg.find_circles(x_stride=1, y_stride=1, threshold=2800)
    print(obj);
    circleX = 0
    circleY = 0
    if len(obj) > 0:
        for index, item in enumerate(obj):
            if(item[2] < 25 and item[2] > 10):
                workImg.draw_circle(item[0], item[1], item[2], thickness=2);
                circleX = item[0]
                circleY = item[1]

    img.draw_image(workImg,0,0)
    sensor.snapshot()
    print("X: " + str(circleX) + ",    Y: " + str(circleY))
    return [circleX, circleY]

#program start
print("0")
gc.collect()
sensor.reset()
sensor.set_vflip(True)
sensor.set_pixformat(sensor.RGB565)
sensor.set_framesize(sensor.QQVGA)
sensor.skip_frames(time = 1000)
clock = time.clock()
clock.tick()

#Recognize Green circle
img = sensor.snapshot()
xy1 = findCircleXY(img, 60, 140, 1)    #for green circle
print(xy1)
#Recognize Blue circle
img = sensor.snapshot()
xy2 = findCircleXY(img, 60, 150, 2)    #for blue circle
print("green: " + str(xy1))
print(xy2)

img.draw_line(xy1[0],xy1[1],xy2[0],xy2[1])
angle = math.atan2(xy2[1] - xy1[1], xy2[0] - xy1[0])
angle = -angle * math.pi    #convert radian to degree
print("C1-C2 angle: " + str(angle))


while(True):
    clock.tick()

    img = sensor.snapshot()
    img2 = img.copy((40,20,img.width(), img.height()-40))
    #img2 = img2.flood_fill(0,0)
    #img2 = img2.cartoon(0.3, 0.2)
    img2 = highPassFilter(img2, 50, 0)    #0:r, 1:g, 2:b
    img2 = lowPassFilter(img2, 100, 0)

    img.draw_image(img2, 20,20)    #writing image to framebuffer to see throughting MaixPy IDE
    sensor.snapshot()    #It should be called to see image through MaxiPy IDE 
    time.sleep(2)

    img2 = img2.cartoon(0.5, 0.5)
    img.draw_image(img2, 20,20)
    sensor.snapshot()
    time.sleep(2)

    img2 = img2.bilateral(1, color_sigma=0.8, space_sigma=1.1)
    img.draw_image(img2, 20,20)
    sensor.snapshot()
    time.sleep(2)

    #img2 = img2.dilate(2, threshold=10)
    #img2 = img2.flood_fill(0,0, seed_threshold=0.05, floating_threshold=0.1, clear_backgroud=False)
    #img.find_edges(image.EDGE_CANNY)
    #img = red2white(img, 100, 0)
    gc.collect()
    findResult = findAngle(img2)
    print("angle: " + str(findResult))
    temp = 0.3889 * (findResult + angle) + 15
    print(str(temp) + "`C")
    img.draw_image(img2,20,20)
    sensor.snapshot()

    time.sleep_ms(500)
    print(clock.fps())

1 recognition of circles

Original image

Cartoon, vibratory, high pass

Low pass

Greakerized Circle Recognition

Same processing for blue

2.Recognition of meter needles

High pass filter

def highPassFilter(img1, limit, rgb):
    for pixel_y in range(img1.height()):
        for pixel_x in range(img1.width()):
            pixel = img1.get_pixel(pixel_x, pixel_y)
            if(limit > pixel[rgb]):
                img1.set_pixel(pixel_x, pixel_y,(0,0,0))
    return img1

Filters all pixels of an image to leave only the specified color within RGB.

The color is specified in < int > RGB.

For R: 0

For G: 1

For B: 2

Highpassfilter (IMG 1, 100, 0)

Then all pixels of the image img1 are scanned and the pixels whose r elements are less than 100 are set to black [color (0, 0, 0)].

Low pass filter

def lowPassFilter(img1, limit, rgb):
    rgb1 = (rgb + 1) % 3
    rgb2 = (rgb + 2) % 3
    for pixel_y in range(img1.height()):
        for pixel_x in range(img1.width()):
            pixel = img1.get_pixel(pixel_x, pixel_y)
            if(limit < pixel[rgb1] or limit < pixel[rgb2]):
                img1.set_pixel(pixel_x, pixel_y,(0,0,0))
    return img1

Filters all pixels of an image to leave only the specified color within RGB.

The color is specified in < int > RGB.

For R: 0

For G: 1

For B: 2

Lowpassfilter (IMG 1, 120, 2)

Then, all pixels of the image img1 are scanned and the pixels whose R or G elements exceed 120 are set to black [color (0, 0, 0)].

Original image

Because the processing is heavy this time, only the necessary parts are cut out and processed.

After cutting, high pass, low pass filter

Cartoon filter

High pass, low pass, cartoon filter

img2 = img2.cartoon(0.5, 0.5)

Bilateral filter

High pass, low pass, cartoon, bilateral filter after passing

img2 = img2.bilateral(1, color_sigma=0.8, space_sigma=1.1)

Write to frame buffer for debugging

img2 = img2.cartoon(0.5, 0.5)
    img.draw_image(img2, 20,20)
    sensor.snapshot()    #framebuffer更新の為に呼ぶ
    time.sleep(2)

You can check the image from the maixpy ide by writing to the frame buffer.

! This method is thought to be a backdrop.

Convenient for jig

It is useful to fix the camera and the subject because it repeatedly carries out the photographing and parameter adjustment.

This time I saw it, but I think that the soft side processing becomes easy if lighting is attached.

Result

If it is the same for the light, it reads fairly stable, but the reading value changes only by a little light.

In QVGA Image It is possible to use the image of the qqvga without the memory shortage, and the qqvga image is used, and it may become somehow somehow, and the micropython has a lot of methods of the image processing system, and there is considerably room to devise. (unlike processing in micropython).

Since the k210 of the SOC used in unit V is a 400MHz dual core, it is likely to be able to be used variously.