Goldfish status test in M5Stack UnitV

Source address https://homemadegarbage.com/ai15 by @HomeMadeGarbage

Here is an examination of whether the goldfish can be recognized from the coordinates by recognizing goldfish in M5StickV/UnitV.

Because the fish are floating in the water when it goes to bed, the coordinates should not change during sleep.

Compose

M5StickV/UnitV and M5StickC are connected by four pin cable to recognize goldfish in M5StickV/UnitV and send coordinates to M5StickC by UART.M5StickC UDP broadcast received goldfish coordinates with WiFi.

Image recognition in the original goldfish model

The goldfish recognition model uses the handmade made before.

Firmware for M5StickV Beta and written in the Kflash GUI.

Goldfish model (kmodel file) written to address 0x600000

M5StickV/Unitv code (maxipy IDE)

Code is Yolo-digit-detector and it was written in the MaixpyIDE.Recognize the goldfish and send the central coordinates (x, y) to UART.

Screen window size 224x224

from machine import UART
from board import board_info
from fpioa_manager import fm
from Maix import GPIO
import sensor,image,lcd
import KPU as kpu

lcd.init()

sensor.reset()
sensor.set_pixformat(sensor.RGB565)
sensor.set_framesize(sensor.QVGA)
sensor.set_windowing((224, 224))
sensor.set_vflip(1)
sensor.set_hmirror(1)
sensor.run(1)

fm.register(35, fm.fpioa.UART1_TX, force=True)
fm.register(34, fm.fpioa.UART1_RX, force=True)

uart = UART(UART.UART1, 115200,8,0,0, timeout=1000, read_buf_len=4096)

classes = ["goldfish"]
task = kpu.load(0x600000)
#task = kpu.load("/sd/model246pic.kmodel")
anchor = (0.57273, 0.677385, 1.87446, 2.06253, 3.33843, 5.47434, 7.88282, 3.52778, 9.77052, 9.16828)
a = kpu.init_yolo2(task, 0.3, 0.3, 5, anchor)
while(True):
    img = sensor.snapshot()
    code = kpu.run_yolo2(task, img)
    if code:
        for i in code:
            a = img.draw_cross(i.x()+int(i.w()/2),i.y()+int(i.h()/2),color = (0, 255, 0),size=5,thickness=4)
            data_packet = bytearray([0xFF,0xD8,0xEA,0x01,i.index(),i.x()+int(i.w()/2),i.y()+int(i.h()/2),0x00,0x00,0x00])
            uart.write(data_packet)
        a = lcd.display(img)
    else:
        a = lcd.display(img)
a = kpu.deinit(task)

M5StickC code (Arduino IDE)

#include <WiFi.h>
#include <WiFiUDP.h>
#include <M5StickC.h>

static WiFiUDP wifiUdp; 
static const char *kRemoteIpadr = "192.168.0.255";
static const int kRmoteUdpPort = 9000; //送信先のポート

const char* ssid = "WiFiのSSID";
const char* password = "パスワード";

static const uint8_t packet_begin[3] = { 0xFF, 0xD8, 0xEA };


void setup() {
  M5.begin();
  M5.Axp.ScreenBreath(0);
  
  Serial.begin(115200);
  Serial2.begin(115200, SERIAL_8N1, 32, 33);

  static const int kLocalPort = 7000;  //自身のポート

  WiFi.mode(WIFI_STA);
  WiFi.begin(ssid, password);
  Serial.println("");

  // Wait for connection
  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
    Serial.print(".");
  }
  
  wifiUdp.begin(kLocalPort);
  
  Serial.println("");
  Serial.print("Connected to ");
  Serial.println(ssid);
  Serial.print("IP address: ");
  Serial.println(WiFi.localIP());
}
 
void loop() {
  if (Serial2.available()) {
    uint8_t rx_buffer[10];
    int rx_size = Serial2.readBytes(rx_buffer, 10);    
    if (rx_size == 10) {
      if ((rx_buffer[0] == packet_begin[0]) && (rx_buffer[1] == packet_begin[1]) && (rx_buffer[2] == packet_begin[2])) {
        Serial.print(rx_buffer[4]);
        Serial.print(":");
        Serial.print(rx_buffer[5]);
        Serial.print(",");
        Serial.println(rx_buffer[6]);
        
        wifiUdp.beginPacket(kRemoteIpadr, kRmoteUdpPort);
        wifiUdp.print(rx_buffer[4]);
        wifiUdp.print(":");
        wifiUdp.print(rx_buffer[5]);
        wifiUdp.write(',');
        wifiUdp.print(rx_buffer[6]);
        wifiUdp.endPacket();  
      }
    }
  }
}

Coordinate plot

It was plotted to receive UDP transmitted from M5StickC (X: horizontal, Y: vertical) by PC.

2/11 14:50～2/12 7:40

2/12 22:48～2/13 9:36

It can be seen that the amount of movement decreases from 1:00 to 5:00 clearly, and it is sleeping.

Conclusion

It came to the conclusion that the goldfish was recognized by the AI camera, and the movement of the moving quantity was monitored from the coordinates, and it was possible to decide the sleeping thing of goldfish.

Next, I want to aim at automatic moving picture by detecting sleeping from this moving quantity.