How to train a custom ResNet18 model for Maix-II Dock board

How to train a custom ResNet18 model for Maix-II board using transfer learning in PyTorch and convert the model to AWNN format for inference

IntermediateWork in progress5 hours580

How to train a custom ResNet18 model for Maix-II Dock board

Things used in this project

Hardware components

Seeed Studio Sipeed MAIX-II

Story

Introduction

In this project I will explain the steps I made to convert a ResNet18 model to AWNN for inference in the Maix-II board.

Note. All files, models and scripts used in this project can be found in this drive.

Training the model

The model was trained using the transfer learning example and the dataset can be found in kaggle.

With only 10 epochs the model got a max accuracy of 98, 7%. I will provide the weights of the model and access the colab notebook if you want to load the model I trained.

params with best accuracy

Converting the model

With the trained model you can use the script provided by Neucrack in his tutorial to convert the model to ONNX.

I also provided the model already converted.

Now you can convert the model from ONNX to NCNN using the online converter provided by daquexian. NOTE: Don't use optimizers.

Now you can use the online tool provided by Sipeed to convert the model from NCNN to AWNN (you need to log in to use the converter).

Before you create the zip file for the converter you need to verify if the input layer of the param file is correct, in my case the input layer had missing values.

You need to insert the resolution of the image in the input layer, in my case it was 224, 224 and since is an RGB image it has 3 channels.

Now you can create a zip file with a image folder containg images for tuning the conversion, the bin file and the param file. With the zip file you can convert to an AWNN model using the online tool.

After the conversion is over you can download a zip folder with de.param and.bin files.

Running the model

Now you have a.bin and.param AWNN files, you have to upload them to the Maix-II board, to do this I used MobaXterm, you can see how to use it to acess the files in this project.

But in a nutshell, you have to connect the board via wifi using serial, discover what is the IP of the board using ifconfig, and connect via SSH using the IP.

Now you can create a folder and upload the AWNN files, after that create an empty file with the extension.py and you can do the inference, the script I used is uploaded at the end of the project.

To run the script you have 'cd' to the directory of the inference.py file and run 'python inference.py'. Here are some tests with the board.

1 / 8

Note. The files for the NCNN, AWNN models and zip folders can be found in this drive.

Code

from maix import nn
from PIL import Image, ImageDraw
from maix import camera, display

model = {
    "param":"/root/models/cat_dog/20220223_000824_awnn_cat_dog_model_int8.param",                    
    "bin": "/root/models/cat_dog/20220223_000824_awnn_cat_dog_model_int8.bin"
}
camera.config(size=(224, 224))
options = {
    "model_type":  "awnn",
    "inputs": {
        "input0": (224, 224, 3)
    },
    "outputs": {
        "output0": (1, 1, 2)
    },
    "first_layer_conv_no_pad": False,
    "mean": [127.5, 127.5, 127.5],
    "norm": [0.00784313725490196, 0.00784313725490196, 0.00784313725490196],
}
print("-- load model:", model)
m = nn.load(model, opt=options)

labels = ['cat', 'dog']

while 1:
    img = camera.capture()
    if not img:
        time.sleep(0.02)
        continue
    out = m.forward(img, quantize=True)
    out = nn.F.softmax(out)
    msg = "{:.2f}: {}".format(out.max(), labels[out.argmax()])
    print(msg)
    draw = ImageDraw.Draw(img)
    draw.text((0, 0), msg, fill=(255, 0, 0))
    display.show(img)

How to train a custom ResNet18 model for Maix-II Dock board