TCM(Traditional Chinese Medic) Intelligent Aided Recognition

Traditional Chinese medicine uses information such as facial appearance, tongue coating, and pulse to comprehensively judge a person's illness. However, accurate judgment requires a doctor to have a lot of diagnostic experience. In order to convert the rich judgment experience of doctors into recognition models, this project establishes a facial recognition model to assist in judging the disease. It should be noted that this project is only aimed at the relationship between facial appearance and illness. Below I will introduce the basic principles of the relationship between the following phases and the disease: the human face is not only a part with rich muscles and sufficient blood supply but also a part where cranial nerves are concentrated. Therefore, Chinese medicine believes that many diseases show a typical facial appearance accordingly. We will show how to construct of TCM auxiliary diagnosis model and run the model on the KV260.

1. Dataset introduction

Under the guidance of experienced traditional Chinese medicine practitioners, we divided the collected face photos into ten categories (there are far more than ten categories of facial features in traditional Chinese medicine, but due to limited time, we only collect ten categories of work). The faces of these ten categories correspond to different disease information, and the specific classification information is shown in the following table. However, it should be noted that due to privacy concerns, it is not convenient for us to disclose the collected data.

2. Preparation conditions

Make sure you have the following work-ready：

• You should have a computer with at least 32G of RAM.
• Install the pycharm software and install the basic deep learning package.
• Install Vitis AI: https://github.com/Xilinx/Vitis-AI （release 1.4）.

3. Train model

Since model training involves real patient face data, the training process is not elaborated. We put the trained model file in the code file, which can be obtained by ourselves.

4. Environment configuration

• Install the system image xilinx-kv260-dpu-v2020.2-v1.4.0.img.gz
• Install the development board AI runtime environment.

sudo dnf install packagegroup-petalinux-vitisai

5. Compile the model

• Enter the vitis-ai docker and activate the pytorch conda environment
• Add Resnet18 network definition file common.py

• Add accuracy-test code to model quantization evaluation.

• Run run.sh to compile DNN and output the executable xmodel file of the development board.

• The compilation process is as follows:

6.Inference on the KV260

• Put the compiled target.tar.gz file on the KV260 and analyze the emotion dnn network results and export the final layer weight parameters to a file.

• DNN output layer CPU calculation, add CPU-side inference calculation code in application.py.

• Execute the command to test: python3 app.py, you can see that the inference accuracy is 86%, and there are 3 computational graphs in the DNN running on the DPU.

• The final auxiliary diagnosis effect is as follows. It should be noted that the model constructed this time is only a preliminary attempt, and the model needs to be optimized under the further guidance of traditional Chinese medicine doctors.