OraCAN

About the disease

Every year 1 in almost every 60 people get affected by Mouth Cancer. The average age of most people diagnosed with these cancers is 63, but they can occur in young people. Just over 20% (1 in 5) of cases occur in patients younger than 55. The overall rate of new cases of the oral cavity and oropharyngeal cancers has risen over the past 20 years. But during this same time, there has been an increase specifically in oropharyngeal cancers associated with human papillomavirus (HPV) infection in both men and women.

In 2018, there were an estimated 396, 937 people living with oral cavity and pharynx cancer in the United States. From 2008 to 2017, oral and oropharyngeal cancers increased by around 1% per year after dropping for several years. Most of this increase occurred in non-Hispanic white people and in a group of oropharynx cancers related to HPV infection.

It is estimated that 10, 850 deaths (7, 620 men and 3, 230 women) from oral and oropharyngeal cancer will occur this year. After dropping for several decades, the death rate for these 2 diseases increased by half a per cent each year from 2009 to 2018.

The Problem

According to the Oral Cancer Foundation, almost 1 person dies every single hour in the US alone due to Oral Cancer, the most common reason being the inability to classify oral cancer in early stages due to a lack of easily accessible diagnostic centres that can classify oral cancer.

"Historically the death rate associated with this cancer is particularly high not because it is hard to discover or diagnose, but due to the cancer being routinely discovered late in its development. Today (2022), that statement is still true, as there is no comprehensive program in the US to screen for the disease opportunistically, and without that, late-stage discovery is more common." - Oral Cancer Foundation, 2022

Our Solution

An AI-powered system consisting of NVIDIA Jetson Nano and an ordinary Endoscope/Intraoral Camera. The Jetson runs a Python GUI that takes input of patient information and takes a picture of the patient's mouth, once the image is clicked the image is automatically fed to the ML model which is made using Edge Impulse and NVIDIA TAO and has been downloaded locally into the device. Within just a few seconds we will get to see a result displaying the patient information and the image classification analysis.

Data Collection

Not much imagery data related to Oral Cancer was found online, however, I have collected images from various articles on the Oral Cancer Foundation, research articles by the University of Southern California, a survey conducted by a Dental Research College in Kolkata, India and many more. Some of these have been combined in my Kaggle dataset for easy usage.

According to NHS (National Health Service) Scotland, "Squamous Cell Carcinoma is the most common type of mouth cancer, accounting for nine out of 10 cases."; followed by Oral Malignant Melanoma and Lichen Planus.

So for this project, I have collected images of 3 of the most common types of Oral Cancer namely Lichen Planus, Oral Malignant Melanoma and Squamous Cell Carcinoma as shown below.

Model Training

Follow the Data Acquisition guide to upload the dataset and use the Impulse Design guide to learn how to make the ML model inside of Edge Impulse Studio.

For this project, I've kept an image size of 160x160 and have used Transfer Learning and RGB as the Colour Depth for model training.

I have used the pre-trained NVIDIA TAO models to design my ML model. Here's a video showing how to do the same:

NVIDIA TAO Toolkit is built on top of TensorFlow and PyTorch, uses the power of transfer learning while simultaneously simplifying the model training process and optimizing the model for inference throughput on practically any platform. The result is an ultra-streamlined workflow.

In this instance, I've used NVIDIA TAO's pre-trained model backbone of MobileNet V2 (800K params) with a learning rate of 0.01 for 1999 training cycles and the GPU as the training processor in Edge Impulse Studio.

Deployment

After the model has been trained we get to see an accuracy of 82.6% with an average precision of 0.84 and an average F1 score of 0.82.

Once the model has been trained we generate the ML model build for NVIDIA Jetson.

After this step, we will have to download the Python GUI that I've designed from GitHub inside of our Jetson Nano.

After the git repo has been downloaded we will have to install Edge Impulse Linux CLI on our Jetson. Follow the steps below to do the same:

sudo apt-get update && sudo apt-get upgrade -y
sudo apt-get install python3-pip
curl -sL https://deb.nodesource.com/setup_20.x | sudo -E bash -
sudo apt-get install -y nodejs
npm install -g edge-impulse-cli

After the Edge Impulse Linux CLI has been installed execute the commands below in the terminal of the Jetson Nano to start the Python GUI locally:

cd oral-cancer
edge-impulse-linux-runner --download modelfile-nano.eim --force-target runner-linux-aarch64-jetson-nano
chmod u+x modelfile-nano.eim
pip3 install -r requirements.txt

Finally after installing all pre-requisites execute the Python file using:

cd oral-cancer
python3 main.py

Here is what you will see on the HDMI screen attached to your Jetson:

Once the patient information is entered, make sure your USB camera is attached to the USB 3.0 port as that would enable better footage quality and faster processing. Here is a picture showing how the entire setup should look in a dental clinic:

Once the patient information consisting of Name, Age, Gender, Contact Number and Location is fed the GUI then asks for the ML model which is already downloaded, the user will simply have to select it and then click on the Take Picture option which then shoots the image using the USB camera and sends the image to the ML model for classification and lays out an entire report within a matter of few seconds.

Here's a report generated using this project:

Demo

Here's a video demo showcasing the final setup:

Conclusion

In this tutorial, we demonstrated how to build a cost-effective and portable device that can classify the three most common types of oral cancer with an accuracy of 82.6% which is considerably much higher than provided by doctors and dentists for detection of oral cancer. Additionally, since the ML model and the Python App GUI get downloaded inside of the Jetson we wouldn't even need any internet connectivity to run this device and can be used effortlessly in remote areas even with a simple battery supply.

In the near future, I will be working on an extension of this project transforming this into an even more portable device combining the Display (here the HDMI Monitor), Jetson Nano and the USB Camera all attached in a single 3D-printed enclosure thus increasing portability and making it more easy to use for remote villages and rural areas.