Hardware components | ||||||
| × | 1 | ||||
| × | 1 | ||||
Software apps and online services | ||||||
 |
| |||||
 |
| |||||
 |
| |||||
Hand tools and fabrication machines | ||||||
 |
| |||||
This tutorial page is associated with North Carolina State University Electrical and Computer Engineering Senior Design Project 03: TI Machine Learning Robotics for the Fall 2021 and Spring 2022 semesters. The goal of this project is to make machine learning more accessible and test the capabilities of Texas Instrument's robotics systems learning kit also known as the RSLK MAX. This page is one of the two pages, with this page showcasing the accelerometer machine learning capabilities using the RSLK MAX.
Check out our Keyword Recognition Project
The GitHub repository will include optional PCB files as well as some of our examples on Edge Impulse Models and prediction code you may needed to get started: https://github.com/asoakley/TI-ML-Tutorial
Our demo videos can be found here.
1. Set Up The Hardware Platform1.1 Assemble the TI-RSLK MAXThis project will be using the TI-RSLK MAX robotics kit that can be purchased from the Texas Instruments website: https://www.ti.com/tool/TIRSLK-EVM. The robotics kit is easy to build, does not require any soldering, and can be constructed within 15-30 minutes.
Texas Instruments has an assembly guide here: https://www.ti.com/lit/ml/sekp164/sekp164.pdf?ts=1650900966929
The assembly guide has four parts: Part 1 is the construction guide for the RSLK MAX kit, Part 2 is the assembly of the MSP432 LaunchPad (not necessary if the kit was purchased), Part 3 is the disassembly of the RSLK MAX, and Part 4 is the option to attach an OLED screen to the robot. While the guide goes through the construction with all the bells and whistles, some parts can be skipped and are optional for this guide. The bumper switches constructed in Step 1.1 and the 8 channel QTRX line following sensor in Step 1.6 are the optional parts of the robotics kit that can be installed if wanted. The entirety of Part 4 is also optional.
Additional Information about the TI-RSLK MAX can be found here:
https://university.ti.com/programs/RSLK/
1.2 Attach the Sensors BoardWe will only be using the Bosch BMI160 for the accelerometer from the Sensors BoosterPack.
TI has a Sensors BoosterPack that can be purchased here: https://www.ti.com/tool/BP-BASSENSORSMKII.
The Sensors BoosterPacks contain sensors that can be used for TinyML. The Sensors BoosterPack contains 6 sensors; TI HDC2080 temperature and humidity sensor, TI TMP117 ultra-high accuracy temperature sensor with breakout board, TI DRV5055 Hall effect sensor, TI OPT3001 ambient light sensor, Bosch BMI160 inertial measurement unit and BMI 150 magnetometer.
All PCB files can be found here: https://github.com/asoakley/TI-ML-Tutorial/tree/main/PCB
If you are interested in building a custom PCB with additional sensors, see the later part of this page to learn more.
This project utilizes the Arduino IDE to program and flash firmware to the TI-RSLK MAX.
Download and install the latest version here: https://www.arduino.cc/en/software
2.2 Install Support for the MSP432Once Arduino IDE is installed, run the program. Navigate to File > Preferences > Additional Board Manager URLs and paste the link below.
http://s3.amazonaws.com/energiaUS/packages/package_energia_index.json
Navigate to Tools > Boards > Boards Manager. Find and Install “Energia MSP432 EMT RED boards” via the search bar or by scrolling down the list. This should take 10-20 minutes.
2.3 Install the Arduino Libraries for the RLSK and BMI160For ease of access, the libraries needed for both the RSLK and BMI160 have been included on the project GitHub repo at https://github.com/asoakley/TI-ML-Tutorial. The BMI160 is an inertial measurement unit included in TI’s BP-BASSENSORSMKII BoosterPack and our custom sensors PCB. Credit for the BMI160 Arduino library goes to Hanya Zou and can also be found at https://github.com/hanyazou/BMI160-Arduino. If you do not plan on using the BMI160 for your projects, then downloading this library is optional.
To add a.ZIP library in Arduino IDE, navigate to Sketch > Include Library > Add.ZIP Library… Select the file path of the library and click Okay. A message saying “Library added” should appear. To check if the library was added, navigate to Sketch > Include Library and see if the name of the library is listed under “Contributed Libraries.”
2.4 Install the Edge Impulse Data ForwarderThe Edge Impulse Data Forwarder, which is part of the Edge Impulse CLI (Command Line Interface), is the tool used in this project to collect and forward data to Edge Impulse via a serial connection.
Install Python 3 on your host computer.
Install Node.js v14 or higher on your host computer. Be sure to install the Additional Node.js tools (called Tools for Native Modules on newer versions) when prompted.
Install Edge Impulses CLI tools by entering the following in the Command Prompt window of your device:
npm install -g edge-impulse-cli --forceBefore you can start training models, you want to program your device to sample and correctly format data. Code for data sampling can be found in the GitHub repository at https://github.com/asoakley/TI-ML-Tutorial/tree/main/testing%20and%20training. For your own custom sensors or data, “DataSamplingTemplate.ino” is a basic implementation that contains instructions on how to edit it for your projects. The other files are for projects that have been created as examples and are further discussed in Section 5.
Once your data sampling code is ready to upload, use a micro USB cable to connect your robot to the Arduino IDE. Under Tools > Boards, select “RED Launchpad w/ msp432p401r EMT (48MHz)” under “Energia MSP432 (32-bits) RED Boards.” Press the “Upload” arrow to flash the program onto the RSLK. The robot should now be continuously sampling data when it is powered on.
3.2 Connect the MSP432 to Edge ImpulseFirst, the robot must be connected via microUSB to a computer’s USB port.
Type this command in the Command Prompt to access the Data Forwarder:
edge-impulse-data-forwarder
If you are starting fresh, you will be asked to:
Log into Edge Impulse
- Log into Edge Impulse
Select your serial port: “COMx (Texas Instruments Incorporated)”
- Select your serial port: “COMx (Texas Instruments Incorporated)”
Select which project you wish to upload data to
- Select which project you wish to upload data to
Provide names for your sensor axes (e.g. “accX, accY, accZ”)
- Provide names for your sensor axes (e.g. “accX, accY, accZ”)
When connecting to the data forwarder again after logging in and selecting a project, you will only be asked to select your serial port. It is important to note that the MSP432 will have two serial connections: one for UART, and the other for the debug probe. If the wrong port is chosen, the data forwarder will fail to auto-detect the data frequency. In this event, simply restart the data forwarder.
To clean out all previous configurations, enter this command on startup:
edge-impulse-data-forwarder --cleanIf the correct serial port is selected in the setup process, the data forwarder will auto-detect the frequency of the incoming data. This auto-detected frequency can be manually overwritten with your desired frequency of xxxx Hz on startup with:
edge-impulse-data-forwarder --frequency xxxxFor additional information on the Edge Impulse Data Forwarder and the rest of the Edge Impulse CLI, please refer to https://docs.edgeimpulse.com/docs/edge-impulse-cli/cli-overview.
3.3 Data AcquisitionAfter the MSP432 is connected to Edge Impulse, navigate to the data acquisition tab, and ensure the correct device is connected.
Select the desired label for the data about to be recorded (in this case motion).
Set the frequency and time to record.
Select the “Record Sample” button and begin recording data, ensuring there is a good split between training and testing data. This data can be set to be split automatically or can be manually divided.
If data has not been split, it can be accomplished by navigating to the dashboard, scrolling down to the “Danger Zone”, selecting “Perform Train / Test Split”, selecting “Yes” on the dialog box, and entering the requested phrase to perform the split on the data. We recommend an 80/20 train/test split.
3.4 Impulse DesignDepending on the desired functionality of the model, the impulse design will vary. In general, there are three different types of blocks that can be selected that will be used in the creation of the machine learning model: Time Series Data, Processing, and Learning.
For time-series data, the window size, window increase, and frequency can be set. From there, a preprocessing block can be selected, or the raw data may be used. The learning block is then selected, which depends on the desired performance of the model. For classification tasks, like our examples, select “Classification (Keras)”.
Select the “Save Impulse” button to move on to the next step, where preprocessing features can be customized. Based on the characteristics of your data, you can adjust how the data is altered before being passed through the classifier. For example, if using the Spectral Analysis block, you can adjust the cut-off frequency, filter order, FFT length, number of peaks, peaks threshold, and power edges. After this step, Edge Impulse uses the preprocessing functions (or lack thereof) and creates features for all of your data points.
Once the features have been extracted, the ML model can be trained by setting the Neural Network block settings (classification for example). Here, you can set the number of training cycles, the learning rate, validation set size, and also the option to automatically balance the dataset. We recommend at least 100 training cycles with the default validation set size (20%) and learning rate (0.0005). The neural network architecture can also be adjusted when it comes to dense or convolutional layers, the number of neurons in each layer, and the addition of more layers or dropout.
Select the “Start Training” button to begin the training process. This may take several minutes, depending on the number of training cycles, learning rate, and amount of features.
After the model training has been completed, the model demonstrates the training performance in terms of accuracy and loss, a confusion matrix for each label with the prediction accuracy, a feature explorer, and the estimated on-device performance. The Feature explorer can be adjusted to view the key features for the dataset to further refine any outliers or ambiguities.
You may also select a model version with the options being “Quantized (int8)” and “Unoptimized (float32)”. Quantized takes up less flash memory, but Unoptimized may be more accurate if some of your class features are very close together on the feature map. Most of the time the two versions will have similar if not the same accuracy, so Quantized is typically recommended. For an example on the difference that the Unoptimized model can make, see the project in Section 5.1.
3.5 Model TestingModel testing involves running the previously generated ML model against test data, where it then outputs the model testing results. These results show the accuracy via a confusion matrix and feature explorer. While this testing is not representative of live data, it allows for further adjustments to be made to improve the results.
Another tool available to further test and optimize a ML model is Live Classification. This can be accomplished by connecting a desired device, setting the data recording settings, and classify either live data or existing test samples.
After classifying each sample of either live data or test samples, the classification includes the detailed results, showing the timestamps of the sampling, the predictions with their confidence ratings, and the spectral features chart.
3.6 DeploymentOnce the ML model has been trained, tested, and updated to the desired level, the impulse can be deployed. For this use-case, under the “Create Library” section, select “Arduino Library”. There is an option to enable the EON Compiler, utilize this if it improves the memory requirements without significantly impacting the model performance. The available optimizations are also demonstrated, with predicted RAM usage, flash usage, latency, and accuracy.
Select the “Build” button at the bottom of the page, which will download the model as a.ZIP file, making it ready for deployment in the Arduino IDE.
4. Flashing Models Onboard The Robot4.1 Upload Model Library to Arduino IDERefer to Section 2.3 on adding.ZIP libraries to Arduino IDE. After adding the.ZIP file deployed from Edge Impulse, navigate to the Documents section on your computer. From there access Arduino > libraries > {name of your Edge Impulse project}_inferencing > src > edge-impulse-sdk > porting > arduino. Open the file “ei_classifier_porting.cpp” with a text editor and add the highlighted line below. Do this or else the code will not compile.
If you have already done this and are uploading a new version of your model with the same name, you will need to delete the “{name of your Edge Impulse project}_inferencing” folder and repeat the process with the new.ZIP file.
4.2 Flash Live Classification CodeGo to https://github.com/asoakley/TI-ML-Tutorial/tree/main/prediction for live onboard classification programs. The file “PredictionsTemplate.ino'' contains the required code for running the model along with instructions on incorporating your own sensor data and robot commands. Edit the template to fit your Edge Impulse model, sensors, and desired outputs. Plug in the RSLK to your computer with a micro USB cable, make sure your board in the IDE is set to “RED Launchpad w/ msp432p401r EMT (48MHz)” and then press “Upload” to flash the code onto your robot. Compiling the Edge Impulse model will take several minutes. After this process is complete, your robot is ready to run your machine learning model. If the robot is plugged into the computer while running, you may view the serial terminal in Arduino IDE to see the confidence values of each class after every classification.
5. Example Projects5.1 Accelerometer Terrain Recognition5.1.1 Data SamplingTo begin training a model for Accelerometer Terrain Recognition, you need to run a program that samples and sends the data to Edge Impulse via a serial connection with the Edge Impulse data forwarder. For this project, you will download and run “AccelTest_V5.ino'' from the “training and testing folder” of the project Github repository to gather your training data. This example requires you to have the BMI160 and RSLK libraries added to Arduino IDE. In this example, the BMI160 is sampled at 50 Hz. A few different motor patterns such as “Forward”, “Forward/Reverse”, and “Oscillate” are available to allow the robot to interact with and traverse the terrain. They are selected by pressing the side buttons on the MSP432. More movement patterns can be added if necessary.
The pre-trained terrain recognition model used in this project was trained only while the robot was moving straight forward.
5.1.2 Model DesignThe model trained in this project contains five different classes: “Brick”, “Carpet”, “Foam”, “Turf”, and “Wood”. Each class was trained for a total of 3 minutes, ranging from 3 to 10 second sample times for each set of data.
As shown in the figure below, this Impulse uses a 1000 ms window size, 80 ms window increase, a 50 Hz sampling frequency, and zero-padded data. The Impulse also includes a Spectral Analysis block and a Classification block.
The parameters used for the Spectral Analysis block are shown below. The filter type was selected as “none” in order to keep all frequencies found within the data.
The parameters of the Neural Network Classifier block are shown in the figure below. The number of neurons for each layer was chosen by starting with the 33 input features and arbitrarily decreasing towards the 5-class output. A dropout layer was added in the middle to keep the model from over-relying on certain features of data.
After training the model, you can see that the accuracy appears to be 51.5% with the default int8 model. This is because the spectral features for the classes other than “Brick” are relatively close together, as seen below, so the resolution of the int8 model is unable to pinpoint these small areas.
For a higher resolution, the model version was switched to the Unoptimized float32 model (this setting can be seen in the top right corner of the figures below). This slightly increased flash memory and inferencing time, but greatly increased the model accuracy to 96.5% with a significantly lower loss.
Our completed Terrain Recognition Edge Impulse project can be found at:
https://studio.edgeimpulse.com/studio/50843.
Our deployed Arduino model library, called “ei-ti-ml-terrain-recognition-arduino-1.0.22.zip”, can be found in the “libraries” folder of the project Github.
5.1.3 Onboard ClassificationThe onboard classification program used in this project, called “AccelPredictions_V2” samples the BMI160 just like before, but instead of sending that data to Edge Impulse, it is fed into the onboard Terrain Recognition model for inferencing. The program declares a buffer for holding 1 second of accelerometer data, which is equivalent to 50 accelerometer samples. The prediction algorithm is started with the press of the MSP432’s Switch 2, causing the robot to drive forward. With each loop of the algorithm, the robot puts measurements in the accelerometer data buffer. Once the buffer is full, it is fed into the classifier for inferencing. Once the classifier finishes, it will output the confidence values of all the classes. These results can be viewed on the serial port. The class with the highest confidence value is chosen as the final prediction. Depending on which class is predicted, the RGB LED on the MSP432 lights up a color that corresponds to the chosen terrain. In this case, blue is foam, green is turf, white is carpet, purple is wood, and red is brick. Pressing Switch 1 on the MSP432 will enable “backup mode.” After each classification, the robot will back up the distance that it had driven forward. This is useful for when you have limited terrain and don’t want the robot to drive off of it.
[OPTIONAL]: Assembling Custom PCB for SensorsTo consolidate multiple TI Boosterpack sensors into one package, we created a custom PCB that integrates the BMI160 and microphone that can be used for tinyML.
All PCB files can be found here: https://github.com/asoakley/TI-ML-Tutorial/tree/main/PCB
The Custom PCB is designed on Eagle which you can replicate or reiterate using the following Eagle libraries:
https://github.com/asoakley/TI-ML-Tutorial/tree/main/PCB/Eagle%20Libraries
The PCB Gerbers can be manufactured by submitting them here: https://www.makerfabs.com/pcb-prototyping-service.html
Following the dropdowns:
Make sure to get a stencil as it will make assembling the custom PCB much easier. The stencil will help you apply solder paste for your components on the custom PCB.
This video can help you on how to use your PCB stencil: https://www.youtube.com/watch?v=5AyxuuFjZSI&ab_channel=Eslam%27sLab
Submit the gerber files by pressing this button and proceed to checkout as normal.
To get materials for the custom PCB, all of the parts can be purchased from vendors such as Mouser, Digikey, and/or Amazon. The bill of materials can be found here: https://github.com/asoakley/TI-ML-Tutorial/blob/main/PCB/Bill%20of%20Material.xlsx
Additionally, you will need access to a soldering station with an iron and a heat gun, solder paste, solder, tweezers, heat resistant tape, desoldering wick, flux and rubbing alcohol (isopropyl 70% or higher should be fine).
To assemble a blank PCB, it is wise to clean the PCB with alcohol (isopropyl) before soldering. Start with soldering on the SMD components, first using the stencil to apply solder paste.
Next, you can use the tweezer to pick and place the SMD components onto the PCB and apply even heat with the heat gun to reflow the SMD components to the board, this includes all the resistors and capacitors as well as the microphone op-amp.
Afterward, you can solder on the microphone and BMI160 breakout board using heat-resistant tape to hold components down while you solder. The microphone (POM-2242P-C33-R) has a datasheet that must be consulted before soldering it down.
Finally, you can solder on the 2x10 header pins and the 2x3 and 2x6 optional header pins. Again use the heat-resistant tape to hold down the headers, making sure that they are even with the board before soldering.
If you are unsure about how to solder there are plenty of tutorials online, and video sharing platforms such as YouTube have many videos that can be watched.
ResourcesArduino IDE for TI-RSLK MAX
https://www.hackster.io/measley2/robotics-system-workshop-arduino-programming-on-ti-rslk-max-d33faa
Edge Impulse Data Forwarder
https://docs.edgeimpulse.com/docs/cli-installation
Arduino BMI160 Library
https://github.com/hanyazou/BMI160-Arduino
Onboard Classification for MSP432
Alan Zheng
Comments