The RASynPuck Demo, what can you build with the RASynBoard?

Introduction

The RASynPuck demo was created to showcase the RASynBoard and Avnet's Cloud Solution IoTConnect. There are many parts to building out this solution including...

• Customizing the RASynBoard Out-of-Box (OOB) application to compliment playing *.gif images on an IoTConnect Dashboard
• Creating the IMU model in Edge Impulse
• Connecting the RASynBoard to IoTConnect on AWS
• Flashing the configuration to SPI flash and separating the core board from the I/O board
• Creating custom *.gif images showing the motion detected by the RASynBoard's Syntiant NDP120 Nural Decision Processor

Note: If you just want to recreate the RASynPuck demo and don't care about how it was put together, you can just follow the instructions here.

In this blog I want to highlight how the OOB application can serve as a good starting place for any custom RASynBoard solution.

The RASynBoard OOB application is a great place to start your own custom RASynBoard solution. The application implements most of the features you'll require for a connected ML at the Edge solution including...

• Complete integration with the on-board Syntiant NDP120 Nural Decision Processor. Bring your own ML model by dropping the model file onto the microSD card and add a new configuration option in the config.ini file. Additionally, there is a switch statement in the code where you can add custom code to define new behaviors based on the NDP120s inference results.
• Full Edge Impulse support. Connect your RASynBoard to Edge Impulse or capture training data to the microSD card and upload your data into Edge Impulse. Edge Impulse also supports generating NDP120 ML models that you can use on your RASynBoard.
• Cloud connectivity built in. We've added the ability to connect your device to either Avnet's IoTConnect cloud solution or AWS IoT Core. You won't need to rebuild the application as all the configuration items are defined in the config.ini file located on the microSD card. We've created detailed documentation and video walkthroughs to help you successfully connect your RASynBoard to the cloud with minimal effort. We expect to have AWS IoTCore certification before September 2024.
• Flash your final configuration including your ML models to the SPI flash on the core board. This allows you to separate the core board from the I/O board for a battery-operated small footprint deployment.
• Mobil device Wi-Fi configuration over BLE. Renesas provides a free Wi-Fi configuration mobile device application you can use to add or change a Wi-Fi connection to your device.

As you read through the content, I encourage you to think about how you could utilize the RASynBoard and the OOB application to solve a problem you face, or to monitor for events to add value into your custom solution.

If you're not familiar with the RASynBoard you can checkout this Hackster blog.

Customizing the RASynBoard Out-of-Box (OOB) application

The OOB application already had all the features that I needed to detect different events and send up telemetry data for each event. On the IoTConnect side, I used a transformation dashboard widget that will display an image, or a gif file in my case based on the inference index. However, when IoTConnect plays a gif file, the gif will continue to play over and over again. I wanted to show a complete gif animation cycle and then send up an idle telemetry message to force the dashboard widget to show a static image of the RASynPuck fixture.

The code changes include...

• Add logic to suppress NDP120 events after sending an inference telemetry message i.e., Rolling, then wait a period of time and send the Idle telemetry message. This allows the gif images to play to completion on the IoTConnect Dashboard.
• Add a new config.ini setting called [Puck Demo]-->Idle_Delay that defines the time period between when the application sends an inference telemetry message i.e., Rolling and sending the Idle telemetry message
• Change the length of time the RGB LED is turned on when the NDP120 detects and inference event. The new period is the same period between sending inference telemetry i.e., Rolling and the Idle message. This is a good visual indication that NDP120 events are suppressed.
• Add a new configuration item to define the period between sending up the event i.e., Rolling and the Idle message.

I decided to create a new FreeRTOS thread to enforce the timing between sending a motion event i.e., Rolling, and sending the Idle message.

To trigger the timing I added a call to a new function called suppressNdp120Events(), after any inference event, that would set a flag called suppressNdp120Events to true each time I received a new inference event. This routine also sets an event bit that the thread is blocking on. Once the event bit is set, the thread is unblocked, delays and then sends up the idle event and clears the suppressNdp120Events flag.

To make sure we did not send up any event telemetry during the suppress time, I added a check right after we receive a new inference event. If we're suppressing events, then return or give back the mutex and semaphore set by the ndp120 thread. This effectively masks the events, or at least keeps the code from sending telemetry messages up or lighting the RGB LED.

All my code changes are checked into the public RASynBoard GitHub repo under a branch called RASynPuckDemo.

Creating the IMU model in Edge Impulse

The RASynBoard is fully supported in Edge Impulse, that means you can...

• Collect training data directly into Edge Impulse using the Edge Impulse CLI tools
• Create Impulses targeting the Syntiant NDP120 Nural Decision Processor
• Generate NDP120 RASynBoard deployments that drop onto the RASynBoard's microSD card

If you're interested in learning more about creating Edge Impulse models for the RASynBoard, I recommend you work through my self-paced lab document. The documentation walks the student through creating the ML model featured in this demo.

I dropped my Edge Impulse RASynBoard model *.synpkg onto my microSD card and updated the configuration.

Connecting the RASynBoard to Avnet's IoTConnect Cloud

Detecting events using ML at the Edge is great for identifying events, problems or Maintenace issues. When a device is local to equipment you can do things like shut down a motor that's about to self-destruct or turn on a red lamp showing where the issue is. However, when we add cloud connectivity to the solution, we can interface with back-end systems to automatically create a work order, or maybe just send out emails or text messages informing the responsible parties that the system needs attention, or some event triggered.

The RASynBoard OOB application supports connecting directly to IoTConnect on AWS, or just AWS IoT Core.

To connect your RASynBoard to IoTConnect, please follow the detailed instructions in the OOB GitHub project. If you work through this document look for special instructions for creating the RASynPuck dashboard. Here's what the predefined dashboard looks like, it contains details on the model as well as inference counts for each inference index.

Deploying the demo to the core board

Once you have the demo configured and working the way you want, the next step is to transfer the configuration and ML models to the SPI flash on the core board. This allows you to separate the core board from the I/O board and mount it in the RASynPuck demo fixture. The process is not hard, but we need to consider a couple of configuration items.

• AWS x.509 certificates. Since the microSD card is on the I/O board (that we want to separate from the core board) we need to make sure that the x.509 certificates are stored in non-volatile memory. This is easily accomplished by first configuring the application to read the certificates from the microSD card, and running the application. This writes the certificate data into the DA16600's (BLE Wi-Fi Device) non-volatile memory. Then change the certificate configuration in the config.ini file to assume that the certificates are already written to non-volatile memory. Don't forget to restart the application to read the new configuration.
• Wi-Fi configuration. If you plan to take the RASynPuck to different locations where you may need to change the Wi-Fi network configuration, then you should select the config.ini option to use the Renesas Wi-Fi configuration application.

Once these items have been changed and confirmed, the user just needs to press the user button (button closest to the corner of the RASynBoard) for > 3 seconds and the application will store the configuration and ML models into the SPI flash on the core board.

Configure the device to run from just the core board: I have a document detailing this process here.

Creating Custom *.gif images

I used Autodesk Fusion to design the RASynPuck fixture, so I have my model in the tool. To create the animation *.gif files I used the Fusion Animation Feature. Here's a quick overview of the process.

• Open the model you want to animate in Autodesk Fusion
• Select the Animation Workspace

• In the animation workspace you'll see a timeline at the bottom. It's likely already set to the 1 second location in the timeline.
• Move the timeline slider to the 3 second mark
• Now move your model using the Transform feature, make sure you have any component visible that you want to capture.
• I'll rotate my model 110 degrees
• You're telling the tool to animate the 110 degree rotation over 2 seconds (3 sec minus the 1sec starting position)

• Move the timeline marker to the 5 second mark
• Next rotate it -220 degrees so it goes the other way

• And finally, move the timeline marker to the 7 second mark, and rotate the model back to the starting position which would be 110 degrees.

• Now using the player controls at the bottom of the workspace play the animation.

• You can play with the time segments to make the animation faster (shorter time) or slower (longer time)

Here's the result...

I used Camtasia to capture the animation, then output the video as a gif file.

Conclusion

Thank you for taking the time to read through this project. I hope I've been able to convey how development teams can leverage the RASynBoard Out-of-Box project to create new custom applications that help you solve your ML at the Edge issues.

If you have any questions or comments, please post them below and I'll be sure to respond.