Stream & Visualize MATRIX Creator Sensor Data with Swim.ai

Required Hardware

Before getting started, let's review what you'll need.

• Raspberry Pi 3 or 4.
• MATRIX Voice or MATRIXCreator - Raspberry Pi does not have a built-in microphone, the MATRIX Voice&MATRIXCreator each have an 8 mic array - Buy MATRIX Voice or Buy MATRIX Creator.
• Micro-USB power adapter for Raspberry Pi.
• Micro SD Card (Minimum 8 GB) - An operating system is required to get started. You can download Raspbian and use the guides for Mac OS, Linux and Windows in the Raspberry Pi website.
• A USB Keyboard & Mouse, and an external HDMI Monitor - we also recommend having a USB keyboard and mouse as well as an HDMI monitor handy. You can also use the Raspberry Pi remotely, see this guide from Google.
• Internet connection (Ethernet or WiFi)
• (Optional) WiFi Wireless Adapter for Pi 2. Note: Pi 3/4 has built-in WiFi.

Let's Get Started

This guide will go over setting up a Swim.ai client and server to visualize data from a MATRIX Creator.

The project demo is meant to serve as a boilerplate for users to build off of.

1. Clone Example Repository

Install git if you don't already have it on your Raspbian OS.

sudo apt install git

Clone the demo repo.

git clone https://github.com/swimos/matrixOne

Enter the cloned git repo directory.

cd matrixOne

Run the following command to make all bash scripts executables with the appropriate permissions before moving onto the next steps.

chmod +x setup.sh runSwim.sh runBridge.sh stopAll.sh server/gradlew

Raspbian Stretch Workaround (Move on to Step 2 otherwise)

IMPORTANT:Only follow the steps below if you are on Raspbian Stretch

Check which version of Raspbian you are on by typing the following command in your terminal.

lsb_release -a

We need java 9 or higher for Swim. On Raspbian Stretch, we also need to add a new source in order to be able to install the correct version of Java. Follow the instructions below to add the new source.

Add deb source url to source list

echo "deb http://deb.debian.org/debian stretch-backports main" | sudo tee /etc/apt/sources.list.d/debian.list
sudo apt update
sudo apt install dirmngr

switch to root and add key for deb.debian.org

sudo su -
gpg --keyserver keys.gnupg.net --recv-keys 7638D0442B90D010 8B48AD6246925553
gpg --armor --export 7638D0442B90D010 | apt-key add -
gpg --armor --export 8B48AD6246925553 | apt-key add -
exit

2. Set Up Pre-Requisites

Run the setup.sh script which installs all pre-requisites including Java9+, NodeJS, and required MATRIX packages. This may take a few minutes as many packages are being installed.

./setup.sh

Once the setup script has finished running without errors, reboot the Raspberry Pi to initialize your MATRIX device.

sudo reboot

3. Run the Server

The swim server hosts the webpages for data visualizations and can be viewed through your browser.

Once booted back into the Pi, enter the project directory once again.

cd matrixOne

Run the swim server application with the following script.

./runSwim.sh

Keep an eye on error logs and wait until the logs show "Running Matrix One plane..." as this indicates the server has successfully been compiled and Swim is running.

tail -f swim.log swim.err

Once the server is running, Ctrl+C out of the logs. At this point, you should be able to see a local website on your PC's browser with the URL http://YOUR_PI_IP:9001.

4. Run client side

The client sends sensor data to the server, acting as a bridge between your device's data and webpage visuals.

Run the client side code with the script below.

./runBridge.sh

Check logs and errors for the bridge with the following command.

tail -f bridge.log bridge.err

At this point, everything should be working, navigate back to your browser and explore the various data visualizations available.

Gyroscope visualization: http://YOUR_PI_IP:9001

All sensor data with history charts: http://YOUR_PI_IP:9001/sensors.html

Color for MATRIX Creator LED animation: http://YOUR_PI_IP:9001/colorPicker.html

5. Stopping the app

Kill the app with the following command (kills both java and node).

./stopall.sh

6. Building off of this example

Basic App Folder Structure

• /bridge - code for data bridge from MATRIX Lite to Swim. See bridge.js
• /server - Swim Java Application codebase
• /templates - collection of JSON files which define various page layouts
• /ui - codebase for web based UI

Swim Application Structure

• Main entry point is server/src/main/java/swim/matrixone/MatrixOnePlane.java
• WebAgents live in server/src/main/java/swim/matrixone/agents
• server/agents/SensorAgent.java is the main WebAgent in the app. Each sensor on the Creator will have its own WebAgent (digital twin) in the Swim Application.

A good place to start playing around with the code could be changing the LED animations.

Check out Swim's website and tutorials to jump start your next data streaming project!