The Sony Spresense board is a compact, versatile and innovation pack micro-controller device that could be used in various areas of applications including research and teaching at various education levels.
Here we are demonstrating some of it inexhaustive education values and how the platform could be used in teaching of STEM [Science Technology Engineering and Mathematics] or STEAM [Science Technology Engineering Arts Mathematics] most especially for age range 6-14 year old in schools and colleges.
About Sony Spresense Board:The board is compact microcontroller [MCU] board with features like:
- Sony's CXD5602 microcontroller
- 6 ARM Cortex-M4F cores
- clock speed of up to 156MHz
- an integrated GPS
- Hi-res audio output and multi mic inputs
Note: Spresense doesn’t come with WiFi or Bluetooth on-board but I think as add-on.
The Sony Spresense board is made of Main and Extension board,
Main Board:Sony’s CXD5602 Processor
8 MB Flash memory
PCB with small footprint
Dedicated camera connector
GNSS (GPS) antenna
Pins and LEDs
Multiple GPIO (UART, SPI, I2C, I2S)
2 ADC channels
Application LED x 4 (Green)
Power LED (Blue)
USB serial port
The Extension board:3.5 mm headphone jack
Micro SD card
An extra USB port
Multiple microphone pins
For more information, comprehensive and well documented information about this versatile kit, please see the following links:
- https://developer.sony.com/develop/spresense/
- https://www.sony-semicon.co.jp/products_en/spresense/index.html
- https://github.com/sonydevworld
Note: As the time of wriring, it seems there are now a Bluetooth added version of the board [CXD5602GG chipset with 185pin FCBGA]. I will follow up more on this to confirm as I am looking for the Bluetooth feature for projects targeting kids and classrooms uses.
UnboxingSony Spresense Board:
When you received your Spresence development evaluation kits, it should comprise of 3 separate boxes as shown below:
As you can seen from the screenshot above, the boxes are colour coded and should act as guide to what is contained in each of the boxes.
Unboxing them revealed that:
- 1.) Mainboard is housed inside the black coloured box
- 2.) Yellow box contains the Camera modules.
- 3.) The Extension board comes in a white box
I am pretty sure that add-on modules may be contained in colour coded boxes as well. This is a very way of packaging development kits most especially for teaching and learning environment. It could go further by having those boards and add-on module having those colours as well. What a good direction!
The next step is bring the kit components of their boxes as shown below:
Now take out the four (4) stand off and insert them onto the Extension board
Next connect the Mainboard onto the Extension board making sure that things are properly align in right order and orientation. See the screenshot below as a guide:
When done yours should look like the images of mine shown below:
Well-done! We almost there, just now need to add the Camera module since we will be doing some data science as part of our demo programs on computer vision. Lets go but hey! Since we are have just installed the mainboard, I think it is better to complete everything that needs done on this component before address the camera module installation. We need to add a shading seal onto the board. According to the installation guide that "Malfunction might occur if the board is placed under direct or in the proximity of a strong light source, e.g. sunlight or a bright lamp if no shading seal is mounted" I hope you are paying attention to this as well but I have pointed it out here so let do it. See below screenshots for how to achieve the task:
Peel off the black adhesive shading seal (item1 in above picture) and place on the IC4 in Spresense main board (item2 in above picture)
When finished it should like my picture below:
Now we are going to connect the Camera module onto the main board
First find the camera label on the main board, next insert the camera module with the reinforcement side towards the camera label on the board. See screenshot below:
I found the camera module slot a bit brittle and fragile so you need extra care when inserting the cable into the slot. It seems like a Bakelite plastic stuff. Insert the cable gently into position and it should be okay! Once again see below for a guide:
Initial Test Drive
On assembly the board, it is usual procedure to test drive it for functionality to see if everything is okay as it should. Hardware hacking takes lot of trial & error paradigm and there is now one way of achieve a particular solution to be honest.
I felt more comfortable in Linux environment than that of Windows and Marc. I do use Windows and the Mac OS X often and when necessary but Centos 7.0 and Ubuntu has been my main play arena. Solaris has died down for sometime now but I now use OpenIndiana as alternative Solaris. Anyway, We will be using the Ubuntu environment as development platform for this project. I may dabble into Windows 7 but then, this will be pointed to if I have to since my Schools and Colleges excel on Windows platform. Good news here are that all of our software and programming platform are cross-platform and run on Linux, Windows and Mac OS X.
Ubuntu Platform: setup is well documented on the Sony seprense website but for our usage here are what needs to be done:
Quick Test on Arduino IDE:1.) Download and install Arduino IDE from: https://www.arduino.cc/en/Main/Software
I normally downloaded the Zip edition and extract it into my Compilers folder but this is up to you, you are in the driver seat here.
Next add yourself to the dial-out group, this is need for serial port access. see steps below for commands:
$ sudo usermod -a -G dialout $USER
Logout and login again to use the new settings. Now connect the Spresense board to the PC via the USB cable.
Next open the terminal and run the following commands:
lsusb and press enter:
Terminal output should list among other device your attached device. See the red arrow from screenshot above showing that I have got: CP210x UART Bridge / myAVR mySmartUSB light. It quickly reveal that the Sony Spresence board is using CP210x (e.g. CP2102 or other version) and on-board programmer. Good insight!
Next run the follow terminal command for further test on serial port which will tell us port value to use in our Arduino IDE:
dmesggrep | tty or grep | ttyUSB
your result should be as shown in the screenshot below:
from the above screenshot it show that our Sony Spresense board port value is ttyUSB0 (usb-serial port 0). Note down this port value needed for programming the board. Good effort! hey!
Stiil on the Test drive but now let us run the Arduino IDE and do some LED Hello World fashion.
Setting up Spresense SDK:click file and select --> preferences option as show below
Next Copy and paste https://github.com/sonydevworld/spresense-arduino- compatible/releases/download/generic/package_spresense_index.json
into the field called Additional Boards Managers URLs
see below screenshot:
Now go to tools menu and underneath it select the boards and click on the Boards Manager
as shown below:
You can either scroll through the boards list or search for Spresense board just type the first three letter in the boards manager search box and it should display the installation link. See below for my step:
The SDK will be downloaded and install for you board:
Next we need to do firmware update:
this then follow by burning the bootloader for the board:
On the screenshot below show a pop up windows for the next instruction on how to download the bootloader and get it installed for the Arduino environment:
Simply click on go to download page, download the bootloader zip file, then drag and drop it onto the pop-up window shown above. (Note: the firmware can be download from here: https://developer.sony.com/file/download/download-spresense-firmware-v1-1-000
The bootloader will extracted and get installed for your Spresense board as shown in the screenshot above.
Now we do the famous "Hello World" LED blink test drive: Create a new sketch file and Ccpy and paste the follow test code into your Arduino IDE sketch. I named mine SpresenceTestDrive1.ino but you can call it want you want!
void setup() {
pinMode(LED0, OUTPUT);
pinMode(LED1, OUTPUT);
pinMode(LED2, OUTPUT);
pinMode(LED3, OUTPUT);
}
void loop() {
digitalWrite(LED0, HIGH);
delay(100);
digitalWrite(LED1, HIGH);
delay(100);
digitalWrite(LED2, HIGH);
delay(100);
digitalWrite(LED3, HIGH);
delay(1000);
digitalWrite(LED0, LOW);
delay(100);
digitalWrite(LED1, LOW);
delay(100);
digitalWrite(LED2, LOW);
delay(100);
digitalWrite(LED3, LOW);
delay(1000);
}
If everything went well then you should be able to verify, compile and upload the above code to your board: See the screenshots below for my results:
Code is compile and uploaded to the board. The on-board LEDs (4 on board LEDs) are now blinking. See highlighted red box below:
Note: Board was tested from 115200 to 1152000 baud-rate and all worked!
Yes! the board is working in Arduino environment! Wait a minute! That's open up a lot of ideas and new ideas started to spring up my heads about the uses of this board kit:
- What about porting all ESP8266 and ESP32 libraries onto Spresense environment?
- On bare-metal development side, I have worked on Ac6 (System Workbench for STM32) often with various STM32F4 family. See: http://www.openstm32.org/. Add another IDE?
- What about bringing STM32F4 Discovery modules to expand this platform?
- We could also port STMicroElectronics SensorTile library as part of Spresence library/modules
- The list can go on and on. this is C-language in concert with ARM microcontrollers to create an ecosystems of technologies for connected devices.... Cheri o!
- One thing Sony has done cleverly here is by providing the pin-mapping for the board
- so that we could use adapt other library quickly : See Energia IDE doing same here: https://energia.nu/
- Online/JavaScript Environment: 1.) https://github.com/soundanalogous/Breakout & http://breakoutjs.com/
- CylonJs: https://cylonjs.com/ & https://github.com/hybridgroup/cylon
- Go language: https://gobot.io/ & https://github.com/hybridgroup/gobot/
- Ruby: oh yes! see: http://artoo.io/ & https://github.com/hybridgroup/artoo/
- Rust Language: https://github.com/rust-embedded
- more to be added here but see below
Spresense board is ARM M4F x6 cores (This is important ARM MCU family information that we must keep in our head during the porting and programming. The next hurdle to get the low-level pin mapping for GPIO, SPI, I2C, etc...
SpresenseBlock [Scratch Programming]:well this seems to be a place to start our coding session for the younger generation that we want to experiment Spresense board doing various IoT, Robotic, Audio, visualisation, data science and many more.
SpresenseScratch is based on modified Google blockly, Ardublock, BlockDuino that is packaged together as an enhance toolkit of those brilliant technologies.
This coding platform is being heavily adapted and remoulding for the Sony Spresense board. The initial target age group is 6-14 year old but guess what? It here for all ages and who ever want to do combine visual/block and text coding. It has a transitional mode to move learner from block based to text based programming. See below:
Simultaneously block and text code can be edited and update synchronised in real-time. Code be uploaded directly into Arduino IDE or built by a back end Python server and flash into the board directly. There is a web interface that cater for remote/network programming using a modified version of codebender plug-in.
A sample dc motor code flashed onto Spresense board and working fine!
The Repository for SpresenseBlock [ScratchProgramming] is at: https://github.com/sanyaade-teachings/spresenserobotics4steam ; work will continue on the this portal and publish also on Hackster.io, twitter, Vimeo and more....
MIT AppInventor:We had a go at create mobile Apps that could give interactive between connected devices, cloud and edge computing. Once again the target age here are the 6-14 years but the resource can be used by any age group.
Appinventor is an open source package for creating mobile apps using block based programming paradigm. Unfortunately this is only for Android platform, do not worry as we wanted to cover more platforms so we already looking at Cordova (formerly PhoneGap) to give a cross platform i.e. Android, iOS and more..
Here for a short demo we are looking at how to interactive with our Spresense based robot. There many more that can be done in Appinventor but now we want to:
- Uses Bluetooth 4.0 (BLE), which connect and control an Spresense powered Robot to drive it around.
- Voice Control the robot making use of Spresense Audio API and Machine Learning
- Control servos motors and dc motors (support for 9G mini servo and more...)
- Demonstrate how sensor values are read through events, easier for block-based programming
Note: This was previously ported to SensorTile by me. I am basing this porting on that work as a jump-start.
Introduction: MicroPython is both a lightweight implementation of Python 3 programming language. It has been ported to various MCU Boards most especially ARM based MCU board including but not limited to Cortex M4.
Advantages:
All MCU I/Os can be accessed and controlled [MCU GPIOs mapped]
Cross-platform: Works on a Windows, Mac, Linux, ARM, ESP, Raspberry Pi, etc...
Interactive Programming Environment: various possible ways:
- REPL – Connect to the board using serial program (minicom, hyperterminal) to get a Python REPL (read–eval–print loop) prompt, where you can execute Python commands just as you would on your PC.
- Remote script – Pressing ctrl-A in the console will switch to raw REPL mode, where you can send any Python script to the board for it to execute immediately. pyboard.py script is provided to your script to the board.
- From file – The pyboard has a filesystem located in the flash memory of the MCU and a micro SD card slot. When you connect the board to your PC it will show as a USB mass storage device, which can be mounted in your PC. Simply copy your Python script (main.py) in the partition, and it will run automatically. This allows to run scripts without being connected to your PC.
- Via Web: using Mu Editor making it possible to do interactive and remote programming on connected devices over the Internet (OTA, ZeroMq, etc..)
Following the procedures and layout on Micropython portal, I steps was successful. A fork was created for further test. I have not be able to get hold of Sony Spresense pin-mapping in order to route I/O as it should be.
The source code and resource is now available on my github at: https://github.com/sanyaade-teachings/spresense-micropython
Cordova Spresense Framework [Android and iOS App]:I have always wanted something that could be quickly realised for mock-up and prototyping most especially when working with the kids. I have dabbled into and used Evothings framework on many occasions. Even before Evothings was MoSync which was a package fro creating mobile Apps and Maqetta was released by IBM as well. Unlike MoSync and Maqetta, Evothings was entirely based around Cordova to be a JavaScript environment for the Internet of Things and connected devices. See: https://evothings.com/ and https://github.com/evothings It excels very well for live coding for internet of things with its Hyperreload framework.
Elementary Data Science using Spresense Development Board [uTensor]:Well I went for SOD from Symisc please see: https://github.com/symisc/sod/
SOD is a single amalgamated file (that is 1 source file plus 1 header file) making it easier to work-on and experiment with. See it at: https://github.com/sanyaade-teachings/spresense-liteml I called it spresense-liteml (Lite Machine Learning Framework). I will put some demos up here soon but Sony Sprense camera was over filling my folder with so many images. I took a while before I realised that I did not set the limit for the amount of captured images to be stored.
Visual Programming/Data-mining in Orange:This is work in progress and I am drawing to the end of submission for this competition. While Orange 3 is Python machine learning and data science package, I will create a port of it for Spresense board..
Virtual Spresense Assistant (Voice Control) with spaCy and scikit-learn: Same apply here. I will do a micro SDCard package for these.
GitHub Links for the resources used on this project:- spresense-liteml: https://github.com/sanyaade-teachings/spresense-liteml
- spresense-micropython: https://github.com/sanyaade-teachings/spresense-micropython
- SpresenseBlock: https://github.com/sanyaade-teachings/SpresenseBlock
- spresense-inventor: https://github.com/sanyaade-teachings/spresense-inventor/
- spresenserobotics4steam : https://github.com/sanyaade-teachings/spresenserobotics4steam
- spresensedroplet: https://github.com/sanyaade-teachings/spresensedroplet
- spresenseflowcode: https://github.com/sanyaade-teachings/spresenseflowcode
Robot DC Motor Code:
/*
This is a test sketch for the Adafruit assembled Motor Shield for Arduino v2
It won't work with v1.x motor shields! Only for the v2's with built in PWM
control
For use with the Adafruit Motor Shield v2
----> http://www.adafruit.com/products/1438
*/
//Adapting this code for use in Sony Spresense Board to drive DC-Motor Robots
// I have about 8 Adafruit Motor Shields in stock, so let put them into use hey!
#include <Wire.h>
#include <Adafruit_MotorShield.h>
// Create the motor shield object with the default I2C address
Adafruit_MotorShield AFMS = Adafruit_MotorShield();
// Or, create it with a different I2C address (say for stacking)
// Adafruit_MotorShield AFMS = Adafruit_MotorShield(0x61);
// Select which 'port' M1, M2, M3 or M4. In this case, M1
Adafruit_DCMotor *myMotor1 = AFMS.getMotor(1);
// You can also make another motor on port M2
Adafruit_DCMotor *myMotor2 = AFMS.getMotor(2); //I am using Sony Spresense on a two wheel robot my initial testing and modification
void setup() {
Serial.begin(9600); // set up Serial library at 9600 bps
Serial.println("Adafruit Motorshield v2 - DC Motor test!");
AFMS.begin(); // create with the default frequency 1.6KHz
//AFMS.begin(1000); // OR with a different frequency, say 1KHz
// Set the speed to start, from 0 (off) to 255 (max speed)
myMotor1->setSpeed(150);
myMotor2->setSpeed(150);
myMotor1->run(FORWARD);
myMotor2->run(FORWARD);
// turn on motor
myMotor1->run(RELEASE);
myMotor2->run(RELEASE);
}
void loop() {
uint8_t i;
Serial.print("tick");
myMotor1->run(FORWARD);
myMotor2->run(FORWARD);
for (i=0; i<255; i++) {
myMotor1->setSpeed(i);
myMotor2->setSpeed(i);
delay(10);
}
for (i=255; i!=0; i--) {
myMotor1->setSpeed(i);
myMotor2->setSpeed(i);
delay(10);
}
Serial.print("Spresense Robot Going backward");
myMotor1->run(BACKWARD);
myMotor2->run(BACKWARD);
for (i=0; i<255; i++) {
myMotor1->setSpeed(i);
myMotor2->setSpeed(i);
delay(10);
}
for (i=255; i!=0; i--) {
myMotor1->setSpeed(i);
myMotor2->setSpeed(i);
delay(10);
}
Serial.print("tech");
myMotor1->run(RELEASE);
myMotor2->run(RELEASE);
delay(1000);
}
Robot Building and testing sessions:
Summary and Conclusion: A very fascinating and intuitive journey or call it pedagogy research for the Sony Spresense board. I kept asking myself all along what does Spresense mean or stand for? I am curios! Cheri o and many thanks for the opportunity given to be part of this versatile device.
God blesses!!!
Best regards,
Sanyaade
Comments