Hardware components | ||||||
 |
| × | 1 | |||
| × | 1 | ||||
 |
| × | 2 | |||
 |
| × | 1 | |||
Software apps and online services | ||||||
 |
| |||||
Hand tools and fabrication machines | ||||||
 |
| |||||
 |
| |||||
Sony Spresense based action camera:
Features:
- video recording (AVI + MJPEG)
- audio recording (MP3)
- location recording (GPS)
- speaker
- SD card support
- USB mass storage
To get started with the Sony Spresense board with the Arduino IDE, we can follow the Spresense Arduino Library Getting Started Guide from the Sony website.
It's pretty much the standard procedure for new boards: install drivers + install the board from the Boards Manager.
The board library comes with a bunch of examples, featuring most of the features of the Spresense platform:
To test the camera, the Camera -> camera example sketch can be used.
The sketch first initializes the camera API in the setup() function:
... theCamera.begin();
... theCamera.startStreaming(true, CamCB);
...
theCamera.setAutoWhiteBalanceMode(CAM_WHITE_BALANCE_DAYLIGHT);
...
theCamera.setStillPictureImageFormat(
CAM_IMGSIZE_QUADVGA_H,
CAM_IMGSIZE_QUADVGA_V,
CAM_IMAGE_PIX_FMT_JPG); ...
Then, in the in the loop() function a picture is taken every second. The first 100 images are stored on the SD card (if present):
...CamImage img = theCamera.takePicture();...if (img.isAvailable()) { ...save image}...
Sample image:
There are different types of microphones that can be used with the Spresense board. I used electret microphones because of they low price and high availability:
The microphones need to be connected to the JP10 header. Their positive terminal needs to be biased with 2.2 kOhm resistors, like follows:
I just soldered the resistor directly to the 2.54 mm connectors:
When the connections are done, we can test the setup using the Audio -> application -> recorder example sketch:
This records sound and saves it to the SD card. In the setup() function the audio library and the SD card is initialized:
... theAudio = AudioClass::getInstance();
theAudio->begin(audio_attention_cb);
...
theAudio->setRecorderMode(AS_SETRECDR_STS_INPUTDEVICE_MIC);
...
theAudio->initRecorder(AS_CODECTYPE_MP3, "/mnt/sd0/BIN", AS_SAMPLINGRATE_48000, AS_CHANNEL_STEREO); ... theAudio->startRecorder();
...
/* Open file for data write on SD card */
myFile = theSD.open("Sound.mp3", FILE_WRITE);
In the loop() function we can record audio frames and save them directly to an MP3 file:
/* Read frames to record in file */
err = theAudio->readFrames(myFile);
For electret microphones you will probably need to increase the gain. This can be done by calling the setRecorderMode function with an additional parameter:
theAudio->setRecorderMode(AS_SETRECDR_STS_INPUTDEVICE_MIC, 210 /* +21.0 dB */ );
Note: for examples the uses MP3 encoding / decoding the appropriate DSP must be installed to the SD card. This can be done using the example sketches from the Audio -> dsp_installer.
The speaker I used is a 23 mm diameter / 8 Ohm one:
First, I tried to connect it to the JP8/9 header, but it didn't worked. After some reading the documentation, it turned out that in order to use the speaker output some tiny SMD components needed to be replaced on the main board.
In the end, I give up that idea and I connected the speaker directly to the headphone jack socket pins:
The Audio -> application -> player example sketch work similarly to the recorder one, but instead of recording sound to a MP3 file, it plays back sound from a MP3 file.
The audio library initialization is done similarly in the setup() function:
theAudio = AudioClass::getInstance();
theAudio->begin(audio_attention_cb);
theAudio->setRenderingClockMode(AS_CLKMODE_NORMAL);
theAudio->setPlayerMode(AS_SETPLAYER_OUTPUTDEVICE_SPHP, AS_SP_DRV_MODE_LINEOUT); err_t err = theAudio->initPlayer(AudioClass::Player0, AS_CODECTYPE_MP3, "/mnt/sd0/BIN", AS_SAMPLINGRATE_AUTO, AS_CHANNEL_STEREO);
theAudio->setVolume(-160);
theAudio->startPlayer(AudioClass::Player0);
The loop() function, frames from the MP3 files are read and sent to the speaker:
int err = theAudio->writeFrames(AudioClass::Player0, myFile);
To try out GPS functionality, we can upload and run the GNSS -> gnss example sketch.
The sketch prints status information on to the Serial port. It needs some time, but in the end it successfully determines our position and the current time as well:
numSatellites: 5
[ 0] Type:GPS, Id:13, Elv:48, Azm:173, CN0:22.059999
[ 1] Type:GPS, Id:17, Elv:50, Azm: 93, CN0:26.789999
[ 2] Type:GPS, Id:19, Elv:44, Azm:137, CN0:27.299999
[ 3] Type:GPS, Id:28, Elv: 0, Azm: 0, CN0:16.340000
[ 4] Type:GPS, Id:30, Elv: 0, Azm: 0, CN0:15.549999
2019/02/03 14:10:00.000586, numSat: 5, Fix, Lat=46.78xxxx, Lon=23.62xxxx
2019/02/03 14:10:01.000579, numSat: 5, Fix, Lat=46.78xxxx, Lon=23.62xxxx
2019/02/03 14:10:02.000586, numSat: 5, Fix, Lat=46.78xxxx, Lon=23.62xxxx
The USB -> UsbMsc shows how to expose the content of the SD card over USB, as the Mass Storage device:
if (!SD.begin()) {
Serial.println("SD card is not present");
}
if (SD.beginUsbMsc()) {
Serial.println("USB MSC Failure!");
} else {
Serial.println("*** USB MSC Prepared! ***");
Serial.println("Insert SD and Connect Extension Board USB to PC.");
}
The USB port used is the one from the extension board.
8. 3D Printed EnclosureAfter testing all the components, I designed and 3D printed a case.
The 1st version turned out a little bit tight and rough on the top.
For the 2nd try, I made some adjustment in the CAD model and printed it with the front face looking up. I turned out much better:
Tools needed: hot glue gun, soldering iron.
To record video, we must encode / compress the camera / image frames, otherwise the output file will be huge. Some popular video encoding / compression algorithms are the MJPEG, MPEG1/2/4, etc.
Because of the limited memory of the Spresense (~1.5MB SRAM) the only viable option is to build up an MJPEG encoded video file from the JPEG frames of the camera. The advantage of the MJPEG is that each image frame is individually encoded as JPEG image. The Spresense camera module has a built in JPEG encoder, so it can provide the image frames directly in JPEG format.
The last thing we need for video recording, is a container file format like AVI. An AVI file is built up from chunks of different type. We will have a header with some meta information (file type, width, height, encoding, etc) and then the JPEG encoded image frames.
I implemented an AVI MJPEG encoder based on an ArduCAM example. It is a AviMJPEG class with a constructor and 3 methods:
writeHeader()- write the header of the AVI filewriteFrame()- writes a JPEG encoded framewriteEnd()- completes some values left empty in the header
The initial idea was to call the .writeFrame() method from the real-time image callback:
void CamCB(CamImage img) {
/* Check the img instance is available or not. */
if (img.isAvailable()) {
video_frames_count++;
aviMjpeg->writeFrame(img.getImgBuff(), img.getImgSize());
//if (video_frames_count >= someNumberOfFrames) {
aviMjpeg->writeEnd();
video_frames_count = 0;
// start new video file
//}
} else {
Serial.print("Failed to get video stream image\n");
}
}
but unfortunately the CAM_IMAGE_PIX_FMT_JPG is implemented just for still image right now in the Spresense camera library (Spresense Developer forum question).
So, as a temporary solution I'm using theCamera.takePicture() to get JPEG frames. It works, but the FPS is a little bit slow (1-3 FPS):
The loop function:
void setup()
{
Serial.begin(115200);
while (!Serial) ; /* wait for the serial port */
Serial.println("Seting up the camera.");
setupCamera();
Serial.println("Seting up the audio system.");
setupAudio();
Serial.println("Seting up the GPS system.");
setupGps(); Serial.println("Seting up the USB mass storage.");
setupUsb();
}
The setup function:
void loop()
{
loopCamera();
loopAudio();
loopGps();
loopUsb();
}
- Camera library: make JPEG work in real-time mode
(Spresense Developer forum question) - add Sound feedback
(start recording, end recording, errors) - add Bluetooth connectivity + mobile app
- add Accelerometer
(for start recording automatically)
- add Battery
(maybe a phone battery + charging module)
- GPS track visualization
Cheers! :)
_3u05Tpwasz.png?auto=compress%2Cformat&w=40&h=40&fit=fillmax&bg=fff&dpr=2)
Comments