Calculating Reading Time with TinyML and Arduino Nano 33 BLE

Being a writer born in the seventies, of course I do like paper books but after many years using also a Kindle I’ve got used to some features I don’t want to lose.

What features? remaining reading time for instance. Can such feature be implemented for paper books? How?

First thought (worst thought) was adding pressing a button with every page turns.

Will it work? Sure but it will be kind of annoying and not at all interesting from engineering point of view.

What about using audio instead? Can you teach Arduino how a page turn it is supposed to sound like?

Sure. With a little Machine Learning help.

What do you need for this project?

• Arduino BLE 33 Sense (which has an on board microphone and a powerful Nina B3 chip)
• Edge Impulse free account, which is a platform to simplify machine learning tasks.
• One Adafruit Oled 128x32 screen (to display reading time)
• One potentiometer (to setup book pages)

Circuits

Oled Screen VCC to Arduino 3.3v, GND to Arduino GND, SDA to A4, SCL to A5

Potentiometer to VIN, GND and A0

TinyML with Edge Impulse

After creating your free Edge Impulse account, go to Data Acquisition, Collect Data, Use your computer. Allow browser permissions and Save at least 5 minutes with 2 seconds between every page turn. When you finish, the audio will be uploaded and a temporary popup will let you know when it is ready. Then you will be able to see the data in Training Data. Do the same but now recording room noise.

Now, make a click in the 3 dots, at the right of the first of the 1 min collected data audios and Split the sample. Do the same for all the samples.

Rebalance the data set from Dashboard so you can have 80% in training and 20% in testing.

Now design your Impulse with MFCC, 2000ms windows size, 1000ms window increase. Save. Go to MFCC Generate Features and confirm that Window size is 101. Then go to NN Classifier, 300 training cycles, 0.05 learning rate, 0.80 confidence rating, Noise high, Mask time bands low and mask freq none. Finally, Start Training.

Now you can go to Model Testing and check the global accuracy. If you get a good percentage you are ready to deploy the model to the Arduino. Go to Deployment, Arduino Library. Save the file and add the zip with Arduino IDE, Include Library, Add Zip library.

Then you can load the example nanoble33 microphone and test the model.

You can then download my complete code for this project at Github Reading Time You should change the include line #include <bandini-project-1_inference.h> to use the same.h indicated in the nanoble33microphone example.

Reading time calculation

Let’s say that the book has 100 pages. If I’ve read 2 pages per minute and I’ve still got 50 pages to go, I will end the book in 50/2 x 1 minute. That is 25 minutes.

This is the code to do that calculation

int remainingMinutes=(elapsedTime/pagesCounter)*(pages-pagesCounter)/1000/60;

Note: 1000 to convert from milliseconds to seconds, then 60 for minutes conversion.

What if we avoid all that Machine Learning trouble using a loud sound detection?

There are 2 problems with this approach. The first one is that any loud sound like a door slam or the phone will be counted and the second problem is that doing that is no fun whatsoever. In the video example below, you can verify that only page turn sounds are detected.

3d Printed Case

With Fusion 360 I've made a minimalistic case holder.

You can find the file at Thingiverse. It is just one piece and it can be printed without support.

Demo video

Some troubleshooting

Serial Monitor problems: you may have to switch between port 5 and port 6 for uploading and serial monitor.

Windows 10 compilation problems: Due to file names too long issue, I did create a temp folder at root and added build.path inside preference file Users/Your User/ AppData/Local/Arduino15/Preferences

Edge Impulse versions: if you are exporting new versions, you may have to delete the old one from Documents/Arduino/Libraries or a previous.h file could be used.

Maker counterculture talk

View also

TinyML using Arduino BLE 33 accelerometer data to recover from shoulder injuries.

Interested in other projects related to literature and electronic arts?

• Ted Talk Inventar es tan preciso with english captions
• Haiku eInk Reader
• Book signer
• Klausner Machine based on Roald Dahl story

/// Follow me on Instagram