We wanted to create a solution for automatic detection of crashes, only using audio signals. The final product could be mounted on traffic lights and enable faster emergency services response times in the event of a crash, due to the immediate alerts of a crash with its exact location.
As a base we took the Arduino BLE sense, on which we implemented our code. The code was mostly put together from example codes that Arduino already provides, specifically the code for sending battery level over Bluetooth was the most helpful. We changed it around so it can send signals that we specify.
We have used the Edge Impulse platform to create a machine learning model for classifying different driving events based on audio recordings. To teach it we provided a database of sounds consisting of emergency vehicle sirens, tire squeals at breaking and crash sounds.
- Data acquisition
We could not record our own data for crashes, so we used a pre-existing dataset of audio recordings of different driving events like normal driving, car crashes, tire skidding and emergency vehicle sirens. The dataset also included some other events like people talking in the car, driving over potholes, listening to music while driving and some others, which we removed as they were not useful for our purpose. Data was gathered from YouTube videos, mostly from dashcam videos. We uploaded the gathered audio samples to Edge Impulse.
The collected data consists of: 22 minutes of emergency vehicle sirens, 12 minutes of normal driving, 9 minutes of crash sounds.
Once the samples were uploaded we cropped some of the samples, so they were all approximately the same duration. Some of the samples also had to be cropped because they included other audio before or after the relevant part of the recording, e.g. a few seconds driving before the crash.
- Creating the impulse
We then created an impulse, which processes the raw data and extracts features and then uses a NN classifier to classify the data.
- The results
When the NN model was created, we tested it. We had to go back to the previous steps and edit the gathered data, retrain the NN model and test it again, until we were happy with the results.
- Generating an Arduino library
Finally, we generated an Arduino Library using our NN model for classifying driving events from audio recordings.
The generated library is then downloaded as a.zip file, which also includes some examples.
Deploying the NN modelWe deployed the model to an Arduino Nano 33 BLE Sense, which already includes a microphone we can use. This Arduino also has Bluetooth Low Energy, which we can use with the ArduinoBLE library to communicate with a mobile phone.
We modified an example from the downloaded Arduino library, adding support for Bluetooth connectivity. When the Arduino detects an event it sends the corresponding number over Bluetooth: 0 / 1 / 2 - driving / crash / emergency vehicle.
The source code for the Arduino app is provided under the code section.
Android appWe developed the Android app using the MIT App Inventor. This tool is useful for quickly creating simple applications, with the use blocks to create the app, you do not need to write any code. The downside of this, is that you are quite limited in what you can achieve.
The applications connects to the predefined device. When the device connects, it displays the status message "Connected". Then the app listens for detected events. It displays the last received event in a label. When a crash or emergency vehicle event is received, the event is added to a list with the event name and the current timestamp. Driving events are only displayed in the label and are not added to the list. If a crash event is received the phone also vibrates.
The app also enables the user to view the map, where the locations of crashes would be shown.
The project“ works as a proof of concept, the final implementation would need to use a different communication method and a server side application, to collect data from multiple sources and display them to the emergency services dispatch centre operators.
Comments
Please log in or sign up to comment.