Engine Speed Recognition based on Exhaust Sound

1. Problem statement

Many aware greenhouse emissions are harmful to environment, but little know about the noise pollution induced by on-road vehicle, by engine sound, drivetrain, exhaust, tyre, brake, wind noise etc. Imagine an onboard monitoring system which map out driving behavior and the degree of pollution by listening to exhaust, tyre, wind noise. Not only this info is fed to driver to remind how gentle driving behavior will lead to lower noise pollution, but it also predicts the degree of wear of tyre, drivetrain and so on. Apart from that, this onboard monitoring system also provide scoring on the level of noise pollution, to local authority or insurance agency (if driver consensus) to give discount/incentive for lower noise induced because driver can make difference by selecting Electric Vehicle, using lower noise tyre, better maintained vehicle condition, and most critical the driving behavior. With this system in place, driver can contribute together to minimize noise pollution to environment as well as the risk on road.

2. Scope & Assumption

There are several processes to be achieved here:

• To recognize the sound for determining what event is that
• To score the event based on recognition result
• To predict known failure in time-series recognition

The scope of this project covers limited condition due to multiple reasons, such as:

• Dataset is limited to only one type of vehicle (in fact only one vehicle)
• Dataset is collected only when the vehicle is not moving due to safety concern because one cannot drive the vehicle & manage the project at the same time.
• Sensory input (audio) is targeting only the exhaust sound.
• Machine learning pipeline runs only M4 core.
• Limited classification output from the SensiML knowledge pack due to Community account.
• SensiML open gateway to be used as GUI due to limited knowledge from the participant to develop front-end application.

Given the scope defined above, the assumption is made that this to serve as prototype study instead of a real-life use case deployment, aiming to enable the participant to explore the use of machine learning technique on an edge device.

1. Proposed solution

The prototype consists of several components as shown in Figure 1.

• QuickFeather by QuickLogic
• ESP32 by NodeMCU
• Li-ion battery 18650

With several other software packages below.

• QORC-SDK by QuickLogic
• ESP-IDF by Espressif
• Data Capture Lab, Analytics Studio, open gateway by SensiML

The connection between QuickFeather and ESP32 is using UART with a simple schematic illustrated in Figure 2.

The plan installation point is to be as close to the exhaust as possible without exposing the prototype to heat from exhaust & water splash etc., shown in Figure 3.

3.1 Proof-of-concept with Test Dataset

There are plenty of effort reading in the datasheet, repo, forum etc. prior to building the project, the process is not without challenge given the new experience on a totally different platform. First off, a proof-of-concept project is built with a simple objective to recognize human voice, to differentiate between character “A”, “B”, C”, “D” and not speaking. The audio datasets are collected in DCL and being segmented manually with length size fixed at 8000 data points with the characters as labels. Further on, in the Analytic Studio, the project is assembled using AutoML pipeline of window size 400 and sliding for 20 feature cascades. The most recommended feature for voice recognition (based on research and survey from internet and forum) will be Mel-Frequency Cepstral Coefficient (MFCC). Therefore, frequency feature is being selected in the AutoML setting. The training result is remarkably impressive, scoring 99% in both the accuracy and sensitivity. The pipeline generates 4 features, i.e., MFCC, Dominant Frequency, Spectral Entropy and Peak Frequencies. And the features selector is using information gain (IG) to select the feature based on the differences between classes. The pipeline ends with Stratified K-Fold Cross-Validation following the feature vector min max scale transformation. Lastly, knowledge pack is downloaded and being compiled in QF_SSI_AI_APP and flash to the QuickFeather.

SensiML open gateway is used to verify the result as shown in Figure 4. The recognition result, however in reality is not as good as the training result. The sensitivity is good while the accuracy is probably 50% to 80%. There are several possible reasons for that, the background sound/noise is affecting the recognition, the dataset is underfitted/overfitted, the model is not optimized, and many other things could go wrong. One of the suspects is, when a character is being pronounced for extended period, the audio waveform can exhibit similar shape, pattern & frequency, for instance, “B” & “D”. Perhaps there are some better technique to catch that, this will be a KIV topic to study in future.

Remarks: Due to the community account, the classification output is limited to 1000 output per power cycle. The audio sensor is configured at 16 kHz and the sliding window size is 8000 data points, it takes only 500 second to reach the maximum output limit.

3.2 Actual Run with Real Dataset

The real dataset is segmented and labelled with the following conditions listed in table 2.

Similar approach is used to build this pipeline. Window size is set at 400 and 10 feature cascades and frequency being selected as the feature generator in the AutoML pipeline. The training result seems acceptable even though there are some outliers in the dataset, as shown in Figure 5.

There are some confusions between some classes, “EngIdle_AC_OFF”, “Eng1k” & “Ramp_DOWN”. Besides, “Ramp_DOWN” & “Ramp_UP” are also having fairly low accuracy issues, shown in Figure 6.

The pipeline is using MFCC & Spectral Entropy as feature generators and IG as feature selector, in which Min Max Scale transformation will be applied prior to feeding it to the model. The model is using Pattern Matching Engine (PME) classifier and optimized by hierarchical clustering with neuron optimization, shown in Figure 7.

Lastly, knowledge pack is compiled with the QF_SSI_AI_APP and loaded to the QuickFeather and streaming to the open gateway via ESP32. The prototype is located as shown in Figure 3, and classification result is being streamed to the laptop at the driver seat, with a steady foot controlling the gas pedal. The classification output is mainly showing “EngIDL_AC_OFF” & “Eng1K” when the engine is idling (about 800 rpm) shown in Figure 8. One of the possible reasons is the exhaust sound between 800 rpm and 1000 rpm are very similar.

Nonetheless, the classification for engine speed above 1000 rpm are remarkably impressive, as shown in Figure 9. It can recognize the correct engine speed when the engine is hovering at a constant rpm. This demonstrate the capability of edge device (QuickFeather) which flash with the SensiML knowledge pack. Best yet, the knowledge pack is built by the AutoML pipeline, without much code cracking or intensive knowledge in data science.

The recognition when engine is ramping up or down is quite random. This is probably something related to the dataset. The dataset is being collected in a garage environment and if there are better resources and environment such as, engine being controlled by ECU to perform a preset speed profile, the dataset will be very much cleaner and improve the model significantly.

The sample video is posted here.

4. Conclusion

The project is a semi-finished product as it only achieves part of the objective, which is to recognize the noise pollution induced by the on-road vehicle and provide feedback to the user. The recognition part is completed even though the scope is not as big as the initial plan to also detect the drivetrain, wind noise etc. It requires multidisciplinary skillsets & resources such as electronic design, script coding etc. to complete the full project which is a constraint in terms of time and knowledge at the time of writing. Therefore, the project is being wrapped up to document all the efforts and studies along the process. Lastly, this project has been so much fun despite the steep learning curve, and what really exciting is, the existence of QuickFeather and SensiML is enabling a new playground for hobbyist & kickstarters to explore ML on Edge devices.