Multiple Safety Service For Electric Scooter

Summary

0. Problems

In Korea, electric scooter accidents are constantly increasing because of dangerous driving habits, neglectful driving on the pothole or speed bump, etc. Also, not only does illegal parking cause inconvenience to pedestrians, but towing from illegal parking causes financial damage to the company.

To solve these problem, the government has proposed various solution such as strengthened regulations and reporting systems about electric scooter, but the problem has not been solved yet.

Thus, we need a new solution to solve this problem and make a sustainable electric scooter service.

1. Electric Scooter And Device

We use electric scooter connected to Raspberry Pi with GPS module, Gyro sensor, Camera module, and LTE module. We transmit the data collected on the scooter to the Mobius server via HTTP communication using the LTE module. The data to be transmitted are GPS, Gyro data, and image.

The upload_gps folder in Raspberry Pi receives NMEA data from the GPS module installed on the electric scooter. And Raspberry Pi uploads this data to the Mobius erver using HTTP communication.

The upload_gyro folder in Raspberry Pi receives acceleration and angular velocity data from Gyro sensor on the electric scooter. And Raspberry Pi uploads this data to the Mobius Server using HTTP communication.

The upload_image folder in Raspberry Pi encodes the image from camera module on the electric scooter into Base64 and transmits them to the Mobius server using HTTP communication.

2. Before Riding

2-1. Register & Login

1) WEB

This is our web page for administrator. We made this using JavaScript, html, CSS.

The administrator can manage risk factors on the map and user information.

Log in to the web page. If the administrator ID and password are correct, you can enter the main page.

The map displays the location of the electric scooter, potholes, speed bumps, school zones, and parking space for electric scooter. By clicking All(Pothole, Speed bump, School zone), Pothole, Speed bump, and School zone, you can see the marker only you want.

2) APP

This is our application for user. We made this using Android Studio with Java and Kotlin.

First, new member must register their own information to use this app. If you're new member, your information is stored in Firebase and you can see a notification which is "Register Success". If your information is already registered in the database, you can't register again.

Second, log in to the app. If user ID and password are correct, you can enter the app. Also, you can see your information in Account page.

2-2. Electric Scooter License

Before scooter riding, you must take written test to confirm your driving ability.

The test can take in the History page. Enter your information and take the test.

The left screen is one of the question. If you get more than 80 points, you can get a scooter license. If you don't, you fail.

You must get this license to drive electric scooter.

3. Riding

If you have logged in to the APP and passed the written test, you're ready to ride.

By clicking the "Start Ride" button on the Home page, you can start ride. Also, if you want to end ride, press the "End Ride" button. After riding, you can check your driving time and price on the History page.

3-1. Risk Detection

We detect [ speed bump and pothole] using Image Detection. We use YOLOv5 (You Only Look Once Version5) to build image Detection. This can be used to maintain the road by using training set related to the state of road.

We gather pictures from the camera on the scooter while driving, and then analyze them using YOLOv5 with GPS on the scooter.

In Mobius Server, data/gps and data/image containers receive a cin value in real time. Decode the image stored in the data/image container to base64 and execute image detection on the server every second. When image detection is complete, save the data in map/pothole/gps and map/pothole/image container.

1) Speed Bump

Speed bump becomes a risk factor depending on the time of day. The user can see the speed bump and slow down in daytime. However, it is difficult for users to identify the speed bump at night. Therefore, the possibility of accidents caused by speed bumps increases.

We can guarantee user's safety by detecting the speed bump in real time.

Before detecting the speed bump, there is no speed bump marker on the map. As soon as you pass the speed bump, you can check the speed bump marker on the map.

This is another speed bump detection result. You can see the picture on the right side in the map/speed_bump/image container, which is the result of the speed bump detection.

Also, you can check the speed bump marker in Web page.

2) Pothole

Electric scooter is smaller than car, so it is more sensitive to small shocks such as pothole.

Electric scooter user can check pothole by detecting pothole and updating them on the map in real time. This helps prevent accident caused by pothole.

Pothole detection method is the same as the speed bump detection method.

We detect pothole in real time. Before detecting pothole, there is no pothole marker on the map. As soon as you pass the pothole, you can check the pothole marker on the map.

Also, you can check the pothole marker in Web page.

3-2. Dangerous Driving Analysis

In web page, all user information is displayed and updated in real time. The administrator can check user's ID, name, birth year, acquisition date, driving time, and rating.

If you click the user information, you can see the detailed information about a specific user. Also, you can see driving rating(A-F), safety/danger score, and penalty score.

1) Sudden Stop

We use GPS and Gyro data from electric scooter to detect sudden stop. If the absolute "gx" value of gyro is greater than 9, it is determined as sudden stop.

When a sudden stop is detected, the app sends a sudden stop notification by sound, vibration, and push message.

In Web, delete cin value where the user's information is stored and increase the total penalty score and sudden stop penalty score by 1. And then generate a new cin value.

2) Speeding on the Speed Bump

We use GPS data from electric scooter to find the speed. Calculate the distance between the current location and the location 1 second ago and divide by 1 second.

Find the distance between electric scooter location and the speed bump location which is stored in Mobius server. If the electric scooter is within 1m from the speed bump and the speed is faster than 20km/h, it is determined that the user is speeding on the speed bump.

When a speeding on the speed bump is detected, the app sends a slow down notification by sound, vibration, and push message. The method of updating penalty points, ratings in the web is the same as the previous method.

3) Speeding in the School Zone

We use electric scooter speed to check the user is speeding or not.

Find the distance between electric scooter location and the school zone location which is stored in the Mobius server. If the electric scooter is within 300m from the school zone and the speed is faster than 20km/h, it is determined that the user is speeding in the school zone.

When a speeding in the school zone is detected, the app sends a slow down notification by sound, vibration, and push message. The method of updating penalty points, ratings in the web is the same as the previous method.

4. After Riding

4-1. Parking Check

After riding, people park electric scooter anywhere. Actually, there is no-parking zone for electric scooter, but people don't know about it. This not only obstructs the traffic and pedestrian, but also causes accidents.

We made parking space for electric scooter to solve this problem. As you know, GPS sometimes fail to recognize the correct location because of an error. Also, it cannot work indoors. Therefore, we use Visual Localization to check whether the electric scooter is parked in the correct location.

We use NetVLAD to extract the feature. As for the data on the parking space, we extracted the feature by learning the pictures taken in all directions which is the inside and outside of the parking space.

Assume that the designated parking space is centered, divide into nine areas around the designated space. And then take a picture at 360 degrees in each area. We gathered data by dividing it into two categories: an outdoor parking space and an indoor parking space.

Then, we get an image from electric scooter and extract a feature. And compare the pre-extracted photo with parking image from electric scooter to determine whether it is parking space or not.

When the user clicks the end button in the app after riding, the app sends an "OK" message to map/parking_space/check container in Mobius server. If parking program gets an "OK" cin value from the same container, it compares the photo.

If electric scooter is in the parking space, this program sends an "SUCCESS" message to map/parking_space/result container in Mobius server. Also, the image from electric scooter is stored in map/parking_space/image container in base64 format. Otherwise, the program sends an "parking out" message to the same container in Mobius server.

And then the app gets the message from the same container. If the message is "SUCCESS", ride button works and returns to the main screen. If the message is "parking out", the user must go to the parking space and try again.

1) Outdoor Parking

This is the result of visual localization for outdoor parking. Since the location was already stored in the parking space photo, GPS correction is possible to a specific parking space location when parking is successful.

The image on the left side is stored in DB. The middle image is from electric scooter. Compare these two images to check the parking. The image on the right side is APP screen when the user succeed in parking.

Also, the image on the left side is stored in DB. The middle image is from electric scooter. The image on the right side is APP screen when the user fails in parking.

2) Indoor Parking

This is the result of visual localization for indoor parking. GPS data is 0.000000 when electric scooter is in indoors. Electric scooter GPS data is updated using the location information which is already stored in the parking space photo when parking is successful.

The image on the left side is stored in DB. The middle image is from electric scooter. Compare these two images to check the parking. The image on the right side is APP screen when the user succeed in parking.

The image on the left side is stored in DB. The middle image is from electric scooter. Compare these two images to check the parking. The image on the right side is APP screen when the user fails in parking.

5. Expectation

1) Data collection

In general, people collect data themselves. But there is a limit to time and space to gather a great deal of data. If we use electric scooter, we can solve these two inconveniences simultaneously.

There are many shared electric scooter service. It is possible to collect data from several people rather than one. It means that we can collect a lot of data in a short time. Also, it can quickly update large amounts of data in real time.

2) Electric Scooter Driving Ability Test Service

Our electric scooter driving ability test service can reduce the number of driver without driver license. It provides them with an opportunity to learn electric scooter safety rules, proper driving and parking method, etc. It will help to establish a proper sense of safety for electric scooter. Thus, electric scooter accidents due to inexperience driving will decrease.

Also, it can help companies to provide stable services in Korea. And the number of shared electric scooter users will increase because of the convenience of getting a electric scooter license.

3) Risk Detection Notification and Dangerous Driving Analysis

Our risk detection and notification service can prevent accidents by providing real-time information on risk factors. Also, it helps users drive safely by detecting user's driving behavior and providing notifications.

The government can use the data we detect to maintain roads. If we develop data set about Image Detection, we can detect more risk factors such as various crack on the road in the future.

4) Visual Localization based Outdoor/Indoor Parking Space Checking Service

Our parking space checking service can solve accidents and problems caused by illegal parking. And it can help the users who don't know where is the parking area for electric scooter.

Since we use Visual Localization, there is no situation in which the electric scooter cannot be returned due to the GPS error. It means that it is possible to identify the user's real-time location and park even when GPS does not work.

Therefore, this service will contribute to the establishment of electric scooter as a new transportation.

6. Full Service Video