•

•

•

•

Published April 24, 2017 © GPL3+

Smartphone Reminder

As part of a student workshop at Brunel University London an app must be developed to remind the user if he forgets his phone in a car

IntermediateProtipOver 20 days595

Things used in this project

Hardware components

Android device

Story

In a world where smartphones gain more and more relevance it’s almost mandatory to always carry the device in the pocket. Permanent online availability belongs to one of the notable standards of living because many people cannot imagine a life without it. This is reflected in the constant increasing data traffic whether it’s streaming media like videos on Youtube or music using a provider like Spotify as well as the increasing number of text, voice or video messages sent through various channels. All this shows, that people have developed a serious dependence to their smartphone.

To keep this availability on the socially expected level, an application shall be provided which helps in a frequent but painful situation. Meant is the situation where people put their phones away in the car. An app was developed that focuses on this problem where drivers forget the most important technical device in this place.

How it works (Data Acquisition)

Based on sensors commonly provided by modern smartphones and activated by users it is possible to precisely determine everyday's situations. For detecting that the smartphone is placed in its holder simple metrics like average and standard deviation along short time intervals like 1/2 second are enough. The following graphs show four typical situations like driving, walking, engine in neutral and engine off for the amplitude, acceleration and rotation sensors.

1 / 3 • Data Distribution - Amplitude

Due to these observations thresholds for the classified datasets can be determined (threshold results are given in the code section). After applying the obtained thresholds for driving, engine in neutral and engine off on a test dataset it can be seen that the classification is quite good. However, not each sensor is able to precisely determine any classified situation.

1 / 3 • Classification Results - Acceleration Sensor

How it works (State Machine)

To ensure that the overall situation is classified correctly, such as the user can be reminded to pick up his phone after driving, are the individual sensor classifications combined by a fully state machine (code provided). (4) The state unclassified is reached if the rotation sensor provides unclassified data. This will mostly happen if the user already uses its smartphone which makes the reminder useless. (3) Driving is reached if acceleration and amplitude values indicate the driving situation. (2) The idle engine state is reached if the amplitude sensor data matches and acceleration sensor is classified as engine off or engine idle. The combination of the acceleration classifications is needed because, as already shown in the graphs, are these classes not distinguishable. (1) For the transition to the state nothing is the combination of the acceleration classes the same as well as the amplitude classification must return engine off.

full state machine - final iteration

With some more steps like sensor debouncing it is possible to build an app which makes "forgetting the forgotten smartphone after car driving" possible.

public class AccAConstants {
   //nothing constants
   public static final double nothingSdUpperLimit = 0.0393101405423961;
   public static final double nothingSdLowerLimit = 0;
   public static final double nothingMeanUpperLimit = 1000;
   public static final double nothingMeanLowerLimit = 0;
   public static final double nothingAvgDUpperLimit = 1000;
   public static final double nothingAvgDLowerLimit = 0;
   //idle constants
   public static final double idleSdUpperLimit = 0.125631060732622;
   public static final double idleSdLowerLimit = 0.0394794492501215;
   public static final double idleMeanUpperLimit = 1000;
   public static final double idleMeanLowerLimit = 0;
   public static final double idleAvgDUpperLimit = 1000;
   public static final double idleAvgDLowerLimit = 0;
   //driving constants
   public static final double drivingSdUpperLimit = 1.29188793550675;
   public static final double drivingSdLowerLimit = 0.223109642215181;
   public static final double drivingMeanUpperLimit = 1000;
   public static final double drivingMeanLowerLimit = 0;
   public static final double drivingAvgDUpperLimit = 1000;
   public static final double drivingAvgDLowerLimit = 0;
}

public class AmplitudeConstants {
   //nothing constants
   public static final double nothingSdUpperLimit = 3.3992677484211;
   public static final double nothingSdLowerLimit = 0.777048047448588;
   public static final double nothingMeanUpperLimit = -6.01400848847711;
   public static final double nothingMeanLowerLimit = -22.1677289662764;
   public static final double nothingAvgDUpperLimit = 3.21216163764692;
   public static final double nothingAvgDLowerLimit = -0.0200115804154526;
   //idle constants
   public static final double idleSdUpperLimit = 3.50929632908844;
   public static final double idleSdLowerLimit = 1.15224759552037;
   public static final double idleMeanUpperLimit = 8.32790809755217;
   public static final double idleMeanLowerLimit = -6.65624617634348;
   public static final double idleAvgDUpperLimit = 3.5521359491125;
   public static final double idleAvgDLowerLimit = 0.357386801081925;
   //driving constants
   public static final double drivingSdUpperLimit = 5.86373046737396;
   public static final double drivingSdLowerLimit = -0.0931564261706317;
   public static final double drivingMeanUpperLimit = 34.6438090289548;
   public static final double drivingMeanLowerLimit = 17.5071043700232;
   public static final double drivingAvgDUpperLimit = 3.1634104378748;
   public static final double drivingAvgDLowerLimit = 1.2648944751908;
}

public class RotAConstants {
   //nothing constants
   public static final double nothingSdUpperLimit = 0.018616528865645;
   public static final double nothingSdLowerLimit = -0.00884175305924402;
   public static final double nothingMeanUpperLimit = 0.0339435654074992;
   public static final double nothingMeanLowerLimit = -0.0139206793817086;
   public static final double nothingAvgDUpperLimit = 0.0169530522747644;
   public static final double nothingAvgDLowerLimit = -0.00763821868216058;
   //idle constants
   public static final double idleSdUpperLimit = 0.0220799625819942;
   public static final double idleSdLowerLimit = -0.00660927425388605;
   public static final double idleMeanUpperLimit = 0.0454714505173014;
   public static final double idleMeanLowerLimit = -0.0111376247146121;
   public static final double idleAvgDUpperLimit = 0.018951330986553;
   public static final double idleAvgDLowerLimit = -0.00600685379681775;
   //driving constants
   public static final double drivingSdUpperLimit = 0.269946969657981;
   public static final double drivingSdLowerLimit = -0.00590720871787567;
   public static final double drivingMeanUpperLimit = 0.501126339199062;
   public static final double drivingMeanLowerLimit = -0.178483366156723;
   public static final double drivingAvgDUpperLimit = 0.250535873872551;
   public static final double drivingAvgDLowerLimit = -0.123693290441407;
}

public class StateMachine {

    /***
     * Buffer for sensor data
     */
    SensorDataBuffer sensorDataBuffer;
    
    /**
     * Instantiates a new state machine.
     */
    public StateMachine() {
        this.idle = new StateIdle(this);
        this.nothing = new StateNothing(this);
        this.unclassified = new SateUnclassified(this);
        this.driving = new StateDriving(this);

        this.actState = this.unclassified;
        this.nextState = this.unclassified;

        this.sensorDataBuffer = new SensorDataBuffer();
    }


    /**
     * This method checks if a transition change has occurred
     */
    public void transisionCheck() {
        if (!sensorDataBuffer.isFull()) {
            sensorDataBuffer.add(CurrentTickData.micMaxAmpl, CurrentTickData.accVecA, CurrentTickData.rotationA);
            return;
        }

        // do state transition check
        double sDAmp = Math.standardDeviation(sensorDataBuffer.getAmpBuffer());
        double sDAcc = Math.standardDeviation(sensorDataBuffer.getAccABuffer());
        double sdRot = Math.standardDeviation(sensorDataBuffer.getRotABuffer());

        double meanAmp = Math.mean(sensorDataBuffer.getAmpBuffer());
        double meanAcc = Math.mean(sensorDataBuffer.getAccABuffer());
        double meanRot = Math.mean(sensorDataBuffer.getRotABuffer());

        double avgDAmp = Math.averageSumDeltas(sensorDataBuffer.getAmpBuffer());
        double avgDAcc = Math.averageSumDeltas(sensorDataBuffer.getAccABuffer());
        double avgDRot = Math.averageSumDeltas(sensorDataBuffer.getRotABuffer());

            
        /* interprete sensor individually */
        SensorInterpretation ampSensor = GetAmplitudeState(meanAmp, sDAmp, avgDAmp);
        SensorInterpretation accSensor = GetAccelerationState(meanAcc, sDAcc, avgDAcc);
        SensorInterpretation rotSensor = GetRotationState(meanRot, sdRot, avgDRot);
        SensorInterpretation bluetoothSensor = GetBluetoothState();

        Log.d("transisoin", ampSensor.toString() + "\n" + accSensor.toString() + "\n" + rotSensor.toString() + "\n" + bluetoothSensor.toString());

        // check next state
        if (rotSensor == SensorInterpretation.UNCLASSIFIED) {
            this.nextState = this.unclassified;

        } else if (
                (
                    ampSensor == SensorInterpretation.NOTHING &&
                    (accSensor == SensorInterpretation.NOTHING || accSensor == SensorInterpretation.ENGINE_IDLE) &&
                    (rotSensor == SensorInterpretation.NOTHING || rotSensor == SensorInterpretation.ENGINE_IDLE || rotSensor == SensorInterpretation.DRIVING)
                ) || bluetoothSensor == SensorInterpretation.NOTHING) {
            this.nextState = this.nothing;

        } else if (ampSensor == SensorInterpretation.ENGINE_IDLE &&
                (accSensor == SensorInterpretation.NOTHING || accSensor == SensorInterpretation.ENGINE_IDLE) &&
                (rotSensor == SensorInterpretation.NOTHING || rotSensor == SensorInterpretation.ENGINE_IDLE || rotSensor == SensorInterpretation.DRIVING)) {
            this.nextState = this.idle;

        } else if (ampSensor == SensorInterpretation.DRIVING &&
                accSensor == SensorInterpretation.DRIVING &&
                (rotSensor == SensorInterpretation.NOTHING || rotSensor == SensorInterpretation.ENGINE_IDLE || rotSensor == SensorInterpretation.DRIVING)) {
            this.nextState = this.driving;
            stateDrivingOccurred = true;

        } else if (ampSensor == SensorInterpretation.UNCLASSIFIED &&
                accSensor == SensorInterpretation.UNCLASSIFIED &&
                (rotSensor == SensorInterpretation.NOTHING || rotSensor == SensorInterpretation.ENGINE_IDLE || rotSensor == SensorInterpretation.DRIVING)) {
            this.nextState = this.unclassified;

        }

        sensorDataBuffer.clear();
        sensorDataBuffer.add(CurrentTickData.micMaxAmpl, CurrentTickData.accVecA, CurrentTickData.rotationA);
    }

    public enum SensorInterpretation {
        NOTHING,
        ENGINE_IDLE,
        DRIVING,
        UNCLASSIFIED
    }

    private ArrayList<SensorInterpretation> ampSensorStates = new ArrayList<SensorInterpretation>();
    private SensorInterpretation ampState = SensorInterpretation.UNCLASSIFIED;

    public SensorInterpretation GetAmplitudeState(double meanAmp, double sDAmp, double avgDAmp) {

        SensorInterpretation ampSensor = SensorInterpretation.UNCLASSIFIED;
        if (AmplitudeConstants.nothingMeanLowerLimit < meanAmp && meanAmp <= AmplitudeConstants.nothingMeanUpperLimit &&
                AmplitudeConstants.nothingSdLowerLimit < sDAmp && sDAmp <= AmplitudeConstants.nothingSdUpperLimit &&
                AmplitudeConstants.nothingAvgDLowerLimit < avgDAmp && avgDAmp <= AmplitudeConstants.nothingAvgDUpperLimit) {
            ampSensor = SensorInterpretation.NOTHING;
        } else if (AmplitudeConstants.idleMeanLowerLimit < meanAmp && meanAmp <= AmplitudeConstants.idleMeanUpperLimit &&
                AmplitudeConstants.idleSdLowerLimit < sDAmp && sDAmp <= AmplitudeConstants.idleSdUpperLimit &&
                AmplitudeConstants.idleAvgDLowerLimit < avgDAmp && avgDAmp <= AmplitudeConstants.idleAvgDUpperLimit) {
            ampSensor = SensorInterpretation.ENGINE_IDLE;
        } else if (AmplitudeConstants.drivingMeanLowerLimit < meanAmp && meanAmp <= AmplitudeConstants.drivingMeanUpperLimit &&
                AmplitudeConstants.drivingSdLowerLimit < sDAmp && sDAmp <= AmplitudeConstants.drivingSdUpperLimit &&
                AmplitudeConstants.drivingAvgDLowerLimit < avgDAmp && avgDAmp <= AmplitudeConstants.drivingAvgDUpperLimit) {
            ampSensor = SensorInterpretation.DRIVING;
        }

        if (Settings.getInstance().getSmoothClassification() == false || ampSensor != SensorInterpretation.UNCLASSIFIED) {
            ampSensorStates.add(ampSensor);
        }

        int debouncingLength = 3;
        if (Settings.getInstance().getSmoothClassification() == true) {
            debouncingLength = 2;
        }
        ensureCircularBuffer(ampSensorStates, debouncingLength);
        ampState = debouncedInterpretation(ampSensorStates, ampState, debouncingLength); // buffer last value
        return ampState;
    }


    private ArrayList<SensorInterpretation> accSensorStates = new ArrayList<SensorInterpretation>();
    private SensorInterpretation accState = SensorInterpretation.UNCLASSIFIED;

    public SensorInterpretation GetAccelerationState(double meanAcc, double sDAcc, double avgDAcc) {

        SensorInterpretation accSensor = SensorInterpretation.UNCLASSIFIED;
        if (AccAConstants.nothingMeanLowerLimit < meanAcc && meanAcc <= AccAConstants.nothingMeanUpperLimit &&
                AccAConstants.nothingSdLowerLimit < sDAcc && sDAcc <= AccAConstants.nothingSdUpperLimit &&
                AccAConstants.nothingAvgDLowerLimit < avgDAcc && avgDAcc <= AccAConstants.nothingAvgDUpperLimit) {
            accSensor = SensorInterpretation.NOTHING;
        } else if (AccAConstants.idleMeanLowerLimit < meanAcc && meanAcc <= AccAConstants.idleMeanUpperLimit &&
                AccAConstants.idleSdLowerLimit < sDAcc && sDAcc <= AccAConstants.idleSdUpperLimit &&
                AccAConstants.idleAvgDLowerLimit < avgDAcc && avgDAcc <= AccAConstants.idleAvgDUpperLimit) {
            if (combineAccANothingAndIdle) {
                accSensor = SensorInterpretation.NOTHING;
            } else {
                accSensor = SensorInterpretation.ENGINE_IDLE;
            }
        } else if (AccAConstants.drivingMeanLowerLimit < meanAcc && meanAcc <= AccAConstants.drivingMeanUpperLimit &&
                AccAConstants.drivingSdLowerLimit < sDAcc && sDAcc <= AccAConstants.drivingSdUpperLimit &&
                AccAConstants.drivingAvgDLowerLimit < avgDAcc && avgDAcc <= AccAConstants.drivingAvgDUpperLimit) {
            accSensor = SensorInterpretation.DRIVING;
        }

        accSensorStates.add(accSensor); // require unclassified to distinguish - do not smooth sensor states here


        int debouncingLength = 3;
        ensureCircularBuffer(accSensorStates, debouncingLength);
        accState = debouncedInterpretation(accSensorStates, accState, debouncingLength);
        return accState;
    }


    private ArrayList<SensorInterpretation> rotSensorStates = new ArrayList<SensorInterpretation>();
    private SensorInterpretation rotState = SensorInterpretation.UNCLASSIFIED;

    public SensorInterpretation GetRotationState(double meanRot, double sDRot, double avgDRot) {

        SensorInterpretation rotSensor = SensorInterpretation.UNCLASSIFIED;
        if (RotAConstants.nothingMeanLowerLimit < meanRot && meanRot <= RotAConstants.nothingMeanUpperLimit &&
                RotAConstants.nothingSdLowerLimit < sDRot && sDRot <= RotAConstants.nothingSdUpperLimit &&
                RotAConstants.nothingAvgDLowerLimit < avgDRot && avgDRot <= RotAConstants.nothingAvgDUpperLimit) {
            rotSensor = SensorInterpretation.NOTHING;
        } else if (RotAConstants.idleMeanLowerLimit < meanRot && meanRot <= RotAConstants.idleMeanUpperLimit &&
                RotAConstants.idleSdLowerLimit < sDRot && sDRot <= RotAConstants.idleSdUpperLimit &&
                RotAConstants.idleAvgDLowerLimit < avgDRot && avgDRot <= RotAConstants.idleAvgDUpperLimit) {
            rotSensor = SensorInterpretation.ENGINE_IDLE;
        } else if (RotAConstants.drivingMeanLowerLimit < meanRot && meanRot <= RotAConstants.drivingMeanUpperLimit &&
                RotAConstants.drivingSdLowerLimit < sDRot && sDRot <= RotAConstants.drivingSdUpperLimit &&
                RotAConstants.drivingAvgDLowerLimit < avgDRot && avgDRot <= RotAConstants.drivingAvgDUpperLimit) {
            rotSensor = SensorInterpretation.DRIVING;
        }

        rotSensorStates.add(rotSensor); // require unclassified to detect movement - do not smooth sensor states here


        int debouncingLength = 3;
        ensureCircularBuffer(rotSensorStates, debouncingLength);
        rotState = debouncedInterpretation(rotSensorStates, rotState, debouncingLength);
        return rotState;
    }

    private SensorInterpretation GetBluetoothState() {
        SensorInterpretation bluetoothSensor = SensorInterpretation.UNCLASSIFIED;
        Log.d("GetBluetoothState", CurrentTickData.bluetooth.toString());
        if (Settings.getInstance().getUseBluetooth()) {
            if (CurrentTickData.bluetooth == BluetoothState.CONNECTION_LOST) {
                bluetoothSensor = SensorInterpretation.NOTHING;
                CurrentTickData.bluetooth = BluetoothState.NOTHING_CONNECTED_YET; // reset state
            } else if (CurrentTickData.bluetooth == BluetoothState.CONNECTED) {
                //bluetoothSensor = SensorInterpretation.DRIVING;
                //stateDrivingOccurred = true; // flag needed for triggering alarm
            }
        }
        return bluetoothSensor;
    }

    private void ensureCircularBuffer(ArrayList<SensorInterpretation> sensorStates, int circularBufferLength) {
        while (sensorStates.size() > circularBufferLength) {
            sensorStates.remove(0);
        }
    }

    private SensorInterpretation debouncedInterpretation(ArrayList<SensorInterpretation> sensorStates, SensorInterpretation state, int debouncingLength) {

        int nothing = 0;
        int idle = 0;
        int driving = 0;
        int unclassified = 0;
        for (SensorInterpretation sensorInterpretation : sensorStates) {
            if (sensorInterpretation == SensorInterpretation.NOTHING) {
                nothing++;
            } else if (sensorInterpretation == SensorInterpretation.ENGINE_IDLE) {
                idle++;
            } else if (sensorInterpretation == SensorInterpretation.DRIVING) {
                driving++;
            } else if (sensorInterpretation == SensorInterpretation.UNCLASSIFIED) {
                unclassified++;
            }
        }


        if (nothing >= debouncingLength) {
            state = SensorInterpretation.NOTHING;
        } else if (idle >= debouncingLength) {
            state = SensorInterpretation.ENGINE_IDLE;
        } else if (driving >= debouncingLength) {
            state = SensorInterpretation.DRIVING;
        } else if (unclassified >= debouncingLength) {
            state = SensorInterpretation.UNCLASSIFIED;
        }

        return state;
    }
}

Credits

DCSE Team C

0 projects • 0 followers

DCSE Team C

Contact

Comments

Please log in or sign up to comment.

Smartphone Reminder

Things used in this project

Hardware components

Story

How it works (Data Acquisition)

How it works (State Machine)

Code

AccAConstants.java

AmplitudeConstants.java

RotAConstants.java

BluetoothState.java

StateMachine.java

Credits

DCSE Team C

Florian Hauff

Frederic Frasch

Marcel Waida

Lukas Jaeckle

Comments

Embed the widget on your own site

Smartphone Reminder

Smartphone Reminder

Things used in this project

Hardware components

Story

How it works (Data Acquisition)

How it works (State Machine)

Code

AccAConstants.java

AmplitudeConstants.java

RotAConstants.java

BluetoothState.java

StateMachine.java

Credits

DCSE Team C

Florian Hauff

Frederic Frasch

Marcel Waida

Lukas Jaeckle

Comments

Related channels and tags