Arduino Menu with APDS 9960 Gesture Sensor Control

I remember several times that I asked the following question: How to create a menu on Arduino without using any mechanical button? In most of my projects, I needed to use an LCD with the Menu for the user to configure the system. However, it was necessary to use buttons and over time, they would wear out and some problems would soon arise.

In addition, I liked to do different projects. I wanted to create something that could be more technological and interesting than using a few buttons on my projects.

After so much time thinking, I found the APDS 9960 gesture sensor and it allowed me to do interesting things that I want to show you in this project. It is simple, but it is powerful for many applications.

With this sensor, I was able to create a fully controlled menu with gestures for load control: the JLCPCB Gesture Control.

In addition, you will learn:

• How the gesture sensor works;
• How to create a menu on the LCD;
• How to manipulate the control functions of the gesture sensor.

Now, let's start building our project.

The JLCPCB Gesture Control

Several times I have developed MENU on Arduino and the circuit and code have become extremely complex. This was due to a large number of buttons and a code control system.

To avoid this problem, I decided to use this gesture sensor. It allowed me to use just 2 wires of data and create a simpler system with several control movements.

Also, see how simple the code is.

#include "SoftwareSerial.h"
#include <LiquidCrystal_I2C.h>
#include "Adafruit_APDS9960.h"

Adafruit_APDS9960 apds;

LiquidCrystal_I2C lcd(0x27,16,2);
SoftwareSerial mySoftwareSerial(10, 11); // RX, TX

byte count = 2;

void setup() 
{
  Serial.begin(9600); 
  Wire.begin();

  lcd.init();                   
  lcd.backlight();

  for(byte i = 2; i <=4; i++)
  {
    pinMode(i, OUTPUT);
  }
  
  if(!apds.begin())
  { 
    Serial.println("Falha ao inicializar o dispositivo. Verifique as conexões!");
  }
  else 
    Serial.println("Dispositivo inicializado!");

  apds.enableProximity(true);
  apds.enableGesture(true);

  show_menu(count);
}

void loop()
{
    uint8_t gesture = apds.readGesture();

          if(gesture == APDS9960_UP)
          {
           count++;
           
           if(count > 4)
           {
            count = 4; 
           }
           
           show_menu(count);
          }
         
          if(gesture == APDS9960_LEFT)
          {
           if(count == 2)
            {
              digitalWrite(2, LOW);  
            }

            if(count == 3)
            {
              digitalWrite(3, LOW);  
            }

            if(count == 4)
            {
              digitalWrite(2, LOW);  
              digitalWrite(3, LOW);  
            }
          }
          
          if(gesture == APDS9960_RIGHT)
          {
            if(count == 2)
            {
              digitalWrite(2, HIGH);  
            }

            if(count == 3)
            {
              digitalWrite(3, HIGH);  
            }

            if(count == 4)
            {
              digitalWrite(2, HIGH);  
              digitalWrite(3, HIGH);  
            }
          }
                    
          if(gesture == APDS9960_DOWN)
          {
           count--;

           if(count < 2)
           {
            count = 2;
           }

           show_menu(count);
          }
}

void show_menu(byte option)
{
  if(option == 2)
  {
    lcd.clear();
    lcd.setCursor(0,0);
    lcd.print("-> LED1");
    lcd.setCursor(0,1);
    lcd.print("   LED2");

    return;
  }

  if(option == 3)
  {
    lcd.clear();
    lcd.setCursor(0,0);
    lcd.print("   LED1");
    lcd.setCursor(0,1);
    lcd.print("-> LED2");

    return;
  }

  if(option == 4)
  {
    lcd.clear();
    lcd.setCursor(0,0);
    lcd.print("-> LED1 and LED2");

    return;
  }
  
  return;
}

First, we declare the library of each component, we create its objects, and, finally, we declare the variables used in the project. The code portion is shown below.

#include "SoftwareSerial.h"
#include <LiquidCrystal_I2C.h>
#include "Adafruit_APDS9960.h"

Adafruit_APDS9960 apds;

LiquidCrystal_I2C lcd(0x27,16,2);
SoftwareSerial mySoftwareSerial(10, 11); // RX, TX

byte count = 2;

After that, I created the void setup function. It is essential to carry out our initial configurations of the devices used in the project.

See below that I started the serial communication, I2C communication, and the LCD operation.

In addition, I configured the LED pins as output through a loop.

void setup() 
{
  Serial.begin(9600);
  Wire.begin();

  lcd.init();
  lcd.backlight();

  for(byte i = 2; i <=4; i++)
  {
    pinMode(i, OUTPUT);
  }
  
  if(!apds.begin())
  {
    Serial.println("Falha ao inicializar o dispositivo. Verifique as conexões!");
  }
  else 
    Serial.println("Dispositivo inicializado!");

  apds.enableProximity(true);
  apds.enableGesture(true);

  show_menu(count);
}

Finally, you need to do something very important: Test the functioning of your gesture sensor. Check that it has been initialized correctly.

If it starts up correctly, configure the sensor to enable gesture detection mode when it detects that something is nearby. The APDS-9960 sensor is shown in the figure below.

This will be done when you move your finger closer to the module and move away. This configuration process is carried out using the last two lines of code.

After that, the system presents the screen with the options menu to the user, as is shown in Figure 2.

After this, it prepares to enter on the loop function.

Void Loop Function and APDS-9960 Gesture Sensor

It is in the void loop function that everything happens. It is the heart of our project. Below is presented the code void loop function.

void loop()
{
    uint8_t gesture = apds.readGesture();

          if(gesture == APDS9960_UP)
          {
           count++;
           
           if(count > 4)
           {
            count = 4; 
           }
           
           show_menu(count);
          }
          
          if(gesture == APDS9960_LEFT)
          {
           if(count == 2)
            {
              digitalWrite(2, LOW);  
            }

            if(count == 3)
            {
              digitalWrite(3, LOW);  
            }

            if(count == 4)
            {
              digitalWrite(2, LOW);  
              digitalWrite(3, LOW);  
            }
          }
          
          if(gesture == APDS9960_RIGHT)
          {
            if(count == 2)
            {
              digitalWrite(2, HIGH);  
            }

            if(count == 3)
            {
              digitalWrite(3, HIGH);  
            }

            if(count == 4)
            {
              digitalWrite(2, HIGH);  
              digitalWrite(3, HIGH);  
            }
          }
                    
          if(gesture == APDS9960_DOWN)
          {
           count--;

           if(count < 2)
           {
            count = 2;
           }

           show_menu(count);
          }
}

The logic developed above is quite simple. First, the system reads the gesture performed by the user using the reading function below.

uint8_t gesture = apds.readGesture();

The system checks 4 types of gestures: left, right, up, or down. The up and down gesture is used to select the load in the options menu

The left gesture is used to disable the load and the right gesture is used to trigger the load, as shown in the code above.

Each time the user makes an up or down gesture, the count variable is increased or decreased. The value of this variable is used to present the screen with the options and allow the user to navigate through the menu.

Basically, we have 3 screens for each count value 2, 3, and 4.

The count starts from the value two since it is the number of the LED pin. See the example below.

 if(gesture == APDS9960_UP)
 {
     count++;
           
     if(count > 4)
     {
       count = 4; 
     }
           
        show_menu(count);
 }

 if(gesture == APDS9960_DOWN)
 {
      count--;

      if(count < 2)
      {
        count = 2;
      }

        show_menu(count);
  }

The up gesture increases and the down gesture decreases the value of the count variable. This allows the user to navigate through the menu, as shown in Figure 3.

After selecting the option, the user must activate or deactivate the LED. For this, he must use the gesture to the left (disable) or to the right (activate).

As is cited before, the system will use the counter variable value to show the screen and activate/disable the LED.

if(gesture == APDS9960_LEFT)
{
           if(count == 2)
            {
              digitalWrite(2, LOW);  
            }

            if(count == 3)
            {
              digitalWrite(3, LOW);  
            }

            if(count == 4)
            {
              digitalWrite(2, LOW);  
              digitalWrite(3, LOW);  
            }
}
          
if(gesture == APDS9960_RIGHT)
{
            if(count == 2)
            {
              digitalWrite(2, HIGH);  
            }

            if(count == 3)
            {
              digitalWrite(3, HIGH);  
            }

            if(count == 4)
            {
              digitalWrite(2, HIGH);  
              digitalWrite(3, HIGH);  
            }
}

In addition to the process of controlling the LEDs and navigation through the menu, we reserve a specific topic to talk about the creation of the screens that will be presented in the menu.

Creation of the screens on the 16x2 LCD Display

To present the navigation screens to the user, we created a function with the name show_menu().

void show_menu(byte option)
{
  if(option == 2)
  {
    lcd.clear();
    lcd.setCursor(0,0);
    lcd.print("-> LED1");
    lcd.setCursor(0,1);
    lcd.print("   LED2");

    return;
  }

  if(option == 3)
  {
    lcd.clear();
    lcd.setCursor(0,0);
    lcd.print("   LED1");
    lcd.setCursor(0,1);
    lcd.print("-> LED2");

    return;
  }

  if(option == 4)
  {
    lcd.clear();
    lcd.setCursor(0,0);
    lcd.print("-> LED1 and LED2");

    return;
  }
  
  return;
}

To present the navigation screens to the user, we created a function with the name show_menu ().

This function must receive the value of the count variable as a parameter. When the value is received, the system compares with values 2, 3 and 4. Each value represents a different option.

• If the value is 2, the system will print the options LED 1 with an arrow on the first line and LED 2 on the second line.
• If the value is 3, the system will print the options LED 1 on the first line and LED 2 with an arrow on the second line.
• Finally, if the value is equal to 4, the system will print the option LED 1 and LED 2.

The developed system is simple and can be useful to develop several types of the menu using the APDS-9960 gesture sensor.

But before finishing this project, I developed a board that will allow you to control any project using the gesture sensor and a control menu as an interface for the user.

The JLCPCB Gesture Control Printed Circuit Board

This printed circuit board in the figure below was developed in a few minutes using the JLCPCB Software - EasyEDA and through this board, you can connect your LCD Display 16x2 and its sensor.

In this project, I decided to put screw connectors for each pin of your Arduino Nano.

Through these connectors, you can connect any type of load and make your project more flexible to facilitate the assembly with several other projects.

From the layout, the PCB was developed with the JLCPCB Company for $2 with 10 PCB's and through this link, you can access the files for download.

Finally, we have the printed circuit board as is shown below.

Through this project, you can assembly your PCB with the components and create several projects using the gesture sensor and LCD Display 16x2 with Arduino Nano.

Conclusion

This project showed a great result to replace mechanical buttons with a control interface totally controlled by gestures.

In addition, it was possible to reduce the complexity of circuit connections and programming, through a single connected sensor. It is possible to control 4 control commands.

The project also made it possible to use the printed circuit board developed at JLCPCB. This PCB is universal, that is, it can be applied in any project and you can use the screw terminals to adapt it to any project.

Therefore, we appreciate your reading and leave the access links to the files for you to obtain your 10 printed circuit boards at JLCPCB.