Published May 22, 2022 © GPL3+

Hand Gesture Volume Control

Use gesture sensing to turn hand movements into keyboard inputs.

BeginnerFull instructions provided1 hour601

Things used in this project

Hardware components

Adafruit QT Py

SENSE: Multipurpose Sensor Development Board

Adafruit Monochrome 128x32 SPI OLED graphic display

SparkFun Qwiic Cable Kit

Jumper wires (generic)

Solderless Breadboard Half Size

Software apps and online services

Arduino IDE

Story

Are you ever in the situation where you need to change your computer volume or press any key, but you are either eating or your hands are dirty so don't want to touch the keyboard? If you have experienced this before, then this project is for you!

In this example, we use gesture sensing to increase/decrease the computer volume. Additionally, you can apply this example to more keyboard inputs such as pressing other commands or typing numbers/letters.

This guide uses a small multipurpose sensor development board called SENSE that can gather over 20 data variables including air quality, sound, light, temperature, proximity, and more. In this application, we will use the gesture sensor.

Step 1: Connections

Connect a Qwiic cable between QTPY SAMD21 and SENSE
Create a I2C bus between SENSE and OLED Display
OLED GND -> SENSE GND
OLD VCC -> SENSE 3V
OLED SCL -> SENSE SCL
OLED SDA -> SENSE SDA

Step 2: CODE

Install Code Libraries: (sketch->include library->manage libraries->typename)

"APDS9960" - Adafruit
"GFX" - Adafruit
"SSD1306" - Adafruit
HID - NicoHood (download from GitHub)

UploadCode

*After the code uploads, open the Serial Monitor and hold your finger on the APDS9960 sensor for 3 seconds to activate it. Then start waving your hand left/right over the sensor to decrease/increase the computer volume.

What's next?

Try using the APDS9960_LEFT and APDS_9960_RIGHT controls to add more functionality. For example, maybe a left gesture mutes the audio a right gesture pauses the audio.

Take a look at the HID API for more information.

Code

SENSE_Gesture_Sensor_Volume_Control.ino

#include <Wire.h>
#include <Adafruit_GFX.h>
#include <Adafruit_SSD1306.h>

#define SCREEN_WIDTH 128 // OLED display width, in pixels
#define SCREEN_HEIGHT 32 // OLED display height, in pixels

// Declaration for an SSD1306 display connected to I2C (SDA, SCL pins)
// The pins for I2C are defined by the Wire-library. 
// On an arduino UNO:       A4(SDA), A5(SCL)
// On an arduino MEGA 2560: 20(SDA), 21(SCL)
// On an arduino LEONARDO:   2(SDA),  3(SCL), ...
#define OLED_RESET     -1 // Reset pin # (or -1 if sharing Arduino reset pin)
#define SCREEN_ADDRESS 0x3C ///< See datasheet for Address; 0x3D for 128x64, 0x3C for 128x32
Adafruit_SSD1306 display(SCREEN_WIDTH, SCREEN_HEIGHT, &Wire, OLED_RESET);
  
#include "Adafruit_APDS9960.h"
Adafruit_APDS9960 apds;

#include "HID-Project.h"

int level = 0;

// the setup function runs once when you press reset or power the board
void setup() {
  Serial.begin(115200);
  Wire.begin();

  // SSD1306_SWITCHCAPVCC = generate display voltage from 3.3V internally
  if(!display.begin(SSD1306_SWITCHCAPVCC, SCREEN_ADDRESS)) {
    Serial.println(F("SSD1306 allocation failed"));
    for(;;); // Don't proceed, loop forever
  }
  
  if(!apds.begin()){
    Serial.println("failed to initialize device! Please check your wiring.");
  }
  else Serial.println("Device initialized!");

  //gesture mode will be entered once proximity mode senses something close
  apds.enableProximity(true);
  apds.enableGesture(true);

  // Clear the buffer
  display.clearDisplay();

  display.setTextSize(1.5);
  display.setTextColor(SSD1306_WHITE);
  display.setCursor(0, 0);
  display.println("Volume"); 
  display.display(); 

  Consumer.begin();
}

// the loop function runs over and over again forever
void loop() {
   

  //read a gesture from the device
    uint8_t gesture = apds.readGesture();
    if(gesture == APDS9960_DOWN) {
      Serial.println("v");
      Consumer.write(MEDIA_VOLUME_DOWN);
      level--; 
      changeVolume();

    }
    if(gesture == APDS9960_UP) {
      Serial.println("^");
      Consumer.write(MEDIA_VOLUME_UP);
      level++;
      changeVolume();
    }
}

void changeVolume() {
  Serial.println(level);
  display.clearDisplay();

  display.setCursor(0, 0);
  display.println("Volume");  
  
  display.setCursor(0, 15);
  
  for (int i = 0; i <= level; i++){
    display.print("[]"); 
    display.display();  
  }
}