Published November 2, 2020 © MIT

Ultrasoinc Gesture Control

Gesture controlled flappy bird using an ultrasonic sensor.

IntermediateFull instructions provided346

Things used in this project

Hardware components

Seeed Studio Seeeduino V4.2

Ultrasonic Sensor - HC-SR04 (Generic)

Software apps and online services

pygame

A python library

pySerial

A python library

Story

In this project, I explored the possibility of using a low-end sensor commonly known as an ultrasonic sensor to control a flappy bird through gestures. The idea is simple, use the ultrasonic sensor to find out the distance of the hand hovering above the sensor when the distance decreases make the bird move up.

Ultrasonic sensor(HC-SR04) :

An ultrasonic sensor has a very interesting working principle, a burst of a high-frequency sound wave is transmitted from one of the two transducers (let's call it TX-transducers) and when the wave hits an object on its way, a part of the wave is traveled back to the sensor which is received/captured by the other transducer (let's call it RX-transducer).

The time taken by the sound wave to travel back and forth determines the distance of the object how far it is located from the sensor.

The sensor has 4 pins,

1) VCC, this pin dedicated 5V power supply.

2) Trig, this a digital input pin, when a 10-microsecond positive electrical pulse is applied on this pin, the sensor starts a series of 8 sound pulses through the TX-transducer.

3) Echo, this is a digital output pin, the sensor pulls this pin HIGH at the same time when the sound pulses are generated through TX-transducer and pull it LOW when those pulses

are received by the RX-transducer. The length of time this pin stays HIGH will be recorded by the Arduino board.

4) GND, this pin is dedicated to the ground connection.

Picture courtesy to electrodragon.com

For more details on working of the sensor : https://www.electrodragon.com/w/File:Frequency..jpg

Interfacing with Arduino :

It is very simple to interface this sensor with an Arduino board. Here is a simple trick you can do to avoid using connecting wires.

It is given in the datasheet of the sensor that it only requires 15mA of current and an Arduino's I/O pin can source up to 40mA of current so instead of connecting the VCC and the GND pin to the actual 5V and ground you can connect them to the regular I/O pins and program Arduino to pull the VCC connected pin high and the ground connected pin low. In this project, we are using pin-7 as VCC and pin-4 as ground. The other two pins 'Trig' and 'Echo' are connected to pin-6 and pin-5 respectively. The following picture shows the connection diagram.

Arduino and Ultrasonic sensor conection diagram

Game development :

It looks overwhelming in the beginning but developing a game is quite easy if you have some programming skills and know what libraries to use.

The python library pygame is utilized in this project to develop a flappy bird. If you are new then watch the following tutorial it will give you a great introduction to game development and in the end, you will have your own flappy bird. Thanks to Clear Code for this tutorial:

Learning pygame

Interfacing with PC :

Now we have a game running on PC and the sensor running on the Arduino board, we need some means of data transfer from the Arduino board to the PC. A simple way to establish this

communication is to write an Arduino sketch to send the sensor input over a serial cable and then read the serial data using the pySerial python library. It is very simple to use this

library as it requires one line of code to set up a serial port, the baud rate, byte size, timeout, and the stop bit. Here is an example:

import serial # importing the library
serial_data = serial.Serial(port = "COM11", baudrate = 115200, bytesize = 8, timeout = 0, stopbits = serial.STOPBITS_ONE)

Change the com port with the number assigned to your Arduino board in the system and make sure to keep the same baud rate as it is set up in the Arduino. For more information on this library, click on the link.

Programming :

The programming is simple as it is for a push-button, consider you are controlling the bird using a push-button then you will write a code to move the bird up when the button is pressed and let the bird go down when the button is not pressed.

Similarly, in the case of the ultrasonic sensor we are looking at the distance of the hand from the sensor, every time the distance decreases to a few centimeters then we will write a logic to move the bird up otherwise the bird will go down.

Remember we have two programs, a python program running on the PC and an Arduino program running on the Arduino board so the movement of the hand or the decrease in distance will be detected by the Arduino board, the board will then send a logic "1" to the PC over the serial port which means a movement has been detected the python code on the PC will then make a necessary move to the bird (i.e go up).

The following video shows the final results of this project:

Flappy bird gesture control

If you need any more information do let me know in the comments below.

Code

// define pins numbers
const int VCC = 7;
const int trigPin = 6;
const int echoPin = 5;
const int GND = 4;
// define variables
long duration;
float distance;
int avg = 0;
int n = 100;

void setup()
{
  pinMode(VCC, OUTPUT); // Sets the VCC as an Output
  digitalWrite(VCC, HIGH); //Pull VCC high
  
  pinMode(GND, OUTPUT); // Sets the GND as an Output
  digitalWrite(GND, LOW); //Pull GND low
  
  pinMode(trigPin, OUTPUT); // Sets the trigPin as an Output
  pinMode(echoPin, INPUT); // Sets the echoPin as an Input
  Serial.begin(115200); // Starts the serial communication
}

void loop() 
{
  for(int i = 0; i < n; i++)
  {
    // Clears the trigPin
    digitalWrite(trigPin, LOW);
    delayMicroseconds(2);
    // Sets the trigPin on HIGH state for 10 micro seconds
    digitalWrite(trigPin, HIGH);
    delayMicroseconds(10);
    digitalWrite(trigPin, LOW);
    // Reads the echoPin, returns the sound wave travel time in microseconds
    duration = pulseIn(echoPin, HIGH, 1000);
    avg += duration;
    }
    if(avg/n > 10)
    {
      Serial.print("1");
    }
    else
    {
      Serial.print("0");
    }
    avg = 0;
}

# -*- coding: utf-8 -*-
"""
Created on Thu Oct 29 20:30:22 2020

@author: khan
"""

import pygame, sys, random 
import serial
import time 

screen_hieght = 1024
screen_width = 576

def draw_floor():
    screen.blit(floor_surface,(floor_x_pos,900))
    screen.blit(floor_surface,(floor_x_pos + 576,900))

def create_pipe():
    random_pipe_pos = random.choice(pipe_height)
    bottom_pipe = pipe_surface.get_rect(midtop = (700,random_pipe_pos))
    top_pipe = pipe_surface.get_rect(midbottom = (700,random_pipe_pos - 500))
    return bottom_pipe,top_pipe

def move_pipes(pipes):
    for pipe in pipes:
        pipe.centerx -= 5
    return pipes

def draw_pipes(pipes):
    for pipe in pipes:
        if pipe.bottom >= 1024:
            screen.blit(pipe_surface,pipe)
        else:
            flip_pipe = pygame.transform.flip(pipe_surface,False,True)
            screen.blit(flip_pipe,pipe)

def check_collision(pipes):
    for pipe in pipes:
        if bird_rect.colliderect(pipe):
            death_sound.play()
            return False

        if bird_rect.top <= -100 or bird_rect.bottom >= 900:
            return False

    return True

def rotate_bird(bird):
    new_bird = pygame.transform.rotozoom(bird,-bird_movement * 3,1)
    return new_bird

def bird_animation():
    new_bird = bird_frames[bird_index]
    new_bird_rect = new_bird.get_rect(center = (100,bird_rect.centery))
    return new_bird,new_bird_rect

def score_display(game_state):
    if game_state == 'main_game':
        score_surface = game_font.render(str(int(score)),True,(255,255,255))
        score_rect = score_surface.get_rect(center = (288,100))
        screen.blit(score_surface,score_rect)
    if game_state == 'game_over':
        score_surface = game_font.render(f'Score: {int(score)}' ,True,(255,255,255))
        score_rect = score_surface.get_rect(center = (288,100))
        screen.blit(score_surface,score_rect)

        high_score_surface = game_font.render(f'High score: {int(high_score)}',True,(255,255,255))
        high_score_rect = high_score_surface.get_rect(center = (288,850))
        screen.blit(high_score_surface,high_score_rect)

def update_score(score, high_score):
    if score > high_score:
        high_score = score
    return high_score

pygame.mixer.pre_init(frequency = 44100, size = 16, channels = 1, buffer = 512)
pygame.init()
screen = pygame.display.set_mode((screen_width,screen_hieght))
clock = pygame.time.Clock()
game_font = pygame.font.Font('04B_19.ttf',40)

# Game Variables
gravity = 0.15
bird_movement = 0
game_active = False
score = 0
high_score = 0

bg_surface = pygame.image.load('assets/background-day.png').convert()
bg_surface = pygame.transform.scale2x(bg_surface)

floor_surface = pygame.image.load('assets/base.png').convert()
floor_surface = pygame.transform.scale2x(floor_surface)
floor_x_pos = 0

bird_downflap = pygame.transform.scale2x(pygame.image.load('assets/bluebird-downflap.png').convert_alpha())
bird_midflap = pygame.transform.scale2x(pygame.image.load('assets/bluebird-midflap.png').convert_alpha())
bird_upflap = pygame.transform.scale2x(pygame.image.load('assets/bluebird-upflap.png').convert_alpha())
bird_frames = [bird_downflap,bird_midflap,bird_upflap]
bird_index = 0
bird_surface = bird_frames[bird_index]
bird_rect = bird_surface.get_rect(center = (100,512))

BIRDFLAP = pygame.USEREVENT + 1
pygame.time.set_timer(BIRDFLAP,200)

pipe_surface = pygame.image.load('assets/pipe-green.png')
pipe_surface = pygame.transform.scale2x(pipe_surface)
pipe_list = []
SPAWNPIPE = pygame.USEREVENT
pygame.time.set_timer(SPAWNPIPE,1200)
pipe_height = [400,600,800]

game_over_surface = pygame.transform.scale2x(pygame.image.load('assets/message.png').convert_alpha())
game_over_rect = game_over_surface.get_rect(center = (288,512))

flap_sound = pygame.mixer.Sound('sound/sfx_wing.wav')
death_sound = pygame.mixer.Sound('sound/sfx_hit.wav')
score_sound = pygame.mixer.Sound('sound/sfx_point.wav')
score_sound_countdown = 100

serial_data = serial.Serial(port = "COM11", 
                           baudrate = 115200, 
                           bytesize = 8, 
                           timeout = 0,
                           stopbits = serial.STOPBITS_ONE)

x = 0;
input_delay = 0.005
t1 = 0.0
t2 = 0.0

while True:
    t2 = time.time()
    if(serial_data.in_waiting > 0) and t2 - t1 >= input_delay:
        x = int(serial_data.read(1))
        t1 = time.time()
    for event in pygame.event.get():
        if event.type == pygame.QUIT:
            pygame.quit()
            sys.exit()

        if x == 1 and game_active == True:
            bird_movement = 0
            bird_movement -= 6
            flap_sound.play()
            x = 0

        if x == 1 and game_active == False:
            game_active = True
            pipe_list.clear()
            bird_rect.center = (100,512)
            score = 0

        if event.type == SPAWNPIPE:
            pipe_list.extend(create_pipe())

        if event.type == BIRDFLAP:
            if bird_index < 2:
                bird_index += 1
            else:
                bird_index = 0

            bird_surface,bird_rect = bird_animation()

    screen.blit(bg_surface,(0,0))

    if game_active:
        # Bird
        bird_movement += gravity
        rotated_bird = rotate_bird(bird_surface)
        bird_rect.centery += bird_movement
        screen.blit(rotated_bird,bird_rect)
        game_active = check_collision(pipe_list)

        # Pipes
        pipe_list = move_pipes(pipe_list)
        draw_pipes(pipe_list)
        
        score += 0.01
        score_display('main_game')
        score_sound_countdown -= 1
        if score_sound_countdown <= 0:
            score_sound.play()
            score_sound_countdown = 100
    else:
        screen.blit(game_over_surface,game_over_rect)
        high_score = update_score(score,high_score)
        score_display('game_over')

    # Floor
    floor_x_pos -= 1
    draw_floor()
    if floor_x_pos <= -576:
        floor_x_pos = 0
    
    pygame.display.update()
    clock.tick(80)

Credits

Khan

1 project • 0 followers

Robotics Engineer

Contact

Comments

Please log in or sign up to comment.

Ultrasoinc Gesture Control

Things used in this project

Hardware components

Software apps and online services

Story

Ultrasonic sensor(HC-SR04) :

Interfacing with Arduino :

Game development :

Interfacing with PC :

Programming :

Schematics

Sensor interface with the Arduino

Code

Arduino Code

Python Code

Github repo

Credits

Khan

Comments

Embed the widget on your own site

Ultrasoinc Gesture Control

Ultrasoinc Gesture Control

Things used in this project

Hardware components

Software apps and online services

Story

Ultrasonic sensor(HC-SR04) :

Interfacing with Arduino :

Game development :

Interfacing with PC :

Programming :

Schematics

Sensor interface with the Arduino

Code

Arduino Code

Python Code

Github repo

Credits

Khan

Comments

Related channels and tags