Imagine you're playing a car racing gameโฆ but instead of using a joystick or keyboard, you just move your hands in front of a camera to drive the car! This project makes that possible using a webcam, Python.
You can:
- ๐ Turn your hands left or right to steer the car.
- ๐ Move your thumb down to speed up or slow down. Itโs like magic! But itโs actually made using something called Computer Vision.
This project is designed for complete beginners and fun enthusiasts who want to control a car using hand gestures with a webcam. The program uses MediaPipe to track your hand, calculates the angle between your wrist, thumb, and index finger, and based on that angle, it sends keyboard commands to the game.
No fancy equipment neededโjust a webcam, your hand, and a game that uses the keyboard (like pressing A/D to steer).
Letโs dive into how it works in the simplest way possible.
SuppliesHereโs everything you need:
- ๏ธ A laptop or desktop computer (Windows, macOS, or Linux)
- A webcam (built-in or USB)
- Python installed (version 3.7 or later)
- Python packages:
- opencv-python
- mediapipe
- pyautogui
If you havenโt already:
- Go to https://www.python.org/
- Click Download Python
- Install it like any other app
These are the special helpers for Python:
- Open Command Prompt (Windows) or Terminal (Mac/Linux)
- Type the following and press Enter:
pip install opencv-python mediapipe pyautogui
What Are These Libraries?Letโs understand them like superheroes:
- ๐ง Mediapipe โ This smart tool sees your hands and finds points like your thumb, wrist, and fingers.
- ๐ OpenCV โ It helps us use the webcam and draw things on the screen.
- โจ๏ธ PyAutoGUI โ It presses keys like W, A, S, D for you.
Hereโs the magic program. You can open Notepad, or better: install VS Code.
Step 4: Explaination of Code and Important Settings in Code Before Using It๐ Explaining the Key Code Linesโ Importing Toolsimport cv2 # To use webcam and show visuals
import mediapipe as mp # To detect hands
import math # To calculate angles
import pyautogui # To press keyboard buttons automatically
import time # To handle timing for button presses
โ Initial Settingshands = mphands.Hands(...)
- This initializes MediaPipe Hands to detect up to 2 hands.
- It finds where your hands are and tracks their movement.
def calculate_angle(x1, y1, x2, y2):
...
- Calculates the angle between your left and right hands.
- This tells whether you're turning left, right, or going straight.
- This was used in a previous version to make turning more or less sensitive.
- BUT in your current code, it's not used anymore, so we can remove it or ignore it.
This value decides how much your hand angle must change before it counts as turning.
- If the angle is less than -6, it means turn left (A key).
- If the angle is more than +6, it means turn right (D key).
- If the angle is between -6 and +6, it means go straight (do nothing).
This helps avoid small hand shakes being treated as turns.
โฑ๏ธ tap_duration = 0.01This sets how quickly to release the A or D key after tapping it.
- 0.01 means the key is held down for just 0.01 seconds (very short).
- This makes the car turn in small steps, so you donโt turn too much with a small hand movement.
If you want smoother, longer turns, increase this value to something like 0.05.
๐๏ธ Detecting Thumb Up or Downdef is_thumb_up(hand_landmarks):
return hand_landmarks.landmark[4].y < hand_landmarks.landmark[3].y
- Checks if the tip of the thumb is above the second joint (on the screen).
- This is used to detect:
- Left thumb up = stop accelerating
- Right thumb up = stop braking
if not is_thumb_up(left_hand):
pyautogui.keyDown('w')
- If your left thumb is down, press the W key (accelerate).
- If your left thumb goes up, release the W key (stop).
if not is_thumb_up(right_hand):
pyautogui.keyDown('s')
- If your right thumb is down, press the S key (brake).
- If your right thumb goes up, release the S key (stop braking).
if angle < -straight_threshold:
pyautogui.keyDown('a')
# release after tap_duration
- Angle < -6: Press A (turn left)
- Angle > +6: Press D (turn right)
These are short taps, so the car turns briefly and doesn't stay turning forever.
๐งน At the Endpyautogui.keyUp('w')
pyautogui.keyUp('s')
pyautogui.keyUp('a')
pyautogui.keyUp('d')
- When you exit the program, it releases all keys to avoid the car getting stuck going forward or turning.
Step 4: Explaination of Code and Important Settings in Code Before Using It๐ Explaining the Key Code Linesโ Importing Toolsimport cv2 # To use webcam and show visualsimport mediapipe as mp # To detect handsimport math # To calculate anglesimport pyautogui # To press keyboard buttons automaticallyimport time # To handle timing for button presses
โ Initial Settingshands = mphands.Hands(...)This initializes MediaPipe Hands to detect up to 2 hands.It finds where your hands are and tracks their movement.
โ Detecting Angle Between Handsdef calculate_angle(x1, y1, x2, y2):...Calculates the angle between your left and right hands.This tells whether you're turning left, right, or going straight.
๐ง Important Settings :
๐ turn_sensitivity = 0.01This was used in a previous version to make turning more or less sensitive.BUT in your current code, It's not used anymore, so we can remove it or ignore it.
๐ straight_threshold = 6This value decides how much your hand angle must change before it counts as turning.If the angle is less than -6, It means turn left (A key).If the angle is more than +6, It means turn right (D key).If the angle is between -6 and +6, It means go straight (do nothing).This helps avoid small hand shakes being treated as turns.
โฑ๏ธ tap_duration = 0.01This sets how quickly to release the A or D key after tapping it.0.01 means the key is held down for just 0.01 seconds (very short).This makes the car turn in small steps, so you donโt turn too much with a small hand movement.If you want smoother, longer turns, increase this value to something like 0.05.
๐๏ธ Detecting Thumb Up or Downdef is_thumb_up(hand_landmarks):return hand_landmarks.landmark[4].y < hand_landmarks.landmark[3].yChecks if the tip of the thumb is above the second joint (on the screen).This is used to detect:Left thumb up = stop acceleratingRight thumb up = stop braking๐ Controls Using Hands
๐ข Accelerate (W key)if not is_thumb_up(left_hand):pyautogui.keyDown('w')If your left thumb is down, press the W key (accelerate).If your left thumb goes up, release the W key (stop).
๐ด Brake (S key)if not is_thumb_up(right_hand):pyautogui.keyDown('s')If your right thumb is down, press the S key (brake).If your right thumb goes up, release the S key (stop braking).
โ๏ธ Turn Left or Right (A and D keys)if angle < -straight_threshold:pyautogui.keyDown('a')# release after tap_durationAngle < -6: Press A (turn left)Angle > +6: Press D (turn right)These are short taps, so the car turns briefly and doesn't stay turning forever.
๐งน At the Endpyautogui.keyUp('w')pyautogui.keyUp('s')pyautogui.keyUp('a')pyautogui.keyUp('d')When you exit the program, it releases all keys to avoid the car getting stuck going forward or turning.โ Final Thoughts๐ฎ What You Can Control
Step 5: Important Settings in Code Before Using It๐ง Important Settings :๐ turn_sensitivity = 0.01- This was used in a previous version to make turning more or less sensitive.
- BUT in your current code, it's not used anymore, so we can remove it or ignore it.
This value decides how much your hand angle must change before it counts as turning.
- If the angle is less than -6, it means turn left (A key).
- If the angle is more than +6, it means turn right (D key).
- If the angle is between -6 and +6, it means go straight (do nothing).
This helps avoid small hand shakes being treated as turns.
โฑ๏ธ tap_duration = 0.01This sets how quickly to release the A or D key after tapping it.
- 0.01 means the key is held down for just 0.01 seconds (very short).
- This makes the car turn in small steps, so you donโt turn too much with a small hand movement.
If you want smoother, longer turns, increase this value to something like 0.05
Comments
Please log in or sign up to comment.