Control Camera Position with Smartphone Gyroscope

Beginning

In the previous tutorial (here) I showed how to display the image from the Raspberry Pi Camera in the browser window. In this tutorial I will show you how to control the position of the camera. We will control in two ways

• By moving the mouse / finger on the video component
• Tilting the phone and using the built-in gyroscope

All this on the website opened in the browser (I used Chrome for both the desktop and smartphone, because it has the best support for the webRTC protocol).

Whole tutorial in video form:

How it's going to work

Just as the camera image is sent via the webRTC protocol, we will use the same channel to send instructions to control the position of the camera. Messages from the browser will go to the Python script on RasbeprryPi, and this script will drive the server driver accordingly. It sounds simple and so it will be – sending messages is completely on the RemoteMe side, in the Python script you only need to implement a function that is run after changing the value of the variable – more about variables here

What is needed:

Camera preview

• Raspberry PI Zero W
• Rasbperry Pi dedicated camera
• Contorl servo 16-bit PWM Adafruit Channel controler I2C
• Two servo mechanisms compatybile wiht the tilt
• A camera tilt
• Power – it does not give specific applications – because they depend on the motors themselves and on what kind of batteries we have at our disposal. I used LiPo 4 * 1.2V batteries with a step-down converter

Connecting servo mechanisms

To control the servo mechanisms, we will use a 12-channel LED driver 16-bit PWM I2C compatible with Adafruit. It communicates with Rpi via the I2C interface. Adafruit provides a python library that works out-of-the-box, If in the second step RemoteMe installation on RPi you have chosen the [Yes] option, you have this library already installed;).

Connections:

SDA, SCL, and ground pins connect to RPI as shown:

Rpi pins:

Full RPI pins image here

Mounting the webcam tilt handle – it should not be difficult, the only thing you need to pay attention to is the assembly of the holder, so that it looks straight ahead and what is most important – the servo mechanisms were in a central position.

In the pictures and the movie you can see the green PCB to which the RPI and the servo driver are mounted, the files you can download from here it was designed in such a way that it can be easily performed using the thermal transfer method. In the repository there are also gerber files for easy ordering, eg from allpcb.com where I ordered my PCB.

The plate will be more useful when we add engine control to our car in the next course.

How its going to work:

• to camera control we will use a variable of the type SMALL_INTEGER_2, which sends two numerical values, one of them we will control the axis x with the second axis y.
• the variable will be changed by phone or computer where the image will be displayed
• by moving the mouse / finger around the area where the video is displayed
• using a gyroscope in the smartphone where the rotation from the starting position will control the camera axes

Let’s start:

We will start by creating the variables – tab variables -> add the appeared window we complete:

We create by clicking Submit more about variables here.

Then, connect our RasbperryPi to RemoteMe.org how to connect here. Next, on the bar of the connected Raspberry, click the burger menu and “Add External script with wizard”

In the first step, select the variable cameraPos

Thanks to this, in the generated code we will have functions ready to be implemented to interact with our variable.

In the second step, we name our device “python” and any unused Id. In the last step, click “Create device”. More about python devices here.

In the open window, we will add a code to support our servo mechanisms:

import logging
import socket
 
import struct
import sys
import os
 
os.chdir(sys.argv[1])
sys.path.append('../base')
 
import remoteme
 
import Adafruit_PCA9685 # Added
import time # Added
import RPi.GPIO as GPIO # Added
 
pwm = None; # Added
 
def setCameraPos(i1, i2):
    remoteMe.getVariables().setSmallInteger2("cameraPos", i1, i2)
 
 
 
def onCameraPosChange(i1, i2):
    global pwm # Added
    logger.info("on camera change {} , {}".format(i1, i2)) # Added
    pwm.set_pwm(1, 0, i1) # Added
    pwm.set_pwm(0, 0, i2) # Added
    pass # Added
 
def setupPWM(): # Added
    global pwm # Added
    pwm = Adafruit_PCA9685.PCA9685() # Added
    pwm.set_pwm_freq(80) # Added
    
try:
 
    logging.basicConfig(level=logging.DEBUG,
    format='%(asctime)s %(name)-12s %(levelname)-8s %(message)s',
    datefmt='%d.%m %H:%M',
    filename="logs.log")
 
    logger = logging.getLogger('application')
    setupPWM() # Added
    remoteMe = remoteme.RemoteMe()
 
    remoteMe.startRemoteMe(sys.argv)
    remoteMe.getVariables().observeSmallInteger2("cameraPos" ,onCameraPosChange);
 
    remoteMe.wait()
 
finally:
    pass

The added lines are marked with the #Added comment

Code overview

import Adafruit_PCA9685 # Added
import time # Added
import RPi.GPIO as GPIO # Added
 
pwm = None; # Added

Adding libraries and global variable declaration pwm to control the PWM controller (Adafruit_PCA9685)

def onCameraPosChange(i1, i2):
    global pwm 
    logger.info("on camera change {} , {}".format(i1, i2)) 
    pwm.set_pwm(1, 0, i1) 
    pwm.set_pwm(0, 0, i2)
    pass

This function will be called when you want to change the position of the camera with a smart phone or computer. In the parameters, we get the values of the x and y axes, and set the servo mechanisms connected to the PWM controller at position 0 and 1. We will already receive variables, with appropriate values, for example, the value (534.234) will mean the central position of the camera – therefore we simply enter the received numbers into the server driver without formatting them.

def setupPWM(): 
    global pwm 
    pwm = Adafruit_PCA9685.PCA9685()
    pwm.set_pwm_freq(80)

Function for PWM controller library initialization. The value of 80 was chosen by trial and error to my servo mechanisms. This function is called in the main program block.

After saving the code, the program on RasbperryPi will be reset to load the new script.

Web Page

Now we will add a web page with two sliders to find the central position of the camera and maximum swing. In the ideal world, the numbers would be, for example, 0.100 for both values, i.e. sending a variable of value (50.50), we would set our camera in the central position, and the value (100, 100) would direct it up and right. Unfortunately, these numbers will certainly be different and we have to find them. For this purpose, we will create a website with two sliders that will send variables and we will know what are the variables.

To add a website – choose “New Device” -> “Webpage” and complete it as follows:

After adding, expand the beam, click index.html and “edit with wizard”. Then “Insert component” and complete:

More about components here

The most important information

• we will control two sliders of the CameraPos variable
• the minimum and maximum number that we will send is 0 – 1000 for both axes

Click the insert, close the wizard window, and open the page in the new tab by clicking index.html and selecting Open in new tab.

We will get our website. The first slider is responsible for the X-axis of the camera (horizontal movements – ie looking left right) and the second one for the Y axis (ie looking up, down). When the sliders control differently, we replace the servo plugs with mechanisms in places. When the servos do not respond, check to see if they have been plugged into the right place.

Moving the sliders set the camera in a central position for me it is x = 564 and y = 474.

Then we tilt the camera up to the minimum values for me then x = 298. y = 223.

Then, calculate the maximum deflection for the camera, the calculation is to make the central position in the middle of the interval. So simple math

Xmax  = Xcenter-Xmin + Xcenter = 564 – 298 + 564 = 830Ymax = Ycenter-Ymin + Ycenter = 474 – 223+ 474 = 725

That is, Xmin = 298, Xcenter = 564, Xmax = 830. has an axis looks like this:

The point is that the distances between the values were the same and additionally the Xcenter value indicated the central position of the camera in the horizontal plane.

Y-axe : Ymin=223, Ycenter=474, Ymax=725

On the sliders, check if the maximum values set the camera in the right position without any problem and the servos do not block – if they block, increase Xmin and repeat the calculation (or calculate Xmin based on Xmax).

We are going back to the index.html page and delete the sliders removing the line from the page’s sources:

<variable  component="slider"  type="SMALL_INTEGER_2"  name="cameraPos" label="camera" min="0" max="1000" valuebox="true" ></variable>

And we re-open the Components Wizard and insert the components

camera:

Status – in the upper right corner it will display the connection status:

Finger / mouse control by moving on the image from the camera:

Please note that I have supplemented the Xmin values with the calculated values.

and Gyroscope – so that we can control the camera position by tilt the smarphone :

I also put the calculated values ​​here. More about components and configurations here

It’s also worth changing the size of the video in index.html sources instead of width and height, enter style = "width: calc (100vmin); height: calc (75vmin)" – this will fill the camera image when the phone is in a vertical position

We open our website in Smaprtphonie – the easiest way is by clicking on index.html -> Get anymous Link -> click on the QR code icon and scan it with a smartphone.

We test by sliding the finger on the screen, when the camera instead of pointing up its moving down in the component <variable component = "cameraMouseTrack" index.html change invertX = "false" into invertX = "true"

If the same applies to the Y axis, we change the value of invertY in the same component

We turn on the gyroscope by clicking the Gyroscope button – it remembers the position of the smartphone and changes the position of the camera – when titl  the phone, when moving is opposite then should be we changed similar like in previous step invertXinvertY but now  in component <variable component = "gyroscope". If we want the gyroscope to react more gently, we increase the values ​​of the xRange and yRange properties

That’s all for today in the video at the beginning of the course are all steps.

Greetings,

Maciek