Kinect-Based Sentry Turret

The Idea

What is there to say? I thought it would be cool to make an automated turret. There probably isn't much (legal and safe) practical use for it, but it's cool and it was a fun build.

How Does It Work?

A Kinect V2 handles the body tracking (if you didn't know, Kinects are pretty cool for this, and it's a shame that the consumer-friendly versions are no longer in production).

The Kinect data is processed by a desktop app, which was built with Unity. This app scans through the tracked body parts, determines what is important, and performs some trigonometry to get the angles that the turret needs to face the target. It then figures out what state it should be in (waiting, tracking, firing, etc). Based on that state, it issues voice commands to the would-be intruder. There are also a few simple gestures that can be used to change the state (hands up, or the secretdeactivationgesture).

Once all that is done, the app sends the angle data and firing state via serial port to an Arduino (an Adafruit Metro Mini, in this case). The Arduino code is pretty simple. It just processes the incoming bytes, sets the angle on a pair of pan/tilt servos, and sets a signal to a relay for firing.

I built a mount out of a piece of scrap wood. It lets me attach the Kinect and servos at a known distance, which helps with the trig calculations. The whole thing is threaded to fit a tripod for easy setup.

There is a little more to it than that, but I suggest watching the video below to see it in action.

Notes and Improvements

You're on your own for finding a suitable airsoft gun. I got one from eBay. I chose electric so I could fire it with a signal. If you go a different route, you'll have to rig something up to pull the trigger on command.

The servos slightly struggle to handle the weight of the gun, and they really struggle to deal with the recoil when firing.

The gestures are extremely simple and therefore finnicky. At the very least, the data could be put through a low-pass filter reduce accidental detections.

Build It Yourself

• Plug in a Kinect and install the drivers
• Download the Kinect SDK Unity Pro Package here: https://developer.microsoft.com/en-us/windows/kinect/
• Create a new Unity project (only tested using Unity 2019.1.8)
• Import the Kinect Asset Package into your new project
• Download the repo for this project
• Copy the "Unity Assets" folder from the repo into your Unity project's Assets folder
• Open the Kinect Turret scene in the Unity Project
• Connect an Adafruit Metro Mini to your PC, and determine what serial port it is connected to (Tera Term is a good tool for this)
• Select the "Serial" object in the Unity scene, and replace the "Port Name" field with the one you found above
• Wire up the Arduino according to the schematic
• Measure the distance from the Kinect to the Turret (X, Y, and Z distances, if necessary)
• Select the "Motor Controller" object and stick the measured distance value in the "Kinect Motor Offset" field (make sure to convert to meters, and note that the sign of the value depends on which direction the offset is from the Kinect)
• Now you should be all set to Play the Unity scene and get shot

Video