Goran Vuksic
Published © MIT

NVIDIA Jetson Orin Nano powered Pit Droid

Star Wars Pit Droid powered by NVIDIA Jetson Orin Nano and vision AI.

AdvancedFull instructions provided8,051
NVIDIA Jetson Orin Nano powered Pit Droid

Things used in this project

Hardware components

Jetson Orin Nano Developer Kit
NVIDIA Jetson Orin Nano Developer Kit
×1
Digital Servo 25kg (270°)
×2
Trust Exis webcam
×1
Arduino relay 5V
×1

Software apps and online services

Microsoft Azure
Microsoft Azure

Hand tools and fabrication machines

3D Printer (generic)
3D Printer (generic)

Story

Read more

Schematics

LED schema

Code

Pit Droid LED

Python
# Pit Droid LED
# Author: Goran Vuksic

import RPi.GPIO as GPIO
from time import sleep

# set mode to BCM
GPIO.setmode(GPIO.BCM)

# define output pin
output_pin = 18

# GPIO setup
GPIO.setup(output_pin, GPIO.OUT)

# turn LED on
GPIO.output(output_pin, 1)
sleep(2)

# turn LED off
GPIO.output(output_pin, 0)
sleep(2)

# cleanup
GPIO.cleanup()

Pit Droid servo

Python
# Pit Droid servo
# Author: Goran Vuksic

import RPi.GPIO as GPIO
from time import sleep

# set mode to BOARD, pins are by numbers on board
GPIO.setmode(GPIO.BOARD)

# define output pin
output_pin = 33

# GPIO setup
GPIO.setup(output_pin, GPIO.OUT)

# start
servo=GPIO.PWM(33, 50)
servo.start(0)
sleep(1)

# move head left
servo.ChangeDutyCycle(5)
sleep(1)

# move head right
servo.ChangeDutyCycle(10)
sleep(1)

# stop and cleanup
servo.stop()
GPIO.cleanup()

Detectnet-led.py

Python
#!/usr/bin/env python3
#
# Copyright (c) 2020, NVIDIA CORPORATION. All rights reserved.
#
# Permission is hereby granted, free of charge, to any person obtaining a
# copy of this software and associated documentation files (the "Software"),
# to deal in the Software without restriction, including without limitation
# the rights to use, copy, modify, merge, publish, distribute, sublicense,
# and/or sell copies of the Software, and to permit persons to whom the
# Software is furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
# THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
# FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
# DEALINGS IN THE SOFTWARE.
#

#######################################################################################
# Pit Droid - detectnet.py is updated to turn on droid lights when person is detected #
#######################################################################################

import sys
import argparse

# import GPIO
import RPi.GPIO as GPIO
from time import sleep

from jetson_inference import detectNet
from jetson_utils import videoSource, videoOutput, Log

# parse the command line
parser = argparse.ArgumentParser(description="Locate objects in a live camera stream using an object detection DNN.", 
                                 formatter_class=argparse.RawTextHelpFormatter, 
                                 epilog=detectNet.Usage() + videoSource.Usage() + videoOutput.Usage() + Log.Usage())

parser.add_argument("input", type=str, default="", nargs='?', help="URI of the input stream")
parser.add_argument("output", type=str, default="", nargs='?', help="URI of the output stream")
parser.add_argument("--network", type=str, default="ssd-mobilenet-v2", help="pre-trained model to load (see below for options)")
parser.add_argument("--overlay", type=str, default="box,labels,conf", help="detection overlay flags (e.g. --overlay=box,labels,conf)\nvalid combinations are:  'box', 'labels', 'conf', 'none'")
parser.add_argument("--threshold", type=float, default=0.5, help="minimum detection threshold to use") 

try:
	args = parser.parse_known_args()[0]
except:
	print("")
	parser.print_help()
	sys.exit(0)

# create video sources and outputs
input = videoSource(args.input, argv=sys.argv)
output = videoOutput(args.output, argv=sys.argv)
	
# load the object detection network
net = detectNet(args.network, sys.argv, args.threshold)

# note: to hard-code the paths to load a model, the following API can be used:
#
# net = detectNet(model="model/ssd-mobilenet.onnx", labels="model/labels.txt", 
#                 input_blob="input_0", output_cvg="scores", output_bbox="boxes", 
#                 threshold=args.threshold)

# set mode to BCM
GPIO.setmode(GPIO.BCM)

# define output pin
output_pin = 18

# GPIO setup
GPIO.setup(output_pin, GPIO.OUT)

# process frames until EOS or the user exits
while True:
    # capture the next image
    img = input.Capture()

    if img is None: # timeout
        continue  
        
    # detect objects in the image (with overlay)
    detections = net.Detect(img, overlay=args.overlay)

    # print the detections
    print("detected {:d} objects in image".format(len(detections)))

    lights = False
    
    for detection in detections:
        # print(detection)
        if int(detection.ClassID) == 1: # person is detected
            lights = True

    if lights:
    	GPIO.output(output_pin, 1)
    else:
        GPIO.output(output_pin, 0)

    # render the image
    output.Render(img)

    # update the title bar
    output.SetStatus("{:s} | Network {:.0f} FPS".format(args.network, net.GetNetworkFPS()))

    # print out performance info
    # net.PrintProfilerTimes()

    # exit on input/output EOS
    if not input.IsStreaming() or not output.IsStreaming():
        break

Pit Droid GitHub

Python code and examples.

Credits

Goran Vuksic

Goran Vuksic

4 projects • 29 followers
Engineering manager, Microsoft AI MVP, cofounder of syntheticAIdata, father, hitchhiker through the galaxy...

Comments

Related channels and tags
  • Internet of Things
  • Machine Learning & AI
  • Microsoft
  • NVIDIA