All you need is Stereo

This is a project built on the Texas Instruments F28379D launchpad board. All the control including motor PI, servo, and ball-retrieving proportional control is done on the CPU1 of this board.

All the components including the resistors, capacitors, headers, and LEDs are soldered and put together myself. I also added the adjustable camera mount for the camera.

Our project is very vision focused – we use a Raspberry PI with an Intel RealSense stereo camera attached to identify objects by color, tell a distance between our robot and the object, and then we will make use of other simpler sensors and machines such as servo motors to do object retrieval tasks.

Our program has 4 states: Fetching State, Scanning State, Grabbing State, and Return State.

First, we will put our robot on the ground along with blue, yellow, and red ping pong balls and a home base. Our goal is to bring them back to base in color order (red-blue-yellow).

• Scanning State: Then, our robot will turn slowly and scan for the color we want.
• Fetching State: Once we find the color we want, we will use the proportional control algorithm to center and chase the ball using the depth information given by the RealSense camera. And the robot will stop at a distance of around 20cm to the ball.
• Grabbing State: The robot will then be hardcoded to lift the gripper and move forward to capture the ball.
• Return State: After successfully retrieving all the balls, it will return to the home position labeled with the green color.

Thank you for watching! Don't hesitate to comment below if you have any questions.