CV to See Me

December 13, 2018

College of Engineering and Applied Science

Department of Electrical, Computer, & Energy Engineering

University of Colorado Boulder

Project Architecture Diagram and Abstract

Fig. 1 - Project Architecture Diagram

The project consisted of using a Raspberry Pi Zero W as our main computer to be able to work with a Camera and LED Strip to implement color detection and identification. In doing so we would be able to create a low-level application of color identification possible for others to recreate.

The camera was operated using OpenCV image manipulation library which is a free open-source software to use in order to allow us to analyze images. Additionally, the LED Strip was operated using a Raspberry Pi library for our WS2812 LED Strip to allow us to control and output information to the LED Strip.

We were able to get the two components working separately by being able to get the camera to function and getting the LED strip to light up in a certain fashion.

Not all of our initial design criteria were met and will be mentioned in the latter portion of this report.

Reason for choice of project

The reason for selecting this project was primarily based on the components that each member of the group had from the previous projects allowing us to avoid having to wait for components to arrive and the additional cost of purchasing components.

In addition to having the components assisting with time and cost, we wanted to challenge ourselves to work with a different board and opted to use the Raspberry Pi Zero W. By using this board we were able to integrate a platform with CV into our project and have this be one of our main design features.

Lastly, we wanted to experiment with a low-level color detector and see if it was possible to create this detector with a simple and easily accessible Raspberry Pi. In doing so we would be able to create a simple project for people to utilize color detection for their projects.

Implementation Methodology

There were two sections to the project dealing with the different components of the project. The First section consisted of working with the Raspberry Pi Camera which involved downloading the appropriate software for the project. The second section consisted of utilizing the GPIO pins with the LED strip.

Initially starting the project, the fundamental process was learning the environment of raspbian. Raspbian was burned as an image file onto a SD card and inserted into the Raspberry Pi. Upon powering up the Raspberry Pi with the SD card and connecting the HDMI, the GUI is launched and we are able to work with the Raspberry Pi.

After getting Raspbian Stretch onto the Raspberry Pi via the SD card, we attempted to install OpenCV and ran through 7 implementations after meeting multiple issues. Once installing OpenCV on the Raspberry Pi 3 B++ model we moved the SD card back into the Raspberry Pi Zero W and were able to operate OpenCV and the camera on our initial board.

Once downloading OpenCV we began by implementing a simple test in which we would identify whether a color was present in an image. After accomplishing this test, we were able to identify that OpenCV was working correctly and an initial step to our project to have color detection was taken.

We found that it was difficult to have a video constantly streaming and having to detect color and went with the route of having the camera take a picture and then having us analyze this picture to see if a certain color appears. If the color is apparent in the picture, we would then send data to the LED strip to indicate that the color appeared in the picture.

Fig. 2 - OpenCV testing color detection (red)

After working with the OpenCV, we began implementing the second section of the project which was controlling the LED Strip. Looking at the hardware of the LED strip, the strip had 3 direct pins (Vout, Data, GND) and two external wires (5V Vin and ground). As a result, the Vout and GND were connected to a power supply without an output of 5V and 0.2A of current and the Data connection was connected to GPIO pin 18 on the Raspberry Pi.

Once the hardware connections were made, the software was written to control the addressable LEDs on the LED strip. This was done by using a Raspberry Pi library for our specific LED strip (WS2812 LED Strip) as a guide. By using the library as a guide we implemented the LED strip to light up to the specified color of the object that was detected by the Raspberry Pi Camera. This specific color lighting was done by sending a PWM signal to the LED strip.

Fig. 3 - LED Strip Detecting Color of Image (red)

Challenges Faced / Limitations with the board and peripherals

We underwent some problems that elongated the project's deadlines. The problem mainly occurred with trying to install OpenCV onto the Raspberry Pi Zero W. With the Raspberry Pi Zero only having a single core, the installation period would have taken 9 hours without any errors occurring; however, we were met with many errors upon the installation.

Upon getting issues with trying to install OpenCV, we tried to work with an alternative CV software and selected to use SimpleCV. We were able to install SimpleCV; however, ran into problems with the shell and it being unable to detect the streaming camera. This issue was found by us using a messed up camera cable as well as not having a certain package.

Once we concluded these errors, we got a new camera and then decided to switch to using the Raspberry Pi 3 B++ model which would be an upgrade from the Raspberry Pi Zero W in terms of downloading speed. With the Raspberry Pi 3 B++ having four core processors we were able to download OpenCV in 1 hour instead of 9 hours. After downloading OpenCV onto the Raspbian Image (SD card), we had issues with the Raspberry Pi 3 B++ not detecting our camera and moved the SD card back into the Raspberry Pi Zero W and were able to get the camera and OpenCV working.

Additionally, we found out that the Raspberry Pi does not have a built-in ADC and therefore we were unable to utilize the flex sensor since this is an additional add-on, we decided to let go of this option and focus on the two important sections of the project (Camera and LED strip).

Future scope of the project

In the future to improve our project, we would want to make some addition that would allow us to meet our initial design criteria and add additional features to the project.

We would want to make an addition of using an external ADC chip to utilize the flex sensors. By using this additional ADC chip, we would communicate by SPI and be able to use a voltage divider and get a digital voltage value for the amount that the flex sensors are bent. By using these flex sensors, we would be able to control the LED strips in regards to the amount of LEDs lit up as well as the brightness of the LED strip.

We would also like to add new peripherals to the project to make the system more volatile and valuable. It would be to include a Bluetooth module that would allow us to print out the information and data of a certain color to a phone via a Bluetooth terminal application. This would allow us to add to the purpose and benefit of the project, by being able to assist those who are color blind to be able to know what color an object is.

Lastly, we want to be able to detect the color of an object during a video stream which would allow us to be able to identify the color of an object in real-time. This would enable us to be able to detect the color of an object while the camera is recording.

Video

References

Installing OpenCV

• https://www.pyimagesearch.com/2017/09/04/raspbian-stretch-install-opencv-3-python-on-your-raspberry-pi/

Working with LED Strip

• https://tutorials-raspberrypi.com/connect-control-raspberry-pi-ws2812-rgb-led-strips/
• https://github.com/BSFEMA/RPi_WS2812

Taking picture on Raspberry Pi Camera

• https://projects.raspberrypi.org/en/projects/getting-started-with-picamera

Working with OpenCV

• https://www.youtube.com/watch?v=rVTab3WgynU

Embedded Video Link of Final Project

• https://drive.google.com/open?id=1yatORFlWKS5ktKJBbx7kxoTKm5lM9scT