Achieve color and image recognition by myPalletizer

Based on the Linux system and a 1:1 simulation model in ROS, the AI Kit composes of the vision, positioned gripping, and automatic sorting modules. Featuring computer vision, an equipped camera can recognize and locate the cubes of different colors or images through OpenCV, and then the core processor of the a robotic arm can calculate their current and targeted spatial coordinate positions, and finally grip a cube into the corresponding barrels.

Now myPalletizer 260 is capitable with AI Kit:

Here is the detailed process of achieving color and image recognition by myPalletizer AI Kit.

1. Color recognition

1. Press "Ctrl"+"Alt"+"T" on the keyboard to open a command window and run the following command to start the master node:

roscore

2. Press "Ctrl"+"Shift"+"T" in the command terminal to open another command window in the same directory, and then type the command to launch the "vision.launch" file:

roslaunch myPalletizer_260 vision.launch

After that, we can get the result as shown in the figure:

3. Press "Ctrl"+"Alt"+"T" to open another command window and enter the command to go to the file directory to be operated

cd catkin_ws/src/mycobot_ros/mycobot_ai/myPalletizer_260/

4. Run the program "detect_obj_color.py", then we can acheive the color recognition and capture

python scripts/detect_obj_color.py

After that, we can get the result as shown in the figure:

2. Image recognition

1. Before running "detect_obj_IMG_FOLder_opt. py" program, we need to ensure that steps 1, 2 and 3 in color recognition have been performed

2. Run the program "add_img.py" and add the images we want to recognize to the specified folder ('red', 'blue', 'green', 'gray').

python scripts/add_img.py

According to the prompts input by the terminal, we capture the image in the second image box.

3. After the image area capturing is completed, press "Enter" and enter the name of the folder to save the image (' Red ', 'Blue ', ' Green ', 'Gray ') as prompted by the terminal. Press "Enter" again to save the image to the corresponding folder.

4. Start the image recognition program "detect_obj_img_folder.py", and verify that the robotic arm can correctly pick the pictures added in step 2 and 3, and place them into the barrel with the corresponding color.

python scripts/detect_obj_img_folder_opt.py

After that, we can get the result as shown in the figure:

This is designed for individuals developers as well as STEM educators who are interested in applications combining AI with robotics. Unless color and image recognition, what application scenarios do you want to achieve if you get the AI Kit? Please share with us in the comment!