OptoInspect3D: AI-driven optical Inspection with a modified

Project Story: Automated PCB Inspection with Modified 3D Printer and AI

In this project, I embarked on a journey to transform a standard 3D printer into an automated PCB inspection machine. My goal was to integrate a USB microscope with additional LED illumination into the printer, allowing for detailed and automated inspection of printed circuit boards (PCBs). The project involved a combination of hardware modifications, 3D design, software development, and machine learning model training.

Modifying the 3D Printer

Hardware Setup: started with a Tronxy X5S 3D printer, which provided a robust and versatile platform for modification. To enhance the printer's capabilities for PCB inspection, I added the following components:

• USB Microscope: This was mounted to provide high-resolution imaging of the PCB.

• Additional LED Illumination: To ensure clear and bright images, I added extra LED lights around the microscope.
• Duet3D Duet 2 Controller: This advanced controller allowed precise control over the printer's movements, essential for scanning PCBs accurately.

3D Design:Using CAD software, I designed a custom 3D-printed part to securely mount the USB microscope onto the printer. The design considered stability and ease of adjustment to accommodate different PCB sizes and inspection angles.

Software Development

Python Project with Tkinter:I developed a Python application using Tkinter for the graphical user interface (GUI). The application integrated several key functionalities:

• Video Acquisition with OpenCV: The software used OpenCV to capture real-time video feed from the USB microscope.
• Printer Control with Duet3D API: The Duet3D API allowed the application to control the printer's movements, enabling precise positioning of the microscope.
• Manual and Automatic Controls: Users could manually control the microscope's position, mark reference points, and take measurements. The auto-scan function allowed the system to automatically capture images across the entire PCB and stitch them together for a comprehensive inspection.
• Settings Storage: A configuration file in JSON format stored user settings and preferences, making the application customizable and user-friendly.

Application Features:

• Camera View with Crosshair: The main interface displayed the live feed from the microscope, overlaid with a crosshair for accurate targeting.
• Manual Motor Positioning: Controls allowed for manual adjustment of the microscope's position to inspect specific areas of the PCB.
• Coordinates Information: The current position of the microscope was displayed, helping users keep track of inspection points.
• Measurement and Mark Tools: Users could measure distances and mark areas of interest directly on the video feed.
• Auto Scan Function: This feature automated the inspection process, capturing and stitching images of the entire PCB for thorough analysis.

Machine Learning Integration

Model Training with Google Colab:To automate defect detection, I trained a YOLO model on Google Colab. I used the "FICS-PCB: A Multi-Modal Image Dataset for Automated Printed Circuit Board Visual Inspection" dataset, focusing on identifying capacitors and resistors (perhaps the most difficult parts to identify).

Deploying the Model with KR260 and Vitis AI:

I haven't been able yet to deploy the model on the Kria, is still work-in-progress. Now the KR260 is capable of running the application, but it is planned to do the inference on the device. This hardware-software combination allowed for efficient and accurate inference on the captured PCB images, detecting and highlighting defects such as misplaced or damaged components.

Future Enhancements:The project has room for further improvements, including integrating more advanced image sensors and expanding the model to detect a wider range of PCB components and defects. These enhancements will make the system even more versatile and powerful.

For now, the model was tested against some scants, and it correctly identifies capacitors and resistor when they are well defined. But some images don't have great quality, and in these images, detection fails.

Conclusion

By combining hardware modifications, custom software development, and machine learning, I created an automated PCB inspection system. The project not only demonstrates the potential of repurposing existing hardware for new applications but also showcases the power of AI in enhancing manufacturing processes. The comprehensive documentation and open-source code will enable others to recreate and build upon this project, fostering further innovation in the field of automated inspection systems.