Portable ADAS system using KRIA KR260 starter kit

My Chaotic Journey into 'Portable ADAS': A Beginner's Tale of Triumph (and Lots of Errors)

Hey folks! Buckle up, because I'm about to take you on a rollercoaster ride through my attempt at creating an Advanced Driver Assistance System (ADAS). Spoiler alert: I'm a total newbie, and this journey was about as smooth as a road full of potholes!

The Beginning: Why ADAS?

Why I Decided to Dive In

So, what possessed me to dive into this world of ADAS? Well, making cars safer sounded cool, and I thought, "How hard could it be?"

(Narrator: It was very hard.)

Advanced Driver Assistance Systems are designed to increase car safety by helping drivers navigate the roads more safely. From lane-keeping assistance to collision warnings, these systems can help prevent accidents and save lives. As someone who loves cars and technology, this was a challenge I couldn't resist. What could possibly go wrong?

What Is ADAS, Anyway?

For those new to the concept (like I was not too long ago), ADAS are electronic systems in vehicles that use advanced technologies to assist the driver. They provide vital information, automate difficult tasks, and improve overall safety. Features include:

• Lane Departure Warning (LDW): Alerts the driver if the vehicle begins to drift out of its lane. Because apparently, we all need a digital nagging spouse on board.
• Adaptive Cruise Control (ACC): Automatically adjusts the vehicle’s speed to maintain a safe distance from the vehicle ahead. It's like your car is playing tag with the car in front of you.
• Collision Avoidance Systems: Warns the driver of potential collisions and can even apply the brakes automatically. Basically, it's your car telling you, "No, you can't smash into that thing."
• Blind Spot Detection: Notifies the driver of vehicles in blind spots that may not be visible. In other words, it’s there to save you from your poor spatial awareness.

With all these features, I was eager to create a system that could potentially make driving safer for everyone. I mean, who wouldn't want to add some digital bells and whistles to their car?

The Hardware Hustle

1. KRIA KR260 Robotics Starter Kit

What Is It?

This little beauty from Xilinx became my new best friend. The KRIA KR260 Robotics Starter Kit is a hardware development platform based on FPGA (Field-Programmable Gate Array) technology, designed to help developers and engineers build advanced robotics and ADAS applications. Picture a tiny computer that makes your big computer feel inadequate.

Why Did I Choose It?

The KR260 is designed for high-performance computing tasks and is highly customizable. It's great for handling parallel processing tasks that are common in ADAS applications, like image and signal processing. Plus, it has the “wow” factor of being something people pretend to understand but don’t.

Challenges Faced:

• Learning Curve: Coming from a software background, the world of hardware and FPGA was entirely new to me. Learning how to program and configure an FPGA was like learning a new language. It's like deciding to learn Mandarin because you've seen a couple of kung fu movies.
• Hardware Setup: Even setting up the board was challenging. I had to ensure all connections were correct and troubleshoot any issues that arose. It was like building a complex Lego set without the instructions, while blindfolded, underwater.

2. LAN Cable

What Is It?

A LAN (Local Area Network) cable is used to connect devices within a network. For my project, it was crucial for connecting my PC to the KR260 and ensuring seamless communication between them. Think of it as a gossip channel for your tech.

Challenges Faced:

• Networking Issues: Ensuring all devices were communicating properly over the network was trickier than expected. I had to configure IP addresses and troubleshoot connectivity issues, which often felt like dealing with stubborn toddlers who didn’t want to cooperate. I learned that my devices were as moody as a teenager who just got their phone taken away.

3. SD Card for PetaLinux

What Is It?

At first, I thought PetaLinux was a new exotic pet. Turns out, it's an embedded Linux distribution tailored for Xilinx hardware platforms like the KR260. It's basically Linux on a diet, made specifically for people who enjoy the thrill of confusion.

Why Did I Need It?

PetaLinux is essential for running software on the KR260. It's a lightweight operating system that allows the development and execution of custom applications directly on the board. Because nothing screams “I know what I’m doing” like running a complex OS on a tiny card.

Challenges Faced:

• Installation Issues: Installing PetaLinux was no walk in the park. I had to ensure the SD card was properly formatted and that the image was correctly written to the card. My card reader became the real MVP, enduring all my trial and error.
• Configuration: Setting up PetaLinux involved configuring various components, which required me to understand more about embedded Linux systems than I ever anticipated. Who knew installing an OS would make me long for the days of using MS Paint?

4. My Wheezing PC

What Is It?

Armed with an NVIDIA graphics card that's probably older than some of you reading this, my PC was tasked with handling all the software and simulations for the project. It was like asking a grandma to run a marathon – bless her soul, she tried her best.

Challenges Faced:

• Performance Limitations: My PC struggled under the weight of the simulations and software. It often felt like pushing a boulder uphill. The poor thing would sigh heavily every time I opened a new program.
• Upgrading Hardware: I had to consider potential upgrades to keep up with the demands of the project. Managing these updates required some financial planning and creative problem-solving. (Read: convincing my wallet to open a bit wider.)

The Software Saga

Now, here's where things got really interesting (read: frustrating).

1. Vivado Design Suite

What Is It?

Vivado Design Suite is a software suite from Xilinx used for synthesizing and analyzing HDL (Hardware Description Language) designs, primarily targeting FPGA products. It’s like Photoshop for hardware, but with way more tears.

Why Use Vivado?

Vivado is crucial for designing and implementing the hardware components of the ADAS system. It allows you to create the logic circuits needed to process data efficiently. In other words, it’s the magical cauldron where all your circuit spells are brewed.

The DPU IP in Vivado:

The real hero here is the DPU (Deep Processing Unit) IP. This magical piece of technology runs AI models directly on the FPGA, making everything faster and more efficient. Think of it as the adrenaline shot your system needs to process data in real-time.

Running Models in DPU:

I managed to run my AI models on the DPU in the Programmable Logic (PL) of the FPGA. This was the heart of my ADAS system, handling all the intensive AI computations like lane detection and object recognition. It’s like having a tiny genius living inside your hardware, doing all the smart stuff so you don’t have to.

Challenges Faced:

Installation Woes: Installing this beast took longer than it takes for paint to dry. Just when I thought it was done – BAM! Error at 99%. This happened multiple times, folks. Days of my life I'll never get back.

• Solution: Installing libcurse before Vivado in Linux was the magic fix. Why isn't this in big, flashing letters on their website?! It’s like they expect you to know these hidden secrets.
• Installation Woes: Installing this beast took longer than it takes for paint to dry. Just when I thought it was done – BAM! Error at 99%. This happened multiple times, folks. Days of my life I'll never get back.Solution: Installing libcurse before Vivado in Linux was the magic fix. Why isn't this in big, flashing letters on their website?! It’s like they expect you to know these hidden secrets.

Configuring the DPU: Getting the DPU to run efficiently was another story. I spent countless hours tinkering with settings and configurations to make it all work. It was like trying to tune a piano with a hammer.

• Configuring the DPU: Getting the DPU to run efficiently was another story. I spent countless hours tinkering with settings and configurations to make it all work. It was like trying to tune a piano with a hammer.

2. CARLA Simulator

What Is It?

CARLA is an open-source autonomous driving simulator designed to support the development, training, and validation of ADAS and autonomous driving systems. In other words, it's a video game for adults who want to pretend they’re working.

Why Use CARLA?

Simulating real-world driving scenarios is vital for testing and refining ADAS applications. CARLA provides a safe and controlled environment for experimentation without the risks of actual road testing. It's like crashing cars without the insurance claims.

Challenges Faced:

Building from Source: The support period was over (great timing, me!), so I had to build it from source. Cue the montage of me pulling my hair out.

XServer Errors: The fix? This magic spell:

bash

Copy code

export VK_ICD_FILENAMES="/usr/share/vulkan/icd.d/nvidia_icd.json"

bash
Copy code
export VK_ICD_FILENAMES="/usr/share/vulkan/icd.d/nvidia_icd.json"

I don't know what it means, but it works! It’s like casting a spell and hoping the incantation is right.

• XServer Errors: The fix? This magic spell:bashCopy codeexport VK_ICD_FILENAMES="/usr/share/vulkan/icd.d/nvidia_icd.json"
I don't know what it means, but it works! It’s like casting a spell and hoping the incantation is right.
• Building from Source: The support period was over (great timing, me!), so I had to build it from source. Cue the montage of me pulling my hair out.XServer Errors: The fix? This magic spell:bashCopy codeexport VK_ICD_FILENAMES="/usr/share/vulkan/icd.d/nvidia_icd.json"
I don't know what it means, but it works! It’s like casting a spell and hoping the incantation is right.

Performance Issues: My PC struggles with CARLA like I struggle with morning jogs. To run it without melting my computer, I had to use:

bash

Copy code

./CarlaUE4.sh -quality-level=Low

bash
Copy code
./CarlaUE4.sh -quality-level=Low

Now it looks like a game from the 90s, but hey, it runs! It’s got that retro vibe, which is totally in, right?

• Performance Issues: My PC struggles with CARLA like I struggle with morning jogs. To run it without melting my computer, I had to use:bashCopy code./CarlaUE4.sh -quality-level=Low
Now it looks like a game from the 90s, but hey, it runs! It’s got that retro vibe, which is totally in, right?

3. Vitis AI

What Is It?

Vitis AI is a development platform from Xilinx for AI inference on AMD and Xilinx hardware, allowing you to deploy AI models on FPGAs and ACAPs. Essentially, it’s the thing that makes your hardware feel like it graduated with honors in AI.

Why Use Vitis AI?

Vitis AI is essential for developing and optimizing AI models for the ADAS system. It provides tools for quantizing and compiling models to run efficiently on the DPU. It’s the secret sauce that makes your AI as efficient as possible.

Challenges Faced:

• Model Optimization: Adapting AI models to run efficiently on the DPU required understanding quantization and compilation techniques. It’s like fitting a square peg into a round hole, only the peg is a neural network.
• Toolchain Complexity: Navigating the Vitis AI toolchain was another steep learning curve. It was like solving a Rubik’s cube, except it changes colors every time you blink.

4. PetaLinux

What Is It?

PetaLinux is an embedded Linux distribution for Xilinx hardware platforms. It allows you to develop custom applications and run them directly on your FPGA. Think of it as the lightweight champion of operating systems, packing a punch without the bloat.

Why Use PetaLinux?

PetaLinux provides a robust and flexible environment for running applications on the KR260. It’s the glue that holds everything together, ensuring the software and hardware work in harmony. Like peanut butter and jelly, only with more command-line interfaces.

Challenges Faced:

Booting and Configuration: Getting PetaLinux up and running was a journey in itself. It required careful configuration and plenty of trial and error.

• Tutorial to the Rescue: An amazing tutorial on Hackster.io saved my bacon. Still not sure what I did, but it worked! Sometimes following instructions feels like casting spells, but when they work, you feel like a wizard.
• Booting and Configuration: Getting PetaLinux up and running was a journey in itself. It required careful configuration and plenty of trial and error.Tutorial to the Rescue: An amazing tutorial on Hackster.io saved my bacon. Still not sure what I did, but it worked! Sometimes following instructions feels like casting spells, but when they work, you feel like a wizard.

The Process (or, How I Learned to Stop Worrying and Love the Error Messages)

Step 1: Data Collection from CARLA

What I Did:

When CARLA wasn't freezing or looking like a flipbook, it was actually pretty cool! I used it to collect data for training and testing my ADAS models.

Challenges Faced:

• Data Management: Collecting and organizing the data was a time-consuming process. I had to ensure the data was clean and properly labeled, which is about as fun as it sounds.
• System Performance: My computer struggled to handle the data processing, requiring creative solutions to optimize performance. My PC whined like a toddler who didn’t want to clean up their room.

Step 2: Lane and Object Detection

What I Did:

Ever tried teaching a computer to see? It's like trying to explain colors to a dog. But with enough AI magic (and a lot of coffee), it started working!

Challenges Faced:

• AI Model Training: Training AI models to accurately detect lanes and objects required experimenting with different architectures and techniques. It was like baking a cake but without a recipe.
• Performance Tuning: Optimizing the models to run efficiently on the DPU involved quantization and fine-tuning. I had to play detective to figure out why my models weren’t performing as expected.

Step 3: Booting PetaLinux on KR260

What I Did:

With the help of an amazing tutorial on Hackster.io, I managed to boot PetaLinux on the KR260. This involved configuring the hardware platform and deploying the operating system.

Challenges Faced:

• Hardware Configuration: Setting up the KR260 and ensuring all components were correctly configured required careful attention to detail. I felt like a surgeon trying to connect the right arteries and veins.
• Debugging: I encountered numerous issues during the boot process, requiring extensive debugging and troubleshooting. The Hackster.io community was a lifesaver in resolving these issues. My error messages looked like a word puzzle, and I became an expert in deciphering cryptic codes.

Step 4: Making Everything Talk

What I Did:

Integrating all the components and ensuring they communicated effectively was like orchestrating a complex symphony. I had to ensure the sensors, FPGA, AI models, and CARLA simulator worked seamlessly together.

Challenges Faced:

• Communication Protocols: Understanding and implementing the correct communication protocols between components was essential. This involved learning about interfaces like I2C, SPI, and Ethernet. It was like playing telephone, but with a lot more wires.
• Synchronization: Ensuring all parts of the system were synchronized and operated in harmony was challenging. It required precise timing and coordination. I felt like a conductor, waving my wand and hoping the orchestra wouldn’t go rogue.

The Magic Moment

After what felt like eons, things started happening! Lanes were recognized, objects were spotted, warnings were given. It was like watching your child take their first steps, if your child was made of silicon and code.

The Breakthrough:

Seeing the system come to life and successfully perform ADAS tasks was incredibly rewarding. It validated all the hard work and late nights spent troubleshooting and debugging. My system was finally doing something useful instead of just making me pull my hair out.

Future Plans

Because clearly, I haven't tortured myself enough, I'm planning to make this thing drive automatically in the next phase. This involves developing autonomous driving capabilities and further refining the ADAS features.

Next Steps:

• Autonomous Driving: Implementing algorithms for autonomous navigation, decision-making, and control. This will require integrating additional sensors and refining the AI models. It’s like teaching a toddler to ride a bike, but with more code.
• Improving Performance: Enhancing the system's performance and scalability, potentially upgrading hardware and optimizing software components. My computer is already whispering sweet nothings about retirement.
• Testing in Real-World Scenarios: Conducting more extensive testing in real-world scenarios to ensure robustness and reliability. I’ll probably need more virtual fire hydrants for the next round of testing.

A Grateful Shoutout

I've got to give a massive thank you to Xilinx and AMD for the hardware giveaway. This opportunity and their support have been incredible. Also, a big shout-out to the Hackster.io community. There are tons of Kria K26 tutorials there that have been lifesavers. Seriously, check them out if you're diving into this crazy world.

Wrapping Up

So, there you have it. My journey from a clueless beginner to a slightly-less-clueless ADAS creator. It's been frustrating, confusing, and sometimes I questioned my life choices. But you know what? It's also been incredibly rewarding. I've learned more in this project than I ever did in school (don't tell my teachers).

Final Thoughts

To all you beginners out there thinking about diving into ADAS or any tech project: Do it! Yes, it's hard. Yes, you'll want to throw your computer out the window sometimes. But the feeling when something finally works? Priceless.

Now, if you'll excuse me, I need to go figure out why my virtual car in CARLA keeps trying to befriend fire hydrants. Wish me luck!

Here are the link of my test videos......Please watch it with

Some useful

https://www.spinny.com/blog/index.php/adas-full-form/https://www.researchgate.net/figure/Flowchart-of-the-lane-detection-algorithm_fig1_331958729https://neptune.ai/blog/object-detection-algorithms-and-librarieshttps://www.mdpi.com/2078-2489/13/6/279