Capstone 2020 - Sydney University w/ Robotics Masters

Background - What is Capstone?

Each Semester, Sydney University asks for industry partners to come on board for final year projects in all faculties. These projects discussed today are conducted in collaboration with Sydney University Engineering and IT Faculty for students across all disciplines.

I have been (and continue to be) an industry partner for Sydney University for their engineering and IT final year projects. This is something I have done ever since leaving university more than 3 years ago.

How it works

The University puts out expressions of interest to work with their final year undergraduate and master’s students. The industry partner (or client) provides a project(s) brief to the University for review and are then assigned teams of 3 to 6 (normally 4) students.

The Projects

This year I am working with 7 teams on various projects that include:

• Sign Detection using Computer Vision
• Sign Detection with TensorFlow
• Object Detection for Simulated Drones
• Optimal Path for Drone Delivery

The teams are given 11 weeks to complete the projects. The goal of all my projects are for students to leave with an excellent understanding on simulation in Unity/Gazebo, Computer Vision / OpenCV and AI technologies. Students will also gain valuable project management, industry and professional experience throughout the process.

They started last week.

Brief - Sign Detection using Computer Vision

Students will be looking at existing techniques and creating new software solutions to detect a number of pre-defined traffic signs using only computer vision (no Neural Networks). Teams will look at different technologies and come up with a solution that is reliable and efficient. Computer Vision is a key part for collecting training data for Neural Networks as it quickly and efficiently automates the process.

Teams will be using the Donkey Car Platform to test their algorithms and solutions on Jetson Nanos. The Donkey Car Platform includes both a physical hardware stack and a simulator built using Unity. Students are expected to initially test solutions in Unity, adding new parts as needed and later on the physical hardware.

At the end of the project, the teams will compete in a challenge course and prove that their algorithms are effective.

So far teams have improved the simulator and are working on improving the sign detection algorithms given to them.

Brief - Sign Detection with TensorFlow

Students will be looking at existing techniques and creating new software solutions to detect a number of pre-defined traffic signs using Neural Networks (TensorFlow). Teams will look at different technologies and come up with a solution that is reliable and efficient. This will involve data collection and cleaning. TensorFlow 2 is recognised as one of the preferred libraries for completing AI tasks.

Like the previous project - they are also using the Donkey Car Simulator for testing their algorithms. So far teams have improved the simulator and are working on and building up their TensorFlow skills.

Brief - Object Detection for Simulated Drones

Students will be looking at existing techniques and creating new software solutions to detect a number of pre-defined types of objects using AI (Tensorflow) and Computer Vision (CV). This project, unlike the others, is drone focused.

Teams will engineer a solution that is reliable and efficient. This will involve data collection and cleaning. Teams will be using the Gazebo simulator for PX4 & ArduPilot to test their algorithms and solutions.

So far (it has only been a few days) teams have set up the Gazebo simulator for ArduPilot and are working on all the other problems given to them.

Brief - Optimal Path for Drone Delivery

This project assumes we are flying drones at high and low altitudes. At high altitudes, the drone is expected to be constantly calculating battery level, nearest charging station and distance to destination. The final goal of the project is to allow a fleet of drones to deliver any number of parcels to destinations in the most optimal way autonomously.

The team will model all of these conditions into the Gazebo Simulator. This is one of the harder projects as it requires a lot of thought for all the conditions in play and research of the best solution.

Final Remarks

I plan on sharing the final results of the student's progress throughout the next few months. If you are interested in any of these projects or have something you want to contribute that would help the students, please message me or let me know in the comments.