Virtual Reality Takes Flight

The remarkable advancements in unmanned autonomous vehicles (UAVs) have revolutionized multiple industries, opening up a world of unprecedented opportunities. In particular, the potential applications of UAVs for tasks like package delivery and disaster response have garnered significant attention. These vehicles, equipped with advanced sensors, navigation systems, and artificial intelligence algorithms, offer unprecedented capabilities for navigating complex environments and performing tasks autonomously.

Despite the immense potential of UAVs for package delivery and disaster response, widespread adoption has been slow, with only a relatively small number of deployments having taken place. Several factors contribute to this limited uptake. Firstly, technological challenges, such as limited battery life, payload capacity, and range, have constrained the capabilities of UAVs for long-distance deliveries or large-scale disaster response operations. While advancements in battery technology and aerodynamics are gradually addressing these limitations, there is still room for improvement to make UAVs more efficient and reliable in demanding real-world scenarios.

Overcoming these technological challenges should be just a matter of time. But other issues, like regulatory hurdles and safety concerns have hindered the integration of UAVs into the national airspace system. Governments around the world are still grappling with issues such as airspace management, privacy regulations, and the risk of mid-air collisions, which has slowed down the approval process for commercial UAV operations. The path to solving these problems is unclear, and is compounded by matters of human supervision.

UAVs may be autonomous, but to be safely deployed at scale, human oversight is essential. Exactly how that can be practically achieved when many thousands, or millions, of UAVs are buzzing around the skies and along the streets has not been fully worked out. Recent research conducted by a team at Oregon State University and SIFT, LLC moves us closer to a solution, however. They have developed a system that allows a single human operator to effectively oversee a swarm of more than 100 UAVs. Moreover, it was shown that the operators are not subjected to any excessive levels of stress, despite the seemingly high workload.

The team’s approach involves the use of a virtual reality (VR) system that they call the Immersive Interactive Interface (I3). It was built with the Unity game engine and utilizes SteamVR and the Valve Index VR headset. This headset is connected to a laptop that manages the swarm control network and receives telemetry from all UAVs in the swarm. A pair of handheld controllers allow the operator to navigate, and interact with, the virtual world. Audio cues signal important incoming information or the presence of hazards.

In general, an I3 operator is issuing high-level commands to the swarm, making adjustments where needed and leaving most of the details to the flight plan initially provided to each UAV. This is an essential design feature, as a single operator cannot micromanage a hundred or more vehicles.

This setup was tested in a series of experiments that included more than 100 commercial, off-the-shelf aerial drones and autonomous wheeled vehicles. Throughout the course of these trials, it was found that a single operator could effectively manage this large fleet of mixed types of vehicles. Questionnaires and sensors that collected physiological measurements from the operators demonstrated that while short periods of stress did occur at certain points, in general, they were able to manage the swarm without being overloaded — even under challenging temperature and wind conditions.

The work done by this team has demonstrated that many long-awaited UAV applications may be just around the corner. The additional human oversight it can provide has the potential to crush regulatory and logistical hurdles that have been holding the technology back.