A Bright Idea in Movement Tracking

Tracking the position of the body has many applications, ranging from interacting with virtual and augmented reality environments to health monitoring, athletic training, and studying animal behavior. This technology plays a pivotal role in enhancing human-computer interaction, enabling more immersive experiences, and advancing various fields of study. It allows us to bridge the gap between the physical and digital worlds, offering a wide array of benefits in terms of convenience, precision, and insights.

The implementation of body tracking technologies often involves a combination of hardware and software components. Common methods include using cameras, depth sensors, or wearable devices to capture data about the body's movements and positions. Computer vision techniques, including image processing and deep learning, are then employed to analyze this data and infer key body points, such as joint positions. These inferred positions are used to drive various applications, whether it is controlling a video game character, tracking a patient's rehabilitation progress, or helping a factory worker maintain ergonomic posture.

One of the key challenges in developing robust body tracking systems is the need for huge amounts of training data. Machine learning models, especially deep learning models, require large sets of manually annotated data to generalize well across diverse body types, movements, and environments. However, collecting and annotating such data is not only costly but also time-consuming. Consequently, models often need to be tailored to specific use cases, as they may not perform well across the board.

Frustrated by the current state-of-the-art systems for tracking animal behavior, scientists at the Salk Institute for Biological Studies developed a novel approach, called GlowTrack, that allows them to effortlessly collect orders of magnitude more labeled body position data than current methods. Using their approach, data is collected after applying a fluorescent dye to the areas to be tracked. This dye serves as an identifiable label that can be paired with normal, RGB images to train a machine learning model to identify body key points.

The fluorescent dye can either be applied to one region of the body at a time, or it can be randomly speckled over a larger area. In the latter case, the random speckles create unique patterns that can be recognized and associated with areas of the body. Data is then captured by rapidly switching between ultraviolet and visible light illumination of a scene while capturing video. Each successive frame will be illuminated by a different light source, allowing the marker positions (that are only visible under ultraviolet light) to be assigned to the following visible light frame. That provides the label for the visible light frame, which can be used to train a body position tracking model.

The non-invasive labeling system used by GlowTrack has the advantages of being both precise and high-definition. It can label tiny structures, like a single digit on the paw of a mouse. The same method can also simultaneously track hundreds of locations on a human hand, providing an unparalleled level of detail about movements of the body.

By using GlowTrack, the team hopes to gain a better understanding of how the brain controls behaviors. They also see future applications in studying movement-related disorders, like amyotrophic lateral sclerosis and Parkinson’s disease. It is also expected that GlowTrack will impact areas outside of the team’s area of expertise, with one of the researchers noting that their “approach can benefit a host of fields that need more sensitive, reliable, and comprehensive tools to capture and quantify movement. I am eager to see how other scientists and non-scientists adopt these methods, and what unique, unforeseen applications might arise.”