This Is a Slam Dunk

Force-feedback technology has been around for quite some time, but recent advances in the field have made it possible for interactive devices to provide users with more realistic and immersive experiences than ever before. These devices use various sensors and motors to simulate sensations such as touch or acceleration, allowing users to feel as if they are interacting with real objects and environments in a virtual space.

Interactive experiences most commonly implement force-feedback in limited ways, perhaps using vibrations or wind to create their effects. This is due to the fact that larger, entire-body effects are much more challenging and costly to produce. The strong forces needed to move a user’s entire body mass require substantial hardware and infrastructure, like motion platforms or pulley systems.

These more advanced setups are completely impractical for home use, and when they are employed at theme parks or virtual reality (VR) arcades, they can be very restrictive for the user, which breaks the illusion. The creative engineers in Pedro Lopes’ lab at The University of Chicago have been working on a way to miniaturize whole-body effects in a clever way — rather than physically moving the entire weight of one’s body, they have created a system that instead simulates the feeling of whole-body motion.

Being from Chicago, the team naturally wants to be like Mike, so they developed an untethered haptic backpack that gives the wearer the sensation of jumping higher than they actually are. In real life their eight inch verticals might not be impressing anyone, but in VR they are winning the Slam Dunk Contest at All-Star Weekend.

Called JumpMod, the device straps on to the wearer like any other backpack. On the back, there is a linear rail with a cube attached to it. This cube needs to be weighted down, so the team decided to place the batteries here so that they could serve that purpose in addition to providing power. That power is used to drive a motor that rapidly moves the cube up or down, along the wearer’s back. An encoder and belt drive mechanism allow for quick, precise movements. A Seeeduino XIAO nRF52840 BLE Sense microcontroller development board provides processing power, wireless connectivity, and control of the motor.

The system detects when a user jumps using the HTC VIVE Lighthouse V2 tracking system, however, it was noted that this function could probably also be achieved on-device using the XIAO’s inertial measurement unit. A custom algorithm then determines when the user is in four different phases of their jump: launch, ascent, descent, and landing.

With knowledge of the user’s jump phase, JumpMod can appropriately time movements of the weight, up or down, to simulate the feeling of either jumping higher, being pulled lower, landing harder or softer, or being pulled higher.

A series of user studies were conducted to assess how well the system performed under real-world conditions. In the first study, a dozen participants were asked to jump as they saw fit, and under some more structured conditions, to determine how accurate the jump detection algorithm was. Overall, the algorithm proved to be 94% accurate in detecting all four jump phases.

In another study, participants' jumps were augmented in various ways by JumpMod to determine how their perceptions of the jumps were altered. Each of the types of sensations that JumpMod can elicit were tested, and users of the system were then asked what they experienced during the jump. Accuracy in producing each of the desired effects was quite high, with the overall accuracy rate observed to be 81%.

In light of these positive results, the team ran one more study in which participants played a VR game that was augmented by JumpMod. The game featured elements like power-ups that give increased jumping abilities, and wind that blows the player down to the ground. The difference between user’s reported sensations when using the backpack, or only the VR headset, was statistically significant, implying that JumpMod noticeably modulates a user’s perceived jumping experience.

Moving forward, the team plans to explore different types of actuators or onboard sensing systems to create additional sensations with the device.