Watch Your Step

Cameras provide robots with a very rich source of information about their surroundings, assisting them in developing accurate motion plans to accomplish their goals. However, cameras are also subject to being blinded by occlusions, leaving the systems that rely on them with an incomplete picture of their environment. This problem is often especially pronounced at the periphery of the camera’s view, where the information that is provided is already limited at best.

Since cameras typically focus most on the area in front of a robot, their feet, and especially the feet of a quadruped, may not get good coverage. That creates many challenges, especially on difficult terrain, as these robots are prone to slip or otherwise lose their footing. Contact sensors and other sensing options have been used to collect additional information from the feet of robots to assist them in these scenarios, but they simply do not provide the same level of detail that can be captured with an image sensor.

A team led by researchers at the Southern University of Science and Technology in Shenzhen came up with a plan to overcome these challenges. Their solution involves placing downward-facing cameras on the feet of robots to get a clear view of the terrain and how the feet are interacting with it. This is not quite as simple as it may sound at first — if you were to just slap cameras on the feet, they would not remain operational for long, and they would collect useful images for even less time.

The researchers started with an off-the-shelf Unitree Go 1 Robot Dog. They then developed soles made of clear acrylic — with just a bit of flexibility to them — that can be attached to the robot dog’s existing feet. These hollow soles are designed to keep their contents safe from dirt, water, and other contaminants. Each sole is then equipped with a 120 FPS camera that is mounted a little over 2 inches from the bottom. A set of LEDs provide illumination of the surface of the ground below.

There is also a marker visible to each camera that is displaced as the foot moves and contacts surfaces. These displacements can be detected to estimate the pose of the foot. As designed, each sole costs about $50, but most of the cost is tied up in the camera itself, so it should be possible to bring that down further by choosing a less expensive option.

In a series of experiments, it was demonstrated that the system can accurately estimate 6D contact forces on terrains, and also the inclination of the contact surface. These capabilities enable the robot to sense the flow of sand or soil around each foot. And that information could help it to deal much better with a variety of terrains, and prevent issues like slipping.

Moving forward, the team plans to enhance the feet by increasing their traction. With some refinement, this novel concept could help make our robots much more robust under real-world conditions.