Giving robots 'superhuman' vision through radio waves

PanoRadar marks a breakthrough in the ever-advancing world of robotics, addressing the challenge of providing effective perception systems for robots that can function in severe weather and unfavorable conditions.

Unlike traditional sensors, PanoRadar uses radio waves to penetrate smoke, fog, and even certain materials, offering a robust alternative to light-based systems.

This innovative approach paves the way for more resilient robotic applications, from autonomous vehicles to rescue missions in hazardous environments.

Limitations of traditional sensors

Current sensor systems, such as cameras or LiDAR, rely on light, which limits their effectiveness in conditions like heavy smoke or fog. Alternatively, they may rely on traditional radar which can ‘see’ through walls and other obstacles, but which produce images lacking in detail.

Nature, however, shows us that perception need not rely solely on light. Bats use sound wave echoes for navigation, while sharks detect electrical fields from their prey’s movements.

These examples highlight alternative perception methods that can inform technological innovations, like radio wave-based sensing.

Radio waves, with their longer wavelengths, can penetrate challenging conditions and even certain materials, surpassing the limitations of human vision and conventional sensors.

PanoRadar: A new era of robot vision

Enter researchers from the University of Pennsylvania School of Engineering and Applied Science (Penn Engineering). They have initiated a new era in robot vision with their creation, the PanoRadar.

This novel invention enables robots to perceive their surroundings in three dimensions with incredible detail, by transforming simple radio waves into detailed, 3-D images of the environment.

“Our initial question was whether we could combine the best of both sensing modalities – the robustness of radio signals, which is resilient to fog and other challenging conditions, and the high resolution of visual sensors,” explained Mingmin Zhao, assistant professor of computer and information science.

The birth of PanoRadar is the novel answer to this complex challenge. It has opened new possibilities for robotic perception in environments where traditional systems fail.

How does PanoRadar work?

Think about how a lighthouse works. It sweeps its beam in a circle, scanning the entire horizon, revealing the presence of ships and coastal features. That’s how the PanoRadar operates.

It consists of a rotating vertical array of antennae that scan the surroundings. These antennae send out radio waves and listen for their reflections from the environment.

But PanoRadar is not just a simple scanner. It intelligently combines measurements from all rotation angles, enhancing its imaging resolution.

“The key innovation is in how we process these radio wave measurements. Our signal processing and machine learning algorithms are able to extract rich 3D information from the environment,” explained Zhao.

This novel approach allows PanoRadar to achieve imaging resolution comparable to that of LiDAR, but at a fraction of the cost.

Comprehending and interpreting complex data

Implementing high-resolution imaging while the robot is moving presented a significant challenge. The team needed to combine measurements from various positions with sub-millimeter accuracy.

“Even small motion errors can significantly impact the imaging quality,” said Haowen Lai, lead author of the paper.

In addition, they had to train their system rigorously to comprehend and interpret the complex data it receives, to ensure that it could accurately identify and understand various objects and environmental features in real time.

“Indoor environments have consistent patterns and geometries. We leveraged these patterns to help our AI system interpret the radar signals, similar to how humans learn to make sense of what they see,” explained Gaoxiang Luo.

Thanks to machine learning algorithms, the model was able to improve its understanding against reality using LiDAR data.

Radio waves and robot vision

PanoRadar’s capabilities extend beyond what traditional sensors can perceive.

“Our field tests across different buildings showed how radio sensing can excel where traditional sensors struggle,” said Liu. “The system maintains precise tracking through smoke and can even map spaces with glass walls.”

This advantage comes from the ability of radio waves to pass through airborne particles with ease, allowing the system to detect elements that LiDAR often misses, such as glass surfaces.

Additionally, PanoRadar’s high resolution enables it to identify people accurately, an essential capability for applications like autonomous vehicles and rescue operations in challenging environments.

These attributes make PanoRadar a game-changer for robots operating in difficult conditions, where reliability and precision are paramount.

The research will be presented at the 2024 International Conference on Mobile Computing and Networking (MobiCom).

—–

Like what you read? Subscribe to our newsletter for engaging articles, exclusive content, and the latest updates. 

Check us out on EarthSnap, a free app brought to you by Eric Ralls and Earth.com.

—–