Human curiosity about the cosmos has driven us to explore the unknown, but planetary landings remain incredibly challenging. To protect human life, robotic explorers have taken the lead—from NASA’s Sojourner in 1997 to the recent Perseverance and Ingenuity rovers on Mars. As we plan for deeper space missions, the need for smaller, cost-effective robots that can operate autonomously under communication delays has become critical. However, their reduced size limits onboard power and compute resources, posing a significant hurdle for performing complex tasks.
To overcome this, NASA’s Jet Propulsion Laboratory (JPL) turned to DINOv2, an open-source vision foundation model released by Meta’s FAIR team in 2023. They developed a Visual Perception Engine that reuses DINOv2 features across multiple tasks, slashing GPU compute and memory needs. This framework, available on GitHub, allows robots to share a single feature extraction backbone instead of relying on separate models for each vision task—such as depth mapping, object detection, and segmentation.
Testing this on the Nebula Spot Robot, JPL restricted sensors to an IMU and RGB camera, processing data with a compact Nvidia Orin AGX computer. The Visual Perception Engine retains extracted features in GPU memory, enabling parallel task execution. This not only boosts image throughput but also frees up resources for onboard learning. Robots can now adapt to unknown terrain in real-time, using DINOv2 features and power usage data to avoid hazards like soft sand—a lesson learned from the 2005 Opportunity rover’s five-week sand trap ordeal.
The results are groundbreaking: a 67% reduction in parameters while maintaining performance. As Meta iterates on DINO, JPL continues experimenting to prepare for space applications. Beyond planetary exploration, this technology holds promise for Earth—such as aiding rescue missions in treacherous caves, like the 2018 Tham Luang operation. Caves offer natural radiation shielding, increasing the likelihood of finding extraterrestrial life, making these small but mighty robots key players in the search for answers beyond our world.
