Abstract: Perception modules are integral in many modern autonomous systems, but their accuracy can be subject to the vagaries of the environment. In this paper, we propose a learning-based approach that can automatically characterize the error of a perception module from data and use this for safe control. The proposed approach constructs an inverse perception contract (IPC) which generates a set that contains the ground-truth value that is being estimated by the perception module, with high probability. We apply the proposed approach to study a vision pipeline deployed on a quadcopter. With the proposed approach, we successfully constructed an IPC for the vision pipeline. We then designed a control algorithm that utilizes the learned IPC, with the goal of landing the quadcopter safely on a landing pad. Experiments show that with the learned IPC, the control algorithm safely landed the quadcopter despite the error from the perception module, while the baseline algorithm without using the learned IPC failed to do so.
Introduction
Modern autonomous systems rely heavily on perception modules, such as computer vision, to interpret their surroundings. However, these modules are rarely perfect; they are frequently compromised by sensor noise, motion blur, and processing latencies. Most standard perception algorithms output a single "best guess"—a point estimate—without providing the downstream controller any indication of how much to trust that estimate.
In safety-critical scenarios, such as landing a drone on a small platform, even minor perception errors can lead to catastrophic failures. While previous research introduced "Perception Contracts" to describe error relative to ground truth, these models were impractical for real-time control because ground truth is precisely what the robot is trying to determine. This paper introduces the Inverse Perception Contract (IPC), which flips this paradigm by starting with noisy observations and predicting the range of possible realities.
The Inverse Perception Contract
The core contribution of this work is the IPC: a mapping that takes the current system state and a perceived value and outputs a set—specifically an ellipsoid—that contains the actual ground truth with high probability.
Learning the Mapping
The authors utilize a three-layer neural network to learn this mapping from empirical data. The network takes the quadcopter's state and the perceived position of the landing pad (transformed into the camera's coordinate frame for better generalization) as inputs. It outputs 12 parameters defining a 3D ellipsoid.
The training process employs a dual-objective loss function:
- Empirical Risk: A hinge loss that penalizes the model if the ground truth falls outside the predicted ellipsoid.
- Regularization: A penalty on the volume of the ellipsoid to prevent the network from outputting excessively large, uninformative sets.
Safe Control Strategy
To utilize the IPC, the researchers implemented a state-machine-based controller. In the Measuring state, the system receives a perception estimate and the corresponding IPC ellipsoid. If the ellipsoid is small enough to fit within the "safety box" of the landing pad, the system transitions to Landing. If the uncertainty (the size of the ellipsoid) is too high, the system moves to a New Waypoint to gather better visual data before attempting the landing again.
Experimental Evaluation
The approach was validated using a quadcopter equipped with a vision pipeline designed to detect ArUco markers on a landing pad. A dataset of 5,000 samples was collected in a lab environment using a high-accuracy Vicon motion capture system to provide ground truth. The primary source of error in this setup was the synchronization lag between camera images and Vicon measurements.
Key Results
- High Reliability: The learned IPC achieved an error rate of only 0.19% on the test set, meaning the ground truth was contained within the predicted ellipsoid over 99.8% of the time.
- Flight Success: In real-world testing, the IPC-based controller achieved a 100% safe landing rate across 10 runs. In contrast, a baseline controller that trusted point estimates failed in 9 out of 10 runs due to perception lag.
- Computational Efficiency: Inference for the IPC takes approximately 3 milliseconds on a Raspberry Pi 3, making it suitable for edge computing on small-scale robotic platforms.
Conclusion
The Inverse Perception Contract provides a mathematically grounded way for autonomous systems to "know what they don't know." By mapping estimates to sets rather than ground truth to error, the contract becomes a practical real-time tool. Future work may explore the joint optimization of perception contracts and controllers to further enhance robust autonomous behavior in changing environments.
Summary updated