TL;DR
This paper introduces object keyline detection, a novel task that extends object detectors like YOLO to predict a fixed, semantically ordered set of infinite lines (keylines) for each object instance. By treating lines as first-class geometric features alongside keypoints, the authors significantly improve the robustness of vision-based aircraft landing systems, ensuring reliable perception even when runway corners are truncated or occluded.
What Problem Does This Paper Address?
In safety-critical applications like autonomous aircraft landing, precise pose estimation is vital. Traditionally, this relies on detecting keypoints (like runway corners). However, keypoints are fragile: if a corner is cut off by the camera's field of view or hidden by a cloud, the geometric constraint is lost.
While general line-segment detectors exist, they typically produce an unordered "soup" of lines across the entire image. They don't know which line belongs to which object, nor do they assign semantic meaning (e.g., "this is the left edge of runway 2"). This paper bridges the gap between object detection and line extraction, providing a "line-level analog" to semantic keypoints that remains observable even when endpoints are missing.
Methods
The authors extend the YOLO-NAS architecture to support keyline prediction through several key innovations:
- Continuous Hesse-Normal Parameterization: Instead of predicting a single, unstable angle $\theta$, the model predicts a signed distance $\rho$ and a unit normal vector $[\cos \theta, \sin \theta]$. This representation is continuous and avoids the "wrap-around" discontinuity (where $0^\circ$ and $360^\circ$ are numerically far but geometrically identical) that plagues standard angle regression.
- Object Keyline Similarity (OKLS): Inspired by the standard Object Keypoint Similarity (OKS) used in pose estimation, the authors propose OKLS. It measures the agreement between a predicted infinite line and ground-truth support endpoints. This serves as a unified evaluation metric, a target assignment signal during training, and a regression loss.
- Architectural Integration: The authors compared "Split" vs. "Fused" prediction heads. In the Fused variant, keypoints and keylines share the same geometric branch, which proved to be highly efficient, adding almost no latency compared to a keypoint-only model.
Datasets
The research primarily uses a curated proprietary dataset of real aircraft approach images.
- Training set: 6,730 images from 230 trajectories across 95 airports.
- Validation/Test sets: ~700 images each, from airports strictly disjoint from the training set to prevent geographic data leakage.
- The dataset includes annotations for 4 runway corners (keypoints) and 5 semantic lines (left, top, right, bottom edges, and the centerline).
The authors also benchmarked against general-purpose line detectors like M-LSD and LINEA using an "oracle matching" protocol to provide a fair comparison on their semantic task.
Key Insights and Ideas
- Lines are more robust than points: The "aha moment" is that while a keypoint provides zero information if occluded, a partially visible edge still provides a strong geometric constraint on orientation and perspective.
- Semantic ordering matters: By assigning a fixed index to each line (e.g., index 0 is always the left edge), the model can learn specific visual priors for each feature, making it much more accurate than general line detectors that treat all lines as equal.
- The "Lever-Arm" Effect: OKLS is superior to standard coordinate losses because it naturally accounts for the fact that small angular errors matter more for long, distant lines. It penalizes errors based on how much they displace the line relative to its physical support.
Key Results
- Superior Parameterization: The continuous Hesse-normal form achieved the highest accuracy (55.7 $AP_{KL}$) compared to point-based or polar-angle alternatives.
- Improved Constraint Availability: For pose estimation (PnPL), having at least three features is a requirement. Using keypoints alone, this was possible in only 71.3% of test images. Adding keylines boosted this "availability" to 99.3%.
- Efficiency: The Fused head architecture added keyline capabilities with a negligible latency increase of only 0.1 ms (on an NVIDIA Tesla T4), making it practical for real-time flight systems.
- Outperforming Baselines: Even when giving general line detectors (like LINEA) the advantage of "oracle matching," the proposed instance-centric approach achieved significantly higher Average Precision.
Limitations and Future Work
While highly effective for runways, the authors note several areas for further exploration:
- Scope: The method assumes objects have a fixed, semantically ordered set of linear features. It may struggle with deformable objects or categories where the number of edges varies.
- Downstream Evaluation: The study evaluates performance in the image domain ($AP$). The next logical step is integrating these keylines into a full 6-DoF pose estimation pipeline to measure the actual reduction in aircraft positioning error.
- Geometric Constraints: Currently, the model predicts lines independently. Future work could enforce global constraints, such as ensuring parallel runway edges share a common vanishing point.
Summary updated