Method for Extracting Multiple Circle Primitives Extraction of Aircraft Surface Based on 3D Point Cloud Deep Learning

Honghua Chen, Wei Zeyong, Qian Xie, Mingqiang Wei, Jun WangJournal of Mechanical Engineering 2022 · DOI

Abstract: In automatic aircraft component hole-making systems, the fast and accurate detection of surface holes is crucial for assembly quality. However, automated, precise, and rapid extraction of hole features from large-scale 3D point cloud data remains a challenge. This paper proposes a method for extracting multiple circular hole primitives from aircraft surfaces based on 3D point cloud deep learning. The network predicts initial boundary points and learns hole normals. A learning-based Weighted Least Squares (WLS) method is designed to fit hole parameters. The network is trained using a multi-task joint loss encompassing boundary detection, parameter estimation, and normal learning errors. Tests on synthetic and real-world data demonstrate superior performance in boundary recognition and parameter calculation accuracy compared to state-of-the-art methods.

Introduction

In modern aircraft manufacturing, thousands of holes must be drilled and filled with rivets to assemble the fuselage and wings. To ensure structural integrity and stealth performance, the parameters of these holes—position, diameter, and orientation (normal)—must be incredibly precise.

While 3D laser scanners can capture the surface of an aircraft as a "point cloud," extracting hole features from this data is notoriously difficult. Real-world scans are often "messy": they contain sensor noise, reflections from metallic surfaces, and missing data due to occlusions. Traditional methods often struggle to distinguish the actual edge of a hole from other surface irregularities, such as points from the internal wall or intersections of different surfaces.

Challenges in 3D point cloud hole detection: distinguishing true boundary points (circles) from internal wall points (squares) and surface intersections (crosses).

Methods

The proposed end-to-end pipeline moves from raw points to precise geometric primitives in three main stages, powered by a multi-task deep learning architecture.

1. Transformer-Based Boundary Learning

The network processes local patches of the point cloud using Graph Convolutions (GC) to extract geometric features. To improve accuracy, it employs a Transformer module (utilizing Multi-Head Attention, or MHA) to fuse local and global neighborhood information. This context-aware approach allows the model to distinguish true hole boundaries from nearby internal wall points or surface noise.

2. Normal Estimation

Once boundary points are identified, the network regresses the 3D normal vector of the hole, defining its orientation in space. This is critical for ensuring the drill or rivet is aligned correctly with the surface.

3. Learning-Based Weighted Least Squares (WLS)

Instead of using a manually designed formula to weight points for circle fitting, the neural network learns to assign weights ($W$) to each point. Points identified as clean boundary edges receive high weights, while noisy outliers or wall points receive low weights. These weights are fed into a differentiable WLS solver to calculate the final center and radius.

The proposed multi-task deep learning architecture, featuring local/global feature fusion and specialized heads for boundary detection, normal estimation, and weight learning.

Datasets and Evaluation

  • Synthetic Data: The researchers used Blender to virtually scan 28 CAD models of aircraft surfaces, generating 10,080 training pairs with simulated Gaussian noise (0.1% to 1.0%) and varying point densities.
  • Real-World Validation: The model was tested on physical aircraft components scanned with industrial sensors, including the Wenglor MLAS202 laser scanner and the PhoXi 3D depth camera.

Key Results

  • Superior Accuracy: The method achieved 98.73% accuracy and a 74.40% IoU for boundary detection, significantly outperforming existing methods like EC-Net and PIE-NET.
  • Sub-Millimeter Precision: In tests on standard hole samples, the method achieved an average radius error of just 0.016 mm.
  • Industrial Performance: When compared against PolyWorks, a leading commercial metrology software, the deep learning approach produced more accurate results with less manual intervention. The system can process a scan of 48 holes (~163,000 points) in approximately 15.8 seconds.

Conclusion

This research demonstrates that integrating deep learning with geometric fitting can solve complex feature extraction problems in aircraft assembly. By learning to weight points and capture global context, the model achieves high robustness against the noise and artifacts typical of industrial 3D scans. Future work will extend this approach to other features like assembly seams and rivet heads, aiming for a comprehensive automated inspection system.

Summary updated

Abstract

摘要: 在飞机部件自动制孔系统中,快速、精确检测飞机表面圆孔对飞机装配质量具有重要作用,但从大规模三维测量点云数据中自动化、精确、快速检测所有圆孔特征依旧是一个难点。鉴于此,提出一种基于三维点云深度学习的飞机表面多圆孔基元提取方法。使用三维点云深度学习网络预测三维测量点云中初始圆孔边界点,并基于初始圆孔边界点,学习圆孔法向。同时,设计基于学习的加权最小二乘(Weighted least square,WLS)方法拟合圆孔参数,并将圆孔边界点检测误差、圆孔参数估计误差、圆孔法向学习误差作为多任务联合损失进行网络训练。通过在不同噪声、不同分辨率的模拟点云数据和实测点云数据上进行测试,并与现有先进边界提取、圆孔拟合方法进行对比。试验结果表明,所提出的方法在边界点识别准确度、圆孔参数计算准确度等方面获得了优越的综合性能。
BibTeX
@article{chen2022method,
  author = {Honghua Chen and Wei Zeyong and Qian Xie and Mingqiang Wei and Jun Wang},
  title = {{Method for Extracting Multiple Circle Primitives Extraction of Aircraft Surface Based on 3D Point Cloud Deep Learning}},
  journal = {Journal of Mechanical Engineering},
  year = {2022},
  doi = {10.3901/jme.2022.14.190},
}
Figure