Multi-modal surface registration for markerless initial patient setup in radiation therapy using microsoft's Kinect sensor

Sebastian Bauer, Jakob Wasza, Sven Haase, Natalia Marosi, Joachim HorneggerWorkshops @ ICCV 2011 · DOI

Introduction

In radiation therapy (RT), precision is paramount. Before treatment begins, a patient’s body must be accurately aligned with the "isocenter" defined in their planning CT scan. Traditionally, this "coarse" setup is a manual, time-consuming process where therapists use wall-mounted lasers and ink markers on the patient's skin to guide positioning. While high-end systems exist for fine-scale verification, they often require a manual starting point to avoid local minima during alignment. This paper presents a novel approach to automate this initial setup using a low-cost Microsoft Kinect sensor and advanced 3D geometry algorithms, eliminating the need for physical markers and reducing manual labor.

The Markerless Setup System

The proposed system utilizes a range imaging (RI) device—specifically a Microsoft Kinect—mounted within the treatment room. The goal is to register the real-time 3D surface captured by the Kinect with the high-resolution surface model extracted from the patient's planning CT.

Physical setup of the markerless positioning system showing the LINAC, RI device (Kinect), and the patient coordinate system.

The system estimates a 4-degree-of-freedom transformation: three translations ($t_{ML}, t_{SI}, t_{AP}$) and one rotation ($\phi$) around the vertical axis of the treatment table. This transformation provides the necessary adjustments to align the patient with the treatment plan.

Methodology and Workflow

The core challenge lies in multi-modal registration: matching noisy, consumer-grade sensor data to clean CT models. The researchers developed a three-stage workflow divided into pre-fractional and intra-fractional phases.

The multi-modal registration workflow, detailing the steps from CT/RI data acquisition to transformation estimation.
  1. Local Encoding: The system uses mathematical "descriptors" to create fingerprints of small surface patches. The authors adapted the MeshHOG descriptor to use "signed distance" to a fitting plane instead of curvature, significantly increasing robustness against the "jitter" and staircase artifacts inherent in Kinect data.
  2. Correspondence Matching: The system identifies matching fingerprints between the Kinect scan and the CT model. A geometric consistency check is applied to filter out false positives, ensuring that anatomical features (like the chest or stomach) are correctly paired.
  3. Transformation Estimation: Finally, the optimal transformation is calculated to align the treatment table.

Results and Clinical Relevance

The system was evaluated using anthropomorphic phantoms and a volunteer study to test resilience to respiratory motion. Key findings include:

  • Success Rate: The modified MeshHOG descriptor achieved a 97.5% success rate in aligning phantoms from gross misalignments (up to 200mm offset and 45° rotation).
  • Accuracy: On average, the system reached a translational error of 13.4 mm and an angular error of 1.5°. This performance is comparable to manual setup using lasers and markers.
  • Robustness: Unlike global alignment algorithms like Iterative Closest Point (ICP), the point-based descriptor approach effectively handles partial overlaps—where the Kinect sees the entire room but the CT only covers the treatment area.

Conclusion

By leveraging low-cost sensors and robust 3D descriptors, this research demonstrates a viable path toward fully automated, markerless initial patient setup in radiation therapy. The system provides a reliable starting point for fine-tuned clinical verification, potentially saving time and improving the patient experience by removing the need for permanent skin markers. Future work will explore multi-camera setups to improve coverage and expand the system to 6 degrees of freedom.

Summary updated

BibTeX
@inproceedings{bauer2011multi,
  author = {Sebastian Bauer and Jakob Wasza and Sven Haase and Natalia Marosi and Joachim Hornegger},
  title = {{Multi-modal surface registration for markerless initial patient setup in radiation therapy using microsoft's Kinect sensor}},
  booktitle = {Workshops @ ICCV},
  year = {2011},
  doi = {10.1109/iccvw.2011.6130383},
}
Figure