Toward Accurate and Accessible Markerless Neuronavigation

Ziye Xie, Oded Schlesinger, Raj Kundu, Jessica Y. Choi, Pablo Iturralde, Dennis A. Turner, Stefan M. Goetz, Guillermo Sapiro, Angel V. Peterchev, J. Matias Di Martino

Neuronavigation is essentially "GPS for the brain." Whether a researcher is targeting a specific cortical region for Transcranial Magnetic Stimulation (TMS) or a surgeon is navigating complex tissue, they need to know exactly where their instruments are relative to the patient's anatomy. Historically, this has required "markers"—clunky retroreflective spheres or electromagnetic coils taped to a patient's forehead.

A new paper titled "Toward Accurate and Accessible Markerless Neuronavigation" suggests we can finally throw those markers away in favor of consumer-grade cameras and sophisticated facial modeling.

TL;DR

Researchers have developed a markerless tracking system using low-cost RGB-D cameras (like the Azure Kinect) that achieves a median accuracy of 2.32 mm. By combining stereo vision, depth sensing, and personalized statistical head models, this approach matches the clinical requirements for brain stimulation while significantly reducing cost and patient discomfort.

What Problem Does This Paper Address?

Traditional neuronavigation is a "high-friction" process. It relies on expensive, specialized hardware (often costing tens of thousands of dollars) and physical markers attached to the subject. These markers have several "fail points":

  1. Skin Shift: The skin moves relative to the skull, introducing errors.
  2. Occlusion: If a doctor's hand blocks the line of sight between the camera and a single marker, the system loses track.
  3. Registration Errors: Manually "pointing" to anatomical landmarks (like the bridge of the nose) to sync the system is time-consuming and prone to human error.

The authors sought to create a system that is markerless (using the face itself as the tracker), low-cost (using $400 consumer cameras), and robust enough for clinical use.

Methods

The team evaluated three distinct computer vision strategies, both with and without a "secret sauce" called the Personalized Head Model (PHM).

  1. Monocular RGB: Uses a single camera to detect 468 facial landmarks (via MediaPipe) and estimates pose using Perspective-n-Point (PnP) algorithms.
  2. Stereo RGB: Uses two cameras to triangulate those landmarks in 3D space.
  3. Depth-Based: Uses time-of-flight sensors to create a dense point cloud of the face, aligning it to a template using Iterative Closest Point (ICP) and Chamfer distance metrics.

The PHM is the most novel component. Instead of treating the face as a random collection of points, the researchers used a Universal Head Model (UHM)—a statistical prior built from thousands of head scans. They "fit" this model to each specific subject during a quick 1-minute scan, creating a regularized, anatomically accurate 3D representation that helps the tracking algorithms ignore noise and handle extreme head turns.

Datasets

The study involved a cohort of 50 human subjects (27 female, 23 male, ages 20–76) with diverse racial and ethnic backgrounds. This is a significant step up from prior "proof of concept" studies that often relied on a single subject or a mannequin.

The ground truth was established using a "gold standard" NDI Polaris Vicra system—the same marker-based infrared system used in high-end surgical suites.

Key Insights and Ideas

  • Complementary Strengths: The researchers found that different modalities excel at different things. Stereo RGB provided the best translational accuracy (moving left/right/up/down), while Depth sensing was superior for rotational accuracy (tilting the head).
  • The Power of Priors: Using the PHM (statistical head model) significantly improved results for stereo and depth methods. It acts as a "sanity check" for the camera data, ensuring the system knows it's looking at a human head with specific geometric constraints.
  • Robustness Through Redundancy: Unlike marker-based systems that fail if a few dots are hidden, markerless systems look at the entire face. Even if a subject turns their head 60 degrees (occluding half their face), the system can still maintain a lock using the remaining visible features.
  • Accessibility: By proving that a $400 Azure Kinect can rival a $20,000+ NDI system, the authors open the door for high-precision brain research in resource-limited settings or mobile clinics.

Key Results

The best-performing algorithms achieved:

  • Median Translation Discrepancy: 2.32 mm (Stereo RGB + PHM).
  • Median Rotation Discrepancy: 2.01°.
  • Failure Rate: The markerless systems actually had a lower failure rate than the NDI system during extreme head movements. While the NDI system lost track in 0.67% of frames due to marker occlusion, the best markerless methods failed in only ~0.015% of frames.

For context, typical TMS clinical targets are 1–2 cm² in size, meaning a ~2.3 mm error is well within the acceptable "operational tolerance" for these procedures.

Limitations and Future Work

  • The "Ground Truth" Ceiling: The study compared markerless tracking to a marker-based system (NDI). However, the NDI system itself has errors (like skin shift). Future work could use medical imaging (MRI/CT) to provide an absolute anatomical ground truth.
  • Real-Time Optimization: While the accuracy is there, the paper doesn't deeply explore the computational "cost" or latency of running these dense models in real-time during a procedure.
  • Temporal Smoothing: The current methods work on a per-frame basis. The authors suggest that adding temporal constraints (e.g., Kalman filters) to account for the fact that heads don't teleport between frames would likely improve accuracy even further.

Paper Citation: Xie, Z., et al. "Toward Accurate and Accessible Markerless Neuronavigation." arXiv:2602.07052v1 [cs.CV], 2026.

Summary updated

Abstract

Neuronavigation is widely used in biomedical research and interventions to guide the precise placement of instruments around the head to support procedures such as transcranial magnetic stimulation. Traditional systems, however, rely on subject-mounted markers that require manual registration, may shift during procedures, and can cause discomfort. We introduce and evaluate markerless approaches that replace expensive hardware and physical markers with low-cost visible and infrared light cameras incorporating stereo and depth sensing combined with algorithmic modeling of the facial geometry. Validation with $50$ human subjects yielded a median tracking discrepancy of only $2.32$ mm and $2.01°$ for the best markerless algorithms compared to a conventional marker-based system, which indicates sufficient accuracy for transcranial magnetic stimulation and a substantial improvement over prior markerless results. The results suggest that integration of the data from the various camera sensors can improve the overall accuracy further. The proposed markerless neuronavigation methods can reduce setup cost and complexity, improve patient comfort, and expand access to neuronavigation in clinical and research settings.
BibTeX
@misc{xie2026toward,
  author = {Ziye Xie and Oded Schlesinger and Raj Kundu and Jessica Y. Choi and Pablo Iturralde and Dennis A. Turner and Stefan M. Goetz and Guillermo Sapiro and Angel V. Peterchev and J. Matias Di Martino},
  title = {{Toward Accurate and Accessible Markerless Neuronavigation}},
  year = {2026},
  eprint = {2602.07052},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2602.07052},
}
Figure