Model-based learning for DoA estimation and array calibration via continuous sparse recovery

Baptiste Chatelier, José Miguel Mateos-Ramos, Vincent Corlay, Matthieu Crussière, Christian Häger, Henk Wymeersch, Nir Shlezinger, Luc Le MagoarouarXiv 2026 · arXiv

This paper tackles a fundamental challenge in wireless sensing: how to accurately estimate the Direction of Arrival (DoA) of signals when the hardware itself is imperfect. By blending classical signal processing with machine learning, the authors propose a method that can calibrate an antenna array and estimate signal directions simultaneously—even without knowing the ground truth positions of the signal sources.

TL;DR

The authors present a model-based machine learning framework that jointly performs antenna array calibration and off-grid DoA estimation. By combining greedy sparse recovery with continuous refinement via gradient descent, the method compensates for severe hardware impairments and outperforms classical baselines, even in the presence of coherent signal sources and without labeled training data.

What Problem Does This Paper Address?

In the emerging field of Integrated Sensing and Communication (ISAC), knowing exactly where a signal is coming from (DoA) is vital. However, real-world antenna arrays suffer from Hardware Impairments (HWIs): antennas aren't perfectly spaced, and their electronic gains and phases vary.

Classical methods struggle here:

  • Subspace methods (like MUSIC): These break down when signal sources are "coherent" (highly correlated), which is common in multipath environments.
  • Sparse recovery methods (like OMP): These are usually "on-grid," meaning they can only pick angles from a predefined set. If a signal falls between grid points, accuracy plummets.
  • Calibration: Traditional calibration requires "labeled data"—knowing the exact location of a transmitter—which is often impossible in practical, "in-the-wild" deployments.

Methods

The authors adopt the Model-Based Machine Learning (MB-ML) paradigm. Instead of using a "black-box" neural network, they use the physical equations of wave propagation to structure their learning process.

1. The Two-Stage Approach

The system operates in two phases:

  • Calibration (Offline/Batch): The system collects a batch of unlabeled measurements. It uses Simultaneous Orthogonal Matching Pursuit (SOMP) to get a rough estimate of DoAs, then uses Projected Gradient Descent (PGD) to "learn" the antenna locations and gains that best explain the data.
  • Estimation (Online): Once calibrated, the system processes new signals. It starts with a coarse SOMP guess and then "refines" the angles in the continuous domain using PGD.

2. Continuous Sparse Recovery

Unlike standard sparse recovery that is stuck on a discrete grid, this method treats the DoA angles as continuous variables. This "off-grid" refinement decouples the final accuracy from the initial grid resolution.

3. Comparison with Subspace Methods

The authors contrast their approach with "differentiable MUSIC" (diffMUSIC). While diffMUSIC tries to align signal and noise subspaces, the proposed method minimizes the residual energy of the actual measurements. As shown in the conceptual comparison below, this makes the method far more robust to coherent sources where subspace separation fails.

Conceptual comparison between the proposed residual-based approach and the subspace-based diffMUSIC.

Datasets

The study relies on rigorous simulations of a 16-antenna Uniform Linear Array (ULA):

  • Impairment Models: They simulated random antenna displacements (up to a quarter-wavelength) and random complex gain variations.
  • Signal Types: Both non-coherent (independent) and fully coherent (identical) source signals were tested.
  • Benchmarks: The method was compared against standard MUSIC, discrete SOMP, and "Oracle" versions of these algorithms that have perfect knowledge of the hardware.

Key Insights and Ideas

  • Unsupervised Calibration is Viable: You don't need to know where your targets are to fix your hardware. By assuming the environment is "sparse" (only a few signal sources), the system can infer the hardware errors just by looking at the consistency of the incoming waves.
  • Efficiency through Refinement: You don't need a massive, computationally expensive grid. You can start with a very coarse grid (e.g., only 50 points for a 180-degree span) and let the gradient descent "nudge" the estimates to their exact continuous values.
  • Breaking the Coherence Barrier: Because the method fits the signal directly rather than analyzing the covariance matrix, it remains highly accurate even when multiple sources are sending the exact same signal (fully coherent).

Key Results

  • Superior Accuracy: The proposed unsupervised method (UC-C-SOMP) actually outperformed the "Physical" D-SOMP baseline (which has perfect hardware knowledge but is stuck on a grid).
  • Snapshot Efficiency: While subspace methods like MUSIC require many "snapshots" (time samples) to build a stable covariance matrix, the proposed method reaches peak performance with as few as 6 snapshots.
  • Robustness: The calibration remained effective even in "degenerate" regimes where antennas were displaced so much they nearly overlapped.
  • Speed: The continuous refinement approach achieved higher accuracy than a 1000-point fine-grid SOMP while being significantly faster.

Limitations and Future Work

  • Inherent Ambiguities: Without at least one reference point, the unsupervised calibration cannot distinguish between a global shift of the array and a global shift of all signal sources. However, the authors note this only marginally affects DoA accuracy.
  • Computational Overhead: While more efficient than fine-grid methods, the PGD steps still add complexity compared to a simple MUSIC scan (see Algorithm 1 below for the simpler, but less robust, baseline).
  • Future Directions: The authors aim to extend this to mutual coupling (where antennas interfere with each other), near-field signals, and scenarios where the number of sources is unknown.
The classical MUSIC algorithm used as a baseline in this study.

Summary updated

Abstract

Direction of arrival (DoA) estimation, a classical signal processing problem, has experienced renewed interest with the advent of integrated sensing and communication (ISAC). While decades of research have yielded highly accurate DoA estimators, hardware impairments have frequently been neglected, despite degrading both DoA estimation accuracy and downstream communication performance in ISAC systems. This work introduces a joint array calibration and DoA estimation method based on the simultaneous orthogonal matching pursuit (SOMP) algorithm, following the model-based machine learning paradigm. The proposed method enables the calibration of antenna arrays subject to severe gain-phase and antenna location impairments through projected gradient descent (PGD), with either supervised or unsupervised objectives. Additionally, DoA estimation is enhanced by extending SOMP to continuous dictionaries via PGD refinement steps. Simulation results demonstrate that the proposed method enables efficient hardware impairment compensation, even without labeled samples, while outperforming conventional sparse recovery and subspace-based DoA estimation baselines.
BibTeX
@misc{chatelier2026model,
  author = {Baptiste Chatelier and José Miguel Mateos-Ramos and Vincent Corlay and Matthieu Crussière and Christian Häger and Henk Wymeersch and Nir Shlezinger and Luc Le Magoarou},
  title = {{Model-based learning for DoA estimation and array calibration via continuous sparse recovery}},
  year = {2026},
  eprint = {2611.09263},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.09263},
}
Figure