Channel Charts That Calibrate Their Own Ray Tracer

Radio signals from a moving phone change smoothly as the phone moves. Channel charting uses that smoothness to lay out unlabeled signal measurements on a map where nearby measurements sit near each other. It needs no surveyed positions, but the map comes out bent, rescaled, and floating in arbitrary coordinates. Differentiable ray tracers do the opposite job. A differentiable ray tracer is a radio-propagation simulator whose outputs can be differentiated with respect to its inputs, so gradient descent can recover a transmitter's position from what was received. They give metric coordinates, but they need a calibrated 3D model of the room, known wall materials, and a starting guess close enough to avoid the many local minima created by wave interference. We propose to train a single channel-charting network whose predicted positions must also be physically consistent: when a ray tracer is run at the predicted position with jointly learned wall materials, it must reproduce the measured signal. The chart gives the simulator a globally sensible starting point. The simulator in turn gives the chart real-world metric coordinates. Because thousands of positions share the same walls, the wall materials become identifiable without the reference measurements current methods need.

The central design choice is where the physics loss is computed. We do not compare raw complex channels, whose phase oscillates on centimeter scales. We compare the fading-robust summary features that channel charting already relies on, angle-delay profiles, which keep large-scale geometry and discard fast fading. Our claim is that this borrowed representation is what makes differentiable wave inversion well-behaved. It could replace the ad-hoc smoothing current systems use.

Core Claim

Let $f_\phi$ be an encoder mapping a channel measurement $h_i$ to a position $\hat{x}_i = f_\phi(h_i)$. Let $\theta$ be unknown per-class wall material parameters (permittivity, conductivity, scattering). Let $\mathcal{S}(x;\theta, G)$ be a differentiable ray tracer over a coarse, uncalibrated geometry $G$, such as a floor plan extruded to walls or a raw point cloud. Let $A(\cdot)$ compute an angle-delay profile. We train

$$\min_{\phi,\theta}\; \sum_i d\big(A(h_i),\,A(\mathcal{S}(f_\phi(h_i);\theta,G))\big) \;+\; \lambda\, \mathcal{L}_{\text{chart}}(\phi),$$

where $d$ is a scale-invariant distance between profiles, $\mathcal{L}_{\text{chart}}$ is a standard temporal triplet channel-charting loss, and $\lambda$ trades them off.

We make three testable claims. First, amortizing the inversion through a shared encoder, rather than solving it point by point, lets $\theta$ be recovered without known device positions or static calibration measurements. Second, the chart term removes the need for loss-landscape smoothing and close initialization. Third, the physics term fixes the non-linear warping and wall "short-circuits" that pure charting suffers in non-line-of-sight regions. That last case is where received-power heuristics fail, because they cannot represent wall attenuation.

Position Relative to Prior Work

Two strands motivate the gap. Channel charting, introduced in Channel Charting, compresses massive-antenna measurements into a chart with high neighborhood preservation, but only up to an unknown deformation. Channel Charting in Real-World Coordinates grounds the chart using known access-point positions and a bilateration loss, which penalizes placing a device closer to the access point that heard it more weakly. That paper reaches 7.64 m mean error on an 83 m × 122 m ray-traced urban scene, close to 5%-labeled semi-supervision at 7.20 m. Its received-power reasoning is a zeroth-order physics model. We replace it with a full propagation model, and the question is whether that buys accuracy, especially behind walls. The optimal-transport approaches of Modality-Agnostic Topology Aware Localization and Deep Learning Frameworks for Weakly-Supervised Indoor Localization align learned manifolds to floor plans without any signal physics. They reach about 1.13–1.26 m on real Wi-Fi. Their method constrains which corridor a point lands in, not where inside it.

The differentiable-physics strand supplies the method. RayLoc inverts a differentiable ray tracer from commodity Wi-Fi channel measurements to device coordinates. It requires a metric CAD model, a background-refinement stage using stationary reference measurements to tune wall properties, and Gaussian smoothing of the loss, and it optimizes each measurement in isolation. WiNeRT trains a neural surrogate ray tracer on data from a commercial simulator and inverts it for position, reaching 0.58 m median error on a synthetic three-room scene. It cannot learn from uncalibrated real data. Differentiable ray tracing over point clouds recovers material parameters with Sionna-backed tracing at under 90 ms per link, but assumes known transmitter and receiver positions. We take its geometry representation so the method can run without CAD. DiffSBR constrains our loss design: it showed that projecting raw multi-antenna radar signals into a spatial representation before computing error tames phase sensitivity. We apply the same logic using channel charting's angle-delay features.

What none of these do is couple the two halves. Charts never meet a simulator, and simulators never exploit the manifold that ties measurements together. In our formulation that coupling is what makes positions and materials jointly identifiable.

Research Program

The system has three parts. The first is a channel-charting encoder trained with temporal triplets on angle-delay profiles, following the feature pipeline of Channel Charting. The second is a Sionna-based differentiable ray tracer operating on coarse geometry. Access-point positions are assumed known, as in Channel Charting in Real-World Coordinates, which also removes the reflection and rotation ambiguity that plagues symmetric layouts. The third is a small set of material parameters, one per semantic wall class.

Training runs in three phases. We warm up with the chart loss only. We then fit a rigid transform plus scale using the physics loss alone, which is cheap and gives a first basin. Finally we train everything jointly. To control cost we trace only for a minibatch of chart points per step, and we cap tracing at a few bounces, following the path budgets reported by the point-cloud tracer.

Three analyses accompany the build. The first is identifiability: we sweep the number of distinct positions per material class and measure material recovery error, to test the amortization claim directly. The second is loss-landscape probing: we plot the physics loss along lines through true positions under raw-channel, power-only, and angle-delay comparisons. This tests whether the feature choice, rather than smoothing, is what widens the basin. The third is model-mismatch robustness: we train with geometry from which furniture is deleted and some walls are displaced, and see whether the chart term absorbs the model bias or whether the physics term drags points to unphysical locations.

Streaming deployment is a secondary goal. We will reuse the similarity-based buffer of Channel Charting for Streaming CSI Data so the physics loss is evaluated on a fixed 1,000-sample core set.

The recipe is modality-agnostic in structure: a manifold learner plus any differentiable forward model evaluated in a fading-robust feature space. As a small portability check we will run it on the acoustic setup of the differentiable multi-sphere scattering model. The main evidence will be radio.

Baselines

The naive reference point is the raw channel chart after a least-squares similarity transform fitted using access-point positions. It shows how much error comes simply from arbitrary chart coordinates.

The strongest label-free method in the file is the bilateration-plus-triplet chart of Channel Charting in Real-World Coordinates. It reports 7.64 m mean distance error on its ray-traced urban scene, against 7.20 m for 5%-labeled semi-supervision. It is the right competitor because it uses exactly our side information, access-point locations and timestamps, with a crude physics prior. We will rerun it on every benchmark with the same encoder, features, epochs, and hyperparameter search budget. Beating it means lower mean error than 7.64 m on that scene, and more importantly, beating the 7.20 m semi-supervised reference without labels. On real data it must also win specifically on the non-line-of-sight subset.

The strongest physics-inversion competitor is RayLoc-style per-point inversion RayLoc. We run it with the same ray tracer and geometry. It gets its background-refinement stage when reference measurements exist, and Gaussian smoothing plus multiple random restarts, with restarts matched in wall-clock time to our training. The file gives no absolute localization error for RayLoc, so we will report our own reproduction.

For topology-aware alignment we include OT-Isomap from Modality-Agnostic Topology Aware Localization given the same floor plan. It reports 1.2 m average on its real passive Wi-Fi dataset. For real mobile data, the map-matched velocity chart of Velocity-Based Channel Charting reports a 90th-percentile error of 1.16 m on walking 5G and 0.90 m on the robotic dataset. Those numbers are the bar where odometry is available.

Supervised fingerprinting on the same encoder serves as an upper reference, not a competitor.

Four ablations isolate the key ingredients. The first is the two-stage pipeline: chart first, then per-point physics refinement with no gradient into the encoder. The second fixes materials at textbook values. The third uses raw channel phase instead of angle-delay profiles in the physics loss. The fourth drops the chart term, which tests the amortized physics loss alone.

Why not just…?

Why not just chart first and then refine each point with RayLoc? This is the most serious threat, and it is our first ablation. The argument against it is that per-point refinement cannot calibrate materials without reference measurements. It also lets neighboring points drift independently into different multipath minima, while joint training shares both material estimates and encoder smoothness. If the two-stage pipeline matches joint training on material recovery and non-line-of-sight error, the joint claim fails and we will report that. The simpler pipeline would then be the recommendation.

Why not just make the bilateration loss wall-aware with a differentiable multi-wall path-loss model? This could plausibly match us at meter scale, since much non-line-of-sight error is attenuation that a per-wall loss term captures. We therefore make it an explicit rung on a fidelity ladder: bilateration, then learned multi-wall path loss, then full ray-traced angle-delay profiles, all with identical training. The ray tracer must earn its cost through delay and angle structure, which power models cannot predict. We expect that to matter in rooms where power alone is ambiguous. If the ladder flattens after the multi-wall rung, the contribution becomes a cheaper method plus a negative result about ray tracing, and we will say so.

Why not just collect a few labeled points? Taner et al. show 5% labels yield 7.20 m on their scene. Labels are a legitimate alternative wherever surveying is cheap. Our target is deployments where it is not, and we report error as a function of label fraction so readers can see the break-even point.

Why not just use derivative-free search with an off-the-shelf simulator? Joint material and trajectory inference has thousands of position variables plus material parameters. Conventional tracing costs seconds per 5-bounce link, per the point-cloud tracer. That makes population-based search impractical at this scale. We include one budget-matched CMA-ES (covariance matrix adaptation, an evolutionary search method) run over materials only, with the chart fixed, as a sanity check.

Proposed Experiments

The controlled testbed is the WiSegRT dataset: ten indoor scenes with semantic material labels and Sionna ray tracing. Its Scene_01 alone has 30 transmitters and 3,325 receivers on a 20 cm grid. We sample smooth trajectories through the receiver grid to create unlabeled streams. Ground-truth materials allow material recovery error to be measured, and we can inject controlled geometry mismatch. The file reports no channel-prediction accuracy for WiSegRT, so all numbers there are ours.

The second benchmark is the Wireless InSite urban scene of Channel Charting in Real-World Coordinates, where the 7.64 m and 7.20 m references apply directly. For inverse-rendering comparison we use WI3ROOMS from WiNeRT. Its 0.58 m median comes from a surrogate trained on simulator output, whereas ours uses no position supervision, so the comparison measures the cost of removing supervision rather than a head-to-head. Real-data evaluation uses DICHASUS, as used in Channel Charting for Streaming CSI Data, with a floor plan and access-point positions. The file reports only trustworthiness there (0.975 full-batch, 0.963 with the 1,000-sample buffer), not position error, so we will reproduce the charting baselines ourselves.

Metrics are mean and 90th-percentile distance error, split by line-of-sight and non-line-of-sight status, plus trustworthiness and continuity. Trustworthiness asks whether points near each other on the chart are truly close; continuity asks whether truly close points stay close on the chart. On synthetic scenes we add relative error in recovered permittivity and conductivity.

Three results would be decisive. On the urban scene, the method should beat 7.64 m and reach or pass 7.20 m without labels. On WiSegRT without reference measurements, joint training should recover materials within a stated tolerance where the two-stage ablation cannot. And the ray-traced rung should beat the multi-wall rung by a clear margin on non-line-of-sight points. Failure on the third check would narrow the paper's claim to physics-consistent charting with the cheapest adequate forward model, which is still an answer to the question.