Noise-Aware Frequency Gains for Deep Interaction Graphs

Robots and self-driving cars increasingly reason about their neighbors with graph neural networks. These are models in which each agent (a car, a pedestrian) is a node, and each layer lets every node average in information from nearby nodes. Two well-known failure modes collide here, and they are usually studied apart. First, the averaging step is a smoothing operation. Stack enough layers and every agent's representation drifts toward the same vector. This is called over-smoothing, and it erases exactly what a planner needs: which car is braking, which pedestrian is cutting across. Second, these models are trained and evaluated as if positions and velocities were known exactly. Real trackers deliver jittered, dropped, and swapped detections. The standard cure for over-smoothing is to add a "high-pass" channel that keeps the differences between a node and its neighbors. But per-agent sensor noise is, by construction, a difference between a node and its neighbors. So the cure for one problem amplifies the other.

We propose to resolve this with a per-scene, per-frequency gain that the network computes from information it actually has at deployment: the tracker's own uncertainty for each agent. The idea in two sentences: break each layer's features into graph frequencies (from "shared by the whole group" to "specific to one agent"), estimate how much of each frequency is real signal versus sensor noise, and keep each frequency in proportion to its signal-to-noise ratio. The noise part can be computed exactly from the per-agent noise variances. The signal part is learned from simulation, where a clean "privileged" copy of every scene is available.

Core Claim

Anti-over-smoothing filters fall short on noisy multi-agent graphs, and so do plain smoothing networks, for the same reason. Each applies one fixed mix of low and high frequencies to every scene, while the right mix depends on the scene. If a far-away, poorly tracked agent dominates a particular high-frequency pattern, that pattern is mostly noise and should be suppressed. If the pattern is carried by well-tracked nearby agents, it is mostly signal and should be kept. We claim that a gain set by a Wiener-style signal-to-noise ratio, with the noise term computed in closed form from tracker covariances, lets deep interaction graphs keep agent-specific information without amplifying jitter. We further claim this beats both shallow graphs and existing filterbank networks once the scene is dense and the noise varies across agents.

Mechanism

Let $u_k$ be the $k$-th eigenvector of the interaction graph's Laplacian, with $k$ ordered from smooth to oscillating. Let $\sigma_i^2$ be the residual position/velocity variance the tracker reports for agent $i$. If tracker errors are independent across agents, the noise power that lands in frequency $k$ is exact:

$$N_k = \sum_i u_k(i)^2\,\sigma_i^2 .$$

For the handful to few dozen agents in a driving or crowd scene, eigendecomposition is cheap. For larger scenes we would use a polynomial approximation. The gain applied to frequency $k$ is

$$g_k = \frac{\hat S_k}{\hat S_k + N_k},$$

where $\hat S_k$ is the predicted signal power in that frequency. Each layer keeps a standard low-pass aggregation channel and adds a diversification channel in the style of Complete the Missing Half, which augments neighborhood averaging with a learnable high-pass channel. In our version, the high-pass channel's spectrum is reweighted by $g_k$ instead of by a single learned scalar.

The noise power changes as features pass through layers. For training, we therefore run each simulated scene twice, once clean and once noisy. This gives the true signal and noise power per frequency at every layer. A small predictor learns to map (graph, $\sigma_i$, layer) to these quantities. This is the privileged-learning recipe of Towards Autonomous Grading In The Real World, which trained a student on noisy heightmaps against a clean-state teacher and reached 91.4% real-world success versus 60.0% for plain behavior cloning. The difference is that we use the clean copy to supervise filter gains rather than actions.

Research Program

The work has three parts.

The first part is diagnostic. Are we measuring oversmoothing in graph neural networks correctly? showed that effective rank (an entropy measure of how many directions the feature matrix really uses) tracks accuracy loss in trained networks, with average Pearson correlation 0.91. The same paper warns that pure noise also has high rank. Paired clean/noisy runs let us split each layer's effective rank into a part shared with the clean twin and a part that is noise. We will measure, for the first time on dynamic interaction graphs, whether noisy inputs make collapsing networks look healthy.

The second part builds the gain mechanism above on two existing models. One is the relational graph network of Relational Graph Learning for Crowd Navigation. It infers pairwise relations between the robot and humans, propagates them with a shallow graph convolution, and assumes perfect positions. The other is the 2-layer graph of GISNet, which explicitly stopped at two layers and assumed clean NGSIM tracks.

The third part maps when depth helps at all. We will sweep depth against crowd density and noise heterogeneity to find the regime where multi-hop interaction matters.

Noise models include distance-dependent Gaussian jitter, detection dropout, and identity swaps. Only the first fits the closed-form noise term exactly. We will treat dropout as inflated variance and report swaps separately as a stress test where the method's assumptions break.

Baselines

The naive reference is each host model trained on clean data and tested on noisy data. This is the perfect-sensing setting, and it shows how large the problem is.

The second reference is the same model trained with noise augmentation. Every method, including ours, receives Kalman-filtered tracks and the same augmentation, so any gain is not just "saw noise during training."

The strongest graph competitor is the filterbank network of Complete the Missing Half, with learnable low/high mixing scalars, trained with noise augmentation. It reported roughly 10% accuracy gains on heterophilic node-classification graphs but was never tested under input noise.

We also run the adaptive gate of When Do GNNs Work. It stops aggregation per node based on neighborhood entropy and center-neighbor similarity, and reached 73.3% versus 59.2% for a plain graph convolution on Cora with 1% labels. Its similarity signal is exactly what jitter corrupts.

Graph-free references come from two papers. Fixed-Dimensional and Permutation Invariant State Representation of Autonomous Driving uses a sum-pooled set encoding that cut approximation error by 62.2% versus sorted vectors. Crowd-Robot Interaction uses single-layer attention pooling.

The published numbers to beat are as follows. On CrowdNav circle crossing with five humans and an invisible robot, MP-RGL-Multistep reports 96% success, 2% collisions, and 1.86 s extra time, against LM-SARL's 90%, 9%, and 3.15 s (Relational Graph Learning for Crowd Navigation). On NGSIM, GISNet reports 3.23 m RMSE at a 5 s horizon (GISNet). We found no reported result on Lacuna for any of these models under injected sensor noise, so we will rerun them ourselves under identical noise.

All methods get the same parameter count within 10%, the same hyperparameter search budget, and the same seeds.

Two ablations isolate the key ingredient. In the first, the predicted gain is replaced by an oracle gain computed from the clean twin. This gives an upper bound and shows how much is lost in prediction. In the second, the per-agent $\sigma_i$ is replaced by its scene average. That destroys the heterogeneity the closed-form noise term exploits. If the second ablation matches the full method, the noise-covariance idea is not what is doing the work.

"Beating" a baseline means lower noisy-input RMSE or higher noisy-input success, across three seeds and beyond seed spread, at matched clean-input performance.

Why not just…?

Why not just keep the graph shallow? GISNet's two layers and attention pooling already work well on sparse highway and five-human scenes. In those scenes we may well find no benefit from depth, and we will say so. The relevant counter-evidence is limited. Multi-step reasoning was needed for MP-RGL's gains over LM-SARL (Relational Graph Learning for Crowd Navigation), and representation collapse appears within about ten trained layers (Are we measuring oversmoothing in graph neural networks correctly?). Neither shows that depth pays off under noise. The density sweep is designed to settle this. Separately, the gain mechanism can be applied to a 2-layer model, and if it helps there too, the contribution survives without requiring depth.

Why not just denoise each agent with a Kalman filter or recurrent network first? Per-agent temporal filtering helps. A 3-step recurrent history lifted CARLA merging success from 93% to 98% under Gaussian noise (Trajectory Planning for Autonomous Vehicles Using Hierarchical Reinforcement Learning). That is why every method here receives Kalman-filtered tracks. But filtering leaves residual error that lags real maneuvers, and closed-loop perception error compounds (Privileged to Predicted). Our method consumes the filter's reported covariance rather than competing with the filter. If filtered tracks alone make the noise-augmented baseline match ours, the claim fails.

Why not just let a learned spectral filter or filterbank absorb the noise during training? This is the real threat. A filterbank trained with noise augmentation could learn good average gains. Our bet is that one global mix cannot adapt to scenes whose noise is concentrated in a few far agents. The heterogeneity ablation and the comparison against an augmented filterbank on scenes with mixed near and far agents test exactly this. If the augmented filterbank ties us there, the per-scene noise term adds nothing.

Why not just pool agents with a sum and skip edges? Sum pooling cannot represent agent-to-agent reactions such as yielding and merging cascades (Crowd-Robot Interaction, PiP). It is also immune to over-smoothing, so it is the right control for whether relational reasoning earns its cost under noise. We report it at every noise level.

Proposed Experiments

The trajectory testbed is NGSIM with 3 s of history and a 5 s horizon, scored by RMSE. We use HighD, whose drone tracks have minimal positioning error, as a near-clean source. We inject calibrated distance-dependent noise into it, so noise variance is known exactly rather than mixed with NGSIM's own camera error.

The closed-loop testbed is CrowdNav using the official CrowdNav code. We extend it from 5 to 10, 20, and 40 humans and add per-human noise that grows with range. We report success, collision, extra time, and freezing episodes.

At every layer we report the split effective rank: the part shared with the clean twin and the part that is noise.

The decisive checks are three. First, under heterogeneous noise in dense scenes, the full method should beat the noise-augmented filterbank, while the scene-averaged-variance ablation should not. Second, the split diagnostic should reveal baselines whose total effective rank stays high while their clean-shared rank collapses. Third, under uniform noise, where the closed-form noise term carries no scene-specific information, our advantage should shrink toward zero.

If the third prediction fails, meaning the method wins even under uniform noise, then the benefit comes from something other than the stated mechanism, and we would report that rather than the original claim.