Traditional multi-microphone audio processing relies on a fundamental assumption: we know exactly where the sensors are. In a standard microphone array, the physical distance between elements is fixed and calibrated. This allows algorithms like delay-and-sum beamforming to calculate precise Time Differences of Arrival (TDOA) and steer a "spatial filter" toward a target sound source.
However, modern audio capture increasingly happens via ad-hoc arrays—spontaneous collections of heterogeneous devices like smartphones, laptops, and IoT gadgets scattered randomly across an environment. As noted by Chang et al. and Wölfel et al., when microphone locations, orientations, and hardware characteristics are unknown and variable, traditional spatial filtering techniques completely break down. Without a fixed sensor manifold, you cannot compute standard steering vectors or align phases constructively.
To solve this, researchers have developed techniques that shift from geometry-dependent processing to geometry-independent or adaptive processing. These methods can be broadly categorized into four intuitive strategies: data-driven calibration, dynamic stream attention, geometry-agnostic neural fusion, and statistical invariance.
Data-Driven Calibration and Subspace Methods
One way to handle spatial uncertainty is to infer the missing geometric relationships directly from the captured audio signals, or to project the signals into a mathematical subspace where the physical layout becomes irrelevant.
If explicit coordinates are needed, Taghizadeh et al. demonstrate that the spatial arrangement of an ad-hoc array can be reconstructed using Euclidean distance matrix completion, even when many sensor-to-sensor distance measurements are missing due to obstructions or failures. For localization tasks, Lebarbenchon et al. map local directional estimates from individual microphone pairs onto a unified global coordinate grid through interpolation, allowing the localization algorithm to remain agnostic to the overall array geometry.
Alternatively, we can bypass coordinate estimation entirely. Xue et al. show that steering vectors for beamforming can be estimated directly from training data using cross-power spectra and signal presence probabilities, rather than relying on physical measurements. For multi-source localization, Grondin leverages Singular Value Decomposition (SVD-PHAT) to isolate the underlying signal subspace. Because this approach focuses on the algebraic structure of the signal rather than a fixed spatial grid, it functions seamlessly across arbitrary linear, planar, or 3D sensor placements.
In specialized hardware setups like acoustic vector-sensor arrays, Weiss treats the physical layout as a "nuisance parameter." By exploiting the internal mathematical constraints between pressure and velocity measurements via tensor decomposition, the direction of arrival can be estimated blindly without any prior knowledge of sensor locations.
Dynamic Stream Attention and Channel Selection
When microphones are scattered blindly across a room, some will inevitably be close to the speaker, while others will be buried in noise or reverberation. Instead of trying to phase-align all channels perfectly, why not dynamically figure out which microphones are performing best and prioritize them?
This intuition drives stream attention and real-time selection mechanisms. Yoshioka et al. introduced PickNet, a lightweight neural network that evaluates an ad-hoc array every few milliseconds to select the single microphone closest to the active speaker, drastically reducing word error rates with minimal latency. To utilize multiple channels without full decoding overhead, Hermansky et al. utilize an unsupervised temporal auto-encoder as a performance monitor to dynamically weight distributed streams based on signal clarity.
In deep learning frameworks, this selection can be optimized end-to-end. Liang et al. employ a residual self-attention mechanism with a sparsemax activation function for speaker verification. The mathematical property of sparsemax forces the network to assign exactly zero weight to highly noisy or distant channels, effectively pruning the ad-hoc array on the fly. Similarly, Wang et al. utilize a stream-attention mechanism within a dual-feature learning network to prioritize channels with the highest signal quality for speech separation.
[Ad-hoc Microphones] ---> [Feature Extraction] ---> [Stream Attention / Sparsemax] ---> [Zeroed Out (Noisy Mics)]
|
+---> [Weighted Fusion] ---------> [Enhanced Output]
Geometry-Agnostic Neural Fusion
Deep neural networks are exceptionally good at learning implicit relationships from unstructured data. Instead of hardcoding physical propagation models, modern architectures use attention mechanisms to map spatial relationships directly from raw multi-channel inputs.
Why does attention work so well for ad-hoc arrays? Traditional convolutional or recurrent layers expect a fixed input structure (e.g., channel 1 is always next to channel 2). Self-attention, however, is permutation-invariant; it computes pairwise relationships between all channels regardless of their order or total number. Wang et al. leveraged this by interleaving self-attention layers with temporal recurrent layers, allowing a speech separation network to remain highly effective regardless of how many devices are used or how they are arranged.
To capture more complex spatial topologies, Guo et al. integrated graph attention modules into a multi-channel U-Net architecture, treating each ad-hoc microphone as a node in a graph to automate channel fusion for dereverberation. When explicit node tracking is required, Gong et al. show that encoding arbitrary node identities as one-hot vectors combined with spatial-temporal attention allows a system to handle a variable number of single-microphone nodes for 2D source localization.
For decentralized environments where a central processor is unavailable, Furnon et al. demonstrate that multi-node Deep Neural Networks can cooperate by sharing compact, compressed signals between nodes to achieve distributed speech enhancement without spatial constraints.
This data-driven mapping mirrors biological systems; Goodman et al. demonstrated that spiking neural networks can learn to localize sounds adaptively by mapping coincidence-detector assemblies to spatial coordinates, completely compensating for unique or changing sensor structures without an explicit physical model.
Statistical and Source-Driven Invariance
A final strategy relies on the physics of the sound source itself or global statistical properties that remain invariant across different recording locations.
For instance, speech is generated by a physical mechanism that can be modeled. Zhang et al. developed Glottal Residual Assisted Beamforming (GRAB), which achieves zero-calibration signal combination by leveraging a fundamental glottal speech production model. Because the underlying speech source is consistent across all channels, the system can align heterogeneous, randomly placed microphones by focusing on source consistency rather than sensor geometry. Similarly, Flego et al. extend fundamental frequency (F0) estimation to multi-channel distributed networks, using a corrective post-processing step that relies on the shared periodicity of the source across unconstrained microphone positions.
From a statistical perspective, Petsatodis et al. introduced a source-agnostic fusion method for voice activity detection that treats each microphone as an independent statistical observer. By computing the geometric mean of likelihood ratios across these independent observers, they bypass direction-of-arrival estimation entirely, maintaining robust performance even in highly reverberant environments.
When dealing with crowdsourced audio from completely uncoordinated devices, Aziz et al. introduced a simple, training-free baseline that operates in the time-frequency domain. By identifying and removing magnitude outliers across different devices, the algorithm isolates the shared "global" signal from localized noises unique to individual microphone positions.
Finally, He et al. bypass source and room geometry entirely by modeling acoustic environments through receiver-to-receiver transformations (SPEAR). Instead of mapping a source to a receiver, they predict how audio sounds at any point in space by "warping" a recording from a known reference position, simplifying deployment in dynamic environments like mobile robotics where spatial parameters are constantly changing.
Key Foundational Work
Multi-source far-distance microphone selection and combination for automatic transcription of lectures (2006) — contributed 1 concept elements to this direction.
On the use of a weighted autocorrelation based fundamental frequency estimation for a multidimensional speech input (2004) — contributed 1 concept elements to this direction.