DNA sequencing tells you which metabolic genes a microbial community carries. It does not tell you which organisms are actually using them. The clearest demonstration in this file is a streambed study. Ammonia-oxidizing archaea carried more copies of the key ammonia-oxidation gene than ammonia-oxidizing bacteria, yet inhibitor experiments showed the bacteria performed over 97% of the oxidation and the archaea less than 1% (Linking Abundance and Activity of Ammonia‐Oxidising Bacteria and Archaea). Getting that answer required selective chemical inhibitors, and those exist for only a handful of pathways. For most metabolites measured in a community, nobody can say which organism made them.
Audio source separation has a closely matching problem with a known cure. When a sound recording is split into frequency bands and each band is unmixed separately, every band returns the speakers in an arbitrary order and at an arbitrary loudness. This is the "frequency-domain permutation problem." It is solved by coupling the bands so that one source keeps the same identity in all of them. Our proposal maps that solution onto microbial communities. Each site, sediment depth, or treatment plays the role of a frequency band, each sample plays the role of a time frame, and the organisms play the role of speakers. There is one twist. In audio, the arbitrary per-band loudness is a nuisance that gets thrown away. In metabolic validation, the per-site loudness of an organism is exactly the quantity we want: its activity in that environment, per unit of DNA.
Core Claim
Attributing measured metabolic output to co-occurring taxa fails when each site or condition is analyzed separately. The failure mirrors per-band source separation. When an abundant dormant taxon and a rare active taxon have correlated abundances within a site, the fit can hand one taxon's credit to the other without hurting the fit at all. We call this a credit swap.
We propose a coupled model with two assumptions. First, each taxon's production profile (which outputs it makes, per cell) is shared across all sites, and it is restricted to reactions that taxon's genome actually encodes. Second, each taxon has a separate activity gain at every site. Under these assumptions, sites where the two taxa's abundance ratio differs break the swap, much as coupling frequency bands fixes speaker order. The fitted site-specific gains are then reported as activity estimates.
Concretely, let $y_s(m,t)$ be the measured production of output $m$ at site $s$ in sample $t$. An output can be a metabolite feature or a process rate such as nitrate production. Let $d_{s,k}(t)$ be the DNA abundance of taxon $k$ in that sample. We model
$$y_s(m,t)=\sum_k B(m,k)\,G(m,k)\,a_{s,k}\,d_{s,k}(t)+\epsilon_s(m,t).$$
Here $B(m,k)\in\{0,1\}$ is a mask that is 1 only if genome $k$ encodes a route to output $m$. $G(m,k)\ge 0$ is the per-cell production profile, shared across sites. $a_{s,k}\ge 0$ is the activity gain of taxon $k$ at site $s$. $\epsilon$ is noise.
Fitting each site separately leaves $G\,a_{s,k}$ free to trade places between taxa whose $d_{s,k}$ are collinear. In the regression, attribution variance grows roughly as $1/(1-\rho_s^2)$, where $\rho_s$ is the within-site abundance correlation. Sharing $G$ across sites pools information from every site where $\rho_s$ is lower. The scale shared between $G$ and $a$ is fixed by normalizing each column of $G$. Absolute activity is the product $G\,a$.
Relation to Prior Work
Four signal-processing papers supply the method.
The ILRMA review (Sawada et al.) quantifies the cost of per-band separation. Per-band unmixing, known as independent component analysis (ICA), reaches 6.27/1.38 dB signal-to-distortion ratio (SDR). Independent vector analysis (IVA), which couples bands through a shared time envelope, reaches 13.52/8.79 dB. ILRMA, which adds low-rank spectral structure, reaches 16.78/12.33 dB.
Geometrically constrained IVA (Li Li et al.) adds direction-of-arrival constraints to the unmixing. Our genome-content mask $B$ plays the same role: a physical prior on what each source can emit.
UNSSOR and ERAS (Wang and Watanabe; Saijo et al.) show that unsupervised separators exploit swaps across frequency bands to satisfy reconstruction losses. ERAS removes this failure with a coupling regularizer applied in a first training stage, converging in 25 of 25 runs. We take from this the lesson that coupling must be enforced early, before the swap is locked in.
Simplex-based activity detection (Laufer-Goldshtein et al.) handles a speaker active only 5% of the time, at 16.8 dB signal-to-interference ratio versus 1.5 dB for ILRMA. Our treatment of rare-but-active taxa will borrow its approach of finding the samples where one source dominates.
We differ from all four in the same way. Our "sources" are labeled by DNA rather than fully blind. The scaling ambiguity is the target rather than a nuisance. Validation comes from biochemical ground truth rather than clean reference signals.
Three lines of microbiome work motivate the gap.
HUMAnN (Abubucker et al.) reconstructs metabolic potential from sequencing reads. It is validated at Pearson correlation >0.9 against mock-community pathway abundances, but it explicitly measures potential, not activity. PICRUSt-style prediction (Li et al.) and path modeling (He et al.) correlate taxa with outputs one link at a time. Paired metabolomics–metagenomics studies (Fusarium wilt; coronary heart disease) use pairwise correlations. None of these models the attribution as a joint unmixing problem across sites.
The streambed study (Wang et al.) plays two roles. It motivates the work, because abundance-proportional attribution gets the answer backwards. It also constrains the work: its inhibitors were incomplete, so even its ground truth carries model-based corrections.
Research Program
We will build a nonnegative, mask-constrained solver for the model above, using multiplicative or alternating updates in the style of ILRMA. The coupling across sites will be applied from initialization, following the staged-regularization lesson from ERAS. Two optional extensions follow the audio literature directly. One is a low-rank pathway structure on $G$, the analogue of ILRMA's shared spectral bases, so that metabolites in the same pathway tend to be credited to the same taxon. The other is a vertex-finding initialization that locates samples dominated by a single taxon, adapted from Max-Corr.
The analysis has three parts. First, we will characterize when the gains $a_{s,k}$ are identifiable. We expect this to depend on the spread of abundance ratios across sites and on how much the genome masks overlap. Second, we will measure how often per-site fitting produces credit swaps as within-site correlation rises. Third, we will test whether the coupled model recovers activity that disagrees with abundance, which is the case that matters scientifically.
Baselines
The naive reference point is abundance-proportional attribution. Each output is credited to taxa in proportion to their copies of the relevant gene. This is what HUMAnN-style potential implies, and it is what the streambed study showed to be inverted. It runs on identical inputs and has no tuning. Our method must beat it by assigning the majority of ammonia oxidation to the bacteria, where abundance-proportional attribution would credit the archaea.
The strongest direct competitor is per-site constrained regression or nonnegative factorization, followed by matching of taxa across sites. This is the analogue of "per-band ICA plus a separate alignment step." It uses the same mask $B$, the same sites, and the same nonnegativity constraints, so it differs from our method only in whether $G$ is shared. Both methods get the same hyperparameter search budget and the same random restarts. Beating it means two things: lower attribution error on synthetic ground truth, and fewer sign or credit swaps across restarts.
The second competitor is a pooled model that shares $G$ but forces a single gain per taxon across all sites. It tests whether site-specific activity is necessary or whether coupling alone does the work.
The third is pairwise Spearman correlation and partial least squares path modeling, as in He et al., with taxon–output links ranked by strength.
Two ablations isolate the key ingredients. One removes the genome mask $B$, testing whether the physical prior or the coupling breaks the swaps. The other removes the coupling across sites while keeping the mask.
On the strongest reported numbers, the file contains no reported result on Lacuna for computational activity attribution on the streambed data. The only reference is the inhibitor-derived ground truth itself: more than 97% bacterial and less than 1% archaeal ammonia oxidation in ammonium-amended microcosms without inhibitors. For metabolite-level attribution in mixed communities, we found no reported result on Lacuna. The nearest figure is HUMAnN's Pearson correlation >0.9 on mock-community pathway potential, which we cite as context and not as a comparable activity metric.
Why not just…?
Why not just correlate taxon abundance with metabolite intensity? When taxa share a niche, their abundances are collinear, so correlation credits all of them equally. In the streambed case, gene abundance and activity were decoupled outright. Pairwise correlation is a baseline in every experiment. The synthetic sweep over within-site correlation will show where it collapses relative to the coupled model.
Why not just map metatranscriptomic reads to genomes and compute RNA-to-DNA ratios? This is a real threat for transcripts. Where uniquely mappable RNA data exist, mapping resolves attribution without any unmixing. We therefore include read-mapping-based ratios as a baseline wherever transcripts are available, and we concede that domain. Our claim is restricted to outputs that carry no sequence label: metabolite features and bulk process rates such as nitrate production. Read mapping cannot assign these at all. If the coupled model adds nothing beyond transcript ratios on outputs that do have transcripts, we will report that.
Why not just use selective inhibitors? Inhibitors exist for few pathways and are imperfect; 1-octyne inhibited the bacteria only incompletely, and the authors needed a Bayesian correction. We use inhibitor experiments as ground truth precisely because they are expensive. The question is whether the coupled model can reproduce their answer from data collected without inhibitors.
Why not just apply a generic joint matrix factorization across omic layers? Generic joint factorization, a family of methods outside this file, typically shares factors but neither restricts profiles to genome content nor treats per-site scale as the estimand. The pooled baseline and the no-mask ablation are exactly these simplifications. If they match the full model, the proposal reduces to known tools, and we will say so.
Proposed Experiments
The first experiment is synthetic and decisive for mechanism. We will simulate communities from genome sets with known content and use the mask derived from that content. We will vary three things: the dormant fraction (abundant taxa with $a\approx 0$), the within-site abundance correlation from 0 to 0.95, and the number of sites from 2 to 20. Outputs are noisy metabolite and process rates. The metrics are attribution error on $G\,a$, rank correlation of the recovered $a_{s,k}$ with the truth, and the swap rate across 25 restarts, mirroring the ERAS protocol. The prediction is that per-site fitting degrades sharply as correlation rises while the coupled model stays stable once enough sites differ in abundance ratio.
The second experiment is a biological test on the streambed design (Wang et al.). Reaches, depths, and ammonium treatments serve as sites. Archaeal and bacterial gene copies serve as $d$. Nitrate production serves as $y$. The inhibitor treatments are held out entirely. Success means recovering a bacterial share consistent with >97% and an archaeal share near the reported <1%, while abundance-proportional attribution assigns the reverse. A second success criterion is that the model flags archaeal gene growth under inhibition as unexplained by ammonia oxidation, matching the authors' observation. If the authors' data cannot be obtained, we will run a replicate microcosm design with the same inhibitors.
The third check is an implementation sanity test. On simulated two-source, two-microphone reverberant audio, the same solver should reproduce the qualitative ICA-to-IVA gap reported by Sawada et al.. This confirms that the coupling code does what the audio literature says it should before we trust it on microbes.
The fourth experiment is exploratory: the redox zones of the hyporheic dataset (He et al.), used to generate inhibitor-testable predictions about which taxa drive dissolved organic nitrogen turnover. There is no ground truth here, so this experiment carries no claim of validation.
References
- Linking Abundance and Activity of Ammonia‐Oxidising Bacteria and Archaea in an Agriculturally Impacted First‐Order Stream
- A review of blind source separation methods: two converging routes to ILRMA originating from ICA and NMF
- Online Directional Speech Enhancement Using Geometrically Constrained Independent Vector Analysis
- UNSSOR: Unsupervised Neural Speech Separation by Leveraging Over-determined Training Mixtures
- Enhanced Reverberation as Supervision for Unsupervised Speech Separation
- Audio source separation by activity probability detection with maximum correlation and simplex geometry
- Metabolic Reconstruction for Metagenomic Data and Its Application to the Human Microbiome
- Comparison Between the Fecal Bacterial Microbiota of Healthy and Diarrheic Captive Musk Deer
- Responses of Microbial Community to Heterogeneous Dissolved Organic Nitrogen Constituents in the Hyporheic Zones of Treated Sewage–Dominated Rivers
- Integrated metabolomics and metagenomics uncover pathogenic mechanisms of Fusarium wilt and faba bean defense responses
- Integrated metabolomics and metagenomics analysis of plasma and urine identified microbial metabolites associated with coronary heart disease