Near-infrared spectroscopy measures how a sample absorbs light between roughly 700 and 2500 nm. Because that light passes through skins, shells and tissue, a spectrometer can watch a fruit ripen or a seed age without destroying it. What it records, though, is a blurred mixture: water, sugars, proteins and lipids each contribute broad, overlapping absorption bands. So the chemicals whose interactions we care about are never measured directly. They are hidden behind a few hundred highly correlated wavelength channels. Methods that recover cause-and-effect networks from dynamics, meaning which constituent's level drives which other constituent's rate of change, assume the opposite. They expect clean, separately measured state variables, and most of them either estimate time derivatives from noisy data or run an ODE solver inside the training loop. Estimating derivatives from noise is ill-posed: differentiation amplifies high-frequency noise in proportion to its frequency. The ODE-solver route is slow.
We propose to fit the hidden chemical dynamics directly to the raw spectra through a "weak form." Instead of differentiating the data in time, we integrate each wavelength channel against smooth, known test functions and move the derivative onto those functions. Under the Beer–Lambert law, absorbance is linear in concentration, so this integral turns every spectrum sequence into a noise-averaged linear mixture of the hidden constituents' rates of change. The central bet is that a sparse, non-linear causal vector field is a strong enough constraint to do two jobs at once: pin down how to unmix the spectra into constituents, and reveal the directed graph among them. It should do this without any pointwise derivative and without an ODE solver.
Core Claim
Let $A(\lambda,t)$ be measured absorbance at wavelength $\lambda$ and time $t$. We model it as
$$A(\lambda,t) = s(t)\sum_{k=1}^{K} S_k(\lambda)\,c_k(t) + b(\lambda,t) + \eta(\lambda,t),$$
where $c_k(t)$ is the concentration of hidden constituent $k$, $S_k(\lambda)\ge 0$ is its pure-component spectrum, $s(t)$ is a per-scan multiplicative scatter factor, $b$ is a baseline that is smooth in wavelength, and $\eta$ is instrument noise. The constituents obey $\dot c = f(c)$. There is an edge $k \to j$ when $c_k$ enters $f_j$. This is the causal definition used by Consistency of mechanistic causal discovery in continuous-time using Neural ODEs.
Take smooth test functions $v_m(t)$ that vanish at the ends of the observation window. Integration by parts then gives a residual in which only the raw data and the exactly known $\dot v_m$ appear on the left:
$$r_m(\lambda) = -\int \dot v_m(t)\,\frac{A(\lambda,t)-b(\lambda,t)}{s(t)}\,dt \;-\; \sum_k S_k(\lambda)\int v_m(t)\, f_k(c(t))\,dt .$$
We fit $S$, a smooth parameterization of $c$, and a neural vector field $f$. The loss has three parts: a reconstruction term, the squared weak-form residuals $\sum_{m,\lambda} r_m(\lambda)^2$, and an adaptive group-lasso penalty on the input weights of each $f_j$, as in the NGM approach of Bellot et al.
What is new is that the dynamics residual is computed against the spectra themselves, not against a previously estimated concentration trajectory. We then make a testable identifiability claim. Mixing $c$ by an invertible matrix $M$ keeps linear dynamics linear, so for linear kinetics the unmixing stays ambiguous. For generic non-linear kinetics, $M f(M^{-1}c)$ is typically dense. Graph sparsity should therefore select the true constituent basis. This is the dynamic counterpart of the principle that non-linearity breaks the observational symmetry of linear-Gaussian models, as used by Least-squares independence regression for non-linear causal inference under non-Gaussian noise.
Relation to Prior Work
Several papers motivate the weak-form ingredient by showing how derivative estimation fails. A stable finite difference ansatz for higher order differentiation of non-exact data formalizes how noise amplification grows with derivative order. PySINDy and Data-driven identification of parametric partial differential equations regress term dictionaries on numerical derivatives and inherit that fragility. Inferring Biological Networks by Sparse Identification of Nonlinear Dynamics recovers rational enzyme kinetics, but it needs $\dot X$ in its library and direct concentration measurements.
Two papers supply the method we reuse. Weak Form Generalized Hamiltonian Learning trains with 200 Gaussian test functions, avoiding both differentiation and solver loops, and is about 45x faster than adjoint regression. D-CODE uses a variational objective for closed-form ODE discovery. Both assume that the physical states are observed. D-CODE explicitly says that with latent states it yields only an empirical approximation. We lift the weak form through a linear optical mixing map.
NGM supplies our graph definition and sparsity penalty, but it assumes fully observed states and integrates an ODE during training. Interpretable Neural ODEs for Gene Regulatory Network Discovery under Perturbations likewise avoids pointwise derivatives, by matching distributions of gene-expression snapshots. It works on directly measured genes and reports high variability on true time series.
Other work sets constraints on what we can claim. Unpaired Multi-Domain Causal Representation Learning and Latent Hierarchical Causal Structure Discovery with Rank Constraints recover latent causal structure from mixtures, but only with linear, static structural equations. The rank-based method identifies graphs only up to Markov equivalence. We keep the linear mixing, which is physically justified in absorbance, but put the non-linearity and time into the latent mechanism. That is exactly where those methods cannot go. On the spectroscopy side, Non-Destructive Prediction of Fruit Ripeness and Firmness Using Hyperspectral Imaging and Lightweight Machine Learning Models reports that PCA degraded accuracy and that field conditions cause 15–30% drops. Both findings shape our baselines and the scatter term in our model.
Research Program
We will build a single model with three parts. The first is non-negative pure-component spectra. The second is concentration trajectories represented by smooth splines, which works under irregular sampling. The third is a group-sparse MLP vector field, optionally with Hill-type units following PerturbODE.
Because the objective contains no ODE integration, each step costs one matrix product over wavelengths and test functions. Test-function design is a known weak point of weak-form methods. We will therefore treat the Gaussian test functions' width and spacing as hyperparameters chosen by held-out weak-residual error, and report how sensitive results are to that choice.
Our analysis will state conditions under which sparsity plus non-linearity rules out alternative unmixings. It will then check these conditions empirically by comparing linear and non-linear kinetics while holding the spectra fixed.
Destructive ground-truth assays will be used only as terminal anchors: one measurement per sample at the end, which fixes the scale of the constituents. They are never needed as trajectories.
Baselines
The naive reference point follows standard chemometrics. We compute PCA scores of Savitzky–Golay-smoothed spectra, take finite differences in time, and regress each score's derivative on all scores with group lasso. Any method that cannot beat this in edge-recovery AUPRC has no reason to exist.
The strongest existing competitor is a two-stage pipeline. First, unmix the spectra into non-negative components using the same bilinear model without dynamics. Then run NGM from Consistency of mechanistic causal discovery in continuous-time using Neural ODEs on the recovered trajectories. NGM is the only method in the file with consistency guarantees for continuous-time non-linear graph recovery under irregular sampling. With fully observed states it reports true-positive rate above 0.95 at $d=100$ on the Rössler system. We will also run NGM on the true concentrations as an oracle upper bound. To make the comparison fair, all methods get the same spectra and train/validation split, the same vector-field architecture, the same number of hyperparameter configurations, and the same GPU budget. Beating NGM means higher edge AUPRC, or lower false-discovery rate at matched true-positive rate, at moderate-to-high noise. It also means doing so at a substantially lower wall-clock cost.
A discrete-time baseline, neural Granger causality on the unmixed components, tests the sampling-consistency argument.
Two ablations isolate our key ingredients. The first keeps our joint model but replaces the weak-form residual with a pointwise residual on Savitzky–Golay time derivatives. This isolates the weak form itself. The second keeps the weak form but applies it to pre-unmixed concentrations rather than raw spectra, as a weak-form version of SINDy. This isolates the joint coupling between unmixing and dynamics.
For context on what weak and variational objectives buy over derivative regression, Weak Form Generalized Hamiltonian Learning reports state error 0.17 versus 0.48 at noise level $\sigma=0.1$, and D-CODE reports RMSE 0.220 versus 0.637 on tumor trajectories. Neither was measured on spectral mixtures. No reported result was found on Lacuna for any method recovering causal graphs from spectroscopic time series.
Why not just…?
Why not just smooth, differentiate and run SINDy or NGM on the unmixed components? This is the two-step route. D-CODE argues that such pipelines need the derivative estimate, not just the trajectory estimate, to converge, and its two-step symbolic-regression baseline was nearly three times worse. The first ablation measures this directly in our setting. We expect a crossover: at low noise and dense sampling, two-step may tie us. We will report where the crossover occurs rather than claim uniform superiority.
Why not just put an encoder in front of NGM and train end-to-end with an ODE solver? This is a real threat. It could match our accuracy, because it also couples unmixing and dynamics. We will implement it as an extra baseline with the same budget. Our predicted advantage comes from two sources. One is the roughly order-of-magnitude training speedup that the weak form showed over adjoint methods. The other is stability on stiff kinetics such as the glycolysis oscillator. If the encoder-plus-NGM model matches us on both accuracy and cost, the contribution reduces to the identifiability analysis, and we will say so.
Why not just apply weak-form differentiation across wavelength instead of time? A Savitzky–Golay derivative already is a convolution with the derivative of a local polynomial smoothing kernel. It is, in effect, a weak-form spectral derivative. Chemometrics has therefore already absorbed that idea. The unsolved instability is in time, where the causal quantities live.
Why not just use PCA scores with discrete Granger tests? PCA components are variance-ranked rotations, not chemical constituents. The ripeness study found that PCA discards low-variance absorption bands. Discrete-time models also give inconsistent graphs as the sampling interval changes, per Bellot et al. The naive baseline and the Granger baseline quantify both effects.
Proposed Experiments
The file contains no longitudinal NIR dataset with known dynamics. The core evaluation is therefore semi-synthetic, and every step of its construction will be released.
Latent kinetics come from the file's canonical systems. These are yeast glycolysis (7 variables), B. subtilis competence (5th-order Hill functions), and Lorenz as a chaotic stress test. We add linear-kinetics counterparts with matched graphs. Pure-component spectra are built from Gaussian bands at the documented absorption sites: C–H overtones in the 700–1100 nm band, water at 1440 and 1920 nm, and the 970/1200/1450 nm aging bands. Noise and scatter are taken from real spectra. We bootstrap residuals from the rice-seed NIR imaging data (924–1595 nm, 200 bands) in Assessment of the vigor of rice seeds by near-infrared hyperspectral imaging combined with transfer learning and from the DeepHS fruit cubes used by Konrad et al. We then sweep noise level, sampling interval (including irregular sampling), number of constituents, and spectral overlap.
Metrics cover four things. Graph recovery is measured by edge AUPRC, true-positive rate and false-discovery rate. Latent recovery is measured by the Hungarian-matched correlation between recovered and true trajectories, and by spectral-angle error on the pure-component spectra. Forecast quality is held-out spectrum forecasting error. Cost is wall-clock training time.
The first decisive check is the identifiability prediction. With the same spectra and noise, unmixing error should be markedly lower under non-linear kinetics than under their linear counterparts, and only for the joint weak-form model. If there is no gap, the claim that sparsity resolves unmixing fails.
The second decisive check is the pair of ablations. The weak form should widen its margin over the pointwise residual as noise grows, and the joint model should beat weak-form fitting on pre-unmixed concentrations when components overlap strongly.
As a real-data sanity check, we will apply the method to the rice-seed harvest-year series, treating storage age as coarse time. We will ask only whether the recovered constituents align with the reported starch, protein and lipid bands and with germination vigor. Graph claims on this data would be unverifiable, so we will not make them.
References
- Neural graphical modelling in continuous-time: consistency guarantees and algorithms
- Least-squares independence regression for non-linear causal inference under non-Gaussian noise
- A stable finite difference ansatz for higher order differentiation of non-exact data
- PySINDy: A Python package for the sparse identification of nonlinear dynamical systems from data
- Data-driven identification of parametric partial differential equations
- Inferring Biological Networks by Sparse Identification of Nonlinear Dynamics
- Weak Form Generalized Hamiltonian Learning
- D-CODE: Discovering Closed-form ODEs from Observed Trajectories
- Interpretable Neural ODEs for Gene Regulatory Network Discovery under Perturbations
- Unpaired Multi-Domain Causal Representation Learning
- Latent Hierarchical Causal Structure Discovery with Rank Constraints
- Non-Destructive Prediction of Fruit Ripeness and Firmness Using Hyperspectral Imaging and Lightweight Machine Learning Models
- Assessment of the vigor of rice seeds by near-infrared hyperspectral imaging combined with transfer learning