Parameter Linearization for Least Squares Estimation

41 concepts32 papers2 proposals

When modeling real-world phenomena—whether tracking the motion of fluid waves, identifying the viscoelastic properties of a biological cell, or estimating the trajectory of a flock of animals—we are almost always confronted with non-linear equations. Directly fitting these non-linear models to experimental data is notoriously difficult. Non-linear optimization algorithms frequently get trapped in local minima, demand heavy computational resources, and rarely offer global convergence guarantees.

To bypass these limitations, researchers rely on a powerful mathematical strategy: parameter linearization. The core intuition is that while a system's underlying physical variables or noise processes may be highly non-linear, the equations can often be mathematically transformed so that the unknown parameters enter the system linearly. Once a problem is framed as linear in its parameters, standard Ordinary Least Squares (OLS) or Weighted Least Squares (WLS) can be applied. This unlocks closed-form analytical solutions, computational efficiency, and rigorous statistical error bounds.

The Core Intuition of Parameter Linearization

To understand why parameter linearization works, consider a physical process where an observed intensity decays exponentially over time, governed by an unknown rate constant. Attempting to fit this curve directly requires non-linear regression. However, by applying a logarithmic transformation or exploiting the differential structure of the process, the non-linear relationship can be reframed into a linear system of equations.

This strategy spans diverse fields of research:

  • Computer Vision: In optical flow estimation, Haussecker et al. demonstrate that complex, non-linear physical processes governing brightness changes (such as thermal diffusion or exponential decay) can be integrated into motion frameworks by reformulating the problem to remain linear in its parameters. This allows standard linear solvers to efficiently compute both optical flow and physical constants simultaneously.
  • Linguistics: When modeling the relationship between text length and vocabulary size, Tuldava linearizes a probabilistic growth equation, enabling the extraction of structural constants using simple least squares fitting from minimal data points.
  • Oceanography: In ocean surface dynamics, Yang et al. transform the non-linear JONSWAP wave spectrum into a linear form using a natural logarithmic scale. This mathematical manipulation makes wind-dependent wave characteristics amenable to linear regression, bypassing the complexities of direct non-linear fitting.

Shifting Domains to Expose Linearity

When a system cannot be easily linearized in its native time or spatial domain, a powerful alternative is to map the data into a completely different mathematical space—such as the spectral, frequency, or characteristic domains. In these alternative spaces, the parameters of interest often decouple and emerge as linear coefficients.

The Log-Characteristic Domain

For signals governed by heavy-tailed, non-Gaussian distributions (such as Symmetric $\alpha$-stable processes), traditional least squares fail because the variance is infinite. To solve this, Aziznejad et al. operate within the log-characteristic function domain. In this transformed space, signal dispersion and noise variance act as linear coefficients relative to different powers of the frequency variable, allowing fast linear least-squares solvers to recover the underlying dictionary parameters.

The Spectral Domain

Similarly, when analyzing stochastic jump processes, Belomestny estimates the fractional order of a Lévy process by transforming historical price or option data into the spectral domain. This transformation linearizes the relationship between the data and the fractional order parameter, enabling a unified weighted least squares procedure to extract the jump activity parameter with strong theoretical accuracy guarantees.

Frequency Warping

In speech processing, the vocal tract introduces non-linear frequency variations that human hearing perceives non-linearly. Quintana-Morales et al. (2004) and Quintana-Morales et al. (2006) address this by adapting Auto-Regressive Moving Average (ARMA) models to a frequency-warped scale. By minimizing a cost function based on the warped reconstruction error, the authors formulate a matrix equation that uses standard least squares to robustly track vocal tract formants even in noisy environments.

System Identification and Dynamical Modeling

System identification is the art of discovering the mathematical equations of a dynamical system by observing its inputs and outputs. Parameter linearization serves as the bedrock for identifying physical constants across mechanical, biological, and electrochemical systems.

Experimental Data ----> Domain Transformation ----> Linear-in-Parameter ----> Least Squares ----> Estimated Physical
(Non-linear Dynamics)     (Logs, Filters, etc.)          Formulation             Solver             Constants

Physical and Biological Systems

In cellular biology, Wang et al. (2017) apply least squares system identification to Atomic Force Microscopy (AFM) data, minimizing the discrepancy between a dynamical model's predicted output and experimental force data to decouple and quantify cellular elasticity and viscosity. In mechatronics, Jiang et al. utilize offline least-squares algorithms on velocity and torque data to calibrate non-linear friction models for servo motors, ensuring control laws match the physical hardware. For electric vehicles, Wang et al. (2023) employ least squares to identify electrochemical parameters from internal reference electrodes, creating an online charging model that accurately prevents lithium plating.

Network and Group Dynamics

Linearization also scales to multi-agent systems. Anderson et al. model the collective behavior of dynamic agents (like herds or flocks) by formulating a second-order difference equation from trajectory data, solving a linear system to minimize future position prediction errors. In neuroscience, Bauer et al. reverse-engineer the functional interactions between brain networks using the non-linear Kuramoto model. By formulating the network's phase dynamics as a linear system of equations, the authors use least-squares optimization to directly solve for the hidden coupling parameters from fMRI data.

Fractional Differentiation Models

Systems with long-term memory effects are frequently modeled using fractional calculus (derivatives of non-integer order). Estimating these models is inherently non-linear because both the coefficients and the fractional orders are unknown. As detailed by Malti et al. (2006) and Malti et al. (2007), researchers simplify this problem by estimating a single "commensurate order" ($\gamma$), meaning all other fractional orders are treated as integer multiples of $\gamma$. By fixing or isolating this order, the equation-error models become linear in their remaining parameters, allowing standard least squares—often aided by fractional State Variable Filters (SVFs) to handle noise—to tractably solve the system.

Unlocking Statistical and Performance Guarantees

Beyond computational simplicity, the ultimate justification for parameter linearization is that it allows researchers to derive rigorous mathematical guarantees regarding sample complexity and error bounds—something rarely possible with non-linear optimization.

In control theory, when a system's internal states are hidden, engineers must design controllers based purely on input-output data. Boczar et al. apply least-squares estimation to input-output data modeled as a Finite Impulse Response (FIR) filter. Because the estimation problem is linear, the authors derive not only a point estimate of the system but also a high-probability bound on the estimation error, which is vital for designing robust controllers.

Expanding on this data-driven control paradigm, Zheng et al. establish an end-to-end sample complexity bound for learning a robust Linear Quadratic Gaussian (LQG) controller from a single trajectory. By applying Ordinary Least Squares to an FIR model, the authors derive non-asymptotic error bounds in the frequency domain, proving that the controller's performance degradation scales linearly with the model estimation error.

This ability to provide guarantees from limited data manifests in other complex settings:

  • Rough Paths: Leahy et al. prove that parameter estimation for rough differential equations driven by non-Brownian noise can be achieved by comparing the observed scaled quadratic variation of a single trajectory against theoretical noise coefficients. When parameters enter the model linearly, this comparison allows least-squares estimators to successfully recover system constants from high-frequency discrete observations.
  • Waveform Deconvolution: In spaceborne remote sensing, Wang et al. (2017) utilize a least squares fitting approach to correct distorted radar waveforms. By finding the pure waveform that, when convolved with a modeled point spread function, best matches the measured data, they solve an inverse problem linearly on the ground rather than relying on complex on-board hardware.
  • Structural Health Monitoring: Qi et al. measure the dynamic deflection of concrete beams under cyclic loading using a two-step least-squares approach. They first use spectral analysis to isolate the dominant frequency of a sinusoidal displacement function, and then utilize linear least squares to refine the remaining amplitude and phase coefficients simultaneously, achieving sub-millimetre accuracy from consumer-grade depth sensors.

Key Foundational Work

Computing Optical Flow with Physical Models of Brightness Variation (2000) — contributed 2 concept elements to this direction.
A Mathematical Model of the Vocabulary-Text Relation (1980) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_least_squares_parameter_estimation_57be1ba2,
  title = {least squares parameter estimation},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/least-squares-parameter-estimation/rdr_57be1ba228714578a78bf10773b03bbf},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_least_squares_parameter_estimation_57be1ba2_v1,
  title = {least squares parameter estimation},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/least-squares-parameter-estimation/rdr_57be1ba228714578a78bf10773b03bbf/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake