Gradient Descent in the Complex Domain with Wirtinger Calculus

You are reading version 1 of this direction, frozen 2026-10-04. The latest version is version 2.

Version 1 of 2101 papers137 conceptsfrozen 2026-10-04

In fields ranging from MRI reconstruction and radar signal processing to quantum machine learning, data is inherently complex-valued. While it is tempting to simply split complex numbers into their real and imaginary parts and treat them as two separate channels, this approach often discards the underlying geometry and phase relationships of the signal.

To truly optimize models in the complex domain, researchers have turned to Wirtinger calculus. This mathematical framework allows us to perform gradient descent on complex-valued parameters even when our objective functions—like the ubiquitous Mean Squared Error—violate the fundamental rules of standard complex analysis.

The Holomorphic Hurdle

In standard complex analysis, a function is only differentiable if it is holomorphic, meaning it satisfies the Cauchy-Riemann equations. This is a remarkably strict requirement. For a function $f(z)$ to be holomorphic, it must be locally linear in the complex sense.

The problem for machine learning is simple: any real-valued function of a complex variable (that isn't a constant) is non-holomorphic.

Because loss functions must output a real-valued scalar to be minimized, they are inherently "broken" from the perspective of traditional complex differentiation. As Pantelis Bouboulis notes in Extension of Wirtinger's Calculus to Reproducing Kernel Hilbert Spaces, this restrictiveness makes standard complex analysis ill-suited for optimizing the error functions used in adaptive filtering and neural networks.

The Wirtinger Workaround

Wirtinger calculus provides a elegant "hack" to bypass this restriction. Instead of trying to find a single complex derivative, we treat the complex variable $z$ and its conjugate $z^*$ as independent variables.

Under this framework, we define two partial derivatives:

  1. The $\mathbb{R}$-derivative (or Wirtinger derivative) $\frac{\partial f}{\partial z}$, which treats $z^*$ as a constant.
  2. The conjugate $\mathbb{R}$-derivative $\frac{\partial f}{\partial z^*}$, which treats $z$ as a constant.

For a real-valued loss function $L(z)$, the direction of steepest descent is simply the gradient with respect to the conjugate, $\nabla_z L = \frac{\partial L}{\partial z^*}$. This realization allows us to apply standard optimization techniques—like SGD or Adam—directly to complex weights. Alexey Minin, Alois Knoll, and Hans-Georg Zimmermann demonstrate in Complex Valued Recurrent Neural Network: From Architecture to Training that this framework enables backpropagation to function exactly like its real-valued counterpart, bypassing the need for Cauchy-Riemann conditions.

Why Complex Optimization Works

By maintaining the complex structure during optimization, models can capture physical phenomena more efficiently than real-valued networks (RVNNs).

  • Phase-Sensitivity: In tasks like speech enhancement or MRI, the phase of the signal carries critical information. Moritz Blumenthal and Martin Uecker show in their work on the BART toolbox that using Wirtinger calculus to train networks on phase-sensitive data outperforms methods that simply split complex numbers into real and imaginary channels.
  • Parameter Efficiency: Complex-valued gates and unitary transitions can solve long-term dependency problems with fewer parameters. Moritz Wolter and Angela Yao found that their Complex Gated RNN achieved state-of-the-art performance on motion prediction using 50% fewer parameters than standard models.
  • Non-Circular Signals: Real-world signals are often "non-circular," meaning their statistical properties are not rotationally invariant in the complex plane. Soroush Javidi and Andrzej Cichocki leverage Wirtinger calculus in K-cBSE to isolate EEG artifacts that standard algorithms miss by explicitly accounting for this non-circularity.

The Complexity Tax

While Wirtinger-based optimization is mathematically powerful, it is not a free lunch. Theoretical analysis by Zhi-Hua Zhou and colleagues in Complex-valued Neurons Can Learn More but Slower suggests a fundamental trade-off: while complex-valued neurons can represent more intricate, phase-dependent functions, they converge at a polynomial rate that is often slower than real-valued neurons on simple tasks.

Furthermore, many common deep learning components, such as the Softmax function, are non-analytic. As Yihong Dong and Songtao Lu point out in their work on Signal Transformers, this necessitates specialized mathematical frameworks to ensure that gradients remain well-defined throughout the network.

Applications Beyond Neural Networks

The utility of Wirtinger calculus extends far beyond deep learning. It has become a cornerstone for:

By treating $z$ and $z^*$ as independent, Wirtinger calculus provides the necessary bridge between the elegant world of complex numbers and the practical requirements of gradient-based optimization.

Papers in this version

Showing 60 of 101 papers.

History & citationVersion 2 · 2026-10-04

Current · Permanent link

  1. Version 2 (latest) · 2026-10-04 · 63 papers
  2. Version 1 · 2026-10-04 · 101 papers
Cite this direction

Latest version (a living page):

@misc{lacuna_wirtinger_calculus_optimization_0500a20c,
  title = {Wirtinger calculus optimization},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/wirtinger-calculus-optimization/rdr_0500a20c85d744b5ae45d3ac3a506724},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_gradient_descent_in_the_complex_domain_w_0500a20c_v1,
  title = {Gradient Descent in the Complex Domain with Wirtinger Calculus},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/gradient-descent-in-the-complex-domain-with-wirtinger-calculus/rdr_0500a20c85d744b5ae45d3ac3a506724/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}