In traditional signal processing, we treat multi-channel data—such as the Red, Green, and Blue channels of a color image—as independent streams or concatenated vectors. However, this approach often ignores the intrinsic correlations between channels. Hypercomplex algebras, specifically quaternions (4D) and octonions (8D), offer a way to represent these channels as a single mathematical entity, preserving inter-dimensional structures.
The challenge is that these algebras do not follow the familiar rules of high-school math. Quaternions are non-commutative ($ab \neq ba$), and octonions are non-associative ($a(bc) \neq (ab)c$). These are not just abstract algebraic quirks; they fundamentally break the calculus and linear algebra tools that modern machine learning relies on.
The Failure of Standard Calculus
Most optimization algorithms, from simple gradient descent to complex neural network backpropagation, rely on the chain rule and the existence of derivatives. In the quaternion domain, standard calculus fails because real-valued cost functions (like Mean Squared Error) are "non-analytic"—they cannot be differentiated using traditional complex-variable methods.
To bridge this gap, Dongpo Xu and Yili Xia and Dongpo Xu developed the Generalized Hamilton-Real (GHR) calculus. This framework provides a way to calculate gradients and Hessians directly within the quaternion field without breaking the variables into four separate real numbers. By establishing these native derivatives, researchers can now apply powerful optimization techniques like Newton’s method and Wirtinger Flow (Michael Ng et al.) to hypercomplex signals, ensuring that local optima are indeed global (Shuning Sun et al.).
The Spectral Norm and Convolution Paradox
In standard signal processing, the Convolution Theorem is a cornerstone: it allows us to perform expensive convolutions by simply multiplying signals in the frequency domain. However, as Giorgos Sfikas and George Retsinas explain, the non-commutative nature of quaternions breaks this relationship. You cannot simply multiply two Quaternion Fourier Transforms (QFT) to get the transform of their convolution.
This lack of commutativity also complicates matrix operations. For instance, Min Xiang et al. demonstrate that a single unitary transform is mathematically incapable of diagonalizing two quaternion covariance matrices simultaneously unless they happen to commute—a rare occurrence in real-world data like EEG signals. To solve this, researchers have had to develop "approximate" uncorrelating transforms that accept a degree of error to achieve the decorrelation necessary for feature extraction.
Scaling to Octonions and the Associativity Barrier
When moving to 8-channel data (like multispectral satellite imagery), researchers turn to octonions. Here, the mathematical difficulty spikes again: octonions lose the associative property. This means the order in which you group operations matters ($a(bc)$ is not the same as $(ab)c$).
As Kumar Vijay Mishra et al. and Roman Jacome et al. point out, the loss of associativity prevents the use of the standard chain rule entirely. Current workarounds involve mapping these 8D operations into "pseudo-real-matrix" spaces. While this allows for gradient-based optimization, it adds significant computational complexity and risks losing the very theoretical benefits—such as sample efficiency and holistic channel representation—that hypercomplex algebras were intended to provide.
Why It Works: Native vs. Real-Valued Processing
Despite these hurdles, the reason this direction persists is performance. By respecting the non-commutative constraints rather than bypassing them, models like QDIP-RED (Qinghua Zhang et al.) achieve superior color image restoration. Treating the signal as a quaternion prevents the "color bleeding" and artifacts that occur when channels are processed independently.
The shift in the field, led by tools like NLCertify (Victor Magron) and new existence theorems for Quaternion Reproducing Kernel Hilbert Spaces (Felipe A. Tobar), is toward building a "native" hypercomplex toolkit. By deriving new versions of the Fourier Transform (Guanlei Xu) and tensor decompositions (Jifei Miao et al.) that account for non-commutativity from the start, the field is moving away from cumbersome real-valued workarounds toward a more elegant, high-dimensional signal theory.
Key Foundational Work
Quaternion Reproducing Kernel Hilbert Spaces: Existence and Uniqueness Conditions (2014) — contributed 1 concept elements to this direction.
Prediction of wide-sense stationary quaternion random signals (2013) — contributed 1 concept elements to this direction.
NLCertify: A Tool for Formal Nonlinear Optimization (2014) — contributed 1 concept elements to this direction.