Semi Nonnegative Matrix Factorization for Mixed Sign Data

72 concepts61 papers1 proposal

In machine learning and signal processing, Non-negative Matrix Factorization (NMF) is celebrated for its ability to find "parts-based" representations. By decomposing a data matrix $X$ into two purely non-negative matrices—a basis matrix $W$ and an activation matrix $H$ ($X \approx WH$)—NMF ensures that features can only be added together, never subtracted. This mathematical constraint mirrors how we intuitively understand the world: images are composed of additive light pixels, documents are bags of positive word counts, and audio spectrograms are combinations of energy frequencies.

However, this strict non-negativity constraint is also NMF's greatest limitation. Real-world data frequently breaks the positivity barrier. Common preprocessing steps like mean centering or kernel transformations routinely introduce negative values, as noted by Zhou et al.. Physical signals, such as neural spikes or acoustic waveforms, naturally oscillate above and below zero (Le Roux et al.). Furthermore, many networks inherently contain negative relationships, such as signed drug-drug interactions that exhibit degressive or inhibitory effects (Yu et al.).

When standard NMF is forced to process mixed-sign data, it fails. It cannot represent negative entries without artificial data truncation or shifting, which distorts the latent feature space and hinders the discovery of descriptive embeddings (Liu et al.). To resolve this conflict, researchers have developed frameworks that bypass strict non-negativity while preserving the core interpretability of matrix decompositions.

The Intuition of Semi-NMF

How can we accommodate negative values without losing the clustering power of a parts-based representation? The answer lies in Semi-Non-Negative Matrix Factorization (Semi-NMF).

In Semi-NMF, we relax the constraints on the input matrix $X$ and the basis matrix $W$, allowing them to contain both positive and negative values. Crucially, however, we keep the activation matrix $H$ strictly non-negative ($H \ge 0$).

$$\underbrace{X}_{\text{Mixed-Sign}} \approx \underbrace{W}_{\text{Mixed-Sign}} \times \underbrace{H}_{\text{Strictly Non-Negative}}$$

To understand why this works, think of $W$ as a dictionary of latent templates or cluster centers, and $H$ as the recipe for reconstructing the data points from those templates.

  • Because $W$ can have mixed signs, the latent templates are free to live anywhere in the continuous feature space. They can capture oscillations, negative correlations, and centered data points perfectly.
  • Because $H$ remains strictly non-negative, every data point is still represented as a purely additive combination of these templates.

This asymmetry preserves the clustering interpretation: a high value in $H_{ij}$ means that data point $j$ strongly belongs to template $i$. As Yu et al. demonstrate, this allows models to capture complex directional differences in signed networks while keeping the latent factors highly interpretable.

Bypassing Positivity via Convex Hulls and Component Splitting

Beyond Semi-NMF, researchers have engineered alternative geometric and optimization mechanisms to handle mixed-sign data without losing structural clarity.

Convex Hull Anchoring

Instead of allowing the basis vectors to wander anywhere in a signed space, Vaz et al. propose restricting the basis vectors to be convex combinations of actual data points on the dataset's outer boundaries (the convex hull).

Because these basis vectors are anchored directly to real data points, the factorization process naturally inherits the data's native sign structure. If the data contains negative values, the basis vectors accommodate them inherently. This eliminates the strict non-negativity requirement of traditional dictionary learning while maintaining a highly interpretable, parts-based representation of multivariate time-series signals.

Positive and Negative Component Splitting

Another powerful strategy addresses the optimization process directly. Standard matrix factorization algorithms often struggle to maintain mathematical stability when signs alternate. To circumvent this, Yuheng Jia demonstrates that optimization frameworks can explicitly separate mixed-sign features into distinct positive and negative components:

$$X = X^+ - X^-$$

By treating these components independently during graph construction and update steps, models can bypass traditional input sign restrictions. This allows graph-based clustering algorithms to maintain high performance on datasets with mixed-sign features without requiring destructive data transformations.

Rewriting the Update Rules for Signed Data

Standard NMF relies on multiplicative update rules, which are popular because they inherently preserve non-negativity—if you start with positive matrices, multiplying them by positive gradients ensures they stay positive. However, these rules break down mathematically when the input data contains negative entries.

To overcome this, researchers have redesigned the underlying optimization mathematics. For instance, Needell et al. formulated specialized iterative update rules for unsupervised feature selection that accommodate both positive and negative entries in input and kernel matrices. This flexibility allows for broader application to datasets where data centering or specific kernel types produce negative entries.

Similarly, Han et al. show that utilizing mixed-sign target matrices in regression-based feature selection offers superior flexibility over traditional NMF, allowing algorithms to accurately model real-world data distributions that do not conform to strictly positive representations.

Even in symmetric matrix factorization—where matrices are typically expected to be positive semi-definite—Jiashun Jin proved that non-negative factors can be successfully recovered even when the matrix contains negative eigenvalues, provided the number of negative eigenvalues is bounded. This theoretical breakthrough confirms that positive semi-definiteness is not a strict prerequisite for obtaining interpretable, non-negative factors in matrix decomposition, opening the door for NMF techniques to be applied safely to a much wider array of real-world systems.

Key Foundational Work

Nonnegative Sparse PCA (2006) — contributed 1 concept elements to this direction.
Compressed Counting (2008) — contributed 1 concept elements to this direction.
On the system of rational difference equations xn+1=f(xn,yn−k),yn+1=f(yn,xn−k) (2006) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_non_negativity_constraint_c966d36e,
  title = {non-negativity constraint},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/non-negativity-constraint/rdr_c966d36e65fc4042b1ede5e4eb387c3d},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_non_negativity_constraint_c966d36e_v1,
  title = {non-negativity constraint},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/non-negativity-constraint/rdr_c966d36e65fc4042b1ede5e4eb387c3d/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake