Sparsity Under Simplex Constraints

You are reading version 1 of this direction, frozen 2026-10-04. The latest version is version 2.

Version 1 of 217 papers27 conceptsfrozen 2026-10-04

In the standard machine learning toolkit, the $L_1$ norm (Lasso) is the go-to mechanism for inducing sparsity. By penalizing the sum of absolute values of a weight vector, we encourage the optimizer to drive unimportant features to zero. However, a significant body of research reveals a fundamental "blind spot" for this technique: when a model is subject to a simplex constraint—where weights must be non-negative and sum to a fixed constant—the $L_1$ norm becomes mathematically useless.

The Invariance Problem

The core issue is a conflict between the geometry of the simplex and the mechanics of $L_1$ regularization. In domains like portfolio management Timothy Hospedales, probability distribution learning Jacky Y. Zhang, and physical systems like MRI fiber reconstruction Alessandro Daducci, the variables are constrained to sum to one.

As noted by Ping Li and Martin Slawski in Methods for Sparse and Low-Rank Recovery under Simplex Constraints, if all weights $w_i \ge 0$ and $\sum w_i = 1$, then the $L_1$ norm $\|w\|_1$ is always exactly $1$. Because the penalty is constant across every possible solution on the simplex, it provides no gradient and no pressure toward sparsity. This renders traditional convex relaxation techniques ineffective for distinguishing between a dense distribution and a sparse one.

Beyond Convexity

Since the standard convex proxy fails, researchers have turned to non-convex alternatives to recover sparse signals in these constrained environments. The research direction focuses on three primary strategies:

  1. Entropic Regularization: Quentin Jacquet and colleagues demonstrate in Entropic Lower Bound of Cardinality for Sparse Optimization that Shannon entropy can act as a sensitive measure of sparsity on the simplex. Unlike the $L_1$ norm, entropy varies significantly depending on how "spread out" the weights are, allowing solvers to push the system toward a few dominant peaks.
  2. Non-Convex Penalties: To break the invariance of the $L_1$ norm, researchers like Suchi Saria and Daphne Koller argue against "premature envelopment"—the reflexive use of convex proxies. Instead, they suggest using smoothed $L_0$ approximations or negative $L_2$ norms, which remain effective even when the sum of weights is fixed.
  3. Stochastic and Greedy Projections: In high-dimensional tasks like index tracking, Yu Zheng and Timothy Hospedales utilize stochastic neural networks and Gumbel-Softmax tricks to learn cardinality-constrained portfolios. Meanwhile, Anastasios Kyrillidis and Volkan Cevher have developed specialized algorithms for Sparse projections onto the simplex, proving that while the combined constraint is complex, exact and efficient projections are possible.

Why It Matters

This shift from $L_1$ to more sophisticated sparsity measures is not just theoretical; it solves practical "mathematical inconsistencies" in applied sciences. In Diffusion MRI, for instance, Alessandro Daducci showed that switching to an $L_0$ prior allows for faster scans by correctly balancing the physical requirement that fiber volumes sum to one with the anatomical reality that only a few fibers cross at any given point. By recognizing that the simplex makes the $L_1$ norm "invisible," these researchers have unlocked more accurate modeling in finance, physics, and natural language processing.

Papers in this version

History & citationVersion 2 · 2026-10-04

Current · Permanent link

  1. Version 2 (latest) · 2026-10-04 · 18 papers
  2. Version 1 · 2026-10-04 · 17 papers
Cite this direction

Latest version (a living page):

@misc{lacuna_simplex_constrained_sparsity_c650f276,
  title = {simplex-constrained sparsity},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/simplex-constrained-sparsity/rdr_c650f27686c341bbbe1172cf621aa2ed},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-22},
}

This exact version (version 1, frozen text):

@misc{lacuna_sparsity_under_simplex_constraints_c650f276_v1,
  title = {Sparsity Under Simplex Constraints},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/sparsity-under-simplex-constraints/rdr_c650f27686c341bbbe1172cf621aa2ed/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-22},
}