Piecewise Constant and Binned Representations for Discontinuous Data

18 concepts12 papers1 proposal

Many foundational tools in machine learning and signal processing share a common assumption: the underlying world is smooth. Fourier analysis maps signals using continuous sine and cosine waves; generalized additive models often rely on smooth polynomial splines; and standard neural networks possess an inherent inductive bias toward low-frequency smoothness.

However, real-world data is frequently full of abrupt transitions, sharp edges, and sudden jumps. Whether it is a digital "on-off" switch in an industrial system, a strict threshold in a medical treatment protocol, a sharp contextual shift in natural language, or a sudden burst of financial volatility, data often changes instantly. When we force these jagged, discontinuous signals into smooth mathematical molds, our models break down.

A growing body of research across signal processing, time series forecasting, and tabular machine learning addresses this mismatch by replacing smooth curves with piecewise constant, step-like, or binned representations. By aligning the mathematical structure of the model with the discontinuous nature of the data, these techniques capture sudden shifts cleanly, efficiently, and without information loss.

The Failure of Smooth Bases

To understand why smooth representations fail, consider what happens when a global, continuous basis function tries to approximate a sharp step. To capture a sudden vertical jump, a Fourier series requires an infinite sum of higher and higher frequency sinusoids. Even then, it suffers from the Gibbs phenomenon—oscillatory ringing artifacts around the discontinuity that introduce significant smoothing errors.

As highlighted by Jiayang Su et al. in Multiscale Attention Wavelet Neural Operator for Capturing Steep Trajectories in Biochemical Systems, global basis functions cannot accurately represent sharp, localized jumps in non-linear trajectories—such as the sudden state switches that trigger cell division—without producing these smoothing errors.

This limitation spans multiple domains:

When a model's mathematical primitives cannot represent a cliff, it turns the cliff into a blurry hill.

Embracing Discontinuity: Step Functions and Wavelets

If a signal contains sharp transitions, the most intuitive solution is to build the model out of functions that already have sharp transitions.

This insight dates back to early pattern recognition. Lucien A. Gerardin et al. in Geometrical Pattern Feature Extraction by Projection on Haar Orthonormal Basis and the authors of Geometrical proposed projecting geometric patterns defined by sharp edges onto a Haar orthonormal basis. Because Haar basis functions are inherently step-like, they match the discrete transitions of the underlying data perfectly. This alignment prevents information loss and summarizes complex data using far fewer coefficients than continuous approximations would require.

Smooth Basis (e.g., Fourier)        Discontinuous Basis (e.g., Haar / Step)
         _.._                                         ______
       .'    `.                                      |      |
_..._.'        `._..._                  _____________|      |_____________
(Gibbs ringing & blurred edges)         (Perfect match for sharp transitions)

In time series analysis, Konstantinos Fokianos et al. apply a similar logic to non-stationary modeling in On Locally Dyadic Stationary Processes. By leveraging Walsh-Fourier analysis—which replaces smooth sinusoids with square waves—the authors constructed a theoretical framework tailored for digital communications and engineering systems characterized by discrete "on-off" states.

Bins and Staircases in Modern Machine Learning

The power of step-like representations explains a long-standing mystery in tabular machine learning: why decision trees still outperform deep learning on standard tables. As Leo Grinsztajn et al. demonstrate in Why do tree-based models still outperform deep learning on typical tabular data?, tabular data frequently features irregular patterns where target variables change abruptly. Tree-based models excel because they naturally utilize piecewise constant approximations, whereas neural networks biased toward low-frequency smoothness struggle to isolate these jagged relationships.

To bring this advantage to interpretable machine learning, researchers are redesigning Generalized Additive Models (GAMs). Traditionally, GAMs use smooth splines to map how individual features affect a prediction. However, splines blur sudden breakpoints caused by real-world thresholds (like age limits or policy cutoffs).

To fix this, Rahul Mazumder et al. introduced the FAST framework in FAST: An Optimization Framework for Fast Additive Segmentation in Transparent ML, which represents feature effects as piecewise constant staircase functions. This approach explicitly preserves sharp discontinuities and sudden shifts in the data distribution, avoiding the obscuring smoothing effects of splines.

        Spline Representation                    Staircase / Binned Representation
                 /\                                             ______
               /    \                                          |      |
   ___________/      \___________               _______________|      |_______________
        (Blurs the threshold)                        (Preserves the exact breakpoint)

Similarly, the GAMformer architecture developed by Andreas C Mueller et al. in GAMformer: Exploring In-Context Learning for Generalized Additive Models and GAMformer: In-Context Learning for Generalized Additive Models abandons continuous weights entirely. Instead, it represents univariate shape functions non-parametrically using discrete bins.

By encoding features into discrete bins, the model can capture non-linear jumps and critical thresholds—such as specific medical treatment boundaries—without imposing an artificial smoothing constraint. Even when dealing with complex, non-Euclidean data evolving over time, applying total variation regularization, as shown by Zhenhua Lin et al. in Total variation regularized Fréchet regression for metric-space valued data, allows regression models to isolate sudden shifts and "jumps" rather than smoothing over the underlying dynamics.

Ultimately, choosing a representation is about matching the geometry of the model to the geometry of the problem. When the world changes in an instant, step functions and discrete bins provide the mathematical language necessary to capture those moments precisely.

Key Foundational Work

Geometrical Pattern Feature Extraction by Projection on Haar Orthonormal Basis (1969) — contributed 3 concept elements to this direction.
Geometrical Pattern Feature Extraction by Projection on Haar Orthonormal Basis (1969) — contributed 2 concept elements to this direction.
On Locally Dyadic Stationary Processes (2017) — contributed 1 concept elements to this direction.
History & citationVersion 2 · 2026-10-04

Current · Permanent link

  1. Version 2 (latest) · 2026-10-04 · 12 papers
  2. Version 1 · 2026-10-04 · 16 papers
Cite this direction

Latest version (a living page):

@misc{lacuna_binned_shape_function_representation_eeb2d84a,
  title = {binned shape function representation},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/binned-shape-function-representation/rdr_eeb2d84a5eea4c809e5bbd91f5034878},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 2, frozen text):

@misc{lacuna_binned_shape_function_representation_eeb2d84a_v2,
  title = {binned shape function representation},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/binned-shape-function-representation/rdr_eeb2d84a5eea4c809e5bbd91f5034878/v/2},
  note = {Version 2, frozen 2026-10-04. Accessed 2026-11-19},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake