TL;DR
The authors introduce STEW (Shrinkage Toward Equal Weights), a regularization method that shrinks model weights toward each other rather than toward zero. By leveraging "feature directions"—simple domain knowledge about whether a variable is generally "good" or "bad"—STEW significantly outperforms Lasso and Ridge regression on small datasets and achieves state-of-the-art learning efficiency in Tetris.
What Problem Does This Paper Address?
In machine learning, small datasets are a minefield of high variance. Standard regularization techniques like the Lasso or Ridge regression combat this by shrinking weights toward zero, effectively assuming that features are either irrelevant or have small effects.
However, in many real-world scenarios, we possess "domain knowledge" that these models ignore. Specifically, we often know the direction of a feature: we know that more "holes" in a Tetris board are bad for the score, or that more "square footage" is good for a house price. The authors argue that when we know these directions, shrinking weights toward zero is counter-intuitive. Instead, we should anchor our model to the assumption that all relevant features contribute roughly equally until the data proves otherwise.
Methods
The core contribution is STEW (Shrinkage Toward Equal Weights). While traditional models minimize a loss function plus a penalty on the magnitude of weights ($\ell_q$ norm), STEW penalizes the differences between weights:
$$\mathcal{L}^{STEW}(\beta, \lambda, q) = \|y - X\beta\|_2^2 + \lambda \sum_{i<j} |\beta_i - \beta_j|^q$$
Key technical highlights include:
- Directability: The authors "direct" environments by recoding features so that all expected weights are positive.
- Closed-form Solution: For $q=2$, STEW becomes a Tikhonov regularization problem with a computationally efficient closed-form solution.
- M-learning: To apply this to Tetris, the authors propose a new reinforcement learning framework called M-learning. It uses multinomial logistic regression to learn an action-utility function based on short "rollouts" (simulated future moves), treating the choice of the best move as a classification problem.
Datasets
The paper evaluates STEW across three distinct types of data:
- Simulated Environments: Various synthetic datasets with different weight distributions and varying levels of sparsity (50% to 90% zero-weights).
- Real-World Benchmarks: 13 standard regression datasets, including the Rent dataset (predicting apartment prices) and the Diabetes dataset (disease progression).
- Tetris: A 10x10 board implementation using 8 classic features (e.g., landing height, number of holes, row transitions) to test sequential decision-making.
Key Insights and Ideas
- The "Equal Weights" Advantage: The authors draw on psychological research showing that "unit weighting" (giving all features a weight of 1) often outperforms complex statistical models when data is scarce. STEW provides a mathematical bridge between OLS (no bias, high variance) and Equal Weights (high bias, zero variance).
- Bias-Variance Tradeoff: The paper proves that the "Equal Weights" model has lower bias than the "Zero-model" (shrinking to zero) whenever features are correctly directed. This explains why STEW dominates in the "small $n$, small $p$" regime.
- Robustness to Wrong Guesses: Unlike the Non-Negative Lasso (NNLasso), which forces weights to be positive and can fail if a direction is guessed incorrectly, STEW is robust. If the "equal weights" assumption is wrong, STEW simply behaves more like standard OLS as it sees more data.
- Sparsity is the Enemy: STEW is not a variable selection tool. If an environment is extremely sparse (e.g., 90% of features are useless), Lasso still wins because STEW tries to keep all weights (even the useless ones) close to each other.
Key Results
- Tetris Efficiency: Using M-learning with the STEW penalty, the agent reached an average score of over 1,000 lines after only 3 training samples. Competing models required significantly more data to achieve the same proficiency.
- Small Data Dominance: On the "Rent" dataset and across the 13 real-world benchmarks, STEW (and the simple Equal Weights model) consistently outperformed Ridge and Lasso when training sets were small (e.g., $n < 50$).
- Directability Matters: The performance gap was most pronounced when feature directions were known. Even when directions were estimated from the training data itself, STEW remained competitive and robust.
Limitations and Future Work
- Lack of Sparsity: As noted, STEW struggles in high-sparsity environments because it lacks a mechanism to "zero out" irrelevant features. A potential two-stage model could use Lasso to select features, then STEW to regularize the remaining weights.
- M-learning Depth: While M-learning showed incredible promise in Tetris, its theoretical properties and behavior in more complex sequential environments require further exploration.
- Computational Scaling: While $q=2$ has a closed-form solution, the pairwise difference matrix grows with the number of features, which might pose challenges for extremely high-dimensional datasets.
Summary updated