Pyglmnet: Python implementation of elastic-net regularized generalized linear models

Mainak Jas, Titipat Achakulvisut, Aid Idrizović, Daniel E. Acuña, Matthew Antalek, Vinícius Marques, Tommy Odland, Ravi Garg, Mayank Agrawal, Yu Umegaki, Peter Foley, Hugo Fernandes, Drew Harris, Beibin Li, Olivier Pieters, Scott Otterson, Giovanni De Toni, Chris C. Rodgers, Eva L. Dyer, Matti S. Hämäläinen, Konrad P. Körding, Pavan RamkumarJ. Open Source Softw 2020 · DOI · DBLP

Introduction

Generalized Linear Models (GLMs) are fundamental tools in statistical modeling, widely utilized across fields ranging from finance to neuroscience due to their interpretability and computational efficiency. However, as datasets grow in complexity and dimensionality, standard GLMs become prone to overfitting. To mitigate this, researchers employ regularization techniques—such as L1 (Lasso) and L2 (Ridge) penalties—to constrain model parameters.

Despite the importance of regularized GLMs, the Python ecosystem was historically fragmented. While statsmodels provided a variety of noise distributions, it lacked robust regularization. Conversely, scikit-learn offered excellent regularization but was limited to Gaussian and Binomial distributions. Other libraries like lightning supported advanced penalties but only for a narrow subset of models. pyglmnet was developed to bridge this gap, providing a unified, scikit-learn-compatible interface for regularized GLMs.

Technical Implementation and Features

The authors implemented the core algorithm described by Friedman, Hastie, and Tibshirani (the creators of the original R glmnet) in pure Python. This approach ensures the library is highly extensible and easily integrated into existing Python workflows.

Key technical features include:

  • Elastic-net Regularization: A hybrid penalty combining L1 (for sparsity and feature selection) and L2 (to prevent coefficient explosion).
  • Broad Distribution Support: Native handling of Gaussian, Binomial, Poisson (including a "softplus" variant), Probit, and Gamma distributions.
  • Advanced Penalties: Support for Group Lasso (penalizing groups of variables) and Tikhonov regularization (Generalized L2).
  • Scikit-learn API: Full implementation of .fit() and .predict() methods, allowing for seamless use in standard machine learning pipelines, cross-validation, and grid searches.

Feature Comparison and Results

The primary contribution of pyglmnet is its comprehensive coverage of both distributions and regularization types within a single package. This reduces the cognitive load on researchers who previously had to navigate multiple libraries with inconsistent syntaxes.

Table 1: Feature comparison of pyglmnet against other popular statistical and machine learning libraries.

As illustrated in the comparison above, pyglmnet is unique among its Python peers for supporting Poisson (softplus), Probit, and Generalized L2 (Tikhonov) regularization simultaneously. It achieves feature parity with the industry-standard R glmnet package while remaining native to the Python environment.

Practical Application: Community and Crime

To demonstrate the library's utility in high-dimensional research, the authors applied pyglmnet to the Community and Crime dataset from the UCI Machine Learning Repository. The task involved predicting the incidence of violent crime per capita across various U.S. counties using 128 demographic attributes. By employing a binomial-distributed GLM with elastic-net regularization, the authors showcased how the library effectively handles real-world, high-dimensional social science data while maintaining model interpretability.

Conclusion

pyglmnet serves as a "one-stop shop" for regularized GLMs in Python. By providing a pure-Python implementation, the authors have created a "hackable" and extensible tool that invites community contributions. The library has already seen adoption in peer-reviewed publications across neuroscience and machine learning, establishing it as a stable and rigorous tool for the modern data science stack.

Summary updated

BibTeX
@article{jas2020pyglmnet,
  author = {Mainak Jas and Titipat Achakulvisut and Aid Idrizović and Daniel E. Acuña and Matthew Antalek and Vinícius Marques and Tommy Odland and Ravi Garg and Mayank Agrawal and Yu Umegaki and Peter Foley and Hugo Fernandes and Drew Harris and Beibin Li and Olivier Pieters and Scott Otterson and Giovanni De Toni and Chris C. Rodgers and Eva L. Dyer and Matti S. Hämäläinen and Konrad P. Körding and Pavan Ramkumar and Vinícius Abrão da Silva Marques and P. J. Foley and Matti Hämäläinen and Chris Rodgers and Konrad Kording},
  title = {{Pyglmnet: Python implementation of elastic-net regularized generalized linear models}},
  journal = {J. Open Source Softw.},
  year = {2020},
  doi = {10.21105/joss.01959},
}
Figure