Abstract
GENO (GENeric Optimization) is a framework designed to bridge the gap between easy-to-use modeling languages and high-performance, hand-coded solvers. By allowing users to define optimization problems in a simple, MATLAB-like syntax, GENO automatically generates Python solvers that are as fast as—and often faster than—specialized, state-of-the-art implementations.
Introduction
Most classical machine learning tasks, such as Support Vector Machines (SVMs), elastic net, or matrix factorization, are fundamentally optimization problems. Currently, researchers and practitioners face a difficult trade-off when implementing these models. Standard toolboxes like scikit-learn are fast but rigid, making it difficult to add custom constraints. Conversely, modeling languages like CVXPY or JuMP offer flexibility but often suffer from significant computational overhead because they transform specific problem instances into standard forms, leading to representational bloat. The third option—hand-writing solvers—is efficient but requires weeks of expert design and debugging.
GENO aims to resolve this tension by providing the flexibility of a modeling language with the performance of a hand-engineered solver.
The GENO Framework
The GENO pipeline consists of two tightly coupled components: a high-level modeling language and an automatic solver generation engine.
Modeling Language
Users specify their objective functions and constraints using vectorized linear algebra expressions, similar to MATLAB or NumPy. A key advantage of GENO is that users do not need to specify dimensions or ensure the problem is convex. As shown in the examples below, the syntax is intuitive and handles various constraints seamlessly.
Automatic Solver Generation
Unlike traditional modeling stacks that transform a specific problem instance, GENO transforms the entire class of the problem into a general standard form. This "class-level transformation" is the core innovation that avoids representational inefficiency. The complexity of the generated solver does not grow with the size of the data.
The generation process involves:
- Automatic Differentiation: A specialized matrix and tensor calculus algorithm computes gradients automatically.
- Optimization Engine: The core solver utilizes the L-BFGS-B algorithm, a quasi-Newton method suited for large-scale problems. To handle non-smooth functions and constraints, GENO extends this with an Augmented Lagrangian approach.
The result is a tailored Python solver produced in just a few milliseconds.
Performance Evaluation
GENO's performance has been evaluated against a variety of baselines, including standard libraries (LIBSVM, LIBLINEAR), recent research-specific solvers from ICML and NeurIPS, and modeling stacks like CVXPY paired with commercial solvers (Gurobi, Mosek).
Key results include:
- Performance Parity: GENO-generated solvers match the efficiency of well-engineered, specialized solvers like LIBSVM.
- Superiority over Specialized Research: In many instances, GENO's automatically generated solvers outperformed specialized solvers published at top-tier conferences by significant margins.
- Massive Speedups: Compared to traditional modeling approaches (CVXPY + Gurobi), GENO is often orders of magnitude faster due to its efficient problem representation.
Conclusion and Future Work
GENO provides a powerful tool for rapid prototyping and deployment of machine learning models. While it currently guarantees only local optima for non-convex problems and is limited to its implemented syntax, its performance is already competitive with pure C++ implementations. Future work involves extending the underlying matrix/tensor calculus to handle even more complex mathematical structures and expanding the library of supported operators.
Summary updated