Abstract
Constraint Optimization Problems (COPs), ranging from factory scheduling to resource allocation, are fundamental to industrial efficiency. However, modeling these problems using Constraint Programming (CP) remains a high-barrier task requiring rare domain expertise to translate natural language requirements into formal declarative code. This paper introduces ConstraintLLM, a neuro-symbolic framework designed to automate industrial-level CP modeling. By combining a fine-tuned Large Language Model (LLM) with a novel constraint-aware retrieval system and iterative self-correction mechanisms, ConstraintLLM significantly outperforms general-purpose models like GPT-4o, achieving a 2x improvement in solving accuracy on complex industrial benchmarks.
1. Introduction
Constraint Programming (CP) is a powerful paradigm for solving combinatorial optimization problems. Unlike imperative programming, CP is declarative; it requires the user to define the logical structure and constraints of a problem rather than the specific steps to solve it. While highly expressive, this requirement creates a significant "modeling gap." Translating ambiguous, natural language descriptions of industrial needs into precise, executable code is error-prone and labor-intensive.
Existing AI-driven optimization tools often focus on simpler Operations Research (OR) models or linear programming. They frequently struggle with the deep logical structures and strict syntax required by symbolic solvers. ConstraintLLM addresses these challenges by integrating neural language processing with symbolic reasoning to bridge the gap between human intent and formal optimization.
2. Methodology
ConstraintLLM utilizes a multi-stage approach to master the nuances of CP modeling:
2.1 Multi-Instruction Supervised Fine-Tuning (SFT)
The framework is built upon the Qwen2.5-Coder-32B model, which underwent specialized fine-tuning on CP modeling tasks. This process trained the model not only to generate code but also to extract specific constraint types and perform self-diagnosis on potential errors.
2.2 Constraint-Aware Retrieval Module (CARM)
A core innovation of ConstraintLLM is the Constraint-Aware Retrieval Module (CARM). Traditional Retrieval-Augmented Generation (RAG) typically relies on semantic similarity—finding examples that sound like the query. In contrast, CARM identifies the "constraint profile" of a problem (e.g., identifying needs for AllDifferent or Cumulative constraints). It retrieves examples with identical logical structures, even if the subject matter differs entirely (e.g., using a nurse scheduling template to solve a factory sequencing problem).
2.3 Tree-of-Thoughts (ToT) Exploration
To handle the complexity of industrial problems, the system employs a Tree-of-Thoughts approach. Instead of a single linear generation, the model explores multiple modeling decision branches, evaluating the promise of each "thought" by testing it against sample cases before proceeding.
2.4 Iterative Self-Correction
ConstraintLLM implements a closed-loop neuro-symbolic system. When generated code is executed through symbolic solvers (such as Choco or ACE), any resulting error messages are fed back into the system. The framework then retrieves a "correction path" from a curated library of known fixes to repair the code iteratively.
3. Experimental Evaluation
3.1 The IndusCP Benchmark
To evaluate performance, the authors introduced IndusCP, the first industrial-level benchmark for CP modeling. While academic benchmarks like NL4OPT average approximately 4 constraints per problem, IndusCP features 140 tasks across domains like pathfinding and resource allocation, averaging over 240 constraints per problem. This represents a significant increase in complexity and a truer test of real-world utility.
3.2 Performance Results
ConstraintLLM demonstrated state-of-the-art performance across several metrics:
- Solving Accuracy (SA): On the IndusCP benchmark, ConstraintLLM achieved a 51.3% SA, nearly doubling the performance of GPT-4o (26.5%) and significantly outperforming the much larger DeepSeek-V3 (38.5%).
- Generalization: In zero-shot tests involving domains absent from the training data, the CARM module allowed the model to successfully generalize logical patterns and maintain high accuracy.
- Efficiency: The system is computationally viable for industrial use; the full ToT version requires approximately 7 minutes per problem, while a streamlined version maintains high accuracy with a 3-minute execution time.
4. Discussion and Insights
The success of ConstraintLLM highlights that for optimization tasks, logic is more important than lexicon. Structural similarity provides a more effective learning signal than semantic similarity. Furthermore, the neuro-symbolic loop—where the LLM handles natural language and the symbolic solver provides ground-truth feedback—proves to be a robust architecture for complex reasoning tasks.
4.1 Limitations and Future Work
Currently, the framework is optimized for the PyCSP3 library. Future iterations could expand to include other CP languages like MiniZinc. Additionally, while the 32B parameter model is highly effective, exploring the scalability of these techniques with larger base models or more diverse case libraries remains a priority for future research.
5. Conclusion
ConstraintLLM represents a significant step forward in automating industrial optimization. By prioritizing structural logic and iterative feedback, it provides a scalable solution for translating complex human requirements into rigorous mathematical models, effectively lowering the barrier to entry for Constraint Programming in industrial settings.
Summary updated