ConstraintLLM: A Neuro-Symbolic Framework for Industrial-Level Constraint Programming

Weichun Shi, Minghao Liu, Wanting Zhang, Langchen Shi, Fuqi Jia, Feifei Ma, Jian Zhang

Abstract

Constraint Optimization Problems (COPs), ranging from factory scheduling to resource allocation, are fundamental to industrial efficiency. However, modeling these problems using Constraint Programming (CP) remains a high-barrier task requiring rare domain expertise to translate natural language requirements into formal declarative code. This paper introduces ConstraintLLM, a neuro-symbolic framework designed to automate industrial-level CP modeling. By combining a fine-tuned Large Language Model (LLM) with a novel constraint-aware retrieval system and iterative self-correction mechanisms, ConstraintLLM significantly outperforms general-purpose models like GPT-4o, achieving a 2x improvement in solving accuracy on complex industrial benchmarks.

1. Introduction

Constraint Programming (CP) is a powerful paradigm for solving combinatorial optimization problems. Unlike imperative programming, CP is declarative; it requires the user to define the logical structure and constraints of a problem rather than the specific steps to solve it. While highly expressive, this requirement creates a significant "modeling gap." Translating ambiguous, natural language descriptions of industrial needs into precise, executable code is error-prone and labor-intensive.

Existing AI-driven optimization tools often focus on simpler Operations Research (OR) models or linear programming. They frequently struggle with the deep logical structures and strict syntax required by symbolic solvers. ConstraintLLM addresses these challenges by integrating neural language processing with symbolic reasoning to bridge the gap between human intent and formal optimization.

2. Methodology

ConstraintLLM utilizes a multi-stage approach to master the nuances of CP modeling:

2.1 Multi-Instruction Supervised Fine-Tuning (SFT)

The framework is built upon the Qwen2.5-Coder-32B model, which underwent specialized fine-tuning on CP modeling tasks. This process trained the model not only to generate code but also to extract specific constraint types and perform self-diagnosis on potential errors.

2.2 Constraint-Aware Retrieval Module (CARM)

A core innovation of ConstraintLLM is the Constraint-Aware Retrieval Module (CARM). Traditional Retrieval-Augmented Generation (RAG) typically relies on semantic similarity—finding examples that sound like the query. In contrast, CARM identifies the "constraint profile" of a problem (e.g., identifying needs for AllDifferent or Cumulative constraints). It retrieves examples with identical logical structures, even if the subject matter differs entirely (e.g., using a nurse scheduling template to solve a factory sequencing problem).

2.3 Tree-of-Thoughts (ToT) Exploration

To handle the complexity of industrial problems, the system employs a Tree-of-Thoughts approach. Instead of a single linear generation, the model explores multiple modeling decision branches, evaluating the promise of each "thought" by testing it against sample cases before proceeding.

2.4 Iterative Self-Correction

ConstraintLLM implements a closed-loop neuro-symbolic system. When generated code is executed through symbolic solvers (such as Choco or ACE), any resulting error messages are fed back into the system. The framework then retrieves a "correction path" from a curated library of known fixes to repair the code iteratively.

3. Experimental Evaluation

3.1 The IndusCP Benchmark

To evaluate performance, the authors introduced IndusCP, the first industrial-level benchmark for CP modeling. While academic benchmarks like NL4OPT average approximately 4 constraints per problem, IndusCP features 140 tasks across domains like pathfinding and resource allocation, averaging over 240 constraints per problem. This represents a significant increase in complexity and a truer test of real-world utility.

3.2 Performance Results

ConstraintLLM demonstrated state-of-the-art performance across several metrics:

  • Solving Accuracy (SA): On the IndusCP benchmark, ConstraintLLM achieved a 51.3% SA, nearly doubling the performance of GPT-4o (26.5%) and significantly outperforming the much larger DeepSeek-V3 (38.5%).
  • Generalization: In zero-shot tests involving domains absent from the training data, the CARM module allowed the model to successfully generalize logical patterns and maintain high accuracy.
  • Efficiency: The system is computationally viable for industrial use; the full ToT version requires approximately 7 minutes per problem, while a streamlined version maintains high accuracy with a 3-minute execution time.

4. Discussion and Insights

The success of ConstraintLLM highlights that for optimization tasks, logic is more important than lexicon. Structural similarity provides a more effective learning signal than semantic similarity. Furthermore, the neuro-symbolic loop—where the LLM handles natural language and the symbolic solver provides ground-truth feedback—proves to be a robust architecture for complex reasoning tasks.

4.1 Limitations and Future Work

Currently, the framework is optimized for the PyCSP3 library. Future iterations could expand to include other CP languages like MiniZinc. Additionally, while the 32B parameter model is highly effective, exploring the scalability of these techniques with larger base models or more diverse case libraries remains a priority for future research.

5. Conclusion

ConstraintLLM represents a significant step forward in automating industrial optimization. By prioritizing structural logic and iterative feedback, it provides a scalable solution for translating complex human requirements into rigorous mathematical models, effectively lowering the barrier to entry for Constraint Programming in industrial settings.

Summary updated

Abstract

Constraint programming (CP) is a crucial technology for solving real-world constraint optimization problems (COPs), with the advantages of rich modeling semantics and high solving efficiency.Using large language models (LLMs) to generate formal modeling automatically for COPs is becoming a promising approach, which aims to build trustworthy neuro-symbolic AI with the help of symbolic solvers.However, CP has received less attention compared to works based on operations research (OR) models.We introduce ConstraintLLM, the first LLM specifically designed for CP modeling, which is trained on an open-source LLM with multiinstruction supervised fine-tuning.We propose the Constraint-Aware Retrieval Module (CARM) to increase the in-context learning capabilities, which is integrated in a Tree-of-Thoughts (ToT) framework with guided selfcorrection mechanism.Moreover, we construct and release IndusCP, the first industriallevel benchmark for CP modeling, which contains 140 challenging tasks from various domains.Our experiments demonstrate that ConstraintLLM achieves state-of-the-art solving accuracy across multiple benchmarks and outperforms the baselines by 2x on the new IndusCP benchmark.Code and data are available at:
BibTeX
@misc{shi2025constraintllm,
  author = {Weichun Shi and Minghao Liu and Wanting Zhang and Langchen Shi and Fuqi Jia and Feifei Ma and Jian Zhang},
  title = {{ConstraintLLM: A Neuro-Symbolic Framework for Industrial-Level Constraint Programming}},
  year = {2025},
  eprint = {2510.05774},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2510.05774},
}
Figure