Abstract
Translating natural language business descriptions into mathematical optimization models is a complex task typically requiring deep domain expertise. This paper introduces a framework that automates this process by fine-tuning code-specific Large Language Models (LLMs) and employing a modular "divide-and-conquer" strategy. Our approach achieves over 95% success in production scheduling tasks and significantly outperforms state-of-the-art prompt engineering methods on general benchmarks, reducing code execution errors and bridging the gap between business requirements and mathematical solvers.
1. Introduction
Operations Research (OR) provides powerful tools for minimizing costs and maximizing efficiency. However, the "expert bottleneck"—the need for specialized skills to translate messy, real-world business descriptions into precise mathematical formulations like Linear Programming (LP) or Mixed-Integer Programming (MIP)—remains a significant barrier to adoption. While general-purpose LLMs like GPT-4 show promise in code generation, they often struggle with the logical rigor and intricate constraints of optimization problems.
Current research often relies on prompt engineering, which frequently fails as problem complexity increases. This work proposes a shift toward domain-specific fine-tuning and modular generation to overcome these limitations.
2. Methodology
The proposed framework utilizes a multi-step approach to ensure accuracy and scalability:
- Base Model Selection: The researchers employed CodeRL, an open-source encoder-decoder LLM specifically optimized for code generation through reinforcement learning.
- Data Augmentation: To improve robustness, ChatGPT was used to "re-style" a small set of manual problem descriptions into hundreds of variations, simulating diverse stakeholder communication styles.
- Modularization: To bypass the context window limits of LLMs, the mathematical formulation is broken into self-sufficient modules (variables, constraints, and objective functions). These are generated sequentially and then integrated.
- Execution-Based Evaluation: Success is measured by running the generated code through Google’s OR-Tools solver to verify that it produces the correct optimal solution, rather than relying on simple text similarity.
3. Datasets and Case Study
The framework was evaluated across three distinct datasets:
- Job Shop Scheduling (JSS): 900 instances derived from 100 base problems.
- Real-World Production Scheduling: A case study of an Australian embroidery factory ("Company XYZ"). This involves "Flexible Job Shop Scheduling," a highly complex domain.
- LPWP (Linear Programming Word Problems): A refined version of the NL4Opt dataset used for benchmarking against state-of-the-art methods.
4. Results and Analysis
The fine-tuned framework demonstrated significant improvements over existing methods:
- Domain Accuracy: The model achieved a success rate exceeding 95% on both conventional and real-world scheduling tasks.
- Benchmark Performance: On the general LPWP benchmark, the framework reached a 72% success rate, nearly doubling the performance of the "Chain-of-Experts" (CoE) method (43%).
- Reliability: Code execution "exceptions" (crashes) were reduced to just 2% on the benchmark, compared to rates as high as 95% for standard prompting methods.
5. Conclusion
This research demonstrates that fine-tuning smaller, domain-specific models can outperform massive general-purpose LLMs in specialized tasks like business optimization. By combining fine-tuning with a modular generation approach, the framework effectively addresses the scalability and accuracy challenges inherent in translating natural language to mathematical models. Future work will explore human-in-the-loop refinement and the integration of broader domain knowledge to help the model handle even more diverse business scenarios.
Summary updated