TL;DR
This paper introduces AIDL, a new domain-specific language (DSL) for Computer-Aided Design (CAD) that leverages the strengths of Large Language Models (LLMs) by offloading complex geometric reasoning to a solver. AIDL enables LLMs to generate editable and semantically meaningful CAD designs from natural language prompts, outperforming existing methods in terms of both visual fidelity and editability.
What Problem Does This Paper Address?
Traditional CAD systems represent geometry as programs, offering precision and editability. While LLMs excel at generating text and code, they struggle with the spatial reasoning and long-range planning required for complex procedural geometry generation in CAD. Existing CAD DSLs often lack features that align with LLM capabilities, such as semantic clarity and hierarchical structure, making it difficult for LLMs to generate not just functional but also easily editable designs. This paper aims to bridge this gap by designing a DSL that complements LLMs, allowing them to focus on high-level design intent while a solver handles the intricate geometric details.
Methods
The core contribution is AIDL (AI Design Language), a hierarchical, solver-aided DSL. AIDL is designed with four key principles:
- Solver-Aided: LLMs concentrate on high-level reasoning, while a geometric constraint solver handles precise spatial relationships. This means AIDL programs specify what relationships should exist, not how to compute them.
- Hierarchical: Designs are broken down into modular, nested structures, promoting modularity and facilitating local editing.
- Semantics: AIDL uses intuitively named variables and operators, leveraging LLMs' natural language understanding.
- Dependencies: AIDL supports implicit constraints through geometric dependencies, allowing parts to remain connected even when edited.
AIDL's backend uses a recursive constraint solver that iteratively refines the design, prioritizing translations over internal geometry modifications to maintain editability.
Datasets
Not discussed in the paper. The experiments use manually designed prompts and existing LLMs (OpenAI's gpt-4o) without specific fine-tuning on AIDL.
Key Insights and Ideas
- LLMs need specialized languages: Instead of solely focusing on training LLMs for existing CAD languages, designing DSLs tailored to LLM strengths is a powerful approach for AI-driven design.
- Solver-assistance is key for LLMs in CAD: Offloading geometric computation to a solver allows LLMs to focus on higher-level design intent, leading to more robust and accurate results.
- Hierarchy and semantics unlock editability: Breaking designs into hierarchical structures with semantically meaningful names makes the generated CAD models significantly easier for both humans and potentially future AI systems to modify.
- Implicit dependencies are crucial for editability: Allowing geometric relationships to be implicitly defined through shared references ensures that edits propagate correctly, preventing broken designs.
Key Results
- In a 2D text-to-CAD setting, AIDL programs generated by an untuned LLM were visually competitive with, and in some cases better than, those generated in OpenSCAD (a language the LLM was familiar with).
- AIDL designs were significantly easier to edit and reason about, as demonstrated by qualitative examples and the ability to modify parts of a design without breaking the entire structure.
- Ablation studies showed that both hierarchy and constraints are crucial for AIDL's effectiveness, contributing to local editability and precise composition, respectively.
- AIDL's success rate in generating valid programs was comparable to OpenSCAD, despite the LLM not being fine-tuned on AIDL.
Limitations and Future Work
- Model Complexity: The current system has limitations with very complex designs, where LLMs might miss components or generate incorrect code.
- Language Feature Hallucination: LLMs sometimes hallucinate non-existent language features or fail to use existing ones correctly (e.g., rectangle rotation).
- Error Recovery: The current validate-until-correct loop might lead to the LLM removing design intent when errors occur.
Future work could involve:
- Fine-tuning LLMs on AIDL to reduce language feature hallucination.
- Developing a more interactive prompting and feedback system to help LLMs recover lost complexity and design intent.
- Exploring the application of AIDL to 3D CAD generation.
- Investigating how AIDL can synergize with advances in multi-modal vision-language models.
Summary updated