Domain-Specific Fine-Tuning of Large Language Models for Interactive Robot Programming

Benjamin Alt, Urs Keßner, Aleksandar Taranovic, Darko Katic, Andreas Hermann, Rainer Jäkel, Gerhard Neumann

TL;DR

This paper explores how to fine-tune large language models (LLMs) to act as interactive assistants for industrial robot programming. By adapting general-purpose LLMs to the specific domain of robot skills and documentation, the authors aim to make complex robot programming more accessible to experts. Their findings suggest that domain-specific fine-tuning is crucial for robust performance, but current methods still face challenges with accuracy and hallucination.

What Problem Does This Paper Address?

Programming industrial robots, especially for advanced tasks involving force sensing or vision, is a complex and time-consuming process that typically requires specialized expertise. While skill-based programming approaches simplify this by sequencing pre-defined robot actions (like "grasp" or "insert"), they still demand deep knowledge of physical interactions and technical nuances. The advent of powerful LLMs like LLaMA presents an opportunity to create more intuitive programming interfaces. This paper tackles the challenge of adapting these general LLMs to understand and assist with the specific language and concepts used in industrial robot programming, aiming to bridge the gap between human language and robot control.

Methods

The core of this research lies in investigating different strategies for domain-specific fine-tuning of LLMs. The authors leverage existing LLMs (Alpaca and LLaMA) and adapt them using limited data and compute resources, mimicking constraints faced by small and medium-sized enterprises.

Key techniques include:

  • QLORA: A parameter-efficient fine-tuning method that significantly reduces memory requirements by quantizing model weights and using low-rank adapters. This is crucial for fitting the fine-tuning process onto a single server-grade GPU.
  • Three Fine-Tuning Regimes:
    1. Instruction-Following Fine-Tuning: Adapting a pre-trained instruction-following model (like Alpaca) directly on domain-specific instruction-response pairs.
  • Streaming Fine-Tuning: Adapting a general-purpose streaming model (like LLaMA) on domain-specific text data (e.g., robot manuals).
  • Merged Fine-Tuning: A two-stage approach where a model is first fine-tuned on streaming data and then on instruction-following data.
  • Prefix Fine-Tuning: Exploring the impact of adding a specific "prefix" to prompts to guide the model towards the correct domain.

Datasets

The paper utilizes two primary types of datasets:

  • Streaming Dataset (Dstream): Created by parsing the ArtiMinds Robot Programming Suite (RPS) user manual. Non-informational content was removed to focus on technical details. This dataset is used for fine-tuning general streaming models.
  • Instruction-Following Dataset (Dinstr): A domain-specific dataset containing questions about RPS templates, their descriptions, use cases, and step-by-step behaviors. To address prompt sensitivity, 10 variants of each question were generated using ChatGPT 3.5. This dataset is used for training instruction-following capabilities.
  • Prefix Dataset (Dprefix): A version of the instruction-following dataset where custom, domain-specific prefixes are used instead of the default Alpaca prefixes.

Key Insights and Ideas

  • Domain-Specific Fine-Tuning is Essential: The results strongly suggest that adapting LLMs to the specific language and concepts of industrial robot programming is critical for effective performance. General-purpose models, even powerful ones, struggle without this specialized training.
  • Instruction-Following Models Excel: Directly fine-tuning models that are already trained for instruction following (like Alpaca) on domain-specific instructions appears to be the most effective approach for achieving robust domain transfer.
  • Model Size Isn't Everything (for BERTScore): The paper found that model size (7B, 13B, 30B parameters) did not directly correlate with BERTScore performance, suggesting that the quality and relevance of the fine-tuning data and method are more influential.
  • Prefix Fine-Tuning's Limited Impact: Prefix fine-tuning, while a promising technique, did not show a discernible positive impact on model performance in this study.
  • Hallucinations Remain a Challenge: Even with fine-tuning, the models exhibited significant issues with "hallucinations" (generating incorrect or out-of-domain information) and repeating words or phrases, indicating that fine-tuning alone may not be sufficient to fully imbue the model with new knowledge.

Key Results

  • BERTScore Performance: Alpaca models fine-tuned on instruction tasks without prefixes achieved the highest BERTScore F1 scores (up to 0.8042), indicating strong semantic similarity to expert-written responses.
  • User Survey Results: Echoing the BERTScore findings, domain experts rated Alpaca models highest in terms of factual correctness, domain adherence, and helpfulness.
  • LLaMA Model Struggles: LLaMA-based models, particularly when fine-tuned, showed more issues with sentence structure, word repetition, and predicting unknown tokens compared to Alpaca models.
  • Limitations of Fine-Tuning: Despite improvements, the models' performance was deemed "not (yet) sufficient for practical use" due to persistent hallucinations and inaccuracies.

Limitations and Future Work

The authors acknowledge several limitations:

  • Practical Usability: The current models are not yet robust enough for direct deployment in real-world industrial settings due to issues like hallucinations and factual inaccuracies.
  • Data Scarcity: The fine-tuning process was performed with limited data, which likely contributes to the models' shortcomings.
  • Prompting Strategies: The paper suggests that prompting strategies, which rely on carefully crafted prompts rather than extensive fine-tuning, might be a more feasible alternative given current LLM capabilities and data constraints.
  • Larger-Scale Evaluation: More extensive user surveys and real-world testing are needed to draw more robust conclusions about model performance.

Future work could focus on exploring more advanced prompting techniques, increasing the scale and diversity of training data, and developing better methods to mitigate hallucinations and improve factual accuracy in domain-specific LLM applications.

Summary updated

Abstract

Industrial robots are applied in a widening range of industries, but robot programming mostly remains a task limited to programming experts. We propose a natural language-based assistant for programming of advanced, industrial robotic applications and investigate strategies for domain-specific fine-tuning of foundation models with limited data and compute.
BibTeX
@inproceedings{alt2024domain,
  author = {Benjamin Alt and Urs Keßner and Aleksandar Taranovic and Darko Katic and Andreas Hermann and Rainer Jäkel and Gerhard Neumann},
  title = {{Domain-Specific Fine-Tuning of Large Language Models for Interactive Robot Programming}},
  booktitle = {Erf},
  year = {2024},
  doi = {10.1007/978-3-031-76424-0\_49},
}
Figure