Beyond Accuracy Optimization: Computer Vision Losses for Large Language Model Fine-Tuning

Daniele Rege Cambrin, Giuseppe Gallipoli, Irene Benedetto, Luca Cagliero, Paolo GarzaEMNLP 2024 · DOI

Abstract

Standard Cross-Entropy (CE) loss, the default objective for fine-tuning Large Language Models (LLMs), often fails to capture the structural nuances required for complex reasoning tasks. This paper explores the adaptation of loss functions from computer vision—specifically Focal and Lovász losses—to LLM fine-tuning. By treating reasoning chains as structured segments, researchers achieved a 36% mean improvement in Exact Match accuracy across mathematical and commonsense benchmarks. This approach offers a data-efficient alternative to Reinforcement Learning from Human Feedback (RLHF) by utilizing a "drop-in" loss replacement during Low-Rank Adaptation (LoRA).

Introduction: The Limitations of Cross-Entropy

When fine-tuning LLMs for specialized domains like mathematics or logic, the industry standard is Cross-Entropy (CE) loss. CE calculates the likelihood of the correct next token, but it suffers from two primary deficiencies in structured tasks:

  1. Token Imbalance: CE treats all tokens with equal weight. In a math problem, the final numerical answer is significantly more critical than transitional phrases like "The answer is," yet CE does not distinguish between them.
  2. Structural Ignorance: CE evaluates tokens in isolation, failing to account for the global "shape" or syntax of the output. A single missing step in a reasoning chain is catastrophic for accuracy, but CE may only penalize it as a minor local error.

While methods like RLHF or Direct Preference Optimization (DPO) address these issues, they are computationally expensive and require vast amounts of human-labeled preference data.

Methodology: Borrowing from Computer Vision

The authors propose a "drop-in" replacement for the fine-tuning objective by borrowing from semantic segmentation, a computer vision task focused on labeling pixels to delineate object boundaries. They argue that generating a structured reasoning chain is analogous to defining boundaries in an image.

Specialized Loss Functions

The study evaluates two primary alternatives to CE:

  • Focal Loss: Originally designed for object detection to handle class imbalance, it down-weights the loss for "easy" examples and focuses the model's attention on "hard" tokens that the model is struggling to predict.
  • Lovász Loss: A region-based loss that optimizes for the Jaccard Index (Intersection-over-Union). Instead of looking at tokens in isolation, it penalizes the model based on how much the entire predicted sequence overlaps with the ground truth, effectively optimizing for global structure.

Implementation

The researchers utilized a convex combination of standard CE (applied to the entire prompt) and the new CV loss, which is applied specifically to the answer and reasoning steps. This targeted approach ensures the model maintains general language capabilities while sharpening its performance on the critical components of the task.

The training objective applies specialized losses (LA) specifically to the answer tokens (AT), while standard Cross-Entropy (CE) is maintained for the full sequence to preserve context.

To maintain efficiency, the models were fine-tuned using LoRA (Low-Rank Adaptation), updating less than 1% of the total parameters.

Experimental Setup

The study evaluated five open-source models—RedPajama-Incite (3B & 7B), StableLM 3B, Falcon 7B, and Llama-2 7B—across four diverse benchmarks:

  • GSM8K & MathQA: Grade school and diverse math word problems requiring multi-step reasoning.
  • OpenBookQA & HellaSwag: Commonsense and multiple-choice question answering.

Results and Analysis

The transition to CV-inspired losses yielded significant performance gains. Across all tested models and datasets, there was a +36% mean improvement in Exact Match (EM) scores compared to the Cross-Entropy baseline.

Exact Match improvement (%) across various models and datasets. The CV-inspired losses consistently outperform the standard Cross-Entropy baseline, particularly in reasoning-heavy tasks.

Key Findings

  • Task-Specific Optimization: The Lovász loss was the standout performer for mathematical reasoning (GSM8K and MathQA). By optimizing for global overlap, it significantly reduced "missing steps," a common failure mode in LLM reasoning. Conversely, Focal loss proved superior for multiple-choice QA (HellaSwag), helping models distinguish between subtle differences in answer choices.
  • Data Efficiency: These losses made models remarkably robust to small datasets. When training data was reduced to 10% of the original size, models using Focal or Lovász losses maintained significantly higher performance than those using standard CE.
  • Competitive Performance: The fine-tuned 7B models often outperformed much larger or more specialized models (such as MAmmoTH or MetaMath) that were trained on significantly more data.

Conclusion

This research demonstrates that the choice of loss function is a critical, yet often overlooked, hyperparameter in LLM fine-tuning. By adopting objectives from computer vision that account for token difficulty and global structure, researchers can achieve state-of-the-art performance without the need for expensive human feedback or massive increases in training data. Future work will explore the scalability of these losses to frontier models (70B+) and their potential integration into the initial pre-training phase.

Summary updated

BibTeX
@inproceedings{cambrin2024beyond,
  author = {Daniele Rege Cambrin and Giuseppe Gallipoli and Irene Benedetto and Luca Cagliero and Paolo Garza},
  title = {{Beyond Accuracy Optimization: Computer Vision Losses for Large Language Model Fine-Tuning}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2024},
  doi = {10.18653/v1/2024.findings-emnlp.704},
}
Figure