TL;DR
This paper introduces an "error-guided multimodal sample selection" method to improve the efficiency and effectiveness of training Large Vision-Language Models (LVLMs). By focusing on samples that the model struggles with and actively suppressing hallucinations during the selection process, they demonstrate significant performance gains on downstream tasks with the same training budget.
What Problem Does This Paper Address?
Training powerful LVLMs requires vast amounts of data, but creating high-quality, diverse instruction datasets is a major bottleneck – it's time-consuming and expensive. While previous work has shown that a smaller, carefully curated dataset can outperform a larger, lower-quality one, automatically selecting these "high-value" samples is challenging. Existing methods often rely on static metrics or external models, failing to directly address the LVLM's actual errors and weaknesses, especially in complex, open-ended tasks where hallucinations are a common problem. This paper aims to bridge this gap by developing an automated, error-driven approach that specifically targets difficult samples and mitigates hallucinations during the selection process.
Methods
The proposed method, "Error-Guided Multimodal Sample Selection with Hallucination Suppression," involves several key stages:
- Clustering: The initial candidate dataset is clustered based on the semantic embeddings of queries. This ensures diversity and prevents bias by preserving representative samples from different categories.
- Hallucination Suppression: A "Visual Contrastive Decoding" (VCD) module is employed during the generation of responses for the candidate dataset. This module helps identify samples that are genuinely difficult for the LVLM, even after hallucination mitigation, by comparing standard decoding with a perturbed version. Samples that remain problematic after VCD are considered high-value.
- Evaluation: The difficulty of samples is assessed differently based on task type:
- Closed-ended tasks (e.g., object detection): Sample value is estimated using prediction accuracy.
- Open-ended tasks (e.g., visual question answering): Perplexity (PPL) of the generated response is used as a proxy for difficulty and model uncertainty.
- Data Selection: Based on the evaluation scores, high-error or high-perplexity samples are preferentially selected from each cluster. This ensures that the selected training data is both informative (exposing model weaknesses) and representative (covering diverse task types).
Datasets
- InsPLAD: A publicly available dataset for object detection in the power grid domain, used for evaluating the method's performance on a closed-ended task.
- PowerQA: A newly constructed multimodal question-answering dataset focused on the power industry. This dataset was created using GPT-40 for initial generation, followed by manual annotation and filtering, and is used to evaluate the method's performance on open-ended tasks.
Key Insights and Ideas
- Error-centricity is key: Simply having more data isn't as effective as having data that specifically targets the model's current weaknesses.
- Hallucination suppression is crucial for selection: By actively mitigating hallucinations during the sample selection process, the method can better identify truly difficult and informative samples, rather than just samples that are hard due to model errors.
- Perplexity as a proxy for uncertainty: For open-ended tasks, PPL effectively captures model uncertainty, making it a valuable metric for identifying challenging samples that require more training.
- Clustering balances diversity and difficulty: Combining clustering with error-guided selection ensures that the training set is not only focused on difficult examples but also maintains broad coverage across different task types.
Key Results
- InsPLAD: The proposed method ("Choice_N") consistently outperformed random sampling and the AlpaGasus baseline. For example, with 2000 samples, the F1 score increased by 0.169 over random sampling and 0.097 over AlpaGasus. With 5000 samples, the F1 score reached 0.809, a significant improvement.
- PowerQA: The method also showed superior performance on generative tasks. For 3000 samples, BLEU score improved to 21.88 and GPT-40 score to 5.75. With 5000 samples, it achieved the highest cosine similarity (0.1925) and a competitive GPT-40 score of 5.82.
- Generalization: The method demonstrated strong generalization capabilities on the MM-Vet benchmark, outperforming baselines across different training data sizes.
- Qualitative Case Study: Visual comparisons showed that models trained with the proposed method produced more accurate, detailed, and semantically rich responses, successfully identifying subtle anomalies and foreign objects missed by models trained on randomly selected data.
Limitations and Future Work
- Static Data Selection: The current pipeline is "one-shot" and static, meaning it selects a fixed set of hard samples at the beginning. Future work could explore adaptive curricula that adjust the mix of easy and hard samples during training.
- Joint Optimization: The paper does not perform end-to-end joint optimization of hallucination mitigation and data selection. Future research could explore this.
- Other Modalities: Extending the hallucination-aware selection to other modalities like video or 3D data is a promising direction.
- Expert Feedback: Incorporating richer expert feedback from practitioners in specific application scenarios could further enhance the method.
- Model Architecture Integration: Investigating the joint optimization of data selection and model architecture, perhaps by adding specialized adapters for hallucination-prone clusters, is another avenue for future work.
Summary updated