No article text was recorded for this version.
data mixture optimization
Version 1 of 177 papers81 conceptsfrozen 2026-10-04
Papers in this version
- Blog: Loss-to-Loss Prediction: Scaling Laws for All Datasets
- Blog: Jina Embeddings: A Novel Set of High-Performance Sentence Embedding Models
- Blog: Holistic Data Scheduler for LLM Pre-training via Multi-Objective Reinforcement Learning
- Blog: Does your data spark joy? Performance gains from domain upsampling at the end of training
- Blog: IDEAL: Data Equilibrium Adaptation for Multi-Capability Language Model Alignment
- Blog: Fast Data Mixture Optimization via Gradient Descent
- Blog: SPRINQL: Sub-optimal Demonstrations driven Offline Imitation Learning
- Blog: Dataset Cartography: Mapping and Diagnosing Datasets with Training Dynamics
- Blog: Exploring Polyglot Harmony: On Multilingual Data Allocation for Large Language Models Pretraining
- Blog: Aya Model: An Instruction Finetuned Open-Access Multilingual Language Model
- Blog: Scaling Laws for Multilingual Neural Machine Translation
- Blog: InfoLaw: Information Scaling Laws for Large Language Models with Quality-Weighted Mixture Data and Repetition
- Blog: RegMix: Data Mixture as Regression for Language Model Pre-training
- Blog: Data Mixture Optimization: A Multi-fidelity Multi-scale Bayesian Framework
- Blog: CLIMB: CLustering-based Iterative Data Mixture Bootstrapping for Language Model Pre-training
- Blog: Scalable Data Ablation Approximations for Language Models through Modular Training and Merging
- Blog: Scaling Laws for Optimal Data Mixtures
- Blog: Retrieval-based Language Models Using a Multi-domain Datastore
- Blog: DataComp-LM: In search of the next generation of training sets for language models
- Blog: Mix, Don’t Pick: Why Synthetic Corpus Composition Matters for Time Series Foundation Model Pretraining
- Blog: Scaling Laws for Predicting Downstream Performance in LLMs
- Blog: Rethinking Data Mixture for Large Language Models: A Comprehensive Survey and New Perspectives
- Blog: Velocitune: A Velocity-based Dynamic Domain Reweighting Method for Continual Pre-training
- Blog: Handling the Alignment for Wake Word Detection: A Comparison Between Alignment-Based, Alignment-Free and Hybrid Approaches
- Blog: Is C4 Dataset Optimal for Pruning? An Investigation of Calibration Data for LLM Pruning
- Blog: Effective Training Data Synthesis for Improving MLLM Chart Understanding
- Blog: Aioli: A Unified Optimization Framework for Language Model Data Mixing
- Blog: AutoMixer: Checkpoint Artifacts as Automatic Data Mixers
- Blog: Shift is Good: Mismatched Data Mixing Improves Test Performance
- Blog: ADMIRE-BayesOpt: Accelerated Data MIxture RE-weighting for Language Models with Bayesian Optimization
- Blog: A Survey on Data Selection for Language Models
- Blog: Olmix: A Framework for Data Mixing Throughout LM Development
- Blog: LongWanjuan: Towards Systematic Measurement for Long Text Quality
- Blog: LongWanjuan: Towards Systematic Measurement for Long Text Quality
- Blog: When Long Helps Short: How Context Length in Supervised Fine-tuning Affects Behavior of Large Language Models
- Blog: COIG-CQIA: Quality is All You Need for Chinese Instruction Fine-tuning
- Blog: AutoMixAlign: Adaptive Data Mixing for Multi-Task Preference Optimization in LLMs
- Blog: DROID: A Large-Scale In-The-Wild Robot Manipulation Dataset
- Blog: DRAW: Domain Weight Randomization with Bayesian Updating for LLM Pre-Training
- Blog: CMR Scaling Law: Predicting Critical Mixture Ratios for Continual Pre-training of Language Models
- Blog: Multitask Learning 1997–2024: Part I Fundamentals
- Blog: Training Dynamics of Parametric and In-Context Knowledge Utilization in Language Models
- Blog: MoExtend: Tuning New Experts for Modality and Task Extension
- Blog: Dolma: an Open Corpus of Three Trillion Tokens for Language Model Pretraining Research
- Blog: Towards Efficient LLMs Annealing with Principled Sample Selection
- Blog: DoReMi: Optimizing Data Mixtures Speeds Up Language Model Pretraining
- Blog: Skill-it! A data-driven skills framework for understanding and training language models
- Blog: Data Mixing Laws: Optimizing Data Mixtures by Predicting Language Modeling Performance
- Blog: Causal Transfer Random Forest: Combining Logged Data and Randomized Experiments for Robust Prediction
- Blog: An Integrated Data Processing Framework for Pretraining Foundation Models
- Blog: DoGE: Domain Reweighting with Generalization Estimation
- Blog: Front-Loading Reasoning: The Synergy between Pretraining and Post-Training Data
- Blog: Adapting Large Language Models via Reading Comprehension
- Blog: LLM360: Towards Fully Transparent Open-Source LLMs
- Blog: TANDEM: Bi-Level Data Mixture Optimization with Twin Networks
- Blog: Towards Neural Scaling Laws for Time Series Foundation Models
- Blog: ScaleBiO: Scalable Bilevel Optimization for LLM Data Reweighting
- Blog: Decouple Searching from Training: Scaling Data Mixing via Model Merging for Large Language Model Pre-training
- Blog: SampleMix: A Sample-wise Pre-training Data Mixing Strategy by Coordinating Data Quality and Diversity
- Blog: SMART: Submodular Data Mixture Strategy for Instruction Tuning
Showing 60 of 77 papers.
History & citation
Current · Permanent link
- Automated Data Mixture Optimization for Language Models merged into this direction (67% overlap)
Cite this direction
Latest version (a living page):
@misc{lacuna_data_mixture_optimization_9ac90158,
title = {data mixture optimization},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/data-mixture-optimization/rdr_9ac90158c7094766895e920a8d5ccd82},
note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-21},
}This exact version (version 1, frozen text):
@misc{lacuna_data_mixture_optimization_9ac90158_v1,
title = {data mixture optimization},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/data-mixture-optimization/rdr_9ac90158c7094766895e920a8d5ccd82/v/1},
note = {Version 1, frozen 2026-10-04. Accessed 2026-11-21},
}