TL;DR
IndustryLLM is an open-weight 35B Mixture-of-Experts (MoE) model specialized for the high-stakes world of industrial procurement. By employing a "failure-driven" training recipe that targets specific domain gaps and enforcing a strict "three-valued logic" for product eligibility, the model delivers substantial gains in transaction volume (GMV) and a 4x reduction in latency compared to general-purpose baselines.
What Problem Does This Paper Address?
Industrial procurement is a minefield of technical jargon, ambiguous standards, and zero-tolerance safety requirements. When a buyer asks for a "42-luo-mu bolt" (a colloquialism for 42CrMo steel) or cites a truncated standard like "16674," general-purpose LLMs often stumble.
The authors identify three core challenges:
- Register Mismatch: The gap between formal engineering standards (dry, technical) and buyer inquiries (informal, messy).
- Factual Brittleness: The tendency of models to hallucinate technical specs or rely on obsolete standards.
- The "Satisfied by Default" Fallacy: In safety-critical engineering, if a product record doesn't mention a required temperature rating, a model shouldn't assume it's fine. It must treat missing information as a hard "Unknown."
Methods
The researchers started with Qwen3.5-35B-A3B-Base and applied a two-stage adaptation process:
1. Failure-Driven Continued Pre-training (CPT)
Instead of just feeding the model more data, they reconstructed a 20B-token subset of their corpus to fix specific failure modes:
- Multi-register Rewriting: They used a teacher model to recast formal standards into 10 different genres (e.g., troubleshooting notes, buyer-seller dialogues) and 8 writing styles to bridge the jargon gap.
- Confidence-Routed Minimal Editing: An automated pipeline identified factual errors in industrial web text (like expired standards) and corrected them via localized "string diffs" only when high-confidence evidence was available.
- Weakness-Targeted QA Synthesis: They generated synthetic Q&A pairs specifically for topics where the model showed high failure rates during intermediate testing.
2. Supervised Fine-Tuning (SFT)
The model was trained on teacher-generated targets (from Qwen3.5-Max) that were strictly filtered for factual correctness and reasoning density. They released two variants: Think (retains the chain-of-thought for complex reasoning) and No-Think (optimized for sub-second latency).
3. The Evidence-Gated Interface
To prevent dangerous hallucinations, they formalized a deployment interface using three-valued logic: Satisfied, Violated, or Unknown. Crucially, "Unknown" is never treated as "Satisfied," triggering a request for clarification rather than a potentially catastrophic product recommendation.
Datasets
- Training Corpus (~100B tokens): Includes 5B tokens of national standards (GB/T) and technical archives, 10B tokens of de-identified real-world transaction records, 25B tokens of filtered industrial web text, and 60B tokens of general-domain replay to prevent catastrophic forgetting.
- IndustryBench: A diagnostic benchmark consisting of 2,049 expert-curated questions used to measure domain mastery and safety.
- General Benchmarks: MMLU-Pro, GSM8K, HumanEval-Plus, etc., were used to ensure the model retained general reasoning capabilities.
Key Insights and Ideas
- Linguistic Diversity Matters: The "aha moment" in their CPT strategy was that rewriting formal knowledge into diverse registers (like blogs or chats) significantly improved the model's ability to understand real-world buyers without losing technical accuracy.
- Sparse Efficiency: By using a Mixture-of-Experts (MoE) architecture (activating only ~3B parameters per token), they achieved domain performance that surpassed much larger dense models, making it viable for production environments.
- Demarcation of Knowledge: The authors argue for a strict boundary: the LLM should structure the intent (what the buyer needs), but the eligibility (whether the product works) must be determined by an auditable system checking fresh, non-parametric evidence.
Key Results
- Production Impact: In randomized online A/B trials (N ≈ 100,000 users/day), IndustryLLM drove a +4.25% increase in Gross Merchandise Value (GMV) and an +8.3% increase in satisfied inquiries.
- Latency: End-to-end response time dropped from 6–7 seconds to 1.5 seconds, a 4x improvement essential for e-commerce.
- Offline Accuracy: The model saw a +2.97 percentage point gain in exact match for query structuring tasks compared to the base model.
- Safety: On the IndustryBench safety-adjusted metric, IndustryLLM (35B) outperformed significantly larger models, including the 397B parameter version of Qwen3.5.
Limitations and Future Work
- Geographic/Language Scope: The current study is centered on the Chinese B2B market. The authors note that cross-lingual transfer and international regulatory compatibility remain unverified.
- Multimodal Gap: While the base model has vision capabilities, the vision encoder was kept frozen for this study. Future work could involve extracting constraints directly from CAD blueprints or technical drawings.
- Model Lineage: The reliance on the Qwen family for both generation and judging introduces potential "intra-family" biases that could be explored with more diverse model ensembles.
Availability: The model weights and inference configurations are publicly available on Hugging Face.
Summary updated