In the world of B2B manufacturing and software, timing is everything. Companies often operate on a "make-to-order" strategy, reserving expensive resources—engineers, factory floor time, and raw materials—based on a scheduled order date. But what happens when a client drags their feet? A delay doesn't just push back a deadline; it leaves reserved resources idling, driving up costs and disrupting entire pipelines.
A new paper from researchers at Hitachi, Ltd., titled "Forecasting Sales Delay through Time-aware Analysis of Sales Events in Textual Reports," introduces a novel way to predict these delays by reading between the lines of sales representatives' reports.
TL;DR
The researchers introduce sales delay forecasting, a task that predicts the gap between a scheduled and actual order date using NLP. By extracting "confidence" and "status" features from textual sales reports, their method achieved a 16% lower Mean Absolute Error (MAE) than traditional project-info baselines and outperformed state-of-the-art Large Language Models (LLMs) like Qwen3.
What Problem Does This Paper Address?
In B2B sales, the "scheduled order date" is often an estimate made early in the process. However, the actual order date ($d_{actual}$) is a moving target influenced by the client’s internal consensus-building and willingness to commit.
Unlike software development or construction, where progress can be measured by physical artifacts (like lines of code or bricks laid), sales progress is largely invisible and highly subjective. Current forecasting methods often rely on static project data (like budget or initial duration), which fails to capture the evolving "vibe" of a deal. The authors argue that the most valuable data lies in the textual reports written by sales reps after client meetings, but this data is unstructured and difficult to quantify.
Methods
The authors propose a three-step pipeline to turn messy text into a precise delay forecast:
- Analysis of Textual Reports: The system uses NLP tools (like spaCy and GiNZA) to extract date expressions and keywords. It maps these to four typical sales phases: Contact, Proposal, Negotiation, and Closing. Crucially, it distinguishes between Facts (past events) and Prospects (future events mentioned by the rep).
- Feature Creation: The core innovation lies in two types of features:
- Confidence Features: These act as a proxy for the client's willingness to order. For example, if a rep writes about a "Closing" event that is significantly later than the scheduled date, the model flags low confidence. It also incorporates sentiment analysis to gauge the rep's optimism.
- Status Features: These provide an objective reality check. They track the current phase of the deal, how long the deal has been stuck in that phase, and how many days remain until the scheduled deadline.
- Regression Model: These features are fed into a Support Vector Machine (SVM) with an RBF kernel to predict the final delay ($y_t$).
Datasets
The study used a proprietary dataset of real-world sales data from a Japanese company.
- Scope: Closed sales projects with multiple forecasting points ($t$) as new reports were added.
- Size: The final evaluation used 1,242 instances for training, 245 for development, and 315 for testing.
- Baselines: The model was compared against trivial baselines (Mean/Zero), a feature-based baseline using only project metadata (ProjectInfo), and several LLMs (RoBERTa, LUKE, BigBird, and Qwen3).
Key Insights and Ideas
- LLMs Aren't Always the Answer: Interestingly, fine-tuned LLMs like RoBERTa and even powerful few-shot models like Qwen3-32B struggled with this task. They tended to predict values close to the mean, failing to capture the wide distribution of actual delays. Explicitly engineering features from the text proved far more effective than letting a model "read" the whole report.
- The "Reality Check" Matters: Sales reps are human and prone to bias. The researchers found that combining Confidence (the rep's subjective view) with Status (the objective timeline) was essential. An optimistic rep reporting from the "Contact" phase just days before a deadline is a major red flag that the model can now catch.
- Time-Awareness is Crucial: The model's accuracy improves significantly as the actual order date approaches. By capturing the "rate of change" in prospects and status, the model becomes a better indicator of reality over time.
Key Results
- Superior Accuracy: The proposed method achieved an MAE of 70.80, a 16% improvement over the ProjectInfo baseline (84.06) and significantly better than the best LLM baseline (Qwen3-32B at 76.10).
- Ablation Success: Removing the "Prospect" feature (anticipated future dates) caused the MAE to jump by +44.17, proving that the rep's future expectations are the single most important signal in the text.
- LLM "Thinking" Helps: For the LLM baselines, enabling Qwen3’s "thinking mode" (Chain-of-Thought) improved performance, as the model attempted to perform arithmetic on the dates mentioned in the text—though it still couldn't beat the specialized feature-based approach.
Limitations and Future Work
- Language Specificity: The current implementation is tailored to Japanese text and specific Japanese sales terminology. While the framework is general, the keyword extractors would need to be rebuilt for other languages.
- Report Quality: The model relies entirely on sales reps writing detailed, honest reports. If a rep is lazy or intentionally misleading in their documentation, the model's "Confidence" features will suffer.
- Inference Speed: While the feature-based model is fast, the researchers noted that using LLMs with "thinking modes" for comparison requires substantially more inference time, making them less practical for real-time enterprise dashboards.
Future work could involve integrating this with resource management systems to automatically trigger alerts when a high-value project is likely to slip.
Summary updated