A Human-in-the-Loop Autonomous Agent for Industry Time Series Forecasting

Xiaoyu Tao, Mingyue Cheng, Ze Guo, Bokai Pan, Qi Liu, Shijin Wang, Enhong ChenarXiv 2026 · arXiv

TL;DR

CastClaw is a human-in-the-loop autonomous forecasting system that wraps specialized time-series models in an inspectable, tool-equipped agentic harness. By combining automated validation checks with natural-language human oversight, it achieves state-of-the-art accuracy across multiple electricity-price datasets without inventing a new forecasting model backbone.

What Problem Does This Paper Address?

Real-world industrial forecasting is a messy, iterative process. It is rarely as simple as a "one-shot" model call; practitioners must constantly inject domain knowledge, verify constraints, investigate anomalies, and adjust to sudden context shifts.

Current solutions fall into two extremes: traditional deep learning models operate in rigid, fixed pipelines that cannot ingest qualitative context, while general-purpose Large Language Model (LLM) agents lack the rigorous statistical checks, hard constraints, and explicit stopping criteria needed for reliable time-series analysis. The authors bridge this gap by designing an autonomous agent specifically engineered to handle the end-to-end forecasting lifecycle alongside human experts.

Methods

CastClaw introduces "forecasting-oriented harness engineering." Rather than acting as a new numerical predictor, the system serves as an intelligent coordinator that manages interchangeable forecasting backbones, analytical tools, and human inputs through a versioned execution record.

The system operates across five distinct stages:

  1. Intent Understanding: Parsing natural language requests to define targets, horizons, and constraints.
  2. Data Profiling: Analyzing datasets and searching for missing contextual evidence.
  3. Experience Accumulation: Evaluating historical rules and past forecasting runs.
  4. Prediction: Running specialized models and assessing forecast plausibility.
  5. Final Answer & Critic: Generating visualizations, final reports, and auditable execution logs.
CastClaw’s forecasting workflow and its supporting execution, tool, and state-management services.

When CastClaw detects a lack of support for a forecast (e.g., a sudden residual spike or a violated constraint), it triggers tools from its library (like web search or data visualizers) or prompts the user for advice. Crucially, any proposed forecast revision must pass strict validation metrics and hard constraints before replacing the active forecast, preventing hallucinated or degraded results.

Datasets

The system was evaluated and compared against baselines using:

  • Five public electricity-price datasets: Belgium (BE), Germany (DE), France (FR), Nord Pool (NP), and PJM. Each was split chronologically using a 7:1:2 ratio for training, validation, and testing.
  • An offline industrial dataset: Real-world provincial electricity-load data from North China covering January–June 2026, used to validate the stability of the offline workflow.

Key Insights and Ideas

  • Orchestration over Architecture: The paper demonstrates that wrapping existing, specialized models in a rigorous agentic framework yields massive performance gains without altering the underlying neural network architectures.
  • Gated Autonomy: Pure LLM agents often suffer from unconstrained drift. CastClaw implements "validation-based candidate checking"—revisions are strictly rejected if they do not improve validation metrics or if they violate user-defined hard constraints.
  • Human-in-the-Loop as a Versioned Input: Human interventions (such as adding a temporary hypothesis or correcting a target) are not just post-hoc feedback; they are treated as timestamped, versioned inputs within the execution record, making the entire decision-making process fully auditable.

Key Results

CastClaw achieved the lowest point-estimate Mean Squared Error (MSE) and Mean Absolute Error (MAE) across all ten dataset-metric combinations, outperforming 16 baselines (including specialized neural models like PatchTST and iTransformer, and forecasting agents like AlphaCast).

Compared to the best competing baseline for each dataset:

  • MSE Reductions: Lowered by 5.5% on BE, 30.1% on DE, 11.4% on FR, 17.7% on NP, and 9.7% on PJM. Across all datasets, the unweighted mean reduction in MSE was 14.9%.
  • MAE Reductions: Achieved an unweighted mean reduction of 18.6% across the five datasets.

Limitations and Future Work

While CastClaw shows strong empirical performance, the authors note several areas for future exploration:

  • Ablation and User Studies: Detailed ablation studies are needed to isolate the exact causal impact of specific tool calls versus user interventions, and formal user studies are required to evaluate human-agent interaction dynamics.
  • Point Estimates: The current evaluation relies on point estimates from controlled runs rather than broader statistical significance testing.
  • Deployment Scope: The provincial-load dataset was used strictly for offline workflow validation; online deployment, production scaling, and direct business impact metrics were not evaluated.

Summary updated

Abstract

Real-world time-series forecasting is rarely a one-shot model invocation: practitioners must formulate tasks, connect data and models, incorporate domain expertise, assess prediction plausibility, and communicate uncertainty. Specialized forecasting models provide strong numerical predictions but usually operate in fixed pipelines, while general-purpose large language model (LLM) agents often lack forecasting-specific checks, constraints, and stopping rules. We present CastClaw, a human-in-the-loop autonomous forecasting system built through forecasting-oriented harness engineering. CastClaw connects data, specialized models, analytical tools, user input, and a versioned execution record in one runtime. Users specify the target, horizon, constraints, and hypotheses in natural language. Starting from a supplied or model-generated forecast, CastClaw checks temporal patterns and user constraints; when evidence is missing, it retrieves context, runs an analysis or another model, or asks the user. It then keeps, revises, or escalates the result under explicit stopping conditions. The output contains the final forecast and an execution report recording inputs, evidence, actions, and revisions. In this five-dataset electricity-price setting, CastClaw reports the lowest point-estimate MSE and MAE among 16 baselines. A Nord Pool case demonstrates the inspectable workflow. CastClaw was also validated offline on provincial electricity-load data from North China covering January--June 2026.
BibTeX
@misc{tao2026human,
  author = {Xiaoyu Tao and Mingyue Cheng and Ze Guo and Bokai Pan and Qi Liu and Shijin Wang and Enhong Chen},
  title = {{A Human-in-the-Loop Autonomous Agent for Industry Time Series Forecasting}},
  year = {2026},
  eprint = {2608.30976},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2608.30976},
}

Loading related content...

Figure