ZARA: Zero-shot Motion Time-Series Analysis via Knowledge and Retrieval Driven LLM Agents

Zechen Li, Baiyu Chen, Hao Xue, Flora D. Salim

Abstract

Motion sensor time-series are central to human activity recognition (HAR), with applications in health, sports, and smart devices. However, existing methods are trained for fixed activity sets and require costly retraining when new behaviours or sensor setups appear. Recent attempts to use large language models (LLMs) for HAR, typically by converting signals into text or images, suffer from limited accuracy and lack verifiable interpretability. We propose ZARA, the first agent-based framework for zero-shot, explainable HAR directly from raw motion time-series. ZARA integrates an automatically derived pair-wise feature knowledge base that captures discriminative statistics for every activity pair, a multi-sensor retrieval module that surfaces relevant evidence, and a hierarchical agent pipeline that guides the LLM to iteratively select features, draw on this evidence, and produce both activity predictions and natural-language explanations. ZARA enables flexible and interpretable HAR without any fine-tuning or task-specific classifiers. Extensive experiments on 8 HAR benchmarks show that ZARA achieves SOTA zero-shot performance, delivering clear reasoning while exceeding the strongest baselines by 2.53x in macro F1. Ablation studies further confirm the necessity of each module, marking ZARA as a promising step toward trustworthy, plug-and-play motion time-series analysis.

Introduction: The Challenge of Generalizable HAR

Human Activity Recognition (HAR) is ubiquitous, powering everything from fitness tracking to clinical gait analysis. However, traditional HAR systems are inherently "brittle," designed for fixed activity sets and specific sensor configurations. Introducing a new activity or changing a sensor's body placement typically necessitates expensive data collection and model retraining.

Comparison of ZARA with existing HAR methods, highlighting its advantages in zero-shot accuracy and interpretability.

While Large Language Models (LLMs) offer potential for zero-shot reasoning, raw motion signals are essentially "gibberish" to text-based models. Previous attempts to bridge this gap by converting signals into images or numerical strings have proven computationally expensive, lossy, and opaque. ZARA addresses these limitations by treating HAR as a structured reasoning task performed by a team of specialized LLM agents.

The ZARA Framework

ZARA enables LLMs to "sense motion" through a three-pillar architecture that combines statistical domain knowledge with retrieval-augmented generation (RAG).

The ZARA architecture: The offline stage builds a pairwise knowledge base and vector database; the inference stage utilizes RAG and a hierarchical multi-agent pipeline for prediction.

1. Pair-Wise Knowledge Base

Instead of global class definitions, ZARA builds a database of discriminative features for pairs of activities (e.g., distinguishing "walking" from "climbing stairs"). This "cheat sheet" uses interpretable statistics like spectral entropy and signal magnitude area, making the system label-agnostic and easily expandable.

2. Class-Wise Multi-Sensor Retrieval

When processing a new signal, ZARA uses a pre-trained time-series encoder to search for similar historical examples across multiple sensors. It employs Reciprocal Rank Fusion (RRF) to synthesize evidence from different body locations into a coherent context for the LLM.

3. Hierarchical Multi-Agent Reasoning

ZARA utilizes four specialized agents (powered by Gemini 2.0-Flash) to process the data:

  • Feature Selector: Identifies critical physical features for the current candidates.
  • Evidence Pruning: Compares the query to retrieved evidence to narrow down the activity list.
  • Refined Feature Selector: Focuses on fine-grained details to separate highly similar activities.
  • Decision Insight: Provides the final prediction and a natural-language rationale.

Experimental Evaluation

ZARA was rigorously tested across 8 public HAR benchmarks, ranging from basic daily activities (UCI-HAR, Shoaib) to complex, multi-class datasets (WISDM, DSADS). These benchmarks include diverse sensor placements such as the wrist, chest, and ankle.

Key Results

  • State-of-the-Art Performance: ZARA achieved an average macro F1 score of 81.4% across all benchmarks.
  • Significant Gains: It outperformed the strongest zero-shot baseline (UniMTS) by 2.53x in macro F1 and improved accuracy by an average of 2.07x.
  • Robustness in Complexity: On the challenging DSADS dataset (19 classes), ZARA maintained 84.2% accuracy, while most baselines performed near chance levels.

Key Insights and Interpretability

ZARA's success stems from turning a signal processing problem into a logical comparison task. By providing the LLM with structured feature distributions and retrieved examples, it overcomes the "sensor literacy" gap. Unlike black-box neural networks, ZARA provides a "Knowledge-Augmented Rationale," explaining its choice based on specific physical characteristics (e.g., vertical acceleration cycles) that are verifiable by humans.

Limitations and Future Work

While ZARA sets a new standard for zero-shot HAR, challenges remain. The multi-agent pipeline introduces latency that may hinder real-time, on-device use. Additionally, the system currently relies on high-tier commercial LLMs. Future research will focus on optimizing latency and exploring knowledge distillation to smaller, open-source models to reduce operational costs and improve privacy.

Summary updated

Abstract

Motion sensor time-series are central to human activity recognition (HAR), with applications in health, sports, and smart devices. However, existing methods are trained for fixed activity sets and require costly retraining when new behaviours or sensor setups appear. Recent attempts to use large language models (LLMs) for HAR, typically by converting signals into text or images, suffer from limited accuracy and lack verifiable interpretability. We propose ZARA, the first agent-based framework for zero-shot, explainable HAR directly from raw motion time-series. ZARA integrates an automatically derived pair-wise feature knowledge base that captures discriminative statistics for every activity pair, a multi-sensor retrieval module that surfaces relevant evidence, and a hierarchical agent pipeline that guides the LLM to iteratively select features, draw on this evidence, and produce both activity predictions and natural-language explanations. ZARA enables flexible and interpretable HAR without any fine-tuning or task-specific classifiers. Extensive experiments on 8 HAR benchmarks show that ZARA achieves SOTA zero-shot performance, delivering clear reasoning while exceeding the strongest baselines by 2.53x in macro F1. Ablation studies further confirm the necessity of each module, marking ZARA as a promising step toward trustworthy, plug-and-play motion time-series analysis. Our codes are available at https://github.com/zechenli03/ZARA.
BibTeX
@inproceedings{li2026zara,
  author = {Zechen Li and Baiyu Chen and Hao Xue and Flora D. Salim},
  title = {{ZARA: Zero-shot Motion Time-Series Analysis via Knowledge and Retrieval Driven LLM Agents}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.684},
}

Loading related content...

Figure