CSI-TextBench: A Dataset and Benchmark for Language-Grounded Ambient Sensing Perception

Guozhen Zhu, Yuqian Hu, Sakila S Jayaweera, Wei-Hsiang Wang, Beibei Wang, K. J. Ray LiuarXiv 2026 · arXiv

TL;DR

CSI-TextBench is a new large-scale dataset and benchmark designed to bridge the gap between ambient WiFi sensing and natural language. By aligning WiFi Channel State Information (CSI) with over 2 million text descriptions, the researchers have enabled "language-grounded" sensing—allowing AI systems to understand indoor activities, proximity, and identity through flexible, open-vocabulary natural language queries rather than fixed, rigid labels.

What Problem Does This Paper Address?

Ambient sensing via WiFi—specifically using Channel State Information (CSI)—is a powerful, privacy-preserving way to monitor indoor environments. Because WiFi signals are perturbed by human movement, they can "see" activities like walking, falling, or even breathing without the need for intrusive cameras.

However, WiFi sensing has traditionally been stuck in a "closed-set" paradigm. Most systems are trained as specialized classifiers to recognize a handful of fixed labels (e.g., "walking" vs. "sitting"). Unlike the worlds of computer vision or audio, where models like CLIP benefit from billions of images paired with captions, CSI lacks naturally occurring text. This makes it difficult to integrate WiFi sensing into modern multimodal AI systems that use language as a universal interface. The authors of this paper aim to turn CSI into a "language-indexed perception layer" that can respond to diverse, complex, and even unseen human queries.

Methods

The authors propose a unified pipeline to convert heterogeneous CSI data into a standardized signal-text format.

The Architecture: They employ a dual-encoder design. A CSI encoder (based on a 1D-adapted ResNet-18) processes the raw signal, while a frozen, pre-trained text encoder (CLIP ViT-B/32) handles the language. By keeping the text encoder frozen, they ensure the model learns to map physical signals into a well-established semantic space.

Alignment Strategies: The researchers benchmarked five different alignment objectives from four paradigm families to see which best handles the "many-to-many" nature of CSI data (where one signal can be described by many sentences, and one sentence can apply to many signals):

  • Contrastive: CLIP-style InfoNCE and SigLIP.
  • Anchor-based: Adapting LanguageBind to align signals to a fixed text space.
  • Non-contrastive: Barlow Twins, which focuses on cross-correlation.
  • Reconstructive: A masked cross-modal prediction objective.

Text Construction: Since CSI doesn't have native captions, the authors created a two-level text pipeline. Level 1 uses hand-written templates for single attributes (e.g., "Someone is walking"). Level 2 creates composite captions that combine multiple attributes into a single scene description (e.g., "User 3 is walking at approximately 1.5m from the sensor").

CSI-TextBench overview showing sensing tasks, example events, and enabled capabilities like attribute-level QA and composite scene understanding.

Datasets

The authors introduced CSI-TextBench, which unifies three major public WiFi sensing benchmarks:

  1. CSI-Bench: Large-scale "in-the-wild" data covering activity, proximity, and user ID.
  2. XRF55: 55 fine-grained indoor activities.
  3. SignFi: 276 sign-language gesture classes.

In total, the dataset contains 285,790 CSI samples across 7 tasks, paired with 2,291 unique descriptions, resulting in roughly 2.0 million training tuples.

Table 2: Dataset statistics for CSI-TextBench. The three multitask CSI-Bench tasks (HAR, Proximity, User ID) share the same 55,684 underlying CSI recordings; each appears separately. The 285,790 task-level instances correspond to 174,422 unique recordings.

Key Insights and Ideas

  • Language Alignment Matches Supervised Performance: The most striking finding is that label-free, text-aligned encoders can match the accuracy of traditional supervised classifiers. This proves that natural language is a viable and powerful supervisory signal for physical sensing.
  • Contrastive Learning is King: Among the methods tested, CLIP-style InfoNCE performed best. It not only achieved high accuracy but also organized the embedding space more reliably for open-set tasks.
  • The Power of Compositionality: A single unified model can reason about multiple attributes (who, what, where) simultaneously. Interestingly, the model performed better at retrieving a full composite scene than it did at answering individual questions about each attribute and combining them. This suggests the model learns to represent the "scene" as a holistic entity.
  • Open-Vocabulary Flexibility: The model can respond to "layperson" language it never saw during training (e.g., "Someone just took a tumble" instead of "Fall detected"), provided the prompts are "anchored" in a natural sentence structure.

Key Results

  • Zero-Shot Accuracy: The CLIP InfoNCE model achieved zero-shot classification accuracy within 0.3 to 2.4 percentage points of fully supervised models across most tasks. For example, it reached 97.8% on activity recognition (HAR) and 100% on user identification.
  • Composite Scene Understanding: In a complex setting with 140 possible scene combinations (Activity × Proximity × User), the unified model achieved 86.9% top-1 accuracy, significantly outperforming ensembles of task-specific classifiers.
  • Few-Shot Transfer: On the difficult 276-class SignFi dataset, the model's zero-shot performance on unseen classes was low (11.3%), but it rapidly improved to 74.6% with just 10 labeled examples, demonstrating strong representation learning.
Table 3: Zero-shot classification accuracy (%, mean±std over 3 seeds).

Limitations and Future Work

Despite the impressive results, the authors note several challenges:

  • Domain Sensitivity: WiFi signals are notoriously sensitive to the specific environment and hardware used. While language alignment helps, it doesn't fully solve the "domain shift" problem when moving a model to a new house or different WiFi router.
  • Fine-Grained Limits: Distinguishing between very similar gestures or activities using only template-based text descriptions remains difficult.
  • Specialized Language: The model struggles with highly technical or "sensor-grounded" language (e.g., "Quasi-periodic Doppler modulation") because the underlying text encoders (like CLIP) weren't trained on such niche scientific data.

Future work will likely focus on domain-aware text representations and alignment objectives specifically tailored to the unique physics of wireless signals.

Summary updated

Abstract

Indoor context deciphered by ambient sensing underpins many applications, from assisted living to smart buildings, yet remains weakly connected to the language-aligned paradigm of modern multimodal systems. Through billions of IoT devices, ambient WiFi Channel State Information (CSI) captures the indoor context of human activity and environmental dynamics passively via ubiquitous, always-on infrastructure. However, unlike vision or audio, CSI lacks naturally paired text like captions or transcripts, limiting its integration with language-driven AI systems. We present CSI-TextBench, a large-scale dataset and benchmark for aligning CSI data with natural language, unifying 3 sensing benchmarks into 285,790 samples across 7 tasks and pairing them with 2,291 hierarchically structured, label-derived descriptions via many-to-many assignment to construct 2.0 million CSI-text training tuples. Benchmarking 5 alignment methods from 4 paradigm families, we show that contrastive CSI-text alignment matches supervised classifiers without class labels, adapts to unseen classes with few samples, and responds to open-vocabulary natural-language prompts. A single unified model supports compositional scene understanding, answering joint queries over activity, proximity, and identity, achieving 86.9% top-1 accuracy on 140 composite scenes, outperforming task-specific classifier ensembles. CSI-TextBench enables ubiquitous ambient sensing as a language-indexed perception layer, enabling indoor contextual awareness in language-driven AI systems.
BibTeX
@misc{zhu2026csi,
  author = {Guozhen Zhu and Yuqian Hu and Sakila S Jayaweera and Wei-Hsiang Wang and Beibei Wang and K. J. Ray Liu},
  title = {{CSI-TextBench: A Dataset and Benchmark for Language-Grounded Ambient Sensing Perception}},
  year = {2026},
  eprint = {2610.34616},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2610.34616},
}
Figure