Camera-free home sensors, such as millimeter-wave radar and simple motion and door sensors, are attractive for monitoring older adults and people with dementia because they record no images and need nothing worn on the body. The activities clinicians most want flagged, like a hesitation before a step or a stalled routine in the kitchen, are rare. They are also fine-grained, and they often never appear in the training set at all. Current radar and ambient systems classify a dozen or so coarse activities and treat each one as an arbitrary integer. That design cannot borrow knowledge from common activities, and it cannot add a new category without collecting new labeled data.
We propose to arrange activity labels in an explicit tree from coarse to fine. Each node of the tree gets a short written definition, and a frozen language model turns that definition into a vector. The sensor model is then trained to match its readings to these vectors. A constraint layer guarantees that a fine-grained prediction never exceeds the confidence of its parent. The central claim is that this combination lets a device-free model recognize rare leaf activities, and leaves added after training from text alone, better than either a hierarchy with symbolic labels or flat language labels. We further claim the advantage grows as both the sensor input and the label counts get sparser.
Core Claim and What Is New
Three existing lines each supply one ingredient. Only the combination addresses the regime of rare and newly added activities seen through sparse sensors.
SHARE Unleashing the Power of Shared Label Structures for Human Activity Recognition decodes wearable-sensor activities as sequences of word tokens such as "open" plus "fridge." It reports 9% to 34% relative error reduction across seven wearable benchmarks. Its labels are flat action–object strings with no parent–child structure, and it was never tested on radar or ambient sensors. This paper motivates our use of language in labels.
Hierarchical methods take a different route. Entailment-cone embeddings Hierarchical Image Classification using Entailment Cone Embeddings raised species-level F1 on the ETHEC butterfly images from 0.4578 to 0.5977. The coherent constraint network C-HMCNN Coherent Hierarchical Multi-Label Classification Networks guarantees parents score at least as high as children and ranked first on 17 of 20 benchmarks. Both treat labels as symbols with no meaning attached, so neither can say anything about a leaf it has never seen. We reuse the C-HMCNN constraint mechanism directly.
On the sensing side, the Star Graph radar model DStar Exploring Spatial-Temporal Representation via Star Graph for mmWave Radar-Based Human Activity Recognition handles radar frames that contain only 10–50 points by linking every point to a fixed virtual center. It reaches 94.27% on 13 flat classes. We reuse it as the sensor encoder.
The closest paper overall is DeSPITE DeSPITE: Exploring Contrastive Deep Skeleton-Pointcloud-IMU-Text Embeddings for Advanced Point Cloud Human Activity Understanding. It aligns LiDAR, IMU, skeleton and text in one shared space, but it uses free captions without hierarchy. It also reports that adding text hurt fine-grained cross-modal matching. That finding constrains our design: we must show when text helps and when it adds noise, rather than assume it always helps.
Mechanism
A sensor encoder $f$ is either DStar for radar or a small Transformer for ambient sensor streams, following Comparison of LSTM and Transformer Models for Activities of Daily Living Recognition using In-Home Ambient Sensors. It maps a window $x$ to a vector. Each taxonomy node $v$ has a one-sentence definition $d_v$, for example "walking forward while swinging the arms," which a frozen text encoder $g$ embeds. The raw score of node $v$ is
$$s_v(x) = \frac{\cos\big(f(x),\, W g(d_v)\big)}{\tau},$$
where $W$ is a learned linear projection shared by all nodes and $\tau$ is a temperature. Because $W$ is shared, a node with no training data still receives a sensible score from its definition alone.
Scores pass through a sigmoid to give $h_v$. The final probability of an internal node is then
$$p_A = \max\Big(h_A,\ \max_{B \in \mathrm{desc}(A)} h_B\Big),$$
which is the constraint module from C-HMCNN. Training uses its max-constraint loss, so a parent can hand its error signal to its most confident child.
Adding a new leaf after training requires only writing its definition and attaching it under a parent. No weights change.
Research Program
The first step builds three-level taxonomies over existing label sets. The 13 radar activities become locomotion, upper-body and whole-body branches; the walking variants, for instance, sit under locomotion. The 12 ambient daily activities become rest, household routine and away. A fixed prompt-and-review protocol writes the definitions, and a second, independently written set measures how sensitive results are to wording.
The second step trains under a controlled sparsity protocol. A set of leaves is held out, and their training windows are cut to 0, 1, 5 or 10 examples. Separately, radar frames are thinned by randomly dropping points to 50% and 25% of the original density.
The third step varies one ingredient at a time to separate their contributions. Real definitions are swapped for random vectors on the same tree, which keeps the hierarchy and removes meaning. Definitions are shuffled across nodes, which keeps realistic text and removes the correct meaning. Finally, the hierarchy is removed while the language is kept.
The research file also contains a direction on amyloid-beta (Aβ) and alpha-synuclein (α-Syn) protein variants. That work Amyloid-β modulates the phase separation and aggregation of α-synuclein and Recruitment of Aβ into α-Synuclein Condensates Catalyzes Primary Nucleation of α-Synuclein Aggregation shows that mixed Aβ and α-Syn biology is mechanistically varied. That variety is one reason to expect patients' behavioral presentations to differ and to fall in the long tail. It motivates the fine-grained, extensible taxonomy and nothing more. No dataset in the file links protein variants to sensor signals, and we make no such claim. As a modest step toward clinical relevance, healthy volunteers would perform scripted proxies on the same radar setup, such as pausing mid-stride or starting a reach and aborting it. These proxies would serve only as zero-shot leaves.
Baselines
The naive reference is DStar with a flat softmax and ordinary cross-entropy. It is trained on exactly the same reduced data, with no help for rare classes. On the full radar data the reported number is 94.27% overall accuracy (DStar, Gao et al.). We expect to match that when nothing is held out. On held-out leaves the naive model scores zero by construction, so it sets the floor.
The strongest flat competitor is the same encoder with a class-balanced focal loss. Its re-weighting strength is tuned on the validation split with the same number of trials our method gets. This is what a practitioner would try first.
On ambient data, the reference is the encoder-only Transformer of Abdalnabi et al. They report 0.91 AUC-ROC and 64% accuracy for their harder participant. We use their 16/3/3-day temporal split.
The strongest hierarchical competitor is C-HMCNN on integer labels with the same encoder and tree. It shares our constraint layer and loss and lacks only the language. It is the most dangerous baseline for the few-shot condition.
The strongest language competitor is SHARE's token decoder, retrained on radar features and run with constrained decoding. It reports 9%–34% relative error reduction on wearables. No reported result was found on Lacuna for SHARE, C-HMCNN, or any hierarchical method on mmWave radar or ambient-sensor data.
The decisive ablations are the random-vector and shuffled-definition versions of our model.
All methods share encoder size, training epochs, and a fixed hyperparameter search budget. Each is run with five seeds. Beating a baseline means a higher macro-F1 on held-out leaves at 1 to 10 shots, with seed intervals that do not overlap. It also requires no loss of more than one point of overall accuracy on the full label set, and zero coherence violations. In the zero-shot condition only the language-based methods can score at all. There we compare against SHARE and the shuffled-definition control.
Why not just…?
Why not just use a flat classifier with focal loss or oversampling? Re-weighting changes how much each example counts but adds no information about a class with five examples. It cannot predict a class with none. The evidence that structure matters beyond loss weighting comes from images and text. On ETHEC images, hierarchy raised rare-leaf F1 from 0.4578 to 0.5977 (Dhall et al.). On text, a hierarchy-aware margin loss raised Macro-F1 on EURLEX57K from 27.91 to 28.77 Hierarchy-aware Biased Bound Margin Loss Function for Hierarchical Text Classification. Whether this carries over to radar is exactly what the focal-loss baseline tests.
Why not just use C-HMCNN with symbolic labels, without any language? This is the real threat. With 5–10 examples per leaf, the hierarchy alone may capture most of the gain. If our model does not beat it at those shot counts, we will say so and narrow the claim. The contribution would then rest on the zero- and one-shot leaves and on adding leaves after training, which symbolic labels cannot support by construction. The random-vector ablation settles the question directly. It keeps everything identical except whether label vectors carry meaning.
Why not just apply SHARE's flat token decoder to radar? Shared words already carry some implicit structure; "walking" appears in several labels, for example. SHARE, however, has no constraint between levels, so it can emit a fine-grained label that contradicts the coarse one. Its decoder also relies on the phrasing of label names. The DeSPITE result warns that text can blur fine physical distinctions. We report coherence violations and fine-leaf F1 for both methods. We also run the two independently written definition sets to measure sensitivity to wording.
Why not just use hyperbolic entailment cones? Cones give tree geometry but, as used by Dhall et al., the labels have no meaning. They therefore share C-HMCNN's inability to handle unseen leaves. Our taxonomies are only three levels deep with a few dozen nodes, so the room that hyperbolic space provides for deep trees is unlikely to matter here. We include cone geometry as a variant of the matching head instead of a separate system.
Proposed Experiments and Decisive Checks
The primary benchmark is the TI-IWR6843ISK radar dataset of Gao et al. (5 participants, 13 activities, about 197,550 frames), evaluated with leave-one-participant-out splits. If the data cannot be obtained, we will collect a matching set with the same sensor and the same Raspberry Pi 4 deployment. The secondary benchmark is the 22-day in-home ambient dataset with its temporal split. PAMAP2 and Opportunity serve as a wearable sanity check, so we can confirm that our SHARE reimplementation behaves sensibly before trusting its radar results.
Metrics are leaf-level macro-F1 at each shot count, overall accuracy, AUC-ROC on ambient data, coherence violation rate, and inference latency on the Pi.
Two decisive checks would change the conclusion. The first is the language test. If real definitions do not beat random vectors and shuffled definitions on 0–5-shot leaves, the language component is not doing work. The second is the two-sparsity interaction. If the gap between our model and C-HMCNN does not widen as radar point density drops from full to 25%, the gap claim fails. The prediction is that it widens, because thinner input makes the encoder less reliable and pushes more weight onto the label prior.
The scripted symptom proxies are a final zero-shot test. Their purpose is to show whether a clinician could add a new category by writing a sentence. They are not evidence about disease.
References
- Unleashing the Power of Shared Label Structures for Human Activity Recognition
- Hierarchical Image Classification using Entailment Cone Embeddings
- Coherent Hierarchical Multi-Label Classification Networks
- Exploring Spatial-Temporal Representation via Star Graph for mmWave Radar-Based Human Activity Recognition
- DeSPITE: Exploring Contrastive Deep Skeleton-Pointcloud-IMU-Text Embeddings for Advanced Point Cloud Human Activity Understanding
- Comparison of LSTM and Transformer Models for Activities of Daily Living Recognition using In-Home Ambient Sensors
- Amyloid-β modulates the phase separation and aggregation of α-synuclein
- Recruitment of Aβ into α-Synuclein Condensates Catalyzes Primary Nucleation of α-Synuclein Aggregation
- Hierarchy-aware Biased Bound Margin Loss Function for Hierarchical Text Classification