TL;DR
This paper demonstrates that training fall-detection models on a diverse pool of existing datasets significantly improves their performance on new, unseen environments. More importantly, this "multi-source" training provides a superior starting point for "target-site personalization," allowing models to adapt effectively to a specific home or hospital ward using as few as five local examples.
What Problem Does This Paper Address?
Deploying video-based fall detection in the real world is notoriously difficult due to domain shift. A model trained on a tidy laboratory dataset often fails when faced with the unique lighting, camera angles, and furniture layouts of a specific nursing home or private residence.
Furthermore, falls are "rare events." It is impractical to ask a user to record dozens of falls in their own home just to train a local model. This creates a "cold start" problem: how do we get a high-performing, personalized detector when local labels are scarce? The authors investigate whether "pre-training" on a collection of diverse public datasets (multi-source training) helps models generalize to new sites (zero-target transfer) and whether it makes subsequent fine-tuning with limited local data (personalization) more effective.
Methods
The researchers employed a Leave-One-Family-Out (LOFO) cross-validation strategy. They treated nine distinct dataset "families" as separate domains. For each experiment, they trained on eight families and held out the ninth as the "unseen" target site.
They evaluated two main categories of models:
- Vision-Language Models (VLMs): Qwen2.5-VL, LLaVA-OneVision, and InternVL3.5. These models were adapted using Low-Rank Adaptation (LoRA), which updates only a tiny fraction of the model's parameters, making them efficient to personalize.
- Visual Baselines:
- CLIP-temporal: A logistic regression classifier built on top of frozen CLIP features.
- VideoMAE: A transformer-based video model that was fully fine-tuned.
- Pose-TCN: A Temporal Convolutional Network that operates on 2D body keypoints extracted by a pose detector.
The evaluation followed a four-stage pipeline:
- Stage 1 (Source Training): Training on the eight source families.
- Stage 2 (Zero-target): Testing the source-trained model on the held-out family with no local data.
- Stage 3 (Few-shot Personalization): Updating the model using 5, 10, or 20 local examples per class.
- Stage 4 (Full-target): Updating using all available local training data.
Datasets
The study utilized a harmonized set of 2,697 video clips drawn from nine sources:
- OmniFall Benchmark: Includes eight families (GMDCSA-24, MCFD, OCCU, CAUCAFall, EDF, Le2i, OOPS-Fall, and UP-Fall). These range from staged laboratory falls to "in-the-wild" accident videos from the web.
- AI-Hub: A large-scale dataset featuring staged falls in outdoor CCTV environments with varying weather and lighting.
The authors mapped all labels to a binary task: a "positive" clip contains the actual transition into a fall, while "negative" clips show activities like walking, sitting, or even lying still on the floor after a fall.
Key Insights and Ideas
- The "Warm Start" Advantage: Source initialization is almost always better than starting from a "base" state (Target-only). Even if the source data looks different from the target, the model learns a general "concept" of a fall that accelerates local learning.
- VLMs are Robust but Not Magic: While InternVL achieved the highest absolute scores, the benefit of source initialization was universal across architectures. Even the relatively simple Pose-TCN saw a massive boost (+0.109 AUROC) from source training when only five local examples were available.
- The "Post-Fall" False Positive: A major finding was that models often continue to assign high "fall" scores long after the person has landed. Because the models are trained to detect the action of falling, they struggle to distinguish between the transition and the resulting state of being on the floor.
- Thresholding Matters: The standard 0.5 decision threshold is rarely optimal for a new site. The authors found that unsupervised methods (like Otsu’s method) could improve binary accuracy without needing any local labels, simply by analyzing the distribution of scores at the new site.
Key Results
- Zero-Target Gains: Multi-source training significantly raised the mean AUROC on unseen sites compared to frozen, off-the-shelf models. For example, Qwen improved from 0.821 to 0.912, and InternVL reached a high of 0.962.
- Few-Shot Success: With just 5 examples per class, the "Source-initialized" models consistently outperformed "Target-only" models. The AUROC benefit was statistically significant for Qwen, CLIP-temporal, VideoMAE, and Pose-TCN.
- Diminishing Returns at Scale: As the amount of local target data increases (Full-target training), the gap between starting from a source-trained model versus a base model narrows. By the time a site has a full training set, the "Target-only" models often catch up, though VLMs like Qwen and LLaVA still maintained a slight edge from their source initialization.
Limitations and Future Work
- Staged vs. Real Falls: Most of the data comes from actors. Real-world falls involving elderly individuals with mobility aids (walkers, canes) may present different visual signatures.
- Temporal Precision: The study used fixed-length clips. In a real monitoring system, the model must process a continuous stream. Future work needs to address how to suppress duplicate alerts and handle the "long lie" (staying on the floor) after a fall.
- Deployment Constraints: Running large VLMs like InternVL (8B parameters) in real-time on edge devices remains a significant engineering challenge.
- Longitudinal Adaptation: This study used static partitions. A real-world system would likely use "online" learning, adapting as caregivers provide feedback on alerts over months of deployment.
Summary updated