Spider-Inspired HCCapture: Beware That What You Are Writing on Mobile Devices Is Becoming Prey for Spiders

Wei Fu, Tingting Zhu, Jing Chen, Peidong Jiang, Kun He, Cheng Zeng, Ruiying DuFrontiers in Bioengineering and Biotechnology 2022 · DOI

TL;DR

Researchers have developed Spider-inspired HCCapture, a stealthy side-channel attack that uses a smartphone’s motion sensors to "read" what a user is handwriting on their screen. By mimicking how spiders detect prey through web vibrations, the system achieves a remarkable 96.1% accuracy while remaining effective across different users and physical postures (sitting vs. standing).

What Problem Does This Paper Address?

While security researchers have long known that motion sensors (accelerometers and gyroscopes) can leak sensitive information like PINs or unlock patterns, these attacks usually focus on "restricted areas"—fixed locations like a virtual keyboard.

Handwriting is much harder to intercept because it is "unrestricted." The authors identify two major hurdles:

  1. Chirography Difference: Everyone has unique writing habits, including varying speeds, pressures, and stroke sequences.
  2. Posture Variation: The "noise" in sensor data changes significantly depending on whether a user is sitting or standing, as limb jitters and holding angles shift.

Existing models often require training data from the specific victim to work. This paper aims to create a user-independent system that can speculate handwritten content without any prior knowledge of the victim’s writing style.

Methods

The framework is divided into several sophisticated stages:

  • Bionic Inspiration: Much like a spider sensing vibrations on a web, HCCapture monitors the subtle movements of the device caused by a finger sliding across the glass.
  • Stealthy Detection: The system uses power consumption profiling to detect when a handwriting-related app is in the foreground, allowing the "malware" to remain dormant and save battery until the victim starts writing.
  • Preprocessing & Segmentation: Raw signals are cleaned using wavelet denoising. To separate individual characters, the authors implemented a modified constant false alarm rate (mCFAR) algorithm, which identifies the start and end of each letter based on signal spikes.
  • Posture-Aware Analysis: The system analyzes background noise (using PSD and MFCC) to determine if the user is sitting or standing. It then selects a specialized recognition model tailored to that specific posture.
  • Character Restoration: Instead of matching raw signals to letters, the system breaks characters down into eight basic stroke types (e.g., circle, horizontal, vertical, oblique). It then uses a neural network to classify these strokes and a Markov model to provide linguistic context, improving the guess based on character correlations in common words.

Datasets

The researchers evaluated the system using:

  • Devices: Three popular off-the-shelf smartphones: Google Pixel 2, Samsung S9, and Xiaomi 10.
  • Participants: 18 volunteers (ages 17–32, balanced by gender) divided into three groups.
  • Data Points: Each volunteer wrote the full alphabet [a-zA-Z] 14 times (7 sitting, 7 standing) and 15 random words from the story "Rip Van Winkle."
  • Validation: They used 3-fold cross-validation, specifically excluding the target user’s data from the training set to prove the system is truly user-independent.

Key Insights and Ideas

  • Generality over Specificity: The "aha moment" here is moving away from raw signal patterns. By focusing on the number and type of strokes, the model becomes robust against individual writing quirks. For example, an "A" always has three strokes, regardless of how fast or messy the user writes.
  • Posture as a Filter: The researchers realized that "universal" models fail because standing introduces more noise than sitting. By treating posture as a detectable variable, they can swap in the most accurate sub-model for the current environment.
  • The Power of Context: Adding a Markov model allows the system to "autocorrect" its guesses. If the sensors are unsure between an 'o' and a 'p' in the middle of a word, the model uses the surrounding letters to make a statistically superior choice.

Key Results

  • High Accuracy: Spider-inspired HCCapture reached an accuracy of 96.1% within two guesses.
  • Posture Performance: The system was more accurate for sitting users (96.1%) than standing users (91.1%), as standing introduces more background vibration (limb jitter).
  • Segmentation Success: The mCFAR algorithm successfully detected 88% of handwriting segments compared to raw data.
  • Linguistic Boost: For word-level recognition, the Markov model increased accuracy from 95.7% to 98.0%.

Limitations and Future Work

  • System Survival: The authors note that deeply customized Android versions (common in the Chinese market) have aggressive background-process management that might kill the malicious service.
  • Limited Postures: The study only focused on sitting and standing. Future work could explore more complex scenarios, such as walking, lying down, or using the phone in a moving vehicle.
  • Language Scope: The current research is limited to the English alphabet; expanding this to logographic systems like Chinese characters would present a significantly higher complexity in stroke detection.

Summary updated

Abstract

On mobile devices, the most important input interface is touchscreen, which can transmit a large amount of sensitive information. Many researchers have proven that sensors can be used as side channels to leak touchscreen interactive information. The research of information leakage in the restricted area has been relatively mature, but in the unrestricted area, still there are two issues to be solved urgently: chirography difference and posture variation. We learn from the way spiders perceive prey through the subtle vibrations of their webs; an unrestricted-area handwriting information speculation framework, called spider-inspired handwriting character capture (spider-inspired HCCapture), is designed. Spider-inspired HCCapture exploits the motion sensor as the side-channel and uses the neural network algorithm to train the recognition model. To alleviate the impact of different handwriting habits, we utilize the generality patterns of characters rather than the patterns of raw sensor signals. Furthermore, each character is disassembled into basic strokes, which are used as recognition features. We also proposed a user-independent posture-aware approach to detect the user's handwriting posture to select a suitable one from some pretrained models for speculation. In addition, the Markov model is introduced into spider-inspired HCCapture, which is used as an enhancement feature when there is a correlation between adjacent characters. In conclusion, spider-inspired HCCapture completes the handwritten character speculation attack without obtaining the victim's information in advance. The experimental results show that the accuracy of spider-inspired HCCapture reaches 96.1%.
BibTeX
@inproceedings{fu2022spider,
  author = {Wei Fu and Tingting Zhu and Jing Chen and Peidong Jiang and Kun He and Cheng Zeng and Ruiying Du},
  title = {{Spider-Inspired HCCapture: Beware That What You Are Writing on Mobile Devices Is Becoming Prey for Spiders}},
  booktitle = {Frontiers in Bioengineering and Biotechnology},
  year = {2022},
  doi = {10.3389/fbioe.2022.858961},
}
Figure