Body Pose Sonification for a View-Independent Auditory Aid to Blind Rock Climbers

Joseph Ramsay, Hyung Jin Chang

Abstract

Rock climbing is a sport in which blind people have traditionally found it extremely difficult to excel due to the high degree of visual problem solving required, and also the requirement to climb with a sighted assistant. We present a system which automates the role of the sighted assistant in order to provide blind people with the freedom to climb and train on their own. We address climbing-specific limitations of a state-of-the-art skeleton tracking system, and discuss the ways in which we mitigated these limitations using post-processing techniques tuned specially for a climbing scenario. We also describe the auditory feedback system used to instruct the blind climber, and demonstrate that a user can learn to follow it in a relatively short time by showing a significant improvement in performance over just a few trials with the system.
BibTeX
@inproceedings{ramsay2020body,
  author = {Joseph Ramsay and Hyung Jin Chang},
  title = {{Body Pose Sonification for a View-Independent Auditory Aid to Blind Rock Climbers}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2020},
  doi = {10.1109/wacv45572.2020.9093462},
}

Loading related content...

Figure