The IPIN 2019 Indoor Localisation Competition - Description and Results

Francesco Potortì, Sangjoon Park, Antonino Crivello, Filippo Palumbo, Michele Girolami, Paolo Barsocchi, Soyeon Lee, Joaquín Torres-Sospedra, Antonio Ramón Jiménez Ruiz, Antoni Pérez-Navarro, Germán Martín Mendoza-Silva, Fernando Seco, Miguel Ortiz, Johan Perul, Valérie Renaudin, Hyunwoong Kang, Soyoung Park, Jae Hong Lee, Chan Gook Park, Jisu Ha, Jaeseung Han, Keunhye Kim, Yonghyun Lee, Seunghun Gye, Keumryeol Lee, Eun-Jee Kim, Jeongsik Choi, Yang-Seok Choi, Shilpa Talwar, Seong Yun Cho, Boaz Ben-Moshe, Alex Scherbakov, Leonid Antsfeld, Emilio Sansano-Sansano, Boris Chidlovskii, Nikolai Kronenwett, Silvia Prophet, Yael Landay, Revital Marbel, Lingxiang Zheng, Ao Peng, Zhichao Lin, Bang Wu, Chengqi Ma, Stefan Poslad, David R. Selviah, Wei Wu, Zixiang Ma, Wenchao Zhang, Dongyan Wei, Hong Yuan, Jun-Bang Jiang, Shao-Yung Huang, Jing-Wen Liu, Kuan-Wu Su, Jenq-Shiou Leu, Kazuki Nishiguchi, Walid Bousselham, Hideaki Uchiyama, Diego Thomas, Atsushi Shimada, Rin-Ichiro Taniguchi, Vicente Cortés Puschel, Tomás Lungenstrass Poulsen, Imran Ashraf, Chanseok Lee, Muhammad Usman Ali, Yeongjun Im, Gunzung Kim, Jeongsook Eom, Soojung Hur, Yongwan Park, Miroslav Opiela, Adriano J. C. Moreira, Maria João Nicolau, Cristiano G. Pendão, Ivo Silva, Filipe Meneses, António Costa, Jens Trogh, David Plets, Ying-Ren Chien, Tzu-Yu Chang, Shih-Hau Fang, Yu TsaoIEEE Access 2020 · DOI

The IPIN 2019 Indoor Localisation Competition represents a significant milestone in the quest for reliable indoor positioning. Held at the CNR Research Area in Pisa, Italy, this premier event provided a "level playing field" for academic and industrial teams to evaluate their algorithms in complex, real-world environments. Unlike typical lab-based studies, this competition forced technologies to navigate multi-floor buildings and transition between indoor and outdoor spaces under standardized conditions.

The Indoor Localization Challenge

While GPS has transformed outdoor navigation, the "indoor problem" remains a fragmented landscape of competing technologies. Indoor environments are inherently "GPS-denied," requiring systems to synthesize data from a diverse array of sources, including Wi-Fi signals, magnetic fields, barometers, and inertial measurement units (IMUs).

The primary hurdle in this field is not just achieving raw accuracy, but ensuring reproducibility and fairness. Research often suffers from results reported in highly controlled settings. The IPIN competition addresses this by utilizing the standardized EvAAL (Evaluation of AAL Systems through Competitive Benchmarking) framework, requiring participants to operate in an unfamiliar 6,000 m² campus featuring stairs, lifts, and varying lighting conditions.

Methodology and Competition Tracks

The competition was structured into four distinct tracks to evaluate different technological approaches:

  1. On-site Smartphone: Real-time positioning using a handheld device.
  2. On-site Video: Real-time positioning leveraging cameras, often fused with IMUs.
  3. Off-site Smartphone: Post-processing of pre-recorded sensor data.
  4. Off-site Foot-mounted IMU: Post-processing data from high-precision sensors attached to a walker's shoe.

Key Technical Approaches

Top-performing teams consistently utilized Sensor Fusion, combining Pedestrian Dead Reckoning (PDR) with Wi-Fi fingerprinting and barometric pressure for robust floor detection. Advanced filtering techniques, such as Particle Filters and Bayesian Grid-based filters, were employed to smooth noisy sensor inputs.

Innovation was also seen in Visual Pedometry, where systems utilized platforms like ARCore and ARKit to estimate movement via scaled optical flow. Furthermore, some teams integrated Deep Learning, replacing traditional PDR models with Deep Neural Networks to predict displacements directly from raw IMU data.

The Competition Environment

The primary dataset was generated at the CNR Research Area in Pisa, encompassing two buildings across three floors. The total area covered approximately 6,000 m² of indoor space and 1,000 m² of outdoor terrain. Ground truth was established using 75 key points measured with laser meters and georeferenced on a map.

Floor plans of Building A and Building B at the CNR Research Area, Pisa, showing the complex multi-room and corridor layouts used for the competition.

For off-site tracks, organizers provided "GetSensorData" log files, which included raw Wi-Fi, IMU, magnetometer, and barometer readings for training and validation.

Key Insights and Results

The competition revealed several critical insights into the current state of the art:

  • Vision-based Fragility: While highly accurate, vision systems (Track 2) proved sensitive to environmental changes, such as exposure variations when moving between dark corridors and bright outdoor paths.
  • Reliable Floor Detection: The use of barometers has largely solved the floor detection problem, with leading teams achieving near 100% accuracy.
  • The Power of Constraints: The most successful algorithms utilized environmental constraints. By matching sensor data to building maps, they could correct trajectory errors that would otherwise suggest impossible movements through walls.

Performance Summary

The competition used a scoring metric based on the 75th percentile of error, including a 15-meter penalty for incorrect floor detection.

Track Winner Best Score (m)
1 (On-site Smartphone) SNU-NESL 3.8
2 (On-site Video) HANA Micron 3.6
3 (Off-site Smartphone) INTEL LABS 2.3
4 (Off-site Foot-mounted) KIU SNU 1.6

Foot-mounted IMUs remain the gold standard for pure dead reckoning, but smartphone-based systems are closing the gap, particularly through the adoption of Wi-Fi RTT (Round Trip Time) and FTM (Fine Timing Measurement).

Limitations and Future Outlook

Despite the progress, several challenges remain. Most systems still struggle with the "cold start" problem, requiring an initial position to begin tracking. Additionally, the limited survey time available to on-site teams proved to be a bottleneck, particularly for vision-based systems. High-accuracy visual SLAM also presents significant power demands, often draining mobile batteries rapidly.

Future efforts will likely focus on making systems fully autonomous from the first step and improving energy efficiency for computer vision tasks. As the community continues to refine these technologies, the goal remains a seamless, high-accuracy navigation experience that matches the reliability of outdoor GPS.

Summary updated

BibTeX
@article{potorti2020ipin,
  author = {Francesco Potortì and Sangjoon Park and Antonino Crivello and Filippo Palumbo and Michele Girolami and Paolo Barsocchi and Soyeon Lee and Joaquín Torres-Sospedra and Antonio Ramón Jiménez Ruiz and Antoni Pérez-Navarro and Germán Martín Mendoza-Silva and Fernando Seco and Miguel Ortiz and Johan Perul and Valérie Renaudin and Hyunwoong Kang and Soyoung Park and Jae Hong Lee and Chan Gook Park and Jisu Ha and Jaeseung Han and Changjun Park and Keunhye Kim and Yonghyun Lee and Seunghun Gye and Keumryeol Lee and Eun-Jee Kim and Jeongsik Choi and Yang-Seok Choi and Shilpa Talwar and Seong Yun Cho and Boaz Ben-Moshe and Alex Scherbakov and Leonid Antsfeld and Emilio Sansano-Sansano and Boris Chidlovskii and Nikolai Kronenwett and Silvia Prophet and Yael Landay and Revital Marbel and Lingxiang Zheng and Ao Peng and Zhichao Lin and Bang Wu and Chengqi Ma and Stefan Poslad and David R. Selviah and Wei Wu and Zixiang Ma and Wenchao Zhang and Dongyan Wei and Hong Yuan and Jun-Bang Jiang and Shao-Yung Huang and Jing-Wen Liu and Kuan-Wu Su and Jenq-Shiou Leu and Kazuki Nishiguchi and Walid Bousselham and Hideaki Uchiyama and Diego Thomas and Atsushi Shimada and Rin-Ichiro Taniguchi and Vicente Cortés Puschel and Tomás Lungenstrass Poulsen and Imran Ashraf and Chanseok Lee and Muhammad Usman Ali and Yeongjun Im and Gunzung Kim and Jeongsook Eom and Soojung Hur and Yongwan Park and Miroslav Opiela and Adriano J. C. Moreira and Maria João Nicolau and Cristiano G. Pendão and Ivo Silva and Filipe Meneses and António Costa and Jens Trogh and David Plets and Ying-Ren Chien and Tzu-Yu Chang and Shih-Hau Fang and Yu Tsao},
  title = {{The IPIN 2019 Indoor Localisation Competition - Description and Results}},
  journal = {IEEE Access},
  year = {2020},
  doi = {10.1109/access.2020.3037221},
}
Figure