Iris Liveness Detection Competition (LivDet-Iris) - The 2023 Edition

Patrick Tinsley, Sandip Purnapatra, Mahsa Mitcheff, Aidan Boyd, Colton R. Crum, Kevin W. Bowyer, Patrick J. Flynn, Stephanie Schuckers, Adam Czajka, Meiling Fang, Naser Damer, Xingyu Liu, Caiyong Wang, Xianyun Sun, Zhaohua Chang, Xinyue Li, Guangzhe Zhao, Juan E. Tapia, Christoph Busch, Carlos M. Aravena, Daniel Schulz

Introduction

Biometric authentication systems, particularly iris recognition, are increasingly integrated into secure environments. However, their reliability hinges on Presentation Attack Detection (PAD)—the ability to distinguish a genuine human eye from a spoof. The 2023 LivDet-Iris competition highlights a critical shift in this landscape: the emergence of high-fidelity synthetic attacks that challenge both human perception and state-of-the-art algorithms.

The Problem: The Open-Set Challenge

Traditional PAD systems often struggle with "open-set" scenarios, where they encounter attack types not present during training. While an algorithm might excel at identifying printed photos, it may fail against novel threats like e-ink displays or AI-generated irises. The 2023 edition specifically targeted this vulnerability by introducing GAN-generated spoofs to measure how current systems handle evolving generative AI capabilities.

Methodology and Benchmarking

The competition evaluated eight algorithms submitted by three teams. Notable approaches included:

  • Fraunhofer IGD: Utilized an attention-based pixel-wise binary supervision network.
  • BUCEA: Employed a fusion convolutional neural network enhanced by Contrast Limited Adaptive Histogram Equalization (CLAHE) to emphasize iris textures.

To establish a baseline, organizers trained standard architectures (ResNet101, DenseNet121, Inception_v3, and Vision Transformers) in two modes: "vanilla" (standard training) and "synth" (trained with GAN images). Uniquely, 300 human subjects were also tested to provide a biological benchmark for "natural" detection capabilities.

Data Diversity

The evaluation utilized a massive dataset of 13,332 images (6,500 bona fide and 6,832 Presentation Attack Instruments (PAIs)). The PAIs were split into two categories:

  1. Physical PAIs: High-resolution printouts, textured contact lenses, Kindle e-ink displays, and prosthetic/doll eyes.
  2. Synthetic PAIs: 6,000 images generated via StyleGAN2-ADA and StyleGAN3. These were categorized by quality (Low, Medium, and High) representing different stages of the GAN training process.

Key Insights

The Closing "Uncanny Valley"

High-quality GAN images (MQ and HQ) were virtually indistinguishable from real irises for human observers. Human accuracy on these samples was near 50%, essentially equivalent to a coin flip. This suggests that generative AI has reached a point where visual inspection alone is no longer a reliable defense.

Generalization vs. Memorization

There is a massive performance gap between closed-set and open-set scenarios. Baseline models trained on GAN images achieved nearly 100% accuracy on them. However, models that had not seen GANs during training failed catastrophically. This indicates that current PAD models are often "memorizing" specific artifact patterns rather than "learning" the fundamental essence of liveness.

Humans vs. Machines

Humans are relatively adept at spotting physical spoofs, such as the "bubbly" textures found on cheap prosthetic eyes. However, they are easily fooled by high-end digital fakes that lack physical artifacts. Conversely, the best-performing algorithms outperformed the human average because they were better at identifying subtle digital artifacts that the human eye overlooks.

Competition Results

The competition recognized two winners based on different error metrics:

  • Fraunhofer IGD won on weighted accuracy (ACER1: 37.31%), demonstrating superior performance when accounting for the sample distribution in each category.
  • BUCEA (Algo1) won on non-weighted accuracy (ACER2: 22.15%), performing better when all attack types were treated as equally important.

Human subjects achieved an overall Attack Presentation Classification Error Rate (APCER1) of 39.68%, trailing the top-performing AI.

Conclusion and Future Directions

The 2023 LivDet-Iris competition underscores that iris PAD remains a significant open problem. The high error rates (22–37%) in this edition, compared to previous years where errors were often below 10%, prove that as PAIs become more sophisticated, detection becomes significantly harder.

Future research must focus on improving open-set generalization—developing models that can detect "fakeness" in general rather than looking for specific, known attack signatures. Additionally, multi-modal defenses that combine texture analysis with physiological cues, such as pupil contraction, will be essential to counter high-fidelity static images.

Summary updated

BibTeX
@inproceedings{tinsley2023iris,
  author = {Patrick Tinsley and Sandip Purnapatra and Mahsa Mitcheff and Aidan Boyd and Colton R. Crum and Kevin W. Bowyer and Patrick J. Flynn and Stephanie Schuckers and Adam Czajka and Meiling Fang and Naser Damer and Xingyu Liu and Caiyong Wang and Xianyun Sun and Zhaohua Chang and Xinyue Li and Guangzhe Zhao and Juan E. Tapia and Christoph Busch and Carlos M. Aravena and Daniel Schulz},
  title = {{Iris Liveness Detection Competition (LivDet-Iris) - The 2023 Edition}},
  booktitle = {IJCB},
  year = {2023},
  doi = {10.1109/ijcb57857.2023.10448637},
}
Figure