Face2QR: A Unified Framework for Aesthetic, Face-Preserving, and Scannable QR Code Generation

Xuehao Cui, Guangyang Wu, Zhenghao Gan, Guangtao Zhai, Xiaohong Liu

TL;DR

Face2QR is a training-free generative pipeline that creates aesthetic, scannable QR codes while faithfully preserving human facial identity. By combining Stable Diffusion with intelligent QR module reshuffling and a spatially dynamic latent optimization, it achieves state-of-the-art performance in balancing visual appeal, face recognition, and functional scannability.

What Problem Does This Paper Address?

Standard QR codes are functional but visually unappealing, consisting of stark black-and-white modules. While there is a growing demand for personalized QR codes—especially those featuring human faces for social media or marketing—existing methods struggle to balance three competing objectives:

  1. Aesthetics: The QR code should look like a high-quality, stylized image.
  2. Face Identity Preservation: The person in the image must remain recognizable.
  3. Scannability: The code must be decodable by standard QR readers.

Previous approaches often fail because QR patterns (the black squares) conflict with facial features. If you remove the patterns from the face, the code becomes unscannable; if you keep them, the face becomes distorted and unrecognizable. Generative models like Stable Diffusion can produce beautiful images but lack the fine-grained control needed to keep a face "legal" for both a human eye and a QR scanner.

Methods

The authors introduce Face2QR, a three-stage, training-free pipeline designed to harmonize these conflicting goals.

The Face2QR pipeline consists of three stages: IDQR for initial generation, IDRS for resolving pattern conflicts, and IDSE for scannability refinement.

1. ID-Refined QR Integration (IDQR)

The process begins with an SDXL-based framework. It uses two specialized control networks: InstantID to inject facial identity and a QR Controller to guide the overall luminance distribution toward a QR pattern. This produces an initial image that looks good but often has a high decoding error rate (over 40%).

2. ID-Aware QR ReShuffle (IDRS)

To fix the high error rate without ruining the face, the authors leverage the inherent flexibility of QR code encoding. They identify the facial region using a pre-trained face recognition model and "fix" the modules in that area to match the image's natural brightness. They then reshuffle the remaining black and white modules in the background to satisfy the QR message requirements. This "ID-aware blueprint" is then used to regenerate the image, cutting the error rate by more than half.

3. ID-Preserved Scannability Enhancement (IDSE)

The final stage is a post-processing step in the latent space. Using gradient descent, the model optimizes the image's latent code to further reduce errors. Crucially, they use a spatially dynamic loss function. It applies a gentler, smaller-variance Gaussian kernel and lower weights to the facial region to prevent distortion, while applying stronger optimization to the background to ensure the QR reader can "see" the modules clearly.

Datasets

The framework is training-free, but several datasets were used for evaluation and component pre-training:

  • MS-COCO: Used to pre-train the VGG-19 architecture for the aesthetic content loss in the IDSE stage.
  • Comparative Analysis Dataset: A custom set of 200 uniquely stylized QR images (1024x1024) with diverse artistic styles.
  • Face Sources: The authors tested the pipeline on real human faces (from Pexels), realistic generated faces (from StyleGAN2), and cartoon faces to demonstrate generalizability.

Key Insights and Ideas

  • Reshuffling is Better Than Blending: Instead of trying to force a QR pattern onto a face, the authors realized they could move the "information" of the QR code elsewhere. By reshuffling modules in the background, they keep the face clear while maintaining the total data integrity required for decoding.
  • Spatially Aware Optimization: A uniform loss function is the enemy of facial identity. By making the scannability loss "aware" of where the face is, the model can be aggressive in the background (where it doesn't matter if a pixel shifts slightly) and conservative on the face (where every pixel matters for recognition).
  • Training-Free Flexibility: By building on top of pre-trained models like SDXL and InstantID, the pipeline is highly adaptable to different artistic styles and facial types without needing expensive retraining.

Key Results

Face2QR achieves state-of-the-art (SOTA) performance across all key metrics:

  • Scannability: It maintains an average success rate of over 94% across various sizes (3cm to 7cm) and scanning angles (up to 45°), comparable to the best existing methods.
  • Identity Preservation: In quantitative tests using ArcFace feature distance, Face2QR significantly outperformed competitors like Text2QR ($d=0.51$ vs $0.43$), meaning the generated faces were much closer to the originals.
  • Aesthetics: It scored highest on the AesBench metric (90.1) compared to ArtCoder (62.0) and Text2QR (87.5).
  • User Preference: In a subjective study, users preferred Face2QR's results over 80% of the time for both identity preservation and aesthetic quality.
Figure 1: Face images (first row) and QR code images (second row) generated by Face2QR. Our QR codes not only faithfully maintain face ID, but also showcase remarkable scanning resilience and aesthetic quality.

Limitations and Future Work

While powerful, Face2QR is still tied to the limitations of its base generative models. It can occasionally produce inconsistent results or require significant computing power for high-resolution images. The authors also noted "bad cases" where the input face and text prompt were in direct conflict (e.g., providing a woman's face but a prompt for a "male man"). Future work will likely focus on improving generation speed and further refining the harmony between complex prompts and facial features.

Summary updated

Abstract

Existing methods to generate aesthetic QR codes, such as image and style transfer techniques, tend to compromise either the visual appeal or the scannability of QR codes when they incorporate human face identity. Addressing these imperfections, we present Face2QR-a novel pipeline specifically designed for generating personalized QR codes that harmoniously blend aesthetics, face identity, and scannability. Our pipeline introduces three innovative components. First, the ID-refined QR integration (IDQR) seamlessly intertwines the background styling with face ID, utilizing a unified Stable Diffusion (SD)-based framework with control networks. Second, the ID-aware QR ReShuffle (IDRS) effectively rectifies the conflicts between face IDs and QR patterns, rearranging QR modules to maintain the integrity of facial features without compromising scannability. Lastly, the ID-preserved Scannability Enhancement (IDSE) markedly boosts scanning robustness through latent code optimization, striking a delicate balance between face ID, aesthetic quality and QR functionality. In comprehensive experiments, Face2QR demonstrates remarkable performance, outperforming existing approaches, particularly in preserving facial recognition features within custom QR code designs. Codes are available at $\href{https://github.com/cavosamir/Face2QR}{\text{this URL link}}$.
BibTeX
@inproceedings{cui2024face,
  author = {Xuehao Cui and Guangyang Wu and Zhenghao Gan and Guangtao Zhai and Xiaohong Liu},
  title = {{Face2QR: A Unified Framework for Aesthetic, Face-Preserving, and Scannable QR Code Generation}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2024},
  url = {https://openreview.net/forum?id=rvBabL7DUu},
}
Figure