Meta-Mining Discriminative Samples for Kinship Verification

Wanhua Li, Shiwei Wang, Jiwen Lu, Jianjiang Feng, Jie Zhou

Abstract

Kinship verification aims to determine biological relationships from facial images. A major hurdle in this field is the extreme data imbalance between positive (related) and negative (unrelated) pairs. Traditional methods often rely on balanced sampling, which discards potentially valuable "hard negative" samples. This paper presents Discriminative Sample Meta-Mining (DSMM), a meta-learning framework that dynamically weights training samples. By utilizing a Meta-Miner to identify discriminative samples from unbalanced batches, the model effectively learns from a broader data distribution, achieving state-of-the-art performance across multiple benchmarks.

Introduction

Determining if two individuals are related based on facial features is a complex task with applications in social media analysis and missing person searches. However, the nature of kinship data creates a massive imbalance: while positive pairs are limited by family size, negative pairs can be generated combinatorially. For instance, 500 positive pairs can yield nearly 250,000 negative pairs.

Most existing models address this by "balanced sampling"—randomly selecting a small subset of negatives to match the number of positives. While this prevents the model from being overwhelmed, it ignores the vast majority of negative data, including "hard negatives" (unrelated individuals with high facial similarity) that are critical for refining decision boundaries.

The DSMM Framework

To overcome the limitations of static sampling, the authors propose Discriminative Sample Meta-Mining (DSMM). This framework consists of two interacting components: a Kinship Model (the primary classifier) and a Meta-Miner (a weight generator).

Comparison between traditional balanced sampling and the proposed Meta-Mining approach.

The Meta-Miner learns to assign importance weights to samples in an unbalanced training batch. Instead of manually defining what constitutes a "hard" sample, the Meta-Miner discovers which samples help the Kinship Model generalize best to a balanced validation set.

The Meta-Learning Loop

The training process follows a three-step cycle at each iteration:

  1. Virtual Training: The Kinship Model performs a temporary update on an unbalanced batch using weights provided by the Meta-Miner.
  2. Meta-Update: This "virtually updated" model is evaluated on a separate, balanced meta-train batch. The resulting loss is used to update the Meta-Miner via second-order gradients, teaching it to provide better weights.
  3. Actual Training: The Meta-Miner generates optimized weights for the original unbalanced batch, and the Kinship Model performs its final update for that iteration.

Architecture: The Relation Module

The Kinship Model utilizes a ResNet-18 backbone for feature extraction. To better capture the nuances of kinship, the authors introduce a Relation Module.

The Kinship Model architecture featuring a CNN backbone and a Relation Module with h-MLP and r-MLP.

As shown in the figure, the module processes face embeddings through a series of Multi-Layer Perceptrons (MLPs). The h-MLP models per-dimension relationships, while the r-MLP produces the final kinship score. This explicit modeling of relationships outperforms simple distance-based metrics.

Experimental Results

The DSMM approach was evaluated on four major kinship benchmarks, consistently outperforming previous state-of-the-art (SOTA) methods:

  • KinFaceW-I: Reached 82.4% accuracy (vs. 78.7% previous best).
  • KinFaceW-II: Reached 93.0% accuracy (vs. 90.4% previous best).
  • TSKinFace: Reached 92.7% accuracy (vs. 87.8% previous best).
  • Cornell KinFace: Reached 90.5% accuracy (vs. 81.7% previous best).

Ablation studies confirmed that the meta-mining strategy is the primary driver of these gains. Interestingly, analysis of the Meta-Miner's behavior showed that it tends to increase the weight of positive samples as training progresses, shifting focus from "easy" negatives to the subtle features defining biological relation.

Conclusion

DSMM demonstrates that data imbalance in kinship verification is not just a challenge to be mitigated, but a source of information to be mined. By automating the discovery of discriminative samples through meta-learning, the framework achieves significant performance boosts without the need for hand-crafted mining heuristics. Future work may involve integrating multi-patch inputs and optimizing the computational cost of the second-order gradient updates.

Summary updated

Abstract

Kinship verification aims to find out whether there is a kin relation for a given pair of facial images. Kinship verification databases are born with unbalanced data. For a database with N positive kinship pairs, we naturally obtain N(N-1) negative pairs. How to fully utilize the limited positive pairs and mine discriminative information from sufficient negative samples for kinship verification remains an open issue. To address this problem, we propose a Discriminative Sample Meta-Mining (DSMM) approach in this paper. Unlike existing methods that usually construct a balanced dataset with fixed negative pairs, we propose to utilize all possible pairs and automatically learn discriminative information from data. Specifically, we sample an unbalanced train batch and a balanced meta-train batch for each iteration. Then we learn a meta-miner with the meta-gradient on the balanced meta-train batch. In the end, the samples in the unbalanced train batch are re-weighted by the learned meta-miner to optimize the kinship models. Experimental results on the widely used KinFaceW-I, KinFaceW-II, TSKinFace, and Cornell Kinship datasets demonstrate the effectiveness of the proposed approach.
BibTeX
@inproceedings{li2021meta,
  author = {Wanhua Li and Shiwei Wang and Jiwen Lu and Jianjiang Feng and Jie Zhou},
  title = {{Meta-Mining Discriminative Samples for Kinship Verification}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year = {2021},
  doi = {10.1109/cvpr46437.2021.01587},
}
Figure