Supervised Fractional-Order Embedding Multiview Canonical Correlation Analysis via Ordinal Label Dequantization for Image Interest Estimation

Masanao Matsumoto, Naoki Saito, Keisuke Maeda, Takahiro Ogawa, Miki Haseyama

Introduction

Estimating user interest in images is a critical task for personalized content delivery and digital marketing. Modern approaches often leverage Multiview Canonical Correlation Analysis (MVCCA) to integrate diverse data sources, such as high-dimensional visual features from images and biological signals like eye-gaze data. By finding a common latent space where these views are maximally correlated, researchers can more accurately predict user preferences. However, traditional supervised CCA methods face a significant "dimensionality bottleneck" when dealing with simple ordinal labels (e.g., 1-4 ratings), leading to substantial information loss.

The Dimensionality Bottleneck in Supervised CCA

In supervised MVCCA, the dimensionality of the integrated feature space is constrained by the view with the lowest number of dimensions. When user interest is represented as a low-dimensional ordinal label, the high-dimensional visual (e.g., CNN features) and biological (e.g., eye-tracking) data are forced into a narrow latent space. This compression discards nuanced information present in the richer data views. Additionally, biological data is often noisy, which can cause "eigenvalue bloating" in standard covariance matrix calculations, leading to suboptimal projections.

Methodology: SFEMCCA-OLD

To address these challenges, the researchers proposed Supervised Fractional-Order Embedding Multiview Canonical Correlation Analysis via Ordinal Label Dequantization (SFEMCCA-OLD). This framework introduces two key innovations:

Ordinal Label Dequantization (OLD)

The OLD component treats ordinal labels not as fixed values, but as "quantized" versions of a higher-dimensional latent representation. The model iteratively expands the label dimensionality (e.g., from 4 to 64) by estimating "dequantized" features that maximize correlation with visual and gaze views. This process effectively uncovers implicit "sub-grades" of interest reflected in user behavior.

The iterative process of Ordinal Label Dequantization (OLD) for expanding label dimensionality.

Fractional-Order Embedding (FE)

To mitigate the impact of noise, the method employs fractional-order embedding during singular value decomposition (SVD). By raising singular values to a fractional power, the model suppresses disproportionately large eigenvalues caused by noise, resulting in more robust projection matrices.

Experimental Results

The SFEMCCA-OLD framework was evaluated using a dataset of 100 images and eye-gaze data from 10 subjects. Visual features were extracted using a pre-trained Inception-v3 model (2,048 dimensions), while eye-gaze features comprised 110 dimensions.

Performance Comparison

The model demonstrated superior performance in Mean Absolute Error (MAE) across two scenarios:

  • Binary Interest: SFEMCCA-OLD achieved an MAE of 0.261, outperforming standard CCA (0.298) and sMVCCA (0.406).
  • 4-Level Interest: The model achieved an MAE of 0.617, significantly better than state-of-the-art methods like LapMCCA (0.700) and DeepCCA (0.770).

Impact of Dimensionality

A key finding was that increasing the dequantized label dimension (e.g., from 4 to 64) consistently improved estimation accuracy. This confirms that the dimensionality constraint was indeed a primary bottleneck in previous supervised CCA applications.

Conclusion

SFEMCCA-OLD represents a significant advancement in image interest estimation by overcoming the dimensionality limitations of traditional supervised CCA. By dequantizing ordinal labels and employing fractional-order embeddings for noise robustness, the model preserves more information and provides more accurate interest predictions. Future work will focus on extending this dequantization approach to categorical labels and matrix-based 2DCCA methods.

Summary updated

Abstract

Supervised fractional-order embedding multiview canonical correlation analysis via ordinal label dequantization (SFEMCCA-OLD) for image interest estimation is presented in this paper. SFEMCCA-OLD is a CCA method that realizes accurate integration of features including low-dimensional ordinal label features. In general, since information is lost due to a limitation of the number of classes, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i.e.</i> , the dimension of ordinal label information is smaller than those of other features, derivation of highly accurate integration of features is difficult. In SFEMCCA-OLD, the dimension of the ordinal label information can be increased by estimation of the canonical correlation between multiview features. We call this approach ordinal label dequantization. In addition, by introducing a fractional-order technique, our method can calculate optimal projections for noisy data such as real data. Experimental results show that the accuracy of SFEMCCA-OLD for image interest estimation is better than that of recent CCA-based methods.
BibTeX
@article{matsumoto2021supervised,
  author = {Masanao Matsumoto and Naoki Saito and Keisuke Maeda and Takahiro Ogawa and Miki Haseyama},
  title = {{Supervised Fractional-Order Embedding Multiview Canonical Correlation Analysis via Ordinal Label Dequantization for Image Interest Estimation}},
  journal = {IEEE Access},
  year = {2021},
  doi = {10.1109/access.2021.3055868},
}
Figure