Abstract
In this paper, we first briefly introduce the multidimensional Principal Component Analysis (PCA) techniques, and then amend our previous N-dimensional PCA (ND-PCA) scheme by introducing multidirectional decomposition into ND-PCA implementation. For the case of high dimensionality, PCA technique is usually extended to an arbitrary n-dimensional space by the Higher-Order Singular Value Decomposition (HO-SVD) technique. Due to the size of tensor, HO-SVD implementation usually leads to a huge matrix along some direction of tensor, which is always beyond the capacity of an ordinary PC. The novelty of this paper is to amend our previous ND-PCA scheme to deal with this challenge and further prove that the revised ND-PCA scheme can provide a near optimal linear solution under the given error bound. To evaluate the numerical property of the revised ND-PCA scheme, experiments are performed on a set of 3D volume datasets.
Introduction
Principal Component Analysis (PCA) is a fundamental tool for dimensionality reduction, yet it faces significant hurdles when applied to high-dimensional data like 3D medical scans or multi-view images. Standard PCA requires flattening data into 1D vectors, which destroys spatial relationships and creates computationally prohibitive covariance matrices. While Higher-Order Singular Value Decomposition (HO-SVD) treats data as multi-dimensional tensors, it often hits memory limits during the "unfolding" process required for computation. This paper introduces a reformulated N-Dimensional PCA (ND-PCA) scheme designed to process massive datasets efficiently on standard hardware.
Methodology
The proposed ND-PCA framework incorporates two key innovations:
- Multidirectional Decomposition: Unlike 2D-PCA, which may only analyze correlations along one axis, this scheme applies decomposition across all $N$ dimensions of a tensor simultaneously. This captures the most significant features from every perspective, ensuring a more robust low-rank approximation.
- Matrix Partitioning SVD: To solve the out-of-memory issue, the authors prove that a large matrix can be partitioned into smaller sub-blocks. By performing SVD on these chunks and sorting the resulting singular values, the algorithm identifies the principal eigenvectors of the original massive matrix with a "near-optimal" accuracy that is mathematically bounded.
Experimental Setup
The researchers validated their approach using a 3D range database of human faces from the Face Recognition Grand Challenge (FRGC). 3D range data was embedded into 3D arrays and re-sampled to a size of 180 × 180 × 90. This setup allowed the team to test the algorithm's ability to handle significant data volumes while preserving multi-way correlations such as those between different subjects and viewpoints.
Results and Discussion
The revised ND-PCA demonstrates superior performance in data compression and error reduction. By maintaining the data's tensor structure, the model avoids the "over-compression" typical of vector-based methods.
As shown in the figure above, the ND-PCA scheme achieves lower residual errors than traditional 2D-PCA, particularly when a limited number of principal components are used. Furthermore, the multidirectional approach (combining 1-mode, 2-mode, and 3-mode subspaces) shows significantly faster convergence than single-mode implementations.
Conclusion
The amended ND-PCA scheme provides a memory-efficient, near-optimal linear solution for high-dimensional data processing. While the current study utilized sparse 3D face data, the benefits of this approach are expected to be even more pronounced for dense 3D volumes, such as MRI or CT scans. Future research will focus on multimodal face data fusion to enhance the robustness of recognition systems.
Summary updated