Cross-View Imputation in Shared Latent Spaces

41 concepts35 papers1 proposal

In an ideal machine learning workflow, multi-view datasets are perfectly complete: every instance is accompanied by all its corresponding modalities, whether they are images, audio tracks, text descriptions, or clinical test scores. In reality, data collection is messy. Sensors fail, users leave profiles partially blank, and biological experiments omit specific measurements.

Faced with this "missing view" problem, a naive approach might involve imputing the missing data directly within the raw input space—for instance, by calculating mean pixel values or using single-view matrix completion. However, raw-space imputation suffers from two fundamental flaws: it forces the model to waste capacity modeling high-dimensional noise, and it struggles to bridge highly heterogeneous data types (such as generating an image directly from an ordinal text tag).

To overcome these limitations, a powerful research direction has emerged: Latent Space Imputation. Instead of filling in the blanks in the noisy, raw feature space, these methods project available views into a shared, low-dimensional latent subspace. By leveraging the semantic correlations between different views, the model can infer missing information where it matters most: in the underlying representation space.


The Core Intuition: Cross-View Redundancy

Why is it possible to recover a missing data view without seeing it? The answer lies in cross-view redundancy. Different perspectives of the same entity typically share an underlying semantic core. For example, a video frame of a barking dog and the corresponding audio track both point to the same high-level concept.

As Akbari et al. demonstrate in multi-signal modeling, disparate data streams can be assumed to be generated by a single, common hidden process. By mapping heterogeneous data into a unified latent space, information from one modality can accurately predict missing values in another (Lian et al.).

[ View 1: Image ]  ───(Encoder 1)───► ┌───────────────────────┐ ◄───(Encoder 2)───  [ View 2: Text ]
                                      │ Shared Latent Space   │
[ View 3: Missing] ───(Imputed) ◄─────│ (Semantic Alignment)  │
                                      └───────────────────────┘

When a dataset is only partially complete, instances that do possess all views act as "anchors" (Wang et al.; Yu et al.). These complete instances establish the geometry of the shared latent space, aligning the modalities. When a partial instance arrives with a missing view, its available views are sufficient to locate its correct coordinates within this pre-aligned space (Li et al.).


Mechanism 1: Projection and Back-Projection

The most direct way to utilize a shared latent space for data recovery is through a generative projection loop. This process operates in three distinct steps:

  1. Encoding: Project the available features of an incomplete sample into the shared latent space.
  2. Alignment: Map view-specific embeddings through individual basis matrices or encoders to find a common representation (Wang et al.).
  3. Decoding (Back-Projection): Pass the shared latent vector through the specific decoder or basis matrix of the missing view to synthesize the absent information.

Wen et al. formalized this using graph-regularized matrix factorization, showing that missing data in one view can be effectively recovered by projecting available features from an alternate view into the shared space and back-projecting them through the target view's basis matrix.

This strategy relies on the principle that different views share redundant semantic information that can be recovered through cross-view autoencoding (Lu et al.). Similarly, Liu et al. enforce a single, unique latent identity across all sources, projecting this shared identity back into individual feature spaces to simultaneously reconstruct available data and synthesize missing values.


Mechanism 2: Pure Latent Imputation

While back-projecting into the raw input space is useful for visualization, it is often an unnecessary detour for downstream tasks like classification or clustering. Reconstructing raw features introduces modeling noise and requires significant computational overhead.

To bypass this, modern architectures perform pure latent imputation. Instead of reconstructing raw inputs, the system learns transformations directly between the latent spaces of different views to predict missing representations (Realize Generative Yet Complete Latent Representation for Incomplete Multi-View Learning).

For instance, in de novo peptide sequencing, Du et al. introduce a "latent imputation" step that reconstructs missing mass spectrometry data within a high-level feature space before the final decoding stage. By utilizing learnable queries to infer missing components based on the context of observed data, this approach avoids raw data noise and focuses entirely on high-level features.

Similarly, Liu et al. use a joint attention mechanism to identify similar samples across available views, using their high-level representations to estimate missing information directly within the embedding space. As Huang argues, precise reconstruction of raw missing views is often less critical than extracting discriminative, task-relevant features that remain invariant to view availability.


Advanced Geometries: Handling Extreme Missingness

When the missingness rate is exceptionally high (e.g., over 70% of the views are absent), simple linear projections fail. Researchers have introduced more sophisticated probabilistic and geometric frameworks to stabilize the shared latent space.

Cluster Centroids and Synergy

When explicit view correspondences are missing, global data structures can guide the imputation. Pu et al. demonstrate that missing features can be estimated by leveraging a sample's cluster membership in available views alongside global cluster centroids. By transferring representative feature information from these centroids, the system maintains a unified representation space even under severe data corruption (Zhao et al.).

Expressive Generative Priors

Standard variational autoencoders often assume simple, restrictive probability distributions in the latent space, which fail to capture complex inter-view dependencies. To resolve this, He et al. utilize Inverse Autoregressive Flows (IAF) to model complex inter-dimensional dependencies, enabling more accurate cross-view completion.

In scenarios with up to 90% missing data, Bernal pairs normalizing flows with a secondary network architecture that maps the latent representations of incomplete data directly to their complete counterparts, bridging the gap to the full data manifold.

Diffusion-Guided Recovery

More recently, diffusion models have been adapted to the latent imputation paradigm. Ou et al. use diffusion processes to generate synthetic latent "summary vectors" conditioned on available labels and views. This allows the system to gracefully hallucinate missing semantic information while pruning redundant noise from the remaining views.


Conclusion

Latent space imputation works because it shifts the problem of data recovery from surface-level reconstruction to semantic alignment. By forcing diverse modalities to pass through a shared bottleneck, models exploit the underlying correlations inherent in multi-view data. Whether the final goal is to synthesize the missing raw features or to perform classification directly on the complete latent vectors, operating within a shared subspace ensures that the model remains robust, efficient, and immune to the noise of the raw input space.

Key Foundational Work

Integrating Features and Similarities: Flexible Models for Heterogeneous Multiview Data (2015) — contributed 2 concept elements to this direction.
Partial Multi-View Clustering (2014) — contributed 2 concept elements to this direction.
Learning to Hash on Partial Multi-Modal Data (2015) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_latent_space_imputation_1a450f87,
  title = {latent space imputation},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/latent-space-imputation/rdr_1a450f877cf048c4a61efa87c8ee88ce},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_latent_space_imputation_1a450f87_v1,
  title = {latent space imputation},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/latent-space-imputation/rdr_1a450f877cf048c4a61efa87c8ee88ce/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake