CHORUS: Cross-Subject Harmonization via Population Relay for Unseen Subjects in Brain Visual Decoding

Jiayu Zuo, Dongyang Li, Chen Wei, Mo Wang, Quanying LiuarXiv 2026 · arXiv

TL;DR

CHORUS is a novel framework for EEG-based visual decoding that solves the "unseen subject" problem by using a population average as an intermediate alignment target. By first mapping a new subject's brain activity to a predicted population response before matching it to images, CHORUS achieves state-of-the-art retrieval accuracy, significantly outperforming methods that attempt direct brain-to-image alignment.

What Problem Does This Paper Address?

Visual decoding—the ability to "read" what someone is looking at from their brain activity—has made huge strides, but it has a major Achilles' heel: inter-subject variability. A decoder trained on one group of people typically fails when presented with a new, "unseen" individual. This is because every brain is unique; the neural "signature" of a cat for Subject A looks different from Subject B's.

Existing solutions try to "calibrate" the model for a new user by aligning their EEG embeddings directly with image features (like those from CLIP). However, the authors argue this is a "bridge too far." The geometric gap between raw brain signals and abstract image embeddings is complex and difficult to bridge without massive amounts of personalized data.

Methods

The authors propose CHORUS (Cross-Subject Harmonization via Population Relay), which introduces a two-stage "relay" process.

  1. Training Stage: Stimulus-matched Cross-subject Averaging (SCA): Instead of training only on individual recordings, the encoder is trained on EEG waveforms averaged across multiple subjects viewing the same image. This "denoises" the signal, forcing the encoder to focus on shared, stimulus-driven features rather than subject-specific quirks.
  2. Deployment Stage: Two-Stage Calibration: When a new subject arrives, CHORUS doesn't jump straight to images.
  3. Stage 1 (Population Alignment): It first aligns the new subject to population anchors—predicted average brain responses for candidate images. These anchors are generated by a ridge regression model that maps visual features to the expected "average" EEG response.
  4. Stage 2 (Image Alignment): Once aligned to the population space, a second refinement step maps these embeddings to the final image space.
Figure 2. CHORUS. (a) Training: SCA averages same-image EEG across subjects. (b) Deployment: target embeddings align first to population anchors, then to images; only the final representation is scored. (c) Measured anchors average source responses; predicted anchors cover unseen images.

Datasets

The study evaluates CHORUS across three diverse datasets:

  • THINGS-EEG2: A high-quality dataset with 63-channel EEG recordings from 10 subjects viewing 1,854 concepts.
  • Alljoined-1.6M: A massive dataset using consumer-grade 32-channel EEG, testing the method's robustness to noisier, lower-resolution hardware.
  • Natural Scenes Dataset (NSD): Used to demonstrate that the "population relay" concept also generalizes to fMRI data.

Key Insights and Ideas

  • The Population is a Better Target: The core "aha!" moment is that an unseen subject's brain activity is much closer to the average brain activity of other people than it is to an image embedding. As shown in Figure 1, the similarity (CKA) is significantly higher for the population average (0.72) than for image embeddings (0.59).
  • Simple Geometry: The authors discovered that the remaining gap between a new subject and the population is largely a simple shift and rotation. This means rigid, low-parameter transforms can effectively "harmonize" a new subject without needing thousands of labeled examples.
  • SCA as a Zero-Cost Regularizer: Simply averaging waveforms during training acts as a powerful regularizer for cross-subject transfer, improving performance even without the subsequent calibration stages.
Figure 1. Where should a new subject be aligned? (a) Across 30 folds, mean pre-calibration linear CKA over 200 stimuli is 0.72 with measured population EEG and 0.59 with image embeddings. (b) Mean CKA for all five references (BHA + SCA, best source selected post hoc). (c) Unlabeled-batch retrieval Top-1.

Key Results

CHORUS sets a new bar for zero-shot brain-to-image retrieval:

  • Top-1 Accuracy: On the THINGS-EEG2 benchmark (200-way retrieval), CHORUS reached 78.37% Top-1 accuracy, a massive 36.32 point lead over the previous state-of-the-art, SCORE.
  • Efficiency: The method works remarkably well with very little data. Even with just 10 labeled EEG-image pairs, it outperforms direct calibration methods that use 200 pairs.
  • Qualitative Success: In image reconstruction tasks, the "population-aligned" embeddings lead to much clearer and more semantically accurate image generations compared to direct calibration.
Figure 3. Direct calibration versus CHORUS. Top-three predic- tions for the held-out subject closest to the mean CHORUS accuracy (BHA + SCA); green: correct. Shown: four random queries that only CHORUS gets right and direct calibration ranks outside its top three.

Limitations and Future Work

While highly effective, CHORUS currently requires training data where multiple subjects have viewed the same shared stimuli to build the population anchors. This limits its use on datasets with purely idiosyncratic stimulus sets.

The authors also note that while they demonstrated a "causal streaming" version of the model (where it calibrates on the fly as data arrives), it is not yet a truly "closed-loop" system. Future work will focus on improving image-to-population prediction and testing the system in live, interactive BCI sessions.

Summary updated

Abstract

EEG-based image retrieval performs well on training subjects but generalizes poorly to unseen individuals. Existing calibration methods address this gap by aligning a new subject directly with image embeddings. We find that the training population is a better alignment target: an unseen subject's embeddings are closer to the stimulus-matched population average than to the image embeddings, and the remaining gap to the population is largely a shift and a rotation. Motivated by this finding, we propose CHORUS. At training, stimulus-matched cross-subject averaging (SCA) teaches the encoder to match responses to the same image across subjects; at deployment, a two-stage calibration first aligns an unseen subject to population anchors predicted from the decoder's visual features, and then to the image embeddings. On leave-one-subject-out THINGS-EEG2 with two backbones, CHORUS improves over direct calibration by 10.92 and 6.63 points on unlabeled test batches and by 4.2-6.9 points with 10-200 labeled pairs; even without the population stage, SCA and direct calibration each outperform prior training and calibration schemes. The complete system reaches 78.37% and 73.40% 200-way Top-1 accuracy, 36.32 and 20.93 points above SCORE.
BibTeX
@misc{zuo2026chorus,
  author = {Jiayu Zuo and Dongyang Li and Chen Wei and Mo Wang and Quanying Liu},
  title = {{CHORUS: Cross-Subject Harmonization via Population Relay for Unseen Subjects in Brain Visual Decoding}},
  year = {2026},
  eprint = {2611.24364},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.24364},
}
Figure