Spectral Adapters: Switching Fine-Tuned Error Correctors on Photonic Hardware Without Retuning Rings

Photonic chips can multiply a vector by a matrix by writing each vector entry onto its own color of light. This is wavelength-division multiplexing (WDM): many wavelengths share one waveguide without interfering. The light then passes tiny ring-shaped filters (microring resonators) that dim each color by an amount equal to a weight, and a photodetector sums the result. The weights live in the rings, and the rings are set by micro-heaters that take milliseconds to settle (Reconfigurable versatile integrated photonic computing chip). Fine-tuning produces a different set of weights for every target domain, so a static-weight photonic chip effectively serves only one fine-tuned model. Switching to another means rewriting every ring. A practical text-correction service does not look like that. It sees beginner essays, advanced learners, native writers, and fluency rewrites interleaved request by request.

This proposal restricts fine-tuning to exactly the parameters the optical hardware can change fast. There are two such controls. The first is the per-wavelength scaling of the input, which is set by the high-speed modulators or folded into the digital-to-analog converter (DAC) values. The second is the per-detector gain and offset on the output. The pre-trained sequence-to-sequence corrector stays frozen in the slow rings. Each domain is a vector of a few thousand scalars, and switching domains costs a vector reload rather than a thermal retune. The central question is empirical and has not been asked before. How much of the benefit of per-domain fine-tuning for grammatical error correction survives when adaptation is confined to these hardware-native diagonal knobs on top of noisy analog weights?

Core Claim

In a broadcast-and-weight photonic layer, every output row sees the same set of input wavelengths, and every output has its own detector. This structure means the cheaply adjustable part of a layer is a two-sided diagonal rescaling of a frozen matrix:

$$y = \mathrm{diag}(g_d)\,\hat{W}\,\mathrm{diag}(s_d)\,x + b_d .$$

Here $x$ is the layer input, $\hat{W}$ is the weight matrix as actually programmed into the rings, including device error and crosstalk, and $d$ indexes the target domain. The vector $s_d$ scales each wavelength and $g_d$ scales each detector's gain. The offset $b_d$ is added electronically after the analog-to-digital converter. For an $n \times n$ layer this is about $3n$ domain-specific numbers instead of $n^2$, and none of them touch a ring.

We claim three things. First, these "spectral adapters" recover most of the gap between a single shared corrector and per-domain full fine-tuning on the BEA-2019 proficiency splits, CoNLL-2014 and JFLEG. Second, they do so with zero added multiply-accumulates and domain switches measured in modulator-reload time rather than thermal settling. Third, the same knobs, refit on a small calibration set, absorb chip-specific weight programming error. That gives one mechanism for both domain transfer and hardware calibration.

This is a claim about where adaptation capacity must live, not a claim that photonics beats GPUs at text correction. We say openly that the input scaling can be folded into the DAC digitally. The contribution is showing that this nearly free parameterization is enough, which makes static-weight photonics viable for multi-domain fine-tuned models.

Relation to Prior Work

The motivating tension comes from two hardware lines. Reconfigurable versatile integrated photonic computing chip builds a microcomb-fed ring and interferometer chip that switches between fully connected, convolutional and recurrent networks. It reports millisecond thermal tuning as the limit on runtime weight updates. Lightening-Transformer avoids static weights by streaming both operands dynamically through coherent dot-product units. That makes switching trivial but pays for weight DACs: 28.06 W at 4-bit rising to 95.92 W at 8-bit. We sit between them. Weights stay static, which is the energy-efficient regime, and only the diagonal terms become dynamic.

The closest prior work on adaptation in photonics is Photonic neuromorphic architecture for tens-of-task lifelong learning. It avoids forgetting across classification tasks by freezing sparse phase-change-material paths per task and adding new ones. Its task capacity is bought with physical area and non-volatile, slow writes. We reuse every ring for every domain, change only fast scalars, and target generation rather than classification.

On the electronic side, Be CIM or Be Memory reconfigures compute-in-memory arrays between arithmetic and buffer modes per layer. It shows reconfiguration pays off when switching overhead is small, reported at 3–5% of execution time. We borrow its framing of reconfiguration as a scheduling cost to be minimized, applied here to domain switches rather than layer phases.

The method and task come from error correction and transfer learning. The CUED's Grammatical Error Correction Systems for BEA-2019 supplies the training recipe we reuse: a Transformer pre-trained on 5 million back-translated synthetic errors, then fine-tuned on W&I+LOCNESS learner data. A Representation Level Analysis of NMT Model Robustness to Grammatical Errors reports that encoders detect errors in intermediate layers and normalize them in deeper layers through specific "robustness heads." It also finds that fine-tuning only the encoder is nearly as effective as fine-tuning everything. That finding shapes our design: we test whether adapters on deeper encoder layers alone suffice. It also raises a risk that analog noise erases the fine token-level distinctions, such as singular versus plural, that detection needs.

Research Program

We will build a simulated static-weight WDM Transformer corrector. Its noise model will include ring programming error, inter-channel crosstalk, detector noise and 4-bit and 8-bit converter quantization. The modeling will follow the dispersion-and-noise treatment that Lightening-Transformer used to report under 1% accuracy loss on classification. All matrix products with fixed weights run through this model. The attention score products, whose operands are both activations, stay electronic or dynamic, because static rings cannot hold them.

The base corrector is a fairseq Transformer trained with the CUED recipe. We then train it noise-aware, with sampled device errors injected during a short phase before freezing.

The domains are the BEA-2019 W&I+LOCNESS proficiency groups (CEFR A, B, C and native writers) described in The BEA-2019 Shared Task on Grammatical Error Correction, the NUCLE essays behind CoNLL-2014, and the fluency rewrites of JFLEG. For each domain we fit $(s_d, g_d, b_d)$ with the rings frozen, under the same noise model, and with the scales constrained so the scaled inputs stay inside the DAC's range.

Three analyses follow. The first measures where adaptation capacity matters, by placing adapters at all layers, at encoder only, and at deep encoder only. The second tests whether one-sided or two-sided scaling is required. The third tests how quantization interacts with scaling, since shrinking an input channel at 4 bits throws away resolution.

A second study reuses the mechanism for calibration. We train on one sampled "chip," then instantiate a new chip with fresh device errors. We ask how many sentences of refitting the diagonal knobs need to restore accuracy without reprogramming rings.

Finally, a serving study replays interleaved multi-domain request streams. It charges ring retunes at millisecond scale and scale reloads at modulator speed, and it reports throughput and energy per sentence.

Baselines

The naive reference is a single shared corrector. It is the CUED-style model fine-tuned once on pooled learner data, mapped onto the noisy photonic simulator, with no per-domain adaptation. If spectral adapters do not beat it clearly on the per-domain test sets, adaptation is not worth any mechanism. It uses the same pre-training, data and simulator.

The accuracy ceiling is per-domain full fine-tuning. Each domain gets its own complete weight set, fine-tuned noise-aware with the same epoch and hyperparameter budget as the adapters. For reference, the CUED restricted-track system reports an ERRANT score of 47.34 on BEA-2019 Dev and an M2 score of 58.90 on CoNLL-2014 (The CUED's Grammatical Error Correction Systems for BEA-2019). On JFLEG the best system in the original benchmark reached 47.2 GLEU against 55.3 for humans (JFLEG). The survey notes top systems reach F0.5 in the high 70s on BEA-2019 (Grammatical Error Correction: A Survey of the State of the Art), but with far larger models than a photonic chip holds. We therefore compare against our own GPU replication of the CUED recipe, and expect it to land near the 47.34 figure. Full fine-tuning is the strongest competitor on accuracy, but on this hardware every domain switch costs a full ring retune.

The strongest cost-competitive method is a low-rank electronic side-car. It is a small digital correction $BAx$ added to the frozen photonic output, trained per domain. It is the obvious way to adapt a frozen analog accelerator. We give it the same parameter count as our adapters and the same tuning budget, and we charge its extra digital multiply-accumulates and memory traffic in the energy model.

Two ablations isolate the key ingredient. One is output-only scaling (gain and offset), which is the simplest electronic-side fix. The other is two-sided scaling trained without the device-noise model, which tests whether noise-awareness matters.

Spectral adapters "win" under two conditions. They must recover at least roughly four-fifths of the F0.5 gap between the shared model and full fine-tuning on each domain. And they must match the equal-parameter side-car within noise while adding no digital multiply-accumulates.

Why not just…?

Why not just use one shared corrector for all domains? This is the cheapest option. If the proficiency splits and JFLEG do not reward domain specialization, our whole premise fails. The shared-model baseline measures this directly. We also note that JFLEG finds systems change only 38% of awkward sentences despite correcting 68% of sentences with grammatical errors. That suggests fluency-oriented and error-oriented domains want different behavior from one model. If the gap turns out to be small, we will report that as a negative result about multi-domain error correction, not bury it.

Why not just add a low-rank electronic correction beside the photonic core? This is the real threat. It is more expressive than diagonal scaling and is easy to build. Its cost is digital multiply-accumulates and weight fetches on every token, which re-imports the electronic bottleneck the optical core was meant to remove. The equal-parameter comparison settles this. If the side-car beats spectral adapters by a wide margin at negligible energy, the right conclusion is a hybrid. In that case we would report the smallest side-car rank that closes the remaining gap on top of the diagonal knobs, rather than defending diagonal scaling alone.

Why not just stream the weights dynamically, as Lightening-Transformer does, so switching is free? Dynamic weights make every product pay DAC energy for both operands, and Lightening-Transformer shows power more than tripling from 4-bit to 8-bit. Our serving study will compare energy per sentence against a dynamic-weight configuration simulated with the same converter model. That comparison shows whether static weights plus fast scalars actually save energy once attention is included.

Why not just switch to a faster non-autoregressive tagger such as GECToR? The survey reports taggers are 5–10× faster than autoregressive correction (Grammatical Error Correction: A Survey of the State of the Art). But speed is orthogonal to our question: a tagger's encoder has the same frozen-weight adaptation problem. We will apply spectral adapters to a tagging encoder as a secondary experiment. That tests whether the result depends on the decoder at all.

Proposed Experiments and Decisive Checks

The primary experiment trains the base corrector and, for each of the six domains, the shared, full fine-tune, side-car, output-only and spectral-adapter variants. We report ERRANT F0.5 on BEA-2019 Dev split by proficiency, M2 F0.5 on CoNLL-2014, and GLEU on JFLEG. Every variant runs at 4-bit and 8-bit converter precision under the device-noise model, with five noise seeds per configuration.

The first decisive check is gap recovery. If two-sided spectral adapters fall below half the shared-to-full-fine-tune gap on most domains, the core claim is false.

The second check follows A Representation Level Analysis of NMT Model Robustness to Grammatical Errors. We compare adapters restricted to deep encoder layers against adapters at all layers. We also probe error-detection F1 by layer under analog noise, to see whether noise degrades the intermediate-layer detection signal. If it does, adapters placed before those layers should help more than deep-only placement.

The third check is chip transfer. We refit the diagonal knobs on a freshly sampled device-error instance using 100, 1,000 and 10,000 sentences, and plot recovered F0.5 against the accuracy of the same model with perfectly programmed weights.

The fourth check is the serving replay. We use interleaved request streams with domain-switch rates from one per batch to one per sentence. Throughput and energy per sentence are compared across the ring-retune, dynamic-weight and spectral-adapter configurations.

The note succeeds if spectral adapters sit on the Pareto front of accuracy versus switch cost and added compute across these four checks. It is informative even if they fail, because it will show how much adaptation capacity a fine-tuned text corrector truly needs beyond its frozen pre-trained weights.