ARCADE: Controllable Codon Design from Foundation Models via Activation Engineering

Jiayi Li, Litian Liang, Shiyi Du, Tang Shi-jie, H. Lai, Carl KingsfordbioRxiv 2025 · DOI

Abstract

Codon sequence design is crucial for generating mRNA sequences with desired functional properties for tasks such as developing mRNA vaccines or gene editing therapies. Yet existing methods lack flexibility and controllability to adapt to various design objectives. We propose a novel machine learning-based framework, ARCADE, that enables flexible and controllable multi-objective codon design. Leveraging inherent knowledge from pretrained genomic language models, ARCADE extends activation engineering, a technique originally developed for controllable text generation, beyond discrete feature manipulation such as concepts and styles, to steering continuous-valued biological metrics. Specifically, we derive biologically meaningful semantic steering vectors in the model's activation space, which directly control properties such as the Codon Adaptation Index, Minimum Free Energy, and GC content. Experimental results demonstrate the flexibility of ARCADE in designing codon sequences with multiple objectives, underscoring its potential for advancing programmable biological sequence design.

Introduction

Designing optimal mRNA sequences is a complex optimization problem due to the redundancy of the genetic code. While a single protein can be encoded by a vast number of synonymous codon sequences, these variants differ significantly in stability, translation efficiency, and immunogenicity. Traditional codon optimization approaches generally fall into two categories: precise but computationally expensive algorithmic methods (e.g., LinearDesign), and fast but rigid deep learning models that often require retraining for new objectives.

ARCADE (Activation engineeRing for ControllAble coDon dEsign) introduces a "third way" by applying activation engineering to genomic foundation models. This approach allows for real-time, multi-objective optimization of continuous biological metrics without the need for model retraining.

Methodology

The ARCADE Framework

ARCADE utilizes a frozen, pretrained genomic foundation model—a BERT-style encoder trained on 10 million mRNA sequences—as its backbone. To transform this encoder into a generative tool, the researchers implemented two key modifications:

  1. Token Classification Head: A classification head is added to predict synonymous codons.
  2. Synonymous Codon Masking: A mask ensures the model only selects codons that preserve the original protein sequence.
Overview of the ARCADE framework for controllable codon design, illustrating the reconstruction process, activation engineering mechanism, and the generation of steering vectors.

Activation Engineering and Steering Vectors

The core innovation lies in the use of steering vectors. To control a specific property, such as GC content or Minimum Free Energy (MFE), the framework compares the internal activations of the model when processing sequences with high versus low values for that property. The resulting difference vector represents a mathematical direction in the model’s latent space corresponding to that biological attribute. During inference, these vectors are injected into the model's layers, allowing researchers to "steer" the output by adjusting a scaling factor ($\lambda$).

Experimental Evaluation

The framework was evaluated using the GENCODE Human Release 47 dataset for general training and the mRFP benchmark, which contains 1,459 variants of the monomeric Red Fluorescent Protein with experimentally measured expression levels. The study focused on steering several critical metrics: Codon Adaptation Index (CAI), Negated MFE (stability), GC content, and immune-related motifs (CpG and UpA densities).

Key Results and Discussion

Efficiency and Flexibility

ARCADE demonstrates a massive performance advantage over traditional methods. It optimized 100 sequences in just 16 seconds on a single GPU, whereas the state-of-the-art algorithmic tool, LinearDesign, required over 8 hours for the same task. Furthermore, ARCADE successfully steered sequences across a wide range of continuous metrics, proving that genomic foundation models naturally organize biological concepts into distinct, steerable directions.

Functional Performance

In tests on the mRFP dataset, ARCADE-designed sequences significantly outperformed other deep learning baselines, such as CodonBERT, in achieving higher experimental expression levels. The results also showed that multiple, sometimes conflicting, objectives (like stability vs. translation efficiency) can be optimized simultaneously by combining different steering vectors, effectively navigating the Pareto front of sequence design.

Conclusion

ARCADE represents a significant advancement in programmable biological sequence design. By repurposing activation engineering for continuous biological metrics, it provides a fast, flexible, and controllable framework for mRNA design. Future work will focus on developing position-aware editing capabilities and exploring non-linear steering vectors to address even more complex biological contexts.

Summary updated

Abstract

Codon sequence design is crucial for generating mRNA sequences with desired functional properties for tasks such as developing mRNA vaccines or gene editing therapies. Yet existing methods lack flexibility and controllability to adapt to various design objectives. We propose a novel machine learning-based framework, ARCADE, that enables flexible and controllable multi-objective codon design. Leveraging inherent knowledge from pretrained genomic language models, ARCADE extends activation engineering, a technique originally developed for controllable text generation, beyond discrete feature manipulation such as concepts and styles, to steering continuous-valued biological metrics. Specifically, we derive biologically meaningful semantic steering vectors in the model's activation space, which directly control properties such as the Codon Adaptation Index, Minimum Free Energy, and GC content. Experimental results demonstrate the flexibility of ARCADE in designing codon sequences with multiple objectives, underscoring its potential for advancing programmable biological sequence design. Our implementation is available at https://github.com/Kingsford-Group/arcade.
BibTeX
@misc{li2025arcade,
  author = {Jiayi Li and Litian Liang and Shiyi Du and Tang Shi-jie and H. Lai and Carl Kingsford},
  title = {{ARCADE: Controllable Codon Design from Foundation Models via Activation Engineering}},
  year = {2025},
  doi = {10.1101/2025.08.19.668819},
  howpublished = {bioRxiv},
}
Figure