TL;DR
This paper introduces a deep learning approach using Recurrent Neural Networks (RNNs) and Long Short-Term Memory (LSTM) networks to predict recombinant protein production in E. coli fermentations. By learning the complex relationships between critical process parameters (CPPs) and real-time optical density (OD600nm), this method offers a promising path towards automated, real-time optimization and control of industrial fermentation processes.
What Problem Does This Paper Address?
Optimizing recombinant protein production in microbial fermentations, particularly in Escherichia coli, is a complex and time-consuming endeavor. The yield of the desired protein is highly sensitive to numerous critical process parameters (CPPs) like pH, temperature, dissolved oxygen, and nutrient feeding rates. Traditionally, fine-tuning these parameters relies on the expertise of experienced biotechnologists, often using empirical rules and extensive trial-and-error experiments. Developing analytical mathematical models for these living systems is exceedingly challenging due to their inherent complexity and dynamic nature. This leaves a gap in achieving efficient, real-time process control and optimization, which is crucial for industrial scalability and cost-effectiveness, especially when producing proteins that are difficult to monitor directly (e.g., inclusion bodies).
Methods
The core of this work is the application of deep learning, specifically Recurrent Neural Networks (RNNs) and Long Short-Term Memory (LSTM) networks, as a "black-box" model. These models are chosen for their ability to handle sequential data, which is characteristic of fermentation processes where past states influence future outcomes.
The process involved several key steps:
- Data Preparation: Critical Process Parameters (CPPs) were identified and selected. Cumulative data for some parameters were converted to non-cumulative forms. Data was then normalized using z-score standardization to ensure equal distribution for training and testing.
- OD600nm Interpolation: A crucial step addressed the temporal inconsistency between CPP measurements (every minute) and OD600nm measurements (hourly). A novel interpolation method combining linear and polynomial fitting was employed to create a consistent time series for both CPPs and OD600nm.
- Data Sequencing: A sliding window approach with a defined stride was used to create sequential data inputs for the RNN/LSTM models, capturing temporal dependencies.
- Model Architecture: A neural network architecture was designed, consisting of an initial fully connected layer to map input data to a high-dimensional latent space, followed by an RNN/LSTM module to process the sequential data, and a final fully connected layer for outputting OD600nm predictions.
- Training and Optimization: The models were trained using a learning rate of 0.001, a batch size of 256, and an SGD optimizer. Hyperparameter optimization was performed using a grid search.
- Evaluation: A leave-one-out cross-validation (LOOCV) strategy was used to evaluate the model's generalization ability. Two key metrics were employed: Root Mean Squared Error (RMSE) for overall prediction accuracy and Relative Error on Final Yield (REFY) to specifically assess the accuracy of the final predicted yield.
Datasets
The study utilized data from eight independent 1L E. coli fermentations conducted in a pre-industrial R&D setup. These fermentations were designed to produce a neurotrophin recombinant protein, often in the form of inclusion bodies. A subset of these fermentations (batches 8, 11, 12, 16, 22, 23, 24, 25, 26, 27, and 28) were selected for training and testing based on data quality and adherence to canonical settings. The paper does not introduce a new benchmark dataset but rather uses its own experimental data.
Key Insights and Ideas
- Black-Box Power: The study demonstrates that even without a deep analytical understanding of the underlying biological mechanisms, machine learning models can effectively learn complex, non-linear relationships between process parameters and fermentation outcomes.
- Real-time Monitoring Proxy: OD600nm, a readily measurable parameter, can serve as a reliable proxy for biomass and, consequently, for inclusion body production, enabling real-time monitoring even when the target protein is not directly quantifiable.
- Temporal Dependencies Matter: The success of RNNs and LSTMs highlights the critical importance of considering the temporal evolution of CPPs, rather than just static values, for accurate prediction in dynamic biological processes.
- Interpolation for Data Mismatch: The developed interpolation technique effectively bridges the gap in sampling frequencies between CPPs and OD600nm, a common challenge in industrial data acquisition.
- Generalization Across Conditions: The model showed promising generalization capabilities, predicting yields even for fermentations run under slightly different protocols (e.g., lower temperature, soluble protein expression), suggesting robustness.
Key Results
- Both LSTM and RNN networks achieved comparable performance in predicting OD600nm values and final yields.
- The models demonstrated good accuracy, with average RMSE values around 4.00-4.10 and average REFY values around 7.21-7.52% across the tested batches.
- Yield estimations for most batches (16, 22, 23, 24, 26, 27, and 28) were "quite promising," indicating the potential for ML-driven process control.
- The models were able to identify deviations from expected fermentation trends, even in challenging cases like batch 12, where unusual CPP trends led to less accurate predictions but still highlighted the anomaly.
Limitations and Future Work
- Data Scarcity for Anomalies: The models struggled with predicting yields for batches with highly unusual CPP trends (e.g., batches 8, 11, 12, and 25) that were underrepresented in the training data. This suggests a need for more diverse training data, potentially exploring a wider design space.
- "Worst-Case" Scenario: While the model performed well on different expression forms (soluble vs. inclusion bodies), the paper acknowledges that predicting in truly "worst-case" or highly anomalous scenarios remains a challenge.
- Future Work: The authors suggest further exploration of the design space to improve generalization ability. They also envision these models paving the way for ML-driven control systems that can recommend optimal CPP set points in real-time, ultimately leading to more robust and efficient GMP fermentation processes. The code for the model is publicly available, encouraging further research and application.
Summary updated