MORAL: A Multimodal Reinforcement Learning Framework for Decision Making in Autonomous Laboratories

Natalie Tirabassi, Sathish A. P. Kumar, Sumit Jha, Arvind RamanathanarXiv 2025 · DOI · arXiv

Abstract

The MORAL (Multimodal Reinforcement Learning) framework significantly enhances the decision-making capabilities of autonomous laboratory robots by integrating visual data with fine-tuned textual captions. By fusing these modalities early in the learning process, the system achieves a 90% task completion rate, representing a 20% improvement over traditional single-modality approaches. This paper details the architecture, data augmentation strategies, and performance benchmarks of the MORAL framework.

Introduction

Autonomous laboratories rely on robots to perform complex, sequential tasks such as pipetting, mixing, and equipment transport. Traditionally, these systems operate primarily on raw visual data. However, pixel-level information often fails to capture the full semantic context of a dynamic environment. For example, a robot might perceive a container and a workstation but struggle to understand their functional relationship across a sequence of actions.

Existing research in multimodal learning has seen success in fields like autonomous driving, yet its application in real-time robotic control for laboratory settings remains underexplored. MORAL addresses this gap by creating a unified framework where language serves as a semantically rich guide for the robot’s actions, bridging the divide between image captioning and reinforcement learning (RL).

Methodology

The MORAL framework operates through a three-stage pipeline designed to synchronize visual and linguistic information.

1. Multimodal Data Augmentation

The system utilizes a pretrained BLIP-2 vision-language model to generate initial captions for robotic task images. To ensure these descriptions are sufficiently precise for laboratory work, a fine-tuning process is implemented. This transforms vague descriptions (e.g., "the robot is moving") into task-aligned, vivid instructions (e.g., "the robot picks up the red block with its left arm").

2. Early Fusion Architecture

Unlike late fusion strategies that combine processed outputs at the final stage, MORAL employs an early fusion approach. Visual features are extracted via a Convolutional Neural Network (CNN), while textual features are processed through a Recurrent Neural Network (RNN). These representations are merged early in the network, allowing the agent to simultaneously process the visual state and the semantic description of the environment.

The MORAL architecture illustrating the early fusion of visual and textual data into a multimodal policy network.

3. Reinforcement Learning Policy

The fused multimodal data is fed into standard RL agents. The framework utilizes Deep Q-Networks (DQN) for discrete task spaces and Proximal Policy Optimization (PPO) for continuous environments, mapping inputs to optimal robotic actions.

Datasets and Benchmarking

The study leverages BridgeData V2, a large-scale dataset of robotic arm trajectories. The researchers augmented this dataset by generating and fine-tuning textual descriptions for every state image within the trajectories. This process created a new multimodal benchmark specifically for laboratory automation.

Example of fine-tuned textual descriptions generated for robotic task sequences in the MORAL framework.

Results and Discussion

Performance Metrics

  • Task Completion: MORAL achieved a 90% task completion rate, significantly outperforming transformer-based (85%) and recurrent-based (78%) multimodal models.
  • Efficiency: The multimodal approach showed a 20% improvement over visual-only baselines and a 30% improvement over textual-only baselines.
  • Caption Accuracy: Iterative fine-tuning improved the BLEU score from 7.66 to 32.56, directly correlating with improved RL performance.

Key Insights

  • The "Slow Start" Phenomenon: Multimodal agents initially lagged behind visual-only agents due to the complexity of processing dual data streams. However, after approximately 40 episodes, the multimodal agents learned to leverage the combined data and consistently surpassed all baselines.
  • Language Grounding: Fine-tuned captions act as a semantic anchor, helping the RL agent generalize across different episodes by providing consistent context that raw pixels might lack due to environmental changes like lighting or camera angles.

Conclusion and Future Work

The MORAL framework demonstrates that integrating language and vision significantly improves robotic precision in complex environments. Future research will explore expanding the modalities to include auditory and tactile sensors. Additionally, the authors plan to investigate advanced fusion mechanisms, such as cross-attention transformers, to further reduce the initial training noise and improve data alignment.

Summary updated

BibTeX
@misc{tirabassi2025moral,
  author = {Natalie Tirabassi and Sathish A. P. Kumar and Sumit Jha and Arvind Ramanathan},
  title = {{MORAL: A Multimodal Reinforcement Learning Framework for Decision Making in Autonomous Laboratories}},
  year = {2025},
  eprint = {2504.03153},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2504.03153},
}
Figure