Introduction
Multimodal Transformers have emerged as a cornerstone of modern artificial intelligence, enabling the simultaneous processing of diverse data types such as text and images. However, the deployment of these models on edge devices is hindered by significant computational demands and irregular workloads. Digital Computing-In-Memory (CIM) architectures offer a promising solution by performing calculations within the memory arrays, thereby reducing the energy-intensive data movement between processor and memory. Despite their potential, existing CIM designs struggle with the dynamic nature of multimodal tasks. This paper introduces StreamDCIM, a novel tile-based digital CIM accelerator designed to overcome these limitations through a reconfigurable architecture and a fine-grained "ping-pong" pipeline.
Addressing Hardware Inflexibility in Multimodal Transformers
Current digital CIM architectures face three primary "inflexibilities" when handling multimodal workloads:
- Microarchitecture Inflexibility: Multimodal models frequently employ "token pruning" to discard redundant information. This results in irregular data patterns that rigid CIM macros cannot process efficiently, leading to underutilized hardware resources.
- Dataflow Inflexibility: Most CIM systems rely on a fixed "weight-stationary" dataflow. This constraint limits the degree of parallelism and prevents the decoupling of execution at a granular level, which is necessary for handling disparate modalities.
- Pipeline Inflexibility: In the self-attention mechanism, certain operations (such as $QK^T$) require weights that are generated dynamically at runtime. In a CIM context, these intermediate results must be written back into the memory arrays. This "rewriting" process is high-latency and typically introduces significant "bubbles" or idle periods in the execution pipeline.
The StreamDCIM Architecture
To resolve these bottlenecks, StreamDCIM introduces three architectural innovations:
Tile-based Reconfigurable CIM (TBR-CIM)
The StreamDCIM hardware is organized into modular tiles. These tiles can dynamically switch between a "normal" mode and a "hybrid" mode. In the hybrid configuration, the CIM macro stores both inputs and weights, significantly improving hardware utilization during the dynamic workloads characteristic of token-pruned multimodal models.
Mixed-Stationary Cross-Forwarding Dataflow
Moving beyond simple data broadcasting, StreamDCIM utilizes a mixed-stationary cross-forwarding dataflow. This mechanism allows tiles to forward data directly to one another, enabling "tile-level execution decoupling." Consequently, different sections of the chip can process distinct computational tasks simultaneously, eliminating the need for global synchronization and increasing overall throughput.
Ping-Pong Fine-Grained Pipeline
One of the most critical challenges in CIM design is the latency associated with rewriting memory. StreamDCIM addresses this by implementing a "ping-pong" pipeline. This design overlaps the writing of new weights into one CIM macro while a secondary macro performs active computation. By hiding the memory update latency, the system maintains a continuous flow of data and execution.
Experimental Evaluation
The StreamDCIM architecture was evaluated using a 28nm process simulation running at 200 MHz. The researchers targeted state-of-the-art multimodal Transformers, specifically ViLBERT-base and ViLBERT-large, performing Visual Question Answering (VQA) v2.0 tasks.
To ensure the high precision required for complex multimodal reasoning, the system utilizes INT16 precision. This choice avoids the accuracy degradation often associated with analog CIM designs, which are susceptible to noise and limited bit-depth.
Results and Impact
StreamDCIM demonstrates significant performance improvements over traditional CIM architectures:
- Throughput: The design achieved a geomean speedup of 2.63× compared to non-streaming CIM solutions and a 1.28× improvement over layer-based streaming designs.
- Energy Efficiency: By optimizing data movement and maximizing utilization, StreamDCIM reduced energy consumption by up to 2.64× on the ViLBERT-base model.
- Physical Implementation: The simulated 28nm design occupies a chip area of 12.10 $mm^2$ with a maximum power envelope of 122.77 mW.
Conclusion and Future Directions
StreamDCIM represents a shift from layer-based to tile-based streaming in CIM accelerators. By treating memory updates as a pipeline stage and enabling reconfigurable tile execution, the architecture provides the elasticity required for modern AI models. While the current implementation focuses on INT16 precision and dual-modality (vision and language) inputs, future research may explore bit-scalable macros to support lower-precision formats (e.g., INT8 or FP8) and scaling the architecture to handle "many-modal" inputs including audio and sensor data. The complexity of the proposed dataflow also suggests the need for advanced compiler support to optimally map diverse model topologies onto the StreamDCIM hardware.
Summary updated