Abstract
Limited by computational capabilities, battery life, and network bandwidth, traditional emotion recognition often fails to provide a seamless user experience. This paper introduces MULTI-EASE, a sustainable multi-modal multi-layer emotion-aware service architecture. By leveraging a hierarchy of terminal devices, edge servers, and the cloud, MULTI-EASE optimizes task scheduling to minimize energy consumption while meeting strict latency requirements. Experimental results from a prototype system demonstrate that this approach significantly improves the efficiency and sustainability of emotion analysis applications in edge environments.
Introduction: The Challenge of Real-Time Emotion AI
Emotion recognition is a computationally intensive task, particularly when processing multi-modal data such as simultaneous video and audio streams. For battery-powered terminal devices like social robots or smartphones, local processing can lead to rapid battery depletion and performance bottlenecks. Conversely, relying solely on the remote cloud introduces intolerable latency, making real-time interactions feel unnatural and "laggy."
The MULTI-EASE framework addresses this sustainability gap by intelligently distributing workloads across a three-tier hierarchy, ensuring high-quality emotion AI without compromising device longevity or user experience.
The MULTI-EASE Architecture
The core of MULTI-EASE is a multi-layer execution environment designed to handle the complexities of multi-modal data sensing and processing.
As shown in the architecture overview, the system consists of:
- Terminal Device Layer: Local hardware (e.g., robots, smartphones) that performs initial data collection and handles small, time-sensitive tasks.
- Edge/Fog Layer: Nearby network nodes that provide a "sweet spot" of moderate computing power and low latency.
- Remote Cloud Layer: Large-scale data centers used for heavy, high-precision analysis where latency is less critical.
Intelligent Task Scheduling
To manage these layers, the authors developed a Task Scheduling Policy formulated as a 0-1 integer linear programming problem. The primary goal is to minimize the total energy consumption of the mobile device while adhering to two critical constraints:
- Latency: Tasks must be completed within a specific deadline ($D_i$).
- Accuracy: The execution layer must meet a minimum accuracy threshold ($A_i$).
A Branch and Bound algorithm is employed to make real-time decisions on whether a task—such as facial expression analysis using VGGNET or voice analysis via an AlexNet-based DCNN—should be processed locally, offloaded to the edge, or sent to the cloud.
Prototype Validation
The researchers implemented a physical prototype to validate the MULTI-EASE architecture in a real-world scenario.
The experimental setup included:
- Robot Terminal: An ARM-based interactive robot.
- Edge Infrastructure: A communication gateway and a local server (Intel Core 2 Quad) acting as a "Super Edge Node."
- Cloud Platform: A remote server (AMD FX 8-Core) for heavy analytics.
Numerical simulations were also conducted to stress-test the system with up to 700 concurrent requests, revealing the "breaking point" of local hardware and the stability provided by edge offloading.
Key Insights and Results
- Energy Efficiency: The MULTI-EASE scheduling policy consistently outperformed "local-only," "cloud-only," and "random" strategies in terms of average energy savings.
- Latency Stability: While local processing latency spiked exponentially under heavy loads, the edge-based approach kept end-to-end delay stable, facilitating natural human-robot interaction.
- Data Size Sensitivity: The system is optimized for data efficiency. For tasks smaller than 55 MB, offloading is highly beneficial. However, for extremely large data sets, the energy cost of transmission may favor local execution if hardware permits.
Conclusion and Future Work
MULTI-EASE provides a robust and sustainable platform for emotion analysis at the edge. By balancing the trade-offs between energy, latency, and accuracy, it enables more responsive and long-lasting IoT applications. Future research will focus on addressing storage constraints for massive multi-modal datasets, improving adaptability to volatile network conditions, and incorporating privacy-preserving mechanisms for sensitive facial and voice data.
Summary updated