TL;DR
This paper introduces a Cyber-Physical System (CPS) that uses integrated AI to predict and influence public emotions within a mixed virtual-physical environment. By fusing sensor data from the real world with controllable elements in a virtual space, the system aims to optimize service experiences by guiding users towards desired emotional states.
What Problem Does This Paper Address?
Creating effective service environments often hinges on managing customer emotions, as emotions significantly impact satisfaction and purchasing decisions. However, directly controlling emotions in a physical space is challenging due to the complexity and limitations of environmental stimuli. Existing approaches often focus on individual emotion recognition, neglecting the collective emotional response within a shared space. This paper tackles this by proposing a system that leverages both physical and virtual environmental factors to proactively induce target emotions in a public setting.
Methods
The core of the proposed system is a two-part emotion control mechanism:
- Emotion Prediction Module: This module uses modular Bayesian Networks (BNs) to predict emotions. It takes inputs from both the physical space (temperature, humidity, dust, current emotion) and the virtual space (illumination, color temperature, video, sound, volume). The BN approach is chosen for its ability to handle uncertainty and complexity inherent in sensor data and emotional states. The model is modularized into a general module, domain selection modules, and domain-specific modules to allow for adaptation to different environments.
- Stimulus Decision Module: This module determines the optimal environmental stimuli to guide the predicted emotion towards a target emotion. It employs utility theory, where a utility table, initialized with domain knowledge and updated via reinforcement learning (Q-learning), quantifies the influence of stimuli on emotions. The system calculates the expected utility of each stimulus and selects the combination that maximizes this value to achieve the target emotion.
The system integrates these modules within a virtual-physical environment built using Unity3D, allowing users to experience the virtual space via VR equipment while interacting with physical sensors and actuators.
Datasets
The paper does not explicitly mention pre-existing datasets for training. Instead, it describes an experimental setup where data was collected from human participants.
- Experimental Data: Data was collected from ten undergraduate and graduate students at Yonsei University. For each scenario, emotions were measured every 20 seconds, and participants input their current emotion using controller buttons corresponding to four categories: positive-arousal, negative-arousal, negative-relax, and positive-relax. A total of 858 data points were collected.
Key Insights and Ideas
- Bridging Physical and Virtual for Emotion Control: The paper demonstrates a novel approach to emotion control by synergistically combining real-world environmental factors with dynamically controllable virtual elements. This expands the range of stimuli beyond what's feasible in a purely physical space.
- Bayesian Networks for Robust Emotion Prediction: The use of modular Bayesian Networks proves effective in predicting emotions, outperforming other classification techniques in accuracy. This highlights the suitability of probabilistic models for handling the inherent uncertainty in sensor data and emotional states.
- Reinforcement Learning for Adaptive Stimulus Optimization: Employing Q-learning to update utility tables allows the system to adapt and refine its stimulus selection based on real-world feedback, moving beyond static domain knowledge.
- Scenario-Based Evaluation: The structured approach of designing specific emotion scenarios (e.g., positive-negative, arousal-relax transitions) provides a controlled way to test the system's ability to induce target emotions under various conditions.
Key Results
- Emotion Prediction Accuracy: The proposed Bayesian Network model achieved an average prediction accuracy of 78.4%, which was significantly higher (27.6% higher) than the average accuracy of other tested methods (DT, MLP, KNN, SVM), which averaged 50.81%.
- Emotion Achievement: The overall emotion achievement rate across all scenarios was 71.79%. Scenario 1 showed the highest achievement (76.64%), while Scenario 4 had the lowest (66.08%). The achievement generally improved as training data accumulated and the utility values were updated.
- Scenario-Specific Performance: Figures 15-19 illustrate that emotion achievement tends to increase over time within each scenario, indicating the system's learning and adaptation capabilities.
Limitations and Future Work
- Limited Participant Pool: The study involved only ten participants, which is a relatively small sample size. The authors suggest increasing the number of participants for more robust validation.
- Virtual Environment Realism: While the virtual environment was designed as a 3D model of a shopping store, further improvements in realism are needed to enhance user immersion and experience.
- Input Method Flexibility: The current input method for emotions is limited to controller buttons. Future work could explore more flexible and natural input methods.
- Scalability and Server Infrastructure: To accommodate a larger number of participants, the authors propose developing a separate server to run the proposed system.
- Diverse Environments: The system's performance needs to be evaluated in various different environments beyond the simulated shopping store.
Summary updated