TL;DR
This paper bridges the gap between subjective self-reported arousal and objective physiological responses (GSR) during movie watching. By developing a novel temporal correlation framework, they demonstrate that GSR can indeed serve as a reliable indicator of arousal, validating its use in affective computing.
What Problem Does This Paper Address?
Affective computing aims to understand and respond to human emotions, with arousal being a key dimension. While Galvanic Skin Response (GSR) is known to correlate with arousal, its temporal relationship with continuous, real-time self-assessments of arousal has not been thoroughly investigated, especially in the context of dynamic media like movies. Existing studies often focus on discrete correlations or specific emotional categories. This paper tackles the challenge of validating the temporal synchronicity between subjective arousal reports and objective GSR signals, providing a more robust understanding of how physiological responses reflect emotional states over time.
Methods
The authors propose a framework for comparing continuous arousal self-assessments with real-time GSR signals.
- GSR Signal Processing: Raw GSR signals undergo several preprocessing steps:
- Low-pass filtering to remove noise.
- Thresholding to focus on positive segments of the derivative, as these indicate increased sweating related to arousal.
- Temporal filtering and sub-sampling to achieve a 5-second time resolution.
- Normalization to remove user-dependent variations.
- Weighted Mean GSR Profile: To account for potential outliers and individual differences in GSR signals, a weighted mean GSR profile is computed for each movie. This involves calculating the agreement between individual GSR profiles using Spearman's rank correlation and assigning confidence scores to participants.
- Arousal Peak Identification: Continuous arousal self-assessments are processed to identify "arousal peaks." These are defined as the highest scores of the mean continuous arousal annotations above a certain threshold (T), which represents the percentage of lowest values to be removed.
- Correlation Calculation: The temporal correlation between the identified arousal peaks and the corresponding values from the weighted mean GSR profile is calculated using Pearson's r and Spearman's rank correlation coefficient (SRCC).
Datasets
- LIRIS-ACCEDE Database: The study utilizes 30 full-length movies from the LIRIS-ACCEDE database, chosen to represent a variety of genres and languages. This database is publicly available and contains video excerpts for affective content analysis.
Key Insights and Ideas
- Temporal Validation is Crucial: The paper highlights the importance of analyzing the temporal correlation between arousal and GSR, moving beyond static or discrete measures.
- GSR as a Reliable Arousal Indicator: The findings strongly suggest that GSR, when properly processed, can reliably track continuous arousal fluctuations in response to dynamic stimuli like movies.
- Arousal Peaks Matter: Focusing on the most intense moments ("arousal peaks") in the movie is key to establishing a significant correlation between GSR and self-reported arousal. Lower arousal periods might not show a strong GSR response.
- Subject-Independent Approach: By using a weighted mean GSR profile and averaging across participants, the study aims for a subject-independent validation, making the findings more generalizable.
Key Results
- Global Correlations: The study achieved a global Pearson's correlation of 0.264 and a Spearman's rank correlation coefficient (SRCC) of 0.336 between arousal peaks and weighted mean GSR profiles across all 30 movies.
- Movie-Specific Correlations: Table 2 shows that for most movies, significant correlations were found. For example, "Lesson Learned" achieved a Pearson's r of 0.754* and SRCC of 0.461*.
- Threshold Effect: The analysis of the threshold T revealed that correlations are strongest when focusing on the most intense arousal peaks, with a global maximum correlation observed around T=46%.
Limitations and Future Work
- Limitations:
- Five movies (Cloudland, Origami, Payload, The Room of Franz Kafka, and Spaceman) showed low and non-significant correlation values. The authors attribute this to potential issues with movie quality (e.g., lack of direct links between scenes) or unconventional/abstract styles that might elicit varied physiological responses.
- The study focused solely on arousal.
- Future Work:
- Investigate the temporal relationship between continuous valence annotations (also available for the dataset) and other physiological signals.
- Explore the use of other relevant physiological signals beyond GSR.
- Further investigate the reasons for low correlations in specific movie types.
Summary updated