Introduction
Financial earnings calls serve as a critical communication channel between corporate executives and the investment community. While traditional analysis has focused on the textual content of these calls, recent advancements in multimodal AI have demonstrated that acoustic features—such as pitch, tone, and intensity—provide significant predictive power regarding future stock market volatility. However, the corporate landscape is characterized by a profound gender imbalance, with women severely underrepresented in executive roles. This study investigates whether state-of-the-art multimodal models, trained on datasets dominated by male voices, inherit and amplify gender bias in their financial risk assessments.
Methodology: The Multimodal Deep Regression Model (MDRM)
The researchers evaluated a Multimodal Deep Regression Model (MDRM) designed to integrate linguistic and paralinguistic signals. The architecture, illustrated in Figure 1, processes text and audio through separate pipelines before fusion.
- Text Processing: Textual embeddings are generated from transcripts using FinBERT, a BERT variant pre-trained specifically on financial communications to capture domain-specific nuances.
- Audio Processing: The system extracts 26 distinct acoustic features using the Praat toolkit, covering pitch analysis, intensity, and voice stability metrics like jitter and shimmer.
- Fusion and Prediction: A Bidirectional Long Short-Term Memory (BiLSTM) network merges these modalities to predict stock volatility across multiple time horizons ($\tau = 3, 7, 15, 30$ days).
To measure bias, the study employs Error Disparity ($\Delta G$), defined as the difference in Mean Square Error (MSE) between female and male speakers ($\Delta G = MSE_F - MSE_M$).
Dataset and Demographics
The study utilized a benchmark dataset comprising 559 public earnings calls from 277 S&P 500 companies in 2017. The data reflects the "glass ceiling" reality of the corporate world: only 12% of the speakers are female, while 88% are male. Genders were verified using self-reported data from Reuters, Crunchbase, and Wikidata.
Key Results: The Performance-Fairness Trade-off
The experimental results reveal a "multimodality paradox": while adding audio features improves the model's overall predictive accuracy, it simultaneously exacerbates gender bias.
As shown in Figure 5, the model consistently achieves lower error rates for male speakers across all modalities and time horizons. For instance, in the 3-day volatility prediction task using audio features, the MSE for female executives was 1.059, significantly higher than the 0.684 recorded for males. Even in the combined "Audio + Text" model, which achieved the best overall MSE (0.185 for $\tau=30$), the disparity remained stark.
Analysis of Bias Amplification
The research identifies several drivers of this disparity:
- Data Underrepresentation: The 88/12 split prevents the model from learning a robust representation of female acoustic profiles.
- Acoustic Semantic Bias: Statistical analysis showed that 13 of the 26 audio features (particularly those related to voice stability) differed significantly by gender. The model often misinterpreted these gender-specific traits as indicators of financial risk.
- Bias Amplification: The model does not merely reflect the data imbalance; it amplifies it. As the training ratio of male-to-female data increases, the error rate for female executives rises disproportionately, suggesting the model develops "imperfect generalizations" based on the majority distribution.
Discussion and Limitations
This investigation highlights a critical ethical challenge in financial AI. The higher error rates for female executives could lead to unfair risk assessments, effectively imposing a "performance tax" where women must deliver superior acoustic performances to be judged as accurately as their male counterparts.
The study is limited by its focus on the S&P 500 and a binary definition of gender. Future research should expand to include diverse accents, ethnicities, and ages. Furthermore, developing de-biasing algorithms—such as data augmentation or cross-domain adaptation—is essential to ensure that multimodal financial models provide equitable assessments for all leaders.
Summary updated