Abstract: In this paper, we seek to understand how politicians use images to express ideological rhetoric through Facebook images posted by members of the U.S. House and Senate. In the era of social media, politics has become saturated with imagery, a potent and emotionally salient form of political rhetoric which has been used by politicians and political organizations to influence public sentiment and voting behavior for well over a century. To date, however, little is known about how images are used as political rhetoric. Using deep learning techniques to automatically predict Republican or Democratic party affiliation solely from the Facebook photographs of the members of the 114th U.S. Congress, we demonstrate that predicted class probabilities from our model function as an accurate proxy of the political ideology of images along a left-right (liberal-conservative) dimension. After controlling for the gender and race of politicians, our method achieves an accuracy of 59.28% from single photographs and 82.35% when aggregating scores from multiple photographs (up to 150) of the same person. Our findings suggest that conservatives tend to use more images supporting status quo political institutions and hierarchy maintenance, featuring individuals from dominant social groups, and displaying greater happiness than liberals.
Introduction
In the contemporary political landscape, imagery has become a primary vehicle for ideological rhetoric. While political science has traditionally relied on "hard" data such as roll-call voting records (e.g., DW-NOMINATE scores) or text-based analysis of speeches and tweets, the visual dimension of political communication remains under-studied at scale. This research addresses whether a "visual grammar" exists for political ideology, investigating if machine learning can detect subtle cues—objects, people, and expressions—that signal a politician's values and align with their legislative behavior.
Methodology
The study utilizes a multi-stage computer vision pipeline to analyze a newly collected Facebook Legislator Dataset comprising 296,461 images from 319 members of the 114th U.S. Congress.
The researchers fine-tuned a ResNet-34 convolutional neural network (CNN) to classify images by party affiliation. To prevent the model from relying on demographic shortcuts, such as the higher diversity within the Democratic party, the training was conducted on a balanced subset of white male politicians. Additional specialized CNNs (trained on ExpW, CelebA, and UTKFace) were used for facial analysis, while Grad-CAM and the Google Vision API provided interpretability by highlighting salient regions and generating semantic labels for objects and scenes.
Visual Rhetoric and Ideological Signals
The analysis revealed distinct "visual rhetorics" associated with different political ideologies. Conservative imagery frequently employs "hierarchy-maintaining" symbols, such as the American flag, formal business attire, and military personnel.
In contrast, liberal imagery tends toward "change-oriented" elements, featuring diverse crowds, protest signs, and informal settings. A notable "happiness gap" was also observed: Republicans and their associates displayed "happy" facial expressions (smiling) more frequently than their Democratic counterparts. This finding aligns with psychological surveys regarding subjective well-being, despite contradicting some recent text-based sentiment studies.
Quantitative Performance
The model's predictive power was validated against DW-NOMINATE scores, the gold standard for measuring legislative ideology. The results demonstrated a strong correlation ($r = 0.67$) between a politician's "visual identity" and their actual voting record.
- Single Image Accuracy: 59.28%
- Aggregated Accuracy (150 images): 82.35%
- Feature Importance: Labels such as "military officer" and "troop" were top predictors for Republicans, while "nature reserve," "sea," and "education" were strongest for Democrats.
Conclusion
This research demonstrates that visual branding on social media is a highly consistent proxy for legislative behavior. While humans and AI perform similarly on single-image classification (~61-63%), the AI's ability to aggregate subtle environmental cues across thousands of images provides a powerful new tool for political science. Future work may explore how these visual signals evolve during national crises or apply these methods to international political contexts.
Summary updated