Five minutes with Prabhakar Raghavan: big data and social science at Google

Prabhakar Raghavan, Rebecca MannLondon 2012

This document is an interview transcript rather than a formal research paper. However, it contains significant strategic insights regarding the intersection of data science and the humanities. Here is a summary tailored for a technical and academic audience.

TL;DR

The tech industry has mastered the "what" of big data through computer science and microeconomics, but it still struggles with the "why" of human behavior. Prabhakar Raghavan argues that the next frontier for Silicon Valley lies in bridging the gap between the statistical rigour of big data and the deep, qualitative insights of social sciences like sociology and ethnography.

What Problem Does This Paper Address?

The "problem" is a fundamental disconnect between the capabilities of tech giants and the needs of social science researchers.

Tech firms like Google and Facebook possess "shallow" but massive datasets—billions of data points on user actions (clicks, page loads) without a clear understanding of the underlying motivation. Conversely, traditional social scientists deal with "deep" but small datasets—studying 50 people intensely over years to understand cultural drivers. Raghavan identifies a "supply chain" issue: there is a lack of researchers who can "straddle" these disciplines to turn massive, anonymized data ensembles into robust social insights.

Methods

As this is an interview, it does not present a single experimental methodology. Instead, it outlines a collaborative framework for interdisciplinary work:

  • Common Language: Using mathematics (specifically game theory and auction design) as the bridge between computer scientists and economists.
  • Cross-Disciplinary Hiring: Placing academic social scientists directly into product groups to reframe "algorithmic problems" as "behavioral or economic problems."
  • Anonymized Ensembles: Utilizing privacy-preserving data analysis to allow social scientists to contemplate "big social problems" without compromising individual user privacy.

Datasets

The text discusses categories of data rather than specific named datasets:

  • Industry Big Data: Terabytes of "voluminous but shallow" data representing the daily actions of billions of people.
  • Social Science Data: Traditionally "small but deep" datasets (e.g., ethnographic field notes, small-group longitudinal studies).
  • The Gap: The interview highlights that social scientists currently lack access to the "big data" held by the tech sector, which limits their ability to test theories at scale.

Key Insights and Ideas

  • The "Math" Bridge: The primary reason microeconomics has succeeded in tech (to the tune of billions of dollars in advertising revenue) is that economists and computer scientists share the language of mathematics. Other social sciences (sociology, anthropology) struggle because they lack this shared default language for reasoning.
  • The "Why" vs. The "What": Computer scientists are "custodians of big data" who measure clicks. Social scientists ask about the "essence of human behavior." The opportunity lies in the confluence of the two: using big data to answer the "why."
  • The Monetary Imperative: Raghavan notes that collaboration flourishes when social science can be tied to the bottom line. Once a CEO sees that a microeconomic insight can "make another billion dollars," the barriers to interdisciplinary work vanish.
  • Institutional Lag: Universities operate on a timescale of decades to create new disciplines, while industry operates in quarters. This creates a talent vacuum for "straddle" researchers who understand both code and culture.

Key Results

  • Economic Impact: The application of game theory and auction design to search and advertising has materially impacted the industry, generating hundreds of billions of dollars.
  • Funding Shifts: In the US, major federal departments (Homeland Security, Defense, Energy) are becoming the biggest funders of this interdisciplinary research, signaling its growing importance to national infrastructure and security.
  • Academic Interest: Workshops on these topics are now attracting more academics than industry professionals, suggesting a shift in the research zeitgeist toward "computational social science."

Limitations and Future Work

  • Nascent Influence: The influence of non-economic social sciences in tech is still in its infancy ("early days").
  • Lack of Commonality: There is currently no "commonality of language" between ethnographers and software engineers, which remains a major barrier to collaboration.
  • Privacy Constraints: All future opportunities must be approached with "utmost regard for individual privacy," restricting analyses to anonymized data, which can sometimes limit the depth of social insight.
  • Future Direction: The goal for the next 20 years is to combine the "big and deep questions" of social science with the "statistical rigour" of computer science.

Summary updated

Abstract

Part of PPG’s Impact of Social Sciences project focuses on how academic research in the social sciences influences decision-makers in business, government and civil society. We will cover a series of salient viewpoints emerging from this interview programme on the blog over the next three months. To launch the series Rebecca Mann talked to Prabhakar Raghavan, who is Vice President of Strategic Technologies at Google, and Consulting Professor of Computer Science at Stanford. He explains the role that social scientists are already playing in the development of the tech sector in Silicon Valley, and discusses the opportunities for impact and some remaining obstacles to collaboration.
BibTeX
@inproceedings{raghavan2012five,
  author = {Prabhakar Raghavan and Rebecca Mann},
  title = {{Five minutes with Prabhakar Raghavan: big data and social science at Google}},
  booktitle = {London},
  year = {2012},
}
Figure