Modeling Diffusion of Competing Products and Conventions in Social Media

Isabel Valera, Manuel Gomez-Rodriguez, Krishna P. Gummadi

TL;DR

Researchers have developed a new continuous-time probabilistic model based on temporal point processes to track how users adopt and repeatedly use competing products in social networks. By incorporating factors like social influence, personal habit (recency), and competitive inhibition, the model significantly outperforms existing methods in predicting user behavior and can even detect sudden market shifts caused by external interventions.

What Problem Does This Paper Address?

While economists and marketers have studied product adoption for decades, most classical models (like the Bass diffusion model) focus on macroscopic trends—how a population adopts a product over time. They often fail to capture the "microscopic" reality: the fine-grained, recurrent usage patterns of individuals and the constant "tug-of-war" between competing products (e.g., choosing between Bitly or TinyURL).

Existing microscopic models typically treat adoption as a one-time event or fail to account for the social network's structure. There is a clear gap for a model that can handle large-scale, continuous-time data while simultaneously accounting for social influence, personal usage history, and the inhibitory effect of competition.

Methods

The authors propose a model based on Multivariate Hawkes Processes. Traditionally, Hawkes processes are "self-exciting," meaning one event increases the probability of future events (like aftershocks following an earthquake).

The Key Innovation: The authors modified the standard Hawkes framework to allow for both excitation (positive influence) and inhibition (negative influence).

  • Recency: If you used a product recently, you’re more likely to use it again (habit).
  • Social Influence: If your friends use a product, you’re more likely to start.
  • Competition: If you or your friends start using a competing product, your intensity for the original product decreases.

To make this computationally feasible for massive social networks, they formulated the parameter estimation as a convex optimization problem. Crucially, this problem "decouples," meaning it can be broken down into thousands of smaller, independent sub-problems (one per user-product pair) that can be solved in parallel. For simulation, they optimized Ogata’s thinning algorithm to exploit the sparsity of social networks, allowing it to scale linearly with the number of products.

Datasets

The model was validated using both synthetic and real-world data:

  1. Synthetic Networks: Generated using Kronecker graph models to simulate different social structures (core-periphery, hierarchical, and random).
  2. Twitter URL Shorteners: A dataset tracking the use of seven popular services (e.g., Bitly, TinyURL, Isgd) among the 3,000 most active users.
  3. Twitter Retweet Conventions: Tracking how users switched between different ways of retweeting (e.g., using "RT", "via", or "retweet") before Twitter introduced an official button in 2009.

Key Insights and Ideas

  • Habit vs. Influence: The data revealed that for popular products (like Bitly), usage is mostly driven by "spontaneous use" and habit. However, for less popular "niche" products, users usually need a "social nudge" from a neighbor to start, after which habit (recency) takes over.
  • Asymmetric Competition: The model captured interesting competitive dynamics. For example, Bitly had a strong inhibitory effect on TinyURL, but TinyURL had almost no effect on Bitly, suggesting Bitly was already the dominant competitor before it became the Twitter default.
  • Detecting "Black Swan" Events: The authors showed that when Twitter suddenly changed its default URL shortener in May 2009, the model’s "goodness of fit" (log-likelihood) plummeted. This suggests the model can be used as a diagnostic tool to detect when an external force—rather than social or personal factors—is driving market changes.

Key Results

  • Superior Prediction: On real Twitter data, the model achieved a prediction probability of 81-89%, vastly outperforming standard Poisson (32-45%) and Weibull (58-76%) models.
  • Better Fit: Using the Akaike Information Criterion (AIC), the model consistently provided a better trade-off between accuracy and complexity than all baselines for over 65% of users.
  • Scalability: The decoupled inference method allowed the researchers to fit over 50 million parameters by solving over a million convex optimization problems, proving the method is ready for "production-scale" social networks with hundreds of thousands of nodes.

Limitations and Future Work

The authors acknowledge several areas for improvement:

  • Simple Kernels: The current model uses a simple exponential decay for influence over time. Future versions could use non-parametric kernels to capture more complex temporal patterns.
  • Uniform Influence: The model currently assumes all neighbors have the same influence. In reality, a close friend's recommendation likely carries more weight than a distant acquaintance's.
  • Product Quality: The model focuses on behavior and network effects but doesn't explicitly account for the intrinsic quality or features of the products themselves.
  • Platform Specificity: The real-world validation was limited to Twitter; testing on other platforms like Facebook or Weibo would help confirm the model's generalizability.

Summary updated

Abstract

The emergence and wide-spread use of social networks and microblogging sites has led to a dramatic increase on the availability of users’ activity data. Importantly, this data can be exploited to solve some of the problems that have captured the attention of economists and marketers for decades as, e.g., product adoption, product competition and product life cycle. In this paper, we leverage on users’ activity data from a popular microblogging site to model and predict the competing dynamics of products and social conventions adoptions. To this aim, we propose a data-driven model, based on continuous-time Hawkes processes, for the adoption and frequency of use of competing products and conventions. We then develop an inference method to efficiently fit the model parameters by solving a convex program. The problem decouples into a collection of smaller subproblems, thus scaling easily to networks with hundred of thousands of nodes. We validate our method over synthetic and real diffusion data gathered from Twitter, and show that the proposed model does not only present a good predictive power but also provides interpretable model parameters, which allow us to gain insights into the fundamental principles that drive product and convention adoptions.
BibTeX
@inproceedings{valera2014modeling,
  author = {Isabel Valera and Manuel Gomez-Rodriguez and Krishna P. Gummadi},
  title = {{Modeling Diffusion of Competing Products and Conventions in Social Media}},
  booktitle = {MPG.PuRe},
  year = {2014},
}

Loading related content...

Figure