Cost Function Learning in Memorized Social Networks with Cognitive Behavioral Asymmetry

Yanbing Mao, Jining Li, Naira Hovakimyan, Tarek F. Abdelzaher, Christian Lebiere

TL;DR

This paper introduces $M^3IRL$, a novel Inverse Reinforcement Learning framework designed to uncover the hidden motivations (cost functions) of actors in social networks. Unlike traditional methods, it explicitly accounts for non-Markovian human memory and asymmetric cognitive biases—like confirmation and novelty bias—requiring only a single trajectory of observed behavior to make accurate inferences.

What Problem Does This Paper Address?

In the fight against misinformation and digital polarization, game-theoretic models are powerful tools. However, these models typically assume we already know the "cost functions" (the underlying goals and priorities) of the players involved. Learning these functions from observation is a classic problem known as Inverse Reinforcement Learning (IRL).

Standard IRL approaches, often used in robotics, face two major hurdles in social contexts:

  1. Human Memory: Most IRL methods rely on the Markov Decision Process (MDP) principle, where the next state depends only on the current one. Humans, however, have memories. Our current opinions are shaped by a "memory horizon" of past information, breaking the MDP assumption.
  2. Cognitive Asymmetry: Human bias isn't symmetric. For example, George might value Alex’s opinion more than Bob’s, even if both are equally "distant" from his own, simply because Alex’s view aligns better with his existing beliefs (confirmation bias).

The authors set out to build a model that captures these human quirks and an algorithm that can infer motivations from limited, real-world social media data.

Methods

The authors propose a two-part solution:

1. A "Memorized" Social Diffusion Model

They develop a dynamic model (Eq. 1) where an individual's opinion at time $k+1$ is a function of their subconscious bias, their own memory, and the influence of their neighbors. Crucially, the influence weights are state-dependent and capture:

  • Confirmation Bias: Giving more weight to information that confirms existing beliefs.
  • Novelty Bias: A preference for outlying or "surprising" content.
  • Memory Horizons: Using a decaying influence function (inspired by the ACT-R cognitive architecture) to model how past information fades over time.
Figure 2. Modeling mechanism aims at capturing the cognitive behavior due to confirmation bias and novelty bias.

2. The $M^3IRL$ Algorithm

Standing for Memorized Model and Maximum-entropy based Inverse Reinforcement Learning, this algorithm avoids the need for massive datasets.

  • Local Optimality: Instead of assuming players are globally optimal, it assumes they are locally optimal, which is more realistic for human decision-making.
  • Single Trajectory: It can infer cost functions from just one finite-time sequence of opinions.
  • Bounded Variables: It handles the fact that opinions are typically bounded (e.g., between -1 for "completely supporting" and +1 for "completely opposing").
Figure 1. Model-based framework of cost function learning.

Datasets

The researchers validated their model using real-world data:

  • Twitter Vaccine Data: They collected tweets from nine users over nine months (Jan–Sept 2021) regarding COVID-19 vaccines.
  • Sentiment Encoding: Tweets were processed using a BERT sentiment analysis model trained on the Sentiment140 dataset, mapping opinions to a numerical range of [-1, 1].
  • Network Structure: They mapped the interaction digraph of these nine users, identifying seven "humans" and two "information sources" (targets for cost function learning).

Key Insights and Ideas

  • Asymmetry is Key: The paper formalizes why "distance" in opinion space isn't enough. As shown in their "Alex, Bob, and George" scenarios, the direction of the distance relative to one's own belief domain drastically changes influence weights.
Figure 3. Asymmetric cognitive bias explanation.
  • Breaking the MDP: By incorporating a memory horizon ($\tau_i$), the model moves beyond the "memoryless" limitations of standard reinforcement learning, making it far better suited for modeling human social behavior.
  • Dynamic Cost Functions: The authors demonstrate that cost functions aren't static; they evolve based on external stimuli (like news events), and $M^3IRL$ is sensitive enough to capture these shifts using recent trajectory data.

Key Results

The algorithm successfully identified the distinct motivations of the two observed information sources (IS8 and IS9):

  • IS8 (Anti-Vaccine): The learned cost function for IS8 showed a strong preference for steering public opinion toward the "opposing" end of the spectrum (-1).
  • IS9 (Pro-Vaccine): Conversely, IS9's cost function was heavily weighted toward steering the public toward "supporting" the vaccine (+1).
  • Influence Differential: The analysis revealed that IS9 had a significantly higher influence ($\alpha_9 = 0.7960$) on the community's evolving opinions compared to IS8 ($\alpha_8 = 0.2040$).
  • Behavioral Traits: Both sources showed a preference for spreading "outlying" opinions rather than persistent, unchanging ones, which was reflected in the sharp jumps observed in their opinion trajectories.

Limitations and Future Work

  • Scale: The empirical validation was performed on a small network of nine users. The authors acknowledge that scaling this to massive social networks is a significant future challenge.
  • Basis Function Library: The method relies on a predefined library of basis functions (e.g., steering toward +1, stubbornness). If the true motivation of a player isn't represented in the library, the inference may be inaccurate.
  • Future Directions: The team plans to investigate how to incorporate communication detection and classification to generalize the framework for large-scale social network analysis.

Summary updated

Abstract

This paper investigates the cost function learning in social information networks, wherein the influence of humans' memory on information consumption is explicitly taken into account. We first propose a model for social information-diffusion dynamics with a focus on systematic modeling of asymmetric cognitive bias, represented by confirmation bias and novelty bias. Building on the proposed social model, we then propose the M$^{3}$IRL: a model and maximum-entropy based inverse reinforcement learning framework for learning the cost functions of target individuals in the memorized social networks. Compared with the existing Bayesian IRL, maximum entropy IRL, relative entropy IRL and maximum causal entropy IRL, the characteristics of M$^{3}$IRL are significantly different here: no dependency on the Markov Decision Process principle, the need of only a single finite-time trajectory sample, and bounded decision variables. Finally, the effectiveness of the proposed social information-diffusion model and the M$^{3}$IRL algorithm are validated by the online social media data.
BibTeX
@article{mao2022cost,
  author = {Yanbing Mao and Jining Li and Naira Hovakimyan and Tarek F. Abdelzaher and Christian Lebiere},
  title = {{Cost Function Learning in Memorized Social Networks with Cognitive Behavioral Asymmetry}},
  journal = {IEEE Transactions on Computational Social Systems},
  year = {2022},
  doi = {10.1109/tcss.2022.3218485},
}
Figure