A dynamical measure of algorithmically infused visibility

Shaojing Sun, Zhiyuan Liu, David Waxman

Abstract

In modern "algorithmically infused" societies, human behavior and algorithmic sorting mechanisms are deeply intertwined. This research proposes a quantitative measure of visibility, $V(D)$, designed to capture how topics trend on social media platforms. By analyzing a topic's rank trajectory—the sequence of ranks it holds over time—the measure incorporates a tunable "discrimination level" ($D$) to weight the importance of top-tier rankings versus total duration. Empirical analysis of a large-scale dataset from Weibo demonstrates that this measure can explain up to 68% of the variability in a topic's total viewership, offering a robust tool for communication studies, marketing, and political science.

The Challenge of Quantifying Visibility

Visibility in the digital age is more than just a snapshot of a "Trending" list; it is a dynamic process. While being ranked #1 is clearly more impactful than being ranked #50, existing metrics often struggle to quantify the trade-off between a short burst at the top and a long-lasting presence at the bottom of a list. This paper addresses the need for a standardized, operational measurement of visibility that accounts for the interplay between human attention and algorithmic curation.

Defining the Visibility Metric $V(D)$

To quantify visibility, the researchers treat a topic's lifecycle as a "rank trajectory." This trajectory is a series of time-stamped ranks $(t_i, R_i)$, where $t_i$ represents a specific time interval and $R_i$ represents the rank the topic held during that interval.

Table 1: Representation of a topic's rank trajectory over time, showing the relationship between time intervals and assigned ranks.

The proposed visibility measure is calculated using the following formula:

$$V(D) = \sum_{i=1}^{f} \frac{1}{R_i^D}$$

In this equation, $f$ is the total number of time intervals, $R_i$ is the rank at time $i$, and $D$ is the discrimination level. The $D$ parameter is the core innovation of this model, acting as a "knob" that adjusts how much weight is given to higher ranks:

  • $D = 0$: The metric simply counts the total time spent on the trending list, treating all ranks as equal.
  • High $D$: Higher ranks (smaller $R$) are weighted exponentially more than lower ranks. For instance, at $D=3$, a single minute at rank 1 is mathematically equivalent to 27 minutes at rank 3.

Empirical Validation: The Weibo Dataset

The researchers tested this metric against a massive dataset crawled from Weibo, one of China’s largest social media platforms. The dataset included over 5.6 million entries covering 27,012 unique topics between December 2022 and March 2023. By capturing data at one-minute intervals for the "Top 50" trending list, the team could precisely map the trajectories of diverse content types.

Key Results and "Ranking Cultures"

The study found that the $V(D)$ measure is highly effective at predicting total reads ($N_{reads}$). For categories like Sports and Domestic News, the measure explained between 65% and 68% of the variability in viewership ($R^2 \approx 0.66-0.68$).

Interestingly, the "optimal" discrimination level ($D_{max}$) varied significantly by category, revealing what the authors call "ranking cultures":

  • Humor/Entertainment: These topics had a lower $D_{max}$ (~0.90), suggesting users are more willing to scroll down the list to find entertaining content.
  • Animation and Social News: These categories had much higher $D_{max}$ values (~1.80), indicating that for these topics, visibility is almost entirely concentrated at the very top of the rankings.

Implications and Future Work

The $V(D)$ metric provides a powerful lens for understanding how algorithmic affordances translate into actual human attention. It demonstrates that visibility is not just about the code of the algorithm, but how users interact with that code within specific cultural contexts.

While the current study relied on aggregate viewership data, future research could refine the model by incorporating real-time click data and semantic analysis of headlines. Furthermore, applying this measure to other platforms like X (Twitter) or TikTok could reveal how different algorithmic ecosystems foster unique ranking cultures.

Summary updated

Abstract

This work focuses on the nature of visibility in societies where the behaviours of humans and algorithms influence each other - termed algorithmically infused societies. We propose a quantitative measure of visibility, with implications and applications to an array of disciplines including communication studies, political science, marketing, technology design, and social media analytics. The measure captures the basic characteristics of the visibility of a given topic, in algorithm/AI-mediated communication/social media settings. Topics, when trending, are ranked against each other, and the proposed measure combines the following two attributes of a topic: (i) the amount of time a topic spends at different ranks, and (ii) the different ranks the topic attains. The proposed measure incorporates a tunable parameter, termed the discrimination level, whose value determines the relative weights of the two attributes that contribute to visibility. Analysis of a large-scale, real-time dataset of trending topics, from one of the largest social media platforms, demonstrates that the proposed measure can explain a large share of the variability of the accumulated views of a topic.
BibTeX
@misc{sun2024dynamical,
  author = {Shaojing Sun and Zhiyuan Liu and David Waxman},
  title = {{A dynamical measure of algorithmically infused visibility}},
  year = {2024},
  eprint = {2412.04735},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2412.04735},
}
Figure