Abstract
In the Web 2.0 era, traditional scholarly metrics like citation counts and journal impact factors are no longer sufficient to capture the immediate and multidimensional nature of scientific communication. This study investigates the dimensions and interaction structures of altmetric indicators using a robust dataset of 33,128 academic articles from the Public Library of Science (PLOS). Through Spearman rank correlation, multidimensional scaling (MDS), and social network analysis (SNA), we identified three distinct clusters of metrics: traditional, active altmetrics, and inactive altmetrics. Our findings suggest a "transfer phenomenon" where active altmetrics, such as blog posts and comment threads, serve as intermediate stations that accelerate an article's social impact toward formal academic citations.
Introduction: Beyond the Impact Factor
For decades, the "Impact Factor" and citation counts have served as the gold standards for measuring scientific importance. However, these metrics are inherently slow, often taking years to reflect the true reach of a paper. In the modern "Web 2.0" landscape, scholars increasingly use social media, blogs, and bookmarking tools to share and discuss research.
These "altmetrics" offer a more immediate, open, and individualized measure of impact. Yet, until recently, the internal structure of these indicators—how they relate to each other and to traditional citations—remained a "black box." This research seeks to reveal the underlying dimensions of altmetrics and the pathways through which they interact to form a cohesive picture of scholarly influence.
Methods and Data
The researchers analyzed a massive dataset of article-level metrics (ALM) from the PLOS API, covering 33,128 articles. They tracked 20 distinct indicators, ranging from traditional citations (CrossRef, Scopus) and usage data (HTML views, PDF downloads) to social signals (tweets, blog posts, and Mendeley bookmarks).
Because altmetric data is typically non-normal and positively skewed, the team employed non-parametric statistical methods:
- Spearman Rank Correlation: Used to measure the strength and direction of relationships between the 20 variables.
- Multidimensional Scaling (MDS): A visualization technique that maps the correlation matrix into a two-dimensional space, allowing researchers to see which metrics naturally cluster together.
- Social Network Analysis (SNA): Using software like NetDraw, the metrics were treated as nodes in a network to identify "hubs" and "bridges" that connect different types of impact.
Key Insights
The Pareto Law of Scholarly Attention
The study found that downloads and page views follow the Pareto law, commonly known as the 80/20 rule. Approximately 80% of the total online "buzz" is generated by only 20% of the articles. This suggests that while altmetrics provide a broader view, scholarly attention remains highly concentrated on a subset of high-impact work.
Three Dimensions of Impact
The MDS analysis revealed that the 20 indicators fall into three distinct functional clusters:
- Traditional Metrics: This cluster includes formal citations and, interestingly, usage data like HTML views and PDF downloads. The extremely high correlation between views and downloads (rho = .899) suggests these are essentially two sides of the same "usage" coin, which serves as a precursor to formal citation.
- Active Altmetrics: This group includes trackbacks, ratings, and comment threads. These represent deep, active engagement where the audience is interacting with the content rather than just consuming it.
- Inactive/Niche Altmetrics: These are less frequent indicators, such as specific social bookmarking services (e.g., CiteULike), which provide unique but less dense signals of impact.
The Impact Transfer Phenomenon
The most compelling discovery is the "transfer phenomenon." Social network diagrams identified specific indicators—notably blog posts (ResearchBlogging.org) and comment/note threads—that act as "transfer stations." These metrics have the highest number of ties to other groups, functioning as the connective tissue that transports an article's social momentum into the realm of formal academic citations.
Conclusion
Altmetrics are not merely a replacement for traditional metrics but a vital complement to them. They are superior to traditional filters for assessing scholarly impact across multiple dimensions and social structures. By understanding the transfer paths from active social engagement to formal citations, academic institutions can better evaluate the multifaceted impact of research in the digital age. Future evaluations should consider not only traditional citations but also "active altmetrics" to capture the full spectrum of a researcher's contribution.
Summary updated