Introduction
In the current academic landscape, the sheer volume of scientific output has made manual oversight of research communities nearly impossible. Traditional evaluation methods often rely on "shallow" metrics like the h-index or raw citation counts. However, these metrics fail to distinguish between positive and negative citations, are susceptible to manipulation via self-citation, and often disadvantage emerging researchers. ACE 2.0 (Academic Community Explorer) addresses these limitations by providing an end-to-end automated system for the digital profiling of researchers, moving beyond simple counts toward a nuanced "Semantic Index."
System Architecture
ACE 2.0 is designed as a modular pipeline that transforms raw input documents into a comprehensive visualization of a scientific community's health and evolution. The system integrates state-of-the-art models for extraction, sentiment analysis, and graph-based profiling.
Core Modules
Bibliographic Reference Extraction (DeepBiRD)
A critical challenge in profiling is the accurate extraction of references from diverse document styles. ACE 2.0 utilizes DeepBiRD, which adopts a layout-based approach rather than traditional text parsing. By treating documents as images, it employs a Mask-RCNN architecture to detect references based on visual cues such as indentation and line spacing.
Sentiment Analysis (ImpactCite)
To provide a qualitative assessment of impact, the ImpactCite module uses an XLNet-based model to classify citation intent. It categorizes citations as Positive, Negative, or Neutral, allowing the system to filter out critiques and focus on constructive influence.
Keyword and Topic Modeling (CoCoNoW)
The CoCoNoW module extracts keywords by building a "Connectivity-Aware Graph." These keywords are mapped to the Computer Science Ontology (CSO), enabling the system to track the dynamic evolution of research topics over time and identify emerging trends.
The Semantic Index
The core innovation of ACE 2.0 is the Semantic Index. Unlike the h-index, this metric calculates a researcher's score by multiplying the number of positive, non-self-citations by a composite of five graph centrality measures (Degree, Eigenvector, Betweenness, Closeness, and Indegree). This approach allows for the identification of specific community roles, such as "Opinion Leaders" or "Community Connectors."
Evaluation and Results
The individual components of ACE 2.0 demonstrated high performance across established benchmarks:
- DeepBiRD: Achieved state-of-the-art results with an AP50 of 97.59% on the BibX dataset and 98.56% on BibLy.
- ImpactCite: Outperformed standard BERT and ALBERT models with a Macro-f1 score of 77.73 in sentiment classification.
- CoCoNoW: Consistently surpassed traditional methods like TF-IDF and TextRank in keyword extraction accuracy across multiple datasets (Hulth2003, SemEval2010, and NLM500).
Discussion and Conclusion
ACE 2.0 represents a significant step toward more transparent and content-oriented research assessment, aligning with the goals of the Declaration on Research Assessment (DORA). By leveraging layout-based extraction and graph theory, the system provides a fairer evaluation of researcher impact.
Future work will focus on improving author disambiguation—a persistent challenge in automated profiling—and scaling the system beyond the Computer Science domain to provide a global view of scientific progress. Currently, the system offers a powerful tool for understanding the internal dynamics and evolutionary trends of specific scientific communities.
Summary updated