Metadata vs. Ground-truth: A Myth behind the Evolution of Community Detection Methods

Tanmoy Chakraborty, Zhe Cui, Noseong Park

Abstract

A community detection (CD) method is usually evaluated by what extent it is able to discover the 'ground-truth' community structure of a network. A certain 'node-centric metadata' is used to define the ground-truth partition. However, nodes in real networks often have multiple metadata types (e.g., occupation, location); each can potentially form a ground-truth partition. Our experiment with 10 CD methods on 5 datasets (having multiple metadata-based ground-truth partitions) show that the metadata-based evaluation is misleading because there is no single CD method that can outperform others by detecting all types of metadata-based partitions. We further show that the community structure obtained from the CD methods is usually topologically stronger than any metadata-based partitions. Finally, we suggest a new task-based evaluation framework for CD methods and show that a certain type of CD methods is useful for a certain type of task.
BibTeX
@inproceedings{chakraborty2018metadata,
  author = {Tanmoy Chakraborty and Zhe Cui and Noseong Park},
  title = {{Metadata vs. Ground-truth: A Myth behind the Evolution of Community Detection Methods}},
  booktitle = {The Web Conference},
  year = {2018},
  doi = {10.1145/3184558.3186921},
}

Loading related content...

Figure