Introduction
In human communication, we rarely state the obvious. When we hear about a "senator from the US," we instinctively form expectations about their likely education, age, and primary language. This cognitive "shortcut" allows humans to process information efficiently even when it is underspecified. Current AI systems, however, struggle with this type of reasoning. Traditional Knowledge Base Completion (KBC) focuses on finding a single "true" missing fact (e.g., an exact birth date), which is often impossible for "long-tail" entities—people or concepts not prominent enough for comprehensive databases like Wikipedia.
This paper introduces profiling, a novel AI task inspired by social psychology. Instead of seeking a single correct fact, a Profiling Machine learns to generate probability distributions of likely values (stereotypes) for unspecified facets, providing a crucial "prior" for Natural Language Understanding (NLU).
Methodology
The researchers evaluated two primary neural architectures designed for profiling:
- Autoencoder (AE): A neural network that uses a "dropout" method on input facets (e.g., nationality, occupation) to force the model to reconstruct missing information from partial data.
- Embedding-based Neural Predictor (EMB): This model leverages pre-trained entity embeddings (from Word2Vec or Freebase) to predict the distribution of 14 target facets.
These models were compared against Most Frequent Value (MFV) and Naive Bayes (NB) baselines using data from Wikidata (covering 3.2 million People, Politicians, and Actors) and a benchmark of human-intuitive expectations collected via crowdsourcing.
The Dynamics of Profiling: Entropy and Information Gain
A key finding is that profiling accuracy is heavily influenced by the "entropy" or unpredictability of a facet. Low-entropy facets like "sex or gender" are easily guessed with minimal information. Conversely, high-entropy facets like "place of birth" require complex inter-facet reasoning.
Furthermore, the model's behavior changes as more information is provided. For some attributes, more data leads to a clear increase in accuracy. For others, as the "denotation group" narrows toward a specific individual, the task can actually become more difficult.
Comparative Performance
Experimental results demonstrate that neural models significantly outperform traditional baselines. The EMB model, in particular, showed that pre-trained embeddings already contain a wealth of stereotypical information that can be effectively extracted.
Beyond raw accuracy, the Autoencoder's predictions aligned closely with human "crowd wisdom." The divergence between the AE and human judgments was lowest on attributes where humans also shared a strong consensus, suggesting the machine is successfully capturing human-like priors.
Limitations and Ethical Considerations
While promising, Profiling Machines face several hurdles:
- Data Representation: Current models struggle with continuous values (like raw dates) and multi-valued facets (like dual citizenship).
- Interpretability: The specific features neural models use to form these "stereotypes" remain a black box.
- Bias: The authors warn that while stereotyping is useful for reasoning, it carries significant risks of propagating gender or racial biases present in training data. Addressing these biases is critical before deployment in sensitive applications.
Summary updated