Knowledge is Power: Symbolic Knowledge Distillation, Commonsense Morality, & Multimodal Script Knowledge

Yejin Choi

Abstract

Scale appears to be the winning recipe in today's AI leaderboards. And yet, extreme-scale neural models are still brittle to make errors that are often nonsensical and even counterintuitive. In this talk, I will argue for the importance of knowledge, especially commonsense knowledge, and demonstrate how smaller models developed in academia can still have an edge over larger industry-scale models, if powered with knowledge.First, I will introduce "symbolic knowledge distillation", a new framework to distill larger neural language models into smaller commonsense models, which leads to a machine-authored KB that wins, for the first time, over a human-authored KB in all criteria: scale, accuracy, and diversity.Next, I will present an experimental conceptual framework toward computational social norms and commonsense morality, so that neural language models can learn to reason that "helping a friend" is generally a good thing to do, but "helping a friend spread fake news" is not.Finally, I will discuss an approach to multimodal script knowledge demonstrating the power of complex raw data, which leads to new SOTA performances on a dozen leaderboards that require grounded, temporal, and causal commonsense reasoning.
BibTeX
@inproceedings{choi2022knowledge,
  author = {Yejin Choi},
  title = {{Knowledge is Power: Symbolic Knowledge Distillation, Commonsense Morality, \& Multimodal Script Knowledge}},
  booktitle = {WSDM},
  year = {2022},
  doi = {10.1145/3488560.3500242},
}

Loading related content...

Figure