Beyond emojis: an insight into the IKON language

Laura Meloni, Phimolporn Hitmeangsong, Bernhard Appelhaus, Edgar Walthert, Cesco RealeNAACL 2022

TL;DR

This paper introduces IKON, a new iconic language designed for universal, language-independent communication. It addresses the limitations of existing visual languages by focusing on semantic compositionality, high iconicity, and a flexible grammatical structure, validated through case studies and an "Iconometer" test.

What Problem Does This Paper Address?

While emojis have become ubiquitous in digital communication, they often lack the grammatical structure and semantic precision needed for complex or specialized domains. Existing visual languages, like Bliss or Zlango, have faced challenges such as being tied to national languages, being too stylized, or having overly simplistic grammars. This paper tackles the need for a robust, universal visual language that can convey nuanced meanings across different cultures and contexts, particularly for applications like medical communication where clarity is paramount.

Methods

The IKON language is built upon a set of philosophical, linguistic, and graphical principles. Linguistically, it emphasizes semantic compositionality (combining icons to create new meanings), the use of grammar categories, and consistent iconemes. Graphically, it prioritizes high iconicity (visual similarity between sign and meaning) and aims for language independence by avoiding text and using culturally neutral elements like grey for skin color. The paper details several strategies for creating icons:

  • Pictographic Icons: Direct visual representations of objects or concepts.
  • Abstract Symbols: Used when a concept is widely understood (e.g., traffic signs).
  • Contextual Icons: Visual scenes that highlight specific elements.
  • Compound Icons: Created through:
  • Juxtaposition: Placing icons side-by-side.
  • Contrastive Form: Using opposing symbols to clarify meaning.
  • Hyperonymic Icons: Representing a broad concept through related elements.
  • Hyponymic Icons: Highlighting a specific instance within a broader category.

The paper also introduces the "Iconometer" test, a software tool designed to assess the degree of polysemy (multiple meanings) of visual representations and measure their adequacy to their intended meaning.

Datasets

Not discussed in the paper. The paper focuses on the design principles and initial validation of the IKON language, rather than large-scale dataset training.

Key Insights and Ideas

  • Beyond Emojis: IKON aims to be a more structured and semantically rich "language" than current emoji sets, addressing their grammatical limitations.
  • Universality through Design: By focusing on iconicity, avoiding text, and using culturally neutral elements, IKON strives for cross-cultural understanding.
  • Semantic Compositionality is Key: The ability to combine simple icons into complex meanings is a core principle, allowing for vocabulary expansion.
  • Context Matters for Clarity: While single icons can be effective, the paper highlights that complex sentences require careful design to avoid misinterpretation, especially in specialized domains like medicine.
  • The Iconometer as a Validation Tool: This test is crucial for empirically assessing how well IKON icons are understood and for refining the language.

Key Results

The paper presents several case studies demonstrating IKON's application to concepts like modality ("must"), verbs of perception ("search," "find"), verbs of communication ("thank"), and verbs of motion ("go").

  • Modality ("must"): Initial attempts using a traffic policeman or a road sign were deemed not intuitive enough. The final design draws inspiration from hand gestures, specifically the index finger pointing, to convey obligation or necessity.
  • Verbs of Perception: Icons for "search" and "find" use a magnifying glass, a familiar symbol from user interfaces, to represent the process and outcome.
  • Verb of Communication ("thank"): A hyperonymic icon was used to represent the diverse cultural ways of expressing gratitude, merging gestures like hand-on-chest, handshake, and bowing.
  • Verb of Motion ("go"): An icon using a dashed purple arrow signifies general movement away from the speaker, with the potential for more specific icons to encode direction and type of motion.

The Iconometer test results, though preliminary due to a small sample size, indicated that single pictograms had an average understanding rate of 81%, while iconic sentences averaged 59%. This suggests that while individual icons are generally well-understood, constructing complex sentences requires further refinement.

Limitations and Future Work

  • Sample Size for Iconometer: The current population for the Iconometer test is insufficient for drawing definitive conclusions. Future work will involve analyzing results from a larger, more diverse group and revising icons that perform poorly.
  • Complexity of Sentences: The study found that semantic considerations for single icons do not always translate to complex sentences, highlighting the cognitive effort required for understanding sentence-level meaning.
  • Further Testing and Refinement: The authors plan to continue testing IKON icons and sentences, particularly in specialized domains like medical communication, to ensure their effectiveness and usability.
  • Expanding Vocabulary: The IKON lexicon is continuously growing, with new meanings being semantically analyzed and translated into visual representations.
  • Exploring Syntactic Flexibility: While currently using a linear word order, the paper acknowledges the potential for more flexible syntactic orders and even bi-dimensional syntax.

Summary updated

BibTeX
@inproceedings{meloni2022beyond,
  author = {Laura Meloni and Phimolporn Hitmeangsong and Bernhard Appelhaus and Edgar Walthert and Cesco Reale},
  title = {{Beyond emojis: an insight into the IKON language}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2022},
}

Loading related content...

Figure