Natural Language and Exploration of an Information Space: The ALFresco Interactive System

Oliviero Stock

Introduction

In the early 1990s, the landscape of information retrieval was divided between two paradigms: rigid natural language interfaces and free-form hypertext systems. While natural language processing (NLP) allowed for specific queries, it often felt "narrow and constraining." Conversely, hypertext offered exploratory freedom but frequently left users "lost in hyperspace," disoriented by a lack of clear search mechanisms. ALFresco was developed as an innovative interactive system to bridge this gap, creating a "habitable" information space for exploring 14th-century Italian frescoes.

System Architecture and Methods

ALFresco integrates structured NLP with the exploratory flexibility of hypermedia. The system architecture coordinates several sophisticated components to manage this interaction, ranging from linguistic analysis to knowledge representation.

The ALFresco system architecture, illustrating the flow from linguistic input to action execution and hypermedia interaction.

The system's core components include:

  • Multimodal Input Handling: Users interact via keyboard and touchscreen. The system resolves deictic references—for instance, understanding "Who is this person?" when a user touches a figure on the screen.
  • Knowledge Representation (YAK): A hybrid system using a KL-One-like language to manage hierarchies of concepts (e.g., painters, towns) and specific instances.
  • Parsing (WEDNESDAY 2): A chart-based parser designed for the flexible word order of Italian, capable of resolving complex linguistic phenomena like ellipses.
  • User Interest Modeling: An activation network tracks user interests across dimensions such as art schools and time periods, allowing the system to tailor its natural language generation (NLG).

Hypermedia and Multimodal Integration

A key innovation of ALFresco is its synergy between NLP and hypermedia. While NLP provides a "jump" facility to prevent disorientation, hypermedia organizes unstructured information, such as art critics' opinions, which are difficult to represent in formal logic.

The integration of hypermedia networks and videodisc images within the ALFresco interface.

As illustrated in Figure 3, the system uses its NLG component to dynamically generate text containing "buttons" or hyperlinks. These links are created based on the current discourse and the user's interest model, connecting formal knowledge to a hypermedia network of canned texts and images from a VideoDisc.

Key Insights and Results

The ALFresco prototype demonstrates that the "sum is greater than the parts" when combining NLP and hypermedia. The system prioritizes "habitability"—the ease of navigating a domain—over simple query accuracy. By allowing users to "drive the game" through a mix of precise questions and serendipitous browsing, ALFresco provides a more effective platform for cultural learning.

Key results from the prototype include:

  1. Multimodal Fusion: Successful resolution of ambiguous screen touches using linguistic context.
  2. Initiative Shifts: Seamless transitions between asking questions and browsing commentary.
  3. Tailored Descriptions: Generation of art piece descriptions that highlight features relevant to the user's established interests.

Limitations and Future Directions

Despite its successes, ALFresco faced several challenges. The reliance on a hand-crafted Knowledge Base (YAK) created a knowledge bottleneck for expanding to wider domains. Additionally, the 1991 hardware requirements—including SUN workstations and analog videodisc units—limited portability. Future work suggested moving toward speech input and the creation of personalized cultural-touristic itineraries based on user interactions.

Summary updated

BibTeX
@inproceedings{stock1991natural,
  author = {Oliviero Stock},
  title = {{Natural Language and Exploration of an Information Space: The ALFresco Interactive System}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {1991},
}
Figure