An Intonational Delphi Poll on Future Trends in “Information Linguistics”

Rainer KuhlenCOLING 1984

TL;DR

This paper presents the findings of an international Delphi poll conducted in 1982-1983 to forecast the future development of "information linguistics." The study surveyed experts to gauge their opinions on the importance, feasibility, and expected realization dates of various sub-areas and topics within this interdisciplinary field, revealing a significant gap between identified problems and existing solutions.

What Problem Does This Paper Address?

The authors aimed to establish a mid-term prognosis for the emerging field of "information linguistics." This interdisciplinary area, defined as the combination of linguistic data processing, applied computer science, linguistics, artificial intelligence, and information science, was rapidly evolving. To understand its future trajectory, the researchers sought to identify which sub-areas and specific topics were considered most important and feasible by leading experts, and when they expected these to be realized. This foresight was crucial for guiding research and development efforts in a nascent and complex field.

Methods

The study employed an international Delphi poll, a structured communication technique for forecasting. This involved two rounds of questionnaires sent to experts. The first round gathered initial opinions, and the results were then summarized and fed back to the participants in the second round, allowing them to react to the emerging trends and refine their assessments. The questionnaires covered demographic data, scientific background, research and application involvement, and crucially, the evaluation of various sub-areas and single topics within information linguistics. Participants rated topics based on importance (for research and application), feasibility (for research and application), and expected date of realization (for research and application). They also assessed their own competence and the desirability of each sub-area.

Datasets

The study utilized data collected from two rounds of an international Delphi poll conducted between 1982 and 1983.

  • Round 1: 385 international experts were surveyed, with 90 questionnaires returned.
  • Round 2: 360 questionnaires were mailed out, and 56 were returned, with 48 of these from participants who had also responded in the first round. The paper refers to four sets of data:

  • Set A: 90 responses from round 1.

  • Set B: 48 responses from round 1 that also included answers in round 2.
  • Set C: 56 responses from round 2.
  • Set D: 48 responses from round 2 that also included answers in round 1. The primary focus for analysis was Set C, as it represents the most current expert opinions after feedback from the first round.

Key Insights and Ideas

  • Expertise Matters: Participants with higher self-assessed competence in a sub-area tended to rate topics within that area as more important and feasible, both for research and application. This highlights the significant role of expertise in shaping future outlooks.
  • The "Problem-Solution Gap": A consistent finding across the study was a notable gap between the perceived importance of scientific and application-oriented topics and their feasibility. This suggests that while experts identified many critical areas and challenges, the solutions and practical implementations lagged behind.
  • Research vs. Application Divergence: There was a distinct difference in how research and application perspectives viewed feasibility and realization dates. While research might see a topic as important, its practical application and realization timeline could be perceived differently, indicating a potential disconnect between theoretical exploration and practical implementation.
  • Delphi Effect Nuances: While the study acknowledges the potential for an averaging effect in Delphi polls, the increase in ratings from round 1 to round 2 suggests that a higher proportion of qualified and motivated experts participated in the second round, leading to generally higher, rather than averaged, evaluations.
  • Information Linguistics is Socially Beneficial: The study found that negative desirability ratings were rarely given, indicating that experts did not view information linguistics as a socially harmful discipline. Instead, it was generally perceived as having positive social effects.

Key Results

  • High Competence: The surveyed experts were highly qualified, with a significant portion holding Ph.D.s or professorships.
  • Extensive Experience: Nearly 60% of respondents had over 10 years of experience in information linguistics.
  • Dominant Sub-areas: "Indexing" and "Automatic Abstracting" were consistently rated as highly important and feasible, particularly from a research perspective. "Automatic Translation" and "Question Answering" also showed high importance.
  • Feasibility Challenges: While importance ratings were generally high, feasibility ratings, especially for application, were often lower, reinforcing the "problem-solution gap." For instance, automatic indexing had high importance but moderate feasibility.
  • Realization Timelines: Expected realization dates varied, with some topics anticipated in the late 1980s and others extending into the mid-1990s and beyond. There was a slight tendency for application-oriented realization dates to be earlier than research-oriented ones.
  • Short-term vs. Long-term: The study identified distinct short-term and long-term topics. For example, automatic indexing and morphological procedures were seen as short-term, while more complex areas like advanced translation systems and sophisticated question answering were considered long-term.

Limitations and Future Work

  • Ranking Limitations: The authors caution against over-interpreting average values for sub-areas and explicitly state that cross-group ranking is not allowed for methodological reasons. The focus should be on individual topics.
  • Delphi Effect Nuance: While the study observed an increase in ratings from round 1 to round 2, the authors note that this might be due to a higher proportion of qualified participants in the second round rather than a pure averaging effect.
  • Future Analysis: The authors mention that a more detailed analysis, including the interplay between research/application orientation and other factors, will be published separately. This suggests further exploration of the nuances within the data.
  • Focus on Specific Topics: The paper highlights the need to apply the evaluation parameters to single topics rather than just sub-areas for more precise insights.

Summary updated

BibTeX
@inproceedings{kuhlen1984intonational,
  author = {Rainer Kuhlen},
  title = {{An Intonational Delphi Poll on Future Trends in “Information Linguistics”}},
  booktitle = {International Conference on Computational Linguistics},
  year = {1984},
}
Figure