Visartico: a visualization tool for articulatory data

Slim Ouni, Loïc Mangeonjean, Ingmar SteinerInterspeech 2012

TL;DR

This paper introduces VisArtico, a user-friendly, Java-based visualization tool for 3D electromagnetic articulography (EMA) data. It addresses the challenge of making complex articulatory data accessible to researchers, enabling synchronized visualization of coil movements, acoustic signals, and phonetic segments.

What Problem Does This Paper Address?

Studying speech production requires understanding both the acoustic output and the physical movements of the vocal tract. While acoustic data is relatively easy to capture, articulatory data, which tracks the precise movements of the tongue, lips, and jaw, has historically been difficult to acquire and, more importantly, to visualize and interpret. Traditional methods like X-ray imaging are now largely prohibited due to health concerns. Modern techniques like Magnetic Resonance Imaging (MRI) and Electromagnetic Articulography (EMA) offer non-invasive ways to capture this data. However, EMA data, while offering high temporal resolution, is complex to process and interpret. Existing software tools for EMA visualization are often outdated, platform-specific, require expensive licenses (like MATLAB), or have steep learning curves, making them inaccessible to many researchers, particularly phoneticians and students. This paper aims to bridge this gap by providing an efficient, easy-to-use, and freely available tool.

Methods

VisArtico is a visualization tool designed to process and display 3D EMA data acquired by the AG500 system. Its core functionality lies in its synchronized multi-view interface:

  • 3D Spatial View: Displays the real-time positions and orientations of EMA coils in a 3D space. It also allows for visualizing coil trajectories as point clouds and connecting coils with segments or spline curves.
  • Midsagittal View: Presents a 2D midsagittal slice of the vocal tract, showing the contours of the tongue, lips, and jaw. This view is crucial for understanding the shape and movement of the vocal tract during speech. The tool can also infer and display the palate contour.
  • Temporal View: This panel displays time-series data for individual coil trajectories (Cartesian and spherical coordinates), synchronized with the acoustic waveform and phonetic segmentations.

Key novelties and features include:

  • Synchronized Animation: All three views are animated simultaneously, allowing users to see how articulatory movements correspond to acoustic events.
  • Phonetic Labeling: VisArtico supports importing and displaying phonetic segmentations, enabling researchers to link articulatory gestures to specific speech sounds.
  • EMA Configuration Module: A user-friendly module allows researchers to map EMA coils to specific articulators (tongue, lips, jaw, velum) and to determine the speaker's midsagittal plane, which is essential for accurate interpretation of the data, especially when the speaker's head is not perfectly aligned with the EMA system's reference frame.
  • Palate Contour Detection: The tool can automatically detect and display the palate contour, either from dedicated palate tracing data or by approximating it from tongue coil positions.
  • Filtering: VisArtico includes options for filtering EMA data to reduce noise and correct unrealistic coil positions, with customizable cutoff frequencies and the ability to ignore intervals with high RMSE.

The software is built using Java, making it cross-platform compatible.

Datasets

The paper does not explicitly mention specific datasets used for training or evaluation. However, it states that VisArtico is designed to work with data from the AG500 3D electromagnetic articulograph (Carstens Medizinelektronik GmbH, Germany). It also mentions compatibility with data formats from TAPADM and the Edinburgh Speech Tools (EST) track files, commonly found in corpora like MOCHA-TIMIT and the mnguo database.

Key Insights and Ideas

  • Democratizing Articulatory Data: The most significant insight is the realization that complex articulatory data is largely inaccessible to many researchers due to the lack of user-friendly tools. VisArtico directly addresses this by prioritizing ease of use and broad accessibility.
  • The Power of Synchronization: The synchronized display of 3D coil movements, midsagittal views, and acoustic signals is crucial. This allows for a holistic understanding of speech production, linking physical actions to sound.
  • Importance of Midsagittal Plane Determination: Accurately determining the midsagittal plane is vital for interpreting EMA data, especially given the unconstrained nature of EMA acquisition. VisArtico's automated method for this is a key feature.
  • Inferring Missing Data: The ability to approximate palate contours from tongue coil data is a clever workaround for situations where dedicated palate tracing data is unavailable, enhancing the tool's utility.

Key Results

The paper focuses on the functionality and design of VisArtico rather than presenting quantitative performance metrics. The key "results" are the successful implementation and demonstration of the tool's capabilities:

  • VisArtico successfully visualizes synchronized 3D EMA data, acoustic signals, and phonetic segmentations.
  • It provides intuitive tools for configuring EMA data, including mapping coils to articulators and determining the midsagittal plane.
  • Advanced features like palate contour detection and data filtering are implemented and functional.
  • The software is designed to be lightweight, cross-platform (Java), and freely available, significantly lowering the barrier to entry for researchers interested in articulatory data.

Limitations and Future Work

Limitations:

  • The current version primarily supports data formats produced by the Carstens CalcPos software.
  • The accuracy of the midsagittal view, particularly for lip shape, depends on the number of coils used.
  • The paper acknowledges that while the jaw position can be approximated from a lower incisor coil, it's difficult to accurately predict without complementary information.

Future Work:

  • Expanded Data Format Support: Plans are in place to support other common EMA data formats.
  • Enhanced Filtering and Analysis: More sophisticated filtering options and advanced data analysis features, such as calculating coil velocity and acceleration, are planned.
  • Acoustic Integration: Adding spectrogram display and formant tracking alongside the acoustic waveform.
  • 3D Tongue Coil Layouts: Extending the configuration module to handle more complex 3D tongue coil arrangements.
  • Localization: Adding support for more languages beyond English and French for the GUI.
  • Adaptability: The software's design is intended to be adaptable for use with other EMA articulograph models (e.g., AG500, AG501, Wave system).

Summary updated

BibTeX
@inproceedings{ouni2012visartico,
  author = {Slim Ouni and Loïc Mangeonjean and Ingmar Steiner},
  title = {{Visartico: a visualization tool for articulatory data}},
  booktitle = {Interspeech},
  year = {2012},
}
Figure