Using Alexa for Flashcard-Based Learning

Lucy Skidmore, Roger K. MooreInterspeech 2019

TL;DR

This paper explores the development of an Alexa skill for flashcard-based language learning, specifically addressing the challenge of implementing "spaced repetition" within a voice assistant's conversational framework. The authors demonstrate that existing Alexa features like persistence can support this, while also identifying crucial limitations, such as the lack of multilingual speech recognition, that hinder more advanced language practice.

What Problem Does This Paper Address?

Voice assistants (VAs) like Amazon Alexa are increasingly being explored for educational applications. One popular method for language learning is flashcard-based learning, which relies on techniques like "spaced repetition" to optimize vocabulary memorization. However, implementing sophisticated features like spaced repetition within Alexa skills is not straightforward. Traditional flashcard systems involve structured, often lengthy, study sessions, which contrast with the typical short, task-oriented interactions users have with VAs. Furthermore, current Alexa development tools have limitations, particularly in supporting multilingual speech recognition, which is essential for practicing speaking skills. This paper investigates how to effectively integrate flashcard-based learning, including spaced repetition, into an Alexa skill, and identifies areas where Alexa's capabilities need to improve to become a truly sophisticated language learning tool.

Methods

The core of this paper is the design and implementation of an Alexa skill called "Japanese Flashcards." To address the spaced repetition requirement, the authors adapted the traditional Leitner System. Instead of daily or multi-day intervals between reviews, they implemented a heuristic approach within a single session, testing cards from four decks at exponentially decreasing intervals (8, 4, 2, and 1 times). This "microlearning" approach aims to balance short-term engagement with long-term retention.

Key technical components include:

  • Alexa Skill Blueprints: Used for initial development, though the authors note limitations for advanced features.
  • Alexa Skills Kit (ASK) SDK for Node.js: Used for writing intent handlers.
  • AWS Lambda: For backend logic.
  • Amazon S3: Utilized for skill persistence, storing user data (like Leitner deck progress) to enable personalized learning across sessions.
  • Amazon Polly: Used for Text-to-Speech (TTS) to generate Japanese vocabulary. A custom switchLang function was developed to handle Japanese voice output, including volume adjustments.
  • Interaction Model: Designed with flexible "sample utterances" to accommodate natural user speech variations (e.g., "erm," "is it").

Datasets

The paper mentions using 100 Japanese words from a freely available resource provided by Tofugu, a Japanese educational platform. This vocabulary was categorized into ten topics, including greetings, basic verbs, and numbers. No new datasets were introduced or benchmarked.

Key Insights and Ideas

  • Skill Persistence is Key: Alexa's ability to maintain user data across sessions (via Amazon S3) is crucial for personalized learning, allowing the skill to remember a user's progress and adapt future practice.
  • Microlearning for VAs: The traditional Leitner System needed adaptation for VAs. A heuristic approach that breaks down spaced repetition into shorter, more frequent "microlearning" sessions is more suitable for the typical user interaction patterns with voice assistants.
  • TTS for Scalability: Using Amazon Polly for TTS is more efficient than pre-recorded audio, saving time and space, especially for large vocabularies. However, the need for manual volume normalization and language switching highlights areas for improvement in the ASK SDK.
  • Flexible Interaction Models Matter: Allowing for variations in user input (like filler words) in the interaction model significantly improves the user experience and reduces frustration, making the skill more forgiving.
  • Multilingual ASR is a Major Hurdle: The absence of built-in multilingual Automatic Speech Recognition (ASR) in Alexa is a significant barrier to effective L2 speaking practice, limiting current skills to L2-L1 translation rather than actual production.

Key Results

The investigation successfully demonstrated that:

  • Alexa's skill persistence can effectively implement personalized spaced repetition.
  • An adapted Leitner System can facilitate both short-term "microlearning" and long-term retention within a VA context.
  • The flexibility of Alexa's interaction model can accommodate natural user speech variations, enhancing the learning experience.

The paper highlights that the "Japanese Flashcards" skill can facilitate simple L2-L1 flashcard-based language learning, showcasing Alexa's potential for sophisticated language learning tools.

Limitations and Future Work

The authors acknowledge several limitations:

  • Empirical Validation: The study focused on the development process, and further empirical testing is needed to validate the skill's effectiveness as a language learning tool. This could involve comparing it to visual or audio-visual learning methods and gathering user attitude data.
  • TTS Effectiveness: The effectiveness of using TTS technology in language learning contexts warrants further investigation.
  • Multilingual ASR: The lack of multilingual ASR is a significant limitation, preventing practice of L2 production.
  • Longitudinal Study: A longitudinal study is recommended to quantitatively assess learners' language improvement and to test various spaced-repetition configurations.

Future work suggested includes:

  • Integrating multilingual ASR into the Alexa development environment.
  • Making raw speech data available for developers to use external ASR tools.
  • Developing built-in functions for language switching in the ASK SDK for easier TTS integration.
  • Exploring how learners interact with and adapt to ASR-enabled systems.
  • Conducting comparative studies with other VAs like Google Assistant and Siri.
  • Expanding beyond flashcards to more advanced learning paradigms like conversational role-play and pronunciation training.

Summary updated

BibTeX
@inproceedings{skidmore2019using,
  author = {Lucy Skidmore and Roger K. Moore},
  title = {{Using Alexa for Flashcard-Based Learning}},
  booktitle = {Interspeech},
  year = {2019},
}

Loading related content...

Figure