LinTO : Assistant vocal open-source respectueux des données personnelles pour les réunions d'entreprise

Jean-Pierre Lorré, Isabelle Ferrané, Francisco Madrigal, Michalis Vazirgiannis, Christophe Bourguignat

Abstract

This paper presents LinTO, an open-source, privacy-centric voice assistant designed specifically for professional environments. By processing data locally and integrating with enterprise productivity suites, LinTO provides a transparent, GDPR-compliant alternative to mainstream commercial solutions. The system addresses the unique challenges of multi-party professional meetings through a combination of sophisticated speech processing, computer vision for participant tracking, and automated summarization. Experimental results demonstrate high accuracy in intent recognition (97%) and competitive transcription quality (Word Error Rate < 14%), establishing LinTO as a viable "sovereign" solution for corporate digital assistants.

Introduction

While consumer voice assistants like Alexa or Siri have become ubiquitous in domestic settings, their adoption in corporate environments remains limited due to significant privacy and technical hurdles. Companies often face a "cornelian choice": sacrifice data sovereignty by handing sensitive internal information to tech giants (GAFAM) or forego the productivity benefits of voice-enabled automation. Furthermore, professional meetings represent complex multi-party environments that differ fundamentally from the one-on-one interactions consumer devices are optimized for.

The LinTO physical device interface designed for professional meeting rooms.

LinTO was developed to bridge this gap, focusing on a sovereign architecture that keeps data within the company's infrastructure. It facilitates meeting management, participant tracking, and automated minute-taking while ensuring full compliance with data protection regulations.

System Architecture and Methods

The LinTO system integrates a custom hardware stack—comprising a Raspberry Pi, a 360° camera, and a microphone array—with a sophisticated software pipeline designed for edge efficiency.

Software Pipeline

The software architecture is modular, moving from raw audio input to actionable commands through several specialized stages:

The LinTO software processing pipeline, illustrating the flow from hotword detection to final action.
  • Speech Processing: The core Automatic Speech Recognition (ASR) engine, LinSTT, is based on the open-source Kaldi toolkit. To trigger actions, the system employs a "hotword" detector utilizing Gated Recurrent Units (GRU). GRUs were selected over LSTMs for their computational efficiency on low-power hardware like the Raspberry Pi while maintaining comparable performance on small datasets.
  • Natural Language Understanding (NLU): Intent detection (e.g., "LinTO, schedule a follow-up") is handled by the open-source TOCK framework, which leverages libraries such as Stanford CoreNLP to parse user commands.
  • Computer Vision: To solve the "who is where" problem in a crowded room, LinTO utilizes a Ricoh Theta 360° camera. Person detection is performed via SSD Mobilenet, supplemented by a spatio-temporal filtering algorithm and the DELF (Deep Local Feature) algorithm. This allows the system to maintain participant tracking even when individuals are momentarily occluded.
  • Summarization: Automated minute-taking is enabled by a neural utterance encoder based on Recurrent Neural Networks (RNNs) with attention mechanisms, which groups dialogue into coherent thematic blocks.

Datasets and Evaluation

The development of LinTO relied on several key benchmarks to ensure professional-grade performance:

  • ESTER Corpus: Used to evaluate ASR performance specifically for the French language.
  • AMI Meeting Corpus: A standard dataset for multi-party meeting analysis, utilized to train summarization and utterance encoding models.
  • Gesture Dataset: A collection of 1 million hand images was used to experiment with non-verbal gesture-based controls.
  • Custom Localization: The project utilized crowdsourcing to translate the AMI corpus into French, improving the system's localized linguistic models.

Key Results

LinTO demonstrates robust performance across its primary functional domains:

  • Intent Recognition: The system achieved a 97% accuracy rate in "command mode," ensuring reliable execution of user requests.
  • Transcription Quality: The ASR engine reached a Word Error Rate (WER) of less than 14% on the ESTER corpus using a DNN-HMM model.
  • Tracking Robustness: The spatio-temporal filtering algorithm successfully maintained participant tracking even during 6-second disappearances from the camera's field of view.
  • Summarization: Evaluations indicated that the neural utterance encoder produced dialogue groupings that often outperformed current state-of-the-art systems in coherence.

Discussion and Future Work

LinTO advocates for a "Third Way" in AI development, suggesting that open-source collaboration is the only viable path for smaller actors to challenge dominant tech monopolies. By performing processing locally and integrating with internal systems (like OpenPaaS), LinTO eliminates the need to transmit voice data to external clouds, providing inherent protection against mass surveillance.

Despite these successes, challenges remain. Environmental factors such as variable room lighting and extreme head rotations can still impact visual tracking accuracy. Future research will focus on "multi-party interaction analysis," aiming to determine not just what is being said, but who is speaking to whom and identifying specific "dialogue acts" (e.g., questions, agreements, or objections) to provide even deeper insights into meeting dynamics.

Summary updated

BibTeX
@inproceedings{lorre2019linto,
  author = {Jean-Pierre Lorré and Isabelle Ferrané and Francisco Madrigal and Michalis Vazirgiannis and Christophe Bourguignat},
  title = {{LinTO : Assistant vocal open-source respectueux des données personnelles pour les réunions d'entreprise}},
  booktitle = {HAL},
  year = {2019},
}
Figure