A framework for FAIR robotic datasets

Corrado Motta, Simona Aracri, R. Ferretti, Marco Bibuli, Gabriele Bruzzone, Massimo Caccia, Angelo Odetti, Fausto Ferreira, Francesca De PascàlisScientific 2023 · DOI

TL;DR

This paper introduces an open-source framework designed to make marine robotic telemetry data "FAIR" (Findable, Accessible, Interoperable, and Reusable). By automating the attachment of standardized metadata to raw robotic logs, the researchers bridge the gap between the robots' behavior and the environmental data they collect, ensuring scientific results are transparent and reproducible.

What Problem Does This Paper Address?

In oceanography, robotic platforms like Autonomous Underwater Vehicles (AUVs) and gliders are generating a flood of new data. However, there is a major disconnect: while the environmental data (like salinity or temperature) often follows strict standards, the robotic data (telemetry, thruster states, and navigation logs) is usually a mess of custom formats and cryptic variable names.

Without knowing exactly how a robot was behaving—its pitch, roll, or power consumption—it is difficult for other scientists to validate the environmental findings or reuse the dataset. Manually labeling these datasets is too time-consuming for researchers in the field, leading to "dark data" that is published as raw, unsearchable files. This paper tackles the lack of a standardized, automated pipeline to make this robotic telemetry scientifically useful.

Methods

The authors developed a Free and Open Source Software (FOSS) pipeline, primarily using Python and Jupyter Notebooks, that draws on established Earth Science standards to format robotic data. Key components include:

  • Standardized Formats: The system outputs data in NetCDF, a self-describing machine-independent format, and generates ISO 19115-compliant XML files for descriptive metadata.
  • Metadata Conventions: It adopts the ACDD (Attribute Convention for Data Discovery) for global metadata and the CF (Climate and Forecast) conventions for variable-level metadata.
  • Two-Level Naming Convention: To handle the "jargon" of different robotic labs, they implemented a mapping system. A raw log name (e.g., xgps) is mapped to a standard long_name (e.g., projection_x_coordinate) using a lightweight JSON database.
  • Automation GUI: A Human-Computer Interface (HCI) allows operators to quickly fill in mandatory "global" metadata (like PI name, license, and project) with a few clicks, while the software automatically calculates spatial and temporal bounds.

Datasets

The framework was validated using real-world data from the SWAMP (Shallow Water Autonomous Multipurpose Platform), a modular catamaran-shaped autonomous surface vehicle.

  • Evaluation Data: The authors used telemetry from a field campaign where SWAMP performed standardized maneuvers (like zig-zag tests) following International Towing Tank Conference (ITTC) criteria.
  • Availability: The resulting FAIR-compliant dataset is hosted on the Zenodo repository, and the source code for the framework is available on GitHub.

Key Insights and Ideas

  • The "Robotic-Environmental" Dichotomy: The authors argue that robotic performance data is not just "metadata" for sensors—it is a primary scientific record necessary for validating in-situ deployments.
  • Vocabulary Gaps: A major hurdle is that a "controlled vocabulary" for robotics (similar to what exists for climate science) doesn't fully exist yet. The authors propose using the Fossen nomenclature as a baseline for standardizing robotic variables.
  • Automation is Mandatory: Because field missions are chaotic and high-pressure, any FAIR-compliance tool must be automated. If it requires manual entry for every variable, researchers simply won't use it.
  • Interoperability through Mapping: By mapping custom "lab-specific" names to a shared database of standard names, researchers can write scripts that analyze data from different robots without needing to manually rename columns.

Key Results

  • Successful Standardization: The framework successfully transformed raw .csv telemetry into fully annotated NetCDF files that are readable by standard Earth Science tools (like the xarray Python library).
  • Metadata Richness: The system automatically populates attributes like units, valid ranges, and instrument sources (e.g., identifying which specific GPS model produced a coordinate).
  • Workflow Integration: The tool was successfully integrated into the existing control interface of the SWAMP vehicle, proving that FAIR data generation can happen "by default" at the end of a mission.

Limitations and Future Work

The authors acknowledge that while they have proposed standard names for many robotic variables, a globally accepted, community-governed "controlled vocabulary" for marine robotics is still a work in progress.

Future work includes:

  • Expanding the Vocabulary: Continuing to build the database of robotic variables to cover more complex platforms.
  • Simulation Integration: Applying the framework to data generated in simulators (like the MARUS simulator) to ensure that virtual experiments are as FAIR as physical ones.
  • Community Adoption: Encouraging other roboticists to contribute to the open-source JSON databases to create a truly universal standard for the field.

Summary updated

Abstract

It is essential to publish and make available environmental data gathered by emerging robotic platforms to contribute to the Global Ocean Observing System (GOOS), supported by the United Nations - Decade of Ocean Science for Sustainable Development (2021-2030). The transparency of these unique observational datasets needs to be supported by the corresponding robotic records. The data describing the observational platform behaviour and its performance are necessary to validate the environmental data and repeat consistently the in-situ robotic deployment. The Free and Open Source Software (FOSS), proposed in this manuscript, describes how, using the established approach in Earth Sciences, the data characterising marine robotic missions can be formatted and shared following the FAIR (Findable, Accessible, Interoperable, Reusable) principles. The manuscript is a step-by-step guide to render marine robotic telemetry FAIR and publishable. State-of-the-art protocols for metadata and data formatting are proposed, applied and integrated automatically using Jupyter Notebooks to maximise visibility and ease of use. The method outlined here aims to be a first fundamental step towards FAIR interdisciplinary observational science.
BibTeX
@inproceedings{motta2023framework,
  author = {Corrado Motta and Simona Aracri and R. Ferretti and Marco Bibuli and Gabriele Bruzzone and Massimo Caccia and Angelo Odetti and Fausto Ferreira and Francesca De Pascàlis},
  title = {{A framework for FAIR robotic datasets}},
  booktitle = {Scientific},
  year = {2023},
  doi = {10.1038/s41597-023-02495-3},
}
Figure