ALFA: A dataset for UAV fault and anomaly detection

Azarakhsh Keipour, Mohammadreza Mousaei, Sebastian Scherer

Abstract

We present a dataset of several fault types in control surfaces of a fixed-wing unmanned aerial vehicle (UAV) for use in fault detection and isolation (FDI) and anomaly detection (AD) research. Currently, the dataset includes processed data for 47 autonomous flights with 23 sudden full engine failure scenarios and 24 scenarios for 7 other types of sudden control surface (actuator) faults, with a total of 66 minutes of flight under normal conditions and 13 minutes of post-fault flight time. It additionally includes many hours of raw data of fully autonomous, autopilot-assisted and manual flights with tens of fault scenarios. The ground truth of the time and type of faults is provided in each scenario to enable evaluation of the methods using the dataset. The dataset and the provided tools can be accessed from https://doi.org/10.1184/R1/12707963.

Introduction

As Autonomous Aerial Vehicles (AAVs) become more integrated into our airspace, ensuring their safety is critical. If a motor fails or a rudder gets stuck mid-flight, the onboard system must detect the issue immediately to trigger emergency protocols. However, developing these detection systems is a "chicken and egg" problem: building a robust detector requires failure data, but intentionally crashing expensive hardware to collect that data is risky and costly. Consequently, most research relies on simulations, which often fail to capture the messy, high-frequency noise and complex aerodynamics of a real aircraft struggling after a mechanical failure. The Air Lab Fault and Anomaly (ALFA) dataset addresses this gap by providing a comprehensive, ground-truthed dataset of real-world flight failures.

Experimental Setup

The researchers engineered a system to "inject" faults safely using a modified Carbon-Z T-28 fixed-wing UAV. They customized the ArduPlane firmware to include parameters that could be triggered via a Ground Control Station (GCS) to disable the engine or fix control surfaces (ailerons, elevators, or rudders) in specific positions, simulating "stuck" or "hardover" failures.

The modified Carbon-Z T-28 UAV equipped with an Nvidia Jetson TX2 and various sensors for fault injection and data logging.

Data was captured using an onboard Nvidia Jetson TX2 running the Robot Operating System (ROS), which logged high-frequency telemetry from the Pixhawk autopilot. A human safety pilot remained ready to take manual control to prevent aircraft loss.

Dataset Description

The ALFA Dataset consists of 47 autonomous flights provided in multiple formats, including processed CSVs, MATLAB .mat files, and original ROS bags.

The hierarchical organization of the ALFA dataset, including processed, raw, telemetry, and dataflash logs.

The dataset includes:

  • 8 Fault Types: Engine failure, rudder stuck (left/right/zero), elevator stuck (zero), and various aileron failure configurations.
  • Flight Time: Approximately 66 minutes of normal flight and 13 minutes of post-fault flight.
  • Ground Truth: Precise timestamps for when each fault was injected.
  • Raw Data: Hours of additional raw telemetry from manual and autopilot-assisted flights.

Key Insights and Metrics

A primary value of the dataset is the high-frequency (20-25 Hz) data comparing autopilot commands versus sensor measurements. This discrepancy is often the "smoking gun" for anomaly detection. Furthermore, the authors propose Maximum Detection Time as a critical metric. In aviation, a delay of even one second in detecting a failure can be the difference between a controlled emergency landing and a catastrophic crash. To facilitate research, helper tools in C++, Python, and MATLAB are provided to load data and evaluate algorithms against the ALFA benchmarks.

Results and Discussion

The processed portion of the dataset is heavily weighted toward engine failures (23 cases), which are critical for training machine learning models. It also provides diverse "edge cases" for actuator failures, including 8 aileron failure cases and 6 rudder/elevator cases. By providing the exact time of failure (down to 0.2s resolution), the dataset allows for rigorous testing of detection latency.

Limitations and Future Work

While ALFA is a significant step forward, it currently only covers fixed-wing aircraft; multicopters have very different failure dynamics. Additionally, the study focuses on "sudden" failures rather than "gradual" faults (e.g., a motor slowly losing efficiency), which are harder to detect. Future work aims to expand the dataset to different airframes and fault types, inviting the broader research community to contribute their own flight data to create a universal benchmark for UAV safety.

Summary updated

Abstract

We present a dataset of several fault types in control surfaces of a fixed-wing unmanned aerial vehicle (UAV) for use in fault detection and isolation (FDI) and anomaly detection (AD) research. Currently, the dataset includes processed data for 47 autonomous flights with 23 sudden full engine failure scenarios and 24 scenarios for 7 other types of sudden control surface (actuator) faults, with a total of 66 minutes of flight under normal conditions and 13 minutes of post-fault flight time. It additionally includes many hours of raw data of fully autonomous, autopilot-assisted and manual flights with tens of fault scenarios. The ground truth of the time and type of faults is provided in each scenario to enable evaluation of the methods using the dataset. We have also provided the helper tools in several programming languages to load and work with the data and to help the evaluation of a detection method using the dataset. A set of metrics is proposed to help to compare different methods using the dataset. Most of the current fault detection methods are evaluated in simulation and, as far as we know, this dataset is the only one providing the real flight data with faults in such capacity. We hope it will help advance the state of the art in AD or FDI research for autonomous aerial vehicles and mobile robots to enhance the safety of autonomous and remote flight operations further. The dataset and the provided tools can be accessed from https://doi.org/10.1184/R1/12707963 .
BibTeX
@article{keipour2020alfa,
  author = {Azarakhsh Keipour and Mohammadreza Mousaei and Sebastian Scherer},
  title = {{ALFA: A dataset for UAV fault and anomaly detection}},
  journal = {The International Journal of Robotics Research},
  year = {2020},
  doi = {10.1177/0278364920966642},
}
Figure