ITRIX - an AI Enabled Solution for Orchestration of Recovery Instructions

Wanita Sherchan, Garfield Vaughn, Shaila Pervin, Bryan BaroneAAAI 2021 · DOI

Introduction

In the event of a major IT system failure, organizations rely on Disaster Recovery (DR) plans to restore services. These plans are typically documented in massive, unstructured "Technical Recovery Procedure" (TRP) files, such as PDFs or spreadsheets. Translating these human-readable instructions into machine-executable "runbooks" is traditionally a grueling manual process. Subject Matter Experts (SMEs) must parse hundreds of pages to identify specific steps and manually input them into orchestration tools—a process that is slow, expensive, and prone to error.

ITRIX (IT Recovery Instructions eXtraction) is an AI-powered pipeline designed to bridge this gap. By combining Deep Learning with rule-based safety checks, ITRIX automates the creation of executable recovery plans, reducing the time required for runbook generation by 80% while maintaining high accuracy.

System Architecture and Methodology

The ITRIX solution utilizes a multi-stage pipeline to transform raw, unstructured text into a structured, executable recovery plan. The orchestration flow is managed through a series of specialized models that handle everything from initial parsing to final code mapping.

The ITRIX Orchestrator pipeline, showing the flow from TRP document to executable recovery plan.

The pipeline consists of five primary stages:

  1. Document Parsing: Extracts text from various formats (Word, PDF, etc.).
  2. Action Classifier Model: A Long Short-Term Memory (LSTM) network identifies sentences representing actual recovery instructions versus general informational text.
  3. Critical Actions Model: A rule-based layer flags high-risk actions (e.g., database shutdowns) and automatically inserts "system-added" pre- and post-condition checks to ensure a graceful recovery.
  4. Scenario Classifier Model: Groups instructions into specific scenarios like "failover," "failback," or "testing."
  5. Recovery Actions Library (RAL) Classifier: Maps extracted text instructions to executable code snippets.

Human-in-the-Loop and Safety Augmentation

A core philosophy of ITRIX is that it does not replace the SME but empowers them. The system features a web-based UI that allows experts to review AI-extracted actions alongside the original source document, providing a collaborative workflow between human expertise and machine speed.

The ITRIX user interface, highlighting extracted actions from the source file and system-added safety checks.

One of the most significant innovations in ITRIX is the "Critical Action" logic. Human-written documentation often omits "obvious" safety checks. ITRIX automatically augments the plan by inserting these checks (e.g., verifying a service is stopped before proceeding). Furthermore, the NLP models can identify "missing" scenarios; for instance, if a document describes a "failover" but lacks a "failback" procedure, the system alerts the user to the gap, ensuring the DR strategy is comprehensive.

Performance and Results

The system was evaluated using internal test data and a one-month pilot study involving five SMEs. Key results include:

  • High Accuracy: The LSTM-based action classifier achieved an F1 score of 0.93, demonstrating high precision and recall in identifying critical instructions.
  • Efficiency Gains: The pilot study showed an 80% reduction in time taken to generate recovery runbooks compared to manual methods.
  • Scalability: By automating the "re-keying" of instructions, ITRIX enables service providers to scale Disaster Recovery as a Service (DRaaS) offerings more effectively in a multi-billion dollar market.

Conclusion and Future Work

ITRIX demonstrates the power of combining deep learning with domain-specific rules to solve a critical bottleneck in IT operations. While highly effective, the system's performance remains dependent on the clarity of the input TRP documents. Future work will focus on refining the mapping of natural language instructions to standardized code libraries and testing the system across a broader variety of IT environments and document styles.

Summary updated

BibTeX
@inproceedings{sherchan2021itrix,
  author = {Wanita Sherchan and Garfield Vaughn and Shaila Pervin and Bryan Barone},
  title = {{ITRIX - an AI Enabled Solution for Orchestration of Recovery Instructions}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2021},
  doi = {10.1609/aaai.v35i18.18026},
}
Figure