Trusting Testcases Using Blockchain-Based Repository Approach

Abdulla Al Zaabi, Chan Yeob Yeun, Ernesto DamianiSymmetry 2021 · DOI

TL;DR

This paper proposes a novel blockchain-based approach for a trusted testcase repository to address the lack of centralized and trustworthy testcase sources for autonomous vehicle software. By leveraging Hyperledger Fabric and IPFS, they introduce a "Proof-of-Validation" mechanism to ensure the integrity and accessibility of testcases, aiming to enhance the security and reliability of autonomous vehicle software.

What Problem Does This Paper Address?

Modern vehicles are increasingly sophisticated, incorporating connected and self-driving capabilities that introduce significant cybersecurity risks. Ensuring the reliability and security of the software underpinning these systems is paramount. While software testing and formal certifications are crucial for building user confidence, the current landscape for autonomous vehicle software testing suffers from a lack of trusted, shared testcase repositories and a common approach. This forces individual developers and organizations to perform testing in isolation, leading to duplicated efforts and potential gaps in security assurance. The paper aims to bridge this gap by creating a decentralized, trustworthy repository for testcases.

Methods

The core of the proposed solution is a blockchain-based repository built using Hyperledger Fabric. This framework is chosen for its ability to support non-incentivized consensus algorithms and its modular architecture. The system introduces a novel "Proof-of-Validation" (PoV) mechanism. In this mechanism, specialized "validating nodes" are responsible for verifying new testcases. Testcases are initially submitted to a private data collection and only become publicly accessible and added to the ledger after being validated by a predefined number of these nodes.

For storing the actual testcase files, the paper utilizes the InterPlanetary File System (IPFS). This distributed file storage system is used to store the testcase content, while the blockchain ledger stores the hash of the file. This approach ensures the integrity of the testcases; any tampering with a testcase file would result in a hash mismatch, making it inaccessible. The system also incorporates a conceptual model for autonomous vehicles to help categorize testcases and identify potential security threats.

Datasets

The paper uses the Linux Test Project (LTP) suite as a use case for their testcase repository. The LTP is described as a realistic repository of testcases used by developers for Linux kernel development and testing. The authors do not introduce a new dataset but rather leverage an existing, well-established one to demonstrate the applicability of their proposed approach.

Key Insights and Ideas

  • Decentralized Trust for Testcases: The fundamental idea is to move away from relying on a single, trusted authority for testcase repositories and instead distribute trust across a blockchain network.
  • Proof-of-Validation (PoV): This novel consensus mechanism for testcase validation is key. It ensures that testcases are rigorously vetted by multiple parties before being accepted, enhancing their trustworthiness.
  • Blockchain + IPFS Synergy: Combining blockchain for metadata and integrity verification with IPFS for efficient, distributed file storage offers a robust and scalable solution for managing large testcase repositories.
  • Autonomous Vehicle Conceptual Model: Developing a structured model for autonomous vehicles helps in organizing and categorizing testcases, which is crucial for systematic security testing in this complex domain.
  • Addressing the "Trust Gap": The paper directly tackles the issue of trust in testcases, which is a significant hurdle in ensuring the security of critical software like that in autonomous vehicles.

Key Results

The paper presents a proof-of-concept implementation and discusses its design. While specific quantitative results like accuracy improvements or performance benchmarks are not detailed, the key outcome is the successful demonstration of a functional blockchain-based testcase repository using Hyperledger Fabric and IPFS. The implementation shows how testcases can be registered, validated through the PoV mechanism, and stored securely. The paper highlights the ability to query and filter testcases based on defined parameters, contributing to a more organized and accessible testing process.

Limitations and Future Work

The authors acknowledge several limitations and suggest future directions:

  • Lack of Authentication/Authorization Middleware: The current implementation lacks robust authentication and authorization for the front-end interface, which is crucial for securing access to the system.
  • Predefined Validators: The current setup assumes a predefined set of validating nodes. Future work could explore dynamic selection or more sophisticated methods for validator assignment.
  • Focus on Testcase Repository: The current work focuses on the repository aspect. Future research aims to extend this to trusted testcase execution, including the generation of evidence for testcase executions.
  • Blockchain Oracles: The adoption of blockchain oracles is proposed to support both off-chain and on-chain generation of evidence and execution of test scripts, further enhancing the system's capabilities.
  • Autonomous Vehicle Specifics: While a conceptual model is provided, further research is needed to evaluate security testing and certification specifically within the autonomous vehicle domain.
  • Testcase Generation: The paper mentions the need for guidelines and templates for writing testcases for autonomous vehicles and plans to develop these in future work.

Summary updated

BibTeX
@article{zaabi2021trusting,
  author = {Abdulla Al Zaabi and Chan Yeob Yeun and Ernesto Damiani},
  title = {{Trusting Testcases Using Blockchain-Based Repository Approach}},
  journal = {Symmetry},
  year = {2021},
  doi = {10.3390/sym13112024},
}
Figure