TL;DR
Deep neural networks (DNNs) are increasingly used in safety-critical systems, but validating their reliability requires assessing the adequacy of test sets. This paper introduces TEASMA, a practical methodology that uses regression analysis on training data to predict a test set's fault detection rate (FDR). TEASMA enables engineers to confidently determine if a test set is sufficient for validating a DNN, even before labeling test data.
What Problem Does This Paper Address?
Deploying Deep Neural Networks (DNNs), especially in safety-critical applications, hinges on ensuring they are thoroughly tested. A key challenge is determining if a given test set is "adequate" – meaning it's effective at uncovering faults and thus provides confidence in the DNN's performance. Traditional software testing methods like mutation analysis and coverage criteria have been adapted for DNNs, but a comprehensive methodology to accurately assess test adequacy, particularly for predicting fault detection capabilities, has been missing. This gap leaves engineers uncertain about trusting high-accuracy test results, potentially leading to the deployment of unreliable models.
Methods
The core of TEASMA lies in building a predictive model for a test set's Fault Detection Rate (FDR) using its Adequacy Score (AS). This process involves:
- Estimating Faults: Using a clustering approach on mispredicted inputs from the DNN's training set to identify distinct root causes of errors.
- Calculating AS and FDR for Training Subsets: Sampling various subsets from the training data, calculating their AS using a chosen adequacy metric (like Mutation Score, Surprise Coverage, or Input Distribution Coverage), and determining their actual FDR based on the estimated faults.
- Performing Regression Analysis: Building a regression model (linear, quadratic, exponential, or regression trees) to capture the relationship between AS and FDR using the data from the training subsets.
- Assessing Test Sets: For a new, unlabeled test set, calculating its AS using the same metric and configuration, and then using the trained regression model to predict its FDR and associated prediction intervals.
The paper emphasizes the use of post-training mutation operators for calculating Mutation Score (MS) due to their computational efficiency compared to pre-training operators.
Datasets
The study utilized a variety of widely-used image recognition datasets:
- MNIST
- Fashion-MNIST
- Cifar-10
- SVHN
- Cifar-100
- ImageNet
These datasets were used in conjunction with seven different DNN models, creating a total of eight experimental subjects.
Key Insights and Ideas
- Adequacy metrics are not direct surrogates for FDR: While metrics like Mutation Score (MS), Distance-based Surprise Coverage (DSC), and Input Distribution Coverage (IDC) show strong correlations with FDR, their relationships are often non-linear and vary significantly across different DNN models. This means they can't be used directly to assess test adequacy without a predictive model.
- Post-training mutation operators are effective: Despite criticisms that they might not realistically represent real faults, mutation operators applied after training (post-training MOs) proved effective in generating mutants that are highly associated with faults, enabling accurate FDR prediction.
- TEASMA provides a practical solution: By building a predictive model on the training set, TEASMA allows for efficient assessment of unlabeled test sets. This is crucial for reducing the cost and effort associated with labeling and executing large test suites.
- MS is the most accurate predictor: Among the investigated metrics, MS (specifically when calculated using DeepMutation's approach) consistently yielded the most accurate FDR predictions with the lowest RMSE.
Key Results
- High correlation: Strong correlations (Spearman's $\rho > 0.98$) were observed between AS and FDR for MS (using DeepMutationMS), DSC, LSC, and IDC on training subsets.
- Accurate FDR prediction: TEASMA, using MS, DSC, and IDC, built regression models that accurately predicted test set FDR with a minimum $R^2$ of 0.94 for MS and 0.90 for DSC and IDC.
- Lowest RMSE with MS: The average RMSE for FDR prediction using MS was 0.09, significantly lower than DSC (0.17) and IDC (0.18), indicating MS's superior accuracy.
- Model shape variation: The shape of the AS-FDR relationship varied significantly across metrics and subjects, often being non-linear, highlighting the need for empirical model selection.
Limitations and Future Work
- Model-specific models: TEASMA requires building a specific prediction model for each DNN and adequacy metric. If the DNN is updated or a new metric is used, the process needs to be reapplied.
- Training/Test set distribution assumption: The methodology implicitly assumes similar distributions between training and test sets. Significant distribution shifts that lead to poor model accuracy would render test adequacy assessment moot, as retraining would be necessary.
- Computational cost of MS: While post-training MOs are more efficient than pre-training ones, MS calculation can still be computationally intensive, especially for large models and datasets.
- Future work: The authors plan to use TEASMA to guide the automated generation of adequate test sets for DNN models. They also suggest that integrating new fault identification approaches could further enhance TEASMA's performance.
Summary updated