TL;DR
This paper addresses the critical issue of uncertainty in toxicity testing by proposing and comparing methods to define "borderline ranges" (BRs). These BRs account for the inherent variability in test results, leading to more transparent hazard assessments and a better understanding of the precision of different testing methods.
What Problem Does This Paper Address?
In toxicology, continuous data from tests (whether animal or non-animal) are often converted into binary "positive" or "negative" classifications using a specific threshold (classification threshold, CT). However, experimental results are never perfectly precise due to biological and technical variability. When a test result falls close to the CT, it can be ambiguous, leading to different classifications upon retesting. This uncertainty is often overlooked. The paper aims to quantify this uncertainty by defining "borderline ranges" (BRs) around the CT, representing the zone where results are equally likely to be positive or negative. This is crucial for accurate hazard assessment and for evaluating the reliability of new testing methods, especially as the field moves towards non-animal alternatives.
Methods
The authors explore and compare several statistical methods for quantifying these borderline ranges:
- Pooled Standard Deviation (pSD): A traditional approach using a weighted average of standard deviations from different datasets. However, this method is sensitive to outliers and non-normal data.
- Pooled Median Absolute Deviation (MAD): A more robust alternative to pSD, using the median absolute deviation, which is less affected by extreme values.
- Bootstrap Percentile Method: A non-parametric approach that uses resampling with replacement to estimate confidence intervals for the MAD, providing an approximate 95% confidence interval for the BR.
- 90% Percentile of All MADs: This method aims to capture the variability of individual MADs rather than just the average, potentially leading to wider, more conservative BRs.
They also applied these methods to both log-transformed and non-transformed data to assess the impact of data transformation.
Datasets
The study utilized experimental data from nine different toxicity testing methods. These included:
- Non-animal tests:
- Direct Peptide Reactivity Assay (DPRA) for skin sensitization.
- LuSens assay for skin sensitization.
- Human Cell Line Activation Test (h-CLAT) for skin sensitization.
- EpiDerm Skin Corrosion test (SCT) for skin corrosion.
- EpiDerm Skin Irritation test (SIT) for skin irritation.
- Bovine Corneal Opacity and Permeability test (BCOP) for eye irritation.
- EpiOcular Eye Irritation test (EIT) for eye irritation.
- Animal test:
- Local Lymph Node Assay (LLNA) for skin sensitization.
The data were generated from routine testing in GLP and ISO 17020 certified laboratories. For one method (DPRA), experimental validation of the derived BR was also performed using a specific test substance (EGDMA).
Key Insights and Ideas
- Precision is Paramount: The size of the borderline range is directly determined by the precision of the testing method. Less precise methods result in wider BRs, indicating greater uncertainty.
- No Single "Perfect" Method: The study highlights that there isn't one universally superior method for calculating BRs. Different statistical approaches yield varying results, suggesting the need for consensus.
- BRs as a Pragmatic Tool: Borderline ranges offer a practical way to acknowledge and quantify the inherent uncertainty in toxicity testing, moving beyond simple binary classifications.
- Log Transformation's Impact: Log-transforming the data generally decreased the size of the BR, suggesting it can help in some cases to normalize data and reduce variability.
- Experimental Validation is Key: Comparing statistically derived BRs with experimentally determined ones (as done for DPRA) is crucial for validating the methods and understanding real-world uncertainty.
Key Results
- The study quantified BRs for various endpoints, including skin corrosion, irritation, sensitization, and eye irritation, using both non-animal and one animal test (LLNA).
- Different statistical methods produced varying BR sizes. Generally, the pooled MAD and bootstrapped MAD methods tended to yield smaller BRs compared to the 90% percentile method, which produced wider intervals.
- The experimental validation for the DPRA showed that the experimentally derived BR was smaller than those calculated by most statistical methods, suggesting that some statistical methods might overestimate the BR. However, one method (log pooled MAD) provided a close approximation in this specific case.
- The paper advocates for incorporating BRs into test guidelines, leading to three possible outcomes: "positive," "negative," or "inconclusive/ambiguous" within the BR.
Limitations and Future Work
- Limited Data Sets: The study primarily used data from a single laboratory, which might limit the generalizability of findings related to inter-laboratory variability.
- No "Perfect" Method: As stated, the lack of a single, universally agreed-upon method for calculating BRs is a limitation.
- Need for Consensus: The authors strongly emphasize the need for a consensus on how to define and report BRs to ensure consistency and facilitate regulatory acceptance.
- Future Work: The authors suggest that BRs should be statistically defined within test guidelines and determined through multi-laboratory ring trials before guideline adoption. They also propose that BRs should be considered when evaluating the concordance and predictive accuracy of testing methods, especially in the context of "defined approaches" that combine multiple non-animal tests. Further work is needed to explore the implications of BRs for regulatory decision-making and to establish standardized approaches.
Summary updated