TL;DR
This paper presents a novel, statistically rigorous methodology for designing quarantine surveillance systems, specifically applied to detect non-indigenous invertebrate species on Barrow Island. It addresses the critical need for objective, evidence-based biosecurity planning by integrating expert knowledge, risk mapping, and statistical power calculations to create an effective and adaptable surveillance program.
What Problem Does This Paper Address?
The authors tackle the challenge of designing effective quarantine surveillance programs, particularly for complex environments like Barrow Island, which is undergoing significant industrial development (the Gorgon gas project). Regulatory requirements mandated a high statistical power (0.8) for detecting non-indigenous species (NIS) across invertebrates, vertebrates, and plants. However, existing methods for designing such surveillance systems were either insufficient, lacked statistical rigor, or were not tailored to the specific challenges of a large, heterogeneous area with numerous potential threats. The core problem was the absence of a systematic, objective framework to balance the need for high detection power with practical constraints like budget, logistics, and the vast number of potential NIS. Traditional approaches often prioritized budget over statistical assurance, leading to potentially ineffective surveillance.
Methods
The paper introduces a new methodology for designing surveillance systems (SS) based on statistical power. Key elements include:
- Exemplar Species Approach: Instead of trying to design for every potential NIS, the authors selected a set of "exemplar" species. These species were chosen for their diverse ecological traits and the availability of well-established detection methods (Surveillance System Components - SSCs). The integrated SS was then designed to cover the broad range of invertebrates by combining the SSCs used for these exemplars.
- Statistical Power as a Design Standard: The core of the design is achieving a specified statistical power (0.8), which represents the probability of detecting an NIS if it is present at a certain threshold population size. This moves beyond simple confidence intervals to a more direct measure of detection capability.
- Risk Mapping and Stratified Design: A multi-stage risk assessment was employed. This involved:
- Decision Trees: Used to model dispersal pathways and conditional probabilities of detection.
- Analytical Hierarchy Process (AHP): Expert opinion was elicited through pairwise comparisons to build risk maps, categorizing areas into three zones (Z1: entry points, Z2: buffer zones, Z3: other areas) based on entry and establishment probabilities.
- Spatial Allocation: The SS was then spatially stratified, with a greater allocation of resources to higher-risk zones (Z1 and Z2).
- Integration of Multiple Data Sources (SSCs): The methodology explicitly addresses how to combine data from various detection methods (e.g., pitfall traps, visual surveys, worker observations). This is crucial for maximizing detection probability and utilizing diverse information streams.
- Expert Elicitation: A significant portion of the design process relied on structured workshops with subject matter experts to gather data, refine models, and make critical decisions about SSC selection, parameterization, and allocation.
Datasets
The primary "dataset" used in this study was derived from expert opinion and existing ecological knowledge of Barrow Island and its potential NIS. This was structured through:
- Expert Workshops: Five workshops over 18 months involving invertebrate ecologists and biosecurity experts.
- Geographical Information System (GIS): Used to map risk zones and guide spatial deployment of surveillance efforts.
- Baseline Surveillance Data: Information from previous baseline surveys on Barrow Island (e.g., Callan et al. 2011) informed the understanding of the existing fauna and potential for NIS establishment.
No traditional "training" or "evaluation" datasets in the machine learning sense were used. Instead, the design was built upon expert knowledge and a priori statistical requirements.
Key Insights and Ideas
- Power-Based Design is Essential for Biosecurity: Moving beyond simple confidence intervals to statistical power provides a more robust and defensible measure of surveillance effectiveness, directly addressing the "what if it's there?" question.
- Exemplar Species Strategy is Scalable: Designing for a diverse set of exemplar species, rather than every single potential NIS, offers a practical and efficient way to create a comprehensive surveillance system for a broad range of targets.
- Risk Mapping is Crucial for Resource Allocation: Stratifying surveillance efforts based on spatially explicit risk assessments (entry and establishment probabilities) allows for more efficient and effective deployment of resources, focusing on high-probability areas.
- Expert Knowledge is Indispensable but Needs Structure: While expert opinion is vital for filling information gaps, the paper demonstrates a structured, systematic approach (e.g., using AHP and decision trees) to elicit and integrate this knowledge into a statistically sound design.
- Integration of Diverse Data Sources is Complex but Achievable: The methodology provides a framework for combining data from various SSCs, acknowledging the challenges but offering a path to a more comprehensive SS.
Key Results
- The developed methodology successfully enabled the design of a quarantine surveillance system for non-indigenous invertebrate species on Barrow Island that met the regulatory requirement of at least 0.8 statistical power.
- The integrated SS was designed to detect a wide range of invertebrate taxa by combining SSCs from five exemplar species.
- Risk mapping identified distinct zones (Z1, Z2, Z3) on Barrow Island, guiding the spatial allocation of surveillance effort, with a focus on Z1 and Z2.
- The implementation of the SS during truncated surveys in 2010-11 demonstrated its practicality and cost-effectiveness, with actual surveillance effort often exceeding design requirements.
- Post-integration power calculations for exemplar species were high (e.g., 1.00 for P. megacephala, T. pisana, P. humilis; 0.87 for C. formosanus; 0.92 for C. claratris).
- No actual NIS were detected during the first year of surveillance, though this was attributed in part to drought conditions affecting invertebrate detectability.
Limitations and Future Work
- Data Uncertainty: The design relies heavily on expert estimates for parameters like detectability (σ) and tolerable population size (K). While sensitivity analyses showed robustness, future work should focus on testing these assumptions with actual data.
- First-Year Data Limitations: The technical performance of the SS could not be fully evaluated after only one year, especially given the impact of drought on invertebrate activity and baseline re-detection rates. More data over time is needed.
- Cost Optimization: While the Gorgon project had high value, the paper notes that surveillance costs should be balanced against exclusion activities. Further refinement of cost-effectiveness for specific SSCs is implied.
- Adaptive Design: The authors suggest that principles of adaptive design could be used to update the SS based on emerging data from successive years of surveillance.
- Web-Based Tool: The development of a web-based tool to facilitate retrospective analysis of SSs is mentioned as a future development requiring investment.
- Termite SS: The termite (C. formosanus) was eventually removed from the integrated SS due to its unique SSCs and area requirements, suggesting that highly specialized threats might still require discrete, tailored surveillance.
Summary updated