Abstract
Background: The physical integrity of insecticidal-treated nets (ITNs) is important for creating a barrier against host-seeking mosquitoes, and therefore influences people's perception of the net's effectiveness and their willingness to use it. Monitoring the physical integrity of ITNs over time provides information for replenishment schedules and purchasing decisions. Currently, the assessment is conducted by manually counting holes and estimating their size to class the net as functional or not. This approach is laborious to routinely conduct during field surveys of ITNs. Automated image analysis may provide a rapid assessment of the physical integrity of ITNs but it is not known if the images can capture sufficient information. As a first step, this study aimed to determine the agreement between estimated hole surface areas derived from 1) manually segmented images, 2) manual hole counting compared to 3) ground truth obtained by calibrated close-up shots of individual holes.
Methods: The physical integrity of 75 ITNs was assessed by manual hole counting, image analysis and ground truth, and these methods were compared. For the image analysis, a graphical user interface was developed and used for the segmentation of holes visible in photographs taken from each side of the net. The hole surface area was then computed from this data. The agreement between the estimates from image analysis and manual hole counting was compared to the ground truth using the Bland-Altman method.
Results: There was substantial agreement between the manually segmented image analysis estimates and the ground truth hole surface areas. The overall bias was small, with a mean ratio of the hole surface area from image analysis to the ground truth of 0.70, and the 95% limits of agreement ranging from 0.35 to 1.38. Manual hole counting underestimated the hole surface area compared to the ground truth, particularly among nets with holes above 10 cm in diameter.
Conclusion: Images coupled with manual segmentation contain sufficient information to calculate hole surface area. This lays the groundwork for incorporating automatic hole detection, offering a fast and objective method for routine assessment of physical integrity of ITNs.
Introduction
Insecticide-treated nets (ITNs) are the frontline defense against malaria, but their effectiveness depends heavily on their physical integrity. As nets develop holes through regular use, they lose their ability to act as a physical barrier, and users often become less likely to utilize them. Monitoring this degradation is crucial for health organizations to manage replenishment schedules and evaluate net brands. Currently, the standard "WHO method" involves a technician manually counting holes and categorizing them into size classes (e.g., "larger than a finger," "larger than a fist"). This process is labor-intensive, slow, and inherently subjective. While digital image analysis offers a potential automated alternative, previous studies noted discrepancies between digital and manual methods without a clear reference for accuracy. This study establishes a definitive "ground truth" to evaluate whether image-based assessment can provide a viable, more accurate monitoring solution.
Methods and Materials
The researchers analyzed 75 field-aged ITNs from five different brands, collected after 36 months of community use in Bagamoyo, Tanzania. Three measurement approaches were compared:
- The WHO Method (Manual): Technicians categorized holes into four size classes (0.5–2 cm, 2–10 cm, 10–25 cm, and >25 cm), estimating total surface area by assuming circular holes and using category mid-points.
- Image Analysis (IA): High-resolution photographs of each net (four sides and the roof) were taken against a high-contrast black background. A custom Graphical User Interface (GUI) allowed for manual segmentation (tracing) of holes within these images.
- Ground Truth (GT): To establish a high-fidelity reference, calibrated close-up shots of every hole (>0.5 cm) were taken using ArUco markers (computer vision reference tags) for exact scaling. This allowed for precise area calculations regardless of hole shape.
Results
The study revealed that the standard WHO manual counting method significantly underestimates physical damage. On average, the WHO method captured only about 30% of the true hole surface area, with an overall median ratio of 0.30 compared to ground truth. This discrepancy is largely attributed to the assumption of circularity; real-world holes are often irregular or elongated, leading to significant data loss when using simplified size categories.
In contrast, image analysis (IA) showed substantial agreement with the ground truth. While IA still exhibited a slight downward bias—estimating approximately 70% to 74% of the true area—this bias was consistent across different levels of damage. This suggests that the necessary information is present in the photographs, and the slight underestimation likely stems from perspective distortion or holes hidden in net folds.
Furthermore, the study found that a net classified as having 1000 cm² of damage by the WHO method (the threshold for being "too torn") actually possesses between 2000 and 3000 cm² of real-world damage. Using this adjusted threshold, image analysis achieved over 80% sensitivity and specificity in identifying nets that meet the WHO's failure criteria.
Discussion and Future Directions
These findings demonstrate that digital images contain sufficient data to transition toward objective, automated monitoring of ITNs. The primary limitations identified include obscured damage near hanging loops or within deep folds, and a relatively small sample size of nets with very low levels of damage (1–100 cm²).
The current manual segmentation process in image analysis still requires human intervention. However, the high-fidelity dataset produced by this study provides the necessary foundation for training machine learning models, such as Mask R-CNN, to perform fully automated hole detection. Moving toward an automated, image-based system will remove human subjectivity and provide more reliable data for malaria control programs worldwide.
Summary updated