Most computer vision benchmarks are built on the assumption that the world is composed of "things"—rigid objects like cars, chairs, and pedestrians that possess stable geometries, distinct edges, and predictable textures. However, a significant portion of the visual world is "amorphous." Substances such as smoke, fire, gas plumes, rip currents, and even food do not conform to fixed shapes. Recognizing these entities requires moving beyond traditional geometric priors toward models that can handle transparency, deformation, and ill-defined boundaries.
The Failure of Rigid Geometric Priors
Standard object detection frameworks typically rely on bounding boxes or precise polygonal masks. As Zuo et al. argue, these models are optimized for objects with consistent geometry, making them ill-suited for "flexible" entities like clouds or fire. When an object lacks a defined structure, the very act of labeling it becomes subjective. Zhu et al. note that the lack of well-defined edges in smoke and wildfires leads to high variance in ground-truth annotations, which complicates the training of traditional detectors.
This challenge extends to industrial and environmental contexts. For instance, Nash and Birbilis describe corrosion as an amorphous phenomenon that lacks clear boundaries, making it difficult for convolutional neural networks (CNNs) to localize. Similarly, Gueguen and Hamid found that damage detection systems often fail on non-rigid structures like mud huts, which lack the sharp geometric signatures of modern buildings and are easily confused with natural terrain.
The Challenge of Transparency and Contrast
Unlike solid objects, many amorphous substances are semi-transparent or diffuse. This physical property creates two primary hurdles:
- Background Interference: The appearance of the substance changes based on what is behind it. Rouhafzay et al. highlight this in the context of liquid detection, where the lack of consistent geometric structure and varying background transparency make it difficult to define stable features.
- Low Contrast: Zhang et al. point out that wildfire smoke often has low contrast against its environment, making precise localization nearly impossible for models expecting distinct textures.
To address this, researchers have turned to physics-inspired features. Han and Pu developed a specialized detection head to handle the semi-transparency of flames, while Zhang et al. utilized "transmission" features derived from image dehazing to better segment smoke plumes.
Modeling Temporal and Fluid Dynamics
Because amorphous objects lack static semantic parts (like the wheels of a car), their identity is often tied to how they move. Bhatt et al. addressed this by using 3D convolutions in a "Spatio-Temporal U-Net" to capture the low-level temporal signatures of deforming gas plumes. By focusing on motion rather than static shape, the authors could distinguish plumes from structured background motion like walking people.
In the case of gaseous emissions, Cao et al. introduced the "Voxel Shift Field," a mechanism designed to model the ever-changing flow of gas across space and time. This shift away from spatial priors is echoed by de Silva et al., who combined fluid dynamics analysis with deep learning to identify rip currents—features that Dumitriu et al. describe as a "nightmare" for computer vision due to their ephemeral and boundary-less nature.
Beyond Simple Classification
Recognizing amorphous objects often requires more than a binary label. Because these substances vary in density and risk, researchers are developing metrics to quantify their properties. Han and Pu introduced a "Burning Intensity" metric for fire, while Nash and Birbilis emphasized the need for "confidence scores" in corrosion detection to mitigate the risks of automated industrial inspection.
Even in everyday tasks like food logging, the amorphous nature of the subject adds complexity. Myers et al. note that food items are visually ambiguous and often hidden within composite meals, requiring a combination of semantic segmentation and depth estimation to predict nutritional volume. Whether detecting a deadly rip current or estimating the calories in a bowl of soup, the core challenge remains the same: the vision system must learn to see the "stuff" as clearly as it sees the "things."
Key Foundational Work
Large-scale damage detection using satellite imagery (2015) — contributed 2 concept elements to this direction.
Im2Calories: Towards an Automated Mobile Vision Food Diary (2015) — contributed 1 concept elements to this direction.
Learning for Feature Selection and Shape Detection (1997) — contributed 1 concept elements to this direction.