Abstract The paper presents the functionality of a system which retrieves images on the basis of automatically generated indexes (i.e. semantic image representations, obtained by automatic image analysis, indicating the content of the images). The system consists of two parts: an off-line indexing part and an on-line image retrieval part. The indexing component is used to automatically generate semantic representations of images so that the image retrieval component can use this information to enable image retrieval. The man-machine communication of the image retrieval component is based on an iconical graphical query language to accomplish geographical query specification for image access. Experiments have been carried out on three different sets of images from the following domains: MRI images of the chest, electronic schemas and topographic maps. The experiments show encouraging results especially for domains which have a high degree of formality in their pictorial expression, such as electronic schemas.
Introduction
In the early 1990s, the rapid growth of digital image storage created a significant challenge: efficient retrieval. Traditional methods relied on manual text-based tagging, which the authors argue is inherently inadequate for large-scale databases. Verbal descriptions often fail to capture the precise geometric and spatial nuances of complex images, such as the specific layout of a heart valve in an MRI or the intricate connections in a topographic map. Furthermore, manual annotation is labor-intensive and prone to human error. To address these limitations, the Enigma system was developed to automate the generation of semantic representations and enable content-based retrieval through a visual interface.
Methodology
The Enigma system architecture is divided into two primary phases: off-line indexing and on-line retrieval.
Off-line Indexing
The indexing phase focuses on extracting domain-specific features, termed "iconograms." For instance, in an electronic circuit diagram, an iconogram might represent a specific transistor symbol. To handle variations in shape and noise, the system utilizes Fuzzy Mathematical Morphology. This approach employs a "structuring element" based on the Euclidean distance transform, assigning higher weights to the center of an object and lower weights to its boundaries. This allows for "fuzzy" matching, effectively accommodating minor structural deviations and noise. These extracted features are then stored in a spatial index to facilitate rapid retrieval.
On-line Retrieval
The retrieval phase introduces an Iconical Graphical Query Language. Instead of text-based queries, users interact with visual metaphors by selecting icons and arranging them on a canvas. This allows for the specification of complex spatial relationships, such as "Feature A is North of Feature B" or "Feature C is Contained within Feature D." The system converts this visual query into a graph structure. Retrieval is then performed using subgraph isomorphism matching, identifying images in the database whose indexed features and spatial arrangements align with the query graph.
Experimental Evaluation
The system was evaluated across three distinct domains to assess its versatility:
- Electronic Schemas: High-formality images with rigid geometric rules.
- Topographic Maps: Formal images utilizing standardized legend symbols.
- MRI Images of the Chest: Low-formality organic images with significant variation. This dataset comprised over 200 images from Yale University Medical School.
Results and Discussion
The experiments revealed that the degree of formality in a domain significantly impacts performance. In the electronic schemas test (20 images), a query involving four components and three spatial relationships successfully retrieved all relevant images, ranked by resemblance, in approximately 0.4 seconds per image.
In the more challenging MRI dataset (200+ images), a query for three anatomical features with two spatial relationships required about one minute of user interaction. Despite the organic complexity, the system correctly identified the anatomical plane in 6 out of the top 7 results. A key feature of Enigma is interactive refinement, allowing users to superimpose found features onto target images to visually assess match quality and adjust thresholds.
Limitations and Future Work
While pioneering, Enigma has notable limitations. It requires domain expertise to define "salient patches" (icons) for indexing. Additionally, the computational cost of subgraph isomorphism may pose scalability issues for massive databases. Finally, while the system excels at geometric and spatial matching, it does not possess deep semantic understanding of the image content (e.g., the medical implications of an MRI finding).
Conclusion
Enigma represents a significant step toward content-based image retrieval by prioritizing spatial logic and visual querying over keyword-based systems. Its success in formal domains demonstrates the power of combining fuzzy morphology with graphical query languages for complex visual data.
Summary updated