Abstract
A captura ou a elaboração de uma imagem pode ser realizada por diversas técnicas: fotografia, pintura, colagem, computação gráfica ou outra forma artística. No âmbito digital, imagens caracterizam-se por serem matrizes numéricas em pixels que podem ser processadas computacionalmente para análise e, inclusive, síntese em novos objetos de mídia. Também no âmbito digital, a produção imagética multiplica-se e diversifica-se. Neste artigo, apresentamos dois conceitos que inauguram possibilidades para ressignificar a produção e a distribuição de imagens no contexto digital: coleção de imagens e visualização de mídia. Para ilustrar tais conceitos, discutimos duas investigações de natureza teórica-prática conduzidas entre 2016 e 2019 no Laboratório de Visão e Computação Gráfica (Visgraf) do Instituto de Matemática Pura e Aplicada (IMPA).
Introduction
We are currently navigating an era of "big image data." The explosion of social media and the massive digitization of cultural archives have moved us beyond viewing individual images toward interacting with massive image collections. Traditional information visualization, which often relies on abstract primitives like dots and lines, frequently fails to capture the semantic richness of the images themselves. This research addresses the gap between possessing vast visual datasets and the ability to meaningfully explore or understand them.
Conceptual Framework: The Image Collection
The researchers at IMPA’s Visgraf Lab propose a shift in perspective: treating an image collection as a distinct media object, situated between a static image and a video. They define this object through a mathematical framework consisting of:
- Affinity Relation: A common link, such as time, location, or theme.
- Common Support: A shared space (geospatial or conceptual) where images are mapped.
- Attribute Model: The metadata and features extracted from the images.
Methodology: Media Visualization and Deep Learning
To process these collections, the authors employ "Media Visualization" principles popularized by Lev Manovich. These include techniques like Zoom out (viewing the entire collection simultaneously to see patterns) and Remapping (rearranging images into new configurations, such as mosaics).
To handle the semantic classification of modern datasets, the team utilized Transfer Learning with the Inception-v3 Convolutional Neural Network. By retraining the model on small, manually labeled samples (approximately 100 images), they achieved high accuracy in identifying specific subjects—such as the Olympic torch—within messy social media feeds.
Case Study 1: Rio 2016 Social Media Analysis
The "Observatór!o2016" project analyzed approximately 1 million tweets and 180,000 unique images from the Rio Olympics. The system successfully identified over 4,000 images of the Olympic torch, even when text tags were missing. The team developed interactive portals to allow users to navigate this data through various lenses, including geospatial mapping and video spheres.
Case Study 2: Historical Archives in 3D
In collaboration with the Instituto Moreira Salles (IMS), the researchers integrated 19th-century photography by Marc Ferrez into modern 3D environments. Using KML (Keyhole Markup Language), they created <photoOverlay> elements that "pin" historical photos within Google Earth. This allows users to align their current 3D view with the historical camera angle, effectively creating a "time machine" that highlights urban evolution.
Insights and Future Work
This research demonstrates that sophisticated AI tools are becoming increasingly accessible for the humanities, requiring relatively small training sets to yield powerful results. By treating collections as media objects, we can extract cultural narratives that are invisible at the level of individual files.
Future work will focus on overcoming resolution bottlenecks in immersive environments and automating the extraction of higher-level semantic metadata, such as complex social actions or specific historical figures, to further enrich these interactive narratives.
Summary updated