TL;DR
This paper introduces a framework for "multimedia journaling" that tightly couples text documents with embedded media, creating robust and portable archives. By building on the Geode multimedia framework, they enable new navigation paradigms where documents and media can point to each other, enhancing discoverability and context.
What Problem Does This Paper Address?
The proliferation of digital media, captured easily by devices like camera phones, has led to vast personal media collections. While blogs and word processors allow embedding media, the relationship between the text and the media is often weak. This fragility arises from reliance on specific software, proprietary formats, and external services, making archives vulnerable to changes in technology or availability. The authors aim to create a more robust, self-contained, and portable multimedia archive that preserves the context and relationships between textual descriptions and the media they reference.
Methods
The core of this work is an extension of the Geode personal media manager. Geode already clusters media based on time and location, and supports activity streams. The authors' contribution lies in:
- Document Integration: They developed a document handler using HTML as the base model, allowing for formatted text and raw HTML.
- Contextual Linking: They expanded Geode's existing contextual link display to show links from media objects back to the documents that contain or reference them. This allows users to navigate from an image, for example, to the journal entry that discusses it.
- Cross-Referencing: When documents are saved, the system analyzes them for media references and builds a network of relationships between documents and media. This allows for discovering documents that refer to similar media objects.
- Attribute Propagation: The framework can propagate attributes like location and time from media objects to the documents that reference them, enriching the context of the document.
- Bottom-Up Navigation: The system facilitates discovery by allowing users to navigate media collections based on events and places, starting from individual media objects.
Datasets
The system was tested using data collected over 6 months from seven participants. The devices used included phones (Nokia 6110, N95), digital still cameras, video cameras, and audio recorders. A variety of media encodings were used, including MP3, WAV, AAC, AC3, MP4 (MOV), H264, DIVX, 3GPP, MJPEG (AVI), JPEG, and MPEG-2. The paper does not introduce new benchmarks but rather evaluates the system's functionality on this diverse, real-world dataset.
Key Insights and Ideas
- Bridging the Text-Media Divide: The most significant insight is the creation of a strong, bidirectional link between textual content and multimedia elements. This moves beyond simple embedding to a truly integrated journaling experience.
- Robustness through Openness: By leveraging open formats (HTML) and frameworks (JMF/FOBS with ffmpeg), the system aims for long-term archival robustness, independent of proprietary software or codecs.
- Enhanced Discoverability: The contextual linking and cross-referencing capabilities significantly improve how users can find and explore their media collections and associated narratives. Discovering related documents based on shared media is a powerful new feature.
- Self-Contained Archives: The emphasis on offline browsing and self-contained archives is crucial for managing large personal media collections, addressing concerns about bandwidth and storage costs.
Key Results
The paper demonstrates the functionality of the proposed framework through its implementation within the Geode system. Key qualitative results include:
- The ability to navigate from a media object (e.g., an image) to the documents that contain it.
- The ability to find other documents that refer to similar media objects.
- The propagation of contextual attributes (like time and place) from media to documents.
- The system successfully handled a wide variety of media formats and encodings using the JMF/FOBS player.
- The user interface allows for composing articles, embedding media, and browsing contextual links, as illustrated in Figures 1, 3, and 4.
Quantitative results are not explicitly detailed in terms of performance metrics like search speed or storage efficiency, but the system's ability to manage and navigate diverse media types is highlighted.
Limitations and Future Work
Limitations:
- The authors acknowledge that their system lacks some of the dynamic facilities of blogs, such as comments.
- While the system is self-contained and browsable offline, it doesn't inherently support online viewing or interaction in the same way a public blog does.
Future Work:
- Blog Publishing Integration: The authors suggest incorporating a blog-publishing facility using APIs like Blogger and MetaWebLog. This would allow content to be viewed online, though potentially at the cost of some of the offline navigation features.
- Further Contextual Propagation: Exploring more sophisticated ways to propagate relationships and attributes between documents and media.
- Enhanced Dynamic Features: Investigating ways to add more interactive elements, similar to those found in modern blogs.
Summary updated