TL;DR
This paper explores how to best integrate interactive visualizations within computational notebooks for complex data analysis tasks, specifically in performance analysis. They propose a task analysis framework to guide design decisions and present a novel notebook-embedded visualization that bridges the gap between direct manipulation and scripting, enabling more fluid and insightful exploration.
What Problem Does This Paper Address?
Interactive visualizations are powerful for exploring data, but often limited to predefined tasks. Scripting offers flexibility but can be cumbersome for free-form exploration. Computational notebooks, like Jupyter, combine code and output, offering a promising environment for integrating these two paradigms. However, designing effective hybrid systems that seamlessly transfer information and state between visualization and scripting components remains a challenge. This paper tackles this by investigating how to design visualizations within notebooks that leverage the strengths of both direct manipulation and scripting, particularly for complex exploratory data analysis in fields like parallel performance analysis.
Methods
The authors employ a design study methodology, starting with a formal task analysis. They categorize tasks based on specificity (complexity/concreteness) and frequency, and also consider suitability for visualization or scripting. This analysis guides their design choices, assigning more frequent, less specific tasks to visualization and more specific, less frequent tasks to scripting. They then design and validate a notebook-embedded interactive visualization for calling context trees (CCTs), a common data structure in performance analysis. A key aspect of their design is enabling two-way data and state transfer between the visualization and Python scripting cells within the notebook. They also investigate different interaction modalities, specifically manual vs. automatic updating of cells.
Datasets
The paper uses datasets from the Kripke application, specifically a 64-core and a 128-core execution, resulting in trees of approximately 1500 and 2700 nodes, respectively. These datasets are used for evaluation tasks.
Key Insights and Ideas
- Task-driven design is crucial: A systematic approach to analyzing tasks based on specificity, frequency, and suitability is essential for deciding where to place functionality (visualization vs. scripting).
- Hybrid is better: For users familiar with scripting, a combined visualization-scripting approach offers significant advantages over purely visual or purely scripting methods, enabling complex queries and easier debugging.
- Seamless state transfer is key: The ability for the visualization to update scripting cells and vice-versa, and for state to be easily saved and restored, is critical for a fluid workflow.
- Automatic updating can reduce friction: While manual updating is more explicit, automatic updating can significantly streamline the workflow by reducing the need for manual re-execution of cells, especially in exploratory scenarios.
- User experience matters: Even with powerful features, the design must be intuitive. Participants expressed a desire for features like undo and clearer state tracking.
Key Results
- Participants successfully completed evaluation tasks using the designed visualization and workflow.
- The task analysis framework proved effective in guiding design decisions.
- Users expressed strong support for the integrated visualization-scripting approach, citing benefits like easier query construction, debugging, and sharing.
- Automatic updating was generally preferred for exploratory tasks due to its fluidity, though concerns about state tracking were raised.
- Participants found the linked table view (showing details of selected nodes) very useful for understanding and communicating findings.
Limitations and Future Work
The authors acknowledge limitations in the generalizability of their findings due to the small, specialized participant pool and the typical one-hour session duration. They suggest future work should focus on improving explicit state tracking for both notebook cells and data objects, and further exploring alternative designs for coordinated views within notebooks. They also note that the effectiveness of automatic updating might depend on users' familiarity with scripting and the complexity of the notebook environment.
Summary updated