Abstract: In this paper, we describe our concepts to visualize very large amounts of data. Our visualization techniques which have been developed to support querying large scientific databases are designed to visualize as many data items as possible on current display devices. Even if we are able to use each pixel of the display device to visualize one data value, the number of data items that can be visualized is quite limited. Therefore, in our system we consider only those data items which are ‘close’ to a user-specified query. The data items are arranged according to their distance from the query. Multiple windows are used for the different attributes of the data and the distance of each of the attributes from the query is represented by color. After briefly describing the basic ideas of our visualization techniques, in this paper we focus on two specific issues which arise in connection with our visualization techniques: First, we compare the data space which is the amount of information available in the database to the visualization space which is the amount of information that can be visualized by our visualization techniques. Second, we discuss the usefulness of different color models for our distance-to-color mapping and define a new color model which produces adequate color scales.
Introduction
By the mid-1990s, the rapid advancement of storage technology allowed for the collection of terabytes of data from sensors, financial transactions, and satellite systems. However, a significant bottleneck emerged: the human ability to perceive and explore this data. Traditional visualization methods, such as scatter plots or textual displays, were limited to a few hundred items, leaving the vast majority of database records as a "drop in the ocean."
In "Issues in Visualizing Large Databases," Keim and Kriegel address this gap by proposing the VisDB system. The core philosophy is to integrate the human user into the data exploration loop by maximizing the "visualization space"—using every available pixel on a display to represent data values.
The VisDB Visualization Strategy
To manage millions of data points, VisDB employs a query-dependent approach. Rather than attempting to show the entire database statically, the system calculates the "distance" of every record from a user-defined query. This distance represents how well a record matches the search criteria.
Visualization Techniques
- Spiral Technique: The most relevant results (exact hits) are placed at the center of the screen. Approximate results spiral outward, filling every pixel to provide a dense overview of the data distribution.
- Axes Technique: This variation assigns specific dimensions to the X and Y axes, allowing users to see not just the distance from a query, but the direction of that distance (e.g., whether a value is higher or lower than the target).
- Grouping Technique: To provide a holistic view of a single record, all dimensions for a data item are grouped together in a small localized area, rather than being split across multiple attribute windows.
Formalizing Visualization Space
A key contribution of the paper is the mathematical definition of "Visualization Space." The authors argue that any visualization is a surjective mapping from a nearly infinite data space to a strictly limited visual one, defined by the number of pixels ($X \times Y$) and the number of distinguishable colors ($Col$).
As shown in the formalization above, the visualization space is constrained by the probability of finding "hits" (yellow pixels) and the arrangement of remaining data points. This framework proves that to maximize information density, every pixel must be utilized as a data carrier, shifting visualization from sparse plots to information-rich, dense displays.
Perceptual Mapping: The HSI Color Model
Mapping data distance to color is not merely an aesthetic choice; it is a perceptual challenge. Standard computer graphics color models like HSV (Hue, Saturation, Value) and HLS (Hue, Lightness, Saturation) are often unsuitable for data visualization because their perceived brightness is not monotonic. For example, a bright yellow might appear "closer" or more important to a user than a dark blue, even if the blue represents a mathematically superior match.
To solve this, the authors developed the HSI (Hue, Saturation, Intensity) model. By using a circular cone geometry, the HSI model ensures that as data relevance decreases, the color intensity decreases linearly and predictably. This allows for "pre-attentive" processing, where a user can instantly identify clusters of relevant data based on brightness alone.
Results and Conclusion
The VisDB system successfully visualized approximately 70,000 items simultaneously on a standard $1024 \times 1280$ resolution display—a massive leap over the 1,000-item limit of contemporary methods. By leveraging the HSI model, the system created intuitive color scales where "brighter equals closer," enabling users to identify correlations across multiple attribute windows.
While the authors acknowledge limits in human perception (such as lateral inhibition) and the complexity of navigating m-dimensional query paths, this work laid the foundation for modern dense pixel visualization and highlighted the critical need for perceptually accurate color mapping in data science.
Summary updated