Data Facilities in Visualization Systems

Reference: Chapter 3, Brodlie, etal, "Scientific Visualization: Techniques and Applications, Springer-Verlag, 1992

Introduction

One of the major problems in science and other areas is the massive amount of data that is collected or generated. Visualization systems must be able to enter, store, and retrieve this data. The data also must be converted into one or more internal formats for the visualization system.

This section discusses a taxonomy of different types of data flow, data formats for import/export, data compression techniques (including image compression techniques), and the facilities necessary for managing data.

Data Sources

Data Sources can be external or internal to the system. External sources would be data collected from experimental measurement or generated by a simulation external to the system. Internal sources would be data that was passed between modules, generated by simulations internal to the system, or stored and then retrieved from the system.

Visualization systems should be able to support the real time collection, storage, and analysis of data.

Data Classification

• External and Internal Data
  • External data is data that is external to the visualization system while internal data is data that is internal to the system. A visualization system must be able to import data from many different sources and in many different formats. It must also have the capability to export data.
• Original and Derived Data
  • We need to differentiate between the original experimental or computed data ("raw" data) and any new data that is derived from the original. Especially for experimental data, the original will likely have noise and need to be filtered. A record should be kept of how the derived data was obtained and what operations have been applied to it. For example, in image processing, the record of what image operations have been applied might be kept with the image, or in an associated file.
• Geometric Data
  • Geometric Data is used to represent the shape of objects. This may be in the form of polygons, surface patches, or coordinates.
• Property Data
  • Property Data is non-geometric data that represents specific properties of the objects or properties measured at certain coordinates, such as temperature, pressure, electron density, etc.
• Metadata

  Metadata is data about the data. It can include the following:

  • the type and structure of the associated dataset
  • attributes of the dataset such as units, scales, etc.
  • comments and records of the operations performed on the dataset
  • color tables or any other mapping attributes
• Command Data
  • A system may be command line driven or, more generally, an interactive mouse driven system. It is useful to have some method of automating certain repetitive tasks, perhaps as a script file, or an automatic recording and playback system.
• Control Data
  • Control Data is used to specify the parameters needed for proper execution of the modules in the system. This can be stored and read into the system each time, as for example a color map.
• Data Relationships
  • There may be certain relations in the data. This can be internal to a dataset or between different data sets. An example of internally linked data is a molecular structure where certain atoms are bonded together.

Data Management

Data Management is a very important issue in data visualization. As scientists collect or compute large amounts of raw data, and then generate derived data, reports, images, etc., the management of this becomes crucial. Scientists have not used database management systems (DBMS) much because most DBMS have been developed for commercial use and are not appropriate for data visualization because of the following reasons:

• the relational model (simple rows and columns) is not suitable for multidimensional or hierarchical data
• commercial DBMS have capabilities that are needed for business related data management but not for data visualization. These include such capabilities as concurrency control, audit trails, and transaction processing. In general, a DBMS used for data visualization will not have a large number of people accessing and updating records.
• the programming support for commercial DBMS are usually a Fourth Generation Language (4GL) or COBOL, neither of which is suitable for scientific programming.

A promising development is the emergence of Object-Oriented DBMS. In these systems, an object could consist of all the different types of data that are associated with a particular experiment or set of experiments. Distributed (over a network) OO-DBMS are also becoming available.

Data Transformation

Before or during an analysis of raw data the user will usually want to perform some type of transformation on the data. There are many types of data transformations, as discussed below.

• Data Normalization is used to scale data to a range of values, e.g. from 0.0 to 1.0. This may be necessary so that a graphics or mathematics package can be used to operate on the data.
• Filtering and Smoothing of data is frequently necessary since raw data that comes from experimental measurement usually has noise. The noise may be introduced by the measurement process or it may be integral to the data. The field of Digital Signal Processing (DSP) is the study of these types of techniques.
• Grid Rezoning is the mapping of data from one type of grid to another, for example, to a rectangular grid of pixels for display.
• Coordinate Transformations may be performed on the data to facilitate the data analysis or display of results. Examples of this might be to transform to a logarithmic scale, or from Cartesian to polar or spherical coordinates.

Feature Detection, Enhancement, and Extraction are different image processing techniques.

Data Compression

A huge volume of data may be generated from the initial experiment or computation and from further data analysis. For purposes of storage or transmission to another system, this data must be compressed.

Here is a further discussion of data compression and image file formats.

HyperVis Table of Contents