Discovering and Visualizing Relations in High Dimensional Data

Abstract

$\mathcal{A}$ dataset with M items has 2^M subsets anyone of which may be the one we really want. With a good data display our fantastic pattern-recognition ability can not only cut great swaths searching through this combinatorial explosion but also extract insights from the visual patterns. These are the core reasons for data visualization. With Parallel Coordinates (abbr.-cs) the search for multivariate relations in high dimensional datasets is transformed into a 2-D pattern recognition problem. Multidimensional exploration is illustrated on real datasets, in the process describing good query design with atomic queries and compound ones using boolean operations. Then complex datasets are classified with a geometric classification algorithm based on-cs. The algorithm has low computational complexity providing the classification rule explicitly and visually. The minimal set of variables required to state the rule is found and ordered by their predictive value. By means of a new divide and conquer technique the classification is extended to previously inaccessible datasets. This new result and others like the triad adjancency problem appear for the first time. A visual economic model of a real country is constructed and analyzed to illustrate how multivariate relations can be modeled by means of hypersurfaces. The overview at the end provides the foundational understanding for-cs, examples of exciting recent results like viewing convexity in any dimension, non-orientability (as in the Mbius strip) and a prelude of what is on the way: the discovery and display of relational information in high-dimensional datasets as visual patterns multidimensional graphs.