Automatic fusion of aerial optical imagery and untextured LiDAR data has been of significant interest for generating photo-realistic 3D urban models in recent years. However, unsupervised, robust registration still remains a challenge. This paper presents a new registration method that does not require priori knowledge such as GPS/INS information. The proposed algorithm is based on feature correspondence between a LiDAR depth map and a depth map from an optical image. Each optical depth map is generated from edge-preserving dense correspondence between the image and another optical image, followed by ground plane estimation and alignment for depth consistency. Our two-pass RANSAC with Maximum Likelihood estimation incorporates 2D-2D and 2D-3D correspondences to yield robust camera pose estimation. Experiments with a LiDAR-optical imagery dataset show promising results, without using initial pose information.

Scatterplots are commonly used to visualize multidimensional data; however, 2D projections of data offer limited understanding of the high-dimensional interactions between data points. We introduce an interactive 3D extension of scatterplots called the Regression Cube (RC), which augments a 3D scatterplot with three facets on which the correlations between the two variables are revealed by sensitivity lines and sensitivity streamlines. The sensitivity visualization of local regression on the 2D projections provides insights about the shape of the data through its orientation and continuity cues. We also introduce a series of visual operations such as clustering, brushing, and selection supported in RC. By iteratively refining the selection of data points of interest, RC is able to reveal salient local correlation patterns that may otherwise remain hidden with a global analysis. We have demonstrated our system with two examples and a user-oriented evaluation, and we show how RCs enable interactive visual exploration of multidimensional datasets via a variety of classification and information retrieval tasks. A video demo of RC is available.

With the advances in dual medical imaging, the requirements for multimodal and multifield volume visualization begin to emerge. One of the challenges in multimodal visualization is how to simplify the process of generating informative pictures from complementary data. In this paper we present an automatic technique that makes use of dual modality information, such as CT and PET, to produce effective focus+context volume visualization. With volume ray casting, per-ray visibility histograms summarize the contribution of samples along each ray to the final image. By quantifying visibility for the region of interest, indicated by the PET data, occluding tissues can be made just transparent enough to give a clear view of the features in that region while preserving some context. Unlike most previous methods relying on costly-preprocessing and tedious manual tuning, our technique achieves comparable and better results based on on-the-fly processing that still enables interactive visualization. Our work thus offers a powerful visualization technique for examining multimodal volume data. We demonstrate the technique with scenarios for the detection and diagnosis of cancer and other pathologies.

Scatterplots remain a powerful tool to visualize multidimensional data. However, accurately understanding the shape of multidimensional points from 2D projections remains challenging due to overlap. Consequently, there are a lot of variations on the scatterplot as a visual metaphor for this limitation. An important aspect often overlooked in scatterplots is the issue of sensitivity or local trend, which may help in identifying the type of relationship between two variables. However, it is not well known how or what factors influence the perception of trends from 2D scatterplots. To shed light on this aspect, we conducted an experiment where we asked people to directly draw the perceived trends on a 2D scatterplot. We found that augmenting scatterplots with local sensitivity helps to fill the gaps in visual perception while retaining the simplicity and readability of a 2D scatterplot. We call this augmentation the generalized sensitivity scatterplot (GSS). In a GSS, sensitivity coefficients are visually depicted as flow lines, which give a sense of continuity and orientation of the data that provide cues about the way data points are scattered in a higher dimensional space. We introduce a series of glyphs and operations that facilitate the analysis of multidimensional data sets using GSS, and validate with a number of well-known data sets for both regression and classification tasks.

This paper introduces a new method for estimating the local neighborhood and scale of data points to improve the robustness of spectral clustering algorithms. We employ a subset of empty region graphs - the β-skeleton - and non-linear diffusion to define a locally-adapted affinity matrix, which, as we demonstrate, provides higher quality clustering than conventional approaches based on κ nearest neighbors or global scale parameters. Moreover, we show that the clustering quality is far less sensitive to the choice of β and other algorithm parameters, and to transformations such as geometric distortion and random perturbation. We summarize the results of an empirical study that applies our method to a number of 2D synthetic data sets, consisting of clusters of arbitrary shape and scale, and to real multi-dimensional classification examples from benchmarks, including image segmentation.

In this paper, we study the sensitivity of centrality metrics as a key metric of social networks to support visual reasoning. As centrality represents the prestige or importance of a node in a network, its sensitivity represents the importance of the relationship between this and all other nodes in the network. We have derived an analytical solution that extracts the sensitivity as the derivative of centrality with respect to degree for two centrality metrics based on feedback and random walks. We show that these sensitivities are good indicators of the distribution of centrality in the network, and how changes are expected to be propagated if we introduce changes to the network. These metrics also help us simplify a complex network in a way that retains the main structural properties and that results in trustworthy, readable diagrams. Sensitivity is also a key concept for uncertainty analysis of social networks, and we show how our approach may help analysts gain insight on the robustness of key network metrics. Through a number of examples, we illustrate the need for measuring sensitivity, and the impact it has on the visualization of and interaction with social and other scale-free networks.

Images that seamlessly combine views at different levels of detail are appealing. However, creating such multiscale images is not a trivial task, and most such illustrations are handcrafted by skilled artists. This paper presents a framework for direct multiscale rendering of geometric and volumetric models. The basis of our approach is a set of non-linearly bent camera rays that smoothly cast through multiple scales. We show that by properly setting up a sequence of conventional pinhole cameras to capture features of interest at different scales, along with image masks specifying the regions of interest for each scale on the projection plane, our rendering framework can generate non-linear sampling rays that smoothly project objects in a scene at multiple levels of detail onto a single image. We address two important issues with non-linear camera projection. First, our streamline-based ray generation algorithm avoids undesired camera ray intersections, which often result in unexpected images. Second, in order to maintain camera ray coherence and preserve aesthetic quality, we create an interpolated 3D field that defines the contribution of each pinhole camera for determining ray orientations. The resulting multiscale camera has three main applications: (1) presenting hierarchical structure in a compact and continuous manner, (2) achieving focus+context visualization, and (3) creating fascinating and artistic images.

This paper presents a study of gradient estimation methods for rendering unstructured-mesh volume data. Gradient estimation is necessary for rendering shaded isosurfaces and specular highlights, which provide important cues for shape and depth. Gradient estimation has been widely studied and deployed for regular-grid volume data to achieve local illumination effects, but has been, otherwise, for unstructured-mesh data. As a result, most of the unstructured-mesh volume visualizations made so far were unlit. In this paper, we present a comprehensive study of gradient estimation methods for unstructured meshes with respect to their cost and performance. Through a number of benchmarks, we discuss the effects of mesh quality and scalar function complexity in the accuracy of the reconstruction, and their impact in lighting-enabled volume rendering. Based on our study, we also propose two heuristic improvements to the gradient reconstruction process. The first heuristic improves the rendering quality with a hybrid algorithm that combines the results of the multiple reconstruction methods, based on the properties of a given mesh. The second heuristic improves the efficiency of its GPU implementation, by restricting the computation of the gradient on a fixed-size local neighborhood.

Sparse, irregular sampling is becoming a necessity for reconstructing large and high-dimensional signals. However, the analysis of this type of data remains a challenge. One issue is the robust selection of neighborhoods--a crucial part of analytic tools such as topological decomposition, clustering and gradient estimation. When extracting the topology of sparsely sampled data, common neighborhood strategies such as k-nearest neighbors may lead to inaccurate results, either due to missing neighborhood connections, which introduce false extrema, or due to spurious connections, which conceal true extrema. Other neighborhoods, such as the Delaunay triangulation, are costly to compute and store even in relatively low dimensions. In this paper, we address these issues. We present two new types of neighborhood graphs: a variation on and a generalization of empty region graphs, which considerably improve the robustness of neighborhood-based analysis tools, such as topological decomposition. Our findings suggest that these neighborhood graphs lead to more accurate topological representations of low- and high- dimensional data sets at relatively low cost, both in terms of storage and computation time. We describe the implications of our work in the analysis and visualization of scalar functions, and provide general strategies for computing and applying our neighborhood graphs towards robust data analysis.

We present topological spines--a new visual representation that preserves the topological and geometric structure of a scalar field. This representation encodes the spatial relationships of the extrema of a scalar field together with the local volume and nesting structure of the surrounding contours. Unlike other topological representations, such as contour trees, our approach preserves the local geometric structure of the scalar field, including structural cycles that are useful for exposing symmetries in the data. To obtain this representation, we describe a novel mechanism based on the extraction of extremum graphs--sparse subsets of the Morse-Smale complex that retain the important structural information without the clutter and occlusion problems that arise from visualizing the entire complex directly. Extremum graphs form a natural multiresolution structure that allows the user to suppress noise and enhance topological features via the specification of a persistence range. Applications of our approach include the visualization of 3D scalar fields without occlusion artifacts, and the exploratory analysis of high-dimensional functions.

Direct volume rendering is an important tool for visualizing complex data sets. However, in the process of generating 2D images from 3D data, information is lost in the form of attenuation and occlusion. The lack of a feedback mechanism to quantify the loss of information in the rendering process makes the design of good transfer functions a difficult and time consuming task. In this paper, we present the general notion of visibility histograms, which are multidimensional graphical representations of the distribution of visibility in a volume-rendered image. In this paper, we explore the 1D and 2D transfer functions that result from intensity values and gradient magnitude. With the help of these histograms, users can manage a complex set of transfer function parameters that maximize the visibility of the intervals of interest and provide high quality images of volume data. We present a semiautomated method for generating transfer functions, which progressively explores the transfer function space toward the goal of maximizing visibility of important structures. Our methodology can be easily deployed in most visualization systems and can be used together with traditional 1D and 2D opacity transfer functions based on scalar values, as well as with other more sophisticated rendering algorithms.

We present a technique which automatically converts a small number of single-view volume rendered images of the same 3D data set into a compact representation of that data set. This representation is a multi-layered image, or an explorable image, which enables interactive exploration of volume data in transfer function space without accessing the original data. We achieve this by automatically extracting layers depicted in composited images. The layers can then be recombined in different ways to simulate opacity changes and recoloring of individual features. Our results demonstrate that explorable images are especially useful when the volume data is too large for interactive exploration, takes too long to render due to the underlying mesh structure or desired shading effect, or if the original volume data is not available. Explorable images can offer real-time image-based interaction as a preview mechanism for remote visualization or visualization of large volume data on low-end hardware, within a mobile device, or a Web browser.

Abstract The selection of an appropriate global transfer function is essential for visualizing time‐varying simulation data. This is especially challenging when the global data range is not known in advance, as is often the case in remote and in‐situ visualization settings. Since the data range may vary dramatically as the simulation progresses, volume rendering using local transfer functions may not be coherent for all time steps. We present an exploratory technique that enables coherent classification of time‐varying volume data. Unlike previous approaches, which require pre‐processing of all time steps, our approach lets the user explore the transfer function space without accessing the original 3D data. This is useful for interactive visualization, and absolutely essential for in‐situ visualization, where the entire simulation data range is not known in advance. Our approach generates a compact representation of each time step at rendering time in the form of ray attenuation functions, which are used for subsequent operations on the opacity and color mappings. The presented approach offers interactive exploration of time‐varying simulation data that alleviates the cost associated with reloading and caching large data sets.

This paper presents a system for generating dynamic narratives from videos. These narratives are characterized for being compact, coherent and interactive, as inspired by principles of sequential art. Narratives depict the motion of one or several actors over time. Creating compact narratives is challenging as it is desired to combine the video frames in a way that reuses redundant backgrounds and depicts the stages of a motion. In addition, previous approaches focus on the generation of static summaries and can afford expensive image composition techniques. A dynamic narrative, on the other hand, must be played and skimmed in real-time, which imposes certain cost limitations in the video processing. In this paper, we define a novel process to compose foreground and background regions of video frames in a single interactive image using a series of spatio-temporal masks. These masks are created to improve the output of automatic video processing techniques such as image stitching and foreground segmentation. Unlike hand-drawn narratives, often limited to static representations, the proposed system allows users to explore the narrative dynamically and produce different representations of motion. We have built an authoring system that incorporates these methods and demonstrated successful results on a number of video clips. The authoring system can be used to create interactive posters of video clips, browse video in a compact manner or highlight a motion sequence in a movie.

Interactivity is key to exploration of volume data. Interactivity may be hindered due to many factors, e.g. large data size,high resolution or complexity of a data set, or an expensive rendering algorithm. We present a novel framework for visualizing volume data that enables interactive exploration using proxy images, without accessing the original 3D data. Data exploration using direct volume rendering requires multiple (often redundant) accesses to possibly large amounts of data. The notion of visualization by proxy relies on the ability to defer operations traditionally used for exploring 3D data to a more suitable intermediate representation for interaction--proxy images. Such operations include view changes, transfer function exploration, and relighting. While previous work has addressed specific interaction needs, we provide a complete solution that enables real-time interaction with large data sets and has low hardware and storage requirements.

Direct volume rendering is an important tool for visualizing complex data sets. However, in the process of generating 2D images from 3D data, information is lost in the form of attenuation and occlusion. The lack of a feedback mechanism to quantify the loss of information in the rendering process makes the design of good transfer functions a difficult and time consuming task. In this paper, we present the notion of visibility-driven transfer functions, which are transfer functions that provide a good visibility of features of interest from a given viewpoint. To achieve this, we introduce visibility histograms. These histograms provide graphical cues that intuitively inform the user about the contribution of particular scalar values to the final image. By carefully manipulating the parameters of the opacity transfer function, users can now maximize the visibility of the intervals of interest in a volume data set. Based on this observation, we also propose a semi-automated method for generating transfer functions, which progressively improves a transfer function defined by the user, according to a certain importance metric. Now the user does not have to deal with the tedious task of making small changes to the transfer function parameters, but now he/she can rely on the system to perform these searches automatically. Our methodology can be easily deployed in most visualization systems and can be used together with traditional 1D opacity transfer functions based on scalar values, as well as with multidimensional transfer functions and other more sophisticated rendering algorithms.

This paper presents a novel methodology for social network discovery based on the sensitivity coefficients of importance metrics, namely the Markov centrality of a node, a metric based on random walks. Analogous to node importance, which ranks the important nodes in a social network, the sensitivity analysis of this metric provides a ranking of the relationships between nodes. The sensitivity parameter of the importance of a node with respect to another measures the direct or indirect impact of a node. We show that these relationships help discover hidden links between nodes and highlight meaningful links between seemingly disparate sub-networks in a social structure. We introduce the notion of implicit links, which represent an indirect relationship between nodes not connected by edges, seen as hidden connections in complex networks. We demonstrate our methodology on two social network data sets and use sensitivity-guided visualizations to highlight our findings. Our results show that this analytic tool, when coupled with visualization, is an effective mechanism for discovering social networks.

Despite the ever-growing improvements on graphics processing units and computational power, classifying 3D volume data remains a challenge.In this paper, we present a new method for classifying volume data based on the ambient occlusion of voxels. This information stems from the observation that most volumes of a certain type, e.g., CT, MRI or flow simulation, contain occlusion patterns that reveal the spatial structure of their materials or features. Furthermore, these patterns appear to emerge consistently for different data sets of the same type. We call this collection of patterns the occlusion spectrum of a dataset. We show that using this occlusion spectrum leads to better two-dimensional transfer functions that can help classify complex data sets in terms of the spatial relationships among features. In general, the ambient occlusion of a voxel can be interpreted as a weighted average of the intensities in a spherical neighborhood around the voxel. Different weighting schemes determine the ability to separate structures of interest in the occlusion spectrum. We present a general methodology for finding such a weighting. We show results of our approach in 3D imaging for different applications, including brain and breast tumor detection and the visualization of turbulent flow.

We present two visualization techniques for curve-centric volume reformation with the aim to create compelling comparative visualizations. A curve-centric volume reformation deforms a volume, with regards to a curve in space, to create a new space in which the curve evaluates to zero in two dimensions and spans its arc-length in the third. The volume surrounding the curve is deformed such that spatial neighborhood to the curve is preserved. The result of the curve-centric reformation produces images where one axis is aligned to arc-length, and thus allows researchers and practitioners to apply their arc-length parameterized data visualizations in parallel for comparison. Furthermore we show that when visualizing dense data, our technique provides an inside out projection, from the curve and out into the volume, which allows for inspection what is around the curve. Finally we demonstrate the usefulness of our techniques in the context of two application cases. We show that existing data visualizations of arc-length parameterized data can be enhanced by using our techniques, in addition to creating a new view and perspective on volumetric data around curves. Additionally we show how volumetric data can be brought into plotting environments that allow precise readouts. In the first case we inspect streamlines in a flow field around a car, and in the second we inspect seismic volumes and well logs from drilling.

The visualization of complex 3D images remains a challenge, a fact that is magnified by the difficulty to classify or segment volume data. In this paper, we introduce size-based transfer functions, which map the local scale of features to color and opacity. Features in a data set with similar or identical scalar values can be classified based on their relative size. We achieve this with the use of scale fields, which are 3D fields that represent the relative size of the local feature at each voxel. We present a mechanism for obtaining these scale fields at interactive rates, through a continuous scale-space analysis and a set of detection filters. Through a number of examples, we show that size-based transfer functions can improve classification and enhance volume rendering techniques, such as maximum intensity projection. The ability to classify objects based on local size at interactive rates proves to be a powerful method for complex data exploration.

Abstract A sampled object representation (SOR) defines a graphical model using data obtained from a sampling process, which takes a collection of samples at discrete positions in space in order to capture certain geometrical and physical properties of one or more objects of interest. Examples of SORs include images, videos, volume datasets and point datasets. Unlike many commonly used data representations in computer graphics, SORs lack in geometrical, topological and semantic information, which is much needed for controlling deformation and animation. Hence it poses a significant scientific and technical challenge to develop deformation and animation methods that operate upon SORs. Such methods can enable computer graphics and computer animation to benefit enormously from the advances of digital imaging technology. In this state of the art report, we survey a wide range of techniques that have been developed for manipulating, deforming and animating SORs. We consider a collection of elementary operations for manipulating SORs, which can serve as building blocks of deformation and animation techniques. We examine a collection of techniques that are designed to transform the geometry shape of deformable objects in sampled representations and pay particular attention to their deployment in surgical simulation. We review a collection of techniques for animating digital characters in SORs, focusing on recent developments in volume animation.

Much of the visualization research has focused on improving the rendering quality and speed, and enhancing the perceptibility of features in the data. Recently, significant emphasis has been placed on focus+context (F+C) techniques (e.g., fisheye views and magnification lens) for data exploration in addition to viewing transformation and hierarchical navigation. However, most of the existing data exploration techniques rely on the manipulation of viewing attributes of the rendering system or optical attributes of the data objects, with users being passive viewers. In this paper, we propose a more active approach to data exploration, which attempts to mimic how we would explore data if we were able to hold it and interact with it in our hands. This involves allowing the users to physically or actively manipulate the geometry of a data object. While this approach has been traditionally used in applications, such as surgical simulation, where the original geometry of the data objects is well understood by the users, there are several challenges when this approach is generalized for applications, such as flow and information visualization, where there is no common perception as to the normal or natural geometry of a data object. We introduce a taxonomy and a set of transformations especially for illustrative deformation of general data exploration. We present combined geometric or optical illustration operators for focus+context visualization, and examine the best means for preventing the deformed context from being misperceived. We demonstrated the feasibility of this generalization with examples of flow, information and video visualization.

In this paper we describe a GPU-based technique for creating illustrative visualization through interactive manipulation of volumetric models. It is partly inspired by medical illustrations, where it is common to depict cuts and deformation in order to provide a better understanding of anatomical and biological structures or surgical processes, and partly motivated by the need for a real-time solution that supports the specification and visualization of such illustrative manipulation. We propose two new feature-aligned techniques, namely surface alignment and segment alignment, and compare them with the axis-aligned techniques which was reported in previous work on volume manipulation. We also present a mechanism for defining features using texture volumes, and methods for computing correct normals for the deformed volume in respect to different alignments. We describe a GPU-based implementation to achieve real-time performance of the techniques and a collection of manipulation operators including peelers, retractors, pliers and dilators which are adaptations of the metaphors and tools used in surgical procedures and medical illustrations. Our approach is directly applicable in medical and biological illustration, and we demonstrate how it works as an interactive tool for focus+context visualization, as well as a generic technique for volume graphics.

Recent proliferation of computing devices has brought attention to heterogeneous collaborative systems, where key challenges arise from the resource limitations and disparities. Sharing data across disparate devices makes it necessary to employ mechanisms for adapting the original data and presenting it to the user in the best possible way. However, this could represent a major problem for effective collaboration, since users may find it difficult to reach consensus with everyone working with individually tailored data. This paper presents a novel approach to controlling the coupling of heterogeneous collaborative systems by combining concepts from complex systems and data adaptation techniques. The key idea is that data must be adapted to each individual's preferences and resource capabilities. To support and promote collaboration this adaptation must be interdependent, and adaptation performed by one individual should influence the adaptation of the others. These influences are defined according to the user's roles and collaboration requirements. We model the problem as a distributed optimization problem, so that the most useful data--both for the individual and the group as a whole--is scheduled for each user, while satisfying their preferences, their resource limitations, and their mutual influences. We show how this approach can be applied in a collaborative 3D design application and how it can be extended to other applications.

In this paper, we describe a methodology and implementation for interactive dataset traversal using motion-controlled transfer functions. Dataset traversal here refers lo the process of translating a transfer function along a specific path. In scientific visualization, it is often necessary to manipulate transfer functions in order to visualize datasets more effectively. This manipulation of transfer functions is usually performed globally, i.e., a new transfer function is applied to the entire dataset. Our approach allows one to locally manipulate transfer functions while controling its movement along a traversal path. The method we propose allows the user to select a traversal path within the dataset, based on the shape of the volumetric model and manipulate a transfer function along this path. Examples of dataset traversal include the animation of transfer functions along a pre-defined path, the simulation of flow in vascular structures, and the visualization of convoluted shapes. For example, this type of traversal is often used in medical illustration to highlight flow in blood vessels. We present an interactive implementation of our method using graphics hardware, based on the decomposition of the volume. We show examples of our approach using a variety of volumetric datasets, and we also demonstrate that with our novel decomposition, the rendering process is faster.

We present a novel paradigm for human to human asymmetric collaboration. There is a need for people at geographically separate locations to seamlessly collaborate in real time as if they are physically co-located. In our system one user (novice) works in the real world and the other user (expert) works in a parallel virtual world. They are assisted in this task by an Intelligent Agent (IA) with considerable knowledge about the environment. Current tele-collaboration systems deal primarily with collaboration purely in the real or virtual worlds. The use of a combination of virtual and real worlds allows us to leverage the advantages from both the worlds.

Heterogeneity aspects in mobile collaborative systems, such as differences in user's interest, semantic conflicts across different domains and representations, and disparate device capabilities, cause difficulties in developing software applications. One of the key problems for collaborative applications is maintaining a consistent shared state.In this paper, we describe a framework that manages several aspects of heterogeneity to maintain consistency across the collaborating sites. We assume graph data structure for application state representation. Our framework is based on structural and semantic mappings between graph structures. The mapping can be customized to meet different requirements through user-defined policies and rules. An important constraint is efficient use of scarce system resources.We describe several applications built using the framework to collaboratively share XML documents. The XML documents in our case are 2D/3D representations of virtual worlds. We also show the performance results of our framework which demonstrate its feasibility for mobile scenarios.