Relative radiometric normalization (RRN) minimizes radiometric differences among images caused by inconsistencies of acquisition condition. In this study, a cross-sensor RRN method is proposed for optical satellite images from Landsat 8 OLI (L8) and Landsat 7 ETM+ (L7) sensors. The data from these two sensors have different pixel depths. Therefore, a rescaling on the radiometry resolution is performed in the preprocessing. Then, multivariate alteration detection (MAD) based on kernel canonical correlation analysis (KCCA) is adopted, which is called KCCA-based MAD, to select pseudo-invariant features (PIFs). The process of RRN is performed by using polynomial regression with Gaussian weighted regression. In experiments, qualitative and quantitative analyses on images from different sensors are conducted. The experimental result demonstrates the superiority of the proposed nonlinear transformation, in terms of regression quality and radiometric consistency, compared with RRN using linear regression.

Super-resolution (SR) has been used in the realm of remote sensing to improve the resolution of an image and get more detailed spatial information than the original image captured by the sensor on the acquisition device. Several SR methods with different approaches, only focusing on sharpening the edges and forgetting non-edge areas. One of the SR methods that utilize prior gradients, can produce high resolution (HR) images in a short time and produce sharp images for nonhomogeneous areas. But for areas that tend to be homogeneous, a lot of noise appears. This problem will affect the remote sensing process due to the amount of noise that arises. This paper offers to use dynamic weighting on the gradient prior that will reduce the noise on the homogeneous area, while still able to maintains to produce the sharp edges in non-homogeneous areas. An experimental comparison is conducted on both homogeneous and non-homogeneous area using the previous method and the proposed method.

Pseudoinvariant features (PIFs) are ground objects with invariant or near-invariant reflectance during data acquisition. The extraction of PIFs from optical satellite images generally plays a crucial role in relative radiometric normalization (RRN) and landcover change detection. Previous studies extract PIFs from bitemporal images while can generally obtain satisfactory results. However, they do not fully consider the problem of inconsistent PIF selection caused by performing pairwise PIF selection on more than two images. To decrease this inconsistency problem, a novel method called multitemporal and multivariate alteration detection (MMAD) is proposed. This method is based on a weighted generalized canonical correlation analysis, which solves canonical coefficients for multivariable and multitemporal data, thereby resulting in consistent PIF selection and RRN. In addition, a new weighting scheme based on pixel similarity, image quality, and temporal coherence is introduced into MMAD to reduce the sensitivity of PIF selection to landcover changes and to stably distinguish PIFs from non-PIFs. Qualitative and quantitative analyses of several multitemporal images acquired by Satellite Pour l'Observation de la Terre 5 are conducted to evaluate the proposed method. Experimental results demonstrate the superiority of the proposed method over related methods in terms of extracted PIF quality and radiometric consistency, particularly for image sequences with considerable landcover changes.

Escher transmutation is a graphic art that smoothly transforms one tile pattern into another tile pattern with dual perception. A classic example is the artwork called Sky and Water, in which a compelling figure-ground arrangement is applied to portray the transmutation of a bird in sky and a fish in water. The shape of a bird is progressively deformed and dissolves into the background while the background gradually reveals the shape of a fish. This paper introduces a system to create a variety of Escher-like transmutations, which includes the algorithms for initializing a tile pattern with dual figure-ground arrangement, for searching for the best matched shape of a user-specified motif from a database, and for transforming the content and shapes of tile patterns using a content-aware warping technique. The proposed system, integrating the graphic techniques of tile initialization, shape matching, and shape warping, allows users to create various Escher-like transmutations with minimal user interaction. Experimental results and conducted user studies demonstrate the feasibility and flexibility of the proposed system in Escher art generation.

The key to content-aware warping and cropping is adapting data to fit displays with various aspect ratios while preserving visually salient contents. Most previous studies achieve this objective by cropping insignificant contents near frame boundaries and consistently resizing frames through an optimization technique with various preservation constraints and fixed boundary conditions. These strategies significantly improve retargeting quality. However, warping under fixed boundary conditions may bound/limit the preservation of visually salient contents. Moreover, dynamic frame cropping and frame alignment may result in unnatural object/camera motions. In this paper, a floating boundary with volumetric warping and object-aware cropping is proposed to address these problems. In the proposed scheme, visually salient objects in the space-time domain are deformed as rigidly and as consistently as possible using information from matched objects and content-aware boundary constraints. The content-aware boundary constraints can retain visually salient contents in a fixed region with a desired resolution and aspect ratio, called critical region, during warping. Volumetric cropping with the fixed critical region is then performed to adjust stereoscopic videos to the desired aspect ratios. The strategies of warping and cropping using floating boundaries and spatiotemporal constraints enable our method to consistently preserve the temporal motions and spatial shapes of visually salient volumetric objects in the left and right videos as much as possible, thus leading to good content-aware retargeting. In addition, by considering shape, motion, and disparity preservation, the proposed scheme can be applied to various media, including images, stereoscopic images, videos, and stereoscopic videos. Qualitative and quantitative analyses of stereoscopic videos with diverse camera and considerable motions demonstrate a clear superiority of the proposed method over related methods in terms of retargeting quality.

Many building models are shared in the website. These models generally have high-quality appearances. However, the accuracy of these building models would not be suitable for each case. Therefore, a model refinement approach is proposed in this paper to refine building models by using airborne LiDAR data. The proposed approach consists of three steps, namely, data processing, geometric reconstruction, and model refinement. The first step is to partition a point cloud into several surface models. The second step is to establish geometric relations and constraints of building models, and the last step is to refine building models with the obtained constraints. By using the proposed method, the building models can be refined more efficiently and accurately, and the original geometric relations can be also maintained.

A word cloud is a visual representation of a collection of text documents that uses various font sizes, colors, and spaces to arrange and depict significant words. The majority of previous studies on time-varying word clouds focuses on layout optimization and temporal trend visualization. However, they do not fully consider the spatial shapes and temporal motions of word clouds, which are important factors for attracting people's attention and are also important cues for human visual systems in capturing information from time-varying text data. This paper presents a novel method that uses rigid body dynamics to arrange multi-temporal word-tags in a specific shape sequence under various constraints. Each word-tag is regarded as a rigid body in dynamics. With the aid of geometric, aesthetic, and temporal coherence constraints, the proposed method can generate a temporally morphable word cloud that not only arranges word-tags in their corresponding shapes but also smoothly transforms the shapes of word clouds over time, thus yielding a pleasing time-varying visualization. Using the proposed frame-by-frame and morphable word clouds, people can observe the overall story of a time-varying text data from the shape transition, and people can also observe the details from the word clouds in frames. Experimental results on various data demonstrate the feasibility and flexibility of the proposed method in morphable word cloud generation. In addition, an application that uses the proposed word clouds in a simulated exhibition demonstrates the usefulness of the proposed method.

This paper addresses the topic of content-aware stereoscopic image retargeting. The key to this topic is consistently adapting a stereoscopic image to fit displays with various aspect ratios and sizes while preserving visually salient content. Most methods focus on preserving the disparities and shapes of visually salient objects through nonlinear image warping, in which distortions caused by warping are propagated to homogenous and low-significance regions. However, disregarding the consistency of object deformation sometimes results in apparent distortions in both the disparities and shapes of objects. An object-coherence warping scheme is proposed to reduce this unwanted distortion. The basic idea is to utilize the information of matched objects rather than that of matched pixels in warping. Such information implies object correspondences in a stereoscopic image pair, which allows the generation of an object significance map and the consistent preservation of objects. This strategy enables our method to consistently preserve both the disparities and shapes of visually salient objects, leading to good content-aware retargeting. In the experiments, qualitative and quantitative analyses of various stereoscopic images show that our results are better than those generated by related methods in terms of consistency of object preservation.

Cloud covers, which are generally present in optical remote sensing images, limit the usage of acquired images and increase the difficulty in data analysis. Thus, information reconstruction of cloud-contaminated images generally plays an important role in image analysis. This paper proposes a novel method to reconstruct cloud-contaminated information in multitemporal remote sensing images. Based on the concept of utilizing temporal correlation of multitemporal images, we propose a patch-based information reconstruction algorithm that spatiotemporally segments a sequence of images into clusters containing several spatially connected components called patches and then clones information from cloud-free and high-similarity patches to their corresponding cloud-contaminated patches. In addition, a seam that passes through homogenous regions is used in information reconstruction to reduce radiometric inconsistency, and information cloning is solved using an optimization process with the determined seam. These processes enable the proposed method to well reconstruct missing information. Qualitative analyses of image sequences acquired by a Landsat-7 Enhanced Thematic Mapper Plus (ETM+) sensor and a quantitative analysis of simulated data with various cloud contamination conditions are conducted to evaluate the proposed method. The experimental results demonstrate the superiority of the proposed method to related methods in terms of radiometric accuracy and consistency, particularly for large clouds in a heterogeneous landscape.

An increasing number of three-dimensional (3D) building models are being made available on Web-based model-sharing platforms. Motivated by the concept of data reuse, an encoding approach is proposed for 3D building model retrieval using point clouds acquired by airborne light detection and ranging (LiDAR) systems. To encode LiDAR point clouds with sparse, noisy, and incomplete sampling, we introduce a novel encoding scheme based on a set of low-frequency spherical harmonic basis functions. These functions provide compact representation and ease the encoding difficulty coming from inherent noises of point clouds. Additionally, a data filling and resampling technique is proposed to solve the aliasing problem caused by the sparse and incomplete sampling of point clouds. Qualitative and quantitative analyses of LiDAR data show a clear superiority of the proposed method over related methods. A cyber campus generated by retrieving 3D building models with airborne LiDAR point clouds demonstrates the feasibility of the proposed method.

Tourist and destination maps are thematic maps designed to represent specific themes in maps. The road network topologies in these maps are generally more important than the geometric accuracy of roads. A road network warping method is proposed to facilitate map generation and improve theme representation in maps. The basic idea is deforming a road network to meet a user-specified mental map while an optimization process is performed to propagate distortions originating from road network warping. To generate a map, the proposed method includes algorithms for estimating road significance and for deforming a road network according to various geometric and aesthetic constraints. The proposed method can produce an iconic mark of a theme from a road network and meet a user-specified mental map. Therefore, the resulting map can serve as a tourist or destination map that not only provides visual aids for route planning and navigation tasks, but also visually emphasizes the presentation of a theme in a map for the purpose of advertising. In the experiments, the demonstrations of map generations show that our method enables map generation systems to generate deformed tourist and destination maps efficiently.

A cloud removal approach based on information cloning is introduced. The approach removes cloud-contaminated portions of a satellite image and then reconstructs the information of missing data utilizing temporal correlation of multitemporal images. The basic idea is to clone information from cloud-free patches to their corresponding cloud-contaminated patches under the assumption that land covers change insignificantly over a short period of time. The patch-based information reconstruction is mathematically formulated as a Poisson equation and solved using a global optimization process. Thus, the proposed approach can potentially yield better results in terms of radiometric accuracy and consistency compared with related approaches. Some experimental analyses on sequences of images acquired by the Landsat-7 Enhanced Thematic Mapper Plus sensor are conducted. The experimental results show that the proposed approach can process large clouds in a heterogeneous landscape, which is difficult for cloud removal approaches. In addition, quantitative and qualitative analyses on simulated data with different cloud contamination conditions are conducted using quality index and visual inspection, respectively, to evaluate the performance of the proposed approach.

Image retargeting is the process of adapting images to fit displays with various aspect ratios and sizes. Most studies on image retargeting focus on shape preservation, but they do not fully consider the preservation of structure lines, which are sensitive to human visual system. In this paper, a patch-based retargeting scheme with an extended significance measurement is introduced to preserve shapes of both visually salient objects and structure lines while minimizing visual distortions. In the proposed scheme, a similarity transformation constraint is used to force visually salient content to undergo as-rigid-as-possible deformation, while an optimization process is performed to smoothly propagate distortions. These processes enable our approach to yield pleasing content-aware warping and retargeting. Experimental results and a user study show that our results are better than those generated by state-of-the-art approaches.

A novel content-aware warping approach is introduced for video retargeting. The key to this technique is adapting videos to fit displays with various aspect ratios and sizes while preserving both visually salient content and temporal coherence. Most previous studies solve this spatiotemporal problem by consistently resizing content in frames. This strategy significantly improves the retargeting results, but does not fully consider object preservation, sometimes causing apparent distortions on visually salient objects. We propose an object-preserving warping scheme with object-based significance estimation to reduce this unpleasant distortion. In the proposed scheme, visually salient objects in 3D space-time space are forced to undergo as-rigid-as-possible warping, while low-significance contents are warped as close as possible to linear rescaling. These strategies enable our method to consistently preserve both the spatial shapes and temporal motions of visually salient objects and avoid overdeformations on low-significance objects, yielding a pleasing motion-aware video retargeting. Qualitative and quantitative analyses, including a user study and experiments on complex videos containing diverse cameras and dynamic motions, show a clear superiority of our method over related video retargeting methods.

This paper introduces double-sided 2.5D graphics, aiming at enriching the visual appearance when manipulating conventional 2D graphical objects in 2.5D worlds. By attaching a back texture image on a single-sided 2D graphical object, we can enrich the surface and texture detail on 2D graphical objects and improve our visual experience when manipulating and animating them. A family of novel operations on 2.5D graphics, including rolling, twisting, and folding, are proposed in this work, allowing users to efficiently create compelling 2.5D visual effects. Very little effort is needed from the user's side. In our experiment, various creative designs on double-sided graphics were worked out by the recruited participants including a professional artist, which show and demonstrate the feasibility and applicability of our proposed method.

Many building models are available in the internet. Based on the concept of data reuse, we propose a system to retrieve models whose shapes are similar to that of point clouds. To consistently encode building models and point clouds, a novel encoding approach is introduced. Our approach can not only encode polygon models but also can handle unorganized, noisy, and incomplete point clouds. In addition, to ease of encoding problem suffering from incomplete data, we propose a resampling technique for point clouds. The experimental results show that our approach can solve the inherent problems of point clouds, i.e., unorganized, noisy and incomplete, and can encode the 3D shape consistently.

This paper presents a method based on mark-based watershed algorithm to automatically extract grains and determine grain sizes from images. Markers generated by the proposed approaches represent rough locations of grains and aperture. Instead of selecting pixels with local minima as marks, we select markers with a priori knowledge, enabling our approach to efficiently ease the problem of over-segmentation in traditional watershed algorithm. The grains patches are extracted correctly after merging the other fragmental patches using both the color and shape properties. Finally, the distribution of grain size is calculated by fitting an ellipse for each detected grain.

Satellite image have recently been wide used in academia and industry, and thus providing high quality and even cloud-free satellite image is a fundamental and important issue. Globally, the land scenes are on average about 35% cloud covered, as reported by Ju and Roy [1], indicating that cloud covers are generally present in optical satellite images. This phenomenon limits the usage of optical images and increases the difficulty of image analysis. Considerable research efforts have been devoted to the issue of cloud removal to ease the difficulties caused by cloud covers [2-13]. These efforts and studies focus on how to detect clouds and how to reconstruct the information of cloud-contaminated pixels. However, information reconstruction is generally sensitive to the feature structures lying on the boundaries of the cloud-contaminated areas. In this paper, we address the topic of determining optimal boundaries of cloud-contaminated areas for the purpose of cloud removal.

The rapid growth of motion capture data increases the importance of motion retrieval. The majority of the existing motion retrieval approaches are based on a labor-intensive step in which the user browses and selects a desired query motion clip from the large motion clip database. In this work, a novel sketching interface for defining the query is presented. This simple approach allows users to define the required motion by sketching several motion strokes over a drawn character, which requires less effort and extends the users’ expressiveness. To support the real-time interface, a specialized encoding of the motions and the hand-drawn query is required. Here, we introduce a novel hierarchical encoding scheme based on a set of orthonormal spherical harmonic (SH) basis functions, which provides a compact representation, and avoids the CPU/processing intensive stage of temporal alignment used by previous solutions. Experimental results show that the proposed approach can well retrieve the motions, and is capable of retrieve logically and numerically similar motions, which is superior to previous approaches. The user study shows that the proposed system can be a useful tool to input motion query if the users are familiar with it. Finally, an application of generating a 3D animation from a hand-drawn comics strip is demonstrated.

Abstract In this paper, a novel graph‐based shape matching scheme for three‐dimensional articulated objects is introduced. The underlying graph structure of a given 3D model is composed of its topological skeleton and local geometric features. Matching two graph structures is generally an NP‐hard combinatorial optimization problem. To reduce computation cost, two graphs are embedded on a high‐dimensional space, and then matched based on an extension of Earth Mover's Distance (EMD). Furthermore, the symmetric components of an articulated object are determined by a voting algorithm with a self‐matching strategy to refine the matching correspondences. Experimental results show that the proposed approach is robust, even when the models are under the surface disturbances of noise addition, smoothing, simplification, similarity transformation, and pose deformation. In addition, the proposed approach is capable of handling both global and partial shape matching. Copyright © 2011 John Wiley & Sons, Ltd.

Abstract Camera path planning for character motions is a fundamental and important research topic, benefiting many animation applications. Existing optimal‐based approaches are generally computationally expensive and infeasible for interactive applications. In this paper, we propose an efficient approach that can take many constraints of finding the camera path into account and can potentially enable interactive camera control. Instead of solving a highly complicated camera optimization problem in a spatiotemporal four‐dimensional space, we heuristically determine the camera path based on an efficient greedy‐based tree traversal approach. The experimental results show that the proposed approach can efficiently generate a smooth, informative, and aesthetic camera path that can reveal the significant features of character motions. Moreover, the conducted user study also shows that the generated camera paths are comparable to those of a state‐of‐the‐art approach and those made by professional animators. Copyright © 2011 John Wiley & Sons, Ltd.

We introduce a template fitting method for 3D surface meshes. A given template mesh is deformed to closely approximate the input 3D geometry. The connectivity of the deformed template model is automatically adjusted to facilitate the geometric fitting and to ascertain high quality of the mesh elements. The template fitting process utilizes a specially tailored Laplacian processing framework, where in the first, coarse fitting stage we approximate the input geometry with a linearized biharmonic surface (a variant of LS-mesh), and then the fine geometric detail is fitted further using iterative Laplacian editing with reliable correspondence constraints and a local surface flattening mechanism to avoid foldovers. The latter step is performed in the dual mesh domain, which is shown to encourage near-equilateral mesh elements and significantly reduces the occurrence of triangle foldovers, a well-known problem in mesh fitting. To experimentally evaluate our approach, we compare our method with relevant state-of-the-art techniques and confirm significant improvements of results. In addition, we demonstrate the usefulness of our approach to the application of consistent surface parameterization (also known as cross-parameterization).

In this paper, we present a very high-capacity and low-distortion 3D steganography scheme. Our steganography approach is based on a novel multilayered embedding scheme to hide secret messages in the vertices of 3D polygon models. Experimental results show that the cover model distortion is very small as the number of hiding layers ranges from 7 to 13 layers. To the best of our knowledge, this novel approach can provide much higher hiding capacity than other state-of-the-art approaches, while obeying the low distortion and security basic requirements for steganography on 3D models.

Three-dimensional animating meshes have been widely used in the computer graphics and video game industries. Reducing the animating mesh complexity is a common way of overcoming the rendering limitation or network bandwidth. Thus, we present a compact representation for animating meshes based on novel key-frames extraction and animating mesh simplification approaches. In contrast to the general simplification and key-frames extraction approaches which are driven by geometry metrics, the proposed methods are based on a deformation analysis of animating mesh to preserve both the geometric features and motion characteristics. These two approaches can produce a very compact animation representation in spatial and temporal domains, and therefore they can be beneficial in many applications such as progressive animation transmission and animation segmentation and transferring.

In this paper, we propose an interactive model decomposition scheme. In preprocess, we automatically build a protrusive graph (PG) for any given 3D model. The purpose of the PG is to give the user a good clue to partition the model into visually-significant parts. Then, this scheme can interactively partition models according to the user-specified partitioning requirement. Finally, an iterative clustering is used to stabilize our partitions and a smoothing refinement is used to smooth the boundary between adjacent partitions. The experimental results show that the proposed scheme is a flexible and powerful method to decompose models into their significant components.

Abstract An easy‐to‐use mesh pose‐editing system is presented. We take advantage of both skeleton‐based and example‐based approaches in order to provide an intuitive way for artists to edit mesh poses. Our system automatically extracts the skeletons of the remaining example models once the skeleton of a reference mesh is constructed. In our editing system the desired skeleton can be easily and naturally posed using an inverse kinematics (IK) algorithm incorporated with searching the optimal weights in the defined skeleton space of examples meshes. Eventually, the desired shape with detailed deformation can be constructed by blending the example meshes. Experimental results show that the proposed system provides an easy and intuitive control on mesh pose‐editing. Copyright © 2007 John Wiley & Sons, Ltd.

Abstract This paper describes a new integrated scheme for metamorphosis between two closed manifold genus‐0 polyhedral models. Spherical parameterizations of the source and target models are created first. To control the morphing, any number of feature vertex pairs is specified and a fold‐over free warping method is used to align two spherical embeddings. Our method does not create a merged meta‐mesh or execute re‐meshing to construct a common connectivity for morphs. Alternatively, a scheme for the progressive connectivity transformation of two spherical parameterizations is employed to generate the intermediate meshes. A novel semi‐overlay with a geomorph scheme is proposed to reduce the popping effects caused by the connectivity transformation. We demonstrate several examples of aesthetically pleasing morphing sequences using the proposed scheme. Copyright © 2005 John Wiley & Sons, Ltd.

Three-dimensional metamorphosis is a powerful technique to produce a 3D shape transformation between two or more existing models. In this paper, we propose a novel 3D morphing technique that avoids creating a merged embedding that contains the faces, edges, and vertices of two given embeddings. This novel 3D morphing technique dynamically adds or removes vertices to gradually transform the connectivity of 3D polyhedrons from a source model into a target model and simultaneously creates the intermediate shapes. In addition, a priority control function provides the animators with control of arising or dissolving of input models' features in a morphing sequence. This is a useful tool to control a morphing sequence more easily and flexibly. Several examples of aesthetically pleasing morphs are demonstrated using the proposed method.

A feature-guided image interpolation scheme is presented. It is an effective and improved, shape-based interpolation method used for interpolating image slices in medical applications. The proposed method integrates feature line-segments to guide the shape-based method for better shape interpolation. An automatic method for finding these line segments is given. The proposed feature-guided shape-based method can manage translation, rotation and scaling situations when the slices have similar shapes. It can also interpolate intermediate shapes when the successive slices do not have similar shapes. This method is experimentally evaluated using artificial and real two-dimensional and three-dimensional data. The proposed method generated satisfactory interpolated results in these experiments. We demonstrate the practicality, effectiveness and reproducibility of the proposed method for interpolating medical images.

Rhinoplasty, or surgery to reshape the nose, is one of the most common of all plastic-surgery procedures. Rhinoplasty can enhance a patient's appearance and self-confidence, may also correct a birth defect or injury, or help relieve some breathing problem. In this paper, we present a three-dimensional (3-D) surgical simulation system, which can assist surgeons in planning rhinoplasty procedures. This system employs computer graphics and image-processing techniques for the simulation of a rhinoplasty. Although the presented algorithms themselves are not new, the proposed system exploits the new idea to apply 3-D morphing for rhinoplasty, and simulation results are useful for the physicians. According to patients' expectation of what they would like their noses to look like, our system simulates expected results. Our tools provide quantitative measurements of a nose structure. Using these quantitative results, surgeons can arrange appropriate preoperative plans for patients. Finally, experimental results and experiences are reported to evaluate the usefulness of the proposed system.

We describe a low-cost three-dimensional (3-D) virtual colonoscopy system that is a noninvasive technique for examining the entire colon and can assist physicians in detecting polyps inside the colon. Using the helical CT data and proposed techniques, we can three-dimensionally reconstruct and visualize the inner surface of the colon. We generate high resolution of video views of the colon interior structures as if the viewer's eyes were inside the colon. The physicians can virtually navigate inside the colon in two different modes: interactive and automatic navigation, respectively. For automatic navigation, the flythrough path is determined a priori using the 3-D thinning and two-pass tracking schemes. The whole colon is spatially subdivided into several cells, and only potentially visible cells are taken into account during rendering. To further improve rendering efficiency, potentially visible cells are rendered at different levels of detail. Additionally, a chain of bounding volume in each cell is used to avoid penetrating through the colon during navigation. In comparison with previous work, the proposed system can efficiently accomplish required preprocessing tasks and afford adequate rendering speeds on a low-cost PC system.