Neural implicit representations have shown remarkable abilities in jointly modeling geometry, color, and camera poses in simultaneous localization and mapping (SLAM). Current methods use coordinates, positional encodings, or other geometry features as input to query neural implicit functions for signed distances and color which produce rendering errors to drive the optimization in overfitting image observations. However, due to the run time efficiency requirement in SLAM systems, we are merely allowed to conduct optimization on each frame in few iterations, which is far from enough for neural networks to overfit these queries. The underfitting usually results in severe drifts in camera tracking and artifacts in reconstruction. To resolve this issue, we propose query quantized neural SLAM which uses quantized queries to reduce variations of input for much easier and faster overfitting a frame. To this end, we quantize a query into a discrete representation with a set of codes, and only allow neural networks to observe a finite number of variations. This allows neural networks to become increasingly familiar with these codes after overfitting more and more previous frames. Moreover, we also introduce novel initialization, losses, and argumentation to stabilize the optimization with significant uncertainty in the early optimization stage, constrain the optimization space, and estimate camera poses more accurately. We justify the effectiveness of each design and report visual and numerical comparisons on widely used benchmarks to show our superiority over the latest methods in both reconstruction and camera tracking.

It is vital to recover 3D geometry from multi-view RGB images in many 3D computer vision tasks. The latest methods infer the geometry represented as a signed distance field by minimizing the rendering error on the field through volume rendering. However, it is still challenging to explicitly impose constraints on surfaces for inferring more geometry details due to the limited ability of sensing surfaces in volume rendering. To resolve this problem, we introduce a method to infer signed distance functions (SDFs) with a better sense of surfaces through volume rendering. Using the gradients and signed distances, we establish a small surface patch centered at the estimated intersection along a ray by pulling points randomly sampled nearby. Hence, we are able to explicitly impose surface constraints on the sensed surface patch, such as multi-view photo consistency and supervision from depth or normal priors, through volume rendering. We evaluate our method by numerical and visual comparisons on scene benchmarks. Our superiority over the latest methods justifies our effectiveness.

This paper introduces a method to synthesize a 3D tensor field within a constrained geometric domain represented as a tetrahedral mesh. Whereas previous techniques optimize for isotropic fields, we focus on anisotropic tensor fields that are smooth and aligned with the domain boundary or user guidance. The key ingredient of our method is a novel computational design framework, built on top of the symmetric orthogonally decomposable (odeco) tensor representation, to optimize the stretching ratios and orientations for each tensor in the domain. In contrast to past techniques designed only for isotropic tensors, we demonstrate the efficacy of our approach in generating smooth volumetric tensor fields with high anisotropy and shape conformity, especially for the domain with complex shapes. We apply these anisotropic tensor fields to various applications, such as anisotropic meshing, structural mechanics, and fabrication.

Despite achieving impressive improvement in accuracy, most existing monocular 3D human mesh reconstruction methods require large-scale 2D/3D ground-truths for supervision, which limits their applications on unlabeled in-the-wild data that is ubiquitous. To alleviate the reliance on 2D/3D ground-truths, we present a self-supervised 3D human pose and shape reconstruction framework that relies only on self-consistency between intermediate representations of images and projected 2D predictions. Specifically, we extract 2D joints and depth maps from monocular images as proxy inputs, which provides complementary clues to infer accurate 3D human meshes. Furthermore, to reduce the impacts from noisy and ambiguous inputs while better concentrate on the high-quality information, we design an uncertainty-aware module to automatically learn the reliability of the inputs at body-joint level based on the consistency between 2D joints and depth map. Experiments on benchmark datasets show that our approach outperforms other state-of-the-art methods at similar supervision levels.

Non-rigid 3D shape correspondence aims to establish dense correspondences between two non-rigidly deformed 3D shapes. However, the variability and symmetry of non-rigid shapes usually lead to mismatches due to shape deformation, topological changes, or data with severe noise. To finding an accurate correspondence between 3D dynamic shapes for the local deformation complexity, this article proposes a Relation Constrained Capsule Graph Network (RC-CGNet), which combines global and local features by encouraging the relation constraints between the embedding feature space and the input shape space based on the functional maps framework. Specifically, we design a Diffusion Graph Attention Network (DGANet) to segment the surface into parts with correct edge boundary between two regions. The Minimum Spanning Tree (MST) of geodesic curves among the singularities obtained from the segmented parts is added as relation constraints, which can compute isometric correspondences in both direct and symmetric directions. Besides that, the relation-and-attention constrained neural networks are designed to learn the shape correspondence via attention-aware CapsNet and functional maps under relation constraints. To improve the convergence speed and matching accuracy, we propose an optimized residual network structure based on the Nesterov Accelerated Gradient (NAG) to extract local features, and use graph convolution structure to extract global features. Moreover, a lightweight Gated Attention Module (GAM) is designed to fuse global and local features to obtain a richer feature representation. Since the capsule network has better spatial reasoning ability than the traditional convolutional neural network, our novel network architecture is a dual-route capsule network based on Routing Attention Fusion Block (RAFB), filtering low-discriminative capsules from a holistic view by exploiting geometric hierarchical relationships of semantic parts. Experiments on open datasets show that our method has excellent accuracy and wide adaptability.

In recent years, huge progress has been made on learning neural implicit representations from multi-view images for 3D reconstruction. As an additional input complementing coordinates, using sinusoidal functions as positional encodings plays a key role in revealing high frequency details with coordinate-based neural networks. However, high frequency positional encodings make the optimization unstable, which results in noisy reconstructions and artifacts in empty space. To resolve this issue in a general sense, we introduce to learn neural implicit representations with quantized coordinates, which reduces the uncertainty and ambiguity in the field during optimization. Instead of continuous coordinates, we discretize continuous coordinates into discrete coordinates using nearest interpolation among quantized coordinates which are obtained by discretizing the field in an extremely high resolution. We use discrete coordinates and their positional encodings to learn implicit functions through volume rendering. This significantly reduces the variations in the sample space, and triggers more multi-view consistency constraints on intersections of rays from different views, which enables to infer implicit function in a more effective way. Our quantized coordinates do not bring any computational burden, and can seamlessly work upon the latest methods. Our evaluations under the widely used benchmarks show our superiority over the state-of-the-art. Our code is available at https://github.com/MachinePerceptionLab/CQ-NIR.

Human-robot interaction (HRI) is a rapidly growing field that encompasses social and industrial applications. Machine learning plays a vital role in industrial HRI by enhancing the adaptability and autonomy of robots in complex environments. However, data privacy is a crucial concern in the interaction between humans and robots, as companies need to protect sensitive data while machine learning algorithms require access to large datasets. Federated Learning (FL) offers a solution by enabling the distributed training of models without sharing raw data. Despite extensive research on Federated learning (FL) for tasks such as natural language processing (NLP) and image classification, the question of how to use FL for HRI remains an open research problem. The traditional FL approach involves transmitting large neural network parameter matrices between the server and clients, which can lead to high communication costs and often becomes a bottleneck in FL. This paper proposes a communication-efficient FL framework for human-robot interaction (CEFHRI) to address the challenges of data heterogeneity and communication costs. The framework leverages pre-trained models and introduces a trainable spatiotemporal adapter for video understanding tasks in HRI. Experimental results on three human-robot interaction benchmark datasets: HRI30, InHARD, and COIN demonstrate the superiority of CEFHRI over full fine-tuning in terms of communication costs. The proposed methodology provides a secure and efficient approach to HRI federated learning, particularly in industrial environments with data privacy concerns and limited communication bandwidth. Our code is available at https://github.com/umarkhalidAI/CEFHRI-Efficient-Federated-Learning.

The fundamental motivation of the proposed work is to present a new visualization-guided computing paradigm to combine direct 3D volume processing and volume rendered clues for effective 3D exploration. For example, extracting and visualizing microstructures in-vivo have been a long-standing challenging problem. However, due to the high sparseness and noisiness in cerebrovasculature data as well as highly complex geometry and topology variations of micro vessels, it is still extremely challenging to extract the complete 3D vessel structure and visualize it in 3D with high fidelity. In this paper, we present an end-to-end deep learning method, VC-Net, for robust extraction of 3D microvascular structure through embedding the image composition, generated by maximum intensity projection (MIP), into the 3D volumetric image learning process to enhance the overall performance. The core novelty is to automatically leverage the volume visualization technique (e.g., MIP - a volume rendering scheme for 3D volume images) to enhance the 3D data exploration at the deep learning level. The MIP embedding features can enhance the local vessel signal (through canceling out the noise) and adapt to the geometric variability and scalability of vessels, which is of great importance in microvascular tracking. A multi-stream convolutional neural network (CNN) framework is proposed to effectively learn the 3D volume and 2D MIP feature vectors, respectively, and then explore their inter-dependencies in a joint volume-composition embedding space by unprojecting the 2D feature vectors into the 3D volume embedding space. It is noted that the proposed framework can better capture the small/micro vessels and improve the vessel connectivity. To our knowledge, this is the first time that a deep learning framework is proposed to construct a joint convolutional embedding space, where the computed vessel probabilities from volume rendering based 2D projection and 3D volume can be explored and integrated synergistically. Experimental results are evaluated and compared with the traditional 3D vessel segmentation methods and the state-of-the-art in deep learning, by using extensive public and real patient (micro- )cerebrovascular image datasets. The application of this accurate segmentation and visualization of sparse and complicated 3D microvascular structure facilitated by our method demonstrates the potential in a powerful MR arteriogram and venogram diagnosis of vascular disease.

Automatic generation of font and text design in the wild is a challenging task since font and text in real world exhibit various visual effects. In this paper, we propose a novel model, JointFontGAN, to derive fonts, including both geometric structures and shape contents in correctness and consistency with very few font samples available. Specifically, we design an end-to-end deep learning based approach for font generation through the new multi-stream extended conditional generative adversarial network (XcGAN) models, which jointly learn and generate both font skeleton and glyph representations simultaneously. It can adapt to the geometric variability and content scalability at the neural network level. Then, we apply it, along with the developed efficient and effective one-stage model, to text generations in letters and sentences / paragraphs with both standard and artistic / handwriting styles. The extensive experiments and comparisons demonstrate that our approach outperforms the state-of-the-art methods on the collected datasets including 20K fonts (letters and punctuations) with different styles.

This paper presents a novel surface registration technique using the spectrum of the shapes, which can facilitate accurate localization and visualization of non-isometric deformations of the surfaces. In order to register two surfaces, we map both eigenvalues and eigenvectors of the Laplace-Beltrami of the shapes through optimizing an energy function. The function is defined by the integration of a smoothness term to align the eigenvalues and a distance term between the eigenvectors at feature points to align the eigenvectors. The feature points are generated using the static points of certain eigenvectors of the surfaces. By using both the eigenvalues and the eigenvectors on these feature points, the computational efficiency is improved considerably without losing the accuracy in comparison to the approaches that use the eigenvectors for all vertices. In our technique, the variation of the shape is expressed using a scale function defined at each vertex. Consequently, the total energy function to align the two given surfaces can be defined using the linear interpolation of the scale function derivatives. Through the optimization of the energy function, the scale function can be solved and the alignment is achieved. After the alignment, the eigenvectors can be employed to calculate the point-to-point correspondence of the surfaces. Therefore, the proposed method can accurately define the displacement of the vertices. We evaluate our method by conducting experiments on synthetic and real data using hippocampus, heart, and hand models. We also compare our method with non-rigid Iterative Closest Point (ICP) and a similar spectrum-based methods. These experiments demonstrate the advantages and accuracy of our method.

This paper introduces a deep neural network based method, i.e., DeepOrganNet, to generate and visualize fully high-fidelity 3D / 4D organ geometric models from single-view medical images with complicated background in real time. Traditional 3D / 4D medical image reconstruction requires near hundreds of projections, which cost insufferable computational time and deliver undesirable high imaging / radiation dose to human subjects. Moreover, it always needs further notorious processes to segment or extract the accurate 3D organ models subsequently. The computational time and imaging dose can be reduced by decreasing the number of projections, but the reconstructed image quality is degraded accordingly. To our knowledge, there is no method directly and explicitly reconstructing multiple 3D organ meshes from a single 2D medical grayscale image on the fly. Given single-view 2D medical images, e.g., 3D / 4D-CT projections or X-ray images, our end-to-end DeepOrganNet framework can efficiently and effectively reconstruct 3D / 4D lung models with a variety of geometric shapes by learning the smooth deformation fields from multiple templates based on a trivariate tensor-product deformation technique, leveraging an informative latent descriptor extracted from input 2D images. The proposed method can guarantee to generate high-quality and high-fidelity manifold meshes for 3D / 4D lung models; while, all current deep learning based approaches on the shape reconstruction from a single image cannot. The major contributions of this work are to accurately reconstruct the 3D organ shapes from 2D single-view projection, significantly improve the procedure time to allow on-the-fly visualization, and dramatically reduce the imaging dose for human subjects. Experimental results are evaluated and compared with the traditional reconstruction method and the state-of-the-art in deep learning, by using extensive 3D and 4D examples, including both synthetic phantom and real patient datasets. The efficiency of the proposed method shows that it only needs several milliseconds to generate organ meshes with 10K vertices, which has great potential to be used in real-time image guided radiation therapy (IGRT).

Analyzing the geometric and semantic properties of 3D point clouds through the deep networks is still challenging due to the irregularity and sparsity of samplings of their geometric structures. This paper presents a new method to define and compute convolution directly on 3D point clouds by the proposed annular convolution. This new convolution operator can better capture the local neighborhood geometry of each point by specifying the (regular and dilated) ring-shaped structures and directions in the computation. It can adapt to the geometric variability and scalability at the signal processing level. We apply it to the developed hierarchical neural networks for object classification, part segmentation, and semantic segmentation in large-scale scenes. The extensive experiments and comparisons demonstrate that our approach outperforms the state-of-the-art methods on a variety of standard benchmark datasets (e.g., ModelNet10, ModelNet40, ShapeNet-part, S3DIS, and ScanNet).

Making machines understand human expressions enables various useful applications in human-machine interaction. In this article, we present a novel facial expression recognition approach with 3D Mesh Convolutional Neural Networks (3DMCNN) and a visual analytics-guided 3DMCNN design and optimization scheme. From an RGBD camera, we first reconstruct a 3D face model of a subject with facial expressions and then compute the geometric properties of the surface. Instead of using regular Convolutional Neural Networks (CNNs) to learn intensities of the facial images, we convolve the geometric properties on the surface of the 3D model using 3DMCNN. We design a geodesic distance-based convolution method to overcome the difficulties raised from the irregular sampling of the face surface mesh. We further present interactive visual analytics for the purpose of designing and modifying the networks to analyze the learned features and cluster similar nodes in 3DMCNN. By removing low-activity nodes in the network, the performance of the network is greatly improved. We compare our method with the regular CNN-based method by interactively visualizing each layer of the networks and analyze the effectiveness of our method by studying representative cases. Testing on public datasets, our method achieves a higher recognition accuracy than traditional image-based CNN and other 3D CNNs. The proposed framework, including 3DMCNN and interactive visual analytics of the CNN, can be extended to other applications.

This article presents a new method to compute a self-intersection free high-dimensional Euclidean embedding (SIFHDE 2 ) for surfaces and volumes equipped with an arbitrary Riemannian metric. It is already known that given a high-dimensional (high-d) embedding, one can easily compute an anisotropic Voronoi diagram by back-mapping it to 3D space. We show here how to solve the inverse problem, i.e., given an input metric, compute a smooth intersection-free high-d embedding of the input such that the pullback metric of the embedding matches the input metric. Our numerical solution mechanism matches the deformation gradient of the 3D → higher-d mapping with the given Riemannian metric. We demonstrate the applicability of our method, by using it to construct anisotropic Restricted Voronoi Diagram (RVD) and anisotropic meshing, that are otherwise extremely difficult to compute. In SIFHDE 2 -space constructed by our algorithm, difficult 3D anisotropic computations are replaced with simple Euclidean computations, resulting in an isotropic RVD and its dual mesh on this high-d embedding. Results are compared with the state-of-the-art in anisotropic surface and volume meshings using several examples and evaluation metrics.

This paper presents a novel approach based on spectral geometry to quantify and visualize non-isometric deformations of 3D surfaces by mapping two manifolds. The proposed method can determine multi-scale, non-isometric deformations through the variation of Laplace-Beltrami spectrum of two shapes. Given two triangle meshes, the spectra can be varied from one to another with a scale function defined on each vertex. The variation is expressed as a linear interpolation of eigenvalues of the two shapes. In each iteration step, a quadratic programming problem is constructed, based on our derived spectrum variation theorem and smoothness energy constraint, to compute the spectrum variation. The derivation of the scale function is the solution of such a problem. Therefore, the final scale function can be solved by integral of the derivation from each step, which, in turn, quantitatively describes non-isometric deformations between two shapes. To evaluate the method, we conduct extensive experiments on synthetic and real data. We employ real epilepsy patient imaging data to quantify the shape variation between the left and right hippocampi in epileptic brains. In addition, we use longitudinal Alzheimer data to compare the shape deformation of diseased and healthy hippocampus. In order to show the accuracy and effectiveness of the proposed method, we also compare it with spatial registration-based methods, e.g., non-rigid Iterative Closest Point (ICP) and voxel-based method. These experiments demonstrate the advantages of our method.

This paper presents a registration-free method based on geometry spectrum for mapping two shapes. Our method can quantify and visualize the surface deformation by the variation of Laplace-Beltrami spectrum of the object. In order to examine our method, we employ synthetic data that has non-isometric deformation. We have also applied our method to quantifying the shape variation between the left and right hippocampus in epileptic human brains. The results on both synthetic and real patient data demonstrate the effectiveness and accuracy of our method.

In order to visualize and analyze complex collective data, complicated geometric structure of each data is desired to be mapped onto a canonical domain to enable map-based visual exploration. This paper proposes a novel volume-preserving mapping and registration method which facilitates effective collective data visualization. Given two 3-manifolds with the same topology, there exists a mapping between them to preserve each local volume element. Starting from an initial mapping, a volume restoring diffeomorphic flow is constructed as a compressible flow based on the volume forms at the manifold. Such a flow yields equality of each local volume element between the original manifold and the target at its final state. Furthermore, the salient features can be used to register the manifold to a reference template by an incompressible flow guided by a divergence-free vector field within the manifold. The process can retain the equality of local volume elements while registering the manifold to a template at the same time. An efficient and practical algorithm is also presented to generate a volume-preserving mapping and a salient feature registration on discrete 3D volumes which are represented with tetrahedral meshes embedded in 3D space. This method can be applied to comparative analysis and visualization of volumetric medical imaging data across subjects. We demonstrate an example application in multimodal neuroimaging data analysis and collective data visualization.

This paper presents the practices observed in successful academic projects. It classifies them by the software lifecycle stage they belong to, most of the academic projects are in stage of evolution. It also classifies them by their purpose into the organizational and code development practices. The classification may help academic project managers and developers, who are often specialist in fields other than software engineering, to select the appropriate practices for their project.

Analysis of dynamic object deformations such as cardiac motion is of great importance, especially when there is a necessity to visualize and compare the deformation behavior across subjects. However, there is a lack of effective techniques for comparative visualization and assessment of a collection of motion data due to its 4-dimensional nature, i.e., timely varying three-dimensional shapes. From the geometric point of view, the motion change can be considered as a function defined on the 2D manifold of the surface. This paper presents a novel classification and visualization method based on a medial surface shape space, in which two novel shape descriptors are defined, for discriminating normal and abnormal human heart deformations as well as localizing the abnormal motion regions. In our medial surface shape space, the geodesic distance connecting two points in the space measures the similarity between their corresponding medial surfaces, which can quantify the similarity and disparity of the 3D heart motions. Furthermore, the novel descriptors can effectively localize the inconsistently deforming myopathic regions on the left ventricle. An easy visualization of heart motion sequences on the projected space allows users to distinguish the deformation differences. Our experimental results on both synthetic and real imaging data show that this method can automatically classify the healthy and myopathic subjects and accurately detect myopathic regions on the left ventricle, which outperforms other conventional cardiac diagnostic methods.

High-quality rendering of a complex system usually depends on accurate segmentation of the corresponding objects. In medical imaging data, it is difficult to extract and visualize multiple related objects simultaneously due to the noise, shape variance, and resolution difference of the objects. In this paper we present a viable method to adaptively extract the isosurfaces of different yet informatively related tissues simultaneously from the multimodality imaging data of the brain based on the statistical Partial Differential Equation (PDE) deformable models. Our system and experiments demonstrate the power of using explicit PDE models in extracting and rendering of multiple objects.

A novel segmentation framework for a prepless virtual colonoscopy (VC) is presented, which reduces the necessity for colon cleansing before the CT scan. The patient is injected rectally with a water-soluble iodinated contrast medium using manual insufflators and a small rectal catheter. Compared to the air-based contrast medium, this technique can better preserve the color lumen and reduce the partial volume effect. However, the contrast medium, together with the fecal materials and air, makes colon wall segmentation challenging. Our solution makes no assumptions about the shape, size, and location of the fecal material in the colon. This generality allows us to label the fecal material accurately and extract the colon wall reliably. The accuracy of our technique has been verified on 60 human subjects. Compared with current VC technologies, our method is shown to be better in terms of both sensitivity and specificity. Further, in our experiments, the accuracy of the technique was comparable to that of optical colonoscopy results.

Parameterization of complex surfaces constitutes a major means of visualizing highly convoluted geometric structures as well as other properties associated with the surface. It also enables users with the ability to navigate, orient, and focus on regions of interest within a global view and overcome the occlusions to inner concavities. In this paper, we propose a novel area-preserving surface parameterization method which is rigorous in theory, moderate in computation, yet easily extendable to surfaces of non-disc and closed-boundary topologies. Starting from the distortion induced by an initial parameterization, an area restoring diffeomorphic flow is constructed as a Lie advection of differential 2-forms along the manifold, which yields equality of the area elements between the domain and the original surface at its final state. Existence and uniqueness of result are assured through an analytical derivation. Based upon a triangulated surface representation, we also present an efficient algorithm in line with discrete differential modeling. As an exemplar application, the utilization of this method for the effective visualization of brain cortical imaging modalities is presented. Compared with conformal methods, our method can reveal more subtle surface patterns in a quantitative manner. It, therefore, provides a competitive alternative to the existing parameterization techniques for better surface-based analysis in various scenarios.

Based on the theory of Markov Random Fields, a Bayesian regularization model for diffusion tensor images (DTI) is proposed in this paper. The low-degree parameterization of diffusion tensors in our model makes it less computationally intensive to obtain a maximum a posteriori (MAP) estimation. An approximate solution to the problem is achieved efficiently using hierarchical Markov Chain Monte Carlo (HMCMC), and a loopy belief propagation algorithm is applied to a coarse grid to obtain a good initial solution for hierarchical MCMC. Experiments on synthetic and real data demonstrate the effectiveness of our methods.

Prostate cancer is the second most frequent cause of cancer deaths among men in the US. In the most reliable screening method, histological images from a biopsy are examined under a microscope by pathologists. In an early stage of prostate cancer, only relatively few gland units in a large region become malignant. Discovering such sparse malignant gland units using a microscope is a labor-intensive and error-prone task for pathologists. In this paper, we develop effective image segmentation and classification methods for automatic detection of malignant gland units in microscopic images. Both segmentation and classification methods are based on carefully designed feature descriptors, including color histograms and texton co-occurrence tables.

One of the fundamental goals of computational neuroscience is the study of anatomical features that reflect the functional organization of the brain. The study of physical associations between neuronal structures and the examination of brain activity in vivo have given rise to the concept of anatomical and functional connectivity, which has been invaluable for our understanding of brain mechanisms and their plasticity during development. However, at present, there is no robust and accurate computational framework for the quantitative assessment of cortical connectivity patterns. In this paper, we present a quantitative analysis and modeling tool that is able to characterize anatomical connectivity patterns based on a newly developed coclustering algorithm, termed the business model-based coclustering algorithm (BCA). We apply BCA to diffusion tensor imaging (DTI) data in order to provide an automated and reproducible assessment of the connectivity patterns between different cortical areas in human brains. BCA not only partitions the cortical mantel into well-defined clusters, but also maximizes the connectivity strength between these clusters. Moreover, BCA is computationally robust and allows both outlier detection as well as parameter-independent determination of the number of clusters. Our coclustering results have showed good performance of BCA in identifying major white matter fiber bundles in human brains and facilitate the detection of abnormal connectivity patterns in patients suffering from various neurological diseases.

In this paper, we propose a novel approach of simultaneous localized feature selection and model detection for unsupervised learning. In our approach, local feature saliency, together with other parameters of Gaussian mixtures, are estimated by Bayesian variational learning. Experiments performed on both synthetic and real-world data sets demonstrate that our approach is superior over both global feature selection and subspace clustering methods.

Scientific workflows have recently emerged as a new paradigm for scientists to formalize and structure complex and distributed scientific processes to enable and accelerate many scientific discoveries. In contrast to business workflows, which are typically control flow oriented, scientific workflows tend to be dataflow oriented, introducing a new set of requirements for system development. These requirements demand a new architectural design for scientific workflow management systems (SWFMSs). Although several SWFMSs have been developed that provide much experience for future research and development, a study from an architectural perspective is still missing. The main contributions of this paper are: 1) based on a comprehensive survey of the literature and identification of key requirements for SWFMSs, we propose the first reference architecture for SWFMSs; 2) according to the reference architecture, we further propose a service-oriented architecture for View (a VIsual sciEntific Workflow management system); 3) we implemented View to validate the feasibility of the proposed architectures; and 4) we present a View-based scientific workflow application system (SWFAS), called FiberFlow, to showcase the application of our View system.

With the rapid growth of the World Wide Web and electronic information services,text corpus is becoming available on-line at an incredible rate.By displaying text data in a logical layout (e.g., color graphs),text visualization presents a direct way to observe the documents as well as understand the relationship between them.In this paper, we propose a novel technique, Exemplar-based Visualization (EV), to visualize an extremely large text corpus. Capitalizing on recent advances in matrix approximation and decomposition, EV presents a probabilistic multidimensional projection model in the low-rank text subspace with a sound objective function. The probability of each document proportion to the topics is obtained through iterative optimization and embedded to a low dimensional space using parameter embedding.By selecting the representative exemplars, we obtain a compact approximation of the data. This makes the visualization highly efficient and flexible. In addition, the selected exemplars neatly summarize the entire data set and greatly reduce the cognitive overload in the visualization, leading to an easier interpretation of large text corpus. Empirically, we demonstrate the superior performance of EV through extensive experiments performed on the publicly available text data sets.

This paper formalizes a novel, intrinsic geometric scale space (IGSS) of 3D surface shapes. The intrinsic geometry of a surface is diffused by means of the Ricci flow for the generation of a geometric scale space. We rigorously prove that this multiscale shape representation satisfies the axiomatic causality property. Within the theoretical framework, we further present a feature-based shape representation derived from IGSS processing, which is shown to be theoretically plausible and practically effective. By integrating the concept of scale-dependent saliency into the shape description, this representation is not only highly descriptive of the local structures, but also exhibits several desired characteristics of global shape representations, such as being compact, robust to noise and computationally efficient. We demonstrate the capabilities of our approach through salient geometric feature detection and highly discriminative matching of 3D scans.

White matter tractography can generate a 3D macroscopic view of the white matter connections in the brain by measuring the diffusion of cellular fluid in tissue and visualize the connections with line strips, tubes, etc. In the past, tractography techniques based on deterministic methodologies have been extensively used. Deterministic tracking follows the primary eigen direction of diffusion that limits its clinical applicability to areas where anisotropy is not clear. More recently, probabilistic tractography methods have been emerging as new tools for extracting fiber tracts. This paper presents a new probability-based framework for quantitative, inter-subject analysis of diffusion tensor imaging data. Advantages of the probabilistic tractography are in its inherent ability to model noise and uncertainty in its global estimation model. This facilitates more accurate tracking in complex neighborhoods, such as branching and crossing fibers. With a combination of statistical measures, our approach can pinpoint abnormalities through the comparison between a set of normal population and patient data. The experiments demonstrate its excellent performance in identifying imaging symptoms of epilepsy and Tourette Syndrome.

This paper presents a novel computational framework based on dynamic spherical volumetric simplex splines for simulation of genuszero real-world objects. In this framework, we first develop an accurate and efficient algorithm to reconstruct the high-fidelity digital model of a real-world object with spherical volumetric simplex splines which can represent with accuracy geometric, material, and other properties of the object simultaneously. With the tight coupling of Lagrangian mechanics, the dynamic volumetric simplex splines representing the object can accurately simulate its physical behavior because it can unify the geometric and material properties in the simulation. The visualization can be directly computed from the object's geometric or physical representation based on the dynamic spherical volumetric simplex splines during simulation without interpolation or resampling. We have applied the framework for biomechanic simulation of brain deformations, such as brain shifting during the surgery and brain injury under blunt impact. We have compared our simulation results with the ground truth obtained through intra-operative magnetic resonance imaging and the real biomechanic experiments. The evaluations demonstrate the excellent performance of our new technique presented in this paper.

With the explosive growth of Web and the recent development in digital media technology, the number of images on the Web has grown tremendously. Consequently, Web image clustering has emerged as an important application. Some of the initial efforts along this direction revolved around clustering Web images based on the visual features of images or textual features by making use of the text surrounding the images. However, not much work has been done in using multimodal information for clustering Web images. In this paper, we propose a graph theoretical framework for simultaneously integrating visual and textual features for efficient Web image clustering. Specifically, we model visual features, images and words from surrounding text using a tripartite graph. Partitioning this graph leads to clustering of the Web images. Although, graph partitioning approach has been adopted before, the main contribution of this work lies in a new algorithm that we propose- Consistent Isoperimetric High-order Co-clustering (CIHC), for partitioning the tripartite graph. Computationally, CIHC is very quick as it requires a simple solution to a sparse system of linear equations. Our theoretical analysis and extensive experiments performed on real Web images demonstrate the performance of CIHC in terms of the quality, efficiency and scalability in partitioning the visual feature-image-word tripartite graph.

Abstract This paper develops a new salient keypoints‐based shape description which extracts the salient surface keypoints with detected scales. Salient geometric features can then be defined collectively on all the detected scale normalized local patches to form a shape descriptor for surface matching purpose. The saliency‐driven keypoints are computed as local extrema of the difference of Gaussian function defined over a curved surface in geodesic scale space. This method can properly function on either manifold or non‐manifold surface without resorting to any surface mapping or parameterization procedures. Therefore, it has a wide utility in many applications such as shape matching, classification, and recognition. Our experiments on 3D shapes demonstrate that the salient keypoints and local feature descriptors are robust and stable to noisy input and insensitive to resolution change. We have applied our technique to the tasks of 3D shape matching, and the experimental results showed good performance and the effectiveness of this new method. Copyright © 2008 John Wiley & Sons, Ltd.

This paper presents a novel and efficient surface matching and visualization framework through the geodesic distance-weighted shape vector image diffusion. Based on conformal geometry, our approach can uniquely map a 3D surface to a canonical rectangular domain and encode the shape characteristics (e.g., mean curvatures and conformal factors) of the surface in the 2D domain to construct a geodesic distance-weighted shape vector image, where the distances between sampling pixels are not uniform but the actual geodesic distances on the manifold. Through the novel geodesic distance-weighted shape vector image diffusion presented in this paper, we can create a multiscale diffusion space, in which the cross-scale extrema can be detected as the robust geometric features for the matching and registration of surfaces. Therefore, statistical analysis and visualization of surface properties across subjects become readily available. The experiments on scanned surface models show that our method is very robust for feature extraction and surface matching even under noise and resolution change. We have also applied the framework on the real 3D human neocortical surfaces, and demonstrated the excellent performance of our approach in statistical analysis and integrated visualization of the multimodality volumetric data over the shape vector image.

Document clustering without any prior knowledge or background information is a challenging problem. In this paper, we propose SS-NMF: a semi-supervised non- negative matrix factorization framework for document clustering. In SS-NMF, users are able to provide supervision for document clustering in terms of pairwise constraints on a few documents specifying whether they "must" or "cannot" be clustered together. Through an iterative algorithm, we perform symmetric tri-factorization of the document- document similarity matrix to infer the document clusters. Theoretically, we show that SS-NMF provides a general framework for semi-supervised clustering and that existing approaches can be considered as special cases of SS-NMF. Through extensive experiments conducted on publicly available data sets, we demonstrate the superior performance of SS-NMF for clustering documents.

In clustering, global feature selection algorithms attempt to select a common feature subset that is relevant for all clusters. Consequently, they are not able to identify individual clusters that exist in different feature subspaces. In this paper, we propose a localized feature selection algorithm for clustering. The proposed algorithm computes adjusted and normalized scatter separability for individual clusters. A sequential backward search is then applied to find the optimal (maybe local) feature subsets for each cluster. Experiment results on both synthetic data clustering and content-based image grouping show the need for feature selection in clustering and the benefits of selecting features locally.

Web image clustering has drawn significant attention in the research community recently. However, not much work has been done in using multi-modal information for clustering Web images. In this paper, we address the problem of Web image clustering by simultaneous integration of visual and textual features from a graph partitioning perspective. In particular, we modelled visual features, images, and words from the surrounding text of the images using a tripartite graph. This graph is actually considered as a fusion of two bipartite graphs that are partitioned simultaneously by the proposed Consistent Isoperimetric High-order Co-clustering(CIHC) framework. Although a similar approach has been adopted before, the main contribution of this work lies in the computational efficiency, quality in Web image clustering and scalability to large image repositories that CIHC is able to achieve. We demonstrate this through experimental results performed on real Web images.

This paper presents a novel integrative information visualization framework for cross-subject neuroimaging data analysis. The framework can integrate multimodal information captured by different imaging modalities and population-based statistical information presented by different subjects. In this framework, accurate registration of cortical structures is the foundation for the information integration across population. We present a non-rigid intersubject brain surface registration method using conformal structure and spherical thin-plate splines. Spherical thin-plate splines are designed to explicitly match prominent homologous landmarks, and meanwhile, interpolate a global deformation field on the spherical domain, registering brain surfaces in a transformed space. Subsequently, an approach for the integrative information fusion and visualization is presented to handle multimodality neuroimaging data. The entire framework demonstrates its usefulness in multimodality neuroimaging data analysis across subjects.

In region-based image annotation, keywords are usually associated with images instead of individual regions in the training data set. This poses a major challenge for any learning strategy. In this paper, we formulate image annotation as a supervised learning problem under Multiple-Instance Learning (MIL) framework. We present a novel Asymmetrical Support Vector Machine-based MIL algorithm (ASVM-MIL), which extends the conventional Support Vector Machine (SVM) to the MIL setting by introducing asymmetrical loss functions for false positives and false negatives. The proposed ASVM-MIL algorithm is evaluated on both image annotation data sets and the benchmark MUSK data sets.

Recent advances in imaging technologies, such as magnetic resonance imaging (MRI), positron emission tomography (PET) and diffusion tensor imaging (DTI) have accelerated brain research in many aspects. In order to better understand the synergy of the many processes involved in normal brain function, integrated modeling and analysis of MRI, PET, and DTI across subjects is highly desirable. The current state-of-art computational tools fall short in offering an analytic approach for intersubject brain registration and analysis. In this paper we present an approach which is based on landmark constrained conformal parameterization of a brain surface from high-resolution structural MRI data to a canonical spherical domain. This model allows natural integration of information from co-registered PET as well as DTI data and lays a foundation for the quantitative analysis of the relationship among diverse datasets across subjects. Consequently, the approach can be extended to provide a software environment able to facilitate detection of abnormal functional brain patterns in patients with neurological disorder.

This paper presents a novel integrative neural network, which contains a feedback loop for adaptive control of learning. Instead of designing a single classifier for the classification task, a finite number of classifiers are simultaneously applied and all outputs from the individual classifiers are processed by the integrative neural network. The stability conditions and supervised learning algorithms are derived for the artificial neural network. We have applied it to unconstrained handwritten numbers recognition. Experiments are carried out and the results are compared to that of multi-layer perceptron. They show that the proposed integrative neural network with feedback control has a better classification rate with no decrease of reliability. This type of neural network scheme provides an alternative approach for ensemble learning.

This paper presents a flexible, scalable framework for interactive hands-on shape design in a haptic, stereoscopic virtual environment. The framework is founded upon the concept of PDE-based geometric surface flow. Given an input polygonal mesh, a user can interactively define implicit functions around regions of interest of the mesh model, and the locally or globally affected regions of the model will automatically deform according to the underlying partial differential equations and reconstruct the implicitly defined shape. During the model deformation process, the model can always maintain its regularity and can properly modify its topology when collisions between different parts of the model occur. With augmented haptics functionality and stereoscopic display, our system provides a more intuitive interface, which allows users to directly manipulate 3D polygonal objects with hands.

We articulate a new modeling paradigm for both local and global editing on complicated point set surfaces of arbitrary topology. In essence, the proposed technique leads to a novel point-set methodology that can unify the topological advantage of the level-set methods and the simplicity of point-sampled surfaces. Any user-specified region of a point set surface in our system can be embedded into a grid-based level-set framework. The super-imposed grid structure enables both powerful local surface editing and global scalar-field free-form deformation anywhere across the point-sampled geometry. Furthermore, the underlying level-set representation, coupled with the concept of digital topology, greatly facilitates the topological modification of the sculpted point-set geometry whenever necessary during shape deformation. We have developed a variety of editing toolkits that can allow users to directly manipulate the point-set surface through interactive sketching, smoothing, embossing, and global free-form deformations with ease. We demonstrate the usefulness and efficacy of our prototype system for the point-sampled geometry via many examples.

Abstract This paper presents HapticFlow, a haptics‐based direct mesh editing system founded upon the concept of PDE‐based geometric surface flow. The proposed flow‐based approach for direct geometric manipulation offers a unified design paradigm that can seamlessly integrate implicit, distance‐field based shape modeling with dynamic, physics‐based shape design. HapticFlow provides an intuitive haptic interface and allows users to directly manipulate 3D polygonal objects with ease. To demonstrate the effectiveness of our new approach, we developed a variety of haptics‐based mesh editing operations such as embossing, engraving, sketching as well as force‐based shape manipulation operations. Copyright © 2004 John Wiley & Sons, Ltd.

This paper systematically presents a novel, interactive solid modeling framework, Haptics-based Dynamic Implicit Solid Modeling, which is founded upon volumetric implicit functions and powerful physics-based modeling. In particular, we augment our modeling framework with a haptic mechanism in order to take advantage of additional realism associated with a 3D haptic interface. Our dynamic implicit solids are semi-algebraic sets of volumetric implicit functions and are governed by the principles of dynamics, hence responding to sculpting forces in a natural and predictable manner. In order to directly manipulate existing volumetric data sets as well as point clouds, we develop a hierarchical fitting algorithm to reconstruct and represent discrete data sets using our continuous implicit functions, which permit users to further design and edit those existing 3D models in real-time using a large variety of haptic and geometric toolkits, and visualize their interactive deformation at arbitrary resolution. The additional geometric and physical constraints afford more sophisticated control of the dynamic implicit solids. The versatility of our dynamic implicit modeling enables the user to easily modify both the geometry and the topology of modeled objects, while the inherent physical properties can offer an intuitive haptic interface for direct manipulation with force feedback.

This paper addresses the problem of surface reconstruction of highly noisy point clouds. The surfaces to be reconstructed are assumed to be 2-manifolds of piecewise C/sup 1/ continuity, with isolated small irregular regions of high curvature, sophisticated local topology or abrupt burst of noise. At each sample point, a quadric field is locally fitted via a modified moving least squares method. These locally fitted quadric fields are then blended together to produce a pseudo-signed distance field using Shepard's method. We introduce a prioritized front growing scheme in the process of local quadrics fitting. Flatter surface areas tend to grow faster. The already fitted regions will subsequently guide the fitting of those irregular regions in their neighborhood.

We present a novel, interactive shape modeling technique: dynamic implicit solid modeling, which unifies volumetric implicit functions and powerful physics-based modeling. Although implicit functions are extremely powerful in graphics, geometric design, and shape modeling, the full potential of implicit functions is yet to be fully realized due to the lack of flexible and interactive design techniques. In order to broaden the accessibility of implicit functions in geometric modeling, we marry the implicit solids, which are semi-algebraic sets of volumetric implicit functions, with the principle of physics-based models and formulate dynamic implicit solids. By using "density springs" to connect the scalar values of implicit functions, we offer a viable solution to introduce the elasticity into implicit representations. As a result, our dynamic implicit solids respond to sculpting forces in a natural and predictive manner. The geometric and physical behaviors are tightly coupled in our modeling system. The flexibility of our modeling technique allows users to easily modify the geometry and topology of sculpted objects, while the inherent physical properties can provide a natural interface for direct, force-based free-form deformation. The additional constraints provide users more control on the dynamic implicit solids. We have developed a sculpting system equipped with a large variety of physics-based toolkits and an intuitive haptic interface to facilitate the direct, natural editing of implicit functions in real-time. Our experiments demonstrate many attractive advantages of our dynamic approach for implicit modeling such as intuitive control, direct manipulation, real-time haptic feedback, and capability to model complicated geometry and arbitrary topology.

Implicit functions characterized by the zero-set of polynomial-based algebraic equations and other commonly-used analytic equations are extremely powerful in graphics, geometric design, and visualization. But the potential of implicit functions is yet to be fully realized due to the lack of flexible and interactive design techniques. The paper presents a haptic sculpting system founded upon scalar trivariate B-spline functions. All the solids sculpted in our environment are semi-algebraic sets of volumetric implicit functions. We develop a large variety of sculpting toolkits equipped with an intuitive haptic interface to facilitate the direct manipulation of implicit functions in real-time. To facilitate multiresolution editing and different levels of details, we employ three techniques: hierarchical B-splines, CSG-based functional composition, and knot insertion. Our experiments demonstrate that our algorithms and haptics-based techniques can greatly overcome the modeling difficulties associated with implicit functions. The novel modeling techniques and their haptics-based design principle are extensible to the design of arbitrary implicit functions.

We have designed an integrated pattern recognition system. Instead of designing a classifier for pattern recognition, a finite number of classifiers are simultaneously applied, and a multilayer artificial neural network with feedback is employed to process all the outputs of the individuals in order to obtain a more accurate classification rate. Because of the introduction of the feedback loop, the pattern recognition system becomes a nonlinear dynamic system rather than a nonlinear mapping. We obtain a sufficient condition on the absolute stability for the integrated network and derive a corresponding learning algorithm to ensure its stability. The system has been applied to totally unconstrained handwritten numeral recognition, and its performance is excellent!.