An efficient and precise diagnosis of retinal diseases is a fundamental goal for auxiliary diagnostic systems in ophthalmology. Inspired by the importance of scattered subtle lesions in manual retinal disease diagnosis, recent research has achieved state-of-the-art performance by mining information related to subtle lesions, including their texture and shape. However, the spatial distribution patterns of subtle lesion areas, which are also crucial in manual diagnosis, have been overlooked in existing research. Neglecting these spatial distribution patterns (e.g., the ring distribution of microaneurysms in diabetic macular edema) may negatively impact the diagnostic process. In this paper, we introduce the Saliency-Image-Graph (SIGraph) network to capture the spatial distribution patterns of lesion areas. We first employ saliency-based perception to identify latent lesion pixels. Subsequently, we propose a novel image-graph block to efficiently capture the global distribution of abundant lesion pixels with minimal information loss. By leveraging additional distribution patterns, SIGraph achieves state-of-the-art performance with at least a 1.5% performance gain across three datasets. Furthermore, ablation studies demonstrate that our image-graph block can be integrated into other visual backbones and effectively boost performance.

Orthodontic path planning is crucial for precise treatment, yet AI -generated trajectories remain inferior to expert designs due to difficulties in perceiving large dental point clouds and modeling multi-step tooth displacements. In this paper, we introduce Efficient ToothAR, a cascade autoregressive framework that decomposes orthodontic planning into tooth perception and transition prediction stages for improved efficiency and accuracy. In the perception stage, we adapt the efficient sequence model Mamba with a tooth-by-tooth scan and hierarchical architecture to encode dental point clouds. In the transition stage, a bidirectional transformer predicts sequential tooth movements autore-gressively from both initial and target configurations, effectively reducing accumulated transition errors. Trained on 5,000 clinical alignment trajectories, Efficient ToothAR achieves state-of-the-art performance across multiple metrics (e.g., displacement error, collision rate) while maintaining high computational efficiency.

We explore Generalizable Tumor Segmentation, aiming to train a single model for zero-shot tumor segmentation across diverse anatomical regions. Existing methods face limitations related to segmentation quality, scalability, and the range of applicable imaging modalities. In this paper, we uncover the potential of the internal representations within frozen medical foundation diffusion models as highly efficient zero-shot learners for tumor segmentation by introducing a novel framework named DiffuGTS. DiffuGTS creates anomaly-aware open-vocabulary attention maps based on text prompts to enable generalizable anomaly segmentation without being restricted by a predefined training category list. To further improve and refine anomaly segmentation masks, DiffuGTS leverages the diffusion model, transforming pathological regions into high-quality pseudo-healthy counterparts through latent space inpainting, and applies a novel pixel-level and feature-level residual learning approach, resulting in segmentation masks with significantly enhanced quality and generalization. Comprehensive experiments on four datasets and seven tumor categories demonstrate the superior performance of our method, surpassing current state-of-the-art models across multiple zero-shot settings. Codes are available at https://github.com/Yankai96/DiffuGTS.

Volume rendering is a costly but widely used technique in scientific visualization. Various methods were proposed to accelerate volume rendering, most of which make use of empty brick skipping to avoid sampling the emtpy space of volumetric data, whereas the empty voxels in the valid bricks are still treated as valid. We introduce Depth-Box VDB, a simple but effective method based on GPU-based voxel database (GVDB) to accelerate volume rendering by further reducing the number of samplings. With the help of depth maps attached to the valid bricks and the displacement mapping technique, our method can skip part of the empty voxels in the valid bricks and has achieved significant improvement in the performance of volume rendering. According to the performance tests on different datasets, the rendering efficiency increases by up to 121.74%, while the use of memory increases by around 18.67%. Comparing with different shading models, we can conclude that our method tends to speed up more as the rendering requires more shading computing.

Relationships in scientific data, such as the numerical and spatial distribution relations of features in univariate data, the scalar-value combinations' relations in multivariate data, and the association of volumes in time-varying and ensemble data, are intricate and complex. This paper presents voxel2vec, a novel unsupervised representation learning model, which is used to learn distributed representations of scalar values/scalar-value combinations in a low-dimensional vector space. Its basic assumption is that if two scalar values/scalar-value combinations have similar contexts, they usually have high similarity in terms of features. By representing scalar values/scalar-value combinations as symbols, voxel2vec learns the similarity between them in the context of spatial distribution and then allows us to explore the overall association between volumes by transfer prediction. We demonstrate the usefulness and effectiveness of voxel2vec by comparing it with the isosurface similarity map of univariate data and applying the learned distributed representations to feature classification for multivariate data and to association analysis for time-varying and ensemble data.

Automatic and accurate retinal disease diagnosis is critical to guide proper therapy and prevent potential vision loss. Previous works simply exploit the most discriminative features while ignoring the pathological visual clues of scattered subtle lesions. Therefore, without a comprehensive understanding of features from different lesion regions, they are vulnerable to noise from complex backgrounds and suffer from misclassification failures. In this paper, we address these limitations with a novel saliency-guided abnormality-aware transformer which explicitly captures the correlation between different lesion features from a global perspective with enhanced pathological semantics. The model has several merits. First, we propose a saliency enhancement module (SEM) which adaptively integrates disease related semantics and highlights potentially salient lesion regions. Second, to the best of our knowledge, this is the first work to explore comprehensive lesion feature dependencies via a tailored efficient self-attention. Third, with the saliency enhancement module and abnormality-aware attention, we propose a new variant of Vision Transformer models, called SatFormer, which outperforms the state-of-the-art methods on two public retinal disease classification benchmarks. Ablation study shows that the proposed components can be easily embedded into any Vision Transformers via a plug-and-play manner and effectively boost the performance.

Abstract Graph transformer has achieved remarkable success in graph‐based segmentation tasks. Inspired by this success, we propose a novel method named MeshFormer for applying the graph transformer to the semantic segmentation of high‐resolution meshes. The main challenges are the large data size, the massive model size, and the insufficient extraction of high‐resolution semantic meanings. The large data or model size necessitates unacceptably extensive computational resources, and the insufficient semantic meanings lead to inaccurate segmentation results. MeshFormer addresses these three challenges with three components. First, a boundary‐preserving simplification is introduced to reduce the data size while maintaining the critical high‐resolution information in segmentation boundaries. Second, a Ricci flow‐based clustering algorithm is presented for constructing hierarchical structures of meshes, replacing many convolutions layers for global support with only a few convolutions in hierarchy structures. In this way, the model size can be reduced to an acceptable range. Third, we design a graph transformer with cross‐resolution convolutions, which extracts richer high‐resolution semantic meanings and improves segmentation results over previous methods. Experiments show that MeshFormer achieves gains from 1.0% to 5.8% on artificial and real‐world datasets.

The surface—volume—surface electric field integral equation (SVS-EFIE) can lead to complex equations, laborious implementation, and unacceptable computational complexity in the method of moments (MoM). Therefore, a general matrix equation (GME) is proposed for electromagnetic scattering from arbitrary metal—dielectric composite objects, and its enhanced solution is presented in this paper. In previous works, MoM solution formulation of SVS—EFIE considering only three-region metal—dielectric composite scatters was presented, and the two-stage process resulted in two integral operators in SVS-EFIE, which is arduous to implement and is incapable of reducing computational complexity. To address these difficulties, GME, which is versatile for homogeneous objects and composite objects consisting of more than three sub-regions, is proposed for the first time. Accelerated solving policies are proposed for GME based on coupling degree concerning the spacing between sub-regions, and the coupling degree standard can be adaptively set to balance the accuracy and efficiency. In this paper, the reformed addition theorem is applied for the strong coupling case, and the iterative method is presented for the weak coupling case. Parallelism can be easily applied in the enhanced solution. Numerical results demonstrate that the proposed method requires only 11.6% memory and 11.8% CPU time on average compared to the previous direct solution.

Detecting neurofibrillary tangles is an important procedure in the assessment of the intensity and distribution pattern of hippocampal tau pathology, which are the principal clinical phenotypes associated with Alzheimer’s disease. Existing deep learning based detectors still face a critical obstacle: the difficulty in detecting extremely small objects in high resolution images. In this paper, we propose a deep learning framework, named DeepNFT, which combines the multilevel feature aggregation pyramid network (MFAPN) and the adversarial feature generation module (AFGM) to acquire precise detection results with significantly reduced false positives. To prove its universality and robustness, DeepNFT has been validated on two datasets. Experiments show the significant performance gain of our proposed approach over state-of-the-art detectors. Ablation study shows our network components improve the performance of various backbones and detectors.

Accurate and reliable segmentation of colorectal tumors and surrounding colorectal tissues on 3D magnetic resonance images has critical importance in preoperative prediction, staging, and radiotherapy. Previous works simply combine multilevel features without aggregating representative semantic information and without compensating for the loss of spatial information caused by down-sampling. Therefore, they are vulnerable to noise from complex backgrounds and suffer from misclassification and target incompleteness-related failures. In this paper, we address these limitations with a novel adaptive lesion-aware attention network (ALA-Net) which explicitly integrates useful contextual information with spatial details and captures richer feature dependencies based on 3D attention mechanisms. The model comprises two parallel encoding paths. One of these is designed to explore global contextual features and enlarge the receptive field using a recurrent strategy. The other captures sharper object boundaries and the details of small objects that are lost in repeated down-sampling layers. Our lesion-aware attention module adaptively captures long-range semantic dependencies and highlights the most discriminative features, improving semantic consistency and completeness. Furthermore, we introduce a prediction aggregation module to combine multiscale feature maps and to further filter out irrelevant information for precise voxel-wise prediction. Experimental results show that ALA-Net outperforms state-of-the-art methods and inherently generalizes well to other 3D medical images segmentation tasks, providing multiple benefits in terms of target completeness, reduction of false positives, and accurate detection of ambiguous lesion regions.

Graph embeddings represent nodes as low-dimensional vectors to preserve the proximity between nodes and communities of graphs for network analysis. The temporal edges (e.g., relationships, contacts, and emails) in dynamic graphs are important for graph evolution analysis, but few existing methods in graph embeddings can capture the dynamic information from temporal edges. In this study, we propose a dynamic graph embedding method to analyze the evolution patterns of dynamic graphs effectively. Our method uses diffuse context sampling to preserve the proximity between nodes, and applies dynamic context graph embeddings to train discrete-time graph embeddings in the same vector space without alignments to preserve the temporal continuity of stable nodes. We compare our method with several state-of-the-art methods for link prediction, and the experiments demonstrate that our method generally performs better at the task. Our method is further verified using a real-world dynamic graph by visualizing the evolution of its community structure at different timesteps.

Dynamic networks evolve with their structures changing over time. It is still a challenging problem to efficiently explore the evolution of dynamic networks in terms of both their structural and temporal properties. In this paper, we propose a visual analytics methodology to interactively explore the temporal evolution of dynamic networks in the context of their structure. A novel diachronic node embedding method is first proposed to learn latent representations of the structural and temporal features of nodes in a vector space. Diachronic node embeddings are then used to discover communities with similar structural proximity and temporal evolution patterns. A visual analytics system is designed to enable users to visually explore the evolutions of nodes, communities, and the network as a whole in terms of their structural and temporal properties. We evaluate the effectiveness of our method using artificial and real-world dynamic networks and comparisons with previous methods.

Text labeling for classification is a time-consuming and unintuitive process. Given an unannotated text collection, it is difficult for users to determine what label to create and how to label the initial training set for classification. Thus, we present an interactive visual analytics system for incremental text classification based on a semi-supervised topic modeling method, modified Gibbs sampling maximum entropy discrimination latent Dirichlet allocation (Gibbs MedLDA). Given a text collection, Gibbs MedLDA generates topics as a summary of the text collection. We design a scatter plot to display documents and topics simultaneously to show the topic information, and this helps users explore the text collection structurally and find labels for creating. After labeling documents, Gibbs MedLDA is applied to the text collection with labels again, and it generates both the topic and classification information. We also provide a scatter plot with the classifier boundary and a matrix view to present weights of classifiers. Users can iteratively label documents to refine each classifier. We evaluate our system via a user study with a benchmark corpus for text classification and case studies with two unannotated text collections.

Understanding the bike-sharing system and traditional taxi system as well as their similarities and differences are essential for bike-sharing rebalancing, taxi dispatching, and urban planning. However, due to the sparseness of record data and the difference in service regions, the relationship between them is indeed obscure, and previous solutions mostly focus only on a single system. In this paper, we propose a visual analytics system to investigate the similarities and differences between bike-sharing and taxi systems. The service region for each bike station is created to fuse bike-sharing data and taxi data. We harness two 3-order tensors to represent them in a unified framework to generate potential patterns by tensor decomposition. The visual analytics system integrates two spatiotemporal data sources by analyzing the patterns that are typical of each data source and the patterns that are common to both data sources to assist users in better discovering the relationships between the taxi system and the bike-sharing system. We demonstrate the effectiveness of our system through real-world case studies.

GPS-based taxi trajectories contain valuable knowledge about movement patterns for transportation and urban planning. Topic modeling is an effective tool to extract semantic information from taxi trajectory data. However, previous methods generally ignore trajectory directions that are important in the analysis of movement patterns. In this paper, we employ the bigram topic model rather than traditional topic models to analyze textualized trajectories and consider the direction information of trajectories. We further propose a modified Apriori algorithm to extract topical sub-trajectories and use them to represent each topic. Finally, we design a visual analytics system with several linked views to facilitate users to interactively explore movement patterns from topics and topical sub-trajectories. The case studies with Chengdu taxi trajectory data demonstrate the effectiveness of the proposed system.

In complex multivariate data sets, different features usually include diverse associations with different variables, and different variables are associated within different regions. Therefore, exploring the associations between variables and voxels locally becomes necessary to better understand the underlying phenomena. In this paper, we propose a co-analysis framework based on biclusters, which are two subsets of variables and voxels with close scalar-value relationships, to guide the process of visually exploring multivariate data. We first automatically extract all meaningful biclusters, each of which only contains voxels with a similar scalar-value pattern over a subset of variables. These biclusters are organized according to their variable sets, and biclusters in each variable set are further grouped by a similarity metric to reduce redundancy and support diversity during visual exploration. Biclusters are visually represented in coordinated views to facilitate interactive exploration of multivariate data from the similarity between biclusters and the correlation of scalar values with different variables. Experiments on several representative multivariate scientific data sets demonstrate the effectiveness of our framework in exploring local relationships among variables, biclusters and scalar values in the data.

A visual summary of the controversial aspects of an item enables both customers and marketers to identify and address complaints and concerns about the item effectively. In this paper, we propose a novel visual analytics system, to visually explore when a controversy occurs and the causes behind the controversy via user-generated reviews with text and ratings in various domains, such as restaurants, home goods, and cultural products. Quantitative analysis of the ratings of an item is first applied to characterize the evolution of controversy over time. A novel aspect-extraction method based on hierarchical clustering is proposed to identify aspect-level reasons garnered from review texts that explain why a controversy occurs. Our system allows the user to interactively explore the time-evolving controversy trend, major aspects of reviews, and sentiment divergences of aspects to understand in depth the controversy in reviews. We evaluate the effectiveness of the proposed aspect-extraction method by means of accuracy of aspect identification, the usefulness of our system using three case studies in different domains, and a user study.

Volume upscaling generates high-resolution volumes from low-resolution volumes to make data exploration more effective. Traditional methods, such as the simple trilinear or cubic-spline interpolation, may blur boundaries of features and lead to jagged artifacts. Inspired by recent progress in image super-resolution with Convolutional Neural Networks (CNN), we propose a CNN-based volume upscaling method. Our CNN contains three hidden layers: block extraction and representation, non-linear mapping, and reconstruction. It directly learns an end-to-end mapping from low-resolution blocks to high-resolution volume. Compared to previous methods, our CNN can preserve better structures and details of features, and provide a better volume quality in both the visualization and evaluation metrics.

Word clouds have been widely used to present the contents and themes in the text for summary and visualization. In this paper, we propose a new semantic word cloud taking into account the word semantic meanings. Distributed word representation is applied to accurately describe the semantic meaning of words, and a word similarity graph is constructed based on the semantic distance between words to lay out words in a more compact and aesthetic manner. Word-related interactions are introduced to guide users fast read and understand the text. We apply the proposed word cloud to user generated reviews in different fields to demonstrate the effectiveness of our method.

Abstract Previous viewpoint selection methods in volume visualization are generally based on some deterministic measures of viewpoint quality. However, they may not express the familiarity and aesthetic sense of users for features of interest. In this paper, we propose an image‐based viewpoint selection model to learn how visualization experts choose representative viewpoints for volumes with similar features. For a given volume, we first collect images with similar features, and these images reflect the viewpoint preferences of the experts when visualizing these features. Each collected image tallies votes to the viewpoints with the best matching based on an image similarity measure, which evaluates the spatial shape and appearance similarity between the collected image and the rendered image from the viewpoint. The optimal viewpoint is the one with the most votes from the collected images, that is, the viewpoint chosen by most visualization experts for similar features. We performed experiments on various volumes available in volume visualization, and made comparisons with traditional viewpoint selection methods. The results demonstrate that our model can select more canonical viewpoints, which are consistent with human perception.

Abstract In volume visualization, noise in regions of homogeneous material and at boundaries between different materials poses a great challenge in extracting, analyzing and rendering features of interest. In this paper, we present a novel volume denoising / smoothing method based on the L0 gradient minimization framework. This framework globally controls how many voxels with a non‐zero gradient are in the result in order to approximate important features’ structures in a sparse way. This procedure can be solved quickly by the alternating optimization strategy with half‐quadratic splitting. While the proposed L0 volume gradient minimization method can effectively remove noise in homogeneous materials, a blurring‐sharpening strategy is proposed to diminish noise or smooth local details on the boundaries. This generates salient features with smooth boundaries and visually pleasing structures. We compare our method with the bilateral filter and anisotropic diffusion, and demonstrate the effectiveness and efficiency of our method with several volumes in different modalities.

This paper proposes an interactive visualization tool, Visitpedia, to detect and analyze social events based on Wikipedia visit history. It helps users discover real-world events behind the data and study how these events evolve over time. Different from previous work based on on-line news or similar text corpora, we choose Wikipedia visit counts as our data source since the visit count data better reflect user concerns of social events. We tackle the event-based task from a time-series pattern perspective rather than semantic perspective. Various visualization and user interaction techniques are integrated in Visitpedia. Two case studies are conducted to demonstrate the effectiveness of Visitpedia.

We present a saliency-aware volume resizing operation called surface carving, which intelligently removes contextual voxels while preserving important features. By iteratively applying surface carving in all directions, we can create a volume of the desired size. For large volume data sets, a multilevel banded method is introduced to gracefully overcome the memory limit and speed up volume resizing. We compare our technique with traditionally cropping and scaling approaches and demonstrate the effectiveness and efficiency of our method with several volume data sets.

Volume up scaling enlarges the size of a volume to make feature analysis more accurate and efficient. Linear interpolation, widely used in volume up scaling, result in jagged artifacts around features and losses of high-frequency components. Based on the example-based up scaling framework, this paper presents a new high-quality volume up scaling technique, predicting the high-frequency components by searching for the best matched patch in the input volume. As each slice can be taken as an image, the existing image up scaling technique based on the local self-similarity assumption can be directly applied to achieve slice up scaling. We further validate that the local self-similarity assumption is still valid for 3D volumes, and we extend this technique to 3D volume up scaling, i.e., isotropic volume up scaling. We compare our volume up scaling technique with traditional linear and cubic spline interpolations, and demonstrate that our method can generate a higher quality volume with better shape and details preserved. The proposed volume up scaling technique is well suitable for legacy low-resolution volumes to improve their visual qualities in visualization and analysis.

Blind motion deblurring estimates a sharp image from a motion blurred image without the knowledge of the blur kernel. Although significant progress has been made on tackling this problem, existing methods, when applied to highly diverse natural images, are still far from stable. This paper focuses on the robustness of blind motion deblurring methods toward image diversity-a critical problem that has been previously neglected for years. We classify the existing methods into two schemes and analyze their robustness using an image set consisting of 1.2 million natural images. The first scheme is edge-specific, as it relies on the detection and prediction of large-scale step edges. This scheme is sensitive to the diversity of the image edges in natural images. The second scheme is nonedge-specific and explores various image statistics, such as the prior distributions. This scheme is sensitive to statistical variation over different images. Based on the analysis, we address the robustness by proposing a novel nonedge-specific adaptive scheme (NEAS), which features a new prior that is adaptive to the variety of textures in natural images. By comparing the performance of NEAS against the existing methods on a very large image set, we demonstrate its advance beyond the state-of-the-art.

Lighting design is a complex, but fundamental, problem in many fields. In volume visualization, direct volume rendering generates an informative image without external lighting, as each voxel itself emits radiance. However, external lighting further improves the shape and detail perception of features, and it also determines the effectiveness of the communication of feature information. The human visual system is highly effective in extracting structural information from images, and to assist it further, this paper presents an approach to structure-aware automatic lighting design by measuring the structural changes between the images with and without external lighting. Given a transfer function and a viewpoint, the optimal lighting parameters are those that provide the greatest enhancement to structural information - the shape and detail information of features are conveyed most clearly by the optimal lighting parameters. Besides lighting goodness, the proposed metric can also be used to evaluate lighting similarity and stability between two sets of lighting parameters. Lighting similarity can be used to optimize the selection of multiple light sources so that different light sources can reveal distinct structural information. Our experiments with several volume data sets demonstrate the effectiveness of the structure-aware lighting design approach. It is well suited to use by novices as it requires little technical understanding of the rendering parameters associated with direct volume rendering.

Seismic volume datasets are usually obtained in 32-bit floating point precision, which makes them difficult to be processed on commodity computer due to their large size and high dynamic range (HDR). In this paper, we present a novel quantization method which not only compresses the size of seismic datasets, but also preserves the important seismic features such as detailed structures, local relevance structures and singularities. Our method first identifies the sub range of detailed structures by measuring the difference between the histograms of original datasets and those smoothed by bilateral filter. Then, local relevance statistics (LRS) is proposed to determine sub ranges of local relevance structures. We also assign relatively small sub ranges for preserving some indispensable singularities. According to the piecewise coordinates, our feature-preserving quantization technique is conducted and the low dynamic range(LDR) datasets are generated. To further preserve the local contrast and keep the continuity of original datasets, tone reduction and Bezier curve are resorted to optimize our quantization process. As will be shown in the comparative study, our quantified LDR datasets could be easily rendered on normal graphic hardwares, and the rendering results still present the features of interest, which facilitate further seismic interpretations.

Halos are usually used to enhance depth perception and display spatial relationships in illustrative visualization. In this paper, we present a simple and effective method to create volumetric halo illustration. In the pre-processing stage, we generate a view-independent halo intensity volume, which contains all potential halos around the boundaries of features, based on the opacity volume on graphic hardware. During halo rendering, the halo intensity volume is used to extract halos only around the contours of structures for the current viewpoint. The performance of our approach is significantly faster than previous halo illustration methods, which perform both halo generation and rendering during volume rendering. Experimental results demonstrate volumetric halo effects and the efficiency of the proposed approach.

Viewpoint selection is becoming a useful part in the volume visualization pipeline, as it further improves the efficiency of data understanding by providing representative viewpoints. We present two structure-aware view descriptors, which are the shape view descriptor and the detail view descriptor, to select the optimal viewpoint with the maximum amount of the structural information. These two proposed structure-aware view descriptors are both based on the gradient direction, as the gradient is a well-defined measurement of boundary structures, which have been proved as features of interest in many applications. The shape view descriptor is designed to evaluate the overall orientation of features of interest. For estimating local details, we employ the bilateral filter to construct the shape volume. The bilateral filter is very effective in smoothing local details and preserving strong boundary structures at the same time. Therefore, large-scale global structures are in the shape volume, while small-scale local details still remain in the original volume. The detail view descriptor measures the amount of visible details on boundary structures in terms of variances in the local structure between the shape volume and the original volume. These two view descriptors can be integrated into a viewpoint selection framework, and this framework can emphasize global structures or local details with flexibility tailored to the user's specific situations. We performed experiments on various types of volume datasets. These experiments verify the effectiveness of our proposed view descriptors, and the proposed viewpoint selection framework actually locates the optimal viewpoints that show the maximum amount of the structural information.

The seismic data is a kind of high dynamic range data, which is very difficult to realize high-quality visualization. In this paper, we present a new method to scale the high-dynamic-range seismic data for visualization, which can quantize the high-dynamic-range data into the low-dynamic-range data, while preserving the high frequency characteristics of the geological structure. We divide the seismic data into two parts, the large-scale geological structure part and the small-scale geological structure part. The large-scale part is obtained using an edge-preserving filter called bilateral filter. Then the small-scale part and the processed large-scale part are merged into a new seismic data. Finally, the new generated data is scaled by the histogram-based quantization algorithm to a low-dynamic-range data. In this paper, we discuss and implement our quantization algorithm, and give the results.

Multi-channel display system is more suitable for collaboration applications than common desktop PC. In tradition, such multi-channel displays are used by a group of collocated users. In this paper, we propose one remote interaction system to allow a group of users to simultaneously access the multi-channel display on common PCs. Our system can achieve real time performance under mainstream network environment, which is difficult for the previous work even under the network testbed of high bandwidth. In the system, the multi-channel video generated on the server side is captured by synchronizing a group of capturing processes. For achieving real time performance, one hierarchical subtile-based approach is designed to represent and compress the multi-channel video using MPEG2. The system selectively streams the bitstream containing the region of visible pixels to the corresponding client. For controlling the multichannel application on client side, we refactor the architecture of VNC software, and integrate the enhanced version into our system. Experimental results show that our system achieves real time performance when simultaneously serving group users.

Matting is of great importance in image editing, which evaluates the opacity value with dependence on the provided foreground and background information. How to keep the subtle details is the main focus in soft matting. In this paper, we extend the previous Poisson matting approach in three points. An acquisition of subtle details is proposed, which is employed for the input to matting procedure. The modification to Poisson equation from divergence-based to eigenvector-based makes the matting more faithful to the details. Moreover, the construction of simultaneous equations is presented. By these improvements, Poisson matting could be finished in one procedure and extract the delicate matte. We name the proposed method as Oriented Poisson Matting. The demonstrations show that ours outperforms previous Poisson matting.

Abstract This paper proposes a patch‐based texture synthesis approach that cuts and stitches irregularly shaped texture patches to generate new texture images with minimized visual discontinuity. It works well on a wide range of textures. A semiautomatic algorithm is developed to obtain the irregularly shaped patches. To synthesize strictly structured textures, a regular pasting method is proposed to identify the texture structures and subsequently position the irregularly shaped patches according to the identified structures. The results and comparisons with related work are given.

Recently, the graphics processing capabilities of graphics systems have improved a lot, and more and more information must be presented to the users. While the display resolution of monitors is still what it is 20 years ago, and becomes the bottleneck of visualization. This leads to recent interest in the research for high-resolution displays. One great solution is to use multiple projectors tiled together to form a large logical display wall and provide high display resolutions and large physical display sizes. This paper presents a survey of the construction process of these kinds of multi-projector tiled display wall systems. The hardware construction problem is first explored. And then we describe the two most important problems of multi-projector tiled display wall systems, i.e. geometric calibration and photometric calibration of projectors. Various algorithms are addressed and compared. Lastly, an introduction of the high resolution image generation via distributed rendering is presented.

This work gives an overview of real-time visualisation algorithms developed under the EC-funded project Multimod to support a novel paradigm for the virtual representation of musculo-skeletal structures. These algorithms are fully integrated into the Multimod Application Framework (MAF), an open-source freely-available software framework for the rapid development of medical visualisation applications. MAF is based on the visualisation toolkit (VTK) and other specialised toolkits, e.g. for image registration and segmentation, collision detection or numerical computation. MAF provides a range of high-level components that can be easily combined for rapid construction of visualisation applications that support synchronised views. The majority of algorithms available within the standard underlying MAF toolkits were frequently either too slow or too general for our purposes. We have thus implemented computationally efficient versions of existing algorithms, e.g. for surface and volume rendering, and more importantly, developed new techniques, e.g. for X-ray rendering and designing volume rendering transfer functions. To achieve interactive rendering we have employed a scheme for space partitioning. The emphasis is on exploiting the characteristics of medical datasets (e.g. density value homogeneity) but further utilising the hardware-accelerated capabilities of modern graphics cards. In this context, calculations are moved into hardware as appropriate while avoiding dependency on specialised features of particular manufacturers so as to ensure real code portability.

We give a brief overview of a new source of data on human anatomy and kinematics created by the EC-funded project VAKHUM. The motivation was to provide support for education at various levels through animated 3D models and finite element meshes of human joints, while making original high-quality raw data available for industrial users. We outline some identified user demands. We further describe the data types and relevant software components employed for accessing and visualising VAKHUM models. The emphasis is on a novel user interface based on focal points and intuitive 3D controls for viewing anatomy that is appropriately registered with kinematics captured by local e.g. electrogoniometry or global e.g. stereophotogrammetry means. The operating scenario for accessing models in the VAKHUM database is outlined. We close with a summary of the work and pointers to future development.

Navigation path planning is a crucial part of virtual endoscopy, but the identification of the optimal fly-through path is a time-intensive process. This paper describes a prototype system for virtual endoscopy and proposes a slice-based approach to calculating the path of the virtual camera that uses the slice borderlines of region-of-interest images to obtain the approximate centreline, centralised locally with respect to the hollow object wall. By using the 2D slice borderlines to express the hollow object boundary, the path searching and polygon intersection computing are simplified and made more efficient. Finally, we present results of our algorithm tested on the Visible Human data set.