Reconstructing a faithful geometric surface from sparse images remains a fundamental challenge in 3D computer vision. While recent methods have achieved remarkable progress, they still struggle to recover reliable geometry due to the lack of multi-view geometric cues, particularly in non-overlapping regions. To address this issue, we introduce VGGS, a Gaussian Splatting (GS) method that exploits multi-view geometric priors from VGGT for efficient and high-fidelity sparse-view surface reconstruction. Our primary contribution is an anchor-calibrated depth estimation scheme, which yields accurate depth maps. The insight is to align the VGGT depth prior to the underlying surface with a sparse set of multi-view consistent anchors, then infer depth for unreliable regions by relative depth estimation. Furthermore, to mitigate misalignment in complex scenes, we propose a relative depth consistency loss that penalizes the rendered depth if its relative depth relationship in local regions is inconsistent to the multi-view prior. Extensive experiments on widely-used benchmarks show that VGGS surpasses state-of-the-art methods in both accuracy and efficiency, delivering 4–7× faster optimization while reducing memory consumption compared to previous GS-based approaches.

Bias in machine learning datasets occurs when certain attributes are unfairly associated, e.g., serious males being mostly linked with law enforcement officers in job-related image datasets. Training models on biased datasets will degrade model performance and lead to fairness issues, particularly for underrepresented groups. Existing bias detection methods mainly focus on explicit biases associated with predefined attributes (e.g., gender and ethnicity) while overlooking implicit biases associated with subtler, dataset-specific attributes (e.g., facial expressions and attire). To address this gap, we present BiasField, an interactive tool that offers a closed-loop workflow for detecting, analyzing, and mitigating bias. Central to BiasField is the adaptive detection of both explicit and implicit biases, a process facilitated by the automatic extraction of the dataset-specific attributes. It then employs a plant-growth metaphor to visualize these biases, enabling structured analysis to identify similar biases and track how they strengthen with additional attributes. Finally, confirmed biases are mitigated through targeted generative data augmentation. A user study, two case studies, and an expert study are conducted to demonstrate its capability to detect, analyze, and mitigate complex biases.

Facial avatar animation methods animate virtual characters based on natural human performance and have been widely used in film and game production. Despite its practical relevance, academic research in this area has been scarce, particularly in the deep learning era. To fill this gap, we propose Facial Avatar Animation via Spatial-Temporal Aggregation (FAST), which leverages Memory-based Spatial-Temporal Aggregation (MSTA) to capture both spatial and temporal dependencies in facial animation. However, directly regressing blendshape and pose coefficients introduces uncertainty and reduces interpretability. Thus, we introduce Implicit Latent Representations (ILRs), which learn the semantic correspondence between the predicted results and blendshapes/poses, enhancing both model interpretability and the vivid tracking of facial expressions. Additionally, monocular RGB-based pose estimation suffers from depth ambiguity that destabilizes animation. To address this, we incorporate a Semantic-Aware Rigid Prior (SRP) to enhance the rigid stability of the animation. To tackle the lack of blendshape coefficient annotations in existing datasets and support the advancement of avatar animation methods, we present the BS500 dataset, which includes 500 individuals and over 4.5 million frames with diverse demographic features such as gender, age, expression, and head pose. Extensive experiments show the superiority of the FAST method over existing approaches. The code and dataset will be released.

The goal of generic multimodal summarization is to extract the most important information from different modalities to form summaries. Yet the importance of scenes and text in a video is often subjective, and users should have the option of customizing the summary by using natural language to specify what is important to them. However, existing methods for fully automatic multimodal summarization have not exploited available language models, which can serve as an effective prior for saliency. To address this issue, we introduce Query-Focused Multimodal Summ arization(QFSumm), a single framework for addressing both generic and query-focused multimodal summarization, typically approached separately in the literature. In addition, we propose a novel gate-guided mixture-of-experts that uses expert gate module to organize three experts (video expert, text expert and shared expert) to model the correlations between multimodal information. In addition, we propose two novel contrastive losses to represent consistency and diversity. Extensive experiments on a query-focused video summarization dataset (QFVS), two standard video summarization datasets (TVSum and SumMe) and three multimodal summarization datasets (CNN, Daily Mail and BLiSS) demonstrate the superiority of QFSumm, achieving state-of-the-art performances on all datasets.

The high performance of tree ensemble classifiers benefits from a large set of rules, which, in turn, makes the models hard to understand. To improve interpretability, existing methods extract a subset of rules for approximation using model reduction techniques. However, by focusing on the reduced rule set, these methods often lose fidelity and ignore anomalous rules that, despite their infrequency, play crucial roles in real-world applications. This paper introduces a scalable visual analysis method to explain tree ensemble classifiers that contain tens of thousands of rules. The key idea is to address the issue of losing fidelity by adaptively organizing the rules as a hierarchy rather than reducing them. To ensure the inclusion of anomalous rules, we develop an anomaly-biased model reduction method to prioritize these rules at each hierarchical level. Synergized with this hierarchical organization of rules, we develop a matrix-based hierarchical visualization to support exploration at different levels of detail. Our quantitative experiments and case studies demonstrate how our method fosters a deeper understanding of both common and anomalous rules, thereby enhancing interpretability without sacrificing comprehensiveness.

Interactive image segmentation aims at obtaining a segmentation mask for an image using simple user annotations. During each round of interaction, the segmentation result from the previous round serves as feedback to guide the user's annotation and provides dense prior information for the segmentation model, effectively acting as a bridge between interactions. Existing methods overlook the importance of feedback or simply concatenate it with the original input, leading to underutilization of feedback and an increase in the number of required annotations. To address this, we propose an approach called Focused and Collaborative Feedback Integration (FCFI) to fully exploit the feedback for click-based interactive image segmentation. FCFI first focuses on a local area around the new click and corrects the feedback based on the similarities of high-level features. It then alternately and collaboratively updates the feedback and deep features to integrate the feedback into the features. The efficacy and efficiency of FCFI were validated on four benchmarks, namely GrabCut, Berkeley, SBD, and DAVIS. Experimental results show that FCFI achieved new state-of-the-art performance with less computational overhead than previous methods. The source code is available at https://github.com/veizgyauzgyauz/FCFI.

Learning per-point semantic features from the hierarchical feature pyramid is essential for point cloud semantic segmentation. However, most previous methods suffered from ambiguous region features or failed to refine per-point features effectively, which leads to information loss and ambiguous semantic identification. To resolve this, we propose Retro-FPN to model the per-point feature prediction as an explicit and retrospective refining process, which goes through all the pyramid layers to extract semantic features explicitly for each point. Its key novelty is a retro-transformer for summarizing semantic contexts from the previous layer and accordingly refining the features in the current stage. In this way, the categorization of each point is conditioned on its local semantic pattern. Specifically, the retro-transformer consists of a local cross-attention block and a semantic gate unit. The cross-attention serves to summarize the semantic pattern retrospectively from the previous layer. And the gate unit carefully incorporates the summarized contexts and refines the current semantic features. Retro-FPN is a pluggable neural network that applies to hierarchical decoders. By integrating Retro-FPN with three representative backbones, including both point-based and voxel-based methods, we show that Retro-FPN can significantly improve performance over state-of-the-art backbones. Comprehensive experiments on widely used benchmarks can justify the effectiveness of our design. The source is available at https://github.com/AllenXiangX/Retro-FPN.

This paper presents a refinement framework for enhancing the accuracy of interactive image segmentation by exploiting all available semantic clues. Interactive image segmentation iteratively improves segmentation masks using an input image and user annotations. The information available in this process ranges from low-level visual features like colors and textures to high-level semantic information, such as user annotations and segmentation results. Despite tremendous efforts to segment the overall object shapes, existing methods underutilize the available semantic clues, causing unsatisfactory boundary quality for segmentation masks. The proposed framework first extracts confidence guidance maps, then suppresses and lifts the predicted probabilities for confident pixels, and finally utilizes color similarities as bases and prediction confidence as guidance to refine the segmentation boundaries. Experimental results demonstrate that the framework has a low computational cost and significantly boosts existing methods on standard benchmarks.

This paper focuses on self-supervised video correspondence learning, which learns effective representations from raw videos without manual annotations and exploits the learned representations for video visual tracking tasks. Previous methods extract temporal correspondence between two frames in fixed geometric structures, which easily leads to mismatches of pixels and overlooks the intra-frame semantic correspondence. To address these issues, we propose a Discriminative Spatiotemporal Alignment (DSA) framework to improve the tracking accuracy in the inference stage. DSA first discriminates representations of different instances for each reference frame through an Instance-Guided Spatial Alignment (IGSA) module. Then, it employs a Focused Temporal Alignment (FTA) module, which samples discriminative pixels from reference frames and propagates the labels of the sampled reference pixels to a target pixel. Experimental results show that DSA possesses flexibility and generalizability and has boosted previous approaches on three tracking tasks, including video object segmentation, human part segmentation, and pose keypoint tracking.

Image animation brings life to the static object in the source image according to the driving video. Recent works attempt to perform motion transfer on arbitrary objects through unsupervised methods without using a priori knowledge. However, it remains a significant challenge for current unsupervised methods when there is a large pose gap between the objects in the source and driving images. In this paper, a new end-to-end unsupervised motion transfer framework is proposed to overcome such issues. Firstly, we propose thin-plate spline motion estimation to produce a more flexible optical flow, which warps the feature maps of the source image to the feature domain of the driving image. Secondly, in order to restore the missing regions more realistically, we leverage multi-resolution occlusion masks to achieve more effective feature fusion. Finally, additional auxiliary loss functions are designed to ensure that there is a clear division of labor in the network modules, encouraging the network to generate high-quality images. Our method11Our source code is publicly available: https://github.com/yoyo-nb/Thin-Plate-Spline-Motion-Model. can animate a variety of objects, including talking faces, human bodies, and pixel animations. Experiments demonstrate that our method performs better on most benchmarks than the state of the art with visible improvements in motion-related metrics.

Image-level annotations allow to achieve semantic segmentation in a weakly-supervised way. Most advanced approaches utilize class activation map (CAM) from deep classifier to generate pseudo-labels. However, CAM generally only focuses on the most discriminative parts of targets. To explore more pixel-level semantic information and recognize all pixels within the objects for segmentation, we propose a Region-based Pixels Integration Mechanism (RPIM) which discovers the intra-region and inter-region information. Firstly, the foreground regions are formed on the basis of superpixels and the initial responses. Each region is regarded as a subtree, whose nodes are the image pixels within the region. Then, an Intra-region Integration (IRI) Module is designed to explore the nodes relationships inside the subtree. Within each subtree, nodes will vote for the most confident class and share the highest probability. Moreover, an Inter-region Spreading (IRS) Module is proposed to further improve the consistency of CAM. For each class, the most confident unprocessed subtree finds their homologous neighbors, connects with them and shares its probability. By iterative refinement, the training process will integrate the individual nodes into region subtrees, and gradually form the subtrees with similar probabilities to the object semantic trees for each foreground class. To our best knowledge, our approach achieves the state-of-the-art performance on PASCAL VOC 2012 validation set with 71.4% mIoU. The experiments also show that our scheme is plug-and-play and can collaborate with different approaches to improve their performance.

The rapid development of deep natural language processing (NLP) models for text classification has led to an urgent need for a unified understanding of these models proposed individually. Existing methods cannot meet the need for understanding different models in one framework due to the lack of a unified measure for explaining both low-level (e.g., words) and high-level (e.g., phrases) features. We have developed a visual analysis tool, DeepNLPVis, to enable a unified understanding of NLP models for text classification. The key idea is a mutual information-based measure, which provides quantitative explanations on how each layer of a model maintains the information of input words in a sample. We model the intra- and inter-word information at each layer measuring the importance of a word to the final prediction as well as the relationships between words, such as the formation of phrases. A multi-level visualization, which consists of a corpus-level, a sample-level, and a word-level visualization, supports the analysis from the overall training set to individual samples. Two case studies on classification tasks and comparison between models demonstrate that DeepNLPVis can help users effectively identify potential problems caused by samples and model architectures and then make informed improvements.

Deep learning based methods, especially convolutional neural networks (CNNs) have been successfully applied in the field of single image super-resolution (SISR). To obtain better fidelity and visual quality, most of existing networks are of heavy design with massive computation. However, the computation resources of modern mobile devices are limited, which cannot easily support the expensive cost. To this end, this paper explores a novel frequency-aware dynamic network for dividing the input into multiple parts according to its coefficients in the discrete cosine transform (DCT) domain. In practice, the high-frequency part will be processed using expensive operations and the lower-frequency part is assigned with cheap operations to relieve the computation burden. Since pixels or image patches belong to low-frequency areas contain relatively few textural details, this dynamic network will not affect the quality of resulting super-resolution images. In addition, we embed predictors into the proposed dynamic network to end-to-end fine-tune the handcrafted frequency-aware masks. Extensive experiments conducted on benchmark SISR models and datasets show that the frequency-aware dynamic network can be employed for various SISR neural architectures to obtain the better tradeoff between visual quality and computational complexity. For instance, we can reduce the FLOPs of SR models by approximate 50% while preserving state-of-the-art SISR performance.

Small object arrangement is very important for creating detailed and realistic 3D indoor scenes. In this article, we present an interactive framework based on active learning to help users create customized arrangements for small objects according to their preferences. To achieve this with minimal user effort, we first learn the prior knowledge about small object arrangement from a 3D indoor scene dataset through a probability mining method, which forms the initial guidance for arranging small objects. Then, users are able to express their preferences on a few small object categories, which are automatically propagated to all the other categories via a novel active learning approach. In the propagation process, we introduce a novel metric to obtain the propagation weights, which measures the degree of interchangeability between two small object categories, and is calculated based on a spatial embedding model learned from the small object neighborhood information extracted from the 3D indoor scene dataset. Experiments show that our framework is able to help users effectively create customized small object arrangements with little effort.

Abstract We present a novel isotropic surface remeshing algorithm that automatically aligns the mesh edges with an underlying directional field. The alignment is achieved by minimizing an energy function that combines both centroidal Voronoi tessellation (CVT) and the penalty enforced by a six‐way rotational symmetry field. The CVT term ensures uniform distribution of the vertices and high remeshing quality, and the field constraint enforces the directional alignment of the edges. Experimental results show that the proposed approach has the advantages of isotropic and field‐aligned remeshing. Our algorithm is superior to the representative state‐of‐the‐art approaches in various aspects.

We propose a method to recover the shape of a 3D room from a full-view indoor panorama. Our algorithm can automatically infer a 3D shape from a collection of partially oriented superpixel facets and line segments. The core part of the algorithm is a constraint graph, which includes lines and superpixels as vertices, and encodes their geometric relations as edges. A novel approach is proposed to perform 3D reconstruction based on the constraint graph by solving all the geometric constraints as constrained linear least-squares. The selected constraints used for reconstruction are identified using an occlusion detection method with a Markov random field. Experiments show that our method can recover room shapes that can not be addressed by previous approaches. Our method is also efficient, that is, the inference time for each panorama is less than 1 minute.

A high-quality label layout is critical for effective information understanding and consumption. Existing labeling methods fail to help users quickly gain an overview of visualized data when the number of labels is large. Visual clutter is a major challenge preventing these methods from being applied to real-world applications. To address this, we propose a context-aware label layout that can measure and reduce visual clutter during the layout process. Our method formulates the clutter model using four factors: confusion, visual connection, distance, and intersection. Based on this clutter model, an effective clutter-aware labeling method has been developed that can generate clear and legible label layouts in different visualizations. We have applied our method to several types of visualizations and the results show promise, especially in support of an uncluttered and informative label layout.

A 2D line drawing is a direct way to illustrate a 3D object. 3D reconstruction from a 2D single line drawing is an important research topic in the area of computer vision and geometric modeling. In this paper, we present an approach to reconstruct objects from single 2D line drawings with line segments, arcs and B-splines. Firstly, planar and curved faces are identified in the line drawing. Then, with a depth estimation algorithm to establish the initial depth of the vertices, the wireframe is reconstructed by hierarchical optimization-based method. Finally, planar and curved faces are reconstructed. The experimental results show that the recovered objects correspond to the human perception. The time complexity shows that our approach is in high-efficiency.

Similar historical forecasts (also called analogs) are important references for the forecasters in their routine work. The forecasts are multi-variate data, and it is difficult to determine whether two forecasts are similar. In this study, we propose a radar glyph based visualization system to analyze the similarity of analogs using multiple variables. The radar glyph, nested in a radar chart, can effectively provide the forecasters with the similarity based on different variables, while keeping an overall similarity context. An interactive system is developed to support the analog data analysis and help the forecasters detect potential biases in the forecast. The usability of the system is then demonstrated using two case studies.

We address the problem of room structure recognition from a 360 cylindrical panorama. The proposed approach first transforms the original panorama into four perspective projected sub-images, then predicts the final layout based on a holistic parameterization framework via candidate evaluation using a linear scoring function, whose weights are trained on a normal indoor image data set. Experiments show that the proposed algorithm is capable of producing reasonable predictions which can be used for further applications such as 3D reconstruction.

In this paper, we present an example-driven symbol recognition algorithm based on its key features in CAD engineering drawings. When user provides an example of a specific symbol, the input symbol is analyzed and its features are extracted automatically. Based on the relation representation, the constrained tree with key feature priority can be established for this type of symbol. By this means, the symbol library can be built and expanded automatically in order to handle variety engineering drawings. In the next stage of the recognition processes, we first locate the key feature nodes in drawings, and then find other elements around which satisfy the topology structure of constrained tree. If all the elements and constrains in the tree are found, the symbol object will be recognized. Because of the accurate position, unnecessary matching calculations are greatly reduced. Experimental results validate that our approach is effective.

This paper presents a new multi-resolution mesh fitting algorithm, extending the adaptive patch-based fitting scheme where each underlying quadrilateral is recursively subdivided into four sub-patches. In this paper, the G1continuity constraints, which mainly consist of perpendicular constraints and twist compatibility constraints, are deduced for B-spline patches. In order to construct a unique B-spline patch for each quadrilateral, the mesh vertices are applied in a least-square approximation, and the energy functions associated with a patch are minimized. In contrast to the original algorithm, this paper fits the mesh into B-spline patches instead of Bezier patches with G1continuity. The B-spline patches make the algorithm have more free control points to be used for optimizing the shape of the quadrilateral patches to achieve higher flexible patch control and less recursive times.

Sectional views are widely used in engineering practice due to their clear and concise expression. However, it is difficult for computers to understand because of the large numbers of omitted entities and their diversified representations. This paper aims at reconstructing 3D models from 2D sectional views by improving the traditional volume based method. First, we present a two-stage loop searching algorithm to extract desired loops from sectional views. Then, sub-objects are identified by the hint-based feature identification algorithm with an intuitive loop-matching criterion. After that, a model-directed algorithm is proposed to guide the generation of sub-objects which are assembled together to form the final objects. The algorithm can handle full sections, partial sections and offset sections, as well as orthographic views. Multiple sectional views are supported in our algorithm. Moreover, the domain of objects is extended to inclined quadric surfaces and intersecting quadric surfaces with higher order curves. Experiment results show its practicability.

An algorithm for reducing control points in NURBS surface skinning is proposed in this paper. We first give distance estimation between the input curves and the surface that has been removed a knot. Then we further reduce the control points in NURBS surface skinning within the given tolerance by utilizing the ldquoresidual distancerdquo. Furthermore, the algorithm is applied to simplify a NURBS surface by giving distance estimation between the original surface and the knot-removal surface. Experimental results demonstrate the usability and quality of the proposed algorithm.

Sectional views are widely used in engineering practice due to their clear and concise expression. However, large numbers of entities are missing in sectional views that makes it hard for computer to understand drawings. Feature identification is the key procedure for solids reconstruction from sectional views. Based on the analysis of the default drawing rules appearing in sectional views, a new algorithm is presented to identify and construct elemental objects in the procedure of volume-based 3D solids reconstruction. In this algorithm, a hint-based search strategy with priority principles is developed for identifying elemental features, even some of them do not have complete expression in 2D views. The algorithm is suitable for full sections, partial sections and offset sections, and there is no restriction on the number of sectional views included in one drawing. Experimental results validate our algorithm.

Realizing the full potential of a multi-radio mesh network involves two main challenges: how to assign channels to radios at each node to minimize interference and how to choose high throughput routing paths in the face of lossy links, variable channel conditions and external load. This paper presents ROMA, a practical, distributed channel assignment and routing protocol that achieves good multi-hop path performance between every node and one or more designated gateway nodes in a dual-radio network. ROMA assigns non-overlapping channels to links along each gateway path to eliminate intra-path interference. ROMA reduces inter-path interference by assigning different channels to paths destined for different gateways whenever possible. Evaluations on a 24-node dual-radio testbed show that ROMA achieves high throughput in a variety of scenarios.

View identification is the basal process for solid reconstruction from engineering drawings. A new method is presented to label various views from a section-involved drawing and identify geometric planes through the object at which the sections are to be located. In the approach, a graph representation is developed for describing multiple relationships among various views in the 2D drawing space, and a reasoning technique based on evidence theory is implemented to validate view relations that are used to fold views and sections in the 3D object space. This is the first automated approach which can handle multiple sections in diverse arrangements, especially accommodating the aligned section for the first time. Experimental results are given to show that the proposed solution makes a breakthrough in the field and builds a promising basis for further expansibility, although it is not a complete one.

Solid reconstruction from engineering drawings is one of the efficient technologies to product solid models. The B-rep oriented approach provides a practical way for reconstructing a wide range of objects. However, its major limitation is the computational complexity involved in the search for all valid faces from the intermediate wire-frame, especially for objects with complicated face topologies. In previous work, we presented a hint-based algorithm to recognize quadric surfaces from orthographic views and generate a hybrid wire-frame as the intermediate model of our B-rep oriented method. As a key stage in the process of solid reconstructing, we propose an algorithm to convert the hybrid wire-frame to the final B-rep model by extracting all the rest faces of planes based on graph theory. The entities lying on the same planar surface are first collected in a plane graph. After all the cycles are traced in a simplified edge-adjacency matrix of the graph, the face loops of the plane are formed by testing loop containment and assigning loop directions. Finally, the B-rep model is constructed by sewing all the plane faces based on the Möbius rule. The method can efficiently construct 2-manifold objects with a variety of face topologies, which is illustrated by results of implementation.

A novel mesh parametrization method for an open connected surface is presented. The parametrization method is based on Hessian-based locally linear embedding (HLLE). Our method operates directly on the surface without using any partition technique and can preserve the local and global structure, while partition-based methods often produce high distortion and discontinuity nearby partition boundaries. In addition, some examples about texture mapping show the efficiency of our method.