The core task of medical anomaly detection is to precisely locate and identify pathological changes within the complex anatomical structure. In the zero-shot and few-shot scenarios, this anomaly detection requires generalization across heterogeneous imaging modalities, as well as the localization of subtle lesions within complex anatomical structures. We propose Anatomic-Decoupling Adaptation (ADAN), a unified framework based on adapters, following the design of “train-heavy, deploy-light”, the auxiliary regularizer is only used during the training phase, while during inference, only the frozen CLIP backbone network, adapter, and hierarchical biological semantic gating mechanism (HBS) are retained. ADAN mitigates anatomy-driven activations and adapts diagnostic focus to supervision granularity, highlighting local texture cues in zero-shot and leveraging global context in few-shot. For zero-shot transfer without target-modality lesion samples, we introduce the Meta-Synthetic Virtual Pathology System (MS-VPS), which constructs diverse pseudo-anomaly meta-tasks to improve cross-modal generalization. Experiments on BMAD show strong gains in anomaly classification and competitive localization quality over CLIP-based baselines under identical backbones, with the largest benefits on anatomy-dominant modalities.

Multi-view clustering has been empirically shown to improve learning performance by leveraging the inherent complementary information across multiple views of data. However, in real-world scenarios, collecting strictly aligned views is challenging, and learning from both aligned and unaligned data becomes a more practical solution. Partially View-aligned Clustering (PVC) aims to learn correspondences between misaligned view samples to better exploit the potential consistency and complementarity across views, including both aligned and unaligned data. However, most existing PVC methods fail to leverage unaligned data to capture the shared semantics among samples from the same cluster. Moreover, the inherent heterogeneity of multi-view data induces distributional shifts in representations, leading to inaccuracies in establishing meaningful correspondences between cross-view latent features and, consequently, impairing learning effectiveness. To address these challenges, we propose a Semantic MAtching contRasTive learning model (SMART) for PVC. The main idea of our approach is to alleviate the influence of cross-view distributional shifts, thereby facilitating semantic matching contrastive learning to fully exploit semantic relationships in both aligned and unaligned data. Specifically, we mitigate view distribution shifts by aligning cross-view covariance matrices, which enables the inference of a semantic graph for all data. Guided by the learned semantic graph, we further exploit semantic consistency across views through semantic matching contrastive learning. After the optimization of the above mechanisms, our model smoothly performs semantic matching for different view embeddings instead of the cumbersome view realignment, which enables the learned representations to enjoy richer category-level semantics and stronger robustness. Extensive experiments on eight benchmark datasets demonstrate that our method consistently outperforms existing approaches on the PVC problem. The code is available at https://github.com/THPengL/SMART.

Reconstructing 3D shapes from monocular freehand sketches is challenging due to their fragmented structures, varying line thickness, and discontinuity. These characteristics cause ambiguity, making it difficult for existing methods to extract sufficient geometric feature information to distinguish subtle shape variations and internal details of the object contours depicted in the sketches, resulting in poor overall reconstruction quality. To address these challenges, we introduce the Part-level Semantic Fusion (PSFusion) module, which combines local units of image features with global feature guidance to enhance the representation of complex geometric structures and subtle contour variations. This approach reduces ambiguity in complex edges and local details, improving shape preservation and edge contour accuracy. Additionally, we propose the coarse-to-fine 3D Decoder consisting of one 3D convolutional network as a coarse-grained regressor and one Shuffle-UNet-based fine-grained refiner, to capture feature dependencies across spatial and channel dimensions. Shuffle operations facilitate information exchange among sub-features, enhancing structural differentiation and cross-feature dependency modelling. This significantly improves the handling of subtle textures and complex intersection boundaries. Extensive experiments on three public benchmarks show that our method outperforms baseline approaches, as demonstrated by both quantitative and qualitative results.

Identifying cancer subtypes is crucial for understanding disease progression. With advancements in high-throughput experimental technology, leveraging multiple types of omic data for subtype identification has become feasible. Various integrative cancer subtyping methods present a promising computational approach for identifying cancer subtypes from heterogeneous datasets. While existing integrative cancer subtyping methods have shown promising results in this task, efficiently integrating and clustering multi-omics datasets remains challenging due to high noise levels in omics data, which hinder accurate relationship capture among samples. To overcome this challenge, we propose a new deep multi-view subspace clustering model that introduces a self-reinforced learning strategy. This strategy iteratively enhances the quality of self-representation, crucial for capturing relationships among samples and for clustering. Specifically, during model training, our method is capable of learning a highly reliable self-representation by leveraging a good neighbor learning approach. This capability enables us to capture more accurate and robust relationships among samples. Subsequently, with the assistance of this highly reliable self-representation, we further develop a learnable view-graph fusion approach, which enables us to learn an accurate consensus for clustering and guides the overall model learning process. Additionally, we introduce a local graph-guided learning mechanism based on an initial graph learned from raw data. This mechanism helps prevent the model from converging to suboptimal solutions, thereby avoiding unsatisfactory and unstable results. Experimental results demonstrate that our method outperforms several state-of-the-art methods, verify the effectiveness of our approach in cancer subtype identification task.

Sketches are an efficient and effective tool for generating 3D human meshes with arbitrary body shapes and poses. However, current mesh reconstruction methods are mainly designed for natural images, which are hard to apply to sketches due to the abstract and sparse characteristics of the latter. Moreover, there is no dataset with sufficient sketch-mesh pairs for developing and evaluating relevant methods. To tackle these issues, we introduce a hybrid framework that fits parametric human models (e.g., skinned multi-person linear model) to sketches in a coarse-to-fine manner. Specifically, the proposed framework consists of three core components: (i) Given a sketch image as the input, a vision transformer-based Local Image Encoder (LIE) is introduced to model the local structures of the sketch and yields a coarse mesh estimation. (ii) A Global Point Encoder (GPE) taking the 2D coordinates of sketch contours as inputs, is also utilized to obtain the global representation of the sketch. (iii) As the local presentation can depict human poses more precisely while the global representation is more suitable for body shapes, we propose a graph-based refiner (GRefiner) to leverage the advantages of both representations and generate the final well-fitted mesh. Furthermore, we collect a large-scale dubbed Sketch3DS, containing approximately 10,000 paired sketches and human meshes with diverse poses and shapes. Extensive experiments on Sketch3DS demonstrate that the proposed approach outperforms existing methods, achieving accurate alignment between input sketches and constructed human meshes.

Sketch-guided point cloud reconstruction aims to provide an efficient and flexible pathway to generate plausible 3D shapes automatically from a free-hand sketch shaping the modeling intentions of end users. However, such a task is still in its infancy due to the complex, challenging, and highly variable patterns of sketches by nature. In this paper, we present a novel sketch-guided framework based on free-form deformation (FFD) for 3D point cloud generation. To capture sufficient meaningful features from a sketch, a dual-branch encoding architecture is devised to extract complementary semantic and geometric clues by formulating the input as a binary image and a 2D point cloud, respectively. The proposed encoder also learns useful features to guide content generation from a template point cloud before decoding the resultant global features into a set of control points for the use of FFD. We have also developed a large and diverse manually collected dataset, Sketch-3DPC, in which there are a total of 13,754 sketch and 3D point cloud pairs categorized into 11 classes. Both qualitative and quantitative experiment results demonstrate the superiority of the proposed methodology and dataset.

In this paper, we introduce a novel 3D shape reconstruction method from a single‐view sketch image based on a deep neural network. The proposed pipeline is mainly composed of three modules. The first module is sketch component segmentation based on multimodal DNN fusion and is used to segment a given sketch into a series of basic units and build a transformation template by the knots between them. The second module is a nonlinear transformation network for multifarious sketch generation with the obtained transformation template. It creates the transformation representation of a sketch by extracting the shape features of an input sketch and transformation template samples. The third module is deep 3D shape reconstruction using multifarious sketches, which takes the obtained sketches as input to reconstruct 3D shapes with a generative model. It fuses and optimizes features of multiple views and thus is more likely to generate high‐quality 3D shapes. To evaluate the effectiveness of the proposed method, we conduct extensive experiments on a public 3D reconstruction dataset. The results demonstrate that our model can achieve better reconstruction performance than peer methods. Specifically, compared to the state‐of‐the‐art method, the proposed model achieves a performance gain in terms of the five evaluation metrics by an average of 25.5% on the man‐made model dataset and 23.4% on the character object dataset using synthetic sketches and by an average of 31.8% and 29.5% on the two datasets, respectively, using human drawing sketches.

The sketch segmentation problem remains largely unsolved because conventional methods are greatly challenged by the highly abstract appearances of freehand sketches and their numerous shape variations. In this work, we tackle such challenges by exploiting different modes of sketch data in a unified framework. Specifically, we propose a deep neural network SPFusionNet to capture the characteristic of sketch by fusing from its image and point set modes. The image modal component SketchNet learns hierarchically abstract ro-bust features and utilizes multi-level representations to produce pixel-wise feature maps, while the point set-modal component SPointNet captures local and global contexts of the sampled point set to produce point-wise feature maps. Then our framework aggregates these feature maps by a fusion network component to generate the sketch segmentation result. The extensive experimental evaluation and comparison with peer methods on our large SketchSeg dataset verify the effectiveness of the proposed framework.

Parsing sketches via semantic segmentation is attractive but challenging, because (i) free-hand drawings are abstract with large variances in depicting objects due to different drawing styles and skills; (ii) distorting lines drawn on the touchpad make sketches more difficult to be recognized; (iii) the high-performance image segmentation via deep learning technologies needs enormous annotated sketch datasets during the training stage. In this paper, we propose a Sketch-target deep FCN Segmentation Network(SFSegNet) for automatic free-hand sketch segmentation, labeling each sketch in a single object with multiple parts. SFSegNet has an end-to-end network process between the input sketches and the segmentation results, composed of 2 parts: (i) a modified deep Fully Convolutional Network(FCN) using a reweighting strategy to ignore background pixels and classify which part each pixel belongs to; (ii) affine transform encoders that attempt to canonicalize the shaking strokes. We train our network with the dataset that consists of 10,000 annotated sketches, to find an extensively applicable model to segment stokes semantically in one ground truth. Extensive experiments are carried out and segmentation results show that our method outperforms other state-of-the-art networks.

Abstract Sketching is a simple and natural way of expression and communication for humans. For this reason, it gains increasing popularity in human computer interaction, with the emergence of multitouch tablets and styluses. In recent years, sketch‐based interactive methods are widely used in many retrieval systems. In particular, a variety of sketch‐based 3D model retrieval works have been presented. However, almost all of these works focus on directly matching sketches with the projection views of 3D models, and they suffer from the large differences between the sketch drawing and the views of 3D models, leading to unsatisfying retrieval results. Therefore, in this paper, during the matching procedure in the retrieval, we propose to match the sketch with each 3D model from historical users instead of projection views. Yet since the sketches between the current user and the historical users can have big difference, we also aim to handle users' personalized deviations and differences. To this end, we leverage recommendation algorithms to estimate the drawing style characteristic similarity between the current user and historical users. Experimental results on the Large Scale Sketch Track Benchmark(SHREC14LSSTB) demonstrate that our method outperforms several state‐of‐the‐art methods.