The goal of Few-Shot Segmentation (FSS) is to segment images of novel categories using few labeled examples. However, FSS tasks face challenges such as over-segmentation and lack of generalization issues. This paper addresses these challenges by employing a triple attention mechanism (TAM) and a hierarchical decoding transformer (HDT). Specifically, TAM is proposed to enhance the model’s ability to focus on spatial regions within query features that are semantically relevant to the target category. The HDT module then aggregates the enhanced query features with the support features in a decoupled manner, generating dense features with pixel-level semantic relevance, which improves the segmentation ability on novel classes. Additionally, considering that class-level labels inside image can provide weak supervision for the segmentation task, this paper introduces a contrastive language image pretraining (CLIP) based model to enhance the segmentation performance. The Grad-CAM mechanism is utilized to convert the class logit scores from CLIP into localization heatmaps, effectively leveraging the text label information to provide prior localization cues for the model. Extensive experiments conducted on the PASCAL-5iand COCO-20idatasets demonstrate state-of-the-art performance. The experimental results validate the effectiveness of the proposed method, significantly improving the generalization and segmentation performance of few-shot semantic segmentation models on novel categories.

Microsatellite instability (MSI), characterized by alterations in short tandem repeat sequences throughout the genome, is an important biomarker for cancer diagnosis, prognosis, and treatment planning. Traditional MSI detection methods, such as immunohistochemistry (IHC) and polymerase chain reaction (PCR), are expensive, time-consuming, and resource-intensive. Recent advances in deep learning have enabled direct MSI prediction from hematoxylin and eosin (H&E) stained whole slide images (WSIs), offering a potentially faster and more cost-effective approach. However, existing convolutional neural network (CNN)-based methods for MSI prediction often focus on patch-level prediction and therefore struggle to capture the complex, multi-scale nature of histological features and their spatial relationships. To address these limitations, we propose a novel three-level multiscale graph neural network (GNN) method that integrates tissue, patch, and cellular-level analyses within a unified framework, namely TPC-GNN. The TPC-GNN extracts comprehensive biological information from WSIs by simulating structural interactions across various scales. At the tissue level, it captures global morphological patterns and tissue organization. The patch level focuses on local texture and architectural features, while the cellular level analyzes individual cell morphology and distribution. This multi-scale approach enables the model to capture complex biomarkers from micro to macro levels, providing a more holistic representation of the histological image. To enhance the integration of multi-scale information, we investigate different learnable fusion schemes based on multi-layer perceptrons (MLP), Transformers, and Mamba architectures. Experimental results demonstrate the superiority of our approach, achieving performance improvements of over 2% and 4% in Area Under the Curve (AUC) on the CRC-MSI and STAD-MSI datasets, respectively, across different network backbones compared to existing methods. These findings not only validate the effectiveness of our multiscale GNN approach but also highlight its application to complex histopathological-level image analysis. Codes are available at https://github.com/zhillusion/TPC-GNN.

The Gaussian homotopy method is a classical optimization approach for solving nonconvex problems. It applies Gaussian smoothing to a given problem, using varying smoothing factors to generate a series of proxy problems that are easier to solve, subsequently addressing them gradually from simpler to more complex ones. Traditional Gaussian homotopy methods typically utilize a predefined sequence or parameter to update a single smoothing factor serially. However, this sequential updating process is inefficient and unsuitable for parallel homotopy pipelines. Moreover, the utilization of predefined sequences or parameters leads to a lack of adaptability. To address the abovementioned challenges, we propose an adaptive parallel Gaussian homotopy optimization method. Initially, we introduce an adaptive model called Scaling. This model is formulated as a scaler that concurrently adjusts multiple smoothing factors, with each factor aligning to a homotopy level. It can be integrated easily into serial and parallel homotopy pipelines. Furthermore, we establish a collaborative training regimen to jointly train the Scaling model and a parallel homotopy model, named continuation path learning (CPL) [1] model. Throughout the training process, the Scaling model furnishes CPL with multiple scaled smoothing factors and updates CPL implicitly. Extensive experiments demonstrate that the proposed Gaussian homotopy approach performs competitively.

The fine-grained object detection, capable of identifying subcategories or types, is thriving in remote sensing scenes. In practice, most existing fine-grained detectors are derived from the two-stage R-CNN paradigm with intricate anchor boxes, focusing on refining features of region of interest (RoI) to boost performance, which often incurs a redundant process. In contrast, the one-stage, anchor-free paradigm possesses a simple yet effective pipeline, but its exploration in fine-grained detections is still far from sufficient. In this article, we propose a one-stage, anchor-free fine-grained detector for remote sensing aircraft recognition. We initially delve into predominant issues when extending the one-stage framework to conduct fine-grained detections, typified by severe interclass confusion and inferior performance in rare categories. Then, we design a fine-grained classification branch, including a region-to-region context distributor (R2CD), a class-aware decoupled focal loss (CDFL), and a cross-shaped sample space (CS3), to address these hindrances. Specifically, the R2CD flexibly integrates the sparse attention mechanism with mask prediction operations to conduct region-level content interactions separately within the foreground and background of feature maps, significantly alleviating the interclass confusion by enhancing subtle features; the CDFL employs dynamic modulation factors driven by optimization gradients to regulate loss contributions across categories while optimizing category-specific heatmaps, thus prioritizing rare categories with hard samples; the CS3 attains a preferable assignment strategy of positive and negative samples by incorporating structure prior, facilitating the capture of foreground features. Extensive experiments conducted on the MAR20 and FAIRPlane11 datasets demonstrate that our model excels at distinguishing fine-grained categories and is well-suited for performing fine-grained detection tasks.

Oriented object detectors provide many scientific solutions for object detection tasks in remote sensing scenes. Among them, anchor-free oriented object detectors have attracted much attention because of their flexibility and conciseness in recent years. However, the positive sampling strategy adopted by most modern anchor-free oriented detectors is inadequate to reflect remote sensing objects characterized by arbitrary orientation, densely packed and extensive scale variation. Besides, there is still a lack of a general representation to describe oriented objects shaped with arbitrary convex quadrilaterals. In this paper, we present a one-stage anchor-free oriented object detector, called M2VDet, which focuses on designing the sampling strategy and oriented bounding box description. First, an adaptive sampling strategy is proposed to generate positive sample spaces, which take full account of objects’ characteristics like scale, shape, and orientation. The resulting candidate regions provide a superior space constraint to construct the heatmap, then 2D oriented Gaussian distribution is employed to generate prior labels for positive candidates. Second, a general oriented bounding box representation is designed by adopting the midpoints-to-vertices strategy, which provides a unified approach to describe the arbitrary convex quadrilaterals without brutal approximation. Extensive experimental results on four public datasets (e.g., DOTA, HRSC2016, UCAS-AOD, and SODA-A) demonstrate that M2VDet is with a simple pipeline, but it is able to achieve more robust and available performance when compared with state-of-the-art baseline detectors, especially on the detection of densely packed objects with arbitrary orientation as well as small objects.

Human beings acquire the ability of image classification through visual concept learning, in which the process of concept formation involves intertwined searches of common properties and concept descriptions. However, in most image classification algorithms using deep convolutional neural network (ConvNet), the representation space is constructed under the premise that concept descriptions are fixed as one-hot codes, which limits the mining of properties and the ability of identifying unseen samples. Inspired by this, we propose a learning strategy of visual concept formation (LSOVCF) based on the ConvNet, in which the two intertwined parts of concept formation, i.e. feature extraction and concept description, are learned together. First, LSOVCF takes sample response in the last layer of ConvNet to induct concept description being assumed as Gaussian distribution, which is part of the training process. Second, the exploration and experience loss is designed for optimization, which adopts experience cache pool to speed up convergence. Experiments show that LSOVCF improves the ability of identifying unseen samples on cifar10, STL10, flower17 and ImageNet based on several backbones, from the classic VGG to the SOTA Ghostnet. The code is available at \url{https://github.com/elvintanhust/LSOVCF}.

Recent development of object detection mainly depends on deep learning with large-scale benchmarks. However, collecting such fully-annotated data is often difficult or expensive for real-world applications, which restricts the power of deep neural networks in practice. Alternatively, humans can detect new objects with little annotation burden, since humans often use the prior knowledge to identify new objects with few elaborately-annotated examples, and subsequently generalize this capacity by exploiting objects from wild images. Inspired by this procedure of learning to detect, we propose a novel Progressive Object Transfer Detection (POTD) framework. Specifically, we make three main contributions in this paper. First, POTD can leverage various object supervision of different domains effectively into a progressive detection procedure. Via such human-like learning, one can boost a target detection task with few annotations. Second, POTD consists of two delicate transfer stages, i.e., Low-Shot Transfer Detection (LSTD), and Weakly-Supervised Transfer Detection (WSTD). In LSTD, we distill the implicit object knowledge of source detector to enhance target detector with few annotations. It can effectively warm up WSTD later on. In WSTD, we design a recurrent object labelling mechanism for learning to annotate weakly-labeled images. More importantly, we exploit the reliable object supervision from LSTD, which can further enhance the robustness of target detector in the WSTD stage. Finally, we perform extensive experiments on a number of challenging detection benchmarks with different settings. The results demonstrate that, our POTD outperforms the recent state-of-the-art approaches. The codes and models are available at https://github.com/Cassie94/LSTD/tree/lstd.

Recent advances in object detection are mainly driven by deep learning with large-scale detection benchmarks. However, the fully-annotated training set is often limited for a target detection task, which may deteriorate the performance of deep detectors. To address this challenge, we propose a novel low-shot transfer detector (LSTD) in this paper, where we leverage rich source-domain knowledge to construct an effective target-domain detector with very few training examples. The main contributions are described as follows. First, we design a flexible deep architecture of LSTD to alleviate transfer difficulties in low-shot detection. This architecture can integrate the advantages of both SSD and Faster RCNN in a unified deep framework. Second, we introduce a novel regularized transfer learning framework for low-shot detection, where the transfer knowledge (TK) and background depression (BD) regularizations are proposed to leverage object knowledge respectively from source and target domains, in order to further enhance fine-tuning with a few target images. Finally, we examine our LSTD on a number of challenging low-shot detection experiments, where LSTD outperforms other state-of-the-art approaches. The results demonstrate that LSTD is a preferable deep detector for low-shot scenarios.

Text recognition has gained significant attention from the computer vision community. Correct character recognition is the premise of text recognition and affects the overall performance to large extent. This paper proposes a novel character representation for scene text recognition. First, a context-based feature that contains local information and relevant key points' feature is extracted from key points. The relativity is measured by the distance of vector that is generated by a trained Gaussian Mixture Model (GMM) between the target key point and other key points in each context bin. In order to recognize each individual character, we adopt a bag-of-words approach, in which the rotation-invariant context features are densely extracted from an individual character. All key points' context features are prone to build a vocabulary of visual words by using k-means clustering. Then we train a set of two-class linear Support Vector Machines in a one-vs-all schema for each category character. By using densely extracted context features that are rotation-invariant and efficient, our method is capable of recognizing perspective texts of arbitrary orientations. The evaluation results on benchmark datasets demonstrate that our proposed scheme of scene character recognition is highly efficient and achieves state-of-the-art performance on not only fontal character recognition but also perspective characters'.

Lines provide important information in images and line detection is crucial in many applications. Many line features can be used to detect line position while line width (i.e., thickness) is a more structured, higher-level feature compared to edge or other line features. Every point of the wide line structure has its own width in spite of the structure's asymmetry. In this paper, we use the parallel edges to get the line orientation and width. We do not need to know the actual width of the line, but rather recover it to get the region of interest (ROI). Then use a modified OTSU obtaining proper region threshold T to segment the wide line structures. By fusing feature of width and gray, a sequence of tests has been conducted on a variety of image samples obtained from simple natural scene and our experimental results demonstrate the practical and robust of the proposed method.

This paper initially develops the discrete-point sampling operator's concept, model, and parameters that we have previously proposed, and makes its belt-shaped regions in a discrete-point sampling map more salient and appropriate for centerline extraction. The cross-sectional features of these belt-shaped regions are then analyzed and seven types of feature points are defined to facilitate descriptions of such features. Based on these feature points, a three-level detection system is proposed, including feature points, line segments, and centerlines, to extract centerlines from the belt-shaped regions. Eight basic types of centerlines and five types of relationships among the centerlines are defined by computational geometry algorithms, and Gestalt laws are used to cluster them into groupings. If some prior information about a desired shape is available, retrieval grouping may be carried out by a discrete-point sampling map, the purpose of which is to find centerlines by best matching with prior information. Discrete-point sampling effectually overcomes the influences of interference from noise, textures, and uneven illumination, and greatly reduces the difficulty of centerline extraction. Centerline clustered groupings and retrieval grouping can offer a strong anti-interference ability with nonlinear deformations such as articulation and occlusion. This method can extract large-scale complex shapes combined of lines and planes from complex images. The wheel location results of noise test and other shape extraction experiments show that our method has a strong capability to persist with nonlinear deformations.

When the scale-invariant feature transform (SIFT) is adopted in the registration of remote sensing images, a lot of incorrect matches of keypoints will appear owing to the significant difference in the image intensity between remote sensing images compared to visible images. Scale–orientation joint restriction criteria are proposed to achieve robust feature matching for keypoints in remote sensing images. Moreover, the feature descriptor of each keypoint is also refined to overcome the difference in the gradient intensity and orientation between remote image pairs. Experimental results for multidate, multispectral, and multisensor remote images indicate that the proposed method improves the match performance compared to intensity- and SIFT-based methods in terms of correct-match rate and aligning accuracy.

Geometric methods are very intuitive and provide a theoretical foundation to many optimization problems in the fields of pattern recognition and machine learning. In this brief, the notion of scaled convex hull (SCH) is defined and a set of theoretical results are exploited to support it. These results allow the existing nearest point algorithms to be directly applied to solve both the separable and nonseparable classification problems successfully and efficiently. Then, the popular S-K algorithm has been presented to solve the nonseparable problems in the context of the SCH framework. The theoretical analysis and some experiments show that the proposed method may achieve better performance than the state-of-the-art methods in terms of the number of kernel evaluations and the execution time.