We challenge the assumption that complex instruction-guided segmentation tasks necessitate equally complex and explicit supervision. This paper introduces RISE (Reasoning via Implicit Self-supervised Emergence), a framework that learns intricate compositional reasoning, spanning spatial relations to world knowledge, without a single ground-truth mask. To achieve this, RISE employs reinforcement learning with GRPO guided by a single, strikingly simple reward: the semantic alignment score between the textual instruction and the predicted image region. Our primary discovery is the implicit emergence of a high-quality chain-of-thought process from this minimalist signal. Within a structured format, the model autonomously learns to understand instructions by accessing its latent knowledge, inferring spatial relationships—capabilities inherent in its architecture but unlocked by our simple objective. Remarkably, our emergent reasoning yields highly competitive results: RISE achieves 58.7 gIoU on the ReasonSeg benchmark, on par with methods using geometric rewards. Furthermore, we show extreme data efficiency: a variant trained on only 2,000 ImageNet-label pairs establishes a new state-of-the-art for annotation-free referring segmentation with 79.6 cIoU on RefCOCO.

Surface-Enhanced Raman Spectroscopy (SERS) combined with deep learning demonstrates considerable potential for liver disease diagnosis. However, acquiring large-scale clinical datasets is challenging due to patient privacy constraints and sample collection complexity, leading to data scarcity that limits deep learning performance. Most existing methods rely heavily on data-driven approaches and fail to effectively utilize prior biomolecular knowledge, making them prone to overfitting. To address these limitations, we present a prior knowledge-guided adaptive multi-scale deep learning model that incorporates literature-validated biomolecular peak positions into feature learning. The model employs a dual-path architecture: an expert-guided path extracts structured features using adaptive multi-scale Gaussian convolutions optimized for distinct biomolecular markers, while a global context path captures comprehensive spectral information. An adaptive fusion mechanism integrates these paths to achieve synergy between prior knowledge and data-driven learning. In a five-class liver disease classification task with 215 subjects, our method achieved 93.66% accuracy, a 5.37% improvement over the baseline convolutional neural network (Baseline CNN, 88.29%). Data constraint experiments demonstrated superior robustness; when training data was reduced to 20%, our approach maintained 86.01% accuracy with a 10.73 percentage point margin over the baseline. Furthermore, independent external validation on a cohort of 35 subjects yielded an overall accuracy of 82.74%, significantly outperforming the Baseline CNN’s 68.23% and reducing the generalization gap from 20.15% to 10.72%, validating the model’s robustness in cross-center clinical scenarios. This work provides an effective integration of domain knowledge with artificial intelligence for Surface-Enhanced Raman Spectroscopy-based medical diagnosis.

Image captioning is a fundamental vision-language task with wide-ranging applications in daily life. The existing methods often struggle to accurately interpret the semantic information in remote sensing images due to the complexity of backgrounds. Target region masks can effectively reflect the shape characteristics of targets and their potential interrelationships. Therefore, incorporating and fully integrating these features can significantly improve the quality of generated captions. However, researchers are hindered by the lack of relevant datasets that contain corresponding object masks. It is natural to ask the following: how to efficiently introduce and utilize object masks? In this article, we provide potential target masks for the publicly available remote sensing image caption (RSIC) datasets, enabling models to utilize the regional features of targets for RSIC. Meanwhile, a novel RSIC algorithm is proposed that combines regional positional features with fine-grained semantic information, abbreviated as$\text {S}^{2}$CPNet. To effectively capture the semantic information from image and position relationship from mask, respectively, the semantic and spatial feature enhancement submodules are introduced at the ends of encoder branches, respectively. Furthermore, the cross-view feature fusion module is designed to integrate regional features and semantic information efficiently. Then, a target recognition decoder is developed to enhance the ability of model to identify and describe critical targets in images. Finally, we improve the caption generation decoder by adaptively merging textual information with visual features to generate more accurate descriptions. Our model achieves satisfactory results on three RSIC datasets compared with the existing method. The related datasets and code will be open-sourced inhttps://github.com/CVer-Yang/SSCPNet.

Road extraction from remote sensing images (RSIs) holds significant application value in various aspects of daily scenarios. However, it is still challenging to extract high-quality road results from RSIs due to the interference of objects sharing similar structures with roads in the background and the occlusion caused by surroundings. To alleviate these problems, a road extraction network based on the global-local Context perception and Cross spatial-scale feature interaction is proposed ($\text {C}^{2}$Net). First, a global-local context perception module (GLCPM) is incorporated to capture the overall topology features of the road, which aims to improve the ability of the model to discriminate between roads and similar objects. Then, the cross spatial-scale feature interaction module is designed in the skip connection to effectively aggregate full-scale features without loss of feature information, which can provide rich and accurate road structural features for the decoder. Experiments conducted on public road datasets demonstrate that$\text {C}^{2}$Net outperforms existing methods in terms of comprehensive metrics such as intersection over union (IoU) and the$F1$-score. The results indicate that$\text {C}^{2}$Net can produce road results with superior connectivity and quality. The source code will be publicly available athttps://github.com/CVer-Yang/CCNet.

Change detection (CD) between heterogeneous images is an increasingly interesting topic in remote sensing. The different imaging mechanisms lead to the failure of homogeneous CD methods on heterogeneous images. To address this challenge, we propose a structure cycle consistency-based image regression method, which consists of two components: the exploration of structure representation and the structure-based regression. We first construct a similarity relationship-based graph to capture the structure information of image; here, a k -selection strategy and an adaptive-weighted distance metric are employed to connect each node with its truly similar neighbors. Then, we conduct the structure-based regression with this adaptively learned graph. More specifically, we transform one image to the domain of the other image via the structure cycle consistency, which yields three types of constraints: forward transformation term, cycle transformation term, and sparse regularization term. Noteworthy, it is not a traditional pixel value-based image regression, but an image structure regression, i.e., it requires the transformed image to have the same structure as the original image. Finally, change extraction can be achieved accurately by directly comparing the transformed and original images. Experiments conducted on different real datasets show the excellent performance of the proposed method. The source code of the proposed method will be made available at https://github.com/yulisun/AGSCC.

The classification of deformable protein shapes, based solely on their macromolecular surfaces, is a challenging problem in protein-protein interaction prediction and protein design. Shape classification is made difficult by the fact that proteins are dynamic, flexible entities with high geometrical complexity. In this paper, we introduce a novel description for such deformable shapes. This description is based on the bifractional Fokker-Planck and Dirac-Kähler equations. These equations analyse and probe protein shapes in terms of a scalar, vectorial and non-commuting quaternionic field, allowing for a more comprehensive description of the protein shapes. An underlying non-Markovian Lévy random walk establishes geometrical relationships between distant regions while recalling previous analyses. Classification is performed with a multiobjective deep hierarchical pyramidal neural network, thus performing a multilevel analysis of the description. Our approach is applied to the SHREC'19 dataset for deformable protein shapes classification and to the SHREC'16 dataset for deformable partial shapes classification, demonstrating the effectiveness and generality of our approach.

Image matching is a primary technology to fuse the complementary information from optical and SAR images. Due to the high nonlinear radiometric and geometric relationship, the optical and SAR image matching task remains a widely unsolved challenge. In this study, we propose to use a Siamese convolutional neural network (CNN) architecture to learn pixelwise deep dense features. The proposed network is able to balance the learning of high-level semantic information and low-level fine-grained information, which is nonnegligible for feature matching task. Under the local searching framework, the loss function is defined based on the score map produced by the sum of squared differences (SSDs) between the learned pixelwise dense features of local optical and the SAR image patches, with a fast implementation in the frequency domain. The hardest negative mining strategy is adopted to increase the discrimination of the network. Extensive experiments are conducted on optical and SAR image pairs of different spatial resolution and different landcover types, verifying the superiority and robustness of the proposed method in terms of matching accuracy and matching precision.

To fully explore the complementary information from optical and synthetic aperture radar (SAR) imageries, they need first to be coregistered with high accuracy. Due to the vast radiometric and geometric disparity, the problem to match high-resolution optical and SAR images is quite challenging. The present deep learning-based methods have shown advantages over the traditional approaches, but the performance increment is not significant. In this article, we explore a better network framework for high-resolution optical and SAR image matching from three aspects. First, we propose an effective multilevel feature fusion method, which helps to take advantage of both the low-level fine-grained features for precious feature location and the high-level semantic features for better discriminative ability. Second, a feature channel excitation procedure is conducted using a novel multifrequency channel attention module, which is able to make image features of different types and multiple levels effectively collaborate with each other and produce image matching features with high diversity. Third, the self-adaptive weighting loss is introduced, with which, each sample is assigned with an adaptive weighting factor, and therefore, information buried in all nearby samples can be better exploited. Under a pseudo-Siamese architecture, the proposed optical and SAR image matching network (OSMNet) is trained and tested on a large and diverse high-resolution optical and SAR dataset. Extensive experiments demonstrate that each component of the proposed deep framework helps to improve the matching accuracy. Also, the OSMNet shows overwhelming superior to the state-of-the-art handcrafted approaches on imageries of different land-cover types.

Protein-protein interactions play a fundamental role in drug design, gene therapy and vaccine development. The study of protein-protein interactions relies heavily on complex and time-consuming experiments, which has a severe impact on research throughputs. Thus, it is important to provide the experimentalist with the most promising cases by screening rapidly through a very large number of potential candidates. We propose a new deep neural network architecture that allows the binding probability for two proteins to be predicted instantly based solely on their amino acid sequences. Subsequently, screenings are performed based on the binding probabilities. The novelty of our approach lies in the fact that we consider self-binding and folding amino acid sequences, rather than just looking at these sequences per se. Our novel Siamese Pyramid Network (SPNet) architecture is inspired by Feature Pyramid Networks and consists of a multi-level Siamese neural network with an attention mechanism and a multilevel, trainable binding probability prediction network. Our experimental evaluation is performed on a strict dataset and shows that SPNet outperforms the state-of-the-art architectures. In addition, we employ SPNet to find the proteins that are most likely to bind with the Covid-2019 spike, thus providing a small and potentially valuable set of candidates for a future therapeutic vaccine.

The scale parameter (SP), to control the sizes of objects, is of great significance in multiscale segmentation which is a prerequisite and foundational step for object-based change detection (OBCD). However, the appropriate SP is not readily apparent and the majority of the existing OBCD algorithms obtain the SPs by empirical or subjective trial-and-error ways that may lead to dissatisfactory accuracy and be time-consuming. To address this issue, an automatic approach for optimal segmentation scale selection for OBCD is proposed in this letter. First, a changed fuzziness image for bitemporal images was generated. Second, multiscale segmentation was implemented in a series of candidate scales, and the merging relationships between adjacent scales were built. Then, mapping the segments to the previous fuzziness image, a statistical metric describing the homogeneity of objects based on the Kullback-Leibler divergence was defined, the increments of the metric between merged objects and their child objects were calculated and weighted to identify the optimal SPs. For performance evaluation, Dempter-Shafer (DS) evidence fusion was utilized in the scales selected by the proposed approach in comparison with other state of art or empirical ones. The experimental results employing GF-1, Google Earth, and aerial images demonstrated the superiority and effectiveness of the SPs identified by the proposed approach in the OBCD task.

Saliency, the distinctive parts of an image, has shown good potential for many applications (e.g., image interpretation and target detection). In this letter, a novel saliency detection method based on background context-aware is proposed for synthetic aperture radar (SAR) images. According to a statistical analysis of SAR image characteristics, several reference background patches (RBPs) are selected. Then, the dissimilarities between the current patch and the RBPs are used to calculate the local saliency, which is further enhanced for the final saliency under the multiscale framework. Experimental results demonstrate the effectiveness of the proposed method, which outperforms some state-of-the-art methods.

This letter presents an effective approach for synthetic aperture radar (SAR) image registration. Corresponding edge features are extracted by exploring the local spatial relationship around corresponding speeded-up robust feature points. Only edge pairs that are matched in the aspect of distance and orientation to the center feature points are taken as effective edge features. For edge-point set matching, we introduce the coherent point drift (CPD) registration method. CPD is able to maintain the global topological structure of edges and is robust to outliers. We refine the original CPD by taking our confidence on edge points as prior information. A new objective function is deduced, resulting in a better registration performance. Experimental results show that the proposed method performs well in terms of registration robustness and accuracy.

TerraSAR-X is a new generation of high-resolution (1 m) space borne Synthetic Aperture Radar (SAR) satellite, which makes ground targets detection (GTD), one of the hot problems in SAR image application fields, possible. As for the mostly used CFAR algorithm for GTD, the statistical analysis of SAR image is an essential step in threshold calculation. However, it has been proved that the traditionally used Rayleigh distribution is no longer suitable to modelling the statistical characteristics of 1 m resolution TerraSAR-X image with ground targets. To model these heavy-tailed distributions, some other distributions, including Gamma, Weibull, lognormal and K distribution models have been applied. According to the analysis of large numbers of experiment results, lognormal distribution model is selected to describe the statistical characteristics of the high resolution SAR images with ground targets. Then genetic algorithm (GA) is adopted to efficiently estimate the model parameters as accurately as possible, which are subsequently used to calculate the global CFAR detection threshold. The experiments indicate that, lognormal distribution model with GA based parameters computation are more suitable than Gamma, Weibull and K distribution models to describe the statistical characteristics of the high resolution SAR images, which is propitious to performance improvement of the global CFAR algorithm.

TerraSAR-X is the first commercially operated high resolution space-borne SAR in the world, which shows good potential in target detection fields. Statistical analysis and modeling are key steps in SAR images based automatic target detection systems. With four typical statistical measures, lognormal distribution is proved well to fit the histograms of TerraSAR-X images over land and ocean regions, and more suitable than Weibull, Gamma, K, G0 and ? stable distributions to modeling statistics of such images. Additionally, the MLE based parameters estimation and simply analytical expression of detection threshold also indicates the effectiveness and efficiency of lognormal based CFAR for target detection.

Feature extraction acts a crucial role in application and research of hyperspectral data and minimum noise fraction (MNF) is one of the most common methods in feature extraction. Estimation of noise covariance matrix is an inevitable step of MNF, but it would bring error which would lead imprecise while using the first some components of MNF transform to represent original data. To solve the problem above, a feature extraction method based on MNF and singular value decomposition was proposed. MNF was first acted on data, then, the transformed data were decomposed by singular value decomposition, and the first some components of reconstruction used singular values and singular vectors were selected as feature components of original data. Experiments with factual hyperspectral data indicated that classified precision after feature extraction by the proposed method was higher than those by traditional MNF in three classified methods and different dimension.