Fusing Visual-Depth-Thermal (VDT) data holds immense potential for robust Salient Object Detection (SOD) in complex environments. However, current research is constrained by dataset scarcity and the limitations of symmetric direct fusion strategies. To address these gaps, we first construct a comprehensive benchmark named LiTR-2654, comprising 2,654 spatially aligned VDT image triplets captured via LiDAR and dual-modality cameras. This dataset features high diversity and reduced center bias, designed to advance practical applications. With this benchmark, we propose the Asymmetric Feature Consistency Reinforcement Network (AFCRNet), effectively utilizing triple-modality cues to achieve accurate SOD. AFCRNet comprises mainly two core technical innovations: "Unify-then-Integrate" fusion strategy investigates modality-complementary information and context-guided decoder module enables the common focus of multi-level features. Specifically, cross-level thermal and visual features are densely interacted to obtain consistent feature representations. Meanwhile, taking depth features as supplements, same-level triple-modality features are integrated with the attention mechanism, significantly suppressing complex background interference and highlighting salient objects. To further improve the segmentation accuracy, high-level contextual information is introduced into multi-level features to accurately distinguish salient objects, and edge supervision is also utilized to optimize the object contour. Comprehensive analysis of different methods is conducted on published and self-built datasets, demonstrating the superiority of the proposed method. The constructed novel benchmark will be made publicly available at: github.com/215HH/LiTR-2654.

Infrared small target detection (IRSTD) plays a crucial role in numerous applications and has benefited significantly from recent advances in deep learning. However, existing methods often face two major challenges: (1) difficulty in detecting non-salient small targets embedded in complex backgrounds, and (2) substantial computational costs during inference, which limit practical deployment. To address these issues, we propose the Frequency Introduced Cascade Feature Fusion Network (FICFFNet), which leverages high-frequency information from infrared images to enhance target saliency, and incorporates a knowledge distillation (KD) framework to balance detection accuracy and efficiency. Specifically, infrared and frequency features are jointly extracted and adaptively fused, enabling effective integration of spatial and frequency-domain cues and improving localization of tiny objects with local temperature variations. Adjacent-level features are further aggregated to combine low-level spatial details with high-level contextual semantics, ensuring robust multi-scale prediction. A lightweight student network is constructed by simplifying the architecture, while multiple distillation paths transfer crucial knowledge from the dual-stream features and prediction maps to the single-stream representation. We also construct two infrared tiny bird detection datasets by collecting and annotating infrared images from transformer substations. Extensive experiments on both self-built and public datasets demonstrate that FICFFNet achieves superior detection performance while maintaining high computational efficiency. Our code will be made public at: github.com/HHUyxt/IRTBD.

Most CNN models exhibit two major flaws in hyper-spectral image (HSI) restoration tasks. First, limited high-dimensional HSI training examples exacerbate the difficulty of deep learning methods in learning effective spatial and spectral representations. Second, the existing CNN-based methods model local relations and present limitations in capturing long-range dependencies. In this paper, we customize a novel dual-stream Transformer (DSTrans) for HSI restoration, which mainly consists of the dual-stream attention and the dual-stream feed-forward network. Specifically, we develop the dual-stream attention consisting of Multi-Dconv-head spectral attention (MDSA) and Multi-head Spatial self-attention (MSSA). MDSA and MSSA respectively calculate self-attention along the spectral and spatial dimensions in local windows to capture long-range spectrum dependencies and model global spatial interactions. Meanwhile, the dual-stream feed-forward network is developed to extract global signals and local details in parallel branches. In addition, we exploit a multi-tasking network to train the auxiliary RGB image (RGBI) task and HSI task jointly so that both numerous RGBI samples and limited HSI samples are exploited to learn parameter distribution for DSTrans. Extensive experimental results demonstrate that our method achieves state-of-the-art results on HSI restoration tasks, including HSI super-resolution and denoising. The source code can be obtained at: https://github.com/yudadabing/Dual-Stream-Transformer-for-Hyperspectral-Image-Restoration.

Real-time scene comprehension is the basis for automatic electric power inspection. However, existing RGB-based scene comprehension methods may achieve unsatisfied performance when dealing with complex scenarios, insufficient illumination or occluded appearances. To solve this problem, by cooperating visual and thermal images, the Dual-Space Graph-based Interaction Network (DSGBINet) is proposed to achieve all-day time semantic segmentation of power equipment in high-voltage power transmission line and electric transformer substation scenes. Specifically, modality-specific features are first extracted via two separate backbone networks with the same architecture. Multi-modality high-level features are first fused via long-range relationship in coordinate space. Then, multi-modality features from regular grids are further clustered and assigned to vertices in feature space. Cross-graph and inner-graph regional relations are utilized for reasoning and enhancement, which could exploit the mutual benefits and extract rich contextual information in a semantic view. Furthermore, to overcome the huge scale difference and the inherent characteristic of thermal images, the idea of multi-task learning is integrated into the decoding process. The edge detection and semantic segmentation are achieved collaboratively, which could segment the different power equipment more accurately and completely. The comparative and ablation experiments on the proposed two RGB-T semantic segmentation datasets evaluate the effectiveness and robustness of the proposed network compared with existing state-of-the-art methods. The extended experiments on the public datasets further demonstrate the superiority of the proposed method. Our dataset and code will be released at:https://github.com/hhujiang/DSGBINet.

Convolutional neural networks (CNNs) have dominated the research of hyperspectral image (HSI) classification, attributing to the superior feature representation capacity. Patch-free global learning (FPGA) as a fast learning framework for HSI classification has received wide interest. Despite their promising results from the perspective of fast inference, recent works have difficulty modeling spectral-spatial relationships with imbalanced samples. In this paper, we revisit the encoder–decoder-based fully convolutional network (FCN) and propose a cross-level spectral-spatial joint encoding framework (CLSJE) for Imbalanced HSI classification. First, a multi-scale input encoder and multiple-to-one multi-scale features connection are introduced to obtain abundant features and facilitate multi-scale contextual information flow between encoder and decoder. Second, in the encoder layer, we propose the spectral-spatial joint attention (SSJA) mechanism consisting of the high-frequency spatial attention (HFSA) and spectral-transform channel attention (STCA). HFSA and STCA encode spectral-spatial features jointly to improve the learning of the discriminative spectral-spatial features. Powered by these two components, CLSJE enjoys a high capability to capture both spatial and spectral dependencies for HSI classification. Besides, a class-proportion sampling strategy is developed to increase the attention to insufficiency samples. Extensive experiments demonstrate the superiority of our proposed CLSJE both at classification accuracy and inference speed, and show the state-of-the-art results on four benchmark datasets. Code can be obtained at: https://github.com/yudadabing/CLSJE.

Abstract The quality of high energy flash X‐ray images is crucial to the high‐precision diagnosis of object density. High energy flash X‐ray radiography is susceptible to the system blur, which usually causes the poor quality of static images. In response to this, a novel restoration algorithm using region extrema and kernel optimization (REKO) is presented. Based on the observation that the region extrema distribution of blurred high energy flash X‐ray images deviates from opposite ends of image grey domain, the sparseness‐inducing prior for regularizing image region extrema is applied to construct the restoration model. Considering the sparse characteristics of blur kernels, the sparseness‐inducing regularization is incorporated to constrain blur kernels in the restoration model. The non‐convex and non‐linear objective function is gradually minimized through energy alternating minimization and dually linear approximation. Furthermore, a continuity enforced kernel optimization algorithm is proposed to estimate more accurate blur kernels. The discontinuous kernel elements are suppressed by extracting the main structure of blur kernels and constructing kernel continuity function in cross windows. Experimental results demonstrate that our algorithm can more accurately estimate blur kernels and achieve restoration results with sharper edges on high energy flash X‐ray images.

Flash Radiography inspections stand to gain from inversion to infer density distribution of object based on X-ray transmission image. It is indispensable to be able to reliably provide uncertainties associated with the inversions. Although many inversion algorithms have been devised, they often perform poorly due to either their sensitivity to regularization parameter chosen in variational optimization or prohibitive computation and noisy results in stochastic simulation. In this paper, we present a gradual reconstruction algorithm, called TLE-Gibbs (two-level efficient Gibbs sampling), for flash radiography. At its core, TLE-Gibbs is a stochastic approach based on efficient Gibbs sampling and reconstruction refinement. A two-level scheme is proposed that enables high-resolution image to be constrained with uncertainty estimation from high-level reconstruction. Furthermore, a splitting variant that increases flexibility and precision is considered in the two-level scheme. An efficient Markov chain Monte Carlo (MCMC) endowed with first-order truncated conjugate gradient (CG) optimizer is developed to achieve minimal cost per sample and to approximate the posterior distribution. Finally, we adopt an effective refinement method to remove noises remained in the sample meanwhile maintaining sharp edges. For performance evaluation, TLE-Gibbs is applied on both synthetic data in which the influence of system blur is specially investigated and real data, and comparison with state-of-the-art reconstruction methods demonstrates the superiority of the proposed method.

Image non-blind deblurring is still an ill-posed problem. Uncertainty in solutions occurs when singular vectors of forward model matrix spanning the noise subspace have rather small singular values. This letter proposes a new image deblurring algorithm, called MNBC-Gibbs (multiple norms and boundary constraint enforced Gibbs sampling). To be more specific, the quadratic and sparseness-inducing norms are combined to construct regularization term, and the objective function is gradually minimized without requirement of regularization parameter choice. In particular, we propose an efficient Markov chain Monte Carlo (MCMC) method equipped with closed-form solution, artifacts processing and non-negative constraint to approximate the posterior distribution and estimate uncertainty for the unknown. Satisfactory deblurring results with sharp edges can be generated while maintaining smoothness without raising extra noise. The quantitative evaluations on different blur kernels and comparison with state-of-the-art image deblurring methods demonstrate the superiority of the proposed method. In addition, we show that our method can effectively deal with real blurry images.

The existing size measurement systems cannot meet the requirements of non-contact measurement tasks, and the main challenge is how to detect various objects and improve measurement accuracy. In order to solve these problems, a novel measurement system was proposed in this paper. In the system, a three-camera model with variable baselines was designed based on the trinocular vision. Three binocular vision subsystems were composed of the three cameras, which were used to obtain the depth information from different shooting angles, and the baselines between the cameras could be adjusted according to the different objects. In the measurement process, the target object was detected automatically based on the visual saliency features and spatial information. Finally, the size of the target object was computed by the cooperative analysis of the three binocular vision subsystems. The experimental results demonstrated that the proposed system is accurate and stable in various objects' detection and measurement tasks.

Since the labelling for the positive images/videos is ambiguous in weakly supervised segment annotation, negative mining‐based methods that only use the intra‐class information emerge. In these methods, negative instances are utilised to penalise unknown instances for ranking their likelihood of being an object, which can be considered as voting in terms of similarity. However, these methods (i) ignore the information contained in positive bags; (ii) only rank the likelihood but cannot generate an explicit decision function. In this study, the authors propose a voting scheme involving not only the definite negative instances but also the ambiguous positive instances to make use of the extra useful information in the weakly labelled positive bags. In the scheme, each instance votes for its label with a magnitude arising from the similarity, and the ambiguous positive instances are assigned soft labels that are iteratively updated during the voting. It overcomes the limitations of voting using only the negative bags. They also propose an expectation kernel density estimation algorithm to gain further insight into the voting mechanism. Experimental results demonstrate the superiority of the authors’ scheme beyond the baselines.

In this paper, we propose a multiple-instance discriminant analysis algorithm for weakly supervised segment annotation. We introduce a selection parameter for each image/video with weak labels and expect that it can sift out object regions from the background clutter to train a better transformation vector. The selection parameter and the transformation parameter are incorporated into a single objective function and optimized in an alternate way. The optimization is an iteration between the eigenvalue decomposition and a set of quadratic programming. We also integrate a regularization term into the objective function to formulate the spatial constraint of segments, which is ignored in ordinary multiple-instance learning methods. The algorithm is able to overcome the limitations that arise when applying ordinary multiple-instance methods to the task. The experimental results validate the effectiveness of our method.

Automatic object annotation for weakly labelled images/videos has attracted great research interests. In the literature, the idea of negative mining has been proposed for the task. Following existing works, the authors start with image/video over‐segmentation. With the assumption that the noisy segments in the concept images and the strongly labelled non‐concept segments are drawn from the same distribution, the authors plan to estimate the non‐concept distribution and apply it to the ambiguous segments to generate a concept ranking. Although this idea was proposed in existing work and was shown ineffective when combined with a naive kernel density estimation strategy, in this study, the authors explore improved density estimation techniques for the ranking and propose a kernel regression model whose parameters are estimated by a maximum likelihood estimation. Experimental results validate the effectiveness of their method.

Sidescan sonar image segmentation is a very important issue in underwater object detection and recognition. In this paper, a robust and fast method for sidescan sonar image segmentation is proposed, which deals with both speckle noise and intensity inhomogeneity that may cause considerable difficulties in image segmentation. The proposed method integrates the nonlocal means-based speckle filtering (NLMSF), coarse segmentation using k -means clustering, and fine segmentation using an improved region-scalable fitting (RSF) model. The NLMSF is used before the segmentation to effectively remove speckle noise while preserving meaningful details such as edges and fine features, which can make the segmentation easier and more accurate. After despeckling, a coarse segmentation is obtained by using k -means clustering, which can reduce the number of iterations. In the fine segmentation, to better deal with possible intensity inhomogeneity, an edge-driven constraint is combined with the RSF model, which can not only accelerate the convergence speed but also avoid trapping into local minima. The proposed method has been successfully applied to both noisy and inhomogeneous sonar images. Experimental and comparative results on real and synthetic sonar images demonstrate that the proposed method is robust against noise and intensity inhomogeneity, and is also fast and accurate.

In this paper, an unsupervised seafloor segmentation method using combined texture features of sidescan sonar images is proposed. Two sets of features are considered in the proposed algorithm. One calculates the statistics from the gray-level co-occurrence matrix (GLCM), and the other obtains the statistics in the nonsubsampled contourlet transform domain (NSCT). The two sets of features are combined together to produce a multi-dimensional feature vector for each pixel. Principal component analysis (PCA) is used to reduce the dimensionality of each feature vector. The Silhouette index is adopted to automatically estimate the number of seafloor types in sonar images. The segmentation is achieved using k-means clustering based on the compact feature vectors. Experimental results show that the proposed method can improve the seafloor segmentation accuracy.