Unpaired image restoration (UIR) is a significant task due to the difficulty of acquiring paired degraded/clear images with identical backgrounds. In this paper, we propose a novel UIR method based on the assumption that an image contains both degradation-related features, which affect the level of degradation, and degradation-unrelated features, such as texture and semantic information. Our method aims to ensure that the degradation-related features of the restoration result closely resemble those of the clear image, while the degradation-unrelated features align with the input degraded image. Specifically, we introduce a Feature Orthogonalization Module optimized on Stiefel manifold to decouple image features, ensuring feature uncorrelation. A task-driven Depth-wise Feature Classifier is proposed to assign weights to uncorrelated features based on their relevance to degradation prediction. To avoid the dependence of the training process on the quality of the clear image in a single pair of input data, we propose to maintain several degradation-related proxies describing the degradation level of clear images to enhance the model's robustness. Finally, a weighted PatchNCE loss is introduced to pull degradation-related features in the output image toward those of clear images, while bringing degradation-unrelated features close to those of the degraded input.

Early smoke segmentation (ESS) enables the accurate identification of smoke sources, facilitating the prompt extinguishing of fires and preventing large-scale gas leaks. But ESS poses greater challenges than conventional object and regular smoke segmentation due to its small scale and transparent appearance, which can result in high miss detection rate and low precision. To address these issues, a Focus and Separation Network (FoSp) is proposed. We first introduce a Focus module employing bidirectional cascade which guides low-resolution and high-resolution features towards mid-resolution to locate and determine the scope of smoke, reducing the miss detection rate. Next, we propose a Separation module that separates smoke images into a pure smoke foreground and a smoke-free background, enhancing the contrast between smoke and background fundamentally, improving segmentation precision. Finally, a Domain Fusion module is developed to integrate the distinctive features of the two modules which can balance recall and precision to achieve high F_beta. Futhermore, to promote the development of ESS, we introduce a high-quality real-world dataset called SmokeSeg, which contains more small and transparent smoke than the existing datasets. Experimental results show that our model achieves the best performance on three available smoke segmentation datasets: SYN70K (mIoU: 83.00%), SMOKE5K (F_beta: 81.6%) and SmokeSeg (F_beta: 72.05%). The code can be found at https://github.com/LujianYao/FoSp.

Smoke segmentation is of great importance in precisely identifying the smoke location, enabling timely fire rescue and gas leak detection. However, due to the visual diversity and blurry edges of the non-grid smoke, noisy labels are almost inevitable in large-scale pixel-level smoke datasets. Noisy labels significantly impact the robustness of the model and may lead to serious accidents. Nevertheless, currently, there are no specific methods for addressing noisy labels in smoke segmentation. Smoke differs from regular objects as its transparency varies, causing inconsistent features in the noisy labels. In this paper, we propose a conditional sample weighting (CoSW). CoSW utilizes a multi-prototype framework, where prototypes serve as prior information to apply different weighting criteria to the different feature clusters. A novel regularized within-prototype entropy (RWE) is introduced to achieve CoSW and stable prototype update. The experiments show that our approach achieves SOTA performance on both real-world and synthetic noisy smoke segmentation datasets.

In the image dehazing task, the haze density is a key feature that affects the performance of dehazing methods. The haze density difference, which has rarely been utilized in previous methods, can guide networks to perceive different global densities and focus on local areas with high density or that are difficult to dehaze. In this paper, we propose a density-aware dehazing method named the Density Feature Refinement Network (DFR-Net), which extracts haze density features from density differences and leverages density differences to refine density features. In DFR-Net, we first generate a proposal image that has a lower overall density than the hazy input, resulting in global density differences. Additionally, the dehazing residual of the proposal image reflects the level of dehazing performance and provides local density differences that indicate localized hard dehazing or high-density areas. Subsequently, we introduce a Global Branch (GB) and a Local Branch (LB) to achieve density awareness. In GB, we use Siamese networks for feature extraction of hazy inputs and proposal images, and we propose a Global Density Feature Refinement (GDFR) module that can refine features by pushing features with different global densities further away. In LB, we explore local density features from the dehazing residuals between hazy inputs and proposal images and introduce an Intermediate Dehazing Residual Feedforward (IDRF) module to update local features and pull them close to clear image features. Sufficient experiments demonstrate that the proposed method outperforms state-of-the-art methods on various datasets.

Undoubtedly, the object is the primary factor in 3D single-object tracking (SOT) tasks. However, prior Siamese-based trackers overlook the adverse effects resulting from randomly dropped object points during backbone sampling, hindering the prediction of accurate bounding boxes (BBoxes). Therefore, developing an approach that maximizes the preservation of object points and their object-aware features is of the utmost significance. To address this, we propose an object-preserving Siamese network (OPSNet) that can effectively maintain object integrity and boost tracking performance. First, anobject highlighting moduleamplifies the object-aware features and extracts discriminative features from the template and search area. Next,object-preserving samplingselects object candidates, obtains object-preserving search area seeds, and discards background points that have less impact on tracking. Finally, anobject localization networkaccurately locates 3D BBoxes based on the object-preserving search area seeds. Extensive experiments demonstrate that the performance of OPSNet exceeds the state-of-the-art performance, achieving success gains of$\sim$9.4% and$\sim$2.5% on the KITTI and Waymo Open datasets, respectively.

In the traditional fault diagnosis task, it is difficult to collect training samples to exhaust all fault classes. There are massive target faults that cannot be collected in advance, which may restrict the performance of fault diagnosis methods. In this article, a novel method named semantic-consistent embedding (SCE) is proposed for zero-shot industrial fault diagnosis. SCE tries to classify unseen class faults only by using seen class faults for training. The fault samples and their human-specified attribute vectors are embedded into a semantic-consistent space and then reconstructed from that space. A specificBarlow matrixis designed to measure the consistency between the embedding of fault samples and the embedding of attribute vectors. The diagonal elements and the off-diagonal elements of the Barlow matrix encode the within-dimension consistency and between-dimension consistency of the cross-modal embeddings, respectively. Through optimizing the Barlow matrix to an identity matrix, SCE learns a significant space where the cross-modal embeddings have consistent representation while reducing the redundant components. Extensive experiments show that SCE gets significant superiority on the three-phase transmission system (26.9% gains) and the Tennessee Eastman process (15.5% gains). Moreover, SCE even gets competitive results with supervised learning methods.

Since both visible and infrared images have their own advantages and disadvantages, RGBT tracking plays an important role in intelligent transportation systems. The key points of RGBT tracking lie in feature extraction and fusion of visible and infrared images. Current RGBT tracking methods mostly pay attention to both individual features (features extracted from images of a single camera) and common features (features extracted and fused from an RGB camera and a thermal camera). Still, they pay less attention to different and dynamic contributions of the individual and common features for different sequences of registered image pairs. This paper proposes a novel RGBT tracking method, called Dynamic Fusion Network (DFNet), which adopts a two-stream structure, in which two non-shared convolution kernels are employed in each layer to extract individual features. Besides, DFNet has shared convolution kernels for each layer to extract common features. Since non-shared and shared convolution kernels are adaptively weighted and summed according to different image pairs, DFNet can deal with different contributions for different sequences. DFNet has a fast speed, which is 28.658 FPS. The experimental results show that when DFNet only increases the Mult-Adds by 0.02% compared with the non-shared-convolution-kernel-based fusion method, Precision Rate (PR) and Success Rate (SR) reach 88.1% and 71.9%, respectively. The model and dataset are available athttps://github.com/PengJingchao/DFNet.

We consider the problem of assigning single-band thermal night vision image with natural day-time color appearance automatically. We present an approach in which supervised learning is first used to estimate colors of monochromic images. Modeling color distribution of thermal imagery is a challenging problem, since there are insufficient local features for estimating the chromatic value at a point. Our model uses a statistical learning algorithm that incorporates multi-scale and spatially arranged image features, and it can be trained on a data set that contains thermal image and registered day-time color image pairs. Experimental results show that our approach leads to relatively accurate description of the desired color distribution and results in thermal images that appear smooth and natural color details, so that the overall scene recognition and situational awareness can be improved.

In the past few years, the computer vision and pattern recognition community has witnessed a rapid growth of a new kind of feature extraction method, the manifold learning methods, which attempt to project the original data into a lower dimensional feature space by preserving the local neighborhood structure. Among these methods, locality preserving projection (LPP) is one of the most promising feature extraction techniques. Based on LPP, this paper proposes a novel feature extraction algorithm, Optimal Locality Preserving Projection (Optimal LPP). Optimal here means that the extracted features are statistically uncorrelated and orthogonal, which are desirable for pattern analysis applications. We compare the proposed Optimal LPP with LPP, Orthogonal Locality Preserving Projection (OLPP) and Uncorrelated Locality Preserving Projection (ULPP) on the public available data sets, FERET and CMU PIE data sets. Experimental results show that the proposed Optimal LPP achieves much higher recognition accuracies.

Dimensionality reduction methods have been successfully employed for face recognition. Among the various dimensionality reduction algorithms, linear (Fisher) discriminant analysis (LDA) is one of the popular supervised dimensionality reduction methods, and many LDA-based face recognition algorithms/systems have been reported in the last decade. However, the LDA-based face recognition systems suffer from the scalability problem. To overcome this limitation, an incremental approach is a natural solution. The main difficulty in developing the incremental LDA (ILDA) is to handle the inverse of the within-class scatter matrix. In this paper, based on the generalized singular value decomposition LDA (LDA/GSVD), we develop a new ILDA algorithm called GSVD-ILDA. Different from the existing techniques in which the new projection matrix is found in a restricted subspace, the proposed GSVD-ILDA determines the projection matrix in full space. Extensive experiments are performed to compare the proposed GSVD-ILDA with the LDA/GSVD as well as the existing ILDA methods using the face recognition technology face database and the Carneggie Mellon University Pose, Illumination, and Expression face database. Experimental results show that the proposed GSVD-ILDA algorithm gives the same performance as the LDA/GSVD with much smaller computational complexity. The experimental results also show that the proposed GSVD-ILDA gives better classification performance than the other recently proposed ILDA algorithms.

Color night vision techniques play a very important role in the night vision field. How to evaluate the perceptual quality of the color night vision image is a great need to assess the performance of algorithms in this technology. Currently, people usually judge the performance of color night vision techniques using subjective evaluation measures, which is time consuming and bothersome. This paper proposes an objective evaluation metric based on affective information and harmonious feeling to assess the color appearance of the color night vision image quality. This objective index is computed based on power spectra density in a decorrelated ιαβ color space and reflects a fluctuation of the image’s color. And this kind of fluctuation consists with the feeling of human vision. Subjective evaluation is also investigated in this paper and the experimental results confirm that the proposed objective metric correlates with the subjective evaluation well.

Recently, a number of studies have demonstrated that thermal infrared (IR) imagery offers a promising alternative to visible imagery in face recognition problems due to its invariance to visible illumination changes. However, thermal IR has other limitations including that it is opaque to glass. As a result, thermal IR imagery is very sensitive to facial occlusion caused by eyeglasses. Fusion of the visible and thermal IR images is an effective way to solve this problem. In this paper, using the face reconstruction information of the visible images, we propose a nonlinear eyeglasses removing algorithm which can successfully reconstruct the thermal images. Experiments on publicly available data set show the excellent performance of our algorithm.

Principal component analysis (PCA) has been proven to be an efficient method in pattern recognition and image analysis. Recently, PCA has been extensively employed for face-recognition algorithms, such as eigenface and fisherface. The encouraging results have been reported and discussed in the literature. Many PCA-based face-recognition systems have also been developed in the last decade. However, existing PCA-based face-recognition systems are hard to scale up because of the computational cost and memory-requirement burden. To overcome this limitation, an incremental approach is usually adopted. Incremental PCA (IPCA) methods have been studied for many years in the machine-learning community. The major limitation of existing IPCA methods is that there is no guarantee on the approximation error. In view of this limitation, this paper proposes a new IPCA method based on the idea of a singular value decomposition (SVD) updating algorithm, namely an SVD updating-based IPCA (SVDU-IPCA) algorithm. In the proposed SVDU-IPCA algorithm, we have mathematically proved that the approximation error is bounded. A complexity analysis on the proposed method is also presented. Another characteristic of the proposed SVDU-IPCA algorithm is that it can be easily extended to a kernel version. The proposed method has been evaluated using available public databases, namely FERET, AR, and Yale B, and applied to existing face-recognition algorithms. Experimental results show that the difference of the average recognition accuracy between the proposed incremental method and the batch-mode method is less than 1%. This implies that the proposed SVDU-IPCA method gives a close approximation to the batch-mode PCA method.

Fisher Linear Discriminant Analysis (LDA) has been successfully used as a data discriminantion technique for face recognition. This paper has developed a novel subspace approach in determining the optimal projection. This algorithm effectively solves the small sample size problem and eliminates the possibility of losing discriminative information. Through the theoretical derivation, we compared our method with the typical PCA-based LDA methods, and also showed the relationship between our new method and perturbation-based method. The feasibility of the new algorithm has been demonstrated by comprehensive evaluation and comparison experiments with existing LDA-based methods.

In the real world, learning is often expected to be a continuous process, which is capable of incorporating new facts into the past experience. However, currently many typical face recognition methods, such as eigenface and Fisherface, have only focused on non-incremental learning tasks, where the learning stops once the training set has been duly processed. In this paper, we present a PCA-based algorithm for face recognition, which takes the incremental learning in account. This method can update the principal subspace without simply re-computing the eigen decomposition from scratch.