The distribution discrepancy metrics are the core foundation of achieving domain confusion. Therefore, they mainly determine the performance of deep transfer diagnosis models. However, their effectiveness relies on the stability of data local distributions, making them unsuitable for cross-domain machine diagnosis tasks under continuous time-varying conditions. Hence, a new integrated-dispersion manifold distance (IDMD) is proposed to enhance the discrepancy representation capability in dynamic data structures. The maximum entropy-based local distribution (MELD) selection mechanism is designed to represent the global distribution information of time-varying monitoring signals adaptively. Furthermore, the ensemble Grassmann manifold geodesic (EGMG) measurement is constructed to characterize the intrinsic distribution discrepancy information due to complex nonlinear structures of high-dimensional data. The proposed IDMD distribution discrepancy metric is validated against two fault transfer diagnosis experiments under time-varying conditions, including laboratory planetary gearboxes and actual wind turbine bearings. The experimental results demonstrate its effectiveness and advantage over the existing advanced methods.

Transfer learning generally addresses to reduce the distribution distance between source-domain and target-domain. However, it is unreasonable to use a distribution to represent the life-cycle signals as they are always time-varying, and the improper assumption affects the efficacy of transfer remaining useful life (RUL) prediction. To fill this gap, this research proposes a micro transfer learning mechanism for multiple differentiated distributions, and a transfer RUL prediction model is constructed. First, a multi-cellular long short-term memory (MCLSTM) neural network is applied to obtain multiple differentiated distributions of the monitoring data at some point. Then the domain adversarial mechanism is used to achieve the knowledge transfer of multiple differentiated distributions at the cell level. Furthermore, an active screen mechanism is designed for weighting the domain discrimination losses of multiple differentiated distributions. Through the transfer RUL prediction experiments on aero-engines and actual wind turbine gearboxes, the superiority of this model over the advanced transfer prediction models is verified.Note to Practitioners—The work is motivated by the accuracy reduction problem caused by the time-varying characteristics of life-cycle data in the cross domain equipment RUL prediction scenario, where a fixed single distribution is difficult to cover the full life-cycle data. This article proposes a micro transfer learning mechanism containing multiple differentiated distributions, and a novel transfer RUL prediction model based on the mechanism is constructed for solving the problem caused by the time-varying characteristics of life-cycle data. There are four steps for implementing this method in practice: 1) collecting the full-life cycle signals of historical equipment; 2) modeling the degradation curves of equipment by MCLSTM; 3) solving the cross domain RUL prediction by narrowing the distributions of degradation curves by the micro transfer learning mechanism; and 4) making prognostics for new equipment. The novelty is that the proposed mechanism can self-adaptively align multiple differentiated subspaces of the source domain and the target domain, that is, it can adaptively extract the domain invariant features over time. As a result, the proposed method has two main advantages: 1) capable of characterizing the degradation processes of different equipment; and 2) superior prognostic results on cross domain RUL prediction.

Low-light images often suffer from varying degrees of visual degradation. Current methods for recovering image texture details fail to rely on the self-adaptive correlation texture direction of the image itself, which leads the network to be unable to address the local texture characteristics of different images. To address this challenge, we propose a semantic-aware detail adaptive network (SDANet) that fully considers the image detail information. The network divides low-light images into high-frequency and low-frequency parts. Learning different forms of noise through a novel total variation regularization module with adaptive weights ensures that the final high-frequency part adequately integrates the texture information of the image. Simultaneously, a detail-adaptive module is incorporated to restore finer details in the resulting image. SDANet not only effectively suppresses noise in real low-light images while considering texture details but also effectively addresses the degradation of visible information, and it performs better than other state-of-the-art methods. The code is available athttps://github.com/cheer79/SDANet.

Underwater images usually exhibit severe color cast, hazy appearance, and/or dark regions because of the complex lighting absorption and scattering in water. How to increase the quality of these degraded underwater images has emerged as a key issue for various underwater application tasks. Recent efforts have been made to deal with single type degradation, however, it is still challenging to deal with multiple degradations that usually coexist in an underwater image with a general network. The degradations in underwater images can be divided into medium-agnostic (hazy or low-light which also encountered in in-air images) and medium-specific (color distortion caused by the specific light attenuation property in water) ones. According to this observation, this article proposes a cascaded multimodule underwater image enhancement (UIE) framework to address the coexisted multiple degradations. In the proposed framework, an in-air image enhancement module and a novel proposed adaptive color channel compensation network (AC3Net) are cascaded, in which the former focuses primarily on solving medium-agnostic degradations and the latter is for handling the medium-specific degradation. This framework has good flexibility by cascading different types of in-air image enhancement networks with AC3Net to achieve various UIE. The effectiveness of the proposed framework has been extensively validated on various degraded underwater images as well as different underwater visual perception tasks.

Many transfer learning methods have been proposed to implement fault transfer diagnosis, and their loss functions are usually composed of task-related losses, distribution distance losses, and correlation regularization losses. The intrinsic parameters and trade-off parameters between losses, however, need to be tuned according to the specific diagnosis tasks; thus, the generalization abilities of these methods in multiple tasks are limited. Besides, the alignment goal of most domain adaptation (DA) mechanisms dynamically changes during the training process, which will result in loss oscillation, slow convergence and poor robustness. To overcome the above-mentioned issues, a novel and simple transfer learning diagnosis method named adaptive intermediate class-wise distribution alignment (AICDA) model is proposed, and it is established via the proposed AICDA mechanism, dynamic intermediate alignment (DIA) adaptive layer and AdaSoftmax loss. The AICDA mechanism develops an adaptive intermediate distribution as the alignment goal of multiple source domains and target domains, and it can simultaneously align the global and class-wise distributions of these domains. The DIA layer is designed to adaptively achieve domain confusion without the distribution distance loss and the correlation regularization loss. Meanwhile, to ensure the classification performance of the AICDA mechanism, AdaSoftmax loss is proposed for boosting the separability of Softmax loss. Finally, in order to evaluate the effectiveness and universality of the AICDA diagnosis model to the most degree, various multisource mixed fault transfer diagnosis tasks of wind turbine planetary gearboxes, including DA and domain generalization (DG), are implemented, and the experimental results indicate that our proposed AICDA model has a higher diagnosis accuracy and a stronger generalization ability than other state-of-the-art transfer learning methods.

Fully connected (FC) layers play a significant role in deep neural networks (DNNs) models. Owing to the complexity of its parameters, an FC layer has sufficient capacity to manage high-dimensional tasks, so a large amount of memory and powerful computing capabilities become essential requirements. However, the large number of parameters in an FC layer greatly limits the practical application of this model. To address this problem, we apply matrix optimization to an FC layer. First, an added penalty term properly maintains the sparsity of the imposed weights. Second, a rank constraint is applied to the two components of the factorized weight matrix. Our compression algorithm can effectively reduce the number of required network parameters, which not only reduces the computational complexity of the network but also results in better generalizability on a test dataset. Finally, the effectiveness of the proposed method is verified in two different computer vision task domains. Experiments show that our sparse reduced-rank method achieves a better compression ratio with a lower accuracy loss relative to the competing approaches. The code is available at https://github.com/cheer79/Compress_FC .

In this article, we propose an end-to-end Transformer-based and structure-aware dual-stream network for low-light image enhancement. First, we divide the dual-stream network into a main stream and a structure stream. The main stream is used to recover an enhanced image and supply structural information to the structure stream, whereas the structure stream is designed to extract structural features from the main stream to provide rich structural information for the enhancement process. Second, we devise a structure-gated Transformer to balance the extraction of global and local features through parallel multihead self-attention with convolution operations following a multilayer perceptron, thus extracting sufficient structural information from the main stream in the encoder part of the dual-stream network. Finally, we develop a cross-attention-based feature fusion module that divides different window sizes into distinct feature fusion stages to achieve multistage, multiscale and multitype feature fusion in the decoder part of the dual-stream network. The experimental results show that our method not only works effectively but also offers favorable computational and storage overheads.

Aiming at the large differences between tasks in continuous remaining useful life (RUL) prediction and the limited information capturing capability of the existing continuous learning (CL) methods, this article develops a novel multibranch horizontal augmentation network (MBHAN). First, a hierarchical self-attention (HSA) mechanism is proposed to capture the local degradation features and dependencies at different scales and enhance the representation capacity of RUL prediction model. Based on HSA and temporal convolutional network (TCN), a time-frequency fusion TCN (TFFTCN) is designed to mine the hidden degradation information from the time-domain and frequency-domain data. Then, a memory weight constraint (MWC) regularization term is built to control the update of important parameters for pervious tasks during the learning of new task. A horizontal network augmentation rule based on the task similarity and MWC is proposed, including the augmentation of a task branch network for small task difference and the augmentation of a feature extraction backbone network for large task difference. On this basis, the MBHAN is proposed to continuously predict RUL of machinery. Finally, the experimental results on the life-cycle bearing and gear datasets demonstrate that TFFTCN achieve an average accuracy of 93% across both datasets, surpassing the existing prediction methods.

The quality of a robot’s environmental perception determines whether it can achieve more intelligent applications, such as semantic interaction with humans. SLAM, on the other hand, is one of the crucial capabilities for a robot to perceive its environment. However, when only a monocular image is provided, dynamic objects in the environment significantly impact the accuracy of map construction by the robot, leading to erroneous perception results. To address this issue, we propose a Visual SLAM framework based on BEV perception results, named BE-SLAM. With this framework, we can handle dynamic objects, occlusions, and incompletely observed objects. It can construct a stable static map by strengthening trust in static objects. Considering that object-level semantic maps can enhance a robot’s perception abilities, we also reconstruct static objects in the map and use them to optimize the pose. Through experiments on existing publicly available dataset, we compare BE-SLAM with several existing methods that have shown good performance. The experimental results demonstrate that BE-SLAM performs exceptionally well on high-dynamic sequences and achieves comparable results on static or low-dynamic sequences.

With the widespread adoption of virtual reality and 360-degree video, there is a pressing need for objective metrics to assess quality in this immersive panoramic format reliably. However, existing image quality assessment models developed for traditional fixed-viewpoint content do not fully consider the specific perceptual issues involved in 360-degree viewing. This paper proposes a 360-degree image full-reference quality assessment (FR-IQA) methodology based on a multi-channel architecture. The proposed 360-degree FR-IQA method further optimizes and identifies the distorted image quality using two easily obtained useful saliency and depth-aware image features. The convolutional neural network (CNN) is designed for training. Furthermore, the proposed method accounts for predicting user viewing behaviors within 360-degree images, which will further benefit the multi-channel CNN architecture and enable the weighted average pooling of the predicted FR-IQA scores. The performance is evaluated on publicly available databases to demonstrate the advantages brought by the proposed multi-channel model in performance evaluation and cross-database evaluation experiments, where it outperforms other state-of-the-art ones. Moreover, an ablation study exhibits good generalization ability and robustness.

Object detection plays a key role in helping to enable industrial quality control and safety monitoring. This article introduces a lightweight and efficient transformer-based object detection network called the encoder-free DEtection TRansformer (EF-DETR). This novel architecture enhances the DETR model through a redesigned network structure, leading to improved accuracy in object detection and a more lightweight network. To address the issue of suboptimal object detection accuracy, especially for small objects in the DETR model, we introduce a multiscale feature extractor and a high-efficiency feature fusion module. These components facilitate the direct extraction of fine-grained features, thereby enabling effective object detection. Departing from the use of a high-complexity encoder structure, we explore the utilization of an encoder-free neck structure to reduce the network's computational complexity. In addition, to expedite convergence, denoising training is incorporated into the decoder. This article presents extensive experiments, and the EF-DETR demonstrates strong performance on the MS COCO2017 dataset compared to other popular models.

In this paper, we propose a graph-represented image distribution similarity (GRIDS) index for full-reference (FR) image quality assessment (IQA), which can measure the perceptual distance between distorted and reference images by assessing the disparities between their distribution patterns under a graph-based representation. First, we transform the input image into a graph-based representation, which is proven to be a versatile and effective choice for capturing visual perception features. This is achieved through the automatic generation of a vision graph from the given image content, leading to holistic perceptual associations for irregular image regions. Second, to reflect the perceived image distribution, we decompose the undirected graph into cliques and then calculate the product of the potential functions for the cliques to obtain the joint probability distribution of the undirected graph. Finally, we compare the distances between the graph feature distributions of the distorted and reference images at different stages; thus, we combine the distortion distribution measurements derived from different graph model depths to determine the perceived quality of the distorted images. The empirical results obtained from an extensive array of experiments underscore the competitive nature of our proposed method, which achieves performance on par with that of the state-of-the-art methods, demonstrating its exceptional predictive accuracy and ability to maintain consistent and monotonic behaviour in image quality prediction tasks. The source code is publicly available at the following website https://github.com/Land5cape/GRIDS.

Various federated transfer learning (FTL) methods have been proposed to address domain shift and safeguard data privacy in the field of fault diagnosis. However, the effectiveness of these methods entirely relies on the presumption that the source clients must be homogenous with the target client. Meanwhile, these methods also require that the testing target-domain data are available during the communication process. Considering that target-domain data are typically unseen and heterogenous with source clients, the traditional FTL-based diagnosis methods cannot meet the demand of high data utilization rate and real-time diagnosis in real engineering. To overcome the above-mentioned issues, a novel heterogeneous federated domain generalization network (HFDGN) is proposed to fill the gap in the heterogeneous multisource federated diagnosis. In the HFDGN, the heterogeneous FTL framework is first proposed to achieve the generalized fault diagnosis of a target client by obtaining the common representation mappings from heterogeneous source clients. Additionally, the disentangled domain adaptation (DDA) base model is designed to remove the negative effect caused by noise. This model can enhance the ability of domain confusion and extract the inherent fault-relevant features. The asynchronous unbalanced update paradigm is utilized to optimize the DDA base model. Experimental results on two heterogeneous federated transfer cases prove that HFDGN outperforms other well-known and advanced diagnosis methods. The related code can be downloaded fromhttps://qinyi-team.github.io/2024/05/Heterogeneous-federated-domain-generalization-network.

Various domain adaptation (DA) methods have been proposed to address distribution discrepancy and knowledge transfer between the source and target domains. However, many DA models focus on matching the marginal distributions of two domains and cannot satisfy fault-diagnosed-task requirements. To enhance the ability of DA, a new DA mechanism, called deep joint distribution alignment (DJDA), is proposed to simultaneously reduce the discrepancy in marginal and conditional distributions between two domains. A new statistical metric that can align the means and covariances of two domains is designed to match the marginal distributions of the source and target domains. To align the class conditional distributions, a Gaussian mixture model is used to obtain the distribution of each category in the target domain. Then, the conditional distributions of the source domain are computed via maximum-likelihood estimation, and information entropy and Wasserstein distance are employed to reduce class conditional distribution discrepancy between the two domains. With joint distribution alignment, DJDA can achieve domain confusion to the highest degree. DJDA is applied to the fault transfer diagnosis of a wind turbine gearbox and cross-bearing with unlabeled target-domain samples. Experimental results verify that DJDA outperforms other typical DA models.

Adaptively reconstruction of the skin surface covering the skeletons of aircrafts, ships, high-speed trains, buildings with steel frames, etc., based on on-site measured point clouds is an important issue in both industry and construction. However, the skeleton has the characteristics of long and narrow shape and variable structure, resulting in a narrow and sparse distribution of measured point cloud on its surface with variable shape in different areas, which brings great challenges to the reconstruction of skin surface. In this article, a method for accurately reconstructing the skin surface covering the skeleton structures based on the arbitrary distributed sparse on-site measured points of the skeleton surface is proposed. At first, a robust B-spline curve fitting method based on the least square principle is proposed to construct the boundary curves of the point cloud. Then, a Coons-B-spline surface fitting method based on the generated boundary curves is proposed to generate an initial skin surface. Next, the initial skin surface is considered as a curved thin plate with stiffness, and a method of surface deformation considering the target points of deformation and the tensile and shear stiffness of the surface is proposed to obtain the surface with high accuracy and good smoothness. To show the feasibility of the proposed method, simulations and experiments are carried out. It is proved that the proposed method can achieve better accuracy while maintaining smoothness.