In industrial internet of things (IIoT), sensor failures, network interruptions, and communication delays often lead to structured missing data, significantly hindering the timely detection of incipient faults. This paper addresses the problem of quality-related incipient fault detection and location (IFDL) in IIoT under incomplete data. Compared with existing methods that rely on fully observed data, IFDL integrates a neighborhood probability guided imputer with a multi-scale graph regularized autoencoder, preserving the temporal continuity and manifold structures that characterize incipient faults. With the restored data and learned features, GSVD-based correlated subspace modeling isolates the quality-related variations embedded in the process to ensure accurate identification of incipient faults. Furthermore, a confidence-weighted trajectory deviation index, combining contextual and Bayesian uncertainty, identifies the variables contributing to process faults. The effectiveness of the proposed method is demonstrated through both simulation studies and an industrial case.

Due to the lack of prior knowledge about unknown classes during training, existing methods for cross-domain open-set image recognition typically rely on threshold-based solutions. However, such approaches often struggle to capture the complex boundary relationships between known and unknown classes, which can lead to negative transfer effects caused by feature confusion between the two. To address this issue, this paper proposes a graph isomorphic distillation diffusion model (GIDDM) that aims to learn the boundary relationships between known and unknown classes from a closed-set classifier that models predictive uncertainty. First, a diffusion classifier is designed to quantify model predictive uncertainty through a Monte Carlo sampling strategy performed on the noise distribution during the reverse denoising process. The uncertainty distribution is modeled, and the cumulative distribution function is used to compute the probability of a sample belonging to an unknown class. Second, an open-set recognition framework is constructed, treating the closed-set diffusion classifier as a teacher classifier, and guiding the student classifier to learn the complex boundary relationships between known and unknown classes through knowledge distillation. Third, the knowledge distillation process is further formalized as a graph isomorphic optimization problem, where the predictive manifolds of the student and teacher classifiers are constrained to be consistent, thereby enhancing knowledge transfer between the classifiers. Finally, the entire process is integrated into a unified open-set adversarial domain adaptation framework, reconstructing the traditional optimization objectives of closed-set adversarial domain adaptation to ensure sufficient separation between known and unknown classes while aligning the distributions of known classes in both the source and target domains. Experiments conducted on multiple hyperspectral image (HSI) datasets demonstrate that the proposed method achieves state-of-the-art performance on cross-domain open-set image recognition tasks. The code demo can be accessed on the following website: https://github.com/wzr78998/GIDDM.

Objects in complex environmental conditions such as low light and smoke occlusion are difficult to be detected accurately, and multimodal fusion of visible and infrared images with complementary physical properties provides a solution. However, different modalities often exhibit significant modal heterogeneity due to differences in imaging mechanisms, which can easily induce intermodal information interference. We decomposed inter modal information interference into task coupling interference, distribution difference interference, and noise superposition interference. To address the above issues, the variational decoupled bottleneck adaptation network (VDBAN) has been proposed. First, we decoupled the class distribution and spatial distribution of the two modalities, which in turn is targeted to captured task-related distribution information from multimodal data. In addition, we also customized differentiated fusion parameters and strategies for object recognition and localization tasks, adapting to the feature learning preferences of different tasks. Second, we designed the global mutual information adaptation mechanism and the structural mutual information adaptation mechanism, which promoted cross-modal knowledge sharing by enhancing the shared information between modalities. Finally, we constructed a variational fusion information bottleneck to compress the information flow in the multimodal feature fusion process to make the model focus on the task related information, and filter out the task-independent noise information. Numerous experimental results have demonstrated that VDBAN shows state-of-the-art detection performance in multimodal object detection task.

Hypersonic flight vehicles (HFVs) exhibit complex nonlinear dynamics and time-varying uncertainties, placing stringent demands on the adaptive learning capabilities of flight control systems. Meanwhile, the limited onboard communication and computational resources further require efficient resource utilization. To address these issues, this article proposes a dynamic event-triggered control (DETC) scheme based on actor–critic framework for HFVs, achieving synergistic optimization of control performance and resource conservation. First, an optimal event-triggered controller is developed within actor–critic framework to regulate the frequency of control input updates. A static event-triggered condition is formulated, and the stability of the closed-loop system is analyzed, with Zeno behavior eliminated, ensuring both stability and feasibility. Building on this, a dynamic variable is introduced, leading to a dynamic event-triggered condition that adaptively adjusts the trigger threshold based on real-time system states. Unlike static event-triggered mechanisms with fixed thresholds, this mechanism further optimizes the frequency of control input updates, achieving more efficient resource utilization while maintaining the desired control performance. Moreover, rigorous theoretical analysis demonstrates that the closed-loop system is stable and Zeno behavior is eliminated under the DETC scheme. Finally, the effectiveness of our proposed DETC scheme is verified through a simulation example.

The fusion of Hyperspectral image (HSI) and LiDAR data can dramatically enhance the classification accuracy, thus becoming a popular research topic. However, many studies are susceptible to the distribution difference across data collected by HSI and LiDAR sensors, resulting in misalignment of intermodal features and inadequate feature fusion. Therefore, a dual-branch information alignment fusion network (DIAFN) is proposed, aiming to exploit the highly correlated spatial information and complementary higher-order semantic features of multimodal data to achieve effective feature fusion. First, the dimension alignment module (DAM) is leveraged to extend the LiDAR data to align with HSI, and the weights of the LiDAR patches are adaptively learned using a dynamic adjustment approach to strengthen the complementary advantages between them. Second, a dual-branch information extraction module (IEM) is used to acquire complementary remote sensing features, and the distribution difference of multimodal data is reduced by optimizing alignment loss. Then, a dual-branch interactive fusion module (IFM) is proposed to fuse the acquired remote sensing features, and the intermodal relationship discriminator and channel discriminator are introduced to deeply mine and utilize the intrinsic connection between the features. Finally, the fused remote sensing features are classified for final category prediction. The model is experimented on two datasets to effectively demonstrate the superiority of DIAFN performance.

Emotion recognition in conversation (ERC) aims at accurately identifying emotional states expressed in conversational content. Existing ERC methods, although relying on semantic understanding, often encounter challenges when confronted with incomplete or misleading semantic information. In addition, when dealing with the interaction between emotional and semantic information, existing methods are often difficult to effectively distinguish the complex relationship between the two, which affects the accuracy of emotion recognition. To address the problems of semantic misdirection and emotional cross-talk encountered by traditional models when confronted with complex conversational data, we propose a semantic and emotional dual channel (SEDC) strategy for emotion recognition in conversations to process emotional and semantic information independently. Under this strategy, emotion information provides an auxiliary recognition function when the semantics are unclear or lacking, enhancing the accuracy of the model. Our model consists of two modules: the emotion processing module accurately captures the emotional features of each utterance through contrastive learning, and then constructs a dialogue emotion propagation map to simulate the emotional information conveyed in the dialogue; the semantic processing module combines an external knowledge base to enhance the semantic expression of the dialogue through knowledge enhancement strategies. This divide-and-conquer approach allows us to more deeply analyze the emotional and semantic dimensions of complex dialogues. Experimental results on the IEMOCAP, EmoryNLP, MELD, and DailyDialog datasets show that our approach significantly outperforms existing techniques and effectively improves the accuracy of dialogue emotion recognition.

A significant problem in sequential recommendation (SR) is the over-recommendation of popular items, leading to popularity bias, as users often follow these items due to conformity. Existing methods measure users’ conformity factors to reduce the impact of popularity bias. However, these methods do not consider the differences in users’ conformity behavior in the long-term and short-term. To address this, we propose LSDRec, a novel debiased SR method structured around three key tasks: degree-centrality conformity awareness, dual-scale interest encoding, and adaptive conformity information fusing. The degree-centrality conformity awareness task constructs a multiuser interaction graph, employs a graph convolutional network (GCN) to obtain global user conformity representations, and uses the degree centrality algorithm to compute users’ long-term and short-term conformity factors. The dual-scale interest encoding task models users’ long-term and short-term interests separately, obtaining corresponding interest representations and further enhancing them through the adaptive conformity information fusing task. The adaptive conformity information fusing task contrasts global conformity representations with long-term and short-term interest representations, adaptively integrating conformity factors and dynamically adjusting the degree of conformity information transfer. Together, these three tasks effectively mitigate popularity bias and improve the accuracy of user interest modeling. Our extensive evaluations of four diverse datasets demonstrate LSDRec's superior performance over current state-of-the-art methods.

Hyperspectral images (HSIs) contain rich spatial and spectral information, while light detection and ranging (LiDAR) data can provide elevation details. Effectively fusing HSI and LiDAR data can help achieve more accurate classification results. However, the joint classification of HSI and LiDAR data still faces several challenges, such as the redundancy of HSI spectral bands, the limitations of singular multimodal data fusion strategy, and the high cost of pixelwise labeling in remote sensing images. To tackle these challenges, we propose a classification method based on candidate pseudo labels pruning and dual mixture of experts (CPLP-DMoEs). First, we employ the multihead mixture of bands (MMoBs) to perform diverse, dense mixing of spectral bands, thereby alleviating the issue of high similarity between adjacent bands. Then, to overcome the limitations of single fusion strategies, we design a mixture of multimodal fusion expert (MoMFE) mechanism, which selects and mixes multiple fusion experts (FEs) to achieve diverse feature fusion of HSI and LiDAR data. Next, we introduce information entropy to balance the selection of FEs. Finally, facing the challenge of limited labeled samples, we propose a candidate pseudo labels pruning (CPLP)-based semi-supervised learning method. CPLP can prune the candidate pseudo label set from both intrasample and intersample perspectives to obtain more reliable pseudo labels, thereby facilitating the learning of a more accurate classification model. The experimental results on three datasets, including Houston 2013, MUUFL, and Augsburg, validate the effectiveness of the proposed method.

Deep learning has achieved outstanding success in the hyperspectral image (HSI) classification task. Almost all the current deep learning methods are used to conduct classification predictions by leveraging the output features from the deepest layer, which generally ignore the attention to multilayer outputs, so that the capability of hierarchical representation is limited. To remedy such deficiency, in this article, we propose to build a novel deep network form, called tensored deep autoencoder network (TDAE), for HSI classification. For this method, the tensor decomposition constraint item is built and introduced into a deep autoencoders network with a fully connection layer. It not only achieves the integration of multilayer output features but also captures the structure information among outputs. By such way, the network’s ability for hierarchical representation is significantly enhanced. Furthermore, to solve such built model, we further design an alternating update optimization scheme and obtain the desired feature forms. The features are further input into the fully connection layer to generate the label of the given HSI. Extensive experiments have been conducted to validate that the proposed TDAE method achieves more competitive performance compared with several state-of-the-art approaches.

Uncrewed underwater vehicles (UUVs) are usually deployed to track noncooperative targets in industrial or military applications. However, the sonar measurement data usually show high unreliability due to the particularity of underwater environments. We propose a novel AKansformer Target Tracking (AKTT) approach to improve the accuracy and robustness of UUV tracking of noncooperative targets under unreliable measurement. First, a historical measurements–based non-Markov state-space model is constructed to capture the complexity of target dynamics under unreliable measurement conditions. Second, we develop a multistep prediction network that integrates the axial attention mechanism with Kolmogorov–Arnold networks (KANs) based on the sonar measurement model and target state transition model. The prediction network learns from massive offline data and predicts the regular transition of the target state, which provides a powerful prediction ability for the tracking approach. In view of the unreliability of the sonar measurement and the uncertainty of the posterior distribution, this article uses the Monte Carlo principle and the proposed multistep prediction network to transform the non-Markov system model into a recursive first-order Markov model and constructs the corresponding recursive filtering model. Finally, we conduct a comprehensive evaluation of the AKTT through an array of statistical experiments and simulation cases. The results validate the efficacy of the AKTT, showcasing its robustness and superiority in tracking noncooperative targets under unreliable, even intermittently failing, sonar measurements.

Safe reinforcement learning aims to ensure the optimal performance while minimizing potential risks. In real-world applications, especially in scenarios that rely on visual inputs, a key challenge lies in the extraction of essential features for safe decision-making while maintaining the sample efficiency. To address this issue, we propose the constrained visual representation learning with bisimulation metrics for safe reinforcement learning (CVRL-BM). CVRL-BM constructs a sequential conditional variational inference model to compress high-dimensional visual observations into low-dimensional state representations. Additionally, safety bisimulation metrics are introduced to quantify the behavioral similarity between states, and our objective is to make the distance between any two latent state representations as close as possible to the safety bisimulation metric between their corresponding states. By integrating these two components, CVRL-BM is able to learn compact and information-rich visual state representations while satisfying pre-defined safety constraints. Experiments on Safety Gym show that CVRL-BM outperforms existing vision-based safe reinforcement learning methods in safety and efficacy. Particularly, CVRL-BM surpasses the state-of-the-art Safe SLAC method by achieving a 19.748% higher reward return, a 41.772% lower cost return, and a 5.027% decrease in cost regret. These results highlight the effectiveness of our proposed CVRL-BM.

Offline reinforcement learning (ORL) has made significant progress in the field of autonomous driving (AD); however, ensuring the safety of autonomous vehicles remains a critical challenge. ORL methods rely on learning from static, pre-collected datasets without interacting with the environment, making them particularly well-suited for AD scenarios where real-time interaction is costly or risky. However, this offline learning paradigm introduces extrapolation errors, which may lead to unsafe behavior during deployment. In this paper, we propose the Uncertainty-Based Alternative Diffusion Policy (UADP), specifically designed to enhance the safety of ORL for AD applications. UADP utilizes two diffusion models as alternative policies, with an ensemble of Q-networks serving as the value network. Both alternative policies and the value network are trained offline. During inference, the trained ensemble Q-network evaluate the actions’ uncertainties and select the action with lower uncertainty for execution to minimize risk and enhance safety in AD scenarios. Following the Datasets for Safe RL (DSRL) benchmark protocol, we compare UADP against a range of state-of-the-art (SOTA) baseline methods on both the MetaDrive and Carla platforms. The results demonstrate that UADP significantly improves driving safety and consistently outperforms existing methods across various AD tasks.

Visual reinforcement learning (VRL) has demonstrated remarkable capabilities in learning behaviors directly from intricate high-dimensional visual inputs. Despite these advancements, existing VRL methods still encounter obstacles such as complete collapse and dimensional collapse, resulting in representation degradation and dimensional redundancy. Contrastive learning, while helping to mitigate the complete collapse issue, is prone to the class collision dilemma. To tackle the aforementioned challenges, this article proposes a novel VRL control method with instance-reweighted alignment and instance-dimension uniformity (IAIU). In this VRL control method, the instance-reweighted alignment representation learning is introduced by minimizing the Kullback-Leibler (KL) divergence between the distributions of predicted next state representations and their weighted actual counterparts. By doing so, we aim to align state representations within the same semantic class, thereby effectively alleviating class collision. Meanwhile, an instance-dimension uniformity regularization mechanism is adopted to suppress the collapse phenomenon. This is realized by leveraging the Hilbert-Schmidt independence criterion (HSIC) and standard orthogonal constraint at the instance and dimension levels, respectively, ensuring the extraction of task-relevant state representations. In essence, IAIU's dual-strategy of alignment and uniformity not only addresses the critical issue of class collision but also guarantees uniformity with respect to both the instances and dimensions. Simulation results from the distracting control suite (DCS) benchmark demonstrate IAIU's superior performance, with substantial enhancements in both representational ability and policy efficacy. The code is available at https://github.com/anonymousforcode/IAIU.

DT, as a conditional sequence modeling (CSM) approach, learns the action distribution for each state using historical information, such as trajectory returns, offering a supervised learning paradigm for offline reinforcement learning (Offline RL). However, due to the fact that decision transformer (DT) solely concentrates on an individual trajectory with high returns-to-go, it neglects the potential for constructing optimal trajectories by combining sequences of different actions. In other words, traditional DT lacks the trajectory stitching capability. To address the concern, a novel DT (PCDT) for Offline RL is proposed. Our approach begins by pretraining a standard DT to explicitly capture behavior sequences. Next, we apply the sequence importance sampling to penalize actions that significantly deviate from these behavior sequences, thereby constructing a pessimistic critic. Finally,Q-values are integrated into the policy update process, enabling the learned policy to approximate the behavior policy while favoring actions associated with the highestQ-value. Theoretical analysis shows that the sequence importance sampling in pessimistic critic decision transformer (PCDT) establishes a pessimistic lower bound, while the value optimality ensures that PCDT is capable of learning the optimal policy. Results on the D4RL benchmark tasks and ablation studies show that PCDT inherits the strengths of actor–critic (AC) and CSM methods, achieving the highest normalized scores on challenging sparse-reward and long-horizon tasks. Our code are available at https://github.com/Henry0132/PCDT.

The piecewise and discontinuous output constraints are prevalent in practical engineering systems but remain insufficiently explored in the literature. Unlike existing results, the discontinuous constraints considered here exhibit two salient characteristics: 1) the bounded constraint boundary function undergoes a jump discontinuity at a specific instant during the initial operational period and 2) the system output becomes unconstrained thereafter. These features render traditional methods inadequate because the derivatives of the constraint boundary function do not exist. To address this problem, we propose a systematic neuroadaptive control framework for nonlinear systems that integrates two novel shift functions with a new barrier Lyapunov function (BLF). The presented control scheme not only ensures good tracking performance under such piecewise and discontinuous output constraints but also can be applied to several other common yet critical constraint scenarios without involving any adjustment to the controller structure. Theoretical analysis and simulation results verify the effectiveness and practicality of the proposed approach.

Emotion recognition in conversation (ERC) is a widely used technology in both affective dialogue bots and dialogue recommendation scenarios, where motivating a system to correctly recognize human emotions is crucial. Uncovering as much contextual information as possible with a limited amount of dialogue information is essential for eventually identifying the correct emotion of each sentence. The integration of contextual information using the existing approaches often results in inadequate access to information or information redundancy. Deeply integrating the different knowledge behind utterances is also difficult. Therefore, to address these problems, we propose a dynamic complementary graph convolutional network (DCGCN) for conversational emotion recognition. Our approach uses commonsense knowledge to complement the contextual information contained in utterances and enrich the extracted conversation information. We creatively propose the concept of utterance density to prevent redundancy and the loss of utterance information in context-dependent contextual information modeling cases. An utterance dependency structure is dynamically determined by the utterance density, and the contextual information is fully integrated into each sentence representation. We evaluate our proposed model in extensive experiments conducted on four public benchmark datasets that are commonly used for ERC. The results demonstrate the effectiveness of the DCGCN, which achieves competitive results in terms of well-known evaluation metrics. Our code is available athttps://github.com/Tars-is-a-robot/Conversational-emotion-recognition.git.

The pretrained vision-language models (VLMs) have achieved outstanding performance in various visual tasks, primarily due to the knowledge they have acquired from massive image-text pairs. This enables VLMs to generalize to a wide range of downstream tasks. This article presents the first attempt to adapt VLMs for the joint classification task of hyperspectral image (HSI) and LiDAR data, aiming to leverage the well-learned VLMs to extract more generalizable features from diverse remote sensing image sources. Initially, using a patch encoder (PE), low-dimensional patches of HSI and LiDAR data are transformed into high-dimensional latent feature representations, meeting the dimensional requirements of VLMs for visual input data. Unlike traditional classifiers that rely on discrete class labels, VLM-based classification methods depend on continuous vectors, which can be derived from textual templates with class names, i.e., prompts. The classification performance of VLM-based methods heavily relies on these prompts, but prompt engineering not only demands extensive expert knowledge but also is extremely time-consuming. To address this, prompt tuning (PT) methods are introduced to enhance the generalizability of VLMs by adding spectral-based prompts to the vision encoder and incorporating randomly initialized, learnable text prompts (TPs) into the text encoder. Finally, through a novel class-discriminative loss function, the distance between text features of different classes is increased, thereby enhancing the model’s discriminative ability. Experimental results on the Houston 2013, Trento, and MUUFL datasets demonstrate that the proposed method can achieve competitive classification accuracy with a limited number of labeled pixels.

Offline reinforcement learning (ORL) learns policy from a static dataset without further interaction with the environment, which holds significant promise in industrial control systems characterized by inefficient online interaction and inherent safety concerns. To mitigate the extrapolation error induced by distribution shift, it is essential for ORL to constrain the learned policy to perform actions within the support set of behavior policy. Existing methods fail to represent the behavior policy properly and typically tend to prefer actions with higher densities within the support set, resulting suboptimal learned policy. This article proposes a novel ORL method which represents the behavior policy with a diffusion model and trains a reverse diffusion guide policy to instruct the pretrained diffusion model in generating actions. The diffusion model exhibits stable training and strong distribution expression ability, and the reverse diffusion guide policy can effectively explore the entire support set to help generate the optimal action. When facing low-quality datasets, a trainable perturbation can be further added to the generated action to help the learned policy escape the performance limitation of behavior policy. Experimental results on D4RL Gym-MuJoCo benchmark demonstrate the effectiveness of the proposed method, surpassing several state-of-the-art ORL methods.

Cardiac magnetic resonance imaging (CMRI) can help experts quickly diagnose cardiovascular diseases. Due to the patient's breathing and slight movement during the magnetic resonance imaging scan, the obtained CMRI may be severely blurred, affecting the accuracy of clinical diagnosis. To address this issue, we propose the quadratic conditional diffusion model for blind CMRI super-resolution (DBSR). Specifically, we propose a conditional blur kernel noise predictor, which predicts the blur kernel from low-resolution images by the diffusion model, transforming the unknown blur kernel in low-resolution CMRI into a known one. Meanwhile, we design a novel conditional CMRI noise predictor, which uses the predicted blur kernel as prior knowledge to guide the diffusion model in reconstructing high-resolution CMRI. Furthermore, we propose a cascaded residual attention network feature extractor, which extracts feature information from CMRI low-resolution images for blur kernel prediction and SR reconstruction of CMRI images. Extensive experimental results indicate that our proposed DBSR achieves better blind super-resolution reconstruction results than several state-of-the-art baselines.

Hyperspectral images (HSIs) and light detection and ranging (LiDAR) are two critical and frequently used types of remote sensing data, each containing rich spectral and elevation information. Fusing HSI and LiDAR can exploit the complementary properties of the two modalities for ground object classification. The performance of existing fusion classification methods is often limited by the difficulty of adapting feature extraction operators to complex spatial distributions, and the correlation and specificity between different modalities are not reasonably exploited. Therefore, the reinforcement learningbased markov edge decoupled fusion network (MEDFN) is proposed. This network can intelligently compose graphs based on different modal characteristics and tasks to adapt to complex spatial distributions; it can also suppress noise to complete fusion classification while fully utilizing complementary information of different modalities. First, a reinforcement learning-based graph construction subnetwork (RLGN) is proposed to learn a twomodal graph construction strategy suitable for classification tasks by transforming regular multimodal data into irregular graph data. Second, a multimodal edge attention module (MEAM) is proposed to extract edge features between spatial neighboring nodes and model the importance of each node, thereby capturing the spatial topology information encompassed in the multimodal data. Finally, the decoupled multimodal fusion module (DMFM) is proposed to decouple multimodal features into shared and unshared parts and enhance the model's ability to distinguish features by targeting the modal-shared feature between modalities and modal-specific feature. The experimental results based on three well-known HSI and LiDAR datasets demonstrate the effectiveness of the proposed MEDFN in fusion classification tasks.

In this article, a novel coupled policy improvement mechanism is developed for improving policy iteration (PI) algorithms. In contrast to the common PI, the developed dual parallel policy iteration (DPPI) with coupled policy improvement mechanism consists of two parallel PIs. At each PI step, the performances of the two parallel policies are evaluated and the better one is defined as the dominant policy. Then, the dominant policy is used to guide the parallel policy improvement in a soft manner by constraining the Kullback-Liebler (KL) divergence between the dominant policy and the policy to be updated. It is proven that the convergence of DPPI can be guaranteed under the designed coupled policy improvement mechanism. Moreover, it is clearly shown that under certain conditions, the Q -functions of the two new policies obtained in each parallel policy improvement are larger than those of all the previous dominant policies, which is conductive to accelerate the PI process and improve the policy learning efficiency to some extent. Furthermore, by combining DPPI with the twin delay deep deterministic (TD3) policy gradient, we propose a reinforcement learning (RL) algorithm: parallel TD3 (PTD3). Experimental results on continuous-action control tasks in the MuJoCo and OpenAI Gym platforms show that the proposed PTD3 outperforms the state-of-the-art RL algorithms.

To mitigate the extrapolation error arising from offline reinforcement learning (RL) paradigm, existing methods typically make learned Q-functions over-conservative or enforce global policy constraints. In this article, we propose a dual behavior regularized offline deterministic Actor-Critic (DBRAC) by simultaneously performing behavior regularization on the coupling-iterative policy evaluation (PE) and policy improvement (PI) in the policy iteration process. In the PE phase, the difference between the Q-function and behavior value is first taken as the anti-exploration behavior value regularization term to drive the Q-function toward its true Q-value, which significantly reduces the conservatism of learned Q-function. In the PI phase, the estimated action variances of behavior policy in different states are then utilized for designing the weight and threshold of mild-local behavior cloning regularization term, which standardizes the local improvement potential of learned policy. Experiments on the well-known datasets for deep data-driven RL (D4RL) demonstrate that the DBRAC can quickly learn more competitive task-solving policies in various offline situations with different data qualities, significantly outperforming state-of-the-art offline RL baselines.

Visual reinforcement learning (VRL) aims to optimize policies by utilizing information derived from visual sensory inputs. To tackle the challenges of high dimensionality and sample efficiency, numerous studies have leveraged self-supervised learning to encode observations into latent state representations. Though these algorithms have shown promise, they may be susceptible to task-irrelevant distractions, such as changes in background, color, and camera angles. Thus, we propose the clustering-driven state embedding for reinforcement learning (RL) under visual distractions (CSE-RL) to learn more robust state representations. The student’s t distribution is first utilized to model the predicted clustering assignment. In this process, the reward centers are defined by hardening the target distribution. Subsequently, the state embeddings, rewards, and their respective centers are fed into the predicted clustering assignment function for clustering. To avoid trivial solutions, the Sinkhorn-Knopp algorithm is adopted to balance observations across clusters. The proposed CSE-RL has the capacity to enhance the comprehension of inherent relationships among different states, while also filtering out irrelevant or redundant information, resulting in the creation of more efficient representations that are less prone to visual distractions. Experimental results from the distracting control suite (DCS) indicate that CSE-RL achieves comparable or even better performance compared with several state-of-the-art VRL algorithms in handling challenging visual distractions. Notably, CSE-RL exhibits remarkable improvements of 31.82% and 18.18% on DCS over baselines based on data-regularized Q (DrQ) and DrQv2 frameworks, respectively. Integrating CSE-RL into offline RL offers promising avenues for future study.

Driven by multi-omics data, some multi-view clustering algorithms have been successfully applied to cancer subtypes prediction, aiming to identify subtypes with biometric differences in the same cancer, thereby improving the clinical prognosis of patients and designing personalized treatment plan. Due to the fact that the number of patients in omics data is much smaller than the number of genes, multi-view spectral clustering based on similarity learning has been widely developed. However, these algorithms still suffer some problems, such as over-reliance on the quality of pre-defined similarity matrices for clustering results, inability to reasonably handle noise and redundant information in high-dimensional omics data, ignoring complementary information between omics data, etc. This paper proposes multi-view spectral clustering with latent representation learning (MSCLRL) method to alleviate the above problems. First, MSCLRL generates a corresponding low-dimensional latent representation for each omics data, which can effectively retain the unique information of each omics and improve the robustness and accuracy of the similarity matrix. Second, the obtained latent representations are assigned appropriate weights by MSCLRL, and global similarity learning is performed to generate an integrated similarity matrix. Third, the integrated similarity matrix is used to feed back and update the low-dimensional representation of each omics. Finally, the final integrated similarity matrix is used for clustering. In 10 benchmark multi-omics datasets and 2 separate cancer case studies, the experiments confirmed that the proposed method obtained statistically and biologically meaningful cancer subtypes.

Labeling hyperspectral images (HSIs) is time-consuming and labor-intensive for researchers, so the deficiency of adequate labeling samples is a giant obstacle to conducting HSI classification. Especially, such issue is exacerbated when there are no available labeled samples in the target scene. For the sake of resolving aforesaid issue, we put forward a novel cross-scene HSI classification method namely bi-classifier adversarial augmentation network (BCAN) so as to transfer knowledge from a similar but different source domain to an unlabeled target domain. First, the source and target domain distributions are aligned by maximizing and minimizing the decision discrepancy between two classifiers, respectively. Then, more accurate samples corresponding to pseudo-labels are selected as reliable samples and added to the training set. Finally, the spectral band random zeroing (SBRZ) method is proposed to expand the training samples for reliable samples, which handles the problem of insufficient network training resulted from insufficient samples in the source domain. By using multi-classifiers for domain adaptation and data augmentation, the accuracy of the network for cross-scene HSI classification tasks are improved. BCAN can extract the source domain’s helpful information to complete the target domain classification task. Experiments conducted on ten HSI data pairs show that BCAN outperforms many state-of-the-art baselines.

The different news clicked by users reflects the diverse interests of users. Most of the existing news recommendation methods do not consider the interaction with candidate news in the process of modeling user interest representation. This method makes it challenging to precisely match candidate news to specific user interests. We propose a user interest activation recommendation method that fuses multichannel information—MIAR. It utilizes the word embedding of the user’s historical clicked news and the news title embedding generated by aggregation and interacts with the candidate news, respectively, to better match the candidate news with the user’s interests. Our proposed method contains two frameworks (interactive framework and distributed framework). In the interactive framework, we propose a user multichannel interest modeling framework MIF from the word embedding level of news headlines to capture more semantic cues related to user interests. In the distributed framework, we design a candidate-aware interest activation module TAR from the news embedding representation level obtained by attention aggregation. It uses different candidate news vectors to adjust the user representations learned from the user’s historical reading records. This allows the model to build candidate-guided user representations to accurately match candidate news to parts of user interests that are relevant to the candidate news. Finally, we effectively assign the weights of the two frame scores so that the models can fuse better. Extensive experiments on the MIND news recommendation dataset demonstrate the effectiveness of our method.

In rating prediction research, how to capture user and item features from review text is a key to improving model prediction accuracy. The sparsity of review text and the accuracy of the description of items in the review text make it difficult to obtain accurate feature representations of users and items by modeling the text content alone. Therefore, it is important to evaluate the usefulness of the reviews at first because not all review texts are valuable. How to analyze the usefulness of a review is a key to modeling the review text. The way previous models use attention to inscribe semantic weights on the review text is not sufficient to indicate the degree of usefulness of a review, so we suggest adding property information to model reviews. Based on this, we propose an interaction recommendation model that is based on enhanced graph convolution and fuses review properties (PGIR), which incorporates property information into text modeling by different activations and matches useful property feature interaction pairs for review text in a self-supervised manner. This allows the model to obtain an accurate feature representation of the review text. In addition, we analyze the high-order connectivity among user–item pairs. Then, we design an enhanced graph convolution method to capture the collaborative signals between users and items and model the dynamic features of users and items on this basis. After extensive experiments conducted on five standard datasets based on Amazon, the results show that the PGIR model achieves a substantial improvement over existing state-of-the-art models in terms of rating prediction. In addition, we experimentally demonstrate the superiority of our proposed property activation method, which further improves the rating prediction performance of the PGIR model.

Since the sample data after one exploration process can only be used to update network parameters once in on-policy deep reinforcement learning (DRL), a high sample efficiency is necessary to accelerate the training process of on-policy DRL. In the proposed method, a submartingale criterion is proposed on the basis of the equivalence relationship between the optimal policy and martingale, and then an advanced value iteration (AVI) method is proposed to conduct value iteration with a high accuracy. Based on this foundation, an anti-martingale (AM) reinforcement learning framework is established to efficiently select the sample data that is conducive to policy optimization. In succession, an AM proximal policy optimization (AMPPO) method, which combines the AM framework with proximal policy optimization (PPO), is proposed to reasonably accelerate the updating process of state value that satisfies the submartingale criterion. Experimental results on the Mujoco platform show that AMPPO can achieve better performance than several state-of-the-art comparative DRL methods.

Adversarial learning-based unsupervised hyperspectral image (HSI) classification methods usually adapt probability distributions by minimizing the statistical distance between similar pixels of different HSIs. Since the adversarial learning may weaken the discriminability of features, the extracted features will contain a lot of non-discriminative information, pixels with similar features may be classified as different classes. Therefore, directly reducing the statistical distance between similar pixels in a latent space may aggravate misclassification. To this end, we propose an unsupervised HSI classification method called soft instance-level domain adaptation with virtual classifier. First, the domain-invariant features of HSI are extracted by a graph convolutional network. Then, a feature similarity metric-based virtual classifier is constructed to output class probabilities of target-domain samples. Furthermore, to enable similar features of HSIs from different domains to be classified into the same class, the divergence between the real and virtual classifiers is reduced by minimizing the real and virtual classifier determinacy disparity. Finally, to reduce the influence of noisy pseudo-labels, a soft instance-level domain adaptation method is proposed. For each target-domain sample, the confidence coefficients are assigned to its corresponding positive and negative samples in the source domain, and a soft prototype contrastive loss is constructed and minimized to adapt two domains in an instance-level way. Experimental results on five real HSI datasets including Botswana, Kennedy Space Center, Pavia Center, Pavia University, and HyRANK demonstrate the effectiveness of our proposed method.

Few-shot hyperspectral image (HSI) classification poses challenges due to sample selection bias in few-shot scenarios, potentially leading to incorrect statistical associations between noncausal factors and category semantics. To address these challenges, an original HSI is treated as a mixture comprising causal and noncausal factors. By integrating the causal learning, meta-learning, and transfer learning, a cross domain few-shot HSI classification method based on causal meta-transfer learning (CMTL) is developed. First, a Mask Transformer is implemented to identify noncausal factors unrelated to categories. Second, an independent causal constraint is applied to separate the causal and noncausal factors, and enhancing the inclusion of pure and independent causal factors in the features. Finally, the meta-transfer learning is leveraged to enable the classification model to extract causal factors highly correlated with category semantics from data, facilitating the cross-domain knowledge transfer. Meanwhile, a causal association module is employed to maximize the mutual information between causal factors and category predictions, thereby ensuring a strong causal association between causal factors and classification tasks. Experimental results show that CMTL achieves competitive performance in cross-domain few-shot HSI classification tasks.

Since obtaining labeled hyperspectral images (HSIs) is difficult and time-consuming, the shortage of training samples has always been a challenge for HSI classification. In practical applications, only a few labeled samples are available in the task domain (target domain), while sufficient labeled samples are available in another domain (source domain). At the same time, these two domains are heterogeneous and contain different categories. This scenario makes it difficult to effectively transfer knowledge from the source domain to the target domain. To address this challenge, we propose a novel heterogeneous few-shot learning (FSL) method, namely graph meta transfer network (GMTN). Specifically, the graph sample and aggregate network (GraphSAGE) and meta-learning, which are both inductive learning, are integrated into a unified framework. In this way, the aggregation function is generalized from abundant few-shot tasks for feature extraction on the source and target domains. The spatial importance strategy (SIS) is designed to guide the feature propagation and alleviate the information interference caused by different categories. The neighborhood receptive field spectral attention (RFSA) mechanism is proposed to model the importance of spectral band using the information of the neighborhood pixels, which enables GMTN to pay more attention to bands with discriminative features in both domains. In addition, the node spatial information reset method is proposed to augment samples based on the spatial position relationship of nodes. Furthermore, to alleviate the domain shift in heterogeneous scenarios, the conditional domain adversarial strategy is used to achieve effective meta-knowledge transfer. Experiments show that GMTN outperforms the compared state-of-the-art methods.

Atrial fibrillation (AF) is an increasing medical burden worldwide, and its pathological manifestations are atrial tissue remodeling and low-pressure atrial tissue fibrosis. Due to the inherent defects of medical image data acquisition systems, the acquisition of high-resolution cardiac magnetic resonance imaging (CMRI) faces many problems. In response to these problems, we propose the Progressive Feedback Residual Attention Network (PFRN) for CMRI super-resolution. Specifically, we directly perform feature extraction on low-resolution images, retain feature information to a large extent, and then build multiple independent progressive feedback modules to extract high-frequency details. To accelerate network convergence and improve image reconstruction quality, we implement the MS-SSIM-L1 loss function. Furthermore, we utilize the residual attention stack module to explore the image's internal relevance and extract the low-resolution image's detailed features. Extensive benchmark evaluation shows that PFRN can improve the detailed information of the image SR reconstruction results, and the reconstructed CMRI has a better visual effect.

Safety limits the application of traditional reinforcement learning (RL) methods to autonomous driving. To address the challenge of safe exploration in autonomous driving tasks, a novel safe RL method called Twin Delayed Deep Deterministic Policy Gradient based on Approximate Safe Action (TD3-ASA) is proposed in this paper. In TD3-ASA, the action output by the current policy during the exploration process is modified to obtain an approximate safe action, and then the approximate safe action is utilized to train a safe policy for deployment. TD3-ASA offers several advantages: 1) TD3-ASA is sample efficient and does not need any prior knowledge; 2) TD3-ASA enhances safety both during training and deployment; 3) TD3-ASA introduces an adjustable safety correction factor that enables a tradeoff between exploration and safety. Experimental results conducted on both the MetaDrive and SpeedLimit autonomous driving test platforms demonstrate the effectiveness of TD3-ASA. TD3-ASA exhibits more than triple safety during training on MetaDrive compared to the current state-of-the-art RL method, achieving a high success rate and low deployment risk.

In existing generative adversarial networks (standard GAN and its variants), the discriminator is trained for recognizing the real data as positive while the generated data as negative. This kind of positive-negative classification criterion ignores the fact that the discriminator is a non-objective evaluator, which means that the image quality evaluated by the discriminator may fluctuate during the whole training progress. Considering this fact, we propose a novel GAN framework called Discriminator-Quality Evaluation GAN (DQE-GAN) by using the discriminator outputs to evaluate image quality. By dynamically classifying images into high discriminator-quality and low discriminator-quality samples, every adversarial iteration step can be more reasonable and objective. The convergence of DQE-GAN framework can be theoretically proved. Through extensive experiments, we demonstrate DQE-GANs’ ability of achieving better generated images faster and more stable.

Generative adversarial networks (GANs) have demonstrated superior performances in image generation. In recent years, various improvements of network structure and learning theory related to GANs have undergone numerous advancement. Among these improvement techniques, the asymmetric training on the generator and discriminator networks has been widely adopted. For example, the batch normalization is used in generator while the spectral normalization is used in discriminator, or using different learning rates for the generator and discriminator. However, the asymmetric training on the real and generated samples has not been taken into consideration till now. In this paper, we proposed a novel asymmetric training-based RealnessGAN (ATRGAN) which applies the idea of asymmetric training on both samples and networks. Specifically, the asymmetric training on samples refers to performing the differential learning on the real and generated samples by controlling the information entropies of real and fake anchor distributions. The asymmetric training on networks is realized via the sampling transmission$G2D$, which abandons the commonly used independent random sampling. With the help of$G2D$, the discriminator can obtain a dominant training position than the generator, so as to ensure that the discriminator can guide the generator more effectively during training. In addition, we proposed the floating anchor distribution technique and constructed the objective function of generator for ATRGAN. Through comparative experiments, we demonstrated ATRGAN's ability of achieving better generation performance than various SOTA GANs on CIFAR-10, CAT, and CelebA-HQ datasets.

The accurate estimation of Q-function and the enhancement of agent's exploration ability have always been challenges of off-policy actor-critic algorithms. To address the two concerns, a novel robust actor-critic (RAC) is developed in this article. We first derive a robust policy improvement mechanism (RPIM) by using the local optimal policy about the current estimated Q-function to guide policy improvement. By constraining the relative entropy between the new policy and the previous one in policy improvement, the proposed RPIM can enhance the stability of the policy update process. The theoretical analysis shows that the incentive to increase the policy entropy is endowed when the policy is updated, which is conducive to enhancing the exploration ability of agents. Then, RAC is developed by applying the proposed RPIM to regulate the actor improvement process. The developed RAC is proven to be convergent. Finally, the proposed RAC is evaluated on some continuous-action control tasks in the MuJoCo platform and the experimental results show that RAC outperforms several state-of-the-art reinforcement learning algorithms.

Oriented object detection has achieved a significant progress in image processing. Compared with horizontal detection methods, oriented detectors add the orientation parameter in regression to locate objects. However, existing rotation and quadrilateral representations are not appropriate for oriented two-stage methods to generate efficient oriented proposals. In this paper, we propose a novel framework to detect oriented objects, termedshort side excursion detection(SSEDet). Inspired by the circle theorem, we propose a transformation method from horizontal rectangles to oriented ones to accurately describe oriented objects. To be specific, we exploit the offset of short sides relative to the top-right vertex to represent the orientation of rectangle. Compared with the horizontal rectangle, the representation parameters of oriented rectangle have only one more orientation parameter. Under the action of the orientation parameter, the one-to-one correspondence between representation parameters and oriented rectangle can be realized. Experimental results on commonly used datasets verify that SSEDet can generate high-quality oriented proposals.

A hyperspectral image (HSI) classification method named graph domain adversarial network with dual-weighted pseudo-label loss (GDAN-DWPL) is proposed in this letter. First, in order to extract more discriminative features, GDAN is applied to the transfer task of HSI. Then, a more reliable spectral–spatial graph is constructed by comprehensively utilizing the abundant spectral features and spatial contextual information. Finally, due to the misalignment of probability distribution on class-level caused by inaccurate pseudo-labels of target domain, a dual-weighted pseudo-label loss is proposed from the perspective of spatiality and confidence. By assigning larger weights to more reliable pixels and eliminating pixels with false pseudo-labels, the negative impact on learning process of prediction model can be reduced. Experimental results on four real HSI datasets show the superiority of GDAN-DWPL.

In recent years, the proximal policy optimization (PPO) algorithm has received considerable attention because of its excellent performance in many challenging tasks. However, there is still a large space for theoretical explanation of the mechanism of PPO's horizontal clipping operation, which is a key means to improve the performance of PPO. In addition, while PPO is inspired by the learning theory of trust region policy optimization (TRPO), the theoretical connection between PPO's clipping operation and TRPO's trust region constraint has not been well studied. In this article, we first analyze the effect of PPO's clipping operation on the objective function of conservative policy iteration, and strictly give the theoretical relationship between PPO and TRPO. Then, a novel first-order policy gradient algorithm called authentic boundary PPO (ABPPO) is proposed, which is based on the authentic boundary setting rule. To ensure the difference between the new and old policies is better kept within the clipping range, by borrowing the idea of ABPPO, we proposed two novel improved PPO algorithms called rollback mechanism-based ABPPO (RMABPPO) and penalized point policy difference-based ABPPO (P3DABPPO), which are based on the ideas of rollback clipping and penalized point policy difference, respectively. Experiments on the continuous robotic control tasks implemented in MuJoCo show that our proposed improved PPO algorithms can effectively improve the learning stability and accelerate the learning speed compared with the original PPO.

Encouraging the agent to explore has always been an important and challenging topic in the field of reinforcement learning (RL). Distributional representation for network parameters or value functions is usually an effective way to improve the exploration ability of the RL agent. However, directly changing the representation form of network parameters from fixed values to function distributions may cause algorithm instability and low learning inefficiency. Therefore, to accelerate and stabilize parameter distribution learning, a novel inference-based posteriori parameter distribution optimization (IPPDO) algorithm is proposed. From the perspective of solving the evidence lower bound of probability, we, respectively, design the objective functions for continuous-action and discrete-action tasks of parameter distribution optimization based on inference. In order to alleviate the overestimation of the value function, we use multiple neural networks to estimate value functions with Retrace, and the smaller estimate participates in the network parameter update; thus, the network parameter distribution can be learned. After that, we design a method used for sampling weight from network parameter distribution by adding an activation function to the standard deviation of parameter distribution, which achieves the adaptive adjustment between fixed values and distribution. Furthermore, this IPPDO is a deep RL (DRL) algorithm based on off-policy, which means that it can effectively improve data efficiency by using off-policy techniques such as experience replay. We compare IPPDO with other prevailing DRL algorithms on the OpenAI Gym and MuJoCo platforms. Experiments on both continuous-action and discrete-action tasks indicate that IPPDO can explore more in the action space, get higher rewards faster, and ensure algorithm stability.

For hyperspectral image (HSI) classification tasks, obtaining sufficient labeled samples is usually difficult, time-consuming, and expensive. To address the aforementioned issue, by transferring the labeled sample information of a relevant source domain to the unlabeled target domain, an HSI classification method based on the domain adversarial broad adaptation network (DABAN) is proposed. First, the bottleneck adaptation module composed of a bottleneck layer and a domain adaptation layer is constructed and introduced to the domain adversarial neural network; thus, the domain adversarial adaptation network (DAAN) is designed. By simultaneously performing domain adversarial learning, reducing both the marginal distribution difference and second-order statistic difference between two domains, the distributions of the source and target domains are aligned. Then, the conditional distribution adaptation regularization term based on the maximum mean discrepancy is embedded into a broad learning system to obtain the conditional adaptation broad network (CABN). On the one hand, CABN can perform the class-level distribution adaptation on the domain-invariant features extracted by DAAN. On the other hand, the representation ability of the domain-invariant features expanded by CABN can be further enhanced. Experimental results on ten real hyperspectral data pairs show that, compared with the existing mainstream methods, DABAN can effectively utilize relevant source-domain information to assist in improving the classification accuracy of the target domain.

Proximal policy optimization (PPO) is a deep reinforcement learning algorithm based on the actor–critic (AC) architecture. In the classic AC architecture, the Critic (value) network is used to estimate the value function while the Actor (policy) network optimizes the policy according to the estimated value function. The efficiency of the classic AC architecture is limited due that the policy does not directly participate in the value function update. The classic AC architecture will make the value function estimation inaccurate, which will affect the performance of the PPO algorithm. For improvement, we designed a novel AC architecture with policy feedback (AC-PF) by introducing the policy into the update process of the value function and further proposed the PPO with policy feedback (PPO-PF). For the AC-PF architecture, the policy-based expected (PBE) value function and discount reward formulas are designed by drawing inspiration from expected Sarsa. In order to enhance the sensitivity of the value function to the change of policy and to improve the accuracy of PBE value estimation at the early learning stage, we proposed a policy update method based on the clipped discount factor. Moreover, we specifically defined the loss functions of the policy network and value network to ensure that the policy update of PPO-PF satisfies the unbiased estimation of the trust region. Experiments on Atari games and control tasks show that compared to PPO, PPO-PF has faster convergence speed, higher reward, and smaller variance of reward.

Most convolutional neural network (CNN) models require large parameter quantity, high computational consumption, and deeper layers to achieve better performance, which limit their further applications. For improvement, a lightweight multiscale multipath ensemble CNN (MSME-CNN) is proposed. First, the shallow-layer and deep-layer convolution output features are directly concatenated to form a multipath ensemble mode, which can not only avoid gradient vanishing but also fuse the multilayer features. In addition, each path of the network is composed of multiscale low-rank convolution kernels in parallel. This structure can extract multiscale features of the input and improve the feature extraction ability of the network. Meanwhile, the convolution kernel low-rank approximation can effectively compress the model complexity. Furthermore, the proposed sparse connection mechanism of the convolution kernel helps to reduce the complexity, so that higher classification accuracy can be obtained with less parameters and computational load. Finally, the linear sparse bottleneck structure is used to fuse multiscale features and compress the convolution channel, which further improves the network performance. Experiments of four commonly used image recognition datasets verify the superiority of MSME-CNN to several baseline models.

Encouraging the agent to explore has become a hot topic in the field of reinforcement learning (RL). The popular approaches to engage in exploration are mainly by injecting noise into neural network (NN) parameters or by augmenting additional intrinsic motivation term. However, the randomness of injecting noise and the metric for intrinsic reward must be chosen manually may make RL agents deviate from the optimal policy during the learning process. To enhance the exploration ability of agent and simultaneously ensure the stability of parameter learning, we proposed a novel proximal parameter distribution optimization (PPDO) algorithm. On the one hand, PPDO enhances the exploration ability of RL agent by transforming NN parameter from a certain single value to a function distribution. On the other hand, PPDO accelerates the parameter distribution optimization by setting two groups of parameters. The parameter optimization is guided by evaluating the parameter quality change before and after the parameter distribution update. In addition, PPDO reduces the influence of bias and variance on the value function approximation by limiting the amplitude of the two consecutive parameter updates, which can enhance the stability of the parameter distribution optimization. Experiments on the OpenAI Gym, Atari, and MuJoCo platforms indicate that PPDO can improve the exploration ability and learning efficiency of deep RL algorithms, including DQN and A3C.

Since the learning of attribute classifiers is independent of the learning of object classifier in zero-shot learning, it is difficult to guarantee that the learned attribute classifiers are optimal for the subsequent object recognition tasks. Therefore, a novel zero-shot learning method based on multitask extended attribute groups (MTEAGs) is proposed by using the multitask learning framework and grouping idea. First, we used an unsupervised clustering method to group the attributes and object classes of training images. Then, based on the obtained attribute and class groups, we constructed the group-based attribute/object classifier collaborative learning model where the class groups are viewed as the extension of attribute groups. In order to explore the shared features within a group as well restrict the feature sharing between groups, we applied the structured sparse method to constrain the model parameter matrix. At last, a hybrid zero-shot classifying model is designed by simultaneously considering the class-class and class-attribute matrices to predict the class labels of testing images, where the class-class relationship is measured by the Jaccard similarity coefficient. Experiments on two popular attribute datasets show that, MTEAG can yield higher zero-shot image classification accuracy compared with several baselines.

In recent years, the deep reinforcement learning (DRL) algorithms have been developed rapidly and have achieved excellent performance in many challenging tasks. However, due to the complexity of network structure and a large amount of network parameters, the training of deep network is time-consuming, and consequently, the learning efficiency of DRL is limited. In this paper, aiming to speed up the learning process of DRL agent, we propose a novel approximate policy-based accelerated (APA) algorithm from the viewpoint of the error analysis of approximate policy iteration reinforcement learning algorithms. The proposed APA is proven to be convergent even with a more aggressive learning rate, making the DRL agent have a faster learning speed. Furthermore, to combine the accelerated algorithm with deep Q-network (DQN), Double DQN and deep deterministic policy gradient (DDPG), we proposed three novel DRL algorithms: APA-DQN, APA-Double DQN, and APA-DDPG, which demonstrates the adaptability of the accelerated algorithm with DRL algorithms. We have tested the proposed algorithms on both discrete-action and continuous-action tasks. Their superior performance demonstrates their great potential in the practical applications.

In zero-shot learning, attributes play as a bridge from original images to class labels. Therefore, to achieve accurate zero-shot image classification, we mainly focus on improving attribute prediction accuracy by taking full advantage of prior information about attribute from two aspects. First, we present a new attribute classifier called deep attribute prediction (DeepAP) model by using supervised deep convolutional neural networks (DCNNs), where the attribute label information participates in the training of DCNNs. Unlike common DCNNs that are usually used to extract image features, the constructed DCNNs are used to directly predict attribute values from the original input images. Thus, the designed DeepAP model can serve as the mapping from low-level image features to high-level semantic attributes in the traditional direct attribute prediction (DAP) model. Second, another prior information about attribute, i.e., class-attribute matrix is used to mine the attribute-class correlation with sparse representation coefficients. Since the attribute-class correlation can reflect different contributions of attributes to classification, we use it to define attribute weights and incorporate the idea of weighted attributes into DeepAP to form the deep weighted attribute prediction (DWAP) model. Experiments on three real datasets show that DWAP outperforms the deep attribute network and DAP on attribute prediction and zero-shot image classification.

For attribute-based zero-shot learning (ZSL), the attribute classifiers learned previously on the training images may not be usable for the testing images due to that the training and testing images may follow different data distributions. Since domain adaptation learning can effectively perform knowledge transfer under the circumstance of different data distributions, we proposed a novel ZSL method, referred to as multisource domain attribute adaptation based on adaptive multikernel alignment learning (A-MKAL), from the point of view of classifier adaptation. Considering there may be a large difference between object classes, we adopt the clustering method to group the training images according to the class–class correlation measured by the whitened cosine similarity, thus multiple source domains are created. The created multiple source domains are then combined into one weighted source domain to participate in the distribution discrepancy match across domains. In order to adapt the attribute classifier learned on the well-defined source domains to the target domain (the training image set), we designed the A-MKAL by applying the centered kernel alignment to align the attribute kernel matrix and the kernel function of adaptive multiple kernel learning. Experiments on Shoes, OSR, and AWA datasets show that, compared with state-of-the-art methods, our proposed method yields more accurate classification.

Due to the difficulty of labeling a large number of training samples of a hyperspectral image (HSI), unsupervised clustering methods have drawn great attention. The recently proposed broad learning (BL) can implement both linear and nonlinear mappings. However, the original BL is a supervised model. In this letter, a novel method named unsupervised BL (UBL) is introduced for HSI clustering. First, a graph-regularized sparse autoencoder is performed on the input and mapped feature of UBL in order to maintain the intrinsic manifold structure of origin HSI. Then, the objective function of UBL composed of an ℓ2-norm of output-layer weights and a graph regularization term is designed, which can be easily solved by choosing eigenvectors corresponding to the smallest eigenvalues. Finally, the HSI clustering results can be obtained by applying spectral clustering on the output of UBL. Experiments on three popular real HSI data sets demonstrate that, compared with several competitive methods, UBL can achieve better clustering performance.

Clustering is a powerful approach to analyze gene expression data which is crucial to the investigation of effective treatment of cancer. Many graph regularize-based clustering methods have been proposed and shown to be superior to the traditional clustering methods. However, they only focus on the inner structure in samples and fail to take the feature manifold into account. In gene expression data, it's practical to hypothesize that both the samples and the genes lie on nonlinear low dimensional manifolds, namely sample manifold and gene manifold, respectively. Therefore in this paper, incorporating the geometric structures in both samples and features, we propose a Dual Hypergraph Regularized PCA (DHPCA) method for biclustering of tumor data. First, for gene expression data, we construct two hypergraphs, i.e., sample hypergraph and gene hypergraph, to estimate the intrinsic geometric structures of samples and genes. Then, we introduce the hypergraph regularization on both gene side and sample side. Finally, our biclustering method is formulated as two hypergraph regularized PCA with closed-form solution. We experimentally validate our proposed DHPCA algorithm on real applications and the promising results indicate its potential in high dimension data analysis.

Tumor clustering is a powerful approach for cancer class discovery which is crucial to the effective treatment of cancer. Many traditional clustering methods such as NMF-based models, have been widely used to identify tumors. However, they cannot achieve satisfactory results. Recently, subspace clustering approaches have been proposed to improve the performance by dividing the original space into multiple low-dimensional subspaces. Among them, low rank representation is becoming a popular approach to attain subspace clustering. In this paper, we propose a novel Low Rank Subspace Clustering model via Discrete Constraint and Hypergraph Regularization (DHLRS). The proposed method learns the cluster indicators directly by using discrete constraint, which makes the clustering task simple. For each subspace, we adopt Schatten -norm to better approximate the low rank constraint. Moreover, Hypergraph Regularization is adopted to infer the complex relationship between genes and intrinsic geometrical structure of gene expression data in each subspace. Finally, the molecular pattern of tumor gene expression data sets is discovered according to the optimized cluster indicators. Experiments on both synthetic data and real tumor gene expression data sets prove the effectiveness of proposed DHLRS.

For the zero-shot image classification with relative attributes (RAs), the traditional method requires that not only all seen and unseen images obey Gaussian distribution, but also the classifications on testing samples are made by maximum likelihood estimation. We therefore propose a novel zero-shot image classifier called random forest based on relative attribute. First, based on the ordered and unordered pairs of images from the seen classes, the idea of ranking support vector machine is used to learn ranking functions for attributes. Then, according to the relative relationship between seen and unseen classes, the RA ranking-score model per attribute for each unseen image is built, where the appropriate seen classes are automatically selected to participate in the modeling process. In the third step, the random forest classifier is trained based on the RA ranking scores of attributes for all seen and unseen images. Finally, the class labels of testing images can be predicted via the trained RF. Experiments on Outdoor Scene Recognition, Pub Fig, and Shoes data sets show that our proposed method is superior to several state-of-the-art methods in terms of classification capability for zero-shot learning problems.

Deep reinforcement learning (RL) comprehensively uses the psychological mechanisms of "trial and error" and "reward and punishment" in RL as well as powerful feature expression and nonlinear mapping in deep learning. Currently, it plays an essential role in the fields of artificial intelligence and machine learning. Since an RL agent needs to constantly interact with its surroundings, the deep Q network (DQN) is inevitably faced with the need to learn numerous network parameters, which results in low learning efficiency. In this paper, a multisource transfer double DQN (MTDDQN) based on actor learning is proposed. The transfer learning technique is integrated with deep RL to make the RL agent collect, summarize, and transfer action knowledge, including policy mimic and feature regression, to the training of related tasks. There exists action overestimation in DQN, i.e., the lower probability limit of action corresponding to the maximum Q value is nonzero. Therefore, the transfer network is trained by using double DQN to eliminate the error accumulation caused by action overestimation. In addition, to avoid negative transfer, i.e., to ensure strong correlations between source and target tasks, a multisource transfer learning mechanism is applied. The Atari2600 game is tested on the arcade learning environment platform to evaluate the feasibility and performance of MTDDQN by comparing it with some mainstream approaches, such as DQN and double DQN. Experiments prove that MTDDQN achieves not only human-like actor learning transfer capability, but also the desired learning efficiency and testing accuracy on target task.

The existing attribute‐based zero‐shot learning models at different levels ignore some necessary prior knowledge. It is essential to improve classification accuracy of zero‐shot learning that how to mine attribute‐related and class‐related prior knowledge further being incorporated into the attribute prediction models. For the mining of class‐related prior knowledge, measurement of the class–class correlation by using whitened cosine similarity is proposed. Likewise for the mining of attribute‐related prior knowledge, measurements of the attribute–class and attribute–attribute correlation are proposed by using sparse representation coefficient. Therefore, a novel indirect attribute prediction (IAP) model is presented by exploiting class‐related and attribute‐related prior knowledge (IAP_CAPK). Experimental results on animals with attributes and a‐Pascal/a‐Yahoo datasets show that, when compared with IAP and direct attribute prediction, the proposed IAP_CAPK not only yields more accurate attribute prediction and zero‐shot image classification, but also achieves much higher computational efficiency.

To take full advantage of hyperspectral information, to avoid data redundancy and to address the curse of dimensionality concern, dimensionality reduction (DR) becomes particularly important to analyze hyperspectral data. Exploring the tensor characteristic of hyperspectral data, a DR algorithm based on class-aware tensor neighborhood graph and patch alignment is proposed here. First, hyperspectral data are represented in the tensor form through a window field to keep the spatial information of each pixel. Second, using a tensor distance criterion, a class-aware tensor neighborhood graph containing discriminating information is obtained. In the third step, employing the patch alignment framework extended to the tensor space, we can obtain global optimal spectral-spatial information. Finally, the solution of the tensor subspace is calculated using an iterative method and low-dimensional projection matrixes for hyperspectral data are obtained accordingly. The proposed method effectively explores the spectral and spatial information in hyperspectral data simultaneously. Experimental results on 3 real hyperspectral datasets show that, compared with some popular vector- and tensor-based DR algorithms, the proposed method can yield better performance with less tensor training samples required.

BACKGROUND: Previous studies have shown modular structures in PPI (protein-protein interaction) networks. More recently, many genome and metagenome investigations have focused on identifying modules in PPI networks. However, most of the existing methods are insufficient when applied to networks with overlapping modular structures. In our study, we describe a novel overlapping module identification method (OMIM) to address this problem. RESULTS: Our method is an agglomerative clustering method merging modules according to their contributions to modularity. Nodes that have positive effects on more than two modules are defined as overlapping parts. As well, we designed de-noising steps based on a clustering coefficient and hub finding steps based on nodal weight. CONCLUSIONS: The low computational complexity and few control parameters prove that our method is suitable for large scale PPI network analysis. First, we verified OMIM on a small artificial word association network which was able to provide us with a comprehensive evaluation. Then experiments on real PPI networks from the MIPS Saccharomyces Cerevisiae dataset were carried out. The results show that OMIM outperforms several other popular methods in identifying high quality modular structures.