Quantum Dempster-Shafer theory (QDST) derives a quantum mass function (QMF), a fuzzy metric obtained from multiple information sources based on quantum interference. In general, QMF effectively represents and processes uncertain information, but managing conflicts among multiple QMFs remains challenging. To address this issue, we propose a novel quantum conflict indicator (QCI) within the QDST framework. It is the first metric satisfying ideal conflict measurement properties, including non-negativity, symmetry, boundedness, extreme consistency, and insensitivity to refinement. Based on QCI, a novel quantum conflict fusion method (QCI-Fusion) is introduced to fuse highly conflicting QMFs. Moreover, traditional methods, including QCI-Fusion, typically constructs the Quantum Frame of Discernment (QFoD) based on predicted labels, which makes it difficult to cover unseen classes. Therefore, a new decision architecture, QCI-Decision, is proposed for unsupervised detection that rejects out-of-distribution (OOD) samples while maintaining in-distribution (ID) classification. Experimental results show that the classification accuracy of QCI-Decision deviates from the original model predictions by at most 1.49%. Meanwhile, compared with the latest OOD detection methods, QCI-Decision improves the Area Under the Receiver Operating Characteristic Curve (AUC) by up to 0.6% and reduces the False Positive Rate at 95% True Negative Rate (FPR) by up to 1.63%. Moreover, compared with QCI-Fusion, QCI-Decision achieves approximately threefold faster fusion speed with negligible performance degradation, offering a promising solution for open-world quantum information decision.

Few-shot learning seeks to recognize novel classes from limited examples. Model-agnostic meta-learning (MAML), known for its simplicity and flexibility, learns an effective initialization for fast adaptation in data-scarce settings. However, MAML-based methods face challenges when there is a significant distributional shift between training and testing tasks, leading to inefficient learning and poor generalization across domains. In this work, we identify the core issues: inflexible weight update rules and limited adaptive learning capabilities. Instead of focusing solely on better initialization, we aim to enhance the adaptation process. Consequently, we propose a novel Layer-Adaptive Proportional-Integral-Derivative (LA-PID) optimizer integrated into a meta-learning framework. This design incorporates classical control theory, utilizing PID control to dynamically adjust task-specific gains at each network layer. Additionally, the theoretical conditions for optimal hyperparameter initialization and global model convergence are addressed from both control and optimization perspectives. Experiments on benchmark datasets show that LA-PID achieves state-of-the-art performance in few-shot classification, cross-domain, and regression tasks, while requiring fewer training steps.

This paper presents a unified framework for continual learning in Audio-Visual Question Answering (AVQACL), designed to enhance fine-grained scene understanding and spatio-temporal reasoning in dynamic multimodal task environments. To simulate realistic incremental learning scenarios, we construct two large-scale benchmark datasets, Split-AVQA and Split-MUSIC-AVQA, by reorganizing existing AVQA corpora into sequential tasks. Empirical results show that conventional models suffer from severe performance degradation and catastrophic forgetting when learning across audio, visual, and textual modalities in a continual setup. To address these challenges, we propose a novel approach that integrates four key modules: (1) Question-Guided Cross-modal Information Fusion (QCIF), which dynamically extracts task-relevant multimodal features via question-aware attention; (2) Task-specific Knowledge Distillation with Spatial-Temporal Feature Constraints (TKD-STFC), which preserves semantic output behavior and internal reasoning trajectories across tasks; (3) Question Semantic Consistency Constraint (QSCC), which regularizes evolving question representations to maintain linguistic stability. (4) Dual-Strategy Exemplar Selection (DSES), a memory-efficient replay strategy that jointly maximizes sample informativeness and diversity. All components are theoretically grounded, and formal analysis is provided to ensure modality alignment, spatial-temporal coherence, and exemplar selection reliability. Extensive experiments on both datasets demonstrate that our method consistently outperforms prior state-of-the-art AVQACL baselines in terms of accuracy, retention, and robustness.

The threat of adversarial attacks on artificial intelligence in uncrewed systems has raised significant concerns regarding security. At a finer granularity, target-level adversarial attacks in instance segmentation play a critical role. However, existing methods are limited to generating disordered pixel perturbations, making it difficult to precisely control the attack targets. To address this challenge, a novel controllable attack for instance segmentation (CAIS) is proposed. CAIS precisely manipulates foreground targets while avoiding misleading the background, thereby enhancing the stealthiness of the attack. Specifically, CAIS introduces a focal gradient enhancement method to guide the victim detector to focus on foreground regions, strengthening the gradient signals for adversarial attacks. In addition, we design an attention guided perturbation method, which reduces the interference of background information through adaptive foreground–background gradient separation. This article introduces a dual loop asynchronous optimization strategy, integrating the aforementioned methods into the CAIS attack. Extensive experiments demonstrate that CAIS achieves superior attack performance on victim instance regions while maintaining prediction consistency in nonvictim regions. Consequently, this advancement will inspire the design of adversarial threat detection and defense mechanisms, ultimately contributing to the safety and reliability of uncrewed systems.

A vision-driven driver monitoring system plays a vital role of vehicular cyber-physical systems (VCPS) to guarantee the driving safety. Recent advances focus on modeling a deep learning-based method to realize the driver monitoring system, which benefits from the powerful capability of data-driven feature extraction. Although the acceptable performances of driver state monitoring methods are achieved, there is still a gap between the emerged techniques and actual application scenarios. First, the human-centric visual appearances are not involved comprehensively to represent the driver states, resulting in ignoring the contextual interaction of behaviors. Second, the inherent uncertainty of driver situation is not considered, while the unreliable samples would lead to the untrustworthy results. In this paper, we focus on a vision-based driver state monitoring method, where a trustworthy driver state perception (TDSP) is proposed via human-centric contextual interaction-driven evidential vision-language fusion in VCPS. Specifically, a vision-language model-based architecture is first modified in temporal dimension to represent the visual human-centric contextual interactions, while a vision-language consistency loss is designed to mitigate the gap between visual and textual representations. Then, an evidence-based learning method is introduced to jointly conduct the classification and uncertainty estimation for driver states. Furthermore, to model the human-centric contextual interactions towards the evidence-based paradigm comprehensively, Dempster-Shafer theory-based combination rule is introduced to fuse the visual and textual representations. Extensive experiments are conducted on two public benchmarks, where the superiority of TDSP is demonstrated compared with the state-of-the-art methods. The superior performance of TDSP to recognize dangerous states are 85.41% and 83.12% in terms of accuracy and F1 score, which outperforms the state-of-the-art methods by 4.68% and 3.99%. Moreover, we validate the reliability of TDSP against the noisy data for VCPS. The code will be public at https://github.com/w64228013/TDSP.

The intelligent fault diagnosis has achieved notable success in identifying known mechanical failures; however, reliably detecting out-of-distribution (OOD) faults remains a key challenge to achieve the diagnostic robustness. In industrial applications, vibration signals are typically collected as time-series data whose dynamic characteristics vary with load, speed, and environmental interference, with weak early fault patterns that blur class boundaries. As a result, models trained under limited laboratory conditions inevitably encounter unseen OOD inputs after deployment, requiring the ability to recognize and reject them reliably. Existing representation- and similarity-based OOD methods have shown promise but typically rely on single-granularity prototypes, capturing only coarse similarity structures and overlooking latent subclass relations—thus limiting the generalization under complex degradation modes. To address these limitations, we propose a multigranularity fuzzy inference (MgFI) framework for enhanced uncertainty quantification in fault diagnosis. MgFI models fine-grained subclass memberships on a hyperspherical manifold, aggregates them into class-level fuzzy sets, and infers coarse-grained In-distribution (ID) confidence through the hierarchical fuzzy reasoning. Extensive experiments demonstrate that MgFI substantially improves the OOD detection accuracy and provides a principled, interpretable framework for trustworthy open-set industrial diagnostics.

Classifiers fusion can be seen as a kind of multisource information fusion (MSIF), and classifiers fusion based on Dempster–Shafer (DS) evidence theory is an effective approach to improve the accuracy of classification tasks. However, different classifiers usually exhibit varying performances, making it challenging to achieve enhanced classification accuracy through direct fusion. Simultaneously, when the frame of discernment (FoD) of the target class expands, the number of focal elements involved in the fusion increases, resulting in a rapid growth in computational complexity. To enhance the classification performance while reducing the time cost of fusion, a novel weighted fusion of classifiers method based on approximate reasoning and reliability evaluation (WFC-AR-RE) is proposed in this article. Specifically, at first, the key focal elements are determined based on the outputs of classifiers, and an approximate basic belief assignment (BBA) is generated. Subsequently, the validation set is utilized to evaluate the performance of each classifier, thus obtaining the self-reliability of each BBA. Afterward, a novel divergence measure is introduced to quantify the discrepancy between BBAs, determining the relative reliability of each BBA. Finally, the fusion weight of each BBA is derived from its self-reliability and relative reliability, and Dempster’s rule is applied to combine the weighted BBA. The proposed WFC-AR-RE algorithm is applied to the MSIF system, and its effectiveness is demonstrated on 12 public datasets.

In this paper, a novel benchmark for audio-visual question answering continual learning (AVQACL) is introduced, aiming to study fine-grained scene understanding and spatial-temporal reasoning in videos under a continual learning setting. To facilitate this multimodal continual leaning task, we create two audio-visual question answering continual learning datasets, named Split-AVQA and Split-MUSIC-AVQA based on the AVQA and MUSIC-AVQA datasets, respectively. The experimental results suggest that the model exhibits limited cognitive and reasoning abilities and experiences catastrophic forgetting when processing three modalities simultaneously in a continuous data stream. To address above challenges, we propose a novel continual learning method that incorporates question-guided cross-modal information fusion (QCIF) to focus on question-relevant details for improved feature representation and task-specific knowledge distillation with spatial-temporal feature constraints (TKD-STFC) to preserve the spatial-temporal reasoning knowledge acquired from previous dynamic scenarios. Furthermore, a question semantic consistency constraint (QSCC) is employed to ensure that the model maintains a consistent understanding of question semantics across tasks throughout the continual learning process. Extensive experimental results on Split-AVQA and Split-MUSIC-AVQA datasets illustrate that our method achieves state-of-the-art audio-visual question answering continual learning performance. The code is available at https://github.com/kx-wu/AVQACL.

Flapping-wing robots exhibit numerous advantages in flight performance, which mimic the natural flight of birds or insects. However, autonomous takeoff remains a significant challenge for large-sized bird-like flapping-wing robots. To address this challenge, we design a jumping mechanism based on a bow-like carbon fiber spring. This mechanism is capable of repeated self-compression and release and can be readily integrated into flapping-wing robots to assist their jumping takeoff. Then we conduct a mechanical analysis of the motion states during the jumping takeoff process. Additionally, we propose a jump-flapping coupling control method based on sensor data to ensure a seamless transition and smooth coordination between the jumping and flapping action, thus enabling a smooth takeoff. Experimental results validate the effectiveness of the proposed jumping mechanism and its collaborative control strategy. This study provides support for advancing flapping-wing robots toward autonomous multi-modal locomotion and further deepens research in this field.

In human communication, referring to a specific object within an environment often involves the combination of a pointing gesture to indicate the object’s direction and linguistic descriptions specifying its name and attributes, thereby enabling precise object identification. Inspired by this natural multimodal interaction, we formalize the zero-shot object navigation driven by language and pointing gesture (LGZSON) task, which aims to more closely approximate real-world human-agent communication scenarios. To address this task, we propose LGNav, an open-set, training-free navigation framework. LGNav estimates the pointing gesture direction by extracting human body landmarks and integrates this directional information with depth images to initialize a versatile candidate position map (VCPM). The framework further employs open-vocabulary object detection to identify all potential candidate objects in the environment, projecting them onto the VCPM. Guided by a motion policy derived from the VCPM, LGNav continuously explores the unknown environment, sequentially visits candidate objects, and utilizes a large vision-language model (LVLM) to verify whether each candidate object satisfies the given navigation instruction. Extensive experimental results validate the effectiveness of LGNav, demonstrating its strong performance in the LG-ZSON task. Furthermore, even in the absence of pointing gestures, LGNav achieves competitive results on standard object navigation benchmarks, including the Gibson and HM3D datasets, outperforming a range of strong baseline methods.

The fault diagnosis based on supervised learning has achieved remarkable results in the intelligent manufacturing, making it an important guarantee for long-term safe and stable operation in modern industry. However, the accuracy heavily relies on high-quality annotation labels, which are expensive to obtain, limiting the diagnosis models applicability in many scenarios. Although obtaining automatically annotated samples from annotators is a promising solution, the generated dataset is always containing incorrect labels (noisy labels), due to perceptual limitations, resulting in low or even invalid the accuracy of model. With the goal of handling this challenge, a diagnostic approach based on multi-granularity information fusion to combat noisy labels, called MgCNL, is proposed, to train the model with high-accuracy, without knowing the specific noise ratio. Specifically, inspired by granular-ball computing, a confidence evaluation method of labels is designed, so that samples with high confidence labels can be selected from dataset with noisy labels for supervised learning, thus avoiding the negative impact of incorrect labels on model performance. Finally, the efficacy was demonstrated on three datasets using different backbones: MgCNL successfully reduced the adverse impact of noisy labels, achieving significantly better results than other advanced methods in various noisy scenarios, which offers a competitive model training strategy for practitioners in intelligent manufacturing or industrial fault diagnosis who are hampered by the costs associated with sample labeling. Note to Practitioners—In modern industry, the cost of manual/expert annotation for high-quality data is is prohibitively expensive, and the data annotated by automatic annotators often contains noisy labels that seriously damages the accuracy of models, which makes many data-driven diagnosis models constrained by training data and difficult to put into practice, posing an urgent challenge to the automation and intelligence of the manufacturing industry. To address this challenge, this article proposed a robust training strategy called MgCNL, aimed at offsetting the negative impact of noisy labels, in the hope that automatic annotation strategy with lower cost can be more widely applied in model training tasks for industrial practice. MgCNL, based on multi-granularity information, can effectively select high-confidence samples from datasets for supervised learning, even under unknown proportions of noise labels, thus reducing the misleading impact of noisy labels on diagnostic models. As a result, MgCNL possesses the ability to robustly train high-accuracy diagnostic models in data with noisy labels, thus enabling automatic annotators to replace experts in dataset construction as a more economical and efficient potential technical approach. Meanwhile, MgCNL also brings value to datasets with uncertain labels, making them applicable without the need to invest significant human resources to verify label reliability.

Dempster-Shafer (DS) evidence theory provides a powerful framework for modeling uncertainty, reasoning, and combining information from multiple sources. However, it may yield counter-intuitive results when handling conflicting evidence, thereby affecting decision reliability and limiting practical applications. To address this issue, this work proposes a novel Evidential Reasoning rule with Divisive Hierarchical Clustering (ER-DHC), consisting of two main modules: evidence clustering and cluster fusion. At first, a new divisive hierarchical algorithm is introduced for evidence clustering, comprising coarse-grained and fine-grained division. In the coarse-grained stage, evidence with different decision preferences is grouped into separate clusters, thus preventing high intra-cluster conflicts and laying a solid foundation for evidence clustering. The fine-grained division adaptively refines cluster structures using an inflection point detection method, thereby enhancing clustering quality. On this basis, a new cluster fusion strategy is developed, involving intra-cluster fusion via classical Dempster's rule and inter-cluster fusion using a fuzzy preference relation-based weighted approach. This fusion strategy can degenerate into classical DS fusion and weighted fusion, while also introducing a new clustering fusion perspective, offering better flexibility. Finally, the proposed ER-DHC method is applied to the multi-source information fusion system, with experimental results demonstrating improved performance of target classification.

Adversarial attacks present a severe threat to the trustworthiness of remote sensing uncrewed systems. Existing detection methods are mostly limited to binary classification, lacking fine-grained traceability and relying heavily on adversarial example (AE) training. To overcome these challenges, we propose a training-free hierarchical adversarial attack traceability framework leveraging the zero-shot transfer ability of contrastive language-image pretraining (CLIP), eliminating the dependency on AE training. For the first time, we establish a multigranularity attack propagation path analysis system. Specifically, the framework leverages CLIP for fine-grained adversarial attack classification without fine-tuning, constructing a hierarchical traceability system (HTS) from coarse-grained to fine-grained semantic levels. Based on Bayesian inference, we design a hierarchical probability fusion method that improves coarse-grained results through maximum a posteriori (MAP) estimation of fine-grained classifiers, collaboratively optimizing hierarchical probability distributions. To capture frequency-domain characteristics of adversarial attacks, we propose a frequency-aware kernel approximation combining high-frequency-enhanced radial basis function (RBF) kernels with ridge regression, improving sensitivity to subtle perturbations in the reproducing kernel Hilbert space (RKHS). This study establishes a comprehensive traceability system for attack propagation chains, providing an interpretable, training-free paradigm for remote sensing security, significantly enhancing fine-grained detection capabilities of uncrewed systems. Extensive experiments on two public remote sensing datasets, RSSSN7 and DWEFS, and one proprietary dataset demonstrate that our method significantly improves adversarial classification accuracy across two victim models, achieving gains of 17.13%/18.89%/13.44% and 17.91%/19.58%/12.17%, respectively, compared with state-of-the-art (SOTA) training-free baselines.

Human activity recognition (HAR) based on wearable devices has received significant attention from scholars in recent years. Nevertheless, the lack of effective exploitation of multiview learning and limited capacity for uncertainty analysis still remain major challenges for high-precision and high-confidence activity recognition. Thus, this article proposes a novel multiview uncertainty-aware graph convolutional network (MVUAGCN) model. Specifically, MVUAGCN first divides the raw time series data into multiview data according to the sensor type, and then structures the derived data into multiview graph topology. After that, the multiview residual graph convolutional networks with the Chebyshev polynomial are deployed to generate the sources of evidence (SoEs). Then, all involved multiview SoEs are mapped into the evidence space through the Dirichlet distribution to obtain the uncertainty degree in MVUAGCN. Finally, all the mapped SoEs are fused sequentially and the decision is made according to the maximum probability. The comprehensive experimental evaluations were conducted on four publicly HAR datasets. With the nearly 5% improvement compared to CNN-based approaches, MVUAGCN achieves 99.06%, 100%, 97.84%, and 98.25% recognition accuracy for all the four datasets: PAMAP2, MHEALTH, OPPORTUNITY, and UCI HAR, respectively.

Deep learning has demonstrated exceptional performance in fault diagnosis tasks that rely on large-scale datasets. However, the high cost of annotating such datasets has led to the emergence of various automatic annotation methods. While these methods reduce labeling costs, they inevitably introduce noisy labels, which pose significant challenges to the generalization and accuracy of deep learning-based diagnostic models. Although Learning with Noisy Labels (LNL) methods mitigate the adverse effects of noisy labels through strategies such as sample separation or label correction, many rely heavily on their own prediction results to guide subsequent training, which often introduces confirmation bias, limiting the effectiveness of the trained models. To address this limitation, this article draws inspiration from quantum collapse and proposes a novel LNL strategy named Multigranularity Labeling (MgL). By integrating observed labels, pseudolabels, and collapsed labels, MgL constructs the multigranularity labels, designed to suppress confirmation bias and improve the model's tolerance to noisy labels. Extensive experiments validate the effectiveness and superiority of MgL, particularly on training datasets with high noise intensity, such as those with 90% symmetric noise. This advancement offers promising opportunities for applying datasets with lower annotation costs in real-world scenarios, ultimately contributing to intelligent diagnostic systems.

Model-Agnostic Meta-Learning (MAML) and its variants have shown remarkable performance in scenarios characterized by a scarcity of labeled data during the training phase of machine learning models. Despite these successes, MAMLbased approaches encounter significant challenges when there is a substantial discrepancy in the distribution of training and testing tasks, resulting in inefficient learning and limited generalization across domains. Inspired by classical proportional-integral-derivative (PID) control theory, this study introduces a Layer-Adaptive PID (LA-PID) Optimizer, a MAML-based optimizer that employs efficient parameter optimization methods to dynamically adjust task-specific PID control gains at each layer of the network, conducting a first-principles analysis of optimal convergence conditions. A series of experiments conducted on four standard benchmark datasets demonstrate the efficacy of the LA-PID optimizer, indicating that LA-PID achieves state-oftheart performance in few-shot classification and cross-domain tasks, accomplishing these objectives with fewer training steps. Code is available on https://github.com/yuguopin/LA-PID.

In the absence of a pre-built indoor map, robot navigation suffers from the limitations of sensors and environments, resulting in decreased efficiency in performing ad-hoc tasks. Given that blueprints are difficult to obtain, an intuitive method is to provide robots with prior knowledge via hand-drawn floor plans. However, due to the inability of robots to directly comprehend hand-drawn styles, the applicability of this method is limited. In this paper, we present a novel framework for hand-drawn floor plan reconstruction that can recognize abstract hand-drawn elements and standardize the reconstruction of hand-drawn floor plans, thereby providing robots with valuable global map information. Specifically, we design a new series of structure descriptors as reconstruction components and employ a deep learning-based model for recognition. Then the standardized results are obtained through the proposed floor plan reconstruction algorithm. To verify the effectiveness of the framework, we conduct experiments on electronic and paper hand-drawn floor plans. Compared with other state-of-the-art methods, our proposed method achieves superior reconstruction results. This work expands the application scenarios for indoor robots, enabling them to quickly comprehend the semantics of complex scenes, thereby enhancing the competitiveness in downstream tasks.

Unmanned aerial vehicles (UAVs) represent an essential component of advanced intelligent equipment that can be used as an aerial perception system by installing various sensors such as vision, hearing, touch, taste, and smell to achieve intelligently integrated perception of environments. However, these perception system with environmental information may be threatened by various internal and external attacks, causing a great challenge to the security of the UAV. The original security system relied on an expert knowledge base to prevent attacks, but the weaknesses of lacking proactivity and flexibility are gradually exposed. The strong resistance and survivability of biological systems can be used to fill this capability gap and provide new ideas for the security of the UAV perception system. Therefore, an endogenous security framework (ESUAV-NI) based on the neural system and immune system is proposed in this article. Through breeding artificial intelligence (AI) vaccines and distributed neural hierarchical control, we achieve the security protection for the UAV perception system. Moreover, we evaluated the AI vaccine breeding approach in the ESUAV-NI by conducting extensive experiments on internal threats and external aerial imagery camouflage data, respectively. The results show that the proposed approach has a superior performance for the UAV perception system.

This paper studies the secure and safe control problem of connected and automated vehicles (CAVs) with false data injection (FDI) attacks. A secure and safe controller with a novel surrounding vehicles’ state estimator and an attack detector is proposed. The state estimation for surrounding vehicles is collectively processed by combining the deep neural network-based predictions with model-based estimations. Additionally, a weight in the loss function is proposed for more accurate predictions of vehicles that are closer to the ego vehicle. For attack detection, a novel scheme that utilizes control outcomes to train detection actions is proposed. Different from the objectives of existing detectors, the reward for the proposed detector is designed to encourage the CAV to fully utilize the observations. A reward setting and the decision preference of the ego vehicle are theoretically analyzed. The effectiveness of the proposed algorithm is validated in an open simulation environment.

In this work, we propose a voxel-based single-stage fine-grained and efficient point cloud 3D object detection algorithm to address the inadequate granularity in point cloud feature extraction tasks and the imbalance between efficiency and accuracy in single-stage point cloud 3D object detection scenarios. We develop a lightweight multibranch cross-sparse convolution network (LMCCN) that is designed to preserve the feature granularity of the original point cloud while achieving enhanced extraction efficiency. Additionally, we introduce a compact fine-grained self-attention augmented bird’s eye view (BEV) feature extraction module (CFSAM). This module aims to further refine BEV features, enabling the acquisition of both locally and globally enhanced features and thereby augmentingthe perceptual capabilities of the constructed model. Without bells and whistles, the proposed method attains excellent performance on many autonomous driving benchmarks, with detection accuracies of up to 81.67% on KITTI, 72.74% on ONCE, and 84.00% on nuScenes. Moreover, it reaches a peak detection speed of 46.08 FPS, effectively balancing accuracy with speed.

The belief functions (BFs) introduced by Shafer in the mid of 1970s are widely applied in information fusion to model epistemic uncertainty and to reason about uncertainty. Their success in applications is however limited because of their high-computational complexity in the fusion process, especially when the number of focal elements is large. To reduce the complexity of reasoning with BFs, we can envisage as a first method to reduce the number of focal elements involved in the fusion process to convert the original basic belief assignments (BBAs) into simpler ones, or as a second method to use a simple rule of combination with potentially a loss of the specificity and pertinence of the fusion result, or to apply both methods jointly. In this article, we focus on the first method and propose a new BBA granulation method inspired by the community clustering of nodes in graph networks. This article studies a novel efficient multigranular belief fusion (MGBF) method. Specifically, focal elements are regarded as nodes in the graph structure, and the distance between nodes will be used to discover the local community relationship of focal elements. Afterward, the nodes belonging to the decision-making community are specially selected, and then the derived multigranular sources of evidence can be efficiently combined. To evaluate the effectiveness of the proposed graph-based MGBF, we further apply this new approach to combine the outputs of convolutional neural networks + attention (CNN + Attention) in the human activity recognition (HAR) problem. The experimental results obtained with real datasets prove the potential interest and feasibility of our proposed strategy with respect to classical BF fusion methods.

Human Activity Recognition (HAR) based on wear-able device has become a hot topic of research due to its wide range of applications in health-care, fitness and smart homes. However, the classification of some activities with similar sensor readings, such as standing and sitting, is usually more challenging for the design of efficient activity recognition algorithms. Considering the inconsistent performance of different classifiers, which can provide information complementary for individual classifier, we propose a novel multi-classifier fusion method based on belief functions (BFs) theory for HAR. Specifically, at first, four classifiers are trained using time-domain and frequency-domain features to obtain basic belief assignments (BBA) of activity, respectively. Then, three assessment criteria are utilized to evaluate the reliability of the classifiers and a scoring matrix is constructed. Next, the algorithm of Belief Function based the Technique for Order Preference by Similarity to Ideal Solution (BF-TOPSIS) is employed to calculate the weighting coefficients for each classifier. Finally, the discounting and Dempster’s rules are adopted to combine the multiple classifiers and further decision making. Several experiments were conducted to illustrate the performance of the proposed method using the UCI smartphone dataset, and the results show that the proposed method is more accurate than the state-of-art methods.

For multi-agent area coverage path planning problem, existing researches regard it as a combination of Traveling Salesman Problem (TSP) and Coverage Path Planning (CPP). However, these approaches have disadvantages of poor observation ability in online phase and high computational cost in offline phase, making it difficult to be applied to energy-constrained Unmanned Aerial Vehicles (UAVs) and adjust strategy dynamically. In this paper, we decompose the task into two sub-problems: multi-agent path planning and sub-region CPP. We model the multi-agent path planning problem as a Collective Markov Decision Process (C-MDP), and design an Energy Constrained Multi-Agent Reinforcement Learning (ECMARL) algorithm based on the centralized training and distributed execution concept. Taking into account energy constraint of UAVs, the UAV propulsion power model is established to measure the energy consumption of UAVs, and load balancing strategy is applied to dynamically allocate target areas for each UAV. If the UAV is under energy-depleted situation, ECMARL can adjust the mission strategy in real time according to environmental information and energy storage conditions of other UAVs. When UAVs reach each sub-region of interest, Back-an-Forth Paths (BFPs) are adopted to solve CPP problem, which can ensure full coverage, optimality and complexity of the sub-problem. Comprehensive theoretical analysis and experiments demonstrate that ECMARL is superior to the traditional offline TSP-CPP strategy in terms of solution quality and computational time, and can effectively deal with the energy-constrained UAVs.

In recent years, wearable sensor-based human activity recognition (HAR) is becoming more and more attractive, especially in health monitoring and sports management. However, in order to obtain high-quality HAR, it is often necessary to get sufficient labeled activity data, which is very difficult, time-consuming, and costly in a natural environment. To tackle this problem, multisource domain adaptation (DA) is a promising method that aims to learn enough multisource prior knowledge from labeled activity data, and then transfer this learned knowledge to the target unlabeled dataset. Thus, this article presents a novel multisource weighted DA with evidential reasoning (w-MSDAER) for HAR, which can effectively utilize complementary knowledge between multiple sources. Specifically, we first use the strategy of distribution alignment to learn local domain-invariant classifiers based on multisource domains. And then the reliabilities of these derived classifiers are comprehensively evaluated according to the belief function based technique for order preference by similarity to ideal solution (BF-TOPSIS). Finally, the discounting fusion method is used to fuse the local classification results. Comprehensive experiments are conducted on two open-source datasets, and the results show that the proposed w-MSDAER significantly outperforms other state-of-art methods.

To improve the efficiency of deep reinforcement learning (DRL)-based methods for robot manipulator trajectory planning in random working environments, we present three dense reward functions. These rewards differ from the traditional sparse reward. First, a posture reward function is proposed to speed up the learning process with a more reasonable trajectory by modeling the distance and direction constraints, which can reduce the blindness of exploration. Second, a stride reward function is proposed to improve the stability of the learning process by modeling the distance and movement distance of joint constraints. Finally, in order to further improve learning efficiency, we are inspired by the cognitive process of human behavior and propose a stage incentive mechanism, including a hard-stage incentive reward function and a soft-stage incentive reward function. Extensive experiments show that the soft-stage incentive reward function is able to improve the convergence rate, get higher mean reward and lower standard deviation after convergence.

Group re-identification (GReID) is an important yet less-studied task. GReID focuses on associating the group images across non-overlapping cameras. The key challenges of GReID include layout variation, membership changes, and occlusion. Most existing methods focus on the group variation but ignore the occlusion that frequently occurs in the group. In practical scenarios, so many activities like conversing, queuing, or fighting consist of groups. To this end, we design a novel Transformer-based Siamese Network for GReID (TSN-GReID) for joint learning of classification and correspondence tasks to learn more robust group features for group layout and membership changes. Furthermore, we put forward an original regrouping random patch module(RRPM) which respectively regroups the member patch embedding and member-level local features to generate group features with improved discrimination ability and more diversified coverage to deal with occlusion. Experimental results demonstrate the effectiveness of our approach, which significantly outperforms state-of-the-art methods by 4.6 % Rank-1 on the CUHK-SYSU Group (CSG) dataset and by 7.1% Rank-1 on the DukeMTMC Group dataset.

Recognizing the human emotion automatically from visual characteristics plays a vital role in many intelligent applications. Recently, gait-based emotion recognition, especially gait skeletons-based characteristic, has attracted much attention, while many available methods have been proposed gradually. The popular pipeline is to first extract affective features from joint skeletons, and then aggregate the skeleton joint and affective features as the feature vector for classifying the emotion. However, the aggregation procedure of these emerged methods might be rigid, resulting in insufficiently exploiting the complementary relationship between skeleton joint and affective features. Meanwhile, the long range dependencies in both spatial and temporal domains of the gait sequence are scarcely considered. To address these issues, we propose a novel two-stream network with transformer-based complementarity, termed as TNTC. Skeleton joint and affective features are encoded into two individual images as the inputs of two streams, respectively. A new transformer-based complementarity module (TCM) is proposed to bridge the complementarity between two streams hierarchically via capturing long range dependencies. Experimental results demonstrate that TNTC outperforms state-of-the-art methods on the latest dataset in terms of accuracy.

To realize a robust robotic grasping system for unknown objects in an unstructured environment, large amounts of grasp data and 3D model data for the object are required; the sizes of these data directly affect the rate of successful grasps. To reduce the time cost of data acquisition and labeling and increase the rate of successful grasps, we developed a self-supervised learning mechanism to control grasp tasks performed by manipulators. First, a manipulator automatically collects the point cloud for the objects from multiple perspectives to increase the efficiency of data acquisition. The complete point cloud for the objects is obtained using the hand-eye vision of the manipulator and the truncated signed distance function algorithm. Then, the point cloud data for the objects are used to generate a series of six-degrees-of-freedom grasp poses, and the force-closure decision algorithm is used to add the grasp quality label to each grasp pose to realize the automatic labeling of grasp data. Finally, the point cloud in the gripper closing area corresponding to each grasp pose is obtained and used to train the grasp-quality classification model for the manipulator. The results of performing actual grasping experiments demonstrate that the proposed self-supervised learning method can increase the rate of successful grasps for the manipulator. Note to Practitioners—Most of the existing grasp planning methods of the manipulator are based on public datasets or simulation data to train model algorithms. Owing to the limited types of objects, the limited amount of data in the public datasets, and the lack of real sensor noise in the simulation data, the robustness of the trained algorithm model is insufficient, and it is difficult to apply to unstructured production environments. To solve the above problems, we propose a 6-DOF capture planning method based on self-supervised learning and introduce a self-supervised learning mechanism to solve the problem of grasp data acquisition in real scenes. The manipulator automatically collects object data from multiple perspectives, performs desktop-level 3D reconstruction, and finally uses the force-closure decision algorithm to automatically label the data in order to realize automatic acquisition and labeling of the grasp data in a real scenario. Preliminary experiments show that this method can obtain high-quality grasp data and can be applied to grasp operations in real multi-target and cluttered environments. However, it has not been tested in actual production environments. This paper focuses on the data acquisition module in the 6-DOF grasp planning framework. In future research, we will design a more efficient grasp planning module to improve the grasp efficiency of the manipulator.

In the original belief function (BF) theory, a precise-valued belief structure has been widely used to represent uncertain information. However, this mentioned belief structure is difficult to effectively measure the specific hesitant situation, especially when decision makers have a set of possible values for the belief assignments of focal elements. In order to model the hesitant nature of the behavior of people to make a decision under uncertainty, we propose a hesitant fuzzy belief structure (HFBS) that is based on the BF theory and the recent hesitant fuzzy set theory. We also present the novel rule of combination of HFBS that is used and evaluated in a wearable human activity recognition (HAR) system coupled with an extreme learning machine. The evaluation of this new HFBS approach is done from two benchmark datasets. We clearly show its effectiveness and its superiority compared to various methods used classically for the wearable HAR.

Because the information from different modalities will complement each other when describing the same contents, multimodal information can be used to obtain better feature representations. Thus, how to represent and fuse the relevant information has become a current research topic. At present, most of the existing feature fusion methods consider the different levels of features representations, but they ignore the significant relevance between the local regions, especially in the high-level semantic representation. In this paper, a general multimodal fusion method based on the co-attention mechanism is proposed, which is similar to the transformer structure. We discuss two main issues: (1) Improving the applicability and generality of the transformer to different modal data; (2) By capturing and transmitting the relevant information between local features before fusion, the proposed method can allow for more robustness. We evaluate our model on the multimodal classification task, and the experiments demonstrate that our model can learn fused featnre representation effectively.

Multisensor fusion strategies have been widely applied in human activity recognition (HAR) in body sensor networks (BSNs). However, the sensory data collected by BSNs systems are often uncertain or even incomplete. Thus, designing a robust and intelligent sensor fusion strategy is necessary for high-quality activity recognition. In this article, Dezert-Smarandache theory (DSmT) is used to develop a novel sensor fusion strategy for HAR in BSNs, which can effectively improve the accuracy of recognition. Specifically, in the training stage, the kernel density estimation (KDE)-based models are first built and then precisely selected for each specific activity according to the proposed discriminative functions. After that, a structure of basic belief assignment (BBA) can be constructed, using the relationship between the test data of unknown class and the selected KDE models of all considered types of activities. In order to deal with the conflict between the obtained BBAs, proportional conflict redistribution-6 (PCR6) is applied to fuse the acquired BBAs. Moreover, the missing data of the involved sensors are addressed as ignorance in the framework of the DSmT without manual interpolation or intervention. Experimental studies on two real-world activity recognition datasets (The OPPORTUNITY dataset; Daily and Sports Activity Dataset (DSAD)) are conducted, and the results shows the superiority of our proposed method over some state-of-the-art approaches proposed in the literature.

The classifier based on rough sets is widely used in pattern recognition. However, in the implementation of rough set-based classifiers, there always exist the problems of uncertainty. Generally, information decision table in Rough Set Theory (RST) always contains many attributes, and the classification performance of each attribute is different. It is necessary to determine which attribute needs to be used according to the specific problem. In RST, such problem is regarded as attribute reduction problems which aims to select proper candidates. Therefore, the uncertainty problem occurs for the classification caused by the choice of attributes. In addition, the voting strategy is usually adopted to determine the category of target concept in the final decision making. However, some classes of targets cannot be determined when multiple categories cannot be easily distinguished (for example, the number of votes of different classes is the same). Thus, the uncertainty occurs for the classification caused by the choice of classes. In this paper, we use the theory of belief functions to solve two above mentioned uncertainties in rough set classification and rough set classifier based on Dezert-Smarandache Theory (DSmT) is proposed. It can be experimentally verified that our proposed approach can deal efficiently with the uncertainty in rough set classifiers.

Multi-source information fusion strategies in target recognition have been widely applied. Generally, each source is defined and modelled over a common frame composed of the hypotheses to discern. However, in practice, the independent sources of evidence can refer to distinct frames of discernment in terms of the hypotheses they consider. Under this condition, the classical combination process cannot be applied directly. Working with distinct frames of discernment for information fusion is a problem often encountered in the development of recognition systems which requires a particular attention. In order to combine such sources, this paper presents a new combination method which splits the process of fusion into two steps: construction of granular structure, calculation of belief mass, followed by the fusion process. Our simulations results show that the proposed method can effectively solve the problem of fusion of sources defined on distinct frames.

In many applications involving epistemic uncertainties usually modeled by belief functions, it is often necessary to approximate general (non-Bayesian) basic belief assignments (BBAs) to subjective probabilities (called Bayesian BBAs). This necessity occurs if one needs to embed the fusion result in a system based on the probabilistic framework and Bayesian inference (e.g. tracking systems), or if one wants to use classical decision theory to make a decision. There exists already several methods (probabilistic transforms) to approximate any general BBA to a Bayesian BBA. From a fusion standpoint, two approaches are usually adopted: 1) one can approximate at first each BBA in subjective probabilities and use Bayes fusion rule to get the final Bayesian BBA, or 2) one can fuse all the BBAs with a fusion rule, typically Dempster-Shafer's, or PCR6 rules (which is very costly in computations), and convert the combined BBA in a subjective probability measure. The former method is the simplest method but it generates a high loss of information included in original BBAs, whereas the latter is intractable for high dimension problems. This paper presents a new method to achieve this task based on hierarchical decomposition (coarsening) of the frame of discernment, which can be seen as an intermediary approach between the two aforementioned methods. After the presentation of this new method, we show through simulations how its performs with respect to other methods.

The study of alternative probabilistic transformation (PT) in DS theory has emerged recently as an interesting topic, especially in decision making applications. These recent studies have mainly focused on investigating various schemes for assigning both the mass of compound focal elements to each singleton in order to obtain Bayesian belief function for real-world decision making problems. In this paper, work by us also takes inspiration from both Bayesian transformation camps, with a novel evolutionary-based probabilistic transformation (EPT) to select the qualified Bayesian belief function with the maximum value of probabilistic information content (PIC) benefiting from the global optimizing capabilities of evolutionary algorithms. Verification of EPT is carried out by testing it on a set of numerical examples on 4D frames. On each problem instance, comparisons are made between the novel method and those existing approaches, which illustrate the superiority of the proposed method in this paper. Moreover, a simple constraint-handling strategy with EPT is proposed to tackle target type tracking (TTT) problem, simulation results of the constrained EPT on TTT problem prove the rationality of this modification.

Aiming at the counterintuitive phenomena of the Dempster–Shafer method in combining the highly conflictive evidences, a combination method of evidences based on the clustering analysis is proposed in this paper. At first, the cause of conflicts is disclosed from the point of view of the internal and external contradiction. And then, a new similarity measure based on it is proposed by comprehensively considering the Pignistic distance and the sequence according to the size of the basic belief assignments over focal elements. This measure is used to calculate the commonality function of evidences to amend the evidence sources; Meanwhile, the Iterative Self-organizing Data Analysis Techniques Algorithm (ISODATA) method based on the new measure is used for clustering according to the clustering characters of the original evidences. The Dempster rule is applied to combining all the evidences in each clustering into an evidential representative, and the reliability is calculated based on the commonality and the occurrence frequency of the evidences in the clustering. At last, Murphy's method is used to combine these evidential representatives of the different clusterings. The experimental results through a series of numeric examples show that the method proposed in this paper is more effective and superior to others.

Real-time navigation and mapping of an autonomous robot is one of the major challenges in intelligent robot systems. In this paper, a novel sensor-based biologically inspired neural network algorithm to real-time collision-free navigation and mapping of an autonomous mobile robot in a completely unknown environment is proposed. A local map composed of square grids is built up through the proposed neural dynamics for robot navigation with restricted incoming sensory information. With equipped sensors, the robot can only sense a limited reading range of surroundings with grid map representation. According to the measured sensory information, an accurate map with grid representation of the robot with local environment is dynamically built for the robot navigation. The real-time robot motion is planned through the varying neural activity landscape, which represents the dynamic environment. The proposed model for autonomous robot navigation and mapping is capable of planning a real-time reasonable trajectory of an autonomous robot. Simulation and comparison studies are presented to demonstrate the effectiveness and efficiency of the proposed methodology that concurrently performs collision-free navigation and mapping of an intelligent robot.

This paper deals with enriched qualitative belief functions for reasoning under uncertainty and for combining information expressed in natural language through linguistic labels. In this work, two possible enrichments (quantitative and/or qualitative) of linguistic labels are considered and operators (addition, multiplication, division, etc) for dealing with them are proposed and explained. We denote them qe-operators, qe standing for “qualitative-enriched” operators. These operators can be seen as a direct extension of the classical qualitative operators (q-operators) proposed recently in the Dezert-Smarandache Theory of plausible and paradoxist reasoning (DSmT). q-operators are also justified in details in this paper. The quantitative enrichment of linguistic label is a numerical supporting degree in [0,∞), while the qualitative enrichment takes its values in a finite ordered set of linguistic values. Quantitative enrichment is less precise than qualitative enrichment, but it is expected more close with what human experts can easily provide when expressing linguistic labels with supporting degrees. Two simple examples are given to show how the fusion of qualitative-enriched belief assignments can be done, and a simulation application is given to show its advantage in rough navigation map building of mobile robot.

We consider in this work evidential sources of information and propose a very general evidence supporting measure of similarity (ESMS) for selecting the most coherent subset of sources to combine among all sources available at each instant. The methodology proposed here coupled with a DSmT-based fusion machine is tested in robotics for the automatic estimation of an unknown simulated environment with obstacles where an autonomous mobile Pioneer II robot with sonar sensors evolves. Our simulation results are based on the fusion of similar and equireliable sensors but same approach can also be used with dissimilar sources as well by using a discounting method taking into account the reliability of each sensor. Our results show clearly the benefit of the selection of the sources as prerequisite for improvement of information fusion