In industrial point cloud analysis, detecting subtle anomalies demands high-resolution spatial data, yet prevailing benchmarks emphasize low-resolution inputs. To address this disparity, we propose a scalable pipeline for generating realistic and subtle 3D anomalies. Employing this pipeline, we developed MiniShift, the inaugural high-resolution 3D anomaly detection dataset, encompassing 2,577 point clouds, each with 500,000 points and anomalies occupying less than 1% of the total. We further introduce Simple3D, an efficient framework integrating Multi-scale Neighborhood Descriptors (MSND) and Local Feature Spatial Aggregation (LFSA) to capture intricate geometric details with minimal computational overhead, achieving real-time inference exceeding 20 fps. Extensive evaluations on MiniShift and established benchmarks demonstrate that Simple3D surpasses state-of-the-art methods in both accuracy and speed, highlighting the pivotal role of high-resolution data and effective feature aggregation in advancing practical 3D anomaly detection.

We propose Anomagic, a zero-shot anomaly generation method that produces semantically coherent anomalies without requiring any exemplar anomalies. By unifying both visual and textual cues through a crossmodal prompt encoding scheme, Anomagic leverages rich contextual information to steer an inpainting‐based generation pipeline. A subsequent contrastive refinement strategy enforces precise alignment between synthesized anomalies and their masks, thereby bolstering downstream anomaly detection accuracy. To facilitate training, we introduce AnomVerse, a collection of 12,987 anomaly–mask–caption triplets assembled from 13 publicly available datasets, where captions are automatically generated by multimodal large language models using structured visual prompts and template‐based textual hints. Extensive experiments demonstrate that Anomagic trained on AnomVerse can synthesize more realistic and varied anomalies than prior methods, yielding superior improvements in downstream anomaly detection. Furthermore, Anomagic can generate anomalies for any normal‐category image using user‐defined prompts, establishing a versatile foundation model for anomaly generation.

Knowledge distillation (KD) has become a fundamental technique for model compression in object detection tasks. The data noise and training randomness may cause the knowledge of the teacher model to be unreliable, referred to as knowledge uncertainty. Existing methods neglect this uncertainty, potentially hindering the student's capacity to capture and understand latent "dark knowledge". In this work, we introduce a novel strategy that explicitly incorporates knowledge uncertainty, named Uncertainty-Driven Knowledge Extraction and Transfer (UET). Given the unknown, high-dimensional nature of the knowledge distribution, we employ Monte Carlo dropout to effectively estimate the teacher's uncertainty. Leveraging information theory, we combine uncertainty with deterministic knowledge, enabling the student to benefit from both precision and diversity. UET is a plug-and-play method that integrates seamlessly with existing distillation techniques. We validate our approach through comprehensive experiments across various distillation strategies, detectors, and backbones. Specifically, UET achieves state-of-the-art results, with a ResNet50-based GFL detector obtaining 44.1% mAP on the COCO dataset-surpassing baseline performance by 3.9%.

Dual-arm robotic manipulation of liquid or biochemical reagents poses critical challenges due to high-dimensional configuration spaces, stringent task-specific end-effector orientation requirements to prevent spillage, frequent inter-arm collisions, and cluttered experimental environments. This paper introduces CLEAR-MP (Clearance Learning-Based Efficient Motion Planning for Dual-Arm Robots under End-Effector Orientation Constraints), a modular framework that integrates multiple innovations: a decoupled learning-driven collision estimation module–comprising aPairwise Link Clearance Networkfor self-collision and aClearance Inference Networkfor environmental obstacles, aLearning-Driven Bidirectional Parallel Search Strategyfor accelerated tree expansion, parallel Cartesian batch sampling for efficient candidate generation, fast inverse-kinematics mapping, andLearning-Guided Batch Shortcut Optimizationto refine trajectories. Together, these components generate smooth, safety-certified paths with substantially reduced planning time and path length. Extensive simulations and real-robot experiments show that CLEAR-MP achieves an average path length of 2.391 m, average planning time of 3.529 s, outperforming state-of-the-art baselines by over 50% in computation and 40% in trajectory quality while maintaining strong generalization without retraining.

The rapid advancement of humanoid robotics has highlighted the challenge of coordinating high-redundancy whole-body motion and posture control under multiple safety constraints. The difficulty lies particularly in dual-arm task planning and the design of high-precision, real-time control strategies. To address these issues, this paper proposes a novel neural-network (NN) based real-time model predictive control (MPC) framework for humanoid robots. The framework innovatively integrates discrete recurrent neural networks (DRNN) with MPC, thereby extending their combined advantages to high-redundancy humanoid motion control. In addition, a primal-dual neural network (PDNN) solver is employed to compute multi-constrained kinematic MPC in a single iteration, avoiding the repeated iterations required in conventional MPC and significantly enhancing real-time performance. The proposed approach is rigorously validated through theoretical derivation and extensive experiments, including both numerical simulations and real-world trials. After theoretical verification in simulation, the NN-based MPC framework is deployed on a self-developed humanoid robotic platform. Experimental results confirm that the NN-based MPC method achieves effective and highly accurate whole-body task planning and real-time control, demonstrating its potential as a reliable solution for advanced humanoid robot control.

In recent years, numerous hyperspectral image (HSI) reconstruction methods have been proposed to enhance the imaging quality of coded aperture snapshot spectral compressive imaging (CASSI) systems. Among these methods, self-supervised Deep Image Prior (DIP)-based approaches have gained attention for their ability to reconstruct three-dimensional (3D) HSIs without the need for external training data. However, DIP methods often suffer from overfitting to high-frequency noise during the optimization process, leading to artifacts and loss of fine details. To address these challenges, we propose a Mutual-Regularized Dual Deep Image Prior (DDIP) framework that employs implicit mutual regularization between two DIP networks. By encouraging mutual constraints, DDIP effectively mitigates high-frequency learning bias and suppresses noise amplification. Additionally, we employ a Half Quadratic Splitting (HQS) optimization strategy to ensure stable and efficient convergence, progressively integrating complementary information from the dual networks. We provide a comprehensive convergence analysis of the DDIP framework and establish theoretical conditions to guide the progressive fusion of the dual networks, ensuring robust and reliable reconstruction. Based on the insights from the convergence analysis, we introduce an Adaptive Deep Image Prior inner-loop strategy that dynamically adjusts the inner-loop updates, ensuring balanced learning of low- and high-frequency components. Moreover, a Residual Spectral-Spatial Feature Attention Network (SSFAN) is designed to enhance spectral-spatial feature extraction, further improving reconstruction accuracy. Extensive experiments on benchmark datasets demonstrate that DDIP achieves competitive HSI reconstruction quality compared to state-of-the-art unsupervised and self-supervised methods.

In industrial bin-picking, robotic systems must estimate the poses of multiple object instances, where accurate pose estimation is essential for reliable downstream manipulation and grasping. Most existing multi-instance registration methods primarily establish point correspondences based on local features to alleviate the challenges posed by occlusion and clutter. However, local features are easily disturbed by neighboring instances and lack global context, leading to unreliable correspondences and degraded registration accuracy. In addition, the absence of rotational invariance further reduces correspondence accuracy in scenes with stacked instances and highly varying object orientations. To address these challenges, we present a one-stage multi-instance point cloud registration framework for stacked-object scenes. Our framework incorporates a rotation-invariant operator to enhance the robustness of feature representations under arbitrary orientations. Then, we propose a Center-Aware Res-Masked Transformer module, which incorporates an object center embedding to enrich global instance-level context and a center-aware residual mask prediction module to balance weight distribution across objects of varying sizes during training. Extensive experiments on the challenging ROBI dataset demonstrate that our method outperforms the competitive baseline MIRETR by more than 10% in mean precision, highlighting its effectiveness in complex bin-picking scenes. Furthermore, evaluations on the unstacked Scan2CAD dataset confirm the generalizability of the proposed framework across different application scenarios.

Effective transmission line (TL) inspection in complex corridor environments is essential for ensuring reliable power delivery. This work presents a 3D-based perception method for this task. The proposed method is designed by considering two key characteristics of TL inspection. First, the point cloud data are sparse and class distributions are highly imbalanced, which weakens the signals from thin conductors and tower components. To address this issue, we model long-range spatial relations along the corridor to mitigate data sparsity and imbalance. Second, strong structural correlations exist between conductors and towers, which can be leveraged to improve perception performance. To exploit this property, we construct a unified 3-D representation that jointly models towers, conductors, and vegetation, while fusing Cartesian and polar geometries through geometry-aware alignment. Experiments on real-world corridor datasets demonstrate that the proposed method, termed multigeometric and long-distance TL perception Network (MGLD-TLNet), consistently improves stability and accuracy under conditions of sparsity, occlusion, and complex environmental interactions.

In this article, an admittance control strategy for aerial manipulators is proposed to achieve contact force tracking in unknown dynamic environments. First, considering the impact of variations in unknown environments on force tracking performance, an adaptive variable stiffness feature is incorporated into an advanced admittance model. The stiffness coefficient is dynamically adjusted using position and force feedback to generate the desired reference trajectory. Second, to address the issue of reference trajectory tracking under disturbances, a pose controller composed of a disturbance observer and barrier Lyapunov function is utilized to achieve stable tracking performance. In the absence of prior knowledge of disturbances, the state variables converge to a constrained range within a finite time, without introducing excessively high control gains. Finally, the stability of the proposed strategy is rigorously analyzed via Lyapunov tools. Both simulations and real-world experimental investigations are conducted to demonstrate the feasibility of the control strategy, highlighting its robust performance in maintaining a stable contact force during interaction with unknown dynamic environments.

Considering situations such as sensor failures and communication losses, limited data samples are a common challenge in actual industrial processes, making the traditional integrated architecture of fault detection (FD) and fault-tolerant control (FTC) based on subspace identification difficult to be applicable. Regarding this problem, this article proposes a nuclear norm-based subspace identification method for FD and FTC. This method leverages key structural matrix properties in the input and output data model, alleviating reliance on data samples. The parameter matrices required to construct the fault detector and fault-tolerant controller can be directly identified within the nuclear norm optimization framework, enabling the design of an integrated FD and FTC architecture. Two case studies demonstrate that the developed method enhances detection and control performance compared with traditional subspace identification methods, particularly in the case of limited data samples.

Ice thickness estimation on overhead transmission lines (OHTL) is essential for mitigating icing-induced mechanical failures and ensuring safe grid operation. To address the challenges of detecting radial ice thickness in complex power line corridors, particularly geometric fragmentation of slender conductors and semantic ambiguity near occluded boundaries, this work proposes a structure-aware framework (SAF) based on 3-D point cloud segmentation and geometry-guided modeling. SAF introduces a structure-aware segmentation network, which integrates a cross-level spatial encoding module to preserve geometric continuity and a partition-aware loss to improve boundary localization under vegetation or tower occlusion. Building on accurate segmentation, a geometry-guided module performs centerline fitting and cross-sectional reconstruction to infer slice-level ice thickness. To support evaluation, a large-scale uncrewed aerial vehicle (UAV)-based point cloud dataset covering 32 OHTL is constructed, including six lines with ground-truth ice labels. Experimental results demonstrate that SAF achieves robust and accurate ice estimation across varied voltage levels and terrains, supporting its practical application in intelligent transmission line inspection and icing risk prevention.

High-precision robotic assembly remains a challenge in intelligent manufacturing. Existing vision-based approaches often learn predefined trajectories from large image corpora yet underutilize task-relevant visual cues during execution, limiting deployment. We present a diffusion-based end-to-end assembly policy that performs redundancy-aware cross-scale fusion and provides stage-dependent conditioning for action generation. Specifically, we introduce a bidirectional-attention complementary visual fusion module that aligns cross-scale observations and produces scale-consistent features with reduced redundancy. We then introduce a dual-stream adaptive modulation module that enables time-state correlated routing and progressively shifts emphasis from scene-level stabilization to contact-level refinement within the denoising process. Coupling complementary visual fusion with adaptive-modulation-conditioned denoising, we develop a staged modulation diffusion policy for end-to-end action generation, producing temporally coherent and geometrically accurate actions. Experiments on five real-world tasks demonstrate consistent improvements over representative baselines, with average gains of 41.10% in overall task success rate and 63.16% in precise assembly rate on three representative assembly benchmarks.

Signal-to-image transformation has been widely used in mechanical fault diagnosis to provide unified visual representations for downstream diagnostic models. However, the resulting diagnostic images can exhibit substantial quality variations caused by measurement noise, sensor degradation, and partial signal loss. Most existing methods either discard low-quality samples or implicitly assume equal reliability across samples, leading to information loss or degraded robustness. This paper proposes an image-quality-guided consistency-alignment network that explicitly estimates sample quality and leverages degraded data during training. First, vibration signals are decomposed into sub-bands, and energy-ratio criteria are used to select informative components for reconstruction. The reconstructed signals are subsequently fused into fixed-layout two-dimensional images via a sector-allocation strategy. Next, a diagnosis-aware quality assessment module assigns a quality score to each image to guide training. Low-quality samples are further regularized via cross-quality feature alignment using a quality-weighted supervised pairwise loss, encouraging them to align with high-quality counterparts from the same class. Finally, a Swin Transformer backbone performs classification. Experiments on the proprietary RGFD dataset and the public SEUB dataset demonstrate consistent gains under controlled mixed-quality settings, with accuracies of up to 99.4% and 98.7%, respectively, while additional evaluations under reproducible degradations further confirm the robustness of the proposed quality-aware learning mechanism.

It is very challenging to precisely control a redundant manipulator with an unknown model during the end-effector tracking task. Adata-driven approach offers a promising solution for manipulator control under such conditions. In this article, two data-driven discrete neurodynamics (DDDN) schemes are proposed for redundant manipulator tracking control. First, utilizing discrete neurodynamics (DN) principles, the DDDN-1 scheme with an adaptive Jacobian matrix is developed. Subsequently, the DDDN-2 scheme is further presented, which eliminates the need for the Jacobian matrix inversion operation. Detailed theoretical analyses verify the effectiveness of DDDN-1 and DDDN-2 schemes. Additionally, detailed comparisons with existing schemes have been provided. Finally, simulative and physical experiments conducted using the UR5 manipulator validate the theoretical analyses, demonstrating the effectiveness and superiority of DDDN-1 and DDDN-2 schemes.

Lung cancer is a leading cause of cancer-related deaths globally. PET-CT is crucial for imaging lung tumors, providing essential metabolic and anatomical information, while it faces challenges such as poor image quality, motion artifacts, and complex tumor morphology. Deep learning-based models are expected to address these problems, however, existing small-scale and private datasets limit significant performance improvements for these methods. Hence, we introduce a large-scale PET-CT lung tumor segmentation dataset, termed PCLT20K, which comprises 21, 930 pairs of PET-CT images from 605 patients. Furthermore, we propose a cross-modal interactive perception network with Mamba (CIPA) for lung tumor segmentation in PET-CT images. Specifically, we design a channel-wise rectification module (CRM) that implements a channel state space block across multi-modal features to learn correlated representations and helps filter out modality-specific noise. A dynamic cross-modality interaction module (DCIM) is designed to effectively integrate position and context information, which employs PET images to learn regional position information and serves as a bridge to assist in modeling the relationships between local features of CT images. Extensive experiments on a comprehensive benchmark demonstrate the effectiveness of our CIPA compared to the current state-of-the-art segmentation methods. We hope our research can provide more exploration opportunities for medical image segmentation. The dataset and code are available at https://github.com/mj129/CIPA.

Aerial-terrestrial amphibious robots excel in search and rescue tasks in unstructured terrains but face challenges in autonomous navigation indoors. Traditional full-mapping methods can degrade global path planning performance, especially when semi-static obstacles shift, leading to suboptimal paths. We propose a method for constructing building structure grid maps that are unaffected by semistatic obstacles. Our approach includes a building structure recognition algorithm based on an octree structure to differentiate between occupied and free grid cells. Experimental results demonstrate that coverage path planning on building structure grid maps produces superior global paths compared to traditional grid maps, offering a more streamlined and robust solution for autonomous navigation of aerial-terrestrial amphibious robots in indoor environments.

The paper proposes a novel Economic Model Predictive Control (EMPC) scheme for Autonomous Surface Vehicles (ASVs) to simultaneously address path following accuracy and energy constraints under environmental disturbances. By formulating lateral deviations as energy-equivalent penalties in the cost function, our method enables explicit trade-offs between tracking precision and energy consumption. Furthermore, a motion-dependent decomposition technique is proposed to estimate terminal energy costs based on vehicle dynamics. Compared with the existing EMPC method, simulations with real-world ocean disturbance data demonstrate the controller’s energy consumption with a 0.06% energy increase while reducing cross-track errors by up to 18.61%. Field experiments conducted on an ASV equipped with an Intel N100 CPU in natural lake environments validate practical feasibility, achieving 0.22 m average cross-track error at nearly 1 m/s and 10 Hz control frequency. The proposed scheme provides a computationally tractable solution for ASVs operating under resource constraints.

In contact-rich tasks such as polishing and drilling, inevitable physical interactions often lead to task deviations due to interference, typically resulting in excessive contact forces and eventual task failure. To tackle these challenges, we propose an innovative human-guided visual-impedance control framework. Specifically, we first introduce an interaction model within image feature space, which models the dynamics of human-robot-environment interactions. Subsequently, human operation skills are characterized through human-guided wrenches, and acts on visual features through a projection matrix, thus integrating human-guided wrenches with visual-impedance interaction dynamics. Finally, leveraging this framework, we develop a novel variable-stiffness visual-impedance control strategy. The impedance parameters are optimized online via Quadratic Program, ensuring that the end-tool contact force converges to desired value while adhering to safety constraints. The validity of the proposed framework was established through polish experiments.

Repetitive motion control of redundant manipulators typically requires precise kinematic models to construct Jacobian matrices. However, model-based approaches are inherently limited when manipulator parameters are unavailable or only partially known. This paper introduces a novel data-driven discrete zeroing neurodynamics (DDZN) model for repetitive motion control. Specifically, a Jacobian matrix estimation method based on data-driven technology is proposed, which eliminates the need for prior models by leveraging historical input-output information. By integrating the Jacobian matrix estimation with a discrete zeroing neurodynamics (DZN) model, the approach enables simultaneous trajectory tracking and repeatable configuration recovery without relying on structural parameters. Theoretical analysis verifies the performance of DDZN model under noise environment. Furthermore, abundant experiment results validate its reliability and superior performance compared with various models.

Precise motion tracking control with unknown structural knowledge and noise disturbance for redundant robots remains a critical and unresolved challenge. This article proposes a novel data-driven fuzzy discrete recurrent neural network (D2-FDRNN) model to address two fundamental limitations of existing models: dependency on known kinematic knowledge and fixed sampling schemes. First, a Jacobian pseudo-inverse estimator is developed to reconstruct the manipulator’s necessary kinematic knowledge using input and output data, eliminating the need for explicit Jacobian inversion. Second, a fuzzy logic-based adaptive sampling strategy dynamically adjusts the step size to balance computational efficiency and tracking precision. In addition, a Kalman filter algorithm is applied to reduce the impact of noise. Rigorous proofs confirm the model’s exponential convergence and noise immunity. To validate the proposed D2-FDRNN model, simulations and physical experiments are carried out. The source code is available at https://github.com/YingluckZ/DD-FDRNN.git.

Since its inception, impedance control has emerged as a fundamental framework for robotic interaction control. Recent advancements in geometric impedance control have demonstrated certain advantages over traditional Cartesian impedance control. However, existing geometric impedance control approaches generally lack force regulation capabilities or rigorous stability guarantees. In this paper, we propose a safety-aware geometric force-impedance controller that addresses these limitations. By incorporating an energy tank mechanism, the proposed approach enables precise force tracking while preserving full compatibility with the impedance behavior. Furthermore, an energy injection and freezing mechanism is introduced, allowing dynamic regulation of energy exchange between the tank and the robotic system. Notably, the proposed method eliminates the need for an offline estimation of the initial energy stored in the tank, facilitating real-time adjustments of force controller parameters. To validate the effectiveness of the proposed framework, we conduct extensive polishing experiments on a real robotic platform. The results demonstrate the capability of the proposed controller to achieve stable and precise force regulation.

The vehicle trajectory prediction is crucial for autonomous driving. The vast majority of existing trajectory prediction schemes depend on high definition (HD) maps. However, the HD knowledge is not invariably valid under many actual traffic scenarios. When map information is unreliable, vehicle trajectory prediction is a fundamental challenge that must be overcome in autonomous driving areas. Over the vehicle–road cooperation, this work has developed a spatiotemporal-attentional-mechanism-based prediction model (STAM-P) for vehicle trajectories under map-free scenarios to precisely forecast future trajectories in the case of unreliable map information. First, a temporal transformer encoder was used to capture and encode the state information of the vehicle at different time steps for extracting the vehicle temporal features of the trajectories. Second, a spatial encoder layer consisting of the static map convolutional layer and the dynamic spatial transformer in the spatial feature extraction layer was designed to capture and encode the interactions between vehicles to obtain the vehicle spatial features of the trajectories. Finally, the obtained temporal–spatial features were inputted into a multimodal decoding layer to decode and complete the trajectory prediction. Experimental results of the Argoverse dataset revealed that the proposed model outperformed other existing map-free prediction schemes and reached the level of other map-based trajectory prediction models in certain performance metrics.

Aiming at the area search task of a multi-robot system in an unknown complex obstacle environment, we propose a cooperative area search algorithm based on a dual improved bio-inspired neural network (DIBNN). First, we improve the BNN model to reduce the interference of the complex obstacle environment on robot decision making. Each robot generally chooses the neuron with the largest sum of surrounding activity values among adjacent neurons as its next movement position. Then, we propose a collaborative search mechanism. When a robot falls into a local deadlock state in the complex obstacle environment, the mechanism will guide the robot to quickly find unsearched areas. Finally, we conduct multi-robot area search simulation experiments under different obstacle environments and compare them with three baseline algorithms in this field. The simulation results verify that the proposed algorithm can efficiently guide the multi-robot to complete the area search task in the complex obstacle environment.Note to Practitioners—The motivation of this article arises from the need to develop fast and effective area search algorithms for practical applications such as UAV swarm reconnaissance and multiple mobile robots area search and rescue. The algorithms based on BNN has been widely used in search tasks under unknown environments due to its good scalability and efficiency. However, the efficiency of area search in complex obstacle environments cannot be guaranteed. In order to achieve efficient area search in unknown complex obstacle environments, the DIBNN algorithm is proposed. It utilizes a cooperative search mechanism and achieves better performance. DIBNN can also be applied to multi-robot systems in different scenarios, demonstrating strong scalability.

This paper presents an intelligent control scheme for multirotors, where accurate trajectory tracking, strong robustness and reliable generalization are guaranteed by the dual-feedback sliding-mode (DFSM) guided deep reinforcement learning (RL). Different from current solutions, the proposed method explores optimal learning strategy on the sliding surface according to the DFSM demonstrations, where the elegantly designed parallel evaluation takes full advantage of model knowledge and learning exploration. Specifically, the intelligent tracking control is achieved in a two-step design. First, the DFSM algorithm is designed for multirotors, where the linear and nonlinear feedback terms work cooperatively. Second, the DFSM-guided deep RL is put forward to achieve intelligent switching on the sliding surface, where position and velocity errors are both considered to generate accurate switching decisions. In the framework, explorations and the DFSM demonstrations are evaluated in parallel, where only the explorations that are better than the DFSM baseline, are kept for policy improvement. In this way, the DFSM algorithm keeps pushing the RL policy to explore better strategy, where the unavoidable bad experiences arisen from exploration are identified accurately. Practical comparative experimental results are included to verify the effectiveness of the proposed strategy.Note to Practitioners—This paper is motivated by the practical problem of controlling multirotor system in uncertain environments. Up until now, most existing approaches are proposed without taking full advantage of model knowledge and deep learning techniques simultaneously, which lacks of reliability in practical application. To deal with the problem, a new dual-feedback sliding-mode (DFSM) guided deep reinforcement learning (RL) strategy is proposed, where the dual feedback and guided RL are designed to achieve satisfactory tracking control and simultaneously handle uncertainties. Specifically, by introducing double-check framework, the RL strategy explores optimal switching policy on the sliding surface according to the DFSM demonstrations, guaranteeing strong robustness and reliable generalization of the obtained RL policy in uncertain environments. The key feature of the framework is that the DFSM-driven training guarantees practice-oriented tracking control in a DFSM-RL cooperative manner. Comparative experiments are implemented to verify the tracking performance of the proposed intelligent control strategy.

Path Integral Strategy Improvement (PI2)-based impedance control is a superior scheme for preventing damage to the physical structure of the fruit during the harvesting process. However, it is highly sensitive to disturbances and has limited generalization ability during the parameter learning process, making it difficult to apply the correct gripping force to fruits with uncertain stiffness. To solve this problem, this paper proposes a variable impedance force control method that integrates PI2 with Dynamic Movement Primitives (DMPs), supplemented by a force error correction loop. Firstly, an adaptive impedance parameter matching mechanism based on gain schedules is designed to facilitate dynamic estimation of impedance parameters and enable precise force control. To further accelerate impedance parameter matching in unknown environments, the PI2 algorithm is introduced to optimize gain schedules, and DMPs are integrated to suppress disturbances, thereby improving the generalization ability of the impedance model’s parameter learning. In addition, an additional force error control loop has been designed to minimize the deviation between the desired and actual gripping force. Finally, the effectiveness of the proposed method is verified through simulation and experiment in fruit teleoperation picking robot.

This article aims to investigate the research achievements related to aerial robots with physical interaction. Various morphologies of aerial physical interaction (APhI) robot prototypes with fixed wing, flapping wing, single main rotor, conventional underactuated multirotor, fully actuated multirotor, even deformed multirotor, and multiple platforms are reviewed for different APhI tasks associated with momentary, loose, and strong interaction coupling. This review also covers APhI robot rigid dynamics and robot-environment coupled interaction dynamics modeling methods, interaction wrench measurement/estimation, decoupled and coupled control, active aerial interaction control, and task-constrained planning approaches. Finally, future development directions and prospects are initially anticipated for aerial robots with physical interaction.Note to Practitioners—Aerial physical interaction (APhI) has been a hot topic in the field of aerial robots in recent years, which is a reflection of the advanced capabilities of aerial robots. However, APhI robots face challenges such as difficulty in flight stability and weak adaptability to dynamic environments while exerting active influence on environments. Under this background, this review aims to offer a reference for researchers and practitioners engaged in the related field from the aspects of system design, modeling, control, and task-constrained planning, which hopes to help them apply APhI robots to polar scientific expeditions, complex environment sampling, infrastructure inspection and maintenance, and other application areas. Further, this review also highlights the design idea of rigid-soft integrated APhI robots from the perspective of design-mechanism-performance to enhance interaction stability and safety.

Highly accurate 3D object detection is critical for autonomous driving and robotic sensing system. However, some objects with few foreground points significantly affect the accuracy of 3D object detection. As the network depth increases, the low-level features of these objects are gradually lost, especially for the hard object. Due to this issue, current LiDAR-only based and multimodal methods often misclassify background as foreground. Therefore, how to leverage the low-level feature that contain information about these objects in the high layer of the network becomes the key to optimizing the issue. In this paper, we propose LDFCDet, a framework boosting 3D object detectors with low-high level feature crosses using Laplace distribution(LD). In our proposed method, we design a low-high level feature crosses module(LHFCM) to embed low-level feature into high-level feature in the deeper layer of the network, and use Laplace distribution to obtain a new low-high level feature that includes information about these objects with few foreground points. In addition, we propose a res-gated feature aggregation module(RGFAM) to fuse the mutli-scale features. Our approach is well-suited for both LiDAR-based and multimodal methods.We evaluate the LDFCDet on the widely used KITTI dataset, and our method outperforms almost current 3D object detection methods on the challenging KITTI test set. Moreover, we conducted comparative experiments on the ONCE dataset, and the results further demonstrate the effectiveness and superiority of our method.

In the electric power scene, fasteners play a pivotal role in securing and connecting electrical equipment, with small fastener detection (SFD) being crucial for ensuring operational stability. Despite the replacement of manual inspection methods by non-destructive techniques employing deep learning, these approaches often demand substantial computational resources and involve numerous parameters. While knowledge distillation (KD) can be a viable solution, existing KD methods may often fail to achieve satisfactory performance when dealing with small object presentation and little inter-class variability in SFD tasks. To alleviate this, we propose a Focal Multi-scale Shape-feature Distillation Network (FMSD) to achieve efficient and precise fastener detection in electric power scenarios. Specifically, we propose a novel Multi-Scale Shape-Aware Feature Aggregation module (MSFA) to augment the network's perception of object shape and scale during the KD process. Additionally, we propose a Contour-Guided Distillation (CGD) module to optimize the transfer of the extracted shape-sensitive knowledge between the teacher and student models. Through a series of experiments compared with existing state-of-the-art (SOTA) methods, our method demonstrates superior performance over existing SOTA techniques, both efficiently and effectively. Furthermore, validation on publicly available power scene datasets confirms the generalizability and adaptability of our proposed FMSD across various settings.

The classification of tree species for overhead transmission lines (OHTL) is of great significance, facilitating the the timely removal of safety hazards posed by trees on power lines. Addressing the challenges in classifying OHTL line tree species, including subtle differences in target shape appearance, densely distributed targets, and limited representation in single-modal data, this article proposes a tree species classification network, VSLNet, based on multimodal data fusion. VSLNet constructs three asymmetric branches, which automatically select more discriminative features among spectra during spectral information processing, and jointly guide the extracted visible light information, ensuring global and local consistency for accurate multispectral classification. Furthermore, in LiDAR processing, the segmentation of individual trees contributes data such as tree height and crown diameter, and seamlessly integrates GPS data with multispectral classification results. Experimental results demonstrate that VSLNet is a feasible and reliable solution for tree classification, with potential applicability to other multimodal tasks.

Accurate semantic segmentation is crucial in distribution network line monitoring to ensure the system’s reliability and security. Due to the complexity of the environment and the diversity of devices, unimodal images (such as RGB images) struggle to provide enough information for effective segmentation. To address these challenges, the complementary nature of RGB and thermal infrared (TIR) images is leveraged to significantly enhance segmentation performance. Therefore, an innovative cross-level multimodal fusion network (CLMFNet) is proposed to improve the accuracy and robustness of semantic segmentation by integrating RGB and TIR data. A dual-branch architecture is employed in CLMFNet to extract features from both RGB and TIR images, which are then effectively integrated through a multimodal fusion strategy. Additionally, a cross-layer guidance mechanism is introduced to facilitate the complementation and optimization of features across different levels. CLMFNet was validated on a custom RGB-T dataset, and experimental results showed that it outperformed state-of-the-art methods in key metrics such as mean accuracy (mAcc) and mean intersection over union (mIoU), demonstrating its effectiveness in performing semantic segmentation in complex power distribution scenarios.

This article proposes a novel hierarchical motion planning strategy for multirotors, where the virtual goal (VG) oriented deep reinforcement learning (RL) and motion optimization are designed cooperatively to achieve efficient, flexible and smooth navigation in unknown environments. Specifically, the intelligent hierarchical motion planning is achieved in a three-step design. First, the dynamic VG generation algorithm is proposed considering the perception range of onboard sensors and current velocity, which transforms the global navigation into a real-time point-to-VG planning, thereby guaranteeing efficient computation even in resource-limited multirotors. Second, instead of generating motion actions, the upper-layer deep RL is designed to make spatial-temporal decisions of VG online, which outputs time allocation and spatial distribution commands according to current observation. Third, based on upper-layer's decisions, local optimization and control are implemented accordingly. Different from existing solutions, high-performance planning is guaranteed by the online VG oriented intelligent decision making, where the data-driven learning and model-driven optimization are integrated to navigate the multirotors. Comparative experiments are carried out in both physical simulation and indoor environments, which demonstrate the satisfactory performance of the proposed motion planning strategy in terms of feasibility, efficiency, navigation smoothness, and flexibility.

The power industry relies on efficient inspection systems to ensure stability and safety. While deep learning has advanced automated inspection, its reliance on custom modules for specific tasks can impact efficiency. Knowledge distillation (KD) offers a balanced solution, but the complex textures and structures of power equipment challenge conventional KD methods, which often fail to capture essential local semantic and topological relationships. To address this, we proposeTopNet, a novel topology-preserved KD framework for power scene detection tasks. Specifically, we model the teacher’s knowledge as a graph, where nodes encode local fine-grained features and edges capture global topological relationships. Based on this, we introduce node feature distillation and edge feature distillation to transfer local–global structural knowledge, which can enhance the student’s ability to perceive objects. Furthermore, we also introduce aggregated feature distillation to incorporate and transfer contextual semantic knowledge. Comprehensive experiments are conducted on two different benchmark datasets to demonstrate that TopNet achieves state-of-the-art detection performance with high efficiency, offering a robust solution for automated power equipment inspection.

Real-world transmission line inspections (RTLIs) ensure power stability and safety. Deep learning (DL) models have become prevalent approaches for performing RTLI tasks. However, the high computational demands and substantial parameter requirements of DL models limit their real-world applicability. This article introduces a novel approach, a multiscale uncertainty-aware knowledge-based network, which is designed to balance the accuracy and efficiency in RTLI tasks. Specifically, we propose an uncertainty-aware knowledge distillation method that incorporates pixel-level uncertainty into the knowledge transfer process, mitigating the impact of noisy knowledge derived from extra background information contained in ground truths. In addition, our method integrates a multiscale relationship distillation technique, thus enhancing the transfer of multiscale information between the teacher and student models. Consequently, RTLI tasks can be efficiently accomplished using the well-learned lightweight student model. Comprehensive experiments conducted on a real-world dataset collected via uncrewedaerial vehicles demonstrate the efficacy of our proposed approach in terms of achieving high detection accuracy with reduced computational costs.

As the most fundamental supporting infrastructure of the distribution network, power towers require regular checks of their tilting status to ensure the system’s smooth operation. To overcome distinguishing objects in large-scale scenes using solely images or point clouds poses significant difficulties, we propose a multimodal fusion semantic segmentation network (MFSS) for power tower semantic segmentation and inclination detection. First, effective near-ground filtering and fixed-area slicing algorithms are proposed to address the issues of sample imbalance and insufficient data. Second, MFSS integrates RGB information and point cloud features to enhance the descriptive ability of the tower. Finally, a novel inclination detection method for distribution towers is proposed, estimating tower tilt from the axis between top and bottom centroids to improve accuracy and stability. Experimental results on our constructed dataset show that the proposed method outperforms existing algorithms, achieving 77.3% IoU and 96.69% per-class accuracy in tower segmentation. The mean angle deviation for tilt detection is 0.78$^{\circ }$, with a state judgment false rate of just 2.4%.

Compressive spectral imaging has garnered significant attention for its ability to effectively enhance the captured spatial and spectral information. Predominant methods, based on compressive sensing, typically formulate the imaging task as a constrained optimization problem and rely on hand-crafted priors to model the sparsity of spectral images. However, these approaches often suffer from suboptimal performance due to the inherent difficulty of identifying an appropriate transform space where spectral images exhibit sparsity. To overcome this limitation, we propose a novel convolutional sparse coding-inspired untrained network prior for fast and adaptive identification of the sparse transform domain and compressible signal. Specifically, a Lightweight Convolutional Thresholding sparse Coding (LCTC) network is designed as the sparse transform domain, with its inputs interpreted as sparse coefficients. Crucially, both the transform domain and its coefficients are solved in a self-supervised learning manner. Furthermore, we demonstrate that LCTC prior can be seamlessly incorporated into the iterative optimization algorithm as a Plug-and-Play (PnP) regularization. Both the LCTC and PnP-LCTC exhibit superior performance compared to previous methods. Experiments under various scenarios validate the effectiveness and efficiency of our approach.

Snapshot Spectral Imaging (SSI) techniques, with the ability to capture both spectral and spatial information in a single exposure, have been found useful in a wide range of applications. SSI systems generally operate within the 'encoding-decoding' framework, leveraging the synergism of optical hardware and reconstruction algorithms. Typically, reconstructing desired spectral images from SSI measurements is an ill-posed and challenging problem. Existing studies utilize either model-based or deep learning-based methods, but both have their drawbacks. Model-based algorithms suffer from high computational costs, while supervised learning-based methods rely on large paired training data. In this paper, we propose a novel Unsupervised range-Nullspace learning (UnNull) prior for spectral image reconstruction. UnNull explicitly models the data via subspace decomposition, offering enhanced interpretability and generalization ability. Specifically, UnNull considers that the spectral images can be decomposed into the range and null subspaces. The features projected onto the range subspace are mainly low-frequency information, while features in the nullspace represent high-frequency information. Comprehensive multispectral demosaicing and reconstruction experiments demonstrate the superior performance of our proposed algorithm.

Semantic foggy scene understanding (SFSU) emerges a challenging task under out-of-domain distribution (OD) due to uncertain cognition caused by degraded visibility. With the strong assumption of data centralization, unsupervised domain adaptation (UDA) reduces vulnerability under OD scenario. Whereas, enlarged domain gap and growing privacy concern heavily challenge conventional UDA. Motivated by gap decomposition and data decentralization, we establish a decentralized domain adaptation (DDA) framework calledTranslate thEnAdapt (abbr.TEA) for privacy preservation. Our highlights lie in. (1) Regarding federated hallucination translation, aDisentanglement andContrastive-learning basedGenerativeAdversarialNetwork (abbr.DisCoGAN) is proposed to impose contrastive prior and disentangle latent space in cycle-consistent translation. To yield domain hallucination, client minimizes cross-entropy of local classifier but maximizes entropy of global model to train translator. (2) Regarding source-free regularization adaptation, aPrototypical-knowledge basedRegularizationAdaptation (abbr.ProRA) is presented to align joint distribution in output space. Soft adversarial learning relaxes binary label to rectify inter-domain discrepancy and inner-domain divergence. Structure clustering and entropy minimization drive intra-class features closer and inter-class features apart. Extensive experiments exhibit efficacy of our TEA which achieves 55.26% or 46.25% mIoU in adaptation from GTA5 to Foggy Cityscapes or Foggy Zurich, outperforming other DDA methods for SFSU.

Coded aperture snapshot spectral imaging (CASSI) captures 3D hyperspectral images (HSIs) in a single shot by encoding incident light into 2D measurements. However, recovering the original hyperspectral data from these measurements is a severely ill-posed inverse problem due to significant information loss during compression. Recent deep learning methods, especially deep unfolding networks, have demonstrated promising reconstruction results by embedding learnable priors into iterative optimization frameworks. However, most existing approaches use a single network to jointly estimate spatial and spectral priors, limiting their ability to handle the distinct properties of HSIs. To overcome this limitation, we propose the Spatial-Spectral Prior Decoupling Model (SSPD), which reformulates HSI reconstruction as a prior absorption problem, enabling independent modeling of spatial and spectral priors with specialized network architectures. To achieve this, we design two attention mechanisms tailored for hyperspectral data: one for capturing spatial correlations and another for preserving spectral signatures. Additionally, we develop a hybrid loss function that combines convergence constraints and cross-prior interactions, ensuring accurate prior fusion and stable reconstruction. Experiments on synthetic and real-world datasets confirm that SSPD outperforms existing methods in spectral snapshot compressive imaging.

Registration of multiview point clouds obtained from 3D scanners is a common method for 3D reconstruction. However, most existing registration methods are designed to handle point clouds with known overlap relationships that are ensured by external equipment (e.g., manipulators, turntables) or acquisition sequences, which limits the application range and increases the acquisition cost. To overcome these limitations, an unknown overlap registration (UOR) method for multiview point clouds is proposed, which can estimate overlap confidence, construct a connected graph, and remove outlier point clouds automatically. First, the overlap confidence between two point clouds is estimated by calculating the average nearest neighbor feature distance within the predicted overlap region. We then construct a minimal spanning tree based on the confidence levels and search for the central node to serve as the world coordinate. Finally, the Lie algebra-based SE(3)-sensitive perturbation scheme is introduced to solve the fine transformations, in which a robust weighting function is designed to weight point correspondences. Our method can find reliable connections among point clouds, and the proposed graph can be combined with different pairwise registration methods. The experimental results on both indoor and industrial datasets demonstrate the accuracy and effectiveness of our method.

Although multisource sensor (MS) signal-based mechanical fault diagnosis (MFD) can significantly improve the diagnostic performance, the existing methods often lack sufficient adaptability and generalization when retraining on single-sensor signals or inferring from partial sensor signals. Thus, a general two-stage signal representation contrastive learning fault diagnosis framework (T-SCF) is proposed to adapt the trained model to varying numbers of sensor signals. This framework enhances model robustness and data fusion by comparing sensor signal views, offering a new approach for information fusion, fault detection, and classification in MFD. In the first stage, an adaptive contrastive algorithm is proposed to generate contrastive samples (C-Ss) and contrastive labels (C-Ls) for MS signals. Then, a supervised contrastive loss (SCL) is designed to minimize the similarity between different fault MS signals while maximizing the similarity between identical ones. By designing a parallel encoder architecture, SCL enables it to merge contrasting the features of different sensor signals during training. This strategy preserves the time-domain dimension properties of different sensors during the training of the second-stage classifier, thereby improving the adaptability of the model to different sensor signals without affecting the global information. The effectiveness of the method was verified from multiple different evaluation dimensions using two public datasets and one self-built dataset.

Wind turbines have emerged as a prominent and environmentally friendly energy generation solution. However, with the widespread use of new materials, ensuring the reliability of these devices has become as a critical issue. Developing efficient and cost-effective monitoring methods for the wind turbine's blades (WTBs), the most expensive components of wind turbine, has become a focal point of research. In this article, we present a novel monitoring system for WTBs that employs a deep convolutional neural network approach based on the medical auscultatory method. The system is designed to balance economic efficiency and engineering reliability. First, we proposed a lightweight WTBs monitoring framework based on edge computing that leverages the signals from the programmable logic controller output of wind turbine to enable efficient collection of relevant aerodynamic audio signals while filtering out irrelevant data. Second, we present a set of audio enhancement algorithms that employ multiscale feature extraction, self-adaptive mask targeting, and deep neural networks to reduce noise in the audio signals generated by WTBs. Third, we introduce a new approach for compressing deep convolution neural networks that makes them suitable for resource-constrained edge computing devices and efficiently utilizes audio-generated spectrograms to diagnose faults in WTBs.

This article introduces CIMAP, a high-performance motion planning algorithm for robotic manipulators in complex environments, based on the clearance inference network (CIN). CIMAP incorporates a batch collision estimation module powered by CIN, which efficiently predicts collisions by dividing the manipulator’s workspace into voxels and estimating clearances between the manipulator and surrounding obstacles. The algorithm also features a batch adaptive bidirectional expansion mechanism, enabling the simultaneous extension of multiple nodes within joint space. Leveraging CIN for batch collision estimation, CIMAP accelerates the discovery of feasible paths. Additionally, CIMAP includes a phased path optimization mechanism that identifies local shortcuts through CIN, improving path efficiency. A geometric collision checker ensures safety, performing necessary repairs when required. To assess CIMAP’s effectiveness in continuous motion planning, we compared its performance against four existing algorithms (CN-RRT, B-RRT, GB-RRT*, and NPB-RRT*-DC) across various obstacle scenarios. Experimental results demonstrate that CIMAP achieves an average motion planning time of under 0.7 s, improving planning efficiency by at least 89% compared to the baseline algorithms, while maintaining shorter path lengths.

The robot industry is developing rapidly, and how to control the redundant manipulators precisely and effectively has become a new hot topic in industry’s development. In recent years, many scholars in the industry have also proposed various control methods. However, most of these methods are proposed assuming that the Jacobian matrix is known. Actually, in practical applications, the detailed information of Jacobian matrix is often not precisely known. Therefore, this article develops a novel data-driven recurrent neural network (RNN) model that can update the Jacobian matrix and joint angles. By defining two dynamic error functions, two RNN designed formulas are used to obtain a continuous RNN (CRNN) model. Subsequently, the CRNN model is discretized by using Euler forward formula, and a discrete RNN (DRNN) model is generated. Then, a classic sliding mode control (SMC) algorithm is introduced, and DRNN-SMC model is further proposed. Moreover, the corresponding rigorous mathematical derivation and proof are carried out. In addition, simulation tests are carried out by using the Kinova Gen2 manipulator, comparing the DRNN model and PD controller, as well as the DRNN-SMC model and DRNN model, validating the precision of the DRNN-SMC model. Additionally, practical experiments using the Kinova Gen3 manipulator are performed to showcase the applicability and versatility of the DRNN-SMC model.

Abstract At present, the multi‐object tracking method based on transformer generally uses its powerful self‐attention mechanism and global modelling ability to improve the accuracy of object tracking. However, most existing methods excessively rely on hardware devices, leading to an inconsistency between accuracy and speed in practical applications. Therefore, a lightweight transformer joint position awareness algorithm is proposed to solve the above problems. Firstly, a joint attention module to enhance the ShuffleNet V2 network is proposed. This module comprises the spatio‐temporal pyramid module and the convolutional block attention module. The spatio‐temporal pyramid module fuses multi‐scale features to capture information on different spatial and temporal scales. The convolutional block attention module aggregates channel and spatial dimension information to enhance the representation ability of the model. Then, a position encoding generator module and a dynamic template update strategy are proposed to solve the occlusion. Group convolution is adopted in the input sequence through position encoding generator module, with each convolution group responsible for handling the relative positional relationships of a specific range. In order to improve the reliability of the template, dynamic template update strategy is used to update the template at the appropriate time. The effectiveness of the approach is validated on the MOT16, MOT17, and MOT20 datasets.

In the field of image descattering, the image formation models employed for restoration approaches are often simplified. In these models, scattering distribution is uniform in homogeneous media when transmission is fixed. Through specifically designed experiments, we discover that scattering exhibits non-uniform characteristics even in homogeneous media. Neglecting non-uniform scattering in these models limits their accuracy in representing scattering distribution, resulting in existing image descattering approaches inadequate. To tackle these issues, this paper proposes a novel image formation model for image descattering, considering more physical parameters, such as zenith angle, azimuth angle, scattering phase function, and camera focal length. Our model describes the light transfer process in scattering media more accurately. For image descattering, we introduce corresponding algorithms for parameter estimation in our model and simultaneous restoration from degraded images. Experimental evaluations demonstrate the effectiveness of our proposed model in various tasks, including physical parameter estimation, pure-scattering removal, image dehazing, and underwater image restoration. In terms of calculating parameters, our results are close to the real values; in terms of underwater image restoration, our work outperforms the state-of-art methods; in terms of image dehazing, our work promotes the performance of existing methods by replacing previous models with our model.

Non-cooperative target herding is one of the concerns in the robotics field. For the non-cooperative target herding problem, a general planning framework is proposed to compel the target to the destination by a mobile robot team with pursuit, encirclement, and guidance operations. In the proposed planning framework, the encirclement strategy and guidance strategy are designed to deal with the unpredictability of the target. Firstly, the mobile robots approach the randomly moving target in the pursuit process. Secondly, the encirclement strategy is applied in the encirclement process to enable the mobile robot team to encircle the non-cooperative target. Finally, the guidance strategy is employed in the guidance process to enable the mobile robot team to form a favorable encirclement formation and compel the non-cooperative target to the destination. During the herding task, the mobile robots encircle the target or other mobile robots in a satellite-like motion. Moreover, two queues (orbiters and wanders) are maintained to change the formation adaptively, improving the flexibility of the framework. Various environments are designed to verify the effectiveness of the planning framework both in simulations and real-world experiments. Furthermore, the proposed framework is a general platform having great potential for various applications, where the numbers of mobile robots in the herding team are changeable, and different planning algorithms can be integrated into the framework.Note to Practitioners—The motivation for this paper is to propose a planning framework to efficiently herd the non-cooperative target in various scenarios for applications such as the maintenance of public safety and the facilitation of wildlife migration. The existing cooperative herding methods typically maintain relatively fixed formations, while the planning framework proposed in this paper introduces a satellite-based motion pattern and diverse formation transformation strategies, thereby enhancing formation flexibility and adaptability. Consequently, this framework can be effectively applied to challenging environments such as those with restricted areas and dynamic obstacles. Furthermore, the proposed planning framework allows for flexible adjustments in path planning methods and the number of mobile robots to meet various practical application requirements. Through a series of simulations and real-world experiments, the reliability and favorable performance of the proposed algorithm framework in practical applications are demonstrated.

The refined aerial operations of the unmanned aerial manipulator (UAM) have been extensively studied for the last decades. Usually, UAM operations are accompanied by several phases, such as free flight, contact operations, and separation. A great challenge is proposed for switching operations in different environments and the high-precision contact force requirements of UAM control, so this paper conducts control stability research in the case of dynamic differences between free flight and contact operation for the UAM system. First, a hybrid force/position control strategy is proposed for switchable UAM system, among them, the adaptive sliding mode control method based on the interference observer is utilized in the free flight phase, and the adaptive impedance force control method is employed in the contact operation phase, where the adaptive estimation method is designed to perform on-board manipulator contact force estimation. Then a robust adaptive control strategy is proposed for the attitude loop to compensate for the torque disturbance generated during the contact operation phase. Meanwhile, the stability of the switching system is analyzed through the continuous Lyapunov function to prove the stability of the switching process. Finally, the effectiveness and superiority of the proposed schemes are verified through contact operation simulations and experiments.Note to Practitioners—This work is motivated by the contact force tracking of an UAM without force sensor. In recent years, hybrid force/position control has been widely used in UAM. However, force measurement is required at the end-effector and the object in most studies. The proposed method divides the contact operation process into free flight and contact operation stages. In the free flight stage, it is not necessary to know the prior information of the environment accurately (i.e. the external disturbance of slowly varying or known upper bound), and the disturbance observer is adopted to compensate for the disturbance caused by the external environment. In the contact operation stage, the impedance force control method is used for force tracking, which reduces the complexity and quality of the end-effector. At the same time, The shock caused by the switch from free flight to contact operation is reduced by calculating the appropriate controller parameters using the Lyapunov function. The proposed method is a promising solution for real applications and is validated via simulation and indoor contact experiments. The experimental results show that the proposed method has better stability and accuracy than the existing methods, and can be extended to industrial inspection, component processing, aerial operation, etc.

Compressive Spectral Imaging (CSI) techniques have attracted considerable attention among researchers for their ability to simultaneously capture spatial and spectral information using low-cost, compact optical components. A prominent example of CSI techniques is the Dual-Camera Coded Aperture Snapshot Spectral Imaging (DC-CASSI), which involves reconstructing hyperspectral images from CASSI measurements and uncoded panchromatic or RGB images. Despite its significance, the reconstruction process in DC-CASSI is challenging. Conventional DC-CASSI techniques rely on different models to explore the similarity between uncoded images and hyperspectral images. Nevertheless, two main issues persist: i) the effective utilization ofspatial informationfrom RGB images to guide the reconstruction process, and ii) the enhancement ofspectral consistencyof recovered images when using panchromatic/RGB images, which inherently lack precise spectral information. To address these challenges, we propose a novel Prior images guided generative autoEncoder (PiE) model. The PiE model leverages RGB images as prior information to enhance spatial details and designs a generative model to improve spectral quality. Notably, the generative model is optimized in a self-supervised manner. Comprehensive experimental results demonstrate that the proposed PiE method outperforms existing techniques, achieving state-of-the-art performance.

Few-shot object detection (FSOD) has brought increasing academic interest by recognizing previously unseen novel classes with very limited well-labeled samples. However, most existing methods identify novel classes via some object-specific characteristics in the few provided samples rather than intrinsic inter-class relations between base and novel classes, which heavily degrades the detection performance on novel classes. Moreover, they cannot learn discriminative proposal representations to distinguish base and novel classes, and thus misclassify novel objects as confusable base classes. To tackle the above challenges, we develop a novel Category-contextual Relation Encoding Network (CRE-Net), which is an early attempt to reason inter-class context relationships for FSOD task. To be specific, we propose a novel category-contextual relation encoding mechanism to capture intrinsic inter-class relations between base and novel classes via knowledge aggregation from global category-contextual descriptors. It utilizes intrinsic inter-class contextual relations to adaptively refine the convolution kernel, thus encoding the local semantic context of query image with category-contextual relation as guidance. Furthermore, to explore discriminative representations for base and novel classes, we develop a scarcity-compensatory contrastive proposal loss by incorporating data scarcity of novel classes and proposal semantic consistency with high confidence. This loss could compact object instances from the same category to a tighter cluster, and enhance the space separability of different classes. Extensive experiments on Pascal VOC and COCO datasets verify the state-of-the-art detection performance of our CRE-Net model when compared with other baseline methods.

Despite the remarkable progress made in learning-based stereo-matching algorithms, it is an open challenge for stereo-matching in disparity discontinuities and textureless regions. In this paper, we propose the deep Markov Random Field based cost aggregation network (DMCA-Net) for stereo matching, which is an end-to-end model-driven network architecture. This architecture introduces an efficient feature extraction network to extract richer textual and contextual feature information for stereo feature similarity representation at multi-stages and levels. Furthermore, with the aim of alleviating the edge-fattening phenomenon at disparity discontinuities and generating accurate disparities in textureless regions, we proposed the differentiable Markov Random Field model for cost aggregation, where the model’s data term utilizes image detail information, such as boundary and contour features, to guide matching cost aggregation, and the model’s smoothness term penalizes the adjacency similarity of the cost between the four-nearest neighboring pixel pairs to predict the disparity in textureless regions. The detailed experiment demonstrates that DMCA network achieves competitive performance on the SceneFlow, KITTI 2012, KITTI 2015, and Middlebury 2014 datasets.

Spectral imaging, with the ability to simultaneously capture the spectral and spatial information of scenes, has obtained researchers' significant interest. Traditional spectral imaging techniques typically suffer from high costs and slow imaging speed. Multispectral filter array-based snapshot imaging is a cutting-edge technology for mitigating these problems. The spectral image demosaicing algorithm plays a key role in reconstructing high-resolution spectral images from the raw measurement. In this article, we first formulate the multispectral demosaicing problem as a compressive spectral imaging problem. Then, a novel deep spectral image prior is introduced as the regularization, which assumes that neural networks can generate the desired spectral image from the raw image in a self-supervised learning manner. Finally, the constructed constrained problem is solved by the alternating direction method of multipliers optimization algorithm. Compared with existing methods, experiments on various scenes demonstrate that the proposed method obtains superior performance.

Transmission line (TL) fasteners play the role of connecting components in smart-grid transmission processes with abnormal TL fastener states, seriously impacting the power supply. Therefore, regular detection of TL fasteners is significant. However, the images taken by unmanned aerial vehicle have problems, such as small size and complex background, which bring great challenges to the existing object detection models. Based on this, this article proposes a multilevel spatial refinement network (MSRN), including an attention-guided receptive field enhanced feature pyramid network (ARFE-FPN) and a double refinement head (DR-Head). For the small target problem, ARFE-FPN first uses dilated convolution to expand the receptive field, and uses global average pooling to extract background activation values. Then, it performs channel weighting on TL fastener features under different fields of view. For the problem of complex background, DR-Head first constructs a semantic prediction task to realize the preseparation of foreground and background, and then combines the high-resolution feature map to further highlight the fastener features in the low-resolution feature map. Experiments on the TL fastener dataset show that MSRN has the best detection accuracy, and its AP can reach 92$\%$.

Due to ability to estimate the spatial transformation of coordinate frames, point cloud registration is a fundamental technique in manufacturing. Previous methods prone to converge to wrong local minima, in the cases of large initialization, noise, outliers, and partiality. This article presents a new learning-based robust point cloud registration approach to predict a rigid transformation in a one-shot way. Our network aims to determine a matchability matrix to yield an accurate registration result. Each element of the matchability matrix refers to similarity of learned per-point embeddings and represents the probability of a potential correspondence. The following two major blocks are developed to guide the matchability matrix to represent correct correspondences: an attention block is introduced to enhance the discriminativeness of learned per-point embeddings, and a zero-mean Gaussian-based annealing layer and a differentiable Sinkhorn normalization layer are designed to enforce a permutation matchability matrix. With the matchability matrix, an intuitive solution is integrated to obtain the relative transformation of the source and target point clouds. Different from the existing work, our network can handle partially overlapped point-cloud pairs effectively. Experimental results demonstrate the superiority of the proposed approach over the state-of-the-art registration approaches in terms of accuracy and robustness.

This article concerns the guaranteed trajectory estimation problem for unmanned surface vehicles (USVs) via set-membership estimation technique. Taking both rigid-body and hydrostatics kinetics into consideration, the nonlinear dynamic model of USV system is derived, where the parameters of system are all unknown. Considering external disturbance and nonlinearities, an offline data-based set-membership estimation algorithm of unknown system parameters is proposed to obtain the set representation of system parameters, which contain the actual parameters of USV. Then, based on the obtained parameter sets, an online guaranteed trajectory estimation algorithm of USV is constructed to provide guaranteed sets enclosing actual trajectory points of USV, which consists of a time-update step and measurement-update step. To tackle the nonlinear transformation of zonotopes, both interval arithmetic and Taylor model are utilized to provide rigorous bounds for nonlinear function. Finally, simulation results on a USV dynamic are provided to demonstrate the effectiveness of the proposed data-based guaranteed trajectory estimation method for USVs.

Hyperspectral images (HSIs) are composed of hundreds of contiguous waveband images, offering a wealth of spatial and spectral information. However, the practical use of HSIs is often hindered by the presence of complicated noise caused by various factors such as non-uniform sensor response and dark current. Traditional methods for denoising HSIs rely on constrained optimization approaches, where selecting appropriate prior knowledge is critical for achieving satisfactory results. Nevertheless, these traditional algorithms are limited by hand-crafted priors, leaving room for improvement in their denoising performance. Recently, the supervised deep learning technique has emerged as a promising approach for HSI denoising. However, their requirement for paired training data and poor generalization ability on untrained noise distributions pose challenges in practical applications. In this paper, we design a novel algorithm by the synergism of optimization-based methods and deep learning techniques. Specifically, we introduce a plug-and-play Deep Low-rank Decomposition (DLD) model into the optimization framework. Furthermore, we propose an effective mechanism to incorporate traditional prior knowledge into the DLD model. Finally, we provide a detailed analysis of the optimization process and convergence of the proposed method. Empirical evaluations on various tasks, including hyperspectral image denoising and spectral compressive imaging, demonstrate the superiority of our approach over state-of-the-art methods.

The tremendous revolutionary progress of cellular vehicle-to-everything (C-V2X) and vehicular edge computing (VEC) technologies provide new opportunities to overcome the autonomous transportation issue of the mining electric locomotives (MELs), in which the accurate and fast detection of obstacles is crucial for the safe operation. With the VEC and C-V2X, we proposed a new high-precision obstacle detection strategy for MELs (MEL-YOLO). Firstly, we investigated the convolutional attention mechanism integrated into the path aggregation network of the Neck layer to strengthen the feature extraction capabilities. Secondly, we added a small-object oriented prediction layer in the Head to form the multi-scale feature prediction. Thirdly, we introduced a more efficient loss function to alleviate the gradient explosion problem in the feature transfer. Finally, we utilized the K-means++ optimization to derive the anchor boxes matchable with the dataset, which was collected and created by featuring different scenes to train validate the model. The MEL-YOLO was compressed by BN layer pruning and implemented on the edge device in a 6G/B5G based-V2X environment. Experimental results verify that the MEL-YOLO can effectively detect obstacles and significantly improve detection accuracy for small obstacles, computationally increasing mAP by 3.3% to original model, while maintaining detection speed and model size nearly unchanged.

High-resolution microscopy hyperspectral (HS) images can provide highly detailed spatial and spectral information, enabling the identification and analysis of biological tissues at a microscale level. Recently, significant efforts have been devoted to enhancing the resolution of HS images by leveraging high spatial resolution multispectral (MS) images. However, the inherent hardware constraints lead to a significant distribution gap between HS and MS images, posing challenges for image super-resolution within biomedical domains. This discrepancy may arise from various factors, including variations in camera imaging principles (e.g., snapshot and push-broom imaging), shooting positions, and the presence of noise interference. To address these challenges, we introduced a unique unsupervised super-resolution framework named R2D2-GAN. This framework utilizes a generative adversarial network (GAN) to efficiently merge the two data modalities and improve the resolution of microscopy HS images. Traditionally, supervised approaches have relied on intuitive and sensitive loss functions, such as mean squared error (MSE). Our method, trained in a real-world unsupervised setting, benefits from exploiting consistent information across the two modalities. It employs a game-theoretic strategy and dynamic adversarial loss, rather than relying solely on fixed training strategies for reconstruction loss. Furthermore, we have augmented our proposed model with a central consistency regularization (CCR) module, aiming to further enhance the robustness of the R2D2-GAN. Our experimental results show that the proposed method is accurate and robust for super-resolution images. We specifically tested our proposed method on both a real and a synthetic dataset, obtaining promising results in comparison to other state-of-the-art methods.

This article presents an event-triggered adaptive neural impedance control (ETANIC) scheme for robotic systems, where the combination of impedance control (IC) and event-triggered mechanism can significantly reduce the computational burden and the communication cost under the premise of ensuring the stability and tracking performances of the robotic systems. The IC is used to achieve the compliant behavior of the robotic systems in response to the environment. The uncertainties of the robotic systems are estimated by the radial basis function neural network (RBFNN), and the update laws for RBFNN are derived from the designed Lyapunov function. The stability of the whole closed-loop control system is analyzed by the Lyapunov theory, and the event-triggered conditions are designed to avoid the Zeno behavior. The numerical simulation and experimental tests demonstrate that the proposed ETANIC scheme can achieve better efficiency for controlling the robotic systems to perform the interaction tasks with the environment in comparison to the adaptive neural IC (ANIC).

Most multilayer Moore-Penrose inverse (MPI)-based neural networks, such as deep random vector functional link (RVFL), are structured with two separate stages: unsupervised feature encoding and supervised pattern classification. Once the unsupervised learning is finished, the latent encoding is fixed without supervised fine-tuning. However, in complex tasks such as handling the ImageNet dataset, there are often many more clues that can be directly encoded, while unsupervised learning, by definition, cannot know exactly what is useful for a certain task. There is a need to retrain the latent space representations in the supervised pattern classification stage to learn some clues that unsupervised learning has not yet been learned. In particular, the residual error in the output layer is pulled back to each hidden layer, and the parameters of the hidden layers are recalculated with MPI for more robust representations. In this article, a recomputation-based multilayer network using Moore-Penrose inverse (RML-MP) is developed. A sparse RML-MP (SRML-MP) model to boost the performance of RML-MP is then proposed. The experimental results with varying training samples (from 3k to 1.8 million) show that the proposed models provide higher Top-1 testing accuracy than most representation learning algorithms. For reproducibility, the source codes are available at https://github.com/W1AE/Retraining.

Unknown distributed incipient thermal fault detection and localization are vital to the safe operation of batteries while they have not been given sufficient attention in existing works compared to the studies on estimation of State of Charge (SoC) as well as State of Health (SoH). In order to fill this gap, a backstepping-based fault localization filter (FLF) is presented. Generally, full-state temperature measurement is required to achieve fault localization, which is impossible in applications. However, with the help of interpolation-based approximation, the required number of sensors decreases from infinity to only a few, which guarantees the usability of FLF. A comprehensive methodology framework, including the FLF design, residual evaluation in a distributed manner, and threshold computation, is introduced to guarantee reliable and robust performance in a real-time pattern. Theoretic analysis as well as experiment validations are presented to for validation.

This article presents a resilient formation control framework for networked nonholonomic mobile robots (NMRs) that enables long-time recovery abilities subject to denial-of-service (DoS) attacks by taking advantage of the Koopman operator. Due to the intermittent interruption of communication under DoS, the transmitted signals among the networked NMRs are incomplete. In the lifted space, the infinite-dimensional Koopman operator is employed to capture a linear characteristic of the missed signals from the available signals. Specifically, a data-driven cost function is developed to approximate the infinite-dimensional Koopman operator, allowing long-term recovery capabilities for the missed signals, where the useful historical data is identified by an event-triggered mechanism (ETM). Then, the least-squares method is implemented to calculate a finite-dimensional approximation of the Koopman operator. Once DoS attacks are active, the missed signals are recovered forward from the latest received signals through the approximation Koopman operator. Furthermore, according to the recovered and transmitted signals, the resilient formation controller with a variable gain takes into account the convergence rate and the steady state formation error. The Lyapunov theorem is introduced to prove that the formation error quickly converges to the minor compact set. A distributed DoS attack example is conducted to validate the efficiency and superiority in numerical simulation, and the proposed method is implemented on the real networked NMRs.

Spectral images with rich spatial and spectral information have wide usage, however, traditional spectral imaging techniques undeniably take a long time to capture scenes. We consider the computational imaging problem of the snapshot spectral spectrometer, i.e., the Coded Aperture Snapshot Spectral Imaging (CASSI) system. For the sake of a fast and generalized reconstruction algorithm, we propose a prior image guidance-based snapshot compressive imaging method. Typically, the prior image denotes the RGB measurement captured by the additional uncoded panchromatic camera of the dual-camera CASSI system. We argue that the RGB image as a prior image can provide valuable semantic information. More importantly, we design the Prior Image Semantic Similarity (PIDS) regularization term to enhance the reconstructed spectral image fidelity. In particular, the PIDS is formulated as the difference between the total variation of the prior image and the recovered spectral image. Then, we solve the PIDS regularized reconstruction problem by the Alternating Direction Method of Multipliers (ADMM) optimization algorithm. Comprehensive experiments on various datasets demonstrate the superior performance of our method.

This article presents the problem of autonomous control of an unmanned aerial manipulator (UAM) developed for operation with unknown disturbances, wherein the disturbances from the coupling effect between the UAM and the external environment need to be considered. Regarding the coupling force as a disturbance to the entire UAM system, an adaptive prescribed performance control (APPC) scheme utilizing the knowledge of prescribed performance is proposed to guarantee the transient and steady-state performance responses. Also, an adaptive law is designed to estimate the upper boundary parameters of the UAM system uncertainties and disturbances, wherein the restrictive constant boundary assumptions and the prior information of the upper bound are not required in the controller design. Furthermore, to enable safe manipulation in a realistic situation, an end-effector trajectory generation method is presented satisfying the joint angle limitation. For the validation of the proposed method, the simulation results of numerical simulation comparisons are shown. Moreover, experimental scenarios including stable flight and simulated co-work with humans in complex environments are designed to verify the proposed method.Note to Practitioners—This article is motivated by the problem of aerial manipulation under unknown disturbances, which may be caused by the wide movement of the manipulator and the sudden loading or unloading of an object. Existing approaches for aerial manipulation often require the assumption of a constant or slowly varying external disturbance. However, a priori bounded disturbance might impose a priori bound on the system state before obtaining closed-loop stability. In this article, the proposed controller with an adaptive law is designed to estimate the upper boundary parameters of the overall disturbances and ensure the predefined performance, so that the prior information of the upper bound of disturbances is not required. The performance of the proposed control strategy is demonstrated via numerical simulation comparisons and experiments, including stale flight and simulated co-work with humans in a complex environment.

Unsupervised domain adaptation has attracted widespread attention as a promising method to solve the labeling difficulties of semantic segmentation tasks. It trains a segmentation network for unlabeled real target images using easily available labeled virtual source images. To improve performance, clustering is used to obtain domain-invariant feature representations. However, most clustering-based methods indiscriminately cluster all features mapped by category from both domains, causing the centroid shift and affecting the generation of discriminative features. We propose a novel clustering-based method that uses an adaptive refining-aggregation-separation framework, which learns the discriminative features by designing different adaptive schemes for different domains and features. The clustering does not require any tunable thresholds. To estimate more accurate domain-invariant centroids, we design different ways to guide the adaptive refinement of different domain features. A critic is proposed to directly evaluate the confidence of target features to solve the absence of target labels. We introduce a domain-balanced aggregation loss and two adaptive separation losses for distance and similarity respectively, which can discriminate clustering features by combining the refinement strategy to improve segmentation performance. Experimental results on GTA$5\rightarrow $Cityscapes and SYNTHIA$\rightarrow $Cityscapes benchmarks show that our method outperforms existing state-of-the-art methods.

In this article, an adaptive sliding-mode disturbance observer (ASMDO)-based finite-time control scheme with prescribed performance is proposed for an unmanned aerial manipulator (UAM) under uncertainties and external disturbances. First, to take into account the dynamic characteristics of the UAM, a dynamic model of the UAM with state-dependent uncertainties and external disturbances is introduced. Then, note that a priori bounded uncertainty may impose a priori constraint on the system state before obtaining closed-loop stability. To remove this assumption, an ASMDO with a nested adaptive structure is introduced to effectively estimate and compensate the external disturbances and state-dependent uncertainties in finite time without the information of the upper bound of the uncertainties and disturbances and their derivatives. Furthermore, based on the proposed ASMDO, the finite-time control scheme with the prescribed performance is presented to ensure finite-time convergence and implement the specified transient and steady-state performance. The Lyapunov tools are utilized to analyze the stability of the proposed controller. Finally, the correctness and performance of the proposed controller are illustrated through numerical simulation comparisons and outdoor experimental comparisons.

Pattern recognition is significantly challenging in real-world scenarios by the variability of visual statistics. Therefore, most existing algorithms relying on the independent identically distributed assumption of training and test data suffer from the poor generalization capability of inference on unseen testing datasets. Although numerous studies, including domain discriminator or domain-invariant feature learning, are proposed to alleviate this problem, the data-driven property and lack of interpretation of their principle throw researchers and developers off. Consequently, this dilemma incurs us to rethink the essence of networks' generalization. An observation that visual patterns cannot be discriminative after style transfer inspires us to take careful consideration of the importance of style features and content features. Does the style information related to the domain bias? How to effectively disentangle content and style features across domains? In this article, we first investigate the effect of feature normalization on domain adaptation. Based on it, we propose a novel normalization module to adaptively leverage the propagated information through each channel and batch of features called disentangling batch instance normalization (D-BIN). In this module, we explicitly explore domain-specific and domaininvariant feature disentanglement. We maneuver contrastive learning to encourage images with the same semantics from different domains to have similar content representations while having dissimilar style representations. Furthermore, we construct both self-form and dual-form regularizers for preserving the mutual information (MI) between feature representations of the normalization layer in order to compensate for the loss of discriminative information and effectively match the distributions across domains. D-BIN and the constrained term can be simply plugged into state-of-the-art (SOTA) networks to improve their performance. In the end, experiments, including domain adaptation and generalization, conducted on different datasets have proven their effectiveness.

In this article, an adaptive fixed-time fuzzy control scheme is proposed for an uncertain robot manipulator with user-defined performance. A novel symmetrical barrier Lyapunov function is designed based on the error conversion mechanism and the performance function such that the tracking errors will not violate the prescribed output constraints. A novel adaptive law is constructed and incorporated into the fixed-time controller design such that all the closed-loop signals can be bounded and achieve practical fixed-time convergence regardless of the initial conditions. Finally, the feasibility and superiority of the proposed scheme are demonstrated based on simulation and experimental studies using a Baxter robot.

Motion planning in an unknown environment is a common challenge because of the existing uncertainties. Representatively, the partially observable Markov decision process (POMDP) is a general mathematical framework for planning in uncertain environments. Recent POMDP solvers generally adopt the sparse reward scheme to solve the planning under uncertainty problem. Subsequently, the robot's exploration may be hindered without immediate rewards, resulting in excessively long planning time. In this article, a POMDP method, information entropy determinized sparse partially observation tree (IE-DESPOT), is proposed to explore a high-quality solution and efficient planning in unknown environments. First, a novel sample method integrating state distribution and Gaussian distribution is proposed to optimize the quality of the sampled states. Then, an information entropy based on sampled states is established for real-time reward calculation, resulting in the improvement of robot exploration efficiency. Moreover, the near-optimality and convergence of the proposed algorithm are analyzed. As a result, compared with general-purpose POMDP solvers, the proposed algorithm exhibits fast convergence to a near-optimal policy in many examples of interest. Furthermore, the IE-DESPOT's performance is verified in real mobile robot experiments.

In this article, an image-based visual servoing (IBVS) control strategy is proposed for the unmanned aerial manipulator (UAM) system to track and grasp a moving target. Specifically, a robust-adaptive velocity observer is designed to estimate the relative velocity between the tracked target and the UAM platform. Based on the velocity observer, an IBVS controller using onboard camera of the UAM platform is proposed for moving target tracking without velocity measurement. Then, the barrier Lyapunov function is introduced into the UAM platform IBVS controller to ensure the safety of target tracking. Besides, another virtual camera is constructed on manipulator end-effector to compensate for the tracking error of the UAM platform. As a benefit, the eye-to-hand onboard camera ensures the global view of the UAM, and the eye-in-hand virtual camera of the manipulator ensures the accuracy of the grasping task. Finally, the stability of the proposed IBVS control strategy is analyzed through Lyapunov theory. The comparative simulations are provided to illustrate the target tracking performance of the proposed method. The experimental results demonstrate that the proposed method can be applied to the UAM with a low-cost sensor suite to realize the tasks of tracking and grasping a moving target.

Mobile robots generally work in harsh and restricted environments, which poses challenges for mobile robots to find a feasible path efficiently. This article presents a planning method, namely, sampling-enhanced exploration tree (SET), to improve computational efficiency in restricted environments while guaranteeing high-quality performance. The core of SET is sampling-enhanced exploration, which consists of critical areas identification, guiding-exploration, and rectifying-exploration. In the critical areas identification phase, the restricted areas are identified based on the distribution of the hybrid samples. Next, the critical samples in restricted areas are selected as the origins of the sampling-enhanced exploration. In the guiding-exploration phase, the sampling-enhanced exploration starts from the origins and marches quickly with the guidance of the leader-samples to capture the spatial feature and connectivity of the restricted areas. The spatial information provides essential guidance for efficient biased sampling. In the rectifying-exploration phase, the directions of sampling-enhanced exploration are rectified to transit the problematic areas and supplement samples. Theoretical analysis is provided to shed light on the properties of SET. Moreover, the generality and effectiveness of SET are verified through a series of mobile robot simulations and real-world experiments.

Vaccines are the most significant and effective way to prevent disease and safeguard human health. However, it is easy to produce or mix foreign matters during the manufacturing process. Moreover, foreign matters are extremely faint that it is difficult to obtain images and detect them accurately. To tackle imaging challenges, in this article, we built a hyperspectral imaging system to construct a first-of-its-kind HSI dataset with pixel-level annotation for vaccine anomaly detection, where the vaccine comes from the actual pharmaceutical company. To address the problem of low detection accuracy, we propose a spectral–spatial anomaly perception module joint with an unsupervised autoencoder network (SSAPN), in which nonlinear features learned from the encoder are divided into nonoverlapping patches and mapped to efficiently encode spectral and spatial feature information. The spectral–spatial multilayer perceptrons (MLP) module consists of continuous and alternating spectral MLP with spatial MLP, which achieves spectral with spatial perception in the global receptive field, captures long-range dependencies, and extracts the most discriminative spectral–spatial features. Experimental results show that our SSAPN model outperforms other state-of-the-art anomaly detection methods in terms of both detection and generalization performance. This work will help speed up the production process in the vaccine pharmaceutical industry and ensure vaccine quality.

Empowered by labeled datasets, supervised pre-training based transfer learning (SPTL) has made significant advances for image classification applications. However, due to privacy-preserving protocol and unaccessible annotation, it emerges as a novel problem in federated learning scenarios whether unsupervised pre-training based transfer learning (UPTL) is available for semantic segmentation. In this work, we define federated transfer learning (FedTL) in the absence of source domain label, and track research progress on pavement crack benchmark. The main challenges of FedTL include: i) a privacy-protecting distributed training framework that extends UPTL to the constraints of federated settings, and ii) a self-learning semantic segmentation approach that develops self-supervised learning paradigm to simultaneously learn category and shape representations. Motivated by that, we propose a FedCrack model to absorb feature disentanglement and prototype clustering into vision Transformer, which obtains the pre-trained encoder on source domain without accessing annotation. Thereafter, a fine-tuning stage is presented to learn decoder with scaling attention on target domain for fine-grained crack segmentation. The effectiveness of proposed FedCrack can be demonstrated with superior performance of 82.14% on mIoU and 9.85 FPS on speed in extensive experiments. To the best of our knowledge, it is the first work in FedTL to gain weights of unsupervised pre-training representations on source domain locally, gradients of which are then aggregated to a federated central model that also fine-tunes the transferable parameters by target domain.

Stereo matching depth estimation based on rectified image pairs is of great importance to many computer vision tasks such as vehicle navigation and autonomous driving. Confidence measures are typically used to refine stereo matching results, which provides robustness and efficiency for disparity estimation. However, previous learning-based confidence methods for stereo matching usually use the middle results or composition as a post-processing step to refine the stereo matching results. This cannot be optimized end-to-end and the performance is limited by the quality of the tri-modal output. To handle this issue, in this paper, we pursue an end-to-end hierarchical architecture and propose a differentiable confidence propagation (DCP) model of a cost aggregation network for stereo matching. The DCP model is integrated into an end-to-end neural network hierarchical architecture to guide matching cost volume aggregation. More specifically, to better represent the similarity of left and right feature maps, we extract unary context feature maps with an effective attention mechanism for matching cost construction. Moreover, we aggregate the cost volume with the multiple stacked DCP cost aggregation (DCPCA) networks to generate a more reliable and finer cost volume. This network suppresses multi-level disparity maps. Each output disparity is supervised with different training weights to learn in a coarse-to-fine way. Our method outperforms previous methods on the Sceneflow dataset by achieving the$0.6735px$EPE error, achieving 1.53% D1-all metric of Non-occluded pixels regions and 0.72% Non-occluded pixels of$5px$metric on KITTI 2015 and 2012 dataset. Extensive experiments carried out on the KITTI Stereo benchmarks demonstrate that our DCPCA-Net can significantly minimize the trade-off between accuracy and efficiency for stereo matching.

Unsupervised Domain Adaptation (UDA) aims to transfer knowledge from a well-labeled source domain to an unlabeled target domain with a correlative distribution. Numerous existing approaches process this hard nut by directly matching the marginal distribution between two domains, which confront the obstacle of rough alignment and blurred decision boundary. Recent advances in UDA introduce target pseudo-label and subdomain adaptation to reduce misalignment and distribution discrepancy. Whereas, they frequently ignore that the production of target pseudo-label is so dependent on the source-trained classifier, which without reasonable restriction to discriminate generated pseudo-label is whether confident. Meanwhile, many methods in the subdomain alignment metric ignore exploring the potential distribution discrepancy between same-class samples of the intra-domain. To address these two issues simultaneously, this paper proposes a Cycle Consistency based Pseudo Label and Fine Alignment (CCPLFA) approach for UDA. In particular, firstly, a novel cycle-consistency based pseudo label module is designed, which is a simple yet effective way to alleviate the noise of pseudo labels and improve their semantic correctness. Secondly, we develop a Fine-Alignment distribution matching metric. Which can maximize the feature distribution density of intra-class cross-domains and not overlook the distribution structure of the global aspect. Comprehensive experiment results on four benchmarks demonstrate the capability of plug and play and the well generalization performance of our proposed method.

This article examines a new measure of combining the vector Lyapunov function with${M}$-matrix for settling the asymptotic stabilization control of fractional-order memristor-based neural networks system (FOMBNNS) has large delays in various dimensional forms. Some new stability and stabilization criteria are deduced. First, the vector Lyapunov function and${M}$-matrix are imported for investigating stabilization control for the above system. Then, we solve the problem for a special type of situation that the activation functions no longer consider Lipschitz parameters via the new method. Finally, four numerical examples from different kinds of situations are simulated for expounding the validity of the novel asymptotic stability and stabilization criteria. Compared with the methods mentioned in the current references, the proposed asymptotic stability and stabilization criteria in this article have strong generality and universality. They can be applied not only to the most common feedback control, accordingly, the feedback control law based on which they are designed but also to all fractional-order parameters from 0 to 1. In addition, the new method has lower conservativeness and fewer constraints. Moreover, the new stability and stabilization criteria can also overcome the difficulty in dealing with the above system owning large delays.

The phenomenon of increasing accidents caused by reduced vigilance does exist. In the future, the high accuracy of vigilance estimation will play a significant role in public transportation safety. We propose a multimodal regression network that consists of multichannel deep autoencoders with subnetwork neurons (MCDAE$_{sn}$). After we define two thresholds of “0.35” and “0.70” from the percentage of eye closure, the output values are in the continuous range of 0–0.35, 0.36–0.70, and 0.71–1 representing the awake state, the tired state, and the drowsy state, respectively. To verify the efficiency of our strategy, we first applied the proposed approach to a single modality. Then, for the multimodality, since the complementary information between forehead electrooculography and electroencephalography features, we found the performance of the proposed approach using features fusion significantly improved, demonstrating the effectiveness and efficiency of our method.

This article aims to solve the problem of formation control of mobile robots based on image and provide a low-cost as well as ease-of-implementation solution for mobile robots relying merely on a monocular camera under field-of-view (FOV) constraints. A low-complexity image-based visual servo controller is proposed, which can achieve the desired relative position on the image plane and solve the FOV constraints without the feature depth and leader’s velocities information. To facilitate the control design, a state transformation is first performed to decouple the visual motion kinematics. Then, an error transformation is introduced to handle the FOV constraints, and performance specifications are incorporated in the error transformation to achieve the predefined control performance. Finally, a simple static controller is derived using only information from images, and the stability of the uncertain system with unknown control direction/coefficients under the given performance control condition is analyzed. The effectiveness and performance of the proposed visual servoing controller can be illustrated using both simulations and experiments.

Anchor or anchor-free based Siamese trackers have achieved the astonishing advancement. However, their parallel regression and classification branches lack the tracked target information link and interaction, and the corresponding independent optimization maybe lead to task-misalignment, such as the reliable classification prediction with imprecisely localization and vice versa. To address this problem, we develop a general Siamese dense regression tracker (SDRT) with both task and feature alignments. It consists of two cooperative and mutual-guidance core branches: dense local regression with RepPoint representation, the global and local multi-classifier fusion with aligned features. They complement and boost each other to constrain the results with well-localized followed to also be well-classified. Specifically, a dense local regression with RepPoint representation, directly estimates and averages multiple dense local bounding box offsets for accurate localization. And then, the refined bounding boxes can be used to learn the global and local affine alignment features for reliable multi-classifier fusion. The classified scores in turn guide the assigned positive bounding boxes for the regression task. The mutual guidance operations can bridge the connection between classification and regression substantially, since the assigned labels of one task depend on the prediction quality of the other task. The proposed tracking module is general, and it can boost both the anchor or anchor-free based Siamese trackers to some extent. The extensive tracking comparisons on six tracking benchmarks verify its favorable and competitive performance over states-of-the-arts tracking modules.

Stereo matching depth estimation for rectified image pairs is of great importance to many compute vision tasks, specifically in autonomous driving. With the flourishing of convolution neural networks, responsible depth estimation of stereo matching with artificial intelligence is the most severe challenge for autonomous driving in recent years. Previous research on end-to-end trainable stereo matching networks has usually used cascading convolution blocks with down-sampling or pooling operations to extract the unary features required for matching cost construction. Such approaches lack a reconstruction stage for increasing feature map pixel-wise alignment and strength, factors which play an important role in representing the similarity between stereo image pairs. To address this issue, in this paper, we propose the progressive fusion stereo matching network (PFSM-Net). We exploit an encoder-decoder feature extraction network architecture for multi-stage and -scale dynamic feature extraction. Moreover, we propose a group-wise concatenation method to construct the cost volume, which provides a more efficient cost volume for cost aggregation. Furthermore, we propose the use of multi-scale cost aggregation networks with a progressive fusion strategy. The aggregated cost volume is progressively fused with the multi-stage and -scale cost volume as the size of the cost volume increases. Multi-stage and -scale outputs are supervised with and learned in a coarse-to-fine manner. Experimental results demonstrate that our method outperforms previous methods on the SceneFlow, KITTI 2012, and KITTI 2015 datasets.

Defects on rail surfaces, which have become critical problems, need to be detected and removed as quickly as possible to ensure the fast, safe, and stable operation of trains. At present, although many solutions have been proposed to address these problems, the comprehensiveness, rapidity, and accuracy of defect detection remain unsatisfactory. This study aims to resolve these existing problems and accordingly proposes a multi-model rail surface defect detection system based on convolutional neural networks (MRSDI-CNN) from the standpoint of studying the squat on the rail surface. The convolutional neural networks utilized include the improved Single Shot MultiBox Detector (SSD) and You Only Look Once version 3(YOLOv3)—two types of one-stage networks. We expounded and analyzed the performance of the convolutional neural networks as well as their applicability to rail surface defect detection. We used a diverse range of rail defect sizes to improve the detection performance of the two deep learning networks, following which they could identify three types of squats in parallel with improved accuracy and without reduction of the detection speed. The experimental results confirm the effectiveness and superiority of the proposed method over those of previous studies.

3D object grounding aims to locate the most relevant target object in a raw point cloud scene based on a freeform language description. Understanding complex and diverse descriptions, and lifting them directly to a point cloud is a new and challenging topic due to the irregular and sparse nature of point clouds. There are three main challenges in 3D object grounding: to find the main focus in the complex and diverse description; to understand the point cloud scene; and to locate the target object. In this paper, we address all three challenges. Firstly, we propose a language scene graph module to capture the rich structure and long-distance phrase correlations. Secondly, we introduce a multi-level 3D proposal relation graph module to extract the object-object and object-scene co-occurrence relationships, and strengthen the visual features of the initial proposals. Lastly, we develop a description guided 3D visual graph module to encode global contexts of phrases and proposals by a nodes matching strategy. Extensive experiments on challenging benchmark datasets (ScanRefer [3] and Nr3D [42]) show that our algorithm outperforms existing state-of-the-art. Our code is available at https://github.com/PNXD/FFL-3DOG.

Pan-sharpening is an effective method to obtain high-resolution multispectral images by fusing panchromatic (PAN) images with fine spatial structure and low-resolution multispectral images with rich spectral information. In this article, a multiscale pan-sharpening method based on dynamic convolutional neural network is proposed. The filters in dynamic convolution are generated dynamically and locally by the filter generation network which is different from the standard convolution and strengthens the adaptivity of the network. The dynamic filters are adaptively changed according to the input images. The proposed multiscale dynamic convolutions extract detail feature of PAN image at different scales. Multiscale network structure is beneficial to obtain effective detail features. The weights obtained by the weight generation network are used to adjust the relationship among the detail features in each scale. The GeoEye-1, QuickBird, and WorldView-3 data are used to evaluate the performance of the proposed method. Compared with the widely used state-of-the-art pan-sharpening approaches, the experimental results demonstrate the superiority of the proposed method in terms of both objective quality indexes and visual performance.

This article presents a new supervised multilayer subnetwork-based feature refinement and classification model for representation learning. The novelties of this algorithm are as follows: 1) different from most multilayer networks that go deeper with increased number of network layers, this work architects a model with wider subnetwork nodes; 2) the conventional classification methods adopt a separate search mechanism to derive a generalized feature space and to get the final cognition, but this work proposes a one-shot process to find the meaningful latent space and recognize the objects; and 3) the traditional feature representation and image classification approaches apply a unimodal feature coding, which suffers from lack of global knowledge. This work overcomes the pitfall through multimodal fusion that fuses various feature sources into one superstate encoding to achieve higher performance. A cross-domain experimental study on camera identification and image classification shows that the proposed method achieves superior performance compared to the existing models.

Anomaly detection refers to the identification of cases that do not conform to the expected pattern, which takes a key role in diverse research areas and application domains. Most of existing methods can be summarized as anomaly object detection-based and reconstruction error-based techniques. However, due to the bottleneck of defining encompasses of real-world high-diversity outliers and inaccessible inference process, individually, most of them have not derived groundbreaking progress. To deal with those imperfectness, and motivated by memory-based decision-making and visual attention mechanism as a filter to select environmental information in human vision perceptual system, in this paper, we propose a Multi-scale Attention Memory with hash addressing Autoencoder network (MAMA Net) for anomaly detection. First, to overcome a battery of problems result from the restricted stationary receptive field of convolution operator, we coin the multi-scale global spatial attention block which can be straightforwardly plugged into any networks as sampling, upsampling and downsampling function. On account of its efficient features representation ability, networks can achieve competitive results with only several level blocks. Second, it's observed that traditional autoencoder can only learn an ambiguous model that also reconstructs anomalies "well" due to lack of constraints in training and inference process. To mitigate this challenge, we design a hash addressing memory module that proves abnormalities to produce higher reconstruction error for classification. In addition, we couple the mean square error (MSE) with Wasserstein loss to improve the encoding data distribution. Experiments on various datasets, including two different COVID-19 datasets and one brain MRI (RIDER) dataset prove the robustness and excellent generalization of the proposed MAMA Net.

Defects on the surface of fabrics seriously affect the production speed and quality of textile products. There are many difficulties in the detection of surface defects on fabrics, such as substantial differences in length-width ratio, uneven distribution, and few features. However, existing methods have the disadvantages of slow detection speed and high misdetection rate. This present study proposes a method of integrating deformable convolution and pyramid network in Cascade R-CNN (IDPNet) for fabric defect detection. First, image data are labeled according to the type and distribution of defects. Then we design a novel multi-stage object detection architecture named IDPNet to detect defects on the surface of fabrics. In the first stage, Resnet50, in combination with feature pyramid network and deformable convolution is used to improve the detection performance of small defects. Besides, we trained a sequence of detectors with increasing IoUs stage by stage based on Cascade R-CNN in the second stage. Finally, experimental results demonstrate that the proposed neural network equip an outstanding performance against other approaches and achieve the accuracy of 91.57% in fabric defect detection, which proves its utility in practice.

Citrus detection is an essential component of the citrus industry. In order to realize the identification, positioning, segmentation, maturity estimation, and quality evaluation of citrus in complex environments, this paper proposes a multi-task deep learning network that can be applied to multiple scenes for citrus detection. The system is based on the Mask R-CNN network framework. By adding multi-task branches, modifying model parameters, and designing multi-task loss function, it can realize multi-task detection of citrus in a complex environment. The mAP on the validation set of the model obtained after training is 91.56%, and it takes an average of 0.35s to detect a citrus image using GeForce GTX 1080 Ti. Through the comparative analysis of the detection effect and performance evaluation index F value of multi-task citrus under different maturity, quality, citrus quantity, and light angle, the experimental results show that the model can effectively and accurately detect the citrus with different maturity and quality in the environment of citrus fruit overlap, tree branch and leaf occlusion, light change and surface shadow. The code is available at https://github.com/wxx-gan/Multitask.

Glass bottles must be thoroughly inspected before they are used for packaging. However, the vision inspection of bottle bottoms for defects remains a challenging task in quality control due to inaccurate localization, the difficulty in detecting defects in the texture region, and the intrinsically nonuniform brightness across the central panel. To overcome these problems, we propose a surface defect detection framework, which is composed of three main parts. First, a new localization method named entropy rate superpixel circle detection (ERSCD), which combines least-squares circle detection and entropy rate superpixel (ERS) with an improved randomized circle detection, is proposed to accurately obtain the region of interest (ROI) of the bottle bottom. Then, according to the structure-property, the ROI is divided into two measurement regions: central panel region and annular texture region. For the former, a defect detection method named frequency-tuned anisotropic diffusion super-pixel segmentation (FTADSP) that integrates frequency-tuned salient region detection (FT), anisotropic diffusion, and an improved superpixel segmentation is proposed to precisely detect the regions and boundaries of defects. For the latter, a defect detection strategy called wavelet transform multiscale filtering (WTMF) based on a wavelet transform and a multiscale filtering algorithm is proposed to reduce the influence of texture and to improve the robustness to localization error. The proposed framework is tested on four data sets obtained by our designed vision system. The experimental results demonstrate that our framework achieves the best performance compared with many traditional methods.

With the development of artificial intelligence, robot swarm systems also frequently appear in complex tasks of different situation. One of the important research directions is the formation of multi-robots. This paper analyzes the limitations of existing algorithms for large-scale mobile robot swarm formation control problems and proposes a consensus control algorithm with two-layer nearest neighbor information. It carries out experimental simulation to verify its convergence performance. At the same time, combined with a distributed structure control strategy that can change the number of robot formation members, the formation control experiment is carried out on the experimental platform consisted of robot state information detection device and multiple mobile robots,to further verify its feasibility.

Automatic image classification has become a necessary task to handle the rapidly growing digital image usage. It has branched out many algorithms and adopted new techniques. Among them, feature fusion‐based image classification methods rely on hand‐crafted features traditionally. However, it has been proven that the bottleneck features extracted through pre‐trained convolutional neural networks (CNNs) can improve the classification accuracy. Thence, this study analyses the effect of fusing such cues from multiple architectures without being tied to any hand‐crafted features. First, the CNN features are extracted from three different pre‐trained models, namely AlexNet, VGG‐16, and Inception‐V3. Then, a generalised feature space is formed by employing principal component reconstruction and energy‐level normalisation, where the features from individual CNN are mapped into a common subspace and embedded using arithmetic rules to construct fused feature vectors (FFVs). This transformation play a vital role in creating a representation that is appearance invariant by capturing complementary information of different high‐level features. Finally, a multi‐class linear support vector machine is trained. The experimental results demonstrate that such multi‐modal CNN feature fusion is well suited for image/object classification tasks, but surprisingly it has not been explored so far by the computer vision research community extensively.

Ampoule injection is a routinely used treatment in hospitals due to its rapid effect after intravenous injection. During manufacturing, tiny foreign particles can be present in the ampoule injection. Therefore, strict inspection must be performed before ampoule injections can be sold for hospital use. In the quality control inspection process, most ampoule enterprises still rely on manual inspection which suffers from inherent inconsistency and unreliability. This paper reports an automated system for inspecting foreign particles within ampoule injections. A custom-designed hardware platform is applied for ampoule transportation, particle agitation, and image capturing and analysis. Constructed trajectories of moving objects within liquid are proposed for use to differentiate foreign particles from air bubbles and random noise. To accurately classify foreign particles, multiple features including particle area, mean gray value, geometric invariant moments, and wavelet packet energy spectrum are used in supervised learning to generate feature vectors. The results show that the proposed algorithm is effective in classifying foreign particles and reducing false positive rates. The automated inspection system inspects over 150 ampoule injections per minute (versus ~ 12 ampoule injections per minute by technologist) with higher accuracy and repeatability. In addition, the automated system is capable of diagnosing impurity types while existing inspection systems are not able to classify detected particles.