Path planning is a fundamental application of an unmanned aerial vehicle (UAV) swarm. Performing such a task in a complex mountain environment with a wind field would encounter challenges such as the sim-to-real (simulation-to-reality) gap. In this paper, we present a digital twin (a virtual replica of the physical system)-assisted path-planning framework for a UAV swarm with two phases: global path planning and real-time trajectory planning. For global path planning, we develop an optimization model that combines the constraints of a single UAV and the swarming rules, while also accounting for wind effects. To solve the optimization model, we improve the polar lights optimization (PLO) algorithm via multiple strategies (named PLOM), enhancing the initialization, exploitation, exploration, and equilibration processes. The major improvement strategies consist of opposition-based learning with refraction and elite, the logarithmic spiral motion, the Cauchy-Gaussian operator, and sine cosine perturbation. We construct a realistic geographical simulation environment based on a digital elevation model (DEM) and dominant wind, and design controlled experiments under different conditions of UAVs, threats, waypoints, and wind speeds. The simulation results demonstrate that the PLOM algorithm always achieves the best solution in swarm path planning scenarios with different complexities. Meanwhile, the PLOM algorithm has the greatest robustness with nearly the shortest runtime.

With the rapid development of low-altitude economies, unmanned aerial vehicle (UAV) swarm has attracted growing interest for cooperative target search. However, most existing studies focus on static targets and assume UAVs operate at a single horizontal altitude, limiting their practical applicability. This paper proposes a novel multi-UAV cooperative search framework for moving targets based on multi-agent deep reinforcement learning (MADRL). By coordinating UAVs across high, medium, and low-altitude layers, the system achieves improved search efficiency through altitude-adaptive operations. We further introduce a revisit-time compensation mechanism to enhance detection performance for moving targets in a multi-layer UAV swarm. To address the challenges of slow convergence and sparse feedback in MADRL, we propose hybrid-reward-based value decomposition networks (HRVDN) algorithm that integrates dense local rewards with sparse global rewards, accelerating learning while encouraging agents to collect high-value information. Simulation results demonstrate that the proposed approach outperforms existing methods in terms of target search rate and area coverage.

No-reference image quality assessment (NR-IQA) aims to quantitatively measure human perception of visual quality without comparing a distorted image to a reference. Despite recent advances, existing NR-IQR approaches often demonstrate insufficient ability to capture perceptual cues in the absence of a reference, limiting their generalisability across diverse and complex real-world image degradations. These limitations hinder their ability to match the reliability of full-reference IQA (FR-IQA) counterparts. A key challenge, therefore, is to enable NR-IQA models to emulate the reference-aware reasoning exhibited by humans and FR-IQA methods. To address this challenge, we propose a novel NR-IQA model based on a knowledge-sharing (KS) strategy to simulate this capability and predict image quality more effectively. Specifically, we designate an FR-IQA model as the teacher and an NR-IQA model as the student. Unlike conventional knowledge distillation (KD), our proposed architecture enables the NR-IQA student and FR-IQA teacher to share a decoder rather than being independent models. Furthermore, the student model contains a Mental Imagery Generation (MIG) module to learn mental imagery as the reference. To fully exploit local and global information, we adopt a vision transformer (ViT) branch and a convolutional neural network branch for feature extraction (FE). Finally, a quality-aware regressor (QAR) combined with deep ordinal regression is constructed to infer the quality score. Experiments show that our proposed NR-IQA model, KSIQA, has class-leading performance against current no-reference (NR) techniques across widespread benchmark datasets.

Digital twin network (DTN) technology provides significant support for intelligent applications of unmanned aerial vehicle (UAV) swarms. However, related research focuses on DTN applications and lacks attention to the construction and maintenance of high-fidelity digital twin (DT) models. In this paper, we first develop a DT simulation platform for UAV swarms. Then, a dynamic data-driven DT model calibration scheme is proposed on the example of the most fundamental flight state of a UAV. The scheme utilizes parameter identification to estimate offline the key parameters of the measurement model and system deviations. Furthermore, the flight state is corrected online utilizing data assimilation actuated on the actual and virtual data. Simulation experiments on the DT simulation platform are conducted, and the effects of data sampling rate on calibration accuracy and computational load are analyzed. The results demonstrate that parameter identification and data assimilation significantly improve the fidelity of the DT model in terms of the optimal sub-pattern assignment (OSPA) metric to different degrees.

As an advanced framework, the digital twin network (DTN) provides effective management and decision support for autonomous aerial vehicle (AAV) swarms and has become a recent research hotspot. The effectiveness of many DTN applications relies on the assumption that high-fidelity digital twin (DT) models exist and are readily available. However, constructing such high-fidelity DT models of AAV swarms is a challenging task, especially in complex and dynamic environments. Despite its importance, there is a notable lack of research focused on the construction of high-fidelity DT models specifically for AAV swarms. This study proposes a dynamic data-driven approach for constructing and calibrating DT models of AAV swarms to achieve long-term consistency with real-world AAV behaviors. The method leverages parameter identification to estimate key parameters of DT models and data assimilation to refine and calibrate the model. It can provide high-fidelity DT AAV models for artificial intelligence model training and facilitate AAV swarm DTN from concept to real application. Additionally, this article developed a DT simulation platform for AAV swarms, validating the proposed method through software-in-the-loop simulations and physical testing. Results indicate that the optimal subpattern assignment metric decreases by an average of 79.2% after calibration, significantly improving the DT model’s fidelity.

Cooperative target search (CTS) technology is highly desirable in various multi-autonomous aerial vehicle (AAV) applications. However, searching for unknown targets in a dynamic threatening environment is a challenging problem, especially for AAVs with limited sensing range and communication capabilities. Besides, traditional searching methods lack scalability and efficient collaboration among the AAV swarm in dynamic environments. In this work, a digital twin (DT)-enabled distributed CTS approach was presented for AAV swarms and achieving sim-to-real transfer. Specifically, a new scalable multi-agent reinforcement learning (MARL) based algorithm called SAMARL is adopted to improve effectiveness and adaptability, combining a multi-head attention mechanism. In SAMARL, a scalable observation space with graph representation and an environmental cognition map is designed to thoroughly consider the target search rate, area coverage, and safety assurance. Then, a DT-driven training framework is proposed to facilitate the continuous evolution of MARL models and address the tradeoff between training speed and environment fidelity. Furthermore, we innovatively develop a distributed AAV swarm digital twin cooperative target search validation system, including real flight control, communication simulation tools, and a 3D physics engine. Extensive simulations validate its superiority compared to state-of-the-art strategies. More importantly, we also conduct real-world flight experiments on different scale mission areas and AAV swarms, further demonstrating the generalization and scalability of trained models.

Advancements in deep learning have led to significant progress in no-reference (NR) image quality assessment (NR-IQA) for evaluating the perceived quality of digital images without relying on a reference. However, existing NR-IQA models remain suboptimal in handling complex and diverse natural images. Visual saliency constitutes a critical element for enhancing the reliability of NR-IQA, but the optimal use of saliency in deep learning-based NR-IQA has not heretofore been significantly explored. In this article, we present a novel method for integrating saliency in NR-IQA, which is motivated by the saliency-based visual search mechanism that different parts of the visual input are visited by the focus of attention (FOA) in the order of decreasing saliency. By dividing saliency into the high and low levels of FOA, we build a bioinspired deep neural network-BioSIQNet-based on a multitask learning (MTL) framework. The network architecture consists of two saliency-specific tasks and one primary image quality assessment (IQA) task. The low and high saliency (HS) are separately encoded and integrated into the early and deeper layers of the IQA network, respectively, analogous to the hierarchical processing in the visual cortex of the brain that allocates low attentional resources to process the simple patterns and high resources to learn intricate representations. We demonstrate that leveraging the synergy between visual attention and image quality perception and joint learning of these interconnected visual tasks can enhance the overall learning capabilities of the primary IQA model. Experiments validate the effectiveness of our proposed BioSIQNet for NR-IQA.

Recent advancements in blind image quality assessment (BIQA) are primarily propelled by deep learning technologies. While leveraging transformers can effectively capture long-range dependencies and contextual details in images, the significance of local information in image quality assessment can be undervalued. To address this challenging problem, we propose a novel feature enhancement framework tailored for BIQA. Specifically, we devise an Adaptive Graph Attention (AGA) module to simultaneously augment both local and contextual information. It not only refines the post-transformer features into an adaptive graph, facilitating local information enhancement, but also exploits interactions amongst diverse feature channels. The proposed technique can better reduce redundant information introduced during feature updates compared to traditional convolution layers, streamlining the self-updating process for feature maps. Experimental results show that our proposed model outperforms state-of-the-art BIQA models in predicting the perceived quality of images. The code of the model will be made publicly available.

When images undergo quality degradation caused by editing, compression or transmission, their saliency tends to shift away from its original position. Saliency shifts indicate visual behaviour change and therefore contain vital information regarding perception of visual content and its distortions. Given a pristine image and its distorted format, we want to be able to detect saliency shifts induced by distortions. The resulting saliency shift map (SSM) can be used to identify the region and degree of visual distraction caused by distortions, and consequently to perceptually optimise image coding or enhancement algorithms. To this end, we first create a largest-of-its-kind eye-tracking database, comprising 60 pristine images and their associated 540 distorted formats viewed by 96 subjects. We then propose a computational model to predict the saliency shift map (SSM), utilising transformers and convolutional neural networks. Experimental results demonstrate that the proposed model is highly effective in detecting distortion-induced saliency shifts in natural images.

Due to the rapid development of deep learning techniques, no-reference image quality assessment (NR-IQA) has achieved significant improvement. NR-IQA aims to predict a real-valued variable for image quality, using the image in question as the sole input. Existing deep learning-based NR-IQA models are formulated as a regression problem and trained by minimising the mean squared error. The error measurement does not consider the relative ordering between different ratings on the quality scale, which consequently affects the efficacy of the model. To account for this problem, we reformulate NR-IQA learning as an ordinal regression problem and propose a simple yet effective framework using deep convolutional neural networks (DCNN) and Transformers. NR-IQA learning is achieved by a deep ordinal loss and using a soft ordinal inference to transform the predicted probabilities to a continuous variable for image quality. Experimental results demonstrate the superiority of our proposed NR-IQA model based on deep ordinal regression. In addition, this framework can be easily extended with various DCNN architectures to build advanced IQA models.