With the explosive growth of e-commerce, efficient last-mile delivery has emerged as a critical challenge. Truck-drone collaborative delivery has garnered significant attention as a promising solution to this problem. In this work, we formulate the truck-drone collaborative delivery problem as a collaborative optimization problem, aiming to minimize the total completion time for delivering packages to customers by leveraging the complementary strengths of the drone’s speed and the truck’s endurance. We propose a two-stage framework to address this challenge. In the first stage, the Lin-Kernighan Helsgaun (LKH) algorithm is employed to generate a high-quality initial Traveling Salesman Problem (TSP) solution, serving as a robust starting point. In the second stage, a Sequence Allocate Policy (SAPPO), based on Proximal Policy Optimization, refines the TSP solution by optimizing the truck-drone collaborative path using a specially designed action space. Extensive experiments conducted on both random dataset and TSPLIB benchmarks demonstrate that our method significantly outperforms existing algorithms regarding delivery time, while exhibiting improved scalability and less training time.

The rise in urbanization and the demand for refined oil pose a significant challenge in improving transportation efficiency while ensuring the safety of gas station inventories. Traditional optimization methods, which rely on deterministic models or demand forecasting, enhance efficiency but struggle with uncertainties such as demand fluctuations. Multi-agent reinforcement learning (MARL) presents a promising approach for adaptive cooperative dispatching. This study models the refined oil dispatch problem as a partially observable constrained Markov game (CPOMG), in which agents make cooperative decisions under inventory constraints with limited observation, and proposes Hierarchical Action-Constrained MAPPO (HAC-MAPPO), a novel cooperation MARL algorithm to find the optimal cooperative dispatching policy of the game. Within the CPOMG framework, the HAC-MAPPO agents optimize order generation decisions based on the states of gas station inventories. The subsequent delivery routing for these orders is optimized using an integrated fuel routing tabu search algorithm, which is part of the environment’s state transition logic. HAC-MAPPO incorporates inventory safety thresholds to constrain order generation decisions across fuel types. It employs a dual-critic architecture and a shared hierarchical actor network for efficient decentralized policy learning. Crucially, the Lagrangian relaxation technique is applied to transform the constrained actor optimization objective (for order generation) into an unconstrained form. Evaluations carried out in real-world scenarios (based on actual geographic coordinates) and synthetic scenarios using multiple types of synthetic fuel consumption datasets demonstrate that the proposed model and algorithm significantly reduce inventory violations and improve dispatch efficiency and safety under varying demand patterns, outperforming baseline methods.

Multi-agent Reinforcement Learning (MARL) has shown considerable promise in enhancing the efficiency of adaptive traffic signal control (ATSC) systems. However, existing MARL approaches primarily focus on optimizing overall traffic flow, often overlooking the issue of fairness in vehicle waiting times. Considering that there is no need to strive for the ultimate fairness, this paper models the ATSC problem as a Constrained Partially Observable Markov Game (CPOMG), where fairness is modeled as a constraint on the maximum waiting time of vehicles on lanes of intersections instead of a reward term that pursues maximization. CPOMG aims to find a cooperative control policy with optimal traffic efficiency within the constrained solution space by multiple agents. On this basis, this paper proposes a new centralized training and decentralized execution cooperative MARL method, i.e., vehicle-level fairness multi-agent proximity policy optimization (VF-MAPPO). VF-MAPPO leverages a centralized trained global Critic Network to estimate the average vehicle traffic efficiency and vehicle maximum waiting time, and an Actor Network shared by all intersections for decentralized execution, which converts the optimization problem with constraints to an unconstrained optimization objective through the Lagrange multiplier method and adopts proximity policy optimization during training. Additionally, VF-MAPPO incorporates spatial-temporal graph attention in the Critic network to efficiently extract state representations in multi-intersection environments. We qualitatively analyzed the monotonic improvement guarantee of VF-MAPPO. Extensive experimental validation across two real-world and one synthetic scenarios substantiates that VF-MAPPO enhances vehicle-level fairness and maintains average traffic efficiency, surpassing state-of-the-art methods.

Existing multi-agent reinforcement learning (MARL) in adaptive traffic signal control (ATSC) typically models cooperative control of multiple intersections as a cooperative Markov game, optimizing the average traffic efficiency of intersections with the same emphasis. However, it is insufficient to meet the requirements in real ATSC scenarios when all intersections are treated equally. To this end, this work proposes the Captain-Member Markov Game (CM-MG) that considers the different priorities between intersections. CM-MG categorizes intersections into special and ordinary intersections, controlled by captain agents and member agents to optimize the traffic efficiency of local and overall road networks, respectively. The cooperative requirements of CM-MG are achieved through a sequential decision-making principle. Captains have priority in choosing actions, and members make decisions sequentially, following the breadth-first traversal order in the road network after obtaining their precursors’ intentions. Then, a cooperative MARL algorithm, i.e, Sequential Decision Deep Graph Network (GNSD-Light), is proposed to learn the optimal joint policy that meets the learning goals of both captains and members. To be unrestricted by the scales of intersections, GNSD-Light adopts an autoregressive framework where all agents make decisions sequentially in a predetermined order by sharing the same decision model. In addition, to obtain sufficient state representation, two relative position encoding-based spatiotemporal representation modules are designed for GNSD-Light based on the characteristics of ATSC scenarios. Finally, through adequate experiments and qualitative analysis, we have confirmed that our method effectively balances traffic efficiency among both the overall road network and special intersections.

WiFi Fingerprint-based Localization (WFL) has recently achieved promising results in the bloom of deep learning techniques. Unfortunately, current studies reveal the great risks of deep-learning models when facing adversarial attacks, raising broader concerns about Deep-learning-based WiFi Fingerprint Localization Models (DFLMs). However, real-world adversarial attacks targeting DFLMs are not fully investigated, making it unclear how to counter this potential threat. In this paper, we take the first step to introduce adversarial examples into the physical world against DFLMs. Specifically, we propose a general attack method named Phy-Adv, consisting of a physical attenuation loss and a differentiable simulation module, the generated adversarial noise could be feasibly produced in the real world and make effects on DFLMs, i.e., misleading the DFLMs from the signal source end. Furthermore, aiming at countering this typical adversarial threat, we propose a Relaxant Multiple Batch Normalization (RMBN) approach, which alleviates the weak robustness of DFLMs by the data-end adaptive training-set segmenting and model-end multiple batch normalization designing. To demonstrate the de facto effectiveness of the proposed physical adversarial examples and the adversarial defense strategy, we conducted extensive experiments on 2 datasets, i.e., BHD and TUT, and multiple deep models, e.g., AlexNet, VGG, and ResNet. The experimental results strongly support that our Phy-Adv shows satisfactory adversarial attacking ability in the physical world, meanwhile, the RMBN enjoys considerable defense ability against the adversarial attacks.

Copy detection is a key task of image copyright protection. This article proposes a robust image hashing algorithm by CP decomposition and discrete cosine transform (DCT) for copy detection. The first contribution is the third-order tensor construction with low-frequency coefficients in the DCT domain. Since the low-frequency DCT coefficients contain most of the image energy, they can reflect the basic visual content of the image and are less disturbed by noise. Hence, the third-order tensor construction with the low-frequency DCT coefficients can ensure robustness of our algorithm. Another contribution is the application of the CP decomposition to the third-order tensor for learning a short binary hash. As the factor matrices learned from the CP decomposition can preserve the topology of the original tensor, the binary hash derived from the factor matrices can reach good discrimination. Lots of experiments and comparisons are done to validate effectiveness and advantage of our algorithm. The results demonstrate that our algorithm has superior classification and copy detection performances than several baseline algorithms. In addition, our algorithm is also better than some baseline algorithms with regard to hash length and computational time.