The inevitable trade-off between perceptual performance and communication bandwidth in collaborative perception poses a significant challenge. To mitigate this constraint, we introduceHAFNet, a hybrid-stage collaborative perception method designed for multi-agent collaborative 3D object detection. This method initially generates dense proposal, serving as the Region of Interest (RoI). Next, all proposals are aggregated through the proposed Agent-Foreground List via RoI association. Moreover, secondary sampling is performed according to those foreground regions. Furthermore, we achieve superior feature extraction through geometric and offset encoding. Concurrently, the setting of proxy points effectively reduces the size of the collective perception messages. In the end, those features are fused and interacted to get the detection. Extensive experiments on existing DAIR-V2X and V2V4Real datasets illustrate thatHAFNetalmost keeps a minimum communication bandwidth, while it surpasses existing state-of-the-art methods in 3D object detection tasks.

With the ever-growing attention on communication security, machine learning-based network intrusion detection system (NIDS) is widely utilized to meet different security requirements. However, most of the existing methods manually extract or learn features from raw traffic, which is usually expensive, complicated, and time-consuming. Moreover, this also brings unprecedented challenges for preserving users’ privacy in the communication process, making it difficult for existing solutions to be deployed in practice due to the privacy requirements from legal policies. This paper proposes a privacy-preserving graph neural network (named NIGNN) for NIDS, which can encode the local structure and traffic features. To address the privacy issues pertaining to the application of graph representation learning, we design a privacy message-passing mechanism with formal privacy guarantees, in which sensitive information potentially contained in graph vertices will be kept private. Specifically, we design a privacy-enhancement graph representation that introduces a degree-sensitive item in vertex-based aggregation to reduce noise. Our theoretical analysis shows that NIGNN can provide a provable privacy guarantee. Extensive experiments demonstrate NIGNN's performance in maintaining a sound privacy-accuracy trade-off.

3D object detection by autonomous vehicles is integral to intelligent transportation. Existing systems often compromise essential foreground point features and local spatial interactions through random down-sampling, focusing primarily on local feature extraction. However, this neglects interactions among distant yet significant points, limiting semantic information and detection performance due to inherent point cloud data sparsity. Addressing this, our proposed Geometric Feature Aggregation and Self-Attention in a Multi-Head Transformer (GFA-SMT) architecture leverages Graph Convolutional Networks and multi-channel transformers to enhance weak semantic information of distant sparse objects. GFA-SMT comprises three modules: Distance Suppression for Local Receptive Fields (DsLRF), Geometric Feature Aggregator with Multi-head Self Attention (GFaSA), and Predicted Key-point Weighting and Refinement (PKwR). DsLRF preserves foreground features, GFaSA encodes similar features and aggregates edge features, while PKwR focuses on key-points for enhancing geometric knowledge of distant and sparse objects. Extensive experiments on KITTI, DIARV2X-I and NuScenes datasets show significant enhancements in widely used techniques, resulting in notable increases in average precision (AP) for 3D object detection: 4.08%, 5.56%, and 4.62%, respectively, on the KITTI test dataset. GFA-SMT enhances point cloud detection accuracy, particularly at medium and long distances, with minimal impact on run-time performance and model parameters.

Detecting camouflaged objects is expected to be a challenging task due to the hard-distinguihsed boundaries of targets. Although existing learning-based methods have concentrated on utilizing boundary information to enhance camouflaged object detection, the absence of boundary difficulty estimation causes them to treat all boundary regions as equal, thereby making it more challenging to distinguish high intrinsic similarity boundary regions. To address this issue, by filtering redundant information on easy boundaries, we have proposed Edge Attention Network (EANet) to extract informative boundary knowledge. Specifically, we propose an Edge-attention Guidance module to prevent misleading segmentation by extracting critical boundary features. Then, Progressive Recognition module is proposed to progressively generate boundary-informative. The experimental results on three real-world datasets have demonstrated that our EANet outperforms existing methods across all three mertrics, while maintaining low computation.

Roadside 3D object detection is crucial for vehicle infrastructure cooperation systems. Due to the distinctive placement of roadside LiDAR, the distribution patterns of roadside point clouds and vehicle-side point clouds differ. In roadside point clouds, the proportion of foreground points in each instance is lower, leading to a notable decline in accuracy when using current sampling methods because of an unguided down-sampling strategy. To address this issue, this paper proposes a point-based single-stage 3D object detector called RDSSD for 3D object detection in roadside scenes. The paper designs a class-guided sampling strategy to efficiently select foreground points associated with potential objects. Furthermore, a task-oriented candidate prediction approach is introduced to generate candidate points that accurately represent the local scene from sampled key points. The experimental results on the DAIR-V2X-I have demonstrated that our method achieves the best detection performance with minimal computational cost.

Task offloading is a widely used technology in Edge Computing (EC), which declines the makespan of user task with the aid of resourceful edge servers. How to solve the competition for computation and communication resources among tasks is a fundamental issue in task offloading. Besides, real-life user tasks often comprise multiple interdependent subtasks. Dependencies among subtasks significantly raises the complexity of task offloading, and makes it difficult to propose generalized approaches for scenarios of different size. In this paper, we study the Dependent Task Offloading (DTO) problem within both single-user single-edge and multi-user multi-edge scenario. First, we use Directed Acyclic Graph (DAG) to model dependent task, where nodes and directed edges represent the subtasks and their interdependencies respectively. Then, we propose a task scheduling method based on Graph Attention Network (GAT) and Deep Reinforcement Learning (DRL) to minimize the makespan of user tasks. More specifically, our method introduces a multi-discrete action DRL scheduler that simultaneously determines which subtask to consider and whether it should be offloaded at each step, and employs GAT to encode the graph-based state representation. To stabilize and speed up DRL scheduler training, we pretrain GAT encoder with unsupervised learning. Extensive experiments demonstrate that our proposed approach can be applied to various environments and outperforms prior methods.

Due to the unstable and complex social network environment, the sole user–item interaction data become insufficient for generating precise recommendations. However, too much emphasis on user–item interactions prevents the discovery of internal connections among them, such as trustworthy user relations. In this work, we have integrated the collaborative and the sequential relations into an end-to-end graph neural network (GNN) simultaneously and proposed a novel framework, namely multirelational collaborative filtering (MRCF), to explore the evolutional social relations. MRCF mainly consists of two components: relational GNN (RGNN) and simple dot-product attention (SDPA), where RGNN is used to capture not only the collaborative but also the sequential relationship from reliable user–item historical interactions through the graph representation, while SDPA can further concentrate on the dominated interaction sequences between users and items. Moreover, a negative sampling method based on user interest is proposed to help train our model. Extensive experiments on three real-world datasets show that the proposed model performs competitively with other state-of-the-art methods in CF.

Camouflaged object detection has been considered a challenging task due to its inherent similarity and interference from background noise. It requires accurate identification of targets that blend seamlessly with the environment at the pixel level. Although existing methods have achieved considerable success, they still face two key problems. The first one is the difficulty in removing texture noise interference and thus obtaining accurate edge and frequency domain information, leading to poor performance when dealing with complex camouflage strategies. The latter is that the fusion of multiple information obtained from auxiliary subtasks is often insufficient, leading to the introduction of new noise. In order to solve the first problem, we propose a frequency domain reconstruction module based on contrast learning, through which we can obtain high-confidence frequency domain components, thus enhancing the model’s ability to discriminate target objects. In addition, we design a frequency domain representation decoupling module for solving the second problem to align and fuse features from theRGBdomain and the reconstructed frequency domain. This allows us to obtain accurate edge information while resisting noise interference. Experimental results show that our method outperforms 12 state-of-the-art methods in three benchmark camouflaged object detection datasets. In addition, our method shows excellent performance in other downstream tasks such as polyp segmentation, surface defect detection, and transparent object detection.

Edge intelligence has emerged as a promising paradigm to accelerate DNN inference by model partitioning, which is particularly useful for intelligent scenarios that demand high accuracy and low latency. However, the dynamic nature of the edge environment and the diversity of end devices pose a significant challenge for DNN model partitioning strategies. Meanwhile, limited resources of the edge server make it difficult to manage resource allocation efficiently among multiple devices. In addition, most of the existing studies disregard the different service requirements of the DNN inference tasks, such as its high accuracy-sensitive or high latency-sensitive. To address these challenges, we propose a Multi-Compression Scale DNN Inference Acceleration (MCIA) based on cloud-edge-end collaboration. We model this problem as a mixed-integer multi-dimensional optimization problem, jointly optimizing the DNN model version choice, the partitioning choice, and the allocation of computational and bandwidth resources to maximize the tradeoff between inference accuracy and latency depending on the property of the tasks. Initially, we train multiple versions of DNN inference models with different compression scales in the cloud, and deploy them to end devices and edge server. Next, a deep reinforcement learning-based algorithm is developed for joint decision making of adaptive collaborative inference and resource allocation based on the current multi-compression scale models and the task property. Experimental results show that MCIA can adapt to heterogeneous devices and dynamic networks, and has superior performance compared with other methods.

Although collaborative 3D perception has successfully improved detection performance by sharing LIDAR information among multiple agents, its impact under adverse weather is under poor investigation. It is non-trivial to reduce the noise effect in the multi-agent system, as each agent may generate defective feature representations with aleatoric uncertainty, and such uncertainty will be further amplified in the collaborative stage due to deterministic collaboration models. To mitigate the negative effects of weather noise on the collaborative framework, we proposed a method called Co-Denoising, which incorporates a two-stage denoising approach within the intermediate collaborative framework. In our method, a sampling-based noise filtering is first performed at each agent to make a coarse denoising. Then, during the collaboration stage, the global feature representations are expanded through Bayesian neural networks to improve the robustness against environmental noise. The extensive experiments on sunny and rainy datasets have indicated the proposed collaborative perception method can significantly reduce performance degradation under adverse weather.

It is inevitable to recognize objects in adverse weather conditions where the uncertainty of contour areas is increased. Although some multi-task learning frameworks have gained from the directional supervision between boundary detection and semantic segmentation, the interaction between those two tasks is poorly investigated. Moreover, the performance of the contour detection is expected to degrade under foggy scenarios, because the auxiliary task also has no benefits from the main task. To address the potential risk in intelligent transportation systems, this paper proposes a mutual learning framework, named CE-MGN (Confidence-Enhanced Mutual Graph Network), to propagate confidence through continuous interaction between different tasks rather than only focusing on the accuracy of the main task. The CE-MGN performs an end-to-end training paradigm and jointly learns two tasks, contour detection and semantic segmentation, through pairwise confidence-enhancement mechanism. Moreover, the task interaction is converted into graph space to further relieve the information loss during the feature aggregation in Euclidean space. Such a framework is capable to improve the robustness of respective tasks because of the encouragement from its peer task. Extensive experiments show that our CE-MGN achieved mean IoU scores of 79.35% and 79.03% on CityScapes and Foggy CityScapes datasets, respectively. Besides, our models have a stable performance on different weather severity, where the performance fluctuation is less than 1%.

Most object detection methods have achieved impressive performance on several public benchmarks, instead, facing underwater detection tasks, it is challenging to detect marine targets because of the inherent illumination inhomogeneity in underwater images. Moreover, the imbalanced foreground-background proposals further aggravate the situation of capturing marine organisms. To address the problems, we analyze the deficiency of existing feature pyramid structures and propose a multi-depth and multi-breadth pyramid architecture named Refined Pyramid Grid (RPG). A Harmonizing Focal Loss (HFL) is then proposed to generalize the discrete labels in focal loss to the continuous version to improve the optimization. Experimental results on the real-world datasets have demonstrated the efficiency and reliability of the proposed framework regarding underwater object detection tasks.

The existing empirical risk minimization algorithms learn the association between inputs and labels, and face substantial difficulties when apply to different distributions because of various confounders. Causal intervention becomes a solid solution to this issue by analyzing the visual causality, instead, those approaches fail at disentangling the confounders and mediators within the causality, and bring negative effects to the prediction. In this paper, we propose a disentangled visual causal model to eliminate the effects of confounders while reserving the corresponding mediators. Specifically, confounders are considered as different objects on the image, while mediators are formulated as some critical components of the targets that contribute to a distinctive identification. Extensive experiments on coco datasets have demonstrated the superiority of our model over other state-of-the-art baselines.

Compared with cloud computing, edge computing is capable of effectively solving the high latency problem in cloud gaming. However, there are still several challenges to address for optimizing system performance. On the one hand, the unpredictable bursts of game requests can cause server overload and network congestion. On the other hand, the mobility of players makes the system highly dynamic. Although existing research has studied game fairness and latency separately to improve the Quality of Experience (QoE), a tradeoff between fairness and latency has been largely ignored. Furthermore, how to balance network and computing load is identified as another constraint during optimization. Focusing on latency, fairness, and load balance simultaneously, we propose an adaptive resource allocation strategy through deep reinforcement learning (DRL) for a dynamic gaming system. The experimental results have demonstrated that the proposed algorithm outperforms the traditional optimization methods and classical reinforcement learning algorithms in solving complex multimodal reward problems.

Due to their inherent capability in the semantic alignment of aspects and their context words, Attention and Long-Short-Term-Memory (LSTM) mechanisms are widely adopted for Aspect-Based Sentiment Classification (ABSC) tasks. Instead, it is challenging to handle long-range word dependencies on multiple entities due to the deficiency in attention mechanisms. To solve this problem, we propose a Context-Focused Aspect-Based Network to align attention before LSTM, making the model focus more on aspect-related words and ignore irrelevant words, improving the accuracy of final classification. This can either alleviate attention distraction or reinforce the text representation ability. Experiments on two benchmark datasets show that the results achieve respectable performance compared to the state-of-the-art methods available in ABSC. Our approach has the potential to improve classification accuracy by adaptively adjusting the focus on context.

Multi-modal fusion is a promising approach to boost the autonomous driving performance and has already received a large amount of attention. Meanwhile, to increase driving reliability under distinct scenarios, it is important to handle unforeseen weather events in the training dataset, which is known as an Out-Of-Distribution (OOD) problem, for autonomous driving algorithms. In this paper, we consider those two aspects and propose an end-to-end multi-modal domain-enhanced framework, namely CrossFuser, to meet the safety orientated driving requirements. CrossFuser first integrates both image and lidar modalities to generate a robust environmental representation through conjoint mapping, elastic disentanglement, and attention mechanism. Further, the perception embedding is used to calculate corresponding waypoints by a waypoint prediction network, consisting of Gate Recurrent Units (GRUs). Finally, the final control commands are calculated by low-level control functions. We conduct experiments on the Car Learning to Act (CARLA) driving simulator involving complex weather conditions under urban scenarios, the results show that CrossFuser can outperform the state of the art.

It is challenging to detect 3D objects from noise point clouds by Graph Neural Networks (GNNs), though graph-based methods have shown promising results in 3D classifications. Since strong robustness against noise is offered by hypergraph, a relative paradigm named HyperGraph Construction-Compression-Conversion (HG3C) is proposed for detecting 3D objects from noise point clouds. Our method presents the capacity of reducing graph redundancy and capturing the variances from multiple features, by pre-encoding the graph, to improve the graph representations in point clouds. A fused graph neural network is further designed to predict the shape and category of the target in converted graphs. The experiments, on both the KITTI and Nuscene, show that the proposed approach achieves leading accuracy. Our results demonstrate the potential of using the hypergraph transformation to extract and compress point cloud information from noisy point clouds.

In a complex environment, simultaneous object recognition and tracking has been one of the challenging topics in computer vision and robotics. Current approaches are usually fragile due to spurious feature matching and local convergence for pose determination. Once a failure happens, these approaches lack a mechanism to recover automatically. In this paper, data-driven unfalsified control is proposed for solving this problem in visual servoing. It recognizes a target through matching image features with a 3-D model and then tracks them through dynamic visual servoing. The features can be falsified or unfalsified by a supervisory mechanism according to their tracking performance. Supervisory visual servoing is repeated until a consensus between the model and the selected features is reached, so that model recognition and object tracking are accomplished. Experiments show the effectiveness and robustness of the proposed algorithm to deal with matching and tracking failures caused by various disturbances, such as fast motion, occlusions, and illumination variation.

This paper deals with acoustic event detection (AED), such as screams, gunshots, and explosions, in noisy environments. The main aim is to improve the detection performance under adverse conditions with a very low signal-to-noise ratio (SNR). A novel filtering method combined with an energy detector is presented. The wavelet packet transform (WPT) is first used for time-frequency representation of the acoustic signals. The proposed filter in the wavelet packet domain then uses a priori knowledge of the target event and an estimate of noise features to selectively suppress the background noise. It is in fact a content-aware band-pass filter which can automatically pass the frequency bands that are more significant in the target than in the noise. Theoretical analysis shows that the proposed filtering method is capable of enhancing the target content while suppressing the background noise for signals with a low SNR. A condition to increase the probability of correct detection is also obtained. Experiments have been carried out on a large dataset of acoustic events that are contaminated by different types of environmental noise and white noise with varying SNRs. Results show that the proposed method is more robust and better adapted to noise than ordinary energy detectors, and it can work even with an SNR as low as -15 dB. A practical system for real time processing and multi-target detection is also proposed in this work.

Advances in image quality assessment have shown the potential added value of including visual attention aspects in objective quality metrics. Numerous models of visual saliency are implemented and integrated in different quality metrics; however, their ability of improving a metric's performance in predicting perceived image quality is not fully investigated. In this paper, we conduct an exhaustive comparison of 20 state-of-the-art saliency models in the context of image quality assessment. Experimental results show that adding computational saliency is beneficial to quality prediction in general terms. However, the amount of performance gain that can be obtained by adding saliency in quality metrics highly depends on the saliency model and on the metric.

Unmanned navigation of vehicles and mobile robots can be greatly simplified by providing environmental intelligence with dispersed wireless sensors. The wireless sensors can work as active landmarks for vehicle localization and routing. However, wireless sensors are often resource scarce and require a resource-saving design. In this paper, a multiple Bloom-filter scheme is proposed to compress a global routing table for a wireless sensor. It is used as a lookup table for routing a vehicle to any destination but requires significantly less memory space and search effort. An error-expectation-based design for a multiple Bloom filter is proposed as an improvement to the conventional false-positive-rate-based design. The new design is shown to provide an equal relative error expectation for all branched paths, which ensures a better network load balance and uses less memory space. The scheme is implemented in a project for wheelchair navigation using wireless camera motes.

This paper proposes a new template model for the simulation of casting defects which are classified, according to shape, into three main types: the single defect with circular or elliptical shape, shrinkage defects with stochastic discontinuities, and the cavity or sponge shrinkage defects. For effective simulation, different nesting stencil plates are designed to reflect the characteristics of different casting defects. These include intensity, orientation, size, and shape. The proposed approach also uses geometric diffusion to demonstrate the production of simulated defects with effective shading and contrast when compared to their background. In order to evaluate the effectiveness of the proposed approach, the simulated casting defects are superposed on real radioscopic images of casting pieces and compared with real defects by extensive visual inspection. On the other hand, in order to verify the similarity of the simulated defects and the real defects, we have used our defect inspection algorithm to recognize both real and simulated defects in the same image. The experimental results show that the proposed defect simulation approach can produce a large range of simulated casting defects, which can be utilized as sample images to tune the parameters of casting inspection algorithms.

Environment Intelligence is becoming ubiquitous. Wireless sensor networks supported environment intelligence is providing an opportunity to service robot navigation to reduce the complexity of conventional centralized and on-board map-building, localization, path-planning and motion control, whilst superior performance can be expected. In terms of robot path planning in a dynamic environment, distributed environment intelligence can take into account both global and local perceptions for path generation and adaptation, which results in better predictability and more prompt reaction. This paper proposes a snake based and distributed path planning algorithm for robot navigation in an intelligent environment with distributed wireless visual sensors. Via communication links between sensors, segments of a path, as an elastic band from start position to goal position, interact each other to react to repulsive forces from obstacles whilst maintain compliance. However, the compliance has to be subject to the robot kinematic constraints and the elastic band may change to rigid or even to a broken state. A state machine is then presented to manage the state switch and control over the network. Simulations and experiments showed that the proposed distributed snake scheme can adapt to dynamic changes in the environment and satisfy the kinematic curvature constraints for the whole path.

This paper presents a discrete learning controller for vision-guided robot trajectory imitation with no prior knowledge of the camera-robot model. A teacher demonstrates a desired movement in front of a camera, and then, the robot is tasked to replay it by repetitive tracking. The imitation procedure is considered as a discrete tracking control problem in the image plane, with an unknown and time-varying image Jacobian matrix. Instead of updating the control signal directly, as is usually done in iterative learning control (ILC), a series of neural networks are used to approximate the unknown Jacobian matrix around every sample point in the demonstrated trajectory, and the time-varying weights of local neural networks are identified through repetitive tracking, i.e., indirect ILC. This makes repetitive segmented training possible, and a segmented training strategy is presented to retain the training trajectories solely within the effective region for neural network approximation. However, a singularity problem may occur if an unmodified neural-network-based Jacobian estimation is used to calculate the robot end-effector velocity. A new weight modification algorithm is proposed which ensures invertibility of the estimation, thus circumventing the problem. Stability is further discussed, and the relationship between the approximation capability of the neural network and the tracking accuracy is obtained. Simulations and experiments are carried out to illustrate the validity of the proposed controller for trajectory imitation of robot manipulators with unknown time-varying Jacobian matrices.

In a virtual enterprise, how to find the best expert to share their experience and information is an important way to improve work efficiency. Semantic matching provides a basic technique to solve this problem. In this paper, semantic matching is measured by a semantic attractive force model, which is based on a domain ontology, and semantic distance computation is a core problem in semantic matching. The depth of ontology tree is considered when computing semantic distance then an agent could account semantic attractive force of optional information and chooses the best matching information to share with each other

The tristability that a class of two-dimensional neuron models may exhibit is investigated in this paper. One proposition and three corollaries on saddles, which play an important role in the dynamics of these models, are proposed and proved using the properties of the Poincare index. Two combinations with three stable states are attained numerically and analyzed geometrically based on INa, p+ IK-model. They are used to interpret visual illusions, and their physiological potentialities and relationship with the short-term memory are discussed.

Today, complex product development requires the involvement of more and more components and actors. Virtual enterprises are alliances which are often temporary and distributed at different locations. One essential aspect that needs to be considered is that of product configuration management. This paper proposes a distributed management system to make the organization and cooperation of such product development alliances more flexible and changeable. Actors can advertise their own skill and knowledge and seek for partners to form alliances. A multi-agent system can then be constructed to realize dynamic collaborative work during product life-cycles.

This paper presents an iterative learning scheme for vision-guided robot trajectory tracking. First, a stability criterion for designing iterative learning controller is proposed. It can be used for a system with initial resetting error. By using the criterion, one can convert the design problem into finding a positive definite discrete matrix kernel and a more general form of learning control can be obtained. Then, a three-dimensional (3D) trajectory tracking system with a single static camera to realize robot movement imitation is presented based on this criterion.

Concerns the time optimal motion planning problem under kinematic and dynamic constraints for a 2-DOF wheeled mobile robot (WMR). The dynamic model of a WMR is derived using a Newton-Euler method and its constraints are analyzed. Kinematic constraints are imposed by its nonholonomy and structural limits while dynamic constraints are due to motor saturation. The motion planning problem is formulated as two stage planning. First, path planning under kinematic constraints is transformed into a pure geometric problem. The shortest path composed of circular arcs and straight lines is obtained. Then, combined with dynamic characteristics of the WMR, a time optimal velocity profile is generated under dynamic constraints. Since constraints of a WMR are fully exploited, the proposed method is simple and effective for motion planning. Simulation results illustrate the capability of the planning scheme.