End-to-end navigation models commonly incorporate multiple sub-modules, each designed for distinct purposes such as searching, obstacle avoidance, and target localization. However, agents equipped with these modules may still struggle to apply the appropriate strategies at the right locations and stages. For instance, agent might incorrectly rely on the search or localization module for obstacle avoidance, reducing adaptability in dynamic environments. Additionally, existing methods assume the recognition for target object is always correct, neglecting the unavoidable misclassification caused by visually similar objects. To apply appropriate strategy for a given situation, we introduce Adaptive Strategy Feature Fusion (ASFF). It heuristically assigns appropriate weights to different sub-modules based on current observation and memory state, enabling flexible integration with arbitrary sub-module combinations. To improve localization in the presence of misclassification, we propose Fault Tolerant Target Memory Aggregator (FTTMA), a module that uses clustering-based sparse self-attention and target cross-attention to minimize interference from misclassified object, providing accurate target orientation to the agent. Experiments on the AI2THOR and RoboTHOR datasets, including both typical and zero-shot navigation tasks, demonstrate that our model outperforms the state-of-the-art (SOTA) methods in both success rate and navigation efficiency.

Inspired by human-like behaviors for navigation: first searching to explore unknown areas before discovering the target, and then the pathfinding of moving towards the discovered target, recent studies design parallel submodules to achieve different functions in the searching and pathfinding stages, while ignoring the differences in reward signals between the two stages. As a result, these models often cannot be fully trained or are overfitting on training scenes. Another bottleneck that restricts agents from learning two-stage strategies is spatial perception ability, since the studies used generic visual encoders without considering the depth information of navigation scenes. To release the potential of the model on strategy learning, we propose the Two-Stage Reward Mechanism (TSRM) for object navigation that decouples the searching and pathfinding behaviours in an episode, enabling the agent to explore larger area in searching stage and seek the optimal path in pathfinding stage. Also, we propose a pretraining method Depth Enhanced Masked Autoencoders (DE-MAE) that enables agent to determine explored and unexplored areas during the searching stage, locate target object and plan paths during the pathfinding stage more accurately. In addition, we propose a new metric of Searching Success weighted by Searching Path Length (SSSPL) that assesses agent’s searching ability and exploring efficiency. Finally, we evaluated our method on AI2-Thor and RoboTHOR extensively and demonstrated it can outperform the state-of-the-art (SOTA) methods in both the success rate and the navigation efficiency.

Text-video retrieval aims to establish a matching relationship between a video and its corresponding text. However, previous works have primarily focused on salient video subjects, such as humans or animals, often overlooking Low-Salient but Discriminative Objects (LSDOs) that play a critical role in understanding content. To address this limitation, we propose a novel model that enhances retrieval performance by emphasizing these overlooked elements across video and text modalities. In the video modality, our model first incorporates a feature selection module to gather video-level LSDO features, and applies cross-modal attention to assign frame-specific weights based on relevance, yielding frame-level LSDO features. In the text modality, text-level LSDO features are captured by generating multiple object prototypes in a sparse aggregation manner. Extensive experiments on benchmark datasets, including MSR-VTT, MSVD, LSMDC, and DiDeMo, demonstrate that our model achieves state-of-the-art results across various evaluation metrics.

Vision-and-Language Navigation (VLN) is a challenging task where an agent is required to navigate to a natural language described location via vision observations. The navigation abilities of the agent can be enhanced by the relations between objects, which are usually learned using internal objects or external datasets. The relationships between internal objects are modeled employing graph convolutional network (GCN) in traditional studies. However, GCN tends to be shallow, limiting its modeling ability. To address this issue, we utilize a cross attention mechanism to learn the connections between objects over a trajectory, which takes temporal continuity into account, termed as Temporal Object Relations (TOR). The external datasets have a gap with the navigation environment, leading to inaccurate modeling of relations. To avoid this problem, we construct object connections based on observations from all viewpoints in the navigational environment, which ensures complete spatial coverage and eliminates the gap, called Spatial Object Relations (SOR). Additionally, we observe that agents may repeatedly visit the same location during navigation, significantly hindering their performance. For resolving this matter, we introduce the Turning Back Penalty (TBP) loss function, which penalizes the agent’s repetitive visiting behavior, substantially reducing the navigational distance. Experimental results on the REVERIE, SOON, Touchdown and R2R datasets demonstrate the effectiveness of the proposed method.

Traditional Vision-and-Language Navigation (VLN) requires an agent to navigate to a target location solely based on visual observations, guided by natural language instructions. Compared to this task, long-instruction VLN involves longer instructions, extended trajectories, and the need to consider more contextual information for global path planning. As a result, it is more challenging and requires accurately aligning the instructions with the agent’s current visual observations, which is accompanied by two significant issues. Firstly, there is a misalignment between actions. The visual observations of the agent at each step lack explicit action-related details, while the instructions contain action-oriented words. Secondly, there is a misalignment between global instructions and local visual observations. The instructions describe the entire navigation trajectory, whereas the agent’s visual observations only provide localized information about a specific position along the trajectory. To address these issues, this article introduces the Action-Perception Alignment Framework (APAF). In this framework, we first design the Action-Contextual Encoding Module (ACEM), which enriches the agent’s visual perception by encoding potential actions with relative heading and elevation angles. We then propose the Dynamic Instruction Weighting Module (DIWM), which adjusts the importance of instruction words based on the agent’s current visual observations, emphasizing those words most relevant to the agent’s visual observations. Our approach significantly outperforms existing methods, achieving state-of-the-art results with improvements of 8.5% and 4.0% in Success Rate (SR) on the long-instruction R4R and RxR datasets, respectively.

This paper addresses the problem of reliable communications for cooperative learning on Internet-of-Vehicles, where a large amount of data from users and services needs to be processed. Previous works have proposed various cooperative learning schemes, but they often assume that the communications between vehicles are reliable, without considering how to achieve this in an Internet-of-Vehicles network. This paper is the first one that implements an abstract MAC layer using a distributed deep reinforcement learning scheme, which can directly meet the reliable communication requirements of cooperative learning in previous works. Our abstract MAC layer performs two operations:acknowledgement, which makes sure that all vehicles can successfully broadcast their messages to all of their neighbors, andprogress, which ensures that each vehicle can receive at least one message from its neighbors. These operations facilitate vehicles to exchange and update their training models in a cooperative learning service. Our simulation results show the efficiency and fairness of our deep reinforcement learning abstract MAC layer.

Vertex engagement has extraordinary significance for social resilience and network stability. There have been lots of existing work studying this fundamental problem in pairwise graphs, but in the more generalized hypergraphs, it has not been well explored, due to the great challenges of sparsity, complex connectivity and dynamicity of hypergraphs. In this work, we initialize the study of the vertex engagement problem in hypergraphs. Based on the observation that the engagement of vertices in hypergraphs needs to consider two critical parameters, group engagement and neighbor engagement, we propose a vertex engagement model integrating the merits of these two measures, called constrained core, to address the ineffectiveness and incomprehensiveness caused by just using a single engagement factor. By giving an algorithm for the constrained core decomposition, we show that the constrained core number of vertices can be computed in linear time. Furthermore, by showing a localized property of contained core, efficient maintenance algorithms for updating the constrained core number of vertices in dynamic hypergraphs are proposed, to avoid the large amount of redundant computations caused by the decomposition from scratch. Extensive experiments conducted on real-world hypergraphs well exhibit the effectiveness of our model and the efficiency of the proposed algorithms.

With the raised privacy concerns and rigorous data regulations, federated learning has become a hot collaborative learning paradigm for the recommendation model without sharing the highly sensitive POI data. However, the time-sensitive, heterogeneous, and limited POI records seriously restrict the development of federated POI recommendation. To this end, in this paper, we design the fine-grained preference-aware personalized federated POI recommendation framework, namely PrefFedPOI, under extremely sparse historical trajectories to address the above challenges. In details, PrefFedPOI extracts the fine-grained preference of current time slot by combining historical recent preferences and periodic preferences within each local client. Due to the extreme lack of POI data in some time slots, a data amount aware selective strategy is designed for model parameters uploading. Moreover, a performance enhanced clustering mechanism with reinforcement learning is proposed to capture the preference relatedness among all clients to encourage the positive knowledge sharing. Furthermore, a clustering teacher network is designed for improving efficiency by clustering guidance. Extensive experiments are conducted on two diverse real-world datasets to demonstrate the effectiveness of proposed PrefFedPOI comparing with state-of-the-arts. In particular, personalized PrefFedPOI can achieve 7% accuracy improvement on average among data-sparsity clients.

The study of graph kernels has been an important area of graph analysis, which is widely used to solve the similarity problems between graphs. Most of the existing graph kernels consider either local or global properties of the graph, and there are few studies on multiscale graph kernels. In this article, the authors propose a framework for graph kernels based on truss decomposition, which allows multiple graph kernels and even any graph comparison algorithms to compare graphs at different scales. The authors utilize this framework to derive variants of five graph kernels and compare them with the corresponding basic graph kernels on graph classification tasks. Experiments on a large number of benchmark datasets demonstrate the effectiveness and efficiency of the proposed framework.

Decentralized Federated Learning (DeFL) plays a critical role in improving effectiveness of training and has been proved to give great scope to the development of edge computing. However, on the one hand, inaccessibility of private data and excessively exploiting the data throughout the learning process have become a public concern, and on the other hand the connections between server-less edge devices are always varying due to the mobility of edge intelligent devices. To address the above issues, we propose aPrivacy-Enhanced -Dynamic -Decentralized -Federated -Learning algorithm called PED$ ^{2}$FL in a dynamic edge environment. We design the PED$ ^{2}$FL under the analog transmission scheme, where mobile edge devices transmit privacy preserving data simultaneously and accomplish efficient information aggregation with doubly-stochastic adjacent matrices. With thorough analysis, it can be demonstrated that PED$ ^{2}$FL satisfies$(\epsilon,\delta)$-differential privacy while the per-device privacy budget decays exponentially with the number of the neighbors, which greatly improved the data utility compared to the fixed budget in the orthogonal transmission strategy. PED$ ^{2}$FL has the same convergence rate$\mathcal {O}(\sqrt{\frac{1}{KN}})$as the non-private decentralized learning algorithm D-PSGD without enhanced privacy protection, where$K$and$N$are the total iterations and the number of nodes, respectively. Extensive experiments show that algorithm PED$ ^{2}$FL also performs well with real-world settings.

Today's business environment is characterized by uncertainty and competition, so the capability to adapt to the evolving era and unforeseen challenges is essential in business strategies. Recent studies on extended enterprise indicate that collaboration among different stakeholders is beneficial for surviving these unexpected changes. However, the barriers such as market uncertainty, privacy and trust concerns, and individual contribution evaluation limit the implementation and application of the extended enterprise concept. Federated learning (FL), in which multiple enterprise entities can use a shared model while retaining all training data locally, has emerged as a promising artificial intelligence (AI) solution for accumulating insights from multiple stakeholders and providing collaborative decision-making. Furthermore, the enhanced privacy-protection benefits of FL remove the barriers to implementing extended enterprise collaboration. In particular, an FL central server manages the local updates of multiple enterprise entities (FL clients) and aggregates their contributions to improve the global model training. Meanwhile, to address the time-series graph learning problem in most business environments, we incorporate temporal convolutional network, graph convolutional neural network, and gated recurrent unit architecture into FL to capture the temporal-spatial dependencies in individual data sources. Furthermore, we use traffic flow forecasting as the use case of our proposed framework to verify its effectiveness. Finally, the experimental results on a real traffic flow dataset and the comparison results with the state-of-the-art baseline methods show that our proposed solution achieves superior performance.

To realize simulation experiments in large-scale Internet of Things (IoT) networks, this work studies the utilization of deep graph generative models to generate IoT networks, which can provide an economic approach facilitating IoT to meet the requirements of real-time performance, interoperability, energy efficiency, and coexistence. In IoT, nodes have different attributes, different connection ways with surrounding nodes, and different compactness of the region, which pose great challenges for network generation. By leveraging the properties of$k$-core and variational autoencoder during network generation, we propose a variable graph autoencoder called Core-GAE incorporating the coreness of nodes. In contrast to previous graph generative models, Core-GAE can preserve the local proximity similarity and maintain the global structural features simultaneously when learning the structural features of graphs. All three of the tasks we experimented with on four data sets show that Core-GAE exhibits better performance than previous ones.

Person reidentification (ReID) is an important application of Internet of Things (IoT). ReID recognizes pedestrians across camera views at different locations and time, which is usually treated as a ranking task. An essential part of this task is the hard sample mining. Technically, two strategies could be employed, i.e., global hard mining and local hard mining. For the former, hard samples are mined within the entire training set, while for the latter, it is done in mini-batches. In literature, most existing methods operate locally. Examples include batch-hard sample mining and semihard sample mining. The reason for the rare use of global hard mining is the high computational complexity. In this article, we argue that global mining helps to find harder samples that benefit model training. To this end, this article introduces a new system to: 1) efficiently mine hard samples (positive and negative) from the entire training set and 2) effectively use them in training. Specifically, a ranking list network coupled with a multiplet loss is proposed. On the one hand, the multiplet loss makes the ranking list progressively created to avoid the time-consuming initialization. On the other hand, the multiplet loss aims to make effective use of the hard and easy samples during training. In addition, the ranking list makes it possible to globally and effectively mine hard positive and negative samples. In the experiments, we explore the performance of the global and local sample mining methods, and the effects of the semihard, the hardest, and the randomly selected samples. Finally, we demonstrate the validity of our theories using various public data sets and achieve competitive results via a quantitative evaluation.

Current state-of-the-art single shot object detection pipelines, composed by an object detector such as Yolo, generate multiple detections for each object, requiring a post-processing Non-Maxima Suppression (NMS) algorithm to remove redundant detections. However, this pipeline struggles to achieve high accuracy, particularly in object counting applications, due to a trade-off between precision and recall rates. A higher NMS threshold results in fewer detections suppressed and, consequently, in a higher recall rate, as well as lower precision and accuracy. In this paper, we have explored a new pedestrian detection pipeline which is more flexible, able to adapt to different scenarios and with improved precision and accuracy. A higher NMS threshold is used to retain all true detections and achieve a high recall rate for different scenarios, and a Pedestrian Similarity Extraction (PSE) algorithm is used to remove redundant detentions, consequently improving counting accuracy. The PSE algorithm significantly reduces the detection accuracy volatility and its dependency on NMS thresholds, improving the mean detection accuracy for different input datasets.

Person re-identification is one of the most important issues in intelligent transportation systems. Recently, the widespread availability of cameras and a growing need for public safety have increasingly motivated interest in the problem of person re-identification in multi-camera networks. The main difficulty of person re-identification arises from the variations in human pose, different viewpoint in multi-camera, cluttered background, occlusion, and low image resolution, which lead person re-identification to a challenging problem. This paper presents a method based on sparse coding for person re-identification. To apply sparse coding method, we firstly solve the problem of aligning person images, and to enhance the discrimination of dictionary, a dictionary learning model is added into our method. Experiments on benchmark dataset (CAVIARa, ETZH, i-LIDS) demonstrate that the proposed method outperforms the state-of-the-art approaches.

In this paper, we propose a novel unsupervised color spatial pyramid matching (UCSPM) approach for person re-identification. It is well motivated by our study on spatial pyramid to build effective structural object representation for person re-identification. Through the combination of illumination invariance color feature, UCSPM can well cope with the variations of viewpoint, illumination and pose. First, local superpixel regions are divided to accurately represent the color feature. Second, human body are divided into increasing fine vertical sub-regions to construct the spatial pyramid matching scheme. Third, the color feature and its spatial distribution information are used in a pyramid match kernel for calculating the similarity between person and person. The effectiveness of our approach is validated on the VIPeR dataset and CUHK campus dataset. Comparing with other approaches, our UCSPM improves the best unsupervised rank-1 matching rate on the VIPeR dataset by 3.08% with only one kind of feature—color.

The data are complexity and heterogeneous in the city-scale. When the tasks are nonlinear, existing systems cannot perform well. This paper proposes a data service oriented architecture that is based on the data vitalization theory. In this perspective, the vitalized cells are the basic units of a system, which are organized through nested and/or layer structure. A smart service platform bases on crowd intelligence network is developed according the theory of data vitalization. It allows the developers upload their own services and data to open for other developers. Then the architecture and implement technologies of the platform are introduced. At last, an application example of this architecture is given, which is used to sense social hot-spots in a period and rebuild the scene of the hot-spots. The platform and example show that the data vitalization theory achieves high scalability and flexibility for both linear and non-linear tasks.

This paper presents a new face recognition approach by using correlation analysis and ensemble classifiers based on Support Vector Machine (SVM). In this approach, image pre-processing techniques such as histogram equalization, edge detection and geometrical transformation are first used in order to improve the quality of the face images. We further employ correlation analysis method to extract features. At last, ensemble classifiers based on SVM are selected to construct the classification committee using Binary Particle Swarm Optimization (BPSO). Comparisons with other popular classification methods show that our scheme is very promising in face recognition.