Existing Human Motion Prediction (HMP) methods based on RGB(D) cameras are sensitive to lighting conditions and raise privacy concerns, limiting their real-world applications such as firefighting and elderly care. Motivated by the robustness and privacy-preserving nature of millimeter-wave (mmWave) radar, this work introduces radar as a novel sensing modality for HMP for the first time. Nevertheless, radar signals often suffer from specular reflections and multipath effects, resulting in noisy and temporally inconsistent measurements, such as body-part miss-detection. To address these radar-specific artifacts, we propose mmPred, the first diffusion-based framework tailored for radar-based HMP. mmPred introduces a dual-domain historical motion representation to guide the generation process, combining a Time-domain Pose Refinement (TPR) branch for fine-grained details and a Frequency-domain Dominant Motion (FDM) branch for capturing global motion trends and suppressing frame-level inconsistency. Furthermore, we design a Global Skeleton-relational Transformer (GST) as the diffusion backbone to model global inter-joint cooperation, enabling corrupted joints to dynamically aggregate information from others. Extensive experiments show that mmPred achieves state-of-the-art performance, outperforming existing methods by 8.6% on mmBody and 22% on mm-Fi.

Monocular 3D Semantic Scene Completion (SSC) is a challenging yet promising task that aims to infer dense geometric and semantic descriptions of a scene from a single image. While recent object-centric paradigms significantly improve efficiency by leveraging flexible 3D Gaussian primitives, they still rely heavily on a large number of randomly initialized primitives, which inevitably leads to 1) inefficient primitive initialization and 2) outlier primitives that introduce erroneous artifacts. In this paper, we propose SplatSSC, a novel framework that resolves these limitations with a depth-guided initialization strategy and a principled Gaussian aggregator. Instead of random initialization, SplatSSC utilizes a dedicated depth branch composed of a Group-wise Multi-scale Fusion (GMF) module, which integrates multi-scale image and depth features to generate a sparse yet representative set of initial Gaussian primitives. To mitigate noise from outlier primitives, we develop the Decoupled Gaussian Aggregator (DGA), which enhances robustness by decomposing geometric and semantic predictions during the Gaussian-to-voxel splatting process. Complemented with a specialized Probability Scale Loss, our method achieves state-of-the-art performance on the Occ-ScanNet dataset, outperforming prior approaches by over 6.3% in IoU and 4.1% in mIoU, while reducing both latency and memory cost by more than 9.3%.

Understanding complex human activities demands the ability to decompose motion into fine-grained, semantic-aligned sub-actions. This motion grounding process is crucial for behavior analysis, embodied AI and virtual reality. Yet, most existing methods rely on dense supervision with predefined action classes, which are infeasible in open-vocabulary, real-world settings. In this paper, we propose ZOMG, a zero-shot, open-vocabulary framework that segments motion sequences into semantically meaningful sub-actions without requiring any annotations or fine-tuning. Technically, ZOMG integrates (1) language semantic partition, which leverages large language models to decompose instructions into ordered sub-action units, and (2) soft masking optimization, which learns instance-specific temporal masks to focus on frames critical to sub-actions, while maintaining intra-segment continuity and enforcing inter-segment separation, all without altering the pretrained encoder. Experiments on three motion-language datasets demonstrate state-of-the-art effectiveness and efficiency of motion grounding performance, outperforming prior methods by 8.7% mAP on HumanML3D benchmark. Meanwhile, significant improvements also exist in downstream retrieval, establishing a new paradigm for annotation-free motion understanding.

Embodied intelligence and related disciplines have identified several mechanisms that help embodied agents learn how to solve complex problems. Reinforcement learning (RL) is one of the most promising computational approaches toward enhancement of the learning-based problem-solving abilities of such agents. Given the recent rapid evolution of artificial intelligence, RL has become a keystone technology, accelerating scientific discoveries and also finding applications in many other domains. In RL, an agent collects data when interacting with the environment, which optimizes a policy ensuring a higher return. Further improvement requires more exploration of the action space. However, not all actions in that space are safe and acceptable. The exploration of an agent must be constrained. In this work, a novel mirror descent safe policy optimization (MDSPO) algorithm is proposed to ensure the safety of an RL agent. The algorithm leverages mirror descent optimization to maximize the return while satisfying the safety constraint. A novel optimization objective is formulated, and an innovative three-stage optimization strategy is employed-comprising gradient descent without the cost constraint, projection onto the nonparametric policy space with the cost constraint, and projection onto the parametric policy space. Compared to previous methods, MDSPO is a simple and easy to implement first-order approach, which does not impose a hard constraint on the trust region. Theoretical analysis of the MDSPO reveals a lower bound on return improvement and an upper bound on constraint violation at the time of each policy update. The numerical results obtained from two sets of different constrained locomotive experiments demonstrate that MDSPO improves the average return by about 12% and better satisfies the cost constraints than other state-of-the-art methods do. In a real-world obstacle avoidance experiment using an unmanned surface vessel, MDSPO both finds the optimal path and guarantees agent safety.

Time-Series (TS) data has grown in importance with the rise of Internet of Things devices like sensors, but its labeling remains costly and complex. While Unsupervised Domain Adaptation (UDAs) offers an effective solution, growing data privacy concerns have led to the development of Source-Free UDA (SFUDAs), enabling model adaptation to target domains without accessing source data. Despite their potential, applying existing SFUDAs to TS data is challenging due to the difficulty of transferring temporal dependencies-an essential characteristic of TS data-particularly in the absence of source samples. Although prior works attempt to address this by specific source pretraining designs, such requirements are often impractical, as source data owners cannot be expected to adhere to particular pretraining schemes. To address this, we propose Temporal Source Recovery (TemSR), a framework that leverages the intrinsic properties of TS data to generate a source-like domain and recover source temporal dependencies. With this domain, TemSR enables dependency transfer to the target domain without accessing source data or relying on source-specific designs, thereby facilitating effective and practical TS-SFUDA. TemSR features a masking-recovery-optimization process to generate a source-like distribution with restored temporal dependencies. This distribution is further refined through local context-aware regularization to preserve local dependencies, and anchor-based recovery diversity maximization to promote distributional diversity. Together, these components enable effective temporal dependency recovery and facilitate transfer across domains using standard UDA techniques. Extensive experiments across seven TS tasks demonstrate the effectiveness of TemSR, which even surpasses existing TS-SFUDA methods that require source-specific designs.

Multi-session SLAM is essential for long-term robotic operations in indoor environments such as warehouses, office buildings, and industrial facilities. However, the thin walls separating enclosed spaces in such environments introduce a challenge known as the double-sided issue, where point clouds from opposite sides are mistakenly associated as a single surface during single-session mapping, and are prone to being grouped into the same voxel during voxelization in multi-session map fusion, leading to poor voxel planarity, which causes voxel invalidation and reduces the available constraints for global optimization. To address this, we propose NVMS-SLAM, a normal vector-based multi-session LiDAR SLAM system tailored for indoor environments. For single-session mapping, an extended voxel map is designed to preserve normal vector information and to distinguish between primary and secondary surfaces, thereby improving data association. At the multi-session level, a density-encoded indoor scan-context descriptor is introduced for robust loop closure. In addition, a two-stage global map fusion strategy is adopted, combining joint pose graph optimization and normal vector-based bundle adjustment to ensure globally consistent mapping. Experiments on simulated datasets and real-world environments demonstrate that NVMS-SLAM can effectively resolve the double-sided issue at both the single-session and multi-session stages.

This article addresses the predefined-time optimal motion control problem of an autonomous surface vehicle (ASV) with disturbances under dynamic self-triggered frameworks via reinforcement learning (RL). Initially, to eliminate the influence of disturbance on the ASV, a predefined-time second-order integral sliding mode control (SOISM) strategy is formulated by establishing a novel integral sliding mode (ISM) function and a terminal sliding mode function. Subsequently, a predefined-time approximate optimal motion (AOM) control strategy is further developed to ensure the ASV maintains a stable state. Furthermore, a single critic network is used to obtain an approximate solution of the Hamilton-Jacobi-Bellman (HJB) equation. The above two strategies are established under the dynamic self-triggered framework, which relies on the current information to predict the next updating time, effectively reducing the computational and communication burden while avoiding the continuous monitoring of the ASV state. In the theoretical analysis, the main challenges lie in the design of Lyapunov functions and triggered conditions to ensure the stability of the sliding mode dynamics and the disturbed ASV. By applying the Lyapunov stability principle and designing two novel Lyapunov functions and triggered conditions that both contain dynamic variables, we demonstrate that the developed control strategies can ensure the stability within the specified time frame. Ultimately, simulation results verify the efficacy of the proposed motion control approach.

Distributed optimization, as a key technology for collaborative intelligence in multiagent systems, has been widely applied in sensor networks, deep learning, and smart grids. Although numerous effective algorithms have been proposed, classical methods typically rely on idealized assumptions, such as accurate objective information, perfect communication channels, and trustworthy system environments. However, these assumptions are frequently violated in real-world applications. To bridge the gap between theory and practice, distributed optimization under information constraints has emerged as a research focus. This survey provides a systematic overview of recent advances in this field. We categorize information constraints based on their origin into three primary types: i) observational constraints, including stochastic objectives, online optimization, and zeroth-order methods; ii) communication constraints, such as random network topologies, delays, asynchronous updates, and communication-efficient strategies; and iii) system-level constraints, encompassing privacy preservation and Byzantine-resilient optimization. This survey reviews the research progress and challenges associated with each constraint category. Furthermore, we use two representative case studies to analyze the practical application of these algorithms and the origins of information constraints in real-world problems. Finally, we explore promising future research directions.

Game theory has emerged as a fundamental framework for modeling and analyzing strategic interactions and decision-making among multiple agents, and has witnessed rapidly growing impact in cyber–physical systems over the past decade. Its integration with dynamic systems has driven major theoretical and technological advances in a wide range of applications, including smart grids, autonomous driving, robotic swarms, and networked control systems. In particular, distributed games in dynamic systems and their equilibrium learning mechanisms have attracted increasing attention due to their scalability, lightweight information exchange, and real-time implementability. This article provides a comprehensive survey of distributed games in dynamic systems, where agents interact only with local neighbors while collectively achieving global equilibrium and stability. First, the foundational theories of distributed dynamic games under three representative classes of systems: linear dynamic systems, nonlinear dynamic systems, and uncertain dynamic systems, are presented. Then, state-of-the-art distributed equilibrium learning and control methods are reviewed, including gradient-based dynamics, payoff-based learning, best-response dynamics, and learning-based approaches. To demonstrate the practical relevance and impact of distributed games in dynamic systems, representative application domains are discussed in detail. Finally, several promising future research directions are outlined, highlighting open challenges at the intersection of distributed games, learning, and dynamic systems.

Tiered Mixed Autonomy (TMA) represents a transformative transportation paradigm where autonomous vehicles (AVs) with varying intelligence levels interact dynamically with human-driven vehicles (HVs) under asymmetric sensing, communication constraints, and task objectives. Unlike conventional autonomy systems, TMA exhibits multidimensional heterogeneity across autonomy tiers, introducing unprecedented challenges in coordination and resilience. However, existing decision-making frameworks fail to resolve coordination complexity and systemic fragility in TMA, particularly in handling noise-induced vulnerability within partially observable environments. Here we propose a novel resilient cluster-based decision-making framework for asymmetrical noisy TMA. First, a Cluster-based Noisy Partially Observable Markov Decision Process (CNMDP) formally characterizes multilayered interactions and asymmetrical observation uncertainties among heterogeneous agents. Additionally, a cluster-graph representation models intra-cluster spatiotemporal dynamics and resolves hierarchical inter-cluster dependencies. Finally, the Resilient Q-Nexus Engine (RQNE) enhances decision robustness via a noise-aware weighting mechanism and a Huber loss function, ensuring stable convergence under dynamic disturbances. Experimental results demonstrate comprehensive performance advantages and superior resilience. Notably, under 12% noise variance, the framework exhibits only 4.34% performance degradation while maintaining 91.486% inter-cluster coordination efficiency. These findings pave the way for deploying resilient TMA systems in real-world dynamic traffic networks, encompassing urban street grids and highway corridors with merging lanes, on-ramps, off-ramps and varying traffic densities.

The rapid expansion of the Internet of Things (IoT) has introduced new challenges in Human Activity Recognition (HAR), particularly in dynamic environments where new and unforeseen activities emerge. Traditional HAR models, relying on predefined labels, struggle to adapt to these scenarios, highlighting the need for zero-shot learning (ZSL) approaches that can generalize beyond fixed training categories. Recent advances in large language models (LLMs) have demonstrated remarkable zero-shot capability in textual and visual domains. However, extending this ability to IoT sensors is substantially more challenging due to their heterogeneous modalities, diverse data structures, and limited semantic annotations. In this paper, we propose TENT (IoT-sEnsorslanguage alignmEnt pre-Training), a novel framework that constructs a unified sensor-language semantic space for zero-shot HAR. Instead of aligning each sensor individually to text, TENT jointly aligns multiple heterogeneous modalities with language, treating them as peers rather than anchors. This balanced multi-modal alignment allows sensors to mutually regularize one another while being grounded in linguistic semantics, transforming heterogeneity from a barrier into a strength. To further enrich the semantic space, TENT incorporates detailed activity descriptions and learnable prompts, enhancing adaptability to unseen activities. Extensive experiments across datasets and evaluation protocols demonstrate that TENT not only achieves robust recognition of both seen and unseen activities but also significantly outperforms existing vision-language and sensor-language baselines, surpassing them by over 20% on zero-shot HAR tasks. These results establish TENT as a new paradigm for generalizable IoT representation learning.

In this paper, we propose a third-order, i.e., white-noise-on-jerk, Gaussian Process (GP) Trajectory Representation (TR) framework for continuous-time (CT) motion estimation (ME) tasks. Our framework features a unified trajectory representation that encapsulates the kinematic models of both SO(3)$times$R3and SE(3) pose representations. This encapsulation strategy allows users to use the same implementation of measurement-based factors for either choice of pose representation, which facilitates experimentation and comparison to make a better choice for the ME task. In addition, unique to our framework, we derive the kinematic models with theclosed-form temporal derivatives of the local variables ofSO(3) and SE(3), which so far has only been approximated based on Taylor expansion in the literature. Our experiments show that these kinematic models can improve the estimation accuracy in high-speed scenarios. All analytical Jacobians of the interpolated states with respect to the support states of the trajectory representation, as well as the motion prior factors, are also provided for accelerated Gauss-Newton (GN) optimization. Our experiments demonstrate the efficacy and efficiency of the framework in various motion estimation tasks such as localization, calibration, and odometry, facilitating fast prototyping for ME researchers. We release the source code for the benefit of the community. Our project is available athttps://github.com/brytsknguyen/gptr.

Multi-robot navigation in complex environments relies on inter-robot communication and mutual observation for situational awareness. This paper studies the multi-robot navigation problem in unknown environments with line-ofsight (LoS) connectivity constraints. While previous works are limited to known environment models to derive the LoS constraints between robots, this paper eliminates such requirements by directly formulating the LoS constraints from realtime LiDAR scans, adopting techniques in point cloud visibility analysis. Based on that, we propose a novel LoS-distance metric to quantify both the urgency and sensitivity of losing LoS between robots considering their potential movements. Moreover, to address the imbalanced urgency of losing LoS between two robots, we design a fusion function to capture the overall urgency while generating gradients that facilitate robots' collaborative behavior to maintain LoS. The team connectivity is guaranteed by encoding the LoS constraints into a potential function that preserves the positivity of the Fiedler eigenvalue of robots' underlying graph. Finally, we establish a LoS-constrained exploration framework integrating the proposed connectivity controller. We showcase its applications in multi-robot exploration in complex unknown environments, where robots can always maintain the LoS connectivity through distributed sensing and communication while collaboratively exploring unknown environments. Our implementations are available at https://github.com/bairuofei/LoS_constrained_navigation.

Robot navigation in dense human crowds poses a significant challenge due to the complexity of human behavior in dynamic and obstacle-rich environments. In this work, we propose a dynamic weight adjustment scheme using a neural network to predict the optimal weights of objectives in an optimization-based motion planner. We adopt a spatial-temporal trajectory planner and incorporate diverse objectives to achieve a balance among safety, efficiency, and goal achievement in complex and dynamic environments. We design the network structure, observation encoding, and reward function to effectively train the policy network using reinforcement learning, allowing the robot to adapt its behavior in real time based on environmental and pedestrian information. Simulation results show improved safety compared to the fixed-weight planner and the state-of-the-art learning-based methods, and verify the ability of the learned policy to adaptively adjust the weights based on the observed situations. The feasibility of the approach is demonstrated in a navigation task using an autonomous delivery robot across a crowded corridor over a 300 m distance. Video: https://youtu.be/nSCbNaaF_VM

Multi-axle autonomous mobile robots (AMRs) are set to revolutionize the future of robotics in logistics. As the backbone of next-generation solutions, these robots face a critical challenge: managing and minimizing swept volume during turns while maintaining precise control. Traditional systems designed for standard vehicles often struggle with the complex dynamics of multi-axle configurations, leading to inefficiency and increased safety risk in confined spaces. Our innovative framework overcomes these limitations by combining swept volume minimization with Signed Distance Field (SDF) path planning and model predictive control (MPC) for independent wheel steering. This approach not only plans paths with an awareness of the swept volume, but actively minimizes it in real-time, allowing each axle to follow a precise trajectory while significantly reducing the space the vehicle occupies. By predicting future states and adjusting the turning radius of each wheel, our method enhances both maneuverability and safety, even in the most constrained environments. Unlike previous works, our solution goes beyond basic path calculation and tracking, offering real-time path optimization with minimal swept volume and efficient individual axle control. To our knowledge, this is the first comprehensive approach to tackle these challenges, delivering life-saving improvements in control, efficiency, and safety for multi-axle AMRs. Furthermore, we will open-source our work to foster collaboration and enable others to advance safer and more efficient autonomous systems.

Helmet-mounted wearable positioning systems are crucial for enhancing safety and facilitating coordination in industrial, construction, and emergency rescue environments. These systems, including LiDAR-Inertial Odometry (LIO) and Visual-Inertial Odometry (VIO), often face challenges in localization due to adverse environmental conditions such as dust, smoke, and limited visual features. To address these limitations, we propose a novel head-mounted Inertial Measurement Unit (IMU) dataset with ground truth, aimed at advancing data-driven IMU pose estimation. Our dataset captures human head motion patterns using a helmet-mounted system, with data from ten participants performing various activities. We explore the application of neural networks, specifically Long Short-Term Memory (LSTM) and Transformer networks, to correct IMU biases and improve localization accuracy. Additionally, we evaluate the performance of these methods across different IMU data window dimensions, motion patterns, and sensor types. We release a publicly available dataset, demonstrate the feasibility of advanced neural network approaches for helmet-based localization, and provide evaluation metrics to establish a baseline for future studies in this field. Data and code can be found at https://lqiutong.github.io/HelmetPoser.github.io/.

Humans learn from observations and experiences to adjust their behaviours towards better performance. Interacting with such dynamic humans is challenging, as the robot needs to predict the humans accurately for safe and efficient operations. Long-term interactions with dynamic humans have not been extensively studied by prior works. We propose an adaptive human prediction model based on the Theory-of-Mind (ToM), a fundamental social-cognitive ability that enables humans to infer others' behaviours and intentions. We formulate the human internal belief about others using a game-theoretic model, which predicts the future motions of all agents in a navigation scenario. To estimate an evolving belief, we use an Unscented Kalman Filter to update the behavioural parameters in the human internal model. Our formulation provides unique interpretability to dynamic human behaviours by inferring how the human predicts the robot. We demonstrate through longterm experiments in both simulations and real-world settings that our prediction effectively promotes safety and efficiency in downstream robot planning. Code will be available at https://github.com/centiLinda/AToM-human-prediction.git.

While UWB-based methods can achieve high localization accuracy in small-scale areas, their accuracy and reliability are significantly challenged in large-scale environments. In this paper, we propose a learning-based framework named ULOC for Ultra-Wideband (UWB) based localization in such complex, large-scale environments. First, anchors are deployed in the environment without knowledge of their actual position. Then, UWB observations are collected when the vehicle travels in the environment. At the same time, map-consistent pose estimates are developed from registering onboard self-localization data (from VIO, LIO, and other SLAM methods) with the prior map to provide the training labels. We then propose a network based on MAMBA that learns the ranging patterns of UWBs over a complex, large-scale environment. The experiment demonstrates that our solution can ensure high localization accuracy on a large scale compared to the state-of-the-art. We release our source code to benefit the community at https://github.com/brytsknguyen/uloc.

Ultra-wideband (UWB) is gaining popularity with devices like AirTags for precise home item localization but faces significant challenges when scaled to large environments like seaports. The main challenges are calibration and localization under obstructed conditions, which are common in logistics environments. Traditional calibration methods, dependent on line-of-sight (LoS), are slow, costly, and unreliable in seaports and warehouses, making large-scale localization a significant pain point in the industry. To overcome these challenges, we propose a one-shot calibration and localization framework based on UWB-LiDAR fusion. Our method uses Gaussian processes to estimate the anchor position from continuous-time LiDAR Inertial Odometry with sampled UWB ranges. This approach ensures accurate and reliable calibration with only one round of sampling in large-scale areas, i.e.,$600 \times 450 ~\mathrm{m}^{2}$. With LoS issues, UWB-only localization can be problematic, even when anchor positions are known. We demonstrate that by applying a UWB-range filter, the search range for LiDAR loop closure descriptors is significantly reduced, improving both accuracy and speed. This concept can be applied to other loop closure detection methods, enabling cost-effective localization in large-scale warehouses and seaports. It significantly improves precision in challenging environments where the UWB-only and LiDAR-Inertial methods fail, as shown in the video https://https://youtu.be/oY8jQKdM7lU. We will open-source our datasets and calibration codes for community use.

Conventional single LiDAR systems are inherently constrained by their limited field of view (FoV), leading to blind spots and incomplete environmental awareness, particularly on robotic platforms with strict payload limitations. Integrating a motorized LiDAR offers a practical solution by significantly expanding the sensor’s FoV and enabling adaptive panoramic 3D sensing. However, the high-frequency vibrations of the quadruped robot introduce calibration challenges: these oscillations continually disturb the LiDAR–motor extrinsics, so parameters calibrated once may drift during operation and degrade sensing accuracy.Existing calibration methods that use artificial targets or dense feature extraction lack feasibility for on-site applications and real-time implementation. To overcome these limitations, we propose LiMo-Calib, an efficient on-site calibration method that eliminates the need for external targets by leveraging geometric features directly from raw LiDAR scans. LiMo-Calib optimizes feature selection based on normal distribution to accelerate convergence while maintaining accuracy and incorporates a reweighting mechanism that evaluates local plane fitting quality to enhance robustness. We integrate and validate the proposed method on a motorized LiDAR system mounted on a quadruped robot, demonstrating significant improvements in calibration efficiency and 3D sensing accuracy, making LiMo-Calib well-suited for real-world robotic applications. We further demonstrate the accuracy improvements of the Lidar Inertial Odometry (LIO) on the panoramic 3D sensing system using the calibrated parameters. The code will be available at: https://github.com/kafeiyin00/LiMo-Calib.

Traditional unmanned aerial vehicle (UAV) swarm missions rely heavily on expensive custom-made drones with onboard perception or external positioning systems, limiting their widespread adoption in research and education. To address this issue, we propose AirSwarm. AirSwarm democratizes multi-drone coordination using low-cost commercially available drones such as Tello or Anafi, enabling affordable swarm aerial robotics research and education. Key innovations include a hierarchical control architecture for reliable multi-UAV coordination, an infrastructure-free visual SLAM system for precise localization without external motion capture, and a ROS-based software framework for simplified swarm development. Experiments demonstrate cm-level tracking accuracy, low-latency control, communication failure resistance, formation flight, and trajectory tracking. By reducing financial and technical barriers, AirSwarm makes multi-robot education and research more accessible. The complete instructions and open source code will be available at https://github.com/vvEverett/tello_ros.

In this paper, the model-free dynamic event-driven safe (MFDEDS) control of unknown nonaffine systems with state and input constraints is investigated via adaptive dynamic programming. To begin with, by introducing a dynamic compensator and performing system transformation, the safe control problem with state and input constraints is transformed into an optimal regulation problem of an unconstrained system. Afterwards, an integral reinforcement learning algorithm is applied to the unconstrained system to derive an optimal safe control policy independent of the original system model, which achieves model-free approximate optimal control for the original system. To conserve computing and communication resources, a novel game-based dynamic event-driven mechanism is established, which models the control policy and the event-driven error as players in a zero-sum game, with the aim of obtaining the worst event-driven error to maximize the triggering interval. Furthermore, an approximate solution to the Hamilton-Jacobi-Bellman equation is derived by constructing a single-critic learning structure, which results in an approximate optimal safe control policy. Theoretical analysis demonstrates that the proposed MFDEDS control scheme ensures the closed-loop system is asymptotically stable. Ultimately, the efficacy of the developed approach is corroborated through two simulation examples.

Angle-constrained formation control has garnered significant attention owing to the advantage of interedge angles invariant under translation, rotation, and scaling. However, most existing approaches addressing this problem are applicable only to single- or double-integrator dynamics, which are often impractical in real-world scenarios. In this article, an angle rigidity-based adaptive formation control framework is introduced for nonlinear multiagent systems subject to mismatched uncertainties. The proposed control framework integrates a prescribed performance control approach with a recursive backstepping procedure, offering several key advantages: the capability to handle unmatched system uncertainties, the preservation of angle rigidity throughout the formation process, and the assurance that the triangulated formation shape is asymptotically achieved without risking collisions between neighboring agents. Furthermore, since the control input of each agent only requires local information related to its neighbors, which can be obtained locally from its own sensors, the proposed control method can be deployed in a communication-free environment. The effectiveness of the proposed control algorithms is validated by extensive numerical simulation.

Most existing prescribed performance formation control methods impose performance requirements on the consensus error rather than directly on the relative states between agents, which limits the physical interpretability of their solutions. This article proposes a novel adaptive prescribed performance formation control strategy that ensures prescribed performance of relative errors in uncertain high-order multiagent systems under both directed and undirected graphs. Since performance constraints are considered for relative errors, the error dynamics involve a coupled nonlinear interaction term that contains global graphical information among agents, making the design of a fully distributed control strategy more challenging. By proposing a series of nonlinear mappings and utilizing the edge Laplacian along with Lyapunov stability theory, the presented formation control scheme offers several advantages over existing approaches. Different performance requirements can be accommodated in a unified manner by solely tuning the design parameters a priori, eliminating the need for control redesign and stability reanalysis under the proposed fixed control protocol. This enhances user-friendliness and reduces implementation complexity. Furthermore, the verification process for the initial constraint, which is often complex and burdensome in existing prescribed performance control methods, is entirely avoided when the performance requirements are global. Additionally, the proposed approach fully decouples nonlinear interactions and ensures the asymptotic stability of the formation manifold through an adaptive parameter estimation technique. The effectiveness of the theoretical results is demonstrated through simulations.

The 3-D point cloud has been widely used in many mobile application scenarios, including autonomous driving and 3-D sensing on mobile devices. However, existing 3-D point cloud models tend to be large and cumbersome, making them hard to deploy on edged devices due to their high memory requirements and nonreal-time latency. There has been a lack of research on how to compress 3-D point cloud models into lightweight models. In this article, we propose a method called T3DNet (tiny 3-D network with augmentation and distillation) to address this issue. We find that the tiny model after network augmentation is much easier for a teacher to distill. Instead of gradually reducing the parameters through techniques, such as pruning or quantization, we predefine a tiny model and improve its performance through auxiliary supervision from augmented networks and the original model. We evaluate our method on several public datasets, including ModelNet40, ShapeNet, and ScanObjectNN. Our method can achieve high compression rates without significant accuracy sacrifice, achieving state-of-the-art performances on three datasets against existing methods. Amazingly, our T3DNet is 58 smaller and 54 faster than the original model yet with only 1.4 accuracy descent on the ModelNet40 dataset. Our code is available at https://github.com/Zhiyuan002/T3DNet.

The modeling problem of converter-based multibus direct current (DC) microgrids (MGs) and the conflict between voltage regulation and current balancing in such MGs have been a hot topic of interest. Voltage regulation is essential for ensuring the stability and power quality of MGs, while current sharing is a reflection of the MGs' ability to coordinate power and is critical to extend the lifespan of the generation units. However, due to the presence of line impedance, currents no longer have the freedom of regulation under consistent voltages across the buses. Additionally, existing models have failed to strike a good balance between accuracy and simplicity in describing DC MGs, resulting in rare research on model-based secondary control. With this in mind, this article develops a DC MG error model containing the dynamics of both the circuit and inner control loops via the fully actuated system theory. Further, a distributed optimal control is proposed based on this model. Compared to existing studies, the suggested error model captures the power characteristics of MGs while possesses a simple structure. For regulation tasks of voltage recovery and precise current allocation, this article unifies these two into a single integrated regulation error, offering a novel approach to address their conflict. Subsequently, the stability of the closed-loop MG system is given. Furthermore, this article includes a consensus analysis of current sharing and a tracking analysis of the average voltages. Finally, a laboratory-scale MG prototype equipped with photovoltaics and batteries is developed to validate the effectiveness of the proposed method.

Large-scale LiDAR Bundle Adjustment (LBA) to refine sensor orientation and point cloud accuracy simultaneously for building navigation maps is a fundamental task in logistics, intelligent transportation, and robotics. In the context of autonomous delivery and smart mobility, the 3D map obtained by accurate and robust LBA plays a pivotal role in enabling reliable localization and navigation across complex, large-scale urban environments. Unlike pose-graph-based methods that rely solely on pairwise relationships between LiDAR frames, LBA leverages raw LiDAR correspondences to achieve more precise results, especially when initial pose estimates are unreliable for low-cost sensors. However, existing LBA methods face challenges such as simplistic planar correspondences, extensive observations, and dense normal matrices in the least-squares problem, which limit robustness, efficiency, and scalability. To address these issues, we propose a Graph Optimality-aware Stochastic Optimization scheme with Progressive Spatial Smoothing, namely PSS-GOSO, to achieverobust,efficient, andscalableLBA. The Progressive Spatial Smoothing (PSS) module extractsrobustLiDAR feature association exploiting the prior structure information obtained by the polynomial smooth kernel. The Graph Optimality-aware Stochastic Optimization (GOSO) module first sparsifies the graph according to optimality for anefficientoptimization. GOSO then utilizes stochastic clustering and graph marginalization to solve the large-scale state estimation problem for ascalableLBA. We validate PSS-GOSO across diverse scenes captured by various platforms, demonstrating its superior performance compared to existing methods. Moreover, the resulting point cloud maps are used for automatic last-mile delivery in large-scale complex scenes, showcasing the practical benefits of our method in modern intelligent transportation systems. The project page can be found at:https://kafeiyin00.github.io/PSS-GOSO/

Remaining useful life (RUL) prediction is an essential component for prognostics and health management of a system. Due to the powerful ability of nonlinear modeling, deep learning (DL) models have emerged as leading solutions by capturing temporal dependencies within time series sensory data. However, in RUL prediction tasks, data are typically collected from multiple sensors, introducing spatial dependencies in the form of sensor correlations. Existing methods are limited in effectively modeling and capturing the spatial dependencies, restricting their performance to learn representative features for RUL prediction. To overcome the limitations, we propose a novel LOcal-GlObal correlation fusion-based framework (LOGO). Our approach combines both local and global information to model sensor correlations effectively. From a local perspective, we account for local correlations that represent dynamic changes of sensor relationships in local ranges. Simultaneously, from a global perspective, we capture global correlations that depict relatively stable relations between sensors. An adaptive fusion mechanism is proposed to automatically fuse the correlations from different perspectives. Subsequently, we define sequential micrographs for each sample to effectively capture the fused correlations. Graph neural network (GNN) is introduced to capture the spatial dependencies within each micrograph, and the temporal dependencies between these sequential micrographs are then captured. This approach allows us to effectively model and capture the dependency information within the data for accurate RUL prediction. Extensive experiments have been conducted, verifying the effectiveness of our method.

A differential dynamic programming (DDP)-based framework for inverse reinforcement learning (IRL) is introduced to recover the parameters in the cost function, system dynamics, and constraints from demonstrations. Different from existing work, where DDP was usually used for the inner forward problem, our proposed framework uses it to efficiently compute the gradient required in the outer inverse problem with equality and inequality constraints. The equivalence between the proposed and existing methods based on Pontryagin's Maximum Principle (PMP) is established. More importantly, using this DDP-based IRL with an open-loop loss function, a closed-loop IRL framework is presented. In this framework, a loss function is proposed to capture the closed-loop nature of demonstrations. It is shown to be better than the commonly used open-loop loss function. We show that the closed-loop IRL framework reduces to a constrained inverse optimal control problem under certain assumptions. Under these assumptions and a rank condition, it is proven that the learning parameters can be recovered from the demonstration data. The proposed framework is extensively evaluated through four numerical robot examples and one real-world quadrotor system. The experiments validate the theoretical results and illustrate the practical relevance of the approach.

Inter-robot relative positions are crucial for executing various multirobot missions, such as formation maneuvering and collaborative inspection. However, the current sensing technology usually provides part of relative position information, such as inter-robot distances, bearings and angles. This prompts the study of determining inter-robot relative positions, i.e., relative localization, from these partial measurements. Based on the existing results of static networks' localizability and mobile robots' relative localization, we propose a novel concept,relative localizabilityto describe whether a multirobot system isrelatively localizable. Given each robot's self-displacement measurements and inter-robot partial measurements in$d$($d\leq 4$) sampling instants, we show that a multirobot system's relative localization can be achieved in a purelyalgebraicanddistributedmanner, in which the multirobot system is said to be$d$-step relatively localizable. To make the results more general, we consider that the multirobot system consists of landmarks, leaders, and followers, and that the inter-robot measurements can be distances, bearings or angles. When robots' coordinate frames have different orientations, we show that the given local measurements can be used to determine robots' relative positions and their coordinate frames' relative orientations simultaneously. Simulations and experiments of relative localization for ground robots are conducted to validate the obtained results.

In this article, we present an efficient visual simultaneous localization and mapping (SLAM) system designed to tackle both short-term and long-term illumination challenges. Our system adopts a hybrid approach that combines deep learning techniques for feature detection and matching with traditional back-end optimization methods. Specifically, we propose a unified convolutional neural network that simultaneously extracts keypoints and structural lines. These features are then associated, matched, triangulated, and optimized in a coupled manner. In addition, we introduce a lightweight relocalization pipeline that reuses the built map, where keypoints, lines, and a structure graph are used to match the query frame with the map. To enhance the applicability of the proposed system to real-world robots, we deploy and accelerate the feature detection and matching networks using C++ and NVIDIA TensorRT. Extensive experiments conducted on various datasets demonstrate that our system outperforms other state-of-the-art visual SLAM systems in illumination-challenging environments. Efficiency evaluations show that our system can run at a rate of$73\,\mathrm{Hz}$on a PC and$40\,\mathrm{Hz}$on an embedded platform.

There is a lack of rigorous stability concept which can stipulate the actual settling time of a dynamic system in existing studies. In this article, a stipulated-time stability for nonautonomous dynamic systems is proposed, which is then extended to stipulated-time boundedness for uncertain systems. By using a class of bounded time-varying functions, Lyapunov-like conditions to ensure a dynamic system to exhibit stipulated-time stability/boundedness are developed. It is interesting that previous Lyapunov-like theorems for predefined-time (PDT) stability can be unified into our framework to achieve stipulated-time stability. To validate the framework, a stipulated-time controller is first designed for a general affine system, which requires a smaller initial control signal than that of PDT control. Furthermore, a generalized design of stipulated-time distributed observer for leader-following multiagent systems is proposed.

Multivariate Time-Series (MTS) data is crucial in various application fields. With its sequential and multi-source (multiple sensors) properties, MTS data inherently exhibits Spatial-Temporal (ST) dependencies, involving temporal correlations between timestamps and spatial correlations between sensors in each timestamp. To effectively leverage this information, Graph Neural Network-based methods (GNNs) have been widely adopted. However, existing approaches separately capture spatial dependency and temporal dependency and fail to capture the correlations between Different sEnsors at Different Timestamps (DEDT). Overlooking such correlations hinders the comprehensive modelling of ST dependencies within MTS data, thus restricting existing GNNs from learning effective representations. To address this limitation, we propose a novel method called Fully-Connected Spatial-Temporal Graph Neural Network (FC-STGNN), including two key components namely FC graph construction and FC graph convolution. For graph construction, we design a decay graph to connect sensors across all timestamps based on their temporal distances, enabling us to fully model the ST dependencies by considering the correlations between DEDT. Further, we devise FC graph convolution with a moving-pooling GNN layer to effectively capture the ST dependencies for learning effective representations. Extensive experiments show the effectiveness of FC-STGNN on multiple MTS datasets compared to SOTA methods. The code is available at https://github.com/Frank-Wang-oss/FCSTGNN.

Contrastive learning, as a self-supervised learning paradigm, becomes popular for Multivariate Time-Series (MTS) classification. It ensures the consistency across different views of unlabeled samples and then learns effective representations for these samples. Existing contrastive learning methods mainly focus on achieving temporal consistency with temporal augmentation and contrasting techniques, aiming to preserve temporal patterns against perturbations for MTS data. However, they overlook spatial consistency that requires the stability of individual sensors and their correlations. As MTS data typically originate from multiple sensors, ensuring spatial consistency becomes essential for the overall performance of contrastive learning on MTS data. Thus, we propose Graph-Aware Contrasting for spatial consistency across MTS data. Specifically, we propose graph augmentations including node and edge augmentations to preserve the stability of sensors and their correlations, followed by graph contrasting with both node- and graph-level contrasting to extract robust sensor- and global-level features. We further introduce multi-window temporal contrasting to ensure temporal consistency in the data for each sensor. Extensive experiments demonstrate that our proposed method achieves state-of-the-art performance on various MTS classification tasks. The code is available at https://github.com/Frank-Wang-oss/TS-GAC.

Perception plays a crucial role in various robot applications. However, existing well-annotated datasets are biased towards autonomous driving scenarios, while unlabelled SLAM datasets are quickly over-fitted, and often lack environment and domain variations. To expand the frontier of these fields, we introduce a comprehensive dataset named MCD (Multi-Campus Dataset), featuring a wide range of sensing modalities, high-accuracy ground truth, and diverse challenging environments across three Eurasian university campuses. MCD comprises both CCS (Classical Cylindrical Spinning) and NRE (Non-Repetitive Epicyclic) lidars, high-quality IMUs (Inertial Measurement Units), cameras, and UWB (Ultra-WideBand) sensors. Further-more, in a pioneering effort, we introduce semantic annotations of 29 classes over 59k sparse NRE lidar scans across three domains, thus providing a novel challenge to existing semantic segmentation research upon this largely unexplored modality. Finally, we propose, for the first time to the best of our knowledge, continuous-time ground truth based on optimization-based registration of lidar-inertial data on three survey-grade prior maps, each several times larger than the next largest publicly available ones. We conduct a rigorous evaluation of numerous state-of-the-art algorithms on MCD, report their performance, and highlight the challenges awaiting solutions from the research community.

This paper addresses the abstract dynamic programming (DP) in the online scenario, where the abstract DP mapping is time-varying, instead of static. In this case, optimal costs and policies at different time instants are not the same in general, and the problem amounts to tracking time-varying optimal costs and policies, which is of interest to many practical problems. It is thus necessary to analyze the performance of classical value iteration (VI) and policy iteration (PI) algorithms in the online case. In doing so, this paper develops and provides the theoretical analysis for several online algorithms, including approximate online VI, online PI, approximate online PI, online optimistic PI, and approximate online optimistic PI algorithms. It is proved that the tracking error bounds for all algorithms critically depend upon the largest difference between any two consecutive abstract mappings. Meanwhile, examples are presented to illustrate the theoretical results.

Unsupervised domain adaptation (UDA) involves adapting a model trained on a label-rich source domain to an unlabeled target domain. However, in real-world scenarios, the absence of target-domain labels makes it challenging to evaluate the performance of UDA models. Furthermore, prevailing UDA methods relying on adversarial training and self-training could lead to model degeneration and negative transfer, further exacerbating the evaluation problem. In this paper, we propose a novel metric called the Transfer Score to address these issues. The proposed metric enables the unsupervised evaluation of UDA models by assessing the spatial uniformity of the classifier via model parameters, as well as the transferability and discriminability of deep representations. Based on the metric, we achieve three novel objectives without target-domain labels: (1) selecting the best UDA method from a range of available options, (2) optimizing hyperparameters of UDA models to prevent model degeneration, and (3) identifying which checkpoint of UDA model performs optimally. Our work bridges the gap between data-level UDA research and practical UDA scenarios, enabling a realistic assessment of UDA model performance. We validate the effectiveness of our metric through extensive empirical studies on UDA datasets of different scales and imbalanced distributions. The results demonstrate that our metric robustly achieves the aforementioned goals.

Multi-modal unsupervised domain adaptation (MM-UDA) for 3D semantic segmentation is a practical solution to embed semantic understanding in autonomous systems without expensive point-wise annotations. While previous MM-UDA methods can achieve overall improvement, they suffer from significant class-imbalanced performance, restricting their adoption in real applications. This imbalanced performance is mainly caused by: 1) self-training with imbalanced data and 2) the lack of pixel-wise 2D supervision signals. In this work, we propose Multi-modal Prior Aided (MoPA) domain adaptation to improve the performance of rare objects. Specifically, we develop Valid Ground-based Insertion (VGI) to rectify the imbalance supervision signals by inserting prior rare objects collected from the wild while avoiding introducing artificial artifacts that lead to trivial solutions. Meanwhile, our SAM consistency loss leverages the 2D prior semantic masks from SAM as pixel-wise supervision signals to encourage consistent predictions for each object in the semantic mask. The knowledge learned from modal-specific prior is then shared across modalities to achieve better rare object segmentation. Extensive experiments show that our method achieves state-of-the-art performance on the challenging MM-UDA benchmark. Code will be available at https://github.com/AronCao49/MoPA.

One-shot LiDAR localization refers to the ability to estimate the robot pose from one single point cloud, which yields significant advantages in initialization and relocalization processes. In the point cloud domain, the topic has been extensively studied as a global descriptor retrieval (i.e., loop closure detection) and pose refinement (i.e., point cloud registration) problem both in isolation or combined. However, few have explicitly considered the relationship between candidate retrieval and correspondence generation in pose estimation, leaving them brittle to substructure ambiguities. To this end, we propose a hierarchical one-shot localization algorithm called Outram that leverages substructures of 3D scene graphs for locally consistent correspondence searching and global substructure-wise outlier pruning. Such a hierarchical process couples the feature retrieval and the correspondence extraction to resolve the substructure ambiguities by conducting a local-to-global consistency refinement. We demonstrate the capability of Outram in a variety of scenarios in multiple large-scale outdoor datasets. Our implementation is open-sourced: https://github.com/Pamphlett/Outram.

In response to the evolving challenges posed by small unmanned aerial vehicles (UAVs), which possess the potential to transport harmful payloads or independently cause damage, we introduce MMAUD: a comprehensive Multi-Modal Anti-UAV Dataset. MMAUD addresses a critical gap in contemporary threat detection methodologies by focusing on drone detection, UAV-type classification, and trajectory estimation. MMAUD stands out by combining diverse sensory inputs, including stereo vision, various Lidars, Radars, and audio arrays. It offers a unique overhead aerial detection vital for addressing real-world scenarios with higher fidelity than datasets captured on specific vantage points using thermal and RGB. Additionally, MMAUD provides accurate Leica-generated ground truth data, enhancing credibility and enabling confident refinement of algorithms and models, which has never been seen in other datasets. Most existing works do not disclose their datasets, making MMAUD an invaluable resource for developing accurate and efficient solutions. Our proposed modalities are cost-effective and highly adaptable, allowing users to experiment and implement new UAV threat detection tools. Our dataset closely simulates real-world scenarios by incorporating ambient heavy machinery sounds. This approach enhances the dataset’s applicability, capturing the exact challenges faced during proximate vehicular operations. It is expected that MMAUD can play a pivotal role in advancing UAV threat detection, classification, trajectory estimation capabilities, and beyond. Our dataset, codes, and designs will be available in https://ntu-aris.github.io/MMAUD.

Accurate and consistent construction of point clouds from LiDAR scanning data is fundamental for 3D modeling applications. Current solutions, such as multiview point cloud registration and LiDAR bundle adjustment, predominantly depend on the local plane assumption, which may be inadequate in complex environments lacking of planar geometries or substantial initial pose errors. To mitigate this problem, this paper presents a LiDAR bundle adjustment with progressive spatial smoothing, which is suitable for complex environments and exhibits improved convergence capabilities. The proposed method consists of a spatial smoothing module and a pose adjustment module, which combines the benefits of local consistency and global accuracy. With the spatial smoothing module, we can obtain robust and rich surface constraints employing smoothing kernels across various scales. Then the pose adjustment module corrects all poses utilizing the novel surface constraints. Ultimately, the proposed method simultaneously achieves fine poses and parametric surfaces that can be directly employed for high-quality point cloud reconstruction. The effectiveness and robustness of our proposed approach have been validated on both simulation and real-world datasets. The experimental results demonstrate that the proposed method outperforms the existing methods and achieves better accuracy in complex environments with low planar structures.

This paper considers the multi-robot active graph exploration problem, where robots need to collaboratively cover a graph environment while maintaining reliable pose estimation in collaborative Simultaneous Localization and Mapping (SLAM). Considering both objectives presents challenges for multi-robot pathfinding, as it involves the expensive covariance propagation for SLAM uncertainty evaluation, especially when considering various combinations of robots’ paths. To reduce the computational complexity, we propose an efficient two-stage strategy where exploration paths are first generated for quick coverage, and then enhanced by adding informative loop-closing actions along the paths for reliable pose estimation. We formulate the latter problem as a non-monotone submodular maximization problem by relating SLAM uncertainty with pose graph topology, which (1) facilitates a more efficient evaluation of SLAM uncertainty than covariance inference, and (2) allows the employment of approximation algorithms in submodular optimization to provide suboptimality guarantees. We further introduce ordering heuristics to improve the objective values while preserving the optimality bound. Simulation experiments over randomly generated graph environments verify the effectiveness of our methods to achieve quick coverage and enhanced pose graph reliability, and benchmark the performance of the approximation algorithms and the greedy-based algorithm in the loop edge selection problem. Our implementations will be open-source at https://github.com/bairuofei/CGE.

Inspired by the behavior of birds, we present AirCrab, a hybrid aerial ground manipulator (HAGM) with a single active wheel and a 3-degree of freedom (3-DoF) manipulator. AirCrab leverages a single point of contact with the ground to reduce position drift and improve manipulation accuracy. The single active wheel enables locomotion on narrow surfaces without adding significant weight to the robot. To realize accurate attitude maintenance using propellers on the ground, we design a control allocation method for AirCrab that prioritizes attitude control and dynamically adjusts the thrust input to reduce energy consumption. Experiments verify the effectiveness of the proposed control method and the gain in manipulation accuracy with ground contact. A series of operations to complete the letters ‘NTU’ demonstrates the capability of the robot to perform challenging hybrid aerial-ground manipulation missions.

LiDAR odometry is a pivotal technology in the fields of autonomous driving and autonomous mobile robotics. However, most of the current works focus on nonlinear optimization methods, and still existing many challenges in using the traditional Iterative Extended Kalman Filter (IEKF) framework to tackle the problem: IEKF only iterates over the observation equation, relying on a rough estimate of the initial state, which is insufficient to fully eliminate motion distortion in the input point cloud; the system process noise is difficult to be determined during state estimation of the complex motions; and the varying motion models across different sensor carriers. To address these issues, we propose the Dual-Iteration Extended Kalman Filter (I2EKF) and the LiDAR odometry based on I2EKF (I2EKF-LO). This approach not only iterates over the observation equation but also leverages state updates to iteratively mitigate motion distortion in LiDAR point clouds. Moreover, it dynamically adjusts process noise based on the confidence level of prior predictions during state estimation and establishes motion models for different sensor carriers to achieve accurate and efficient state estimation. Comprehensive experiments demonstrate that I2EKF-LO achieves outstanding levels of accuracy and computational efficiency in the realm of LiDAR odometry. Additionally, to foster community development, our code is open-sourced.1

Online 3D multi-object tracking (MOT) has recently received significant research interests due to the expanding demand of 3D perception in advanced driver assistance systems (ADAS) and autonomous driving (AD). Among the existing 3D MOT frameworks for ADAS and AD, conventional point object tracking (POT) framework using the tracking-by-detection (TBD) strategy has been well studied and accepted for LiDAR and 4D imaging radar point clouds. In contrast, extended object tracking (EOT), another important framework which accepts the joint-detection-and-tracking (JDT) strategy, has rarely been explored for online 3D MOT applications. This paper provides the first systematical investigation of the EOT framework for online 3D MOT in real-world ADAS and AD scenarios. Specifically, the widely accepted TBD-POT framework, the recently investigated JDT-EOT framework, and our proposed TBD-EOT framework are compared via extensive evaluations on two open source 4D imaging radar datasets: View-of-Delft and TJ4DRadSet. Experiment results demonstrate that the conventional TBD-POT framework remains preferable for online 3D MOT with high tracking performance and low computational complexity, while the proposed TBD-EOT framework has the potential to outperform it in certain situations. However, the results also show that the JDT-EOT framework encounters multiple problems and performs inadequately in evaluation scenarios. After analyzing the causes of these phenomena based on various evaluation metrics and visualizations, we provide possible guidelines to improve the performance of these MOT frameworks on real-world data. These provide the first benchmark and important insights for the future development of 4D imaging radar-based online 3D MOT algorithms.

Time-of-Flight (ToF) RGB-D cameras provide a wealth of information for SLAM systems. However, the limited field of view (FOV) of a single ToF RGB-D camera and the small range of its depth measurement module make it prone to degeneracy when relying solely on visual or depth information for SLAM, a problem typical of unimodal SLAM algorithms. To address this issue, this article presents M-DIVO: an IEKF-based odometry that fuses visual, depth (similar to LiDAR), and inertial modules from multiple ToF RGB-D cameras. It comprises two direct method subsystems: 1) the depth–inertial odometry (DIO) subsystem, which constructs point-to-plane constraints from multiple depth modules and 2) the visual–inertial odometry (VIO) subsystem, which optimizes pose using photometric error constructed by multiple cameras. Additionally, to manage the significant computational load from processing multiple sensors and multimodal information, we introduce a multimodal redundancy scheduling mechanism (MRSM): prioritizing the DIO subsystem with the VIO subsystem as auxiliary, executing the VIO subsystem only when degeneracy occurs in the DIO subsystem. We also propose a “External First, Internal Last” strategy for calibrating multiple external and internal sensors. Experiments demonstrate that compared to unimodal SLAM, our method achieves higher robustness and precision, as well as satisfactory real-time performance. The proposed calibration strategy is demonstrated to be more accurate than the traditional inertial measurement unit-centric approach. The code is open source.

WiFi-based pose estimation is a technology with great potential for the development of smart homes and metaverse avatar generation. However, current WiFi-based pose estimation methods are predominantly evaluated under controlled laboratory conditions with sophisticated vision models to acquire accurately labeled data. Furthermore, WiFi channel state information (CSI) is highly sensitive to environmental variables, and direct application of a pretrained model to a new environment may yield suboptimal results due to domain shift. In this article, we propose a domain adaptation algorithm, AdaPose, designed specifically for WiFi-based pose estimation. The proposed method aims to identify consistent human poses that are highly resistant to environmental dynamics and WiFi signal noises. To achieve this goal, we introduce instance-wise consistency alignment loss that aligns domain shifts considering instance-wise pose distribution variance, and cross-environment channel enhancement module that enhances WiFi CSI feature representation by emphasizing channel-wise similarity between source and target domains. We conduct extensive experiments on both our self-collected pose estimation data set and a large public MM-Fi data set. The results demonstrate the effectiveness and robustness of AdaPose in eliminating domain shift, thereby facilitating the widespread application of WiFi-based pose estimation in smart cities.

Unsupervised Domain Adaptation (UDA) methods have been successful in reducing label dependency by minimizing the domain discrepancy between labeled source domains and unlabeled target domains. However, these methods face challenges when dealing with Multivariate Time-Series (MTS) data. MTS data typically originates from multiple sensors, each with its unique distribution. This property poses difficulties in adapting existing UDA techniques, which mainly focus on aligning global features while overlooking the distribution discrepancies at the sensor level, thus limiting their effectiveness for MTS data. To address this issue, a practical domain adaptation scenario is formulated as Multivariate Time-Series Unsupervised Domain Adaptation (MTS-UDA). In this paper, we propose SEnsor Alignment (SEA) for MTS-UDA, aiming to address domain discrepancy at both local and global sensor levels. At the local sensor level, we design endo-feature alignment, which aligns sensor features and their correlations across domains. To reduce domain discrepancy at the global sensor level, we design exo-feature alignment that enforces restrictions on global sensor features. We further extend SEA to SEA++ by enhancing the endo-feature alignment. Particularly, we incorporate multi-graph-based higher-order alignment for both sensor features and their correlations. Extensive empirical results have demonstrated the state-of-the-art performance of our SEA and SEA++ on six public MTS datasets for MTS-UDA.

This article investigates the similar formation control problem for multirobot systems. Specifically, we propose an integrated relative localization and similar formation control scheme to navigate multirobot systems to a desired configuration, which is a similar transformation of a given template, based on interrobot and robot-landmark range measurements and odometry measurements of robots themselves. To achieve the exact relative localization, a persistent excitation (P.E.) signal is introduced in the controller which, however, perturbs the motion of each robot and affects the formation accuracy. To resolve the conflict, an autonomous system with its output regulated by a carefully designed function of range measurements is introduced to generate the persistent excitation. It is proved that the similar formation control problem can be solved by our proposed scheme with global asymptotic convergence for directed acyclic graphs (DAGs). Both numerical simulation and physical experiment are presented to verify and validate the effectiveness of our theoretical findings.

This article introduces a novel self-supervised method that leverages incoherence detection for video representation learning. It stems from the observation that the visual system of human beings can easily identify video incoherence based on their comprehensive understanding of videos. Specifically, we construct the incoherent clip by multiple subclips hierarchically sampled from the same raw video with various lengths of incoherence. The network is trained to learn the high-level representation by predicting the location and length of incoherence given the incoherent clip as input. Additionally, we introduce intravideo contrastive learning to maximize the mutual information between incoherent clips from the same raw video. We evaluate our proposed method through extensive experiments on action recognition and video retrieval using various backbone networks. Experiments show that our proposed method achieves remarkable performance across different backbone networks and different datasets compared to previous coherence-based methods.

This article presents a bio-inspired safety control scheme for unmanned aerial vehicles (UAVs) in confined environments with disturbance. Although there has been some existing research on the effect of disturbance for a single UAV, multi-UAV formation under external wind disturbances remains challenging, especially in a tight and confined environment. Inspired by nature, this study concentrates on an anti-disturbance mechanism for safe multi-UAV formation in a tight environment. The presented safety control system combines disturbance observer-based control (DOBC), bionic formation switching (BFS) strategy, and safety evaluation. Two safety issues are considered in this article. For a single UAV, the estimated disturbance is compensated in the inner-loop controller. While for multi-UAV formation, the BFS strategy attenuates the effect of external wind disturbance leveraging the formation configuration. The so-called group perturbation immune factor (GPIF) is designed to analyze and evaluate the safety of the overall formation. The experimental results validate the comprehensiveness and anti-disturbance capability of the system.

Cooperative perception techniques empower connected and automated vehicles (CAVs) perception capabilities through Vehicle-to-Everything (V2X) communication. However, this advancement introduces a significant influx of information within CAVs, posing a new challenge in managing potentially intricate aspects of asynchrony and correlation in this information landscape. In this context, this paper proposes a cooperative perception algorithm that considers localization uncertainty and information correlation and asynchrony. Specifically, a hierarchical split covariance intersection (SCI) approach is proposed to efficiently fuse the information from local sensors and connected devices. To bridge the reference disparities of information among CAVs, we incorporate localization uncertainty into the connected information fusion through a coordinate transformation approach based on the cubature rule, which unifies the references. Then, the covariance boundedness of the whole proposed algorithm is theoretically analyzed, demonstrating to some extent the safety guaranteed by our algorithm in practical applications. Finally, we build a high-fidelity driving simulator and collected real trajectory data from 80 drivers. The simulation and driver data testing results show the effectiveness and superiority of the proposed algorithm.

Over the past decade, automated methods have been developed to detect cracks more efficiently, accurately, and objectively, with the ultimate goal of replacing conventional manual visual inspection techniques. Among these methods, semantic segmentation algorithms have demonstrated promising results in pixel-wise crack detection tasks. However, training such networks requires a large amount of human-annotated datasets with pixel-level annotations, which is a highly labor-intensive and time-consuming process. Moreover, supervised learning-based methods often struggle with poor generalizability in unseen datasets. Therefore, we propose an unsupervised pixel-wise road crack detection network, known as UP-CrackNet. Our approach first generates multi-scale square masks and randomly selects them to corrupt undamaged road images by removing certain regions. Subsequently, a generative adversarial network is trained to restore the corrupted regions by leveraging the semantic context learned from surrounding uncorrupted regions. During the testing phase, an error map is generated by calculating the difference between the input and restored images, which allows for pixel-wise crack detection. Our comprehensive experimental results demonstrate that UP-CrackNet outperforms other general-purpose unsupervised anomaly detection algorithms, and exhibits satisfactory performance and superior generalizability when compared with state-of-the-art supervised crack segmentation algorithms. Our source code is publicly available at mias.group/UP-CrackNet.

WiFi-based smart human sensing technology enabled by Channel State Information (CSI) has received great attention in recent years. However, CSI-based sensing systems suffer from performance degradation when deployed in different environments. Existing works solve this problem by domain adaptation using massive unlabeled high-quality data from the new environment, which is usually unavailable in practice. In this paper, we propose a novel augmented environment-invariant robust WiFi gesture recognition system named AirFi that deals with the issue of environment dependency from a new perspective. The AirFi is a novel domain generalization framework that learns the critical part of CSI regardless of different environments and generalizes the model to unseen scenarios, which does not require collecting any data for adaptation to the new environment. AirFi extracts the common features from several training environment settings and minimizes the distribution differences among them. The feature is further augmented to be more robust to environments. Moreover, the system can be further improved by few-shot learning techniques. Compared to state-of-the-art methods, AirFi is able to work in different environment settings without acquiring any CSI data from the new environment. The experimental results demonstrate that our system remains robust in the new environment and outperforms the compared systems.

Deep neural networks have empowered accurate device-free human activity recognition, which has wide applications. Deep models can extract robust features from various sensors and generalize well even in challenging situations such as data-insufficient cases. However, these systems could be vulnerable to input perturbations, i.e., adversarial attacks. We empirically demonstrate that both black-box Gaussian attacks and modern adversarial white-box attacks can render their accuracies to plummet. In this paper, we first point out that such phenomenon can bring severe safety hazards to device-free sensing systems, and then propose a novel learning framework, SecureSense, to defend common attacks. SecureSense aims to achieve consistent predictions regardless of whether there exists an attack on its input or not, alleviating the negative effect of distribution perturbation caused by adversarial attacks. Extensive experiments demonstrate that our proposed method can significantly enhance the model robustness of existing deep models, overcoming possible attacks. The results validate that our method works well on wireless human activity recognition and person identification systems. To the best of our knowledge, this is the first work to investigate adversarial attacks and further develop a novel defense framework for wireless human activity recognition in mobile computing research.

The nonsmooth finite-sum minimization is a fundamental problem in machine learning. This article develops a distributed stochastic proximal-gradient algorithm with random reshuffling to solve the finite-sum minimization over time-varying multiagent networks. The objective function is a sum of differentiable convex functions and nonsmooth regularization. Each agent in the network updates local variables by local information exchange and cooperates to seek an optimal solution. We prove that local variable estimates generated by the proposed algorithm achieve consensus and are attracted to a neighborhood of the optimal solution with an O((1/T)+(1/√T)) convergence rate, where T is the total number of iterations. Finally, some comparative simulations are provided to verify the convergence performance of the proposed algorithm.

Domain adaptation (DA) approaches address domain shift and enable networks to be applied to different scenarios. Although various image DA approaches have been proposed in recent years, there is limited research toward video DA. This is partly due to the complexity in adapting the different modalities of features in videos, which includes the correlation features extracted as long-range dependencies of pixels across spatiotemporal dimensions. The correlation features are highly associated with action classes and proven their effectiveness in accurate video feature extraction through the supervised action recognition task. Yet correlation features of the same action would differ across domains due to domain shift. Therefore, we propose a novel adversarial correlation adaptation network (ACAN) to align action videos by aligning pixel correlations. ACAN aims to minimize the distribution of correlation information, termed as pixel correlation discrepancy (PCD). Additionally, video DA research is also limited by the lack of cross-domain video datasets with larger domain shifts. We, therefore, introduce a novel HMDB-ARID dataset with a larger domain shift caused by a larger statistical difference between domains. This dataset is built in an effort to leverage current datasets for dark video classification. Empirical results demonstrate the state-of-the-art performance of our proposed ACAN for both existing and the new video DA datasets.

When the edge between two nodes is not measured, is there any hint to know the edge property, and will the inferred edge property be useful? To answer these questions, this paper uniformly defines the properties of unmeasurable edges in generic graphs. For an unmeasurable edge$(i,j)$, it is called rangeable if its length is unique in any realization of the graph, rigid if the number of its possible lengths is finite, and flexible if it has infinite possible lengths. The rangeable edge can provide deterministic hidden knowledge as if the edge is measured. A condition for an unmeasured edge being rangeable in 2D space is firstly proposed, based on which a centralized identification algorithm (DRE) is designed. However, the centralized rangeable edge identification has the overhead of global information collection. Therefore distributed condition and algorithm to identify rangeable edges are further investigated. We prove that an unmeasurable edge$(i,j)$is rangeable if there are at least two Disjoint Minimally Rigid Branches (DMRBs) between$i$and$j$. The unmeasurable edge$(i,j)$is rigid and flexible when the number of DMRB is one and zero, respectively. A distributed Branching and Blacklisting (BB) algorithm is proposed to find DMRBs, so that rangeable edges are identified distributively. Then, the applications of rangeable, rigid, and flexible edges are discussed. Experimental evaluations show that the centralized and distributed algorithms can identify a rich set of unmeasurable but rangeable edges in distance graphs, even more than the number of directly measured edges. Moreover, BB has a similar identification performance as the centralized DRE algorithm and outperforms existing distributed unmeasurable edge inference algorithms significantly.

Unsupervised Domain Adaptation (UDA) methods can reduce label dependency by mitigating the feature discrepancy between labeled samples in a source domain and unlabeled samples in a similar yet shifted target domain. Though achieving good performance, these methods are inapplicable for Multivariate Time-Series (MTS) data. MTS data are collected from multiple sensors, each of which follows various distributions. However, most UDA methods solely focus on aligning global features but cannot consider the distinct distributions of each sensor. To cope with such concerns, a practical domain adaptation scenario is formulated as Multivariate Time-Series Unsupervised Domain Adaptation (MTS-UDA). In this paper, we propose SEnsor Alignment (SEA) for MTS-UDA to reduce the domain discrepancy at both the local and global sensor levels. At the local sensor level, we design the endo-feature alignment to align sensor features and their correlations across domains, whose information represents the features of each sensor and the interactions between sensors. Further, to reduce domain discrepancy at the global sensor level, we design the exo-feature alignment to enforce restrictions on the global sensor features. Meanwhile, MTS also incorporates the essential spatial-temporal dependencies information between sensors, which cannot be transferred by existing UDA methods. Therefore, we model the spatial-temporal information of MTS with a multi-branch self-attention mechanism for simple and effective transfer across domains. Empirical results demonstrate the state-of-the-art performance of our proposed SEA on two public MTS datasets for MTS-UDA. The code is available at https://github.com/Frank-Wang-oss/SEA

Deep learning has had remarkable success in robotic perception, but its data-centric nature suffers when it comes to generalizing to ever-changing environments. By contrast, physics-based optimization generalizes better, but it does not perform as well in complicated tasks due to the lack of high-level semantic information and reliance on manual parametric tuning. To take advantage of these two complementary worlds, we present PyPose: a robotics-oriented, PyTorch-based library that combines deep perceptual models with physics-based optimization. PyPose's architecture is tidy and well-organized, it has an imperative style interface and is efficient and user-friendly, making it easy to integrate into real-world robotic applications. Besides, it supports parallel computing of any order gradients of Lie groups and Lie algebras and 2nd-order optimizers, such as trust region methods. Experiments show that PyPose achieves more than 10× speedup in computation compared to the state-of-the-art libraries. To boost future research, we provide concrete examples for several fields of robot learning, including SLAM, planning, control, and inertial navigation.

Continual Test-Time Adaptation (CTTA) generalizes conventional Test-Time Adaptation (TTA) by assuming that the target domain is dynamic over time rather than stationary. In this paper, we explore Multi-Modal Continual Test-Time Adaptation (MM-CTTA) as a new extension of CTTA for 3D semantic segmentation. The key to MMCTTA is to adaptively attend to the reliable modality while avoiding catastrophic forgetting during continual domain shifts, which is out of the capability of previous TTA or CTTA methods. To fulfill this gap, we propose an MM-CTTA method called Continual Cross-Modal Adaptive Clustering (CoMAC) that addresses this task from two perspectives. On one hand, we propose an adaptive dual-stage mechanism to generate reliable cross-modal predictions by attending to the reliable modality based on the class-wise feature-centroid distance in the latent space. On the other hand, to perform test-time adaptation without catastrophic forgetting, we design class-wise momentum queues that capture confident target features for adaptation while stochastically restoring pseudo-source features to revisit source knowledge. We further introduce two new benchmarks to facilitate the exploration of MM-CTTA in the future. Our experimental results show that our method achieves state-of-the-art performance on both benchmarks. Visit our project website at https://sites.google.com/view/mmcotta.

This paper investigates the stochastic moving target encirclement problem in a realistic setting. In contrast to typical assumptions in related works, the target in our work is non-cooperative and capable of escaping the circle containment by boosting its speed to maximum for a short duration. In extreme conditions, where GPS signals are not available, weight restrictions are present, and ground guidance is absent, the agents can rely solely on their onboard single-modality perception tools to measure the distances to the target. The distance measurement allows for creating a position estimator by providing a target position-dependent variable. Furthermore, the construction of the unique distributed anti-synchronization controller (DASC) can guarantee that the two agents track and encircle the target swiftly. The convergence of the estimator and controller is rigorously evaluated using the Lyapunov technique. A real-world UAV-based experiment is conducted to illustrate the performance of the proposed methodology in addition to a simulated Matlab numerical sample. Our video demonstration can be found in the URL https://youtu.be/EDVLvP-bk8M.

This paper presents Segregator, a global point cloud registration framework that exploits both semantic information and geometric distribution to efficiently build up outlier-robust correspondences and search for inliers. Current state-of-the-art algorithms rely on point features to set up putative correspondences and refine them by employing pair-wise distance consistency checks. However, such a scheme suffers from degenerate cases, where the descriptive capability of local point features downgrades, and unconstrained cases, where length-preserving (1-TRIMs)-based checks cannot sufficiently constrain whether the current observation is consistent with others, resulting in a complexified NP-complete problem to solve. To tackle these problems, on the one hand, we propose a novel degeneracy-robust and efficient corresponding procedure consisting of both instance-level semantic clusters and geometric-level point features. On the other hand, Gaussian distribution-based translation and rotation invariant measurements (G-TRIMs) are proposed to conduct the consistency check and further constrain the problem size. We validated our proposed algorithm on extensive real-world data-based experiments. The code is available: https://github.com/Pamphlett/Segregator.

In this paper, we present the dynamic model and control of DoubleBee, a novel hybrid aerial-ground vehicle consisting of two propellers mounted on tilting servo motors and two motor-driven wheels. DoubleBee exploits the high energy efficiency of a bicopter configuration in aerial mode, and enjoys the low power consumption of a two-wheel self-balancing robot on the ground. Furthermore, the propeller thrusts act as additional control inputs on the ground, enabling a novel decoupled control scheme where the attitude of the robot is controlled using thrusts and the translational motion is realized using wheels. A prototype of DoubleBee is constructed using commercially available components. The power efficiency and the control performance of the robot are verified through comprehensive experiments. Challenging tasks in indoor and outdoor environments demonstrate the capability of DoubleBee to traverse unstructured environments, fly over and move under barriers, and climb steep and rough terrains.

This paper proposes an illumination-robust visual odometry (VO) system that incorporates both accelerated learning-based corner point algorithms and an extended line feature algorithm. To be robust to dynamic illumination, the proposed system employs the convolutional neural network (CNN) and graph neural network (GNN) to detect and match reliable and informative corner points. Then point feature matching results and the distribution of point and line features are utilized to match and triangulate lines. By accelerating CNN and GNN parts and optimizing the pipeline, the proposed system is able to run in real-time on low-power embedded platforms. The proposed VO was evaluated on several datasets with varying illumination conditions, and the results show that it outperforms other state-of-the-art VO systems in terms of accuracy and robustness. The open-source nature of the proposed system allows for easy implementation and customization by the research community, enabling further development and improvement of VO for various applications.

In this study, we introduce AV-PedAware, a self-supervised audio-visual fusion system designed to improve dynamic pedestrian awareness for robotics applications. Pedestrian awareness is a critical requirement in many robotics applications. However, traditional approaches that rely on cameras and LIDARs to cover multiple views can be expensive and susceptible to issues such as changes in illumination, occlusion, and weather conditions. Our proposed solution replicates human perception for 3D pedestrian detection using low-cost audio and visual fusion. This study represents the first attempt to employ audio-visual fusion to monitor footstep sounds for the purpose of predicting the movements of pedestrians in the vicinity. The system is trained through self-supervised learning based on LIDAR-generated labels, making it a cost-effective alternative to LIDAR-based pedestrian awareness. AV-PedAware achieves comparable results to LIDAR-based systems at a fraction of the cost. By utilizing an attention mechanism, it can handle dynamic lighting and occlusions, overcoming the limitations of traditional LIDAR and camera-based systems. To evaluate our approach's effectiveness, we collected a new multimodal pedestrian detection dataset and conducted experiments that demonstrate the system's ability to provide reliable 3D detection results using only audio and visual data, even in extreme visual conditions. We will make our collected dataset and source code available online for the community to encourage further development in the field of robotics perception systems.

4D human perception plays an essential role in a myriad of applications, such as home automation and metaverse avatar simulation. However, existing solutions which mainly rely on cameras and wearable devices are either privacy intrusive or inconvenient to use. To address these issues, wireless sensing has emerged as a promising alternative, leveraging LiDAR, mmWave radar, and WiFi signals for device-free human sensing. In this paper, we propose MM-Fi, the first multi-modal non-intrusive 4D human dataset with 27 daily or rehabilitation action categories, to bridge the gap between wireless sensing and high-level human perception tasks. MM-Fi consists of over 320k synchronized frames of five modalities from 40 human subjects. Various annotations are provided to support potential sensing tasks, e.g., human pose estimation and action recognition. Extensive experiments have been conducted to compare the sensing capacity of each or several modalities in terms of multiple tasks. We envision that MM-Fi can contribute to wireless sensing research with respect to action recognition, human pose estimation, multi-modal learning, cross-modal supervision, and interdisciplinary healthcare research.

As an important biomarker for human identification, human gait can be collected at a distance by passive sensors without subject cooperation, which plays an essential role in crime prevention, security detection, and other human identification applications. Presently, most research works are based on cameras and computer vision techniques to perform gait recognition. However, vision-based methods are not reliable when confronting poor illuminations, leading to degrading performances. In this article, we propose a novel multimodal gait recognition method, namely, GaitFi, which leverages WiFi signals and videos for human identification. In GaitFi, channel state information (CSI) that reflects the multipath propagation of WiFi is collected to capture human gaits, while videos are captured by cameras. To learn robust gait information, we propose a lightweight residual convolution network (LRCN) as the backbone network and further propose the two-stream GaitFi by integrating WiFi and vision features for the gait retrieval task. The GaitFi is trained by the triplet loss and classification loss on different levels of features. Extensive experiments are conducted in the real world, which demonstrates that the GaitFi outperforms state-of-the-art gait recognition methods based on single WiFi or camera, achieving 94.2% for human identification tasks of 12 subjects.

Accurate indoor pedestrian localization and tracking are crucial in many practical applications. One efficient yet low-cost sensing scheme is the integration of inertial measurement unit and WiFi received signal strength (RSS) due to the popularity of smart devices and WiFi networks. Many approaches have been proposed to enhance the localization performance. However, they heavily rely on prerequisites, including prior knowledge (e.g., map information) and beacon corrections, which degrades the generalization of the approaches and their accuracy in complex environments. To address this issue, in this article, we propose a novel localization approach named VariFi, which incorporates variational inference techniques to estimate the location of pedestrian. Variational inference is applied in this work, whose inference network can produce accurate estimates as its parameters are optimized in terms of the reconstruction loss and regularization loss in real time. A signal map is constructed to provide a conditional RSS distribution at any given location, which is further applied to generate the reconstruction loss based on the real measurements. Also, a filtering mechanism is designed to reduce local optimum cases in optimization by utilizing the prior estimate and RSS fingerprinting estimate. In addition, VariFi can be further applied to conduct online optimization following the existing localization approaches. We conduct experiments, including static localization and trajectory estimation scenarios to validate the performance of our approach. The trajectory estimation results show that our approach outperforms the mainstream approaches in terms of both localization accuracy and robustness, respectively. Furthermore, the combination of existing approaches and VariFi has also been validated effectively in the experiments of two environments, where VariFi has the ability to bring enhanced localization accuracy.

Wi-Fi sensing technology has shown superiority in smart homes among various sensors for its cost-effective and privacy-preserving merits. It is empowered by channel state information (CSI) extracted from Wi-Fi signals and advanced machine learning models to analyze motion patterns in CSI. Many learning-based models have been proposed for kinds of applications, but they severely suffer from environmental dependency. Though domain adaptation methods have been proposed to tackle this issue, it is not practical to collect high-quality, well-segmented, and balanced CSI samples in a new environment for adaptation algorithms, but randomly captured CSI samples can be easily collected. In this article, we first explore how to learn a robust model from these low-quality CSI samples, and propose AutoFi, an annotation-efficient Wi-Fi sensing model based on a novel geometric self-supervised learning algorithm. The AutoFi fully utilizes unlabeled low-quality CSI samples that are captured randomly, and then transfers the knowledge to specific tasks defined by users, which is the first work to achieve cross-task transfer in Wi-Fi sensing. The AutoFi is implemented on a pair of Atheros Wi-Fi APs for evaluation. The AutoFi transfers knowledge from randomly collected CSI samples into human gait recognition and achieves state-of-the-art performance. Furthermore, we simulate cross-task transfer using public data sets to further demonstrate its capacity for cross-task learning. For the UT-HAR and Widar data sets, the AutoFi achieves satisfactory results on activity recognition and gesture recognition without any prior training. We believe that AutoFi takes a huge step toward automatic Wi-Fi sensing without any developer engagement. Our codes have been included inhttps://github.com/xyanchen/Wi-Fi-CSI-Sensing-Benchmark.

In the metaverse, digital avatar plays an important role in representing human beings for various interaction with virtual objects and environments, which puts a high demand on effective pose estimation. Though camera-based solutions yield remarkable performance, they encounter privacy issues and degraded performance caused by varying illumination, especially in the smart home. In this article, we propose a WiFi-based Internet of Things-enabled human pose estimation scheme for metaverse avatar simulation, namely, MetaFi++. Specifically, WPFormer is designed with a shared convolutional module and a Transformer block to map the channel state information of WiFi signals to human pose landmarks, effectively exploring spatial information of human pose through self-attention. It is enforced to learn the annotations from the accurate computer vision model, thus achieving cross-modal supervision. Due to the ubiquitous existence of WiFi and robustness to various illumination conditions, WiFi-based human poses are suitable to instruct the movement of digital avatars in the metaverse, promoting avatar applications in smart homes. The experiments are conducted in the real world, and the results show that the MetaFi++ achieves very high performance with a PCK@50 of 97.30%. Our codes are available inhttps://github.com/pridy999/metafi_pose_estimation.

Unsupervised domain adaptation methods have been proposed to tackle the problem of covariate shift by minimizing the distribution discrepancy between the feature embeddings of source domain and target domain. However, the standard evaluation protocols assume that the conditional label distributions of the two domains are invariant, which is usually not consistent with the real-world scenarios such as long-tailed distribution of visual categories. In this article, the imbalanced domain adaptation (IDA) is formulated for a more realistic scenario where both label shift and covariate shift occur between the two domains. Theoretically, when label shift exists, aligning the marginal distributions may result in negative transfer. Therefore, a novel cluster-level discrepancy minimization (CDM) is developed. CDM proposes cross-domain similarity learning to learn tight and discriminative clusters, which are utilized for both feature-level and distribution-level discrepancy minimization, palliating the negative effect of label shift during domain transfer. Theoretical justifications further demonstrate that CDM minimizes the target risk in a progressive manner. To corroborate the effectiveness of CDM, we propose two evaluation protocols according to the real-world situation and benchmark existing domain adaptation approaches. Extensive experiments demonstrate that negative transfer does occur due to label shift, while our approach achieves significant improvement on imbalanced datasets, including Office-31, Image-CLEF, and Office-Home.

This brief proposes a game-theoretic inverse reinforcement learning (GT-IRL) framework, which aims to learn the parameters in both the dynamic system and individual cost function of multistage games from demonstrated trajectories. Different from the probabilistic approaches in computer science community and residual minimization solutions in control community, our framework addresses the problem in a deterministic setting by differentiating Pontryagin's maximum principle (PMP) equations of open-loop Nash equilibrium (OLNE), which is inspired by Jin et al. (2020). The differentiated equations for a multi-player nonzero-sum multistage game are shown to be equivalent to the PMP equations for another affine-quadratic nonzero-sum multistage game and can be solved by some explicit recursions. A similar result is established for two-player zero-sum games. Simulation examples are presented to demonstrate the effectiveness of our proposed algorithms.

Despite recent progress in trajectory planning for multiple robots and a single tethered robot, trajectory planning for multiple tethered robots to reach their individual targets without entanglements remains a challenging problem. In this article, a complete approach is presented to address this problem. First, a multirobot tether-aware representation of homotopy is proposed to efficiently evaluate the feasibility and safety of a potential path in terms of 1) the cable length required to reach a target following the path, and 2) the risk of entanglements with the cables of other robots. Then the proposed representation is applied in a decentralized and online planning framework, which includes a graph-based kinodynamic trajectory finder and an optimization-based trajectory refinement, to generate entanglement-free, collision-free, and dynamically feasible trajectories. The efficiency of the proposed homotopy representation is compared against the existing single and multiple tethered robot planning approaches. Simulations with up to eight UAVs show the effectiveness of the approach in entanglement prevention and its real-time capabilities. Flight experiments using three tethered UAVs verify the practicality of the presented approach. The software implementation is publicly available online.1

This article studies the problem of distributed framework matching (FM), which originates from the assignment task in multirobot coordination and the matching task in pattern recognition. The objective of distributed FM is to distributively seek a correspondence which minimizes some metrics describing the disagreement between two frameworks (i.e., graphs and their embeddings). In view of the type of the underlying graph in the framework, two formulations, undirected framework matching (UFM) and directed framework matching (DFM), and their convex relaxations, relaxed UFM (RUFM), and relaxed DFM (RDFM), are presented. UFM is converted into a graph matching (GM) problem with the adjacency matrix being replaced by a matrix constructed from the undirected framework under certain graphical conditions, and can be solved distributively. Sufficient conditions for the equivalence between UFM and RUFM, and the perturbation admitting exact recovery of correspondence are established. On the other hand, DFM embeds the configuration of the directed framework via another type of matrix, whose computation is distributed, and can deal with the case of two frameworks with different sizes of node sets. A distributed optimization algorithm for solving RDFM is proposed and its convergence results are established which allows DFM to be solved in a fully distributed manner. Simulation examples on both synthetic data and real world datasets demonstrate the applicability and efficacy of our theoretical results in formation control and object matching problems.

In this article, we study the problem of estimating the four-degree-of-freedom (3-D position and heading) robot-to-robot relative frame transformation using onboard odometry and interrobot distance measurements. First, we present a theoretical analysis of the problem, namely, the derivation and interpretation of the Cramèr–Rao lower bound, the Fisher information matrix, and its determinant. Second, we propose optimization-based solutions, including a quadratically constrained quadratic programming (QCQP) formulation and its semidefinite programming (SDP) relaxation. Third, based on the theoretical results, we can detect singular configurations as well as measure the uncertainty of each individual parameter. We perform extensive simulations and real-life experiments with aerial robots to show that the proposed QCQP and SDP methods can outperform state-of-the-art approaches, especially in geometrically poor or large measurement noise conditions. In general, the QCQP method provides the best results at the expense of computational time, while the SDP method runs much faster and is sufficiently accurate in most cases.

In this article, we consider the problem of using a multirobot system to conduct sweep coverage over a region with uneven and unknown workload distribution. Uneven workload distribution means that a robot has to spend different amounts of time covering a unit area at different locations in the region. Unknown workload distribution means that the amount of workload at any location is unknown prior to the operation, hence online sensing and allocation of workload is needed for better efficiency. In this work, we adopt the formulation in which the entire region is separated into multiple stripes, and a discrete-time distributed workload allocation algorithm is used to allocate workload on a stripe to each robot. Previous works that adopt similar formulations do not provide rigorous stability analysis and experimental verification and lack consideration of practical aspects, such as limited sensor range. This work addresses these weaknesses and bridges the gap between theory and practice. First, compared with the existing works, the convergence of the distributed workload allocation algorithm to the optimal workload assignment is established under a more realistic assumption, and less conservative error bounds are derived, which serve as a better indicator of the effectiveness of the algorithm. Second, we propose a new algorithm that addresses the limited sensor range of robots, which is an important constraint in applications, such as agricultural spraying and building inspection. The stability analysis and error bound of the proposed algorithm are also provided. Third, realistic simulations and actual flight experiments using unmanned aerial vehicles are carried out to demonstrate the practicality and validate the theoretical results.

This article addresses the formation maneuver control problem of leader–follower multiagent systems with high-order integrator dynamics. A distributed output feedback formation maneuver controller is proposed to achieve desired maneuvers so that the scale, orientation, translation, and shape of formation can be manipulated continuously, where the followers do not need to know or estimate the time-varying maneuver parameters only known to the leaders. Compared with existing relative-measurement-based formation maneuver control, the advantages of the proposed method are that it is output (relative output) feedback based and shows how to realize different types of formation shape. In addition, it can be applied to nongeneric and nonconvex nominal configurations and the leaders are allowed to be maneuvered. It is worth noting that the proposed method can also be extended to general linear multiagent systems under some additional conditions. The theoretical results are demonstrated by a simulation example.

This article studies the security synthesis of cyber–physical systems subject to stealthy attacks via zonotopic set theory. The set is used to quantify the effect of potential stealthy attacks on systems. Control performance and security level are characterized using$L_{\infty }$performance index and the radius of the attack-induced state set, respectively. Sufficient design conditions are given to optimize the security while guaranteeing a prescribed level of control performance. Simulation examples are conducted to demonstrate the effectiveness of the proposed method.

In this article, we focus on enabling the autonomous perception and control of a small unmanned aerial vehicle (UAV) for a façade inspection task. Specifically, we consider the perception as a planar object pose estimation problem by simplifying the building structure as a concatenation of planes, and the control as an optimal reference tracking control problem. First, a vision-based adaptive observer is proposed for plane pose estimation which converges fast and is insensitive to noise under very mild observation conditions. Second, a model predictive controller (MPC) is designed to achieve stable plane following and smooth transition in a multiple-plane scenario, while the persistent excitation (PE) condition of the observer and the maneuver constraints of the UAV are satisfied. The stability of the observer and the MPC controller is also investigated to ensure theoretical completeness. The proposed autonomous plane pose estimation and plane tracking methods are tested in both simulation and practical building façade inspection scenarios, which demonstrate their effectiveness and practicability.

Vision-based SLAM has played an important role in many robotic applications. However, most existing visual SLAM methods are developed under a static world assumption and the robustness in dynamic environments remains a challenging problem. In this paper, we propose a robust RGB-D SLAM system for autonomous vehicles in dynamic scenarios which uses geometry-only information to reduce the impact of moving objects. To achieve this, we introduce an effective and efficient dynamic points detection module in a feature- based SLAM system. Specifically, for each new RGB-D image pair, we first segment the depth image into a few regions using the KMeans algorithm, and then identify the dynamic regions via their reprojection errors. The feature points located in these dynamic regions are then removed and only static ones are used for pose estimation. A dense map that contains only static parts of the environment is also produced by removing dynamic regions in the keyframes. Extensive experiments on public dataset and in real-world scenarios demonstrate that our method provides significant improvement in localization accuracy and mapping quality in dynamic environments.

3D semantic segmentation is a vital problem in autonomous driving. Vehicles rely on semantic segmentation to sense the surrounding environment and identify pedestrians, roads, and other vehicles. Though many datasets are publicly available, there exists a gap between public data and real-world scenarios due to the different weathers and environments, which is formulated as the domain shift. These days, the research for Unsupervised Domain Adaptation (UDA) rises for solving the problem of domain shift and the lack of annotated datasets. This paper aims to introduce adversarial learning and cross-modal networks (2D and 3D) to boost the performance of UDA for semantic segmentation across different datasets. With this goal, we design an adversarial training scheme with a domain discriminator and render the domain-invariant feature learning. Furthermore, we demonstrate that introducing 2D modalities can contribute to the improvement of 3D modalities by our method. Experimental results show that the proposed approach improves the mIoU by 7.53% compared to the baseline and has an improvement of 3.68% for the multi-modal performance.

Deep learning based semantic segmentation models have achieved remarkable results in recent years. However, many deep learning based models encounter the problem of catastrophic forgetting, i.e. when the model is required to learn a new task without labels for old objects, its performance drops significantly for the previous tasks. To solve this problem, an incremental learning method, a Combination of Old Prediction and Modified Label (COPML), is developed in this paper. The proposed method utilizes the prediction results of the old model and the modified labels of the new task to create pseudo labels which are close to the ground truths. By using these pseudo labels for training, the model is expected to preserve the knowledge of old tasks. In addition, knowledge distillation, the replay and parameter freezing strategy are also applied to the proposed method to further assist the model in overcoming catastrophic forgetting. The effectiveness of the proposed method is validated on two semantic segmentation models: Unet and Deeplab3 in Pascal- VOC 2012 dataset and a self-made dataset. The experimental results demonstrate that COPML enables the model to maintain most of the old knowledge while obtaining an excellent performance on a new task.

Existing channel-state information (CSI)-based human authentication systems in the literature require a large amount of CSI data to train deep neural network (DNN) models and are ineffective for unknown intruder detection. To address this issue, we propose a CSI-based human authentication system (CAUTION) which is able to learn distinctive gait features of different users through CSI data to perform human authentication in this article. By taking advantage of few-shot learning, CAUTION is able to construct an accurate user identification model with a very limited number of CSI training data. By converting the CSI samples into low-dimensional representations on the feature plane, it computes central points for different users as their CSI profiles and introduces an intruder threshold to measure whether the CSI data matches one of the user classes by a margin. The intruder threshold is able to be optimized without any intruders’ data. CAUTION does not require a large number of training data and provides an effective way to train the system for unknown intruder detection. We have tested CAUTION at different places and compared it with state-of-the-art CSI-based authentication systems. The experimental results demonstrate that CAUTION is able to perform accurate human authentication with a limited amount of CSI training data (one-fifth of data needed by compared systems) and outperforms the compared human authentication systems.

WiFi technology has been applied to various places due to the increasing requirement of high-speed Internet access. Recently, besides network services, WiFi sensing is appealing in smart homes since it is device free, cost effective and privacy preserving. Though numerous WiFi sensing methods have been developed, most of them only consider single smart home scenario. Without the connection of powerful cloud server and massive users, large-scale WiFi sensing is still difficult. In this article, we first analyze and summarize these obstacles, and propose an efficient large-scale WiFi sensing framework, namely, EfficientFi. The EfficientFi works with edge computing at WiFi access points and cloud computing at center servers. It consists of a novel deep neural network that can compress fine-grained WiFi channel state information (CSI) at edge, restore CSI at cloud, and perform sensing tasks simultaneously. A quantized autoencoder and a joint classifier are designed to achieve these goals in an end-to-end fashion. To the best of our knowledge, the EfficientFi is the first Internet of Things-cloud-enabled WiFi sensing framework that significantly reduces communication overhead while realizing sensing tasks accurately. We utilized human activity recognition (HAR) and identification via WiFi sensing as two case studies, and conduct extensive experiments to evaluate the EfficientFi. The results show that it compresses CSI data from 1.368 Mb/s to 0.768 kb/s with extremely low error of data reconstruction and achieves over 98% accuracy for HAR.

To steer a team of multiple mobile agents to desired collective maneuvers so that the geometric pattern, translation, orientation, and scale of formation can be changed continuously, this article studies the formation maneuver control of single-integrator and double-integrator multiagent systems by a leader-follower strategy. Unlike most existing results requiring generic configurations or convex configurations, the proposed control algorithms can be applied to either nongeneric or nonconvex configurations. Distributed control algorithms are designed for the leaders and followers over directed graphs, respectively, where the formation's maneuver parameters, such as geometric pattern, translation, orientation, and scale of formation are decided by the first leader. It is worth noting that the closed-loop tracking errors converge to zero globally. Some numerical simulations are given to illustrate the theoretical results.

This article addresses a multi-pursuer single-evader pursuit-evasion game where the free-moving evader moves faster than the pursuers. Most of the existing works impose constraints on the faster evader, such as limited moving area and moving direction. When the faster evader is allowed to move freely without any constraint, the main issues are how to form an encirclement to trap the evader into the capture domain, how to balance between forming an encirclement and approaching the faster evader, and what conditions make the capture possible. In this article, a distributed pursuit algorithm is proposed to enable pursuers to form an encirclement and approach the faster evader. An algorithm that balances between forming an encirclement and approaching the faster evader is proposed. Moreover, sufficient capture conditions are derived based on the initial spatial distribution and the speed ratios of the pursuers and the evader. Simulation and experimental results on ground robots validate the effectiveness and practicability of the proposed method.

Finite-time control is concerned with steering a system state to the origin before a certain settling-time limit, ignoring any consideration of when each state element converges relative to the others. In this article, a control problem called time-synchronized control is investigated, where all the system state elements have to converge to the origin at the same time. To facilitate this problem formulation, we introduce the notion of time-synchronized stability together with sufficient Lyapunov conditions. Based on these, the analytical solution of a time-synchronized stable system is obtained and discussed, explicitly offering a quantitative method to preview and predesign the control system performance in prior. Following these results, a robust time-synchronized control law is designed for multivariable systems under external disturbances and model uncertainties. Finally, comparative numerical simulations between time-synchronized control and finite/fixed/prescribed-time control are conducted to showcase the time-synchronized features attained.

This article investigates the reinforcement-learning (RL)-based disturbance rejection control for uncertain nonlinear systems having nonsimple nominal models. An extended state observer (ESO) is first designed to estimate the system state and the total uncertainty, which represents the perturbation to the nominal system dynamics. Based on the output of the observer, the control compensates for the total uncertainty in real time, and simultaneously, online approximates the optimal policy for the compensated system using a simulation of experience-based RL technique. Rigorous theoretical analysis is given to show the practical convergence of the system state to the origin and the developed policy to the ideal optimal policy. It is worth mentioning that the widely used restrictive persistence of excitation (PE) condition is not required in the established framework. Simulation results are presented to illustrate the effectiveness of the proposed method.

This article develops an adaptive observation-based efficient reinforcement learning (RL) approach for systems with uncertain drift dynamics. A novel concurrent learning adaptive extended observer (CL-AEO) is first designed to jointly estimate the system state and parameter. This observer has a two-time-scale structure and does not require any additional numerical techniques to calculate the state derivative information. The idea of concurrent learning (CL) is leveraged to use the recorded data, which leads to a relaxed verifiable excitation condition for the convergence of parameter estimation. Based on the estimated state and parameter provided by the CL-AEO, a simulation of experience-based RL scheme is developed to online approximate the optimal control policy. Rigorous theoretical analysis is given to show that the practical convergence of the system state to the origin and the developed policy to the ideal optimal policy can be achieved without the persistence of excitation (PE) condition. Finally, the effectiveness and superiority of the developed methodology are demonstrated via comparative simulations.

In recent years, onboard self-localization (OSL) methods based on cameras or lidar have achieved many significant progresses. However, some issues such as estimation drift and robustness in low-texture environment still remain inherent challenges for OSL methods. On the other hand, infrastructure-based methods can generally overcome these issues, but at the expense of some installation cost. This poses an interesting problem of how to effectively combine these methods, so as to achieve localization with long-term consistency as well as flexibility compared to any single method. To this end, we propose a comprehensive optimization-based estimator for the 15-D state of an unmanned aerial vehicle (UAV), fusing data from an extensive set of sensors: inertial measurement unit (IMU), ultrawideband (UWB) ranging sensors, and multiple onboard visual-inertial and lidar odometry subsystems. In essence, a sliding window is used to formulate a sequence of robot poses, where relative rotational and translational constraints between these poses are observed in the IMU preintegration and OSL observations, while orientation and position are coupled in thebody-offsetUWB range observations. An optimization-based approach is developed to estimate the trajectory of the robot in this sliding window. We evaluate the performance of the proposed scheme in multiple scenarios, including experiments on public datasets, high-fidelity graphical-physical simulation, and field-collected data from UAV flight tests. The result demonstrates that our integrated localization method can effectively resolve the drift issue, while incurring minimal installation requirements.

Due to the scarcity of annotated scene flow data, self-supervised scene flow learning in point clouds has attracted increasing attention. In the self-supervised manner, establishing correspondences between two point clouds to ap-proximate scene flow is an effective approach. Previous methods often obtain correspondences by applying point-wise matching that only takes the distance on 3D point co-ordinates into account, introducing two critical issues: (1) it overlooks other discriminative measures, such as color and surface normal, which often bring fruitful clues for ac-curate matching; and (2) it often generates sub-par performance, as the matching is operated in an unconstrained situation, where multiple points can be ended up with the same corresponding point. To address the issues, we formulate this matching task as an optimal transport problem. The output optimal assignment matrix can be utilized to guide the generation of pseudo ground truth. In this optimal transport, we design the transport cost by considering multiple descriptors and encourage one-to-one matching by mass equality constraints. Also, constructing a graph on the points, a random walk module is introduced to encourage the local consistency of the pseudo labels. Comprehensive experiments on FlyingThings3D and KITTI show that our method achieves state-of-the-art performance among self-supervised learning methods. Our self-supervised method even performs on par with some supervised learning approaches, although we do not need any ground truth flow for training.

This paper considers online convex optimization with long term constraints, where constraints can be violated in intermediate rounds, but need to be satisfied in the long run. The cumulative constraint violation is used as the metric to measure constraint violations, which excludes the situation that strictly feasible constraints can compensate the effects of violated constraints. A novel algorithm is first proposed and it achieves an $\mathcal{O}(T^{\max\{c,1-c\}})$ bound for static regret and an $\mathcal{O}(T^{(1-c)/2})$ bound for cumulative constraint violation, where $c\in(0,1)$ is a user-defined trade-off parameter, and thus has improved performance compared with existing results. Both static regret and cumulative constraint violation bounds are reduced to $\mathcal{O}(\log(T))$ when the loss functions are strongly convex, which also improves existing results. %In order to bound the regret with respect to any comparator sequence, In order to achieve the optimal regret with respect to any comparator sequence, another algorithm is then proposed and it achieves the optimal $\mathcal{O}(\sqrt{T(1+P_T)})$ regret and an $\mathcal{O}(\sqrt{T})$ cumulative constraint violation, where $P_T$ is the path-length of the comparator sequence. Finally, numerical simulations are provided to illustrate the effectiveness of the theoretical results.

Most of the existing visual SLAM methods heavily rely on a static world assumption and easily fail in dynamic environments. Some recent works eliminate the influence of dynamic objects by introducing deep learning-based semantic information to SLAM systems. However such methods suffer from high computational cost and cannot handle unknown objects. In this paper, we propose a real-time semantic RGBD SLAM system for dynamic environments that is capable of detecting both known and unknown moving objects. To reduce the computational cost, we only perform semantic segmentation on keyframes to remove known dynamic objects, and maintain a static map for robust camera tracking. Furthermore, we propose an efficient geometry module to detect unknown moving objects by clustering the depth image into a few regions and identifying the dynamic regions via their reprojection errors. The proposed method is evaluated on public datasets and realworld conditions. To the best of our knowledge, it is one of the first semantic RGB-D SLAM systems that run in real-time on a low-power embedded platform and provide high localization accuracy in dynamic environments.

In recent years, thanks to the continuously reduced cost and weight of 3D lidar, the applications of this type of sensor in the community have become increasingly popular. Despite many progresses, estimation drift and tracking loss are still prevalent concerns associated with these systems. However, in theory these issues can be resolved with the use of some observations to fixed landmarks in the operation environments. This motivates us to investigate a sensor fusion scheme of lidar and inertia measurements with Ultra-Wideband (UWB) range measurements to such landmarks, which can be easily deployed in the environments with minimal cost and time. Hence, data from IMU, lidar and UWB are tightly-coupled with the robot's states on a sliding window based on their timestamps. Then, we construct a cost function comprising of factors from UWB, lidar and IMU preintegration measurements. Finally an optimization process is carried out to estimate the robot's position and orientation. It is demonstrated through some real world experiments that the method can effectively resolve the drift issue, while only requiring two or three anchors deployed in the environment.

Simultaneous Localization and Mapping (SLAM) has wide robotic applications such as autonomous driving and unmanned aerial vehicles. Both computational efficiency and localization accuracy are of great importance towards a good SLAM system. Existing works on LiDAR based SLAM often formulate the problem as two modules: scan-to-scan match and scan-to-map refinement. Both modules are solved by iterative calculation which are computationally expensive. In this paper, we propose a general solution that aims to provide a computationally efficient and accurate framework for LiDAR based SLAM. Specifically, we adopt a non-iterative two-stage distortion compensation method to reduce the computational cost. For each scan input, the edge and planar features are extracted and matched to a local edge map and a local plane map separately, where the local smoothness is also considered for iterative pose optimization. Thorough experiments are performed to evaluate its performance in challenging scenarios, including localization for a warehouse Automated Guided Vehicle (AGV) and a public dataset on autonomous driving. The proposed method achieves a competitive localization accuracy with a processing rate of more than 10 Hz in the public dataset evaluation, which provides a good trade-off between performance and computational cost for practical applications. It is one of the most accurate and fastest open-sourced SLAM systems1in KITTI dataset ranking.

Channel state information (CSI)-based human activity recognition (HAR) has received great attention in recent years due to its advantages in privacy protection, insensitivity to illumination, and no requirement for wearable devices. In this article, we propose a multimodal channel state information-based activity recognition (MCBAR) system that leverages existing WiFi infrastructures and monitors human activities from CSI measurements. MCBAR aims to address the performances degradation of WiFi-based human recognition systems due to environmental dynamics. Specifically, we address the issue of nonuniformly distributed unlabeled data with rarely performed activities by taking advantages of the generative adversarial network (GAN) and semisupervised learning. We apply a multimodal generator to approximate the CSI data distribution in different environment settings with limited measured CSI data. The generated CSI data using the multimodal generator can provide better diversity for knowledge transfer. This multimodal generator improves the ability of MCBAR to recognize specific activities with various CSI patterns caused by environmental dynamics. Compared to state-of-the-art CSI-based recognition systems, MCBAR is more robust as it is able to handle the nonuniformly distributed CSI data collected from a new environment setting. In addition, diverse generated data from the multimodal generator improves the stability of the system. We have tested MCBAR under multiple experimental settings at different places. The experimental results demonstrate that our algorithm overcomes environmental dynamics and outperforms existing HAR systems.

Deep models have achieved prominent results in pattern recognition tasks, especially computer vision and natural language processing. However, the dataset bias caused by the distribution discrepancy between the training and testing data hinders the generalization ability of deep models. Though many domain adaptation approaches have been proposed to mitigate such negative effect, most of them improve the transferability of features by aligning global distributions of deep models. Few researchers pay attention to the versatility of deep features which can play a vital role in cross-domain recognition. In this paper, we propose to enrich the classic deep learning models by capturing high-low-frequency information and multi-scale features, which deal with the domain shift that cannot be easily addressed by merely feature-level alignment. The Hierarchical Transfer Network (HTN) leverages octave convolution, pyramid features, and self-attention mechanism for revamping the classic models, which can be further integrated with any domain alignment approaches by replacing the feature extractor with the proposed HTN. Extensive experiments have been conducted on three public domain adaptation benchmarks. The results show that the proposed HTN can effectively improve adversarial-based, statistics-based, and norm-based domain adaptation approaches, achieving competitive performance without involving model complexity.

In this article, we propose a general graph optimization-based framework for localization, which can accommodate different types of measurements with varying measurement time intervals. Special emphasis will be on range-based localization. Range and trajectory smoothness constraints are constructed in a position graph, then the robot trajectory over a sliding window is estimated by a graph-based optimization algorithm. Moreover, convergence analysis of the algorithm is provided, and the effects of the number of iterations and window size in the optimization on the localization accuracy are analyzed. Extensive experiments on quadcopter under a variety of scenarios verify the effectiveness of the proposed algorithm and demonstrate a much higher localization accuracy than the existing range-based localization methods, especially in the altitude direction.

The linear quadratic regulation (LQR) problem for discrete-time networked control systems (NCSs) is investigated in this article. The difference from most previous works is that input delay and packet losses occur simultaneously in the communication channel connecting the controller to the actuator. Moreover, the data packet dropout is modeled as a time-homogeneous Markov process which poses challenges due to the temporal correlation. The contributions of this article are twofold. First, by applying the maximum principle involving Markov jumps and delay, the linear quadratic optimal control problem in finite horizon is solved and the solution is given in terms of a forward and a backward stochastic difference equations (FBSDEs-M). Second, under a basic assumption, the infinite horizon optimal control problem is solved and the necessary and sufficient condition for mean square stabilization is given in terms of the solutions to coupled algebraic Riccati-type equations with Markov jumps. The presented results are new to the best of our knowledge since there is no existing work that tackles delay and Markovian packet dropouts simultaneously. It is a generalization of the previous work in which the packet dropouts is modeled as an independent identically distributed Bernoulli process.

Point clouds provide intrinsic geometric information and surface context for scene understanding. Existing methods for point cloud segmentation require a large amount of fully labeled data. Using advanced depth sensors, collection of large scale 3D dataset is no longer a cumbersome process. However, manually producing point-level label on the large scale dataset is time and labor-intensive. In this paper, we propose a weakly supervised approach to predict point-level results using weak labels on 3D point clouds. We introduce our multi-path region mining module to generate pseudo point-level labels from a classification network trained with weak labels. It mines the localization cues for each class from various aspects of the network feature using different attention modules. Then, we use the point-level pseudo label to train a point cloud segmentation network in a fully supervised manner. To the best of our knowledge, this is the first method that uses cloud-level weak labels on raw 3D space to train a point cloud semantic segmentation network. In our setting, the 3D weak labels only indicate the classes that appeared in our input sample. We discuss both scene- and subcloud-level weakly labels on raw 3D point cloud data and perform in-depth experiments on them. On ScanNet dataset, our result trained with subcloud-level labels is compatible with some fully supervised methods.

Time-varying formation tracking problem for unmanned aerial vehicle (UAV) swarm systems subject to unknown nonlinear dynamics, external disturbances, and switching topologies is studied. The states of the following UAV s are required to form a predefined time-varying formation while tracking the state of the leading UAV. For each following UAV, the unknown nonlinear dynamics and external disturbance are regarded as an extended state of the UAV, and then an extended state observer (ESO) is correspondingly designed. Based on the output of the ESO, a novel time-varying formation tracking protocol is proposed. Rigorous theoretical analysis is given to show that, with the application of the proposed protocol, the ESO estimation error and the time-varying formation tracking error can be made arbitrarily small. A numerical simulation demonstrates the effectiveness of the proposed approach.

Channel State Information (CSI) based human activity recognition has received great attention in recent years due to its advantages in privacy protection, insensitive to illumination and no requirement for wearable devices. However, for practical deployment, it needs to greatly enhance the performance robustness against dynamic changes of the surrounding environment. To address this problem, we propose a novel CSI based activity recognition using Roaming Generator (CSIRoG) system for human activity detection. CSIRoG leverages existing WiFi infrastructures and monitors human behaviours from CSI measurements. It utilizes the generative adversarial network (GAN) to transfer the CSI information from one environment to another with dynamic changes such as people passing by, furniture layout changes, etc. The proposed method aims to approximate the CSI distribution in the new environment setting which has very limited CSI data. Therefore, the system can learn to handle multiple environment dynamics. Compared to the existing works, CSIRoG leverages a multimodal system model for better diversity of the generated CSI data for knowledge transfer. This improves the ability of CSIRoG to recognize various kinds of CSI information for one specific user activity caused by various dynamic conditions, thus enhancing system robustness. We have tested CSIRoG under multiple environment settings at different places. The experimental results demonstrate that our algorithm overcomes environmental dynamics and outperforms existing human activity recognition systems.

Remote scene classification serves a vital role in many applications. However, satellite images are often blurred and degraded due to aerosol scattering under fog, haze, and other weather conditions, reducing the image contrast and color fidelity. State-of-the-art remote sensing classification models building upon convolutional neural networks (CNNs) are mostly trained on annotated datasets of clear satellite images. When applied to blurred images, they will suffer a great degradation in performance. To address this problem, we adopt the domain adaptation algorithm TADA and propose Transferable Attention enhanced Adversarial Adaptation Network (TA3N), which utilizes annotated data in clear images by applying knowledge transferring from clear image domain to blurred image domain. Our TA3N first integrates spatial attention to focus on salient areas which are discriminative and transferable. In addition, domain discriminator and adversarial training via gradient reversal layer are used to minimize the discrepancies in extracted features from clear and degraded domains. We synthesize degraded remote scene classification dataset SSI based on FoHIS model. Experiments on degraded SSI showed that TA3N significantly outperforms baseline and other state-of-the-art domain adaptation methods.

In this work, we propose a tightly-coupled odometry framework, which combines monocular visual feature observations with distance measurements provided by a single ultra-wideband (UWB) anchor with an initial guess for its location. Firstly, the scale factor and the anchor position in the vision frame will be simultaneously estimated using a variant of Levenberg-Marquardt non-linear least squares optimization scheme. Once the scale factor is obtained, the map of visual features is updated with the new scale. Subsequent ranging errors in a sliding window are continuously monitored and the estimation procedure will be reinitialized to refine the estimates. Lastly, range measurements and anchor position estimates are fused when needed into a pose-graph optimization scheme to minimize both the landmark reprojection errors and ranging errors, thus reducing the visual drift and improving the system robustness. The proposed method is implemented in Robot Operating System (ROS) and can function in real-time. The performance is validated on both public datasets and real-life experiments and compared with state-of-the-art methods.

Loop closure detection is an essential and challenging problem in simultaneous localization and mapping (SLAM). It is often tackled with light detection and ranging (LiDAR) sensor due to its view-point and illumination invariant properties. Existing works on 3D loop closure detection often leverage on matching of local or global geometrical-only descriptors which discard intensity reading. In this paper we explore the intensity property from LiDAR scan and show that it can be effective for place recognition. We propose a novel global descriptor, intensity scan context (ISC), that explores both geometry and intensity characteristics. To improve the efficiency for loop closure detection, an efficient two-stage hierarchical re-identification process is proposed, including binary-operation based fast geometric relation retrieval and intensity structure re-identification. Thorough experiments including both local experiment and public datasets test have been conducted to evaluate the performance of the proposed method. Our method achieves better recall rate and recall precision than existing geometric-only methods.

As an essential component of visual simultaneous localization and mapping (SLAM), place recognition is crucial for robot navigation and autonomous driving. Existing methods often formulate visual place recognition as feature matching, which is computationally expensive for many robotic applications with limited computing power, e.g., autonomous driving and cleaning robot. Inspired by the fact that human beings always recognize a place by remembering salient regions or landmarks that are more attractive or interesting than others, we formulate visual place recognition as saliency reidentification. In the meanwhile, we propose to perform both saliency detection and re-identification in frequency domain, in which all operations become element-wise. The experiments show that our proposed method achieves competitive accuracy and much higher speed than the state-of-the-art feature-based methods. The proposed method is open-sourced and available at https://github.com/wh200720041/SRLCD.git.

Deep neural networks (DNNs) have made significant advances in computer vision and sensor-based smart sensing. DNNs achieve prominent results based on standard data sets and powerful servers, whereas, in real applications with domain-shift data and resource-constrained environments such as Internet-of-Things (IoT) devices in the edge computing, DNNs are likely to have degraded performance in terms of accuracy and efficiency. To this end, we develop the MobileDA framework that learns transferable features while keeping the simple structure of the deep model. Our method allows a novel teacher network trained in the server to distill the knowledge for a student network running in the edge device, which is achieved by a cross-domain distillation. Leveraging unlabeled data in the new environment, our student model amends the feature learning to be domain invariant, then being our objective model running in the edge device. Our approach is evaluated on a challenging IoT-based WiFi gesture recognition scenario, and three classic visual adaptation benchmarks. The empirical studies corroborate the effectiveness of distillation for domain transfer, and the overall results show that our model achieves state-of-the-art performance merely using a simple network.

Location-based service (LBS) has become an indispensable part of our daily lives. Realizing accurate LBS in indoor environments is still a challenging task. WiFi fingerprinting-based indoor positioning system (IPS) has achieved encouraging results recently, but the time and labor overhead of constructing a dense WiFi radio map remains the key bottleneck that hinders it for real-world large-scale implementation. In this article, we propose WiGAN an automatic fine-grained indoor ratio map construction and the adaptation scheme empowered by the Gaussian process regression conditioned least-squares generative adversarial networks (GPR-GANs) with a mobile robot. First, we develop a mobile robotic platform that constructs the spatial map and radio map simultaneously in the easily accessed free space. GPR-GAN first establishes a Gaussian process regression (GPR) model using the real received signal strength (RSS) measurements collected by our robotic platform via LiDAR SLAM in the free space. Then, the outputs of the GPR are adopted as the input of GAN's generator. The learning objective of GAN is to synthesize realistic RSS data in a constrained space where it has not been covered and model the irregular RSS distributions in complex indoor environments. Real-world experiments were conducted in a real-world indoor environment, which confirms the feasibility, high accuracy, and superiority of WiGAN over existing solutions in terms of both RSS estimation accuracy and localization accuracy.

This paper studies the dynamic formation control problem for cooperative agents with discrete-time dynamics over directed graphs. Unlike using absolute coordinate, relative coordinate, interagent distance, or interagent bearing to specify the target formation and coordinate agents to achieve the formation, we study a coordination problem where the desired formation varies with time and only its geometric shape is predefined. Matrix-valued Laplacian approach has been adopted to address this problem in the continuous-time setting. However, the discrete-time counterpart is more challenging due to the constraint in information exchange. On the other hand, observe that in many real operations, at a given time instant, the agents will be able to plan their target formation configurations for a period of time ahead. We propose preview-based P-like and PD-like controllers for the formation control. The controllers with proper parameter setting are proved to be effective to address the dynamic formation control problem. Numerical simulations are given to validate the effectiveness of the proposed controllers.

This puts forth an infrastructure-free cooperative relative localization (RL) for unmanned aerial vehicles (UAVs) in global positioning system (GPS)-denied environments. Instead of estimating relative coordinates with vision-based methods, an onboard ultra-wideband (UWB) ranging and communication (RCM) network is adopted to both sense the inter-UAV distance and exchange information for RL estimation in 2-D spaces. Without any external infrastructures prepositioned, each agent cooperatively performs a consensus-based fusion, which fuses the obtained direct and indirect RL estimates, to generate the relative positions to its neighbors in real time despite the fact that some UAVs may not have direct range measurements to their neighbors. The proposed RL estimation is then applied to formation control. Extensive simulations and real-world flight tests corroborate the merits of the developed RL algorithm.

In this article, we investigate the problem of controlling a multirobot team to follow a leader in formation, supported by a relative position estimate derived from distance and self-displacement measurements, thus waiving the need of external localization infrastructure. The main challenge of the problem, which is to simultaneously fulfill both relative localization and control tasks, is efficiently and novelly resolved by embedding a distance-displacement-based persistently excited adaptive relative localization technique into a time-varying formation with bounded control input (PEARL-TVF). By assuming that the leader is globally reachable and by selecting proper parameters, it is shown that the PEARL-TVF ensures exponentially convergent localization, which leads to exponentially convergent formation when the leader's behavior is deterministic, and bounded formation error for a nondeterministic leader. Numerical simulations and experiments on quadcopters are provided to verify the theoretical findings.

This paper considers leader-following bipartite consensus of single-integrator multiagent systems in the presence of measurement noise. To attenuate the noise, a time-varying consensus gain q(t) is introduced into the stochastic approximation-type protocol. Necessary and sufficient conditions for ensuring a strong mean square leader-following bipartite consensus are given. In particular, in the absence of measurement noise, the convergence speed of error dynamics is dependent on the eigenvalues of Laplacian and the rate of ∫0tq(s)ds approaching infinity. By appropriately choosing q(t), the speed of leader-following bipartite consensus convergence can be improved in a fixed communication topology. It is proven that conditions for the signed digraph to be structurally balanced and having a spanning tree are necessary and sufficient to ensure leader-following bipartite consensus, regardless of measurement noise.

Convolutional neural networks (CNNs) have enabled the state-of-the-art performance in many computer vision tasks. However, little effort has been devoted to establishing convolution in non-linear space. Existing works mainly leverage on the activation layers, which can only provide point-wise non-linearity. To solve this problem, a new operation, kervolution (kernel convolution), is introduced to approximate complex behaviors of human perception systems leveraging on the kernel trick. It generalizes convolution, enhances the model capacity, and captures higher order interactions of features, via patch-wise kernel functions, but without introducing additional parameters. Extensive experiments show that kervolutional neural networks (KNN) achieve higher accuracy and faster convergence than baseline CNN.

Though vision-based techniques have become quite popular for autonomous docking of Unmanned Aerial Vehicles (UAVs), due to limited field of view (FOV), the UAV must rely on other methods to detect and approach the target before vision can be used. In this paper we propose a method combining Ultra-wideband (UWB) ranging sensor with vision-based techniques to achieve both autonomous approaching and landing capabilities in GPS-denied environments. In the approaching phase, a robust and efficient recursive least-square optimization algorithm is proposed to estimate the position of the UAV relative to the target by using the distance and relative displacement measurements. Using this estimate, UAV is able to approach the target until the landing pad is detected by an onboard vision system, then UWB measurements and vision-derived poses are fused with onboard sensor of UAV to facilitate an accurate landing maneuver. Real-world experiments are conducted to demonstrate the efficiency of our method.

We propose a gesture recognition system that leverages existing WiFi infrastructures and learns gestures from channel state information (CSI) measurements. Having developed an innovative OpenWrt-based platform for commercial WiFi devices to extract CSI data, we propose a novel deep Siamese representation learning architecture for one-shot gesture recognition. Technically, our model extends the capacity of spatio-temporal patterns learning for the standard Siamese structure by incorporating convolutional and bidirectional recurrent neural networks. More importantly, the representation learning is ameliorated by our Siamese framework and transferable pairwise loss which helps to remove structured noise, such as individual heterogeneity and various measurement conditions during domain-different training. Meanwhile, our Siamese model also enables one-shot learning for higher availability in reality. We prototype our system on commercial WiFi routers. The experiments demonstrate that our model outperforms state-of-the-art solutions for temporal-spatial representation learning and achieves satisfactory results under one-shot conditions.

Our study is concerned with the time-varying formation tracking problem for second-order multi-agent systems that are subject to unknown nonlinear dynamics and external disturbance, and the states of the followers form a predefined time-varying formation while tracking the state of the leader. The total uncertainty lumps the unknown nonlinear dynamics and the external disturbance, and is regarded as an extended state of the agent. To estimate the total uncertainty, we design an extended state observer (ESO). Then we propose a novel ESO based time-varying formation tracking protocol. It is proved that, under the proposed protocol, the ESO estimation error and the time-varying formation tracking error can be made arbitrarily small. An application to the target enclosing problem for multiple unmanned aerial vehicles (UAVs) verifies the effectiveness and superiority of the proposed approach.

Human activity recognition plays a unique role in many important applications, including ubiquitous computing, health-care services, and smart buildings. Due to the nonintrusive property of smartphones, smartphone sensors are widely used for the identification of human activities. Since the signals of smartphone sensors are quite noisy, feature engineering will be performed to extract more discriminant representations. Then, various machine learning algorithms can be employed to recognize different human activities. Extreme learning machine (ELM) has been shown to be effective in classification tasks with extremely fast learning speed. Due to its randomness property, it is naturally suitable for ensemble learning. In this paper, we propose a novel ensemble ELM algorithm for human activity recognition using smartphone sensors. Gaussian random projection is employed to initialize the input weights of base ELMs. By doing this, more diversities can be generated to boost the performance of ensemble learning. Real experimental data has been applied to evaluate the performance of our proposed approach. We also conduct a comparison of the proposed approach with some state-of-the-art approaches in the literature. The experimental results indicate that our proposed ensemble ELM approach outperforms these approaches and can achieve recognition accuracies of$\text{97.35}\%$and$\text{98.88}\%$on two datasets.

The use of multichannel data in line spectral estimation (or frequency estimation) is common for improving the estimation accuracy in array processing, structural health monitoring, wireless communications, and more. Recently proposed atomic norm methods have attracted considerable attention due to their provable superiority in accuracy, flexibility, and robustness compared with conventional approaches. In this paper, we analyze atomic norm minimization for multichannel frequency estimation from noiseless compressive data, showing that the sample size per channel that ensures exact estimation decreases with the increase of the number of channels under mild conditions. In particular, given L channels, order K (log K) (1 + L/1 log N) samples per channel, selected randomly from N equispaced samples, suffice to ensure with high probability exact estimation of K frequencies that are normalized and mutually separated by at least 4/N. Numerical results are provided corroborating our analysis.

Efficient flight routing and scheduling play an important role in air traffic flow management, which aims to maximize the utilization of airport and enroute capacities to ensure safety and efficiency of air transportation. In this paper, we first propose a novel discrete-time flow dynamic model for an air traffic network, consisting of airports, waypoints, and air links, upon which we formulate an air flow routing and scheduling problem as an integer linear programming problem. Considering the NP-hard nature of the problem, we present a novel hierarchical flow routing and scheduling approach, where the hierarchical architecture is derived naturally from the network containment relationship, and computation is carried out in a bottom-up manner, which relies on an incremental strategy. On the resulting flow routes and schedules, a heuristic algorithm is carried out to determine flight plans for individual aircrafts. The effectiveness of the proposed hierarchical approach is illustrated by air traffic data in four flight information regions in the association of Southeast Asian nations.

Probabilistic Boolean network (PBN) is a kind of stochastic logical system in which update functions are randomly selected from a set of candidate Boolean functions according to a prescribed probability distribution at each time step. In this brief, a pinning controller design algorithm is proposed to set stabilize any PBN with probability one. First, an algorithm is given to change the columns of its transition matrix. Then, according to the newly obtained transition matrix, a fraction of nodes can be selected as pinning nodes to inject control inputs to achieve set stabilization. The problem is challenging since the Boolean functions in a PBN are not deterministic but are randomly chosen among several Boolean functions. Furthermore, the structure matrices of the pinning controllers are given by solving some logical matrices equations based on which a pinning controller design algorithm is provided to set stabilize the PBN with probability one. Finally, the theoretical results are validated using several examples.

Cross-correlator plays a significant role in many visual perception tasks, such as object detection and tracking. Beyond the linear cross-correlator, this paper proposes a kernel cross-correlator (KCC) that breaks traditional limitations. First, by introducing the kernel trick, the KCC extends the linear cross-correlation to non-linear space, which is more robust to signal noises and distortions. Second, the connection to the existing works shows that KCC provides a unified solution for correlation filters. Third, KCC is applicable to any kernel function and is not limited to circulant structure on training data, thus it is able to predict affine transformations with customized properties. Last, by leveraging the fast Fourier transform (FFT), KCC eliminates direct calculation of kernel vectors, thus achieves better performance yet still with a reasonable computational cost. Comprehensive experiments on visual tracking and human activity recognition using wearable devices demonstrate its robustness, flexibility, and efficiency. The source codes of both experiments are released at https://github.com/wang-chen/KCC.

We propose AutoID, a human identification system that leverages the measurements from existing WiFi-enabled Internet of Things (IoT) devices and produces the identity estimation via a novel sparse representation learning technique. The key idea is to use the unique fine-grained gait patterns of each person revealed from the WiFi Channel State Information (CSI) measurements, technically referred to as shapelet signatures, as the "fingerprint" for human identification. For this purpose, a novel OpenWrt-based IoT platform is designed to collect CSI data from commercial IoT devices. More importantly, we propose a new optimization-based shapelet learning framework for tensors, namely Convex Clustered Concurrent Shapelet Learning (C3SL), which formulates the learning problem as a convex optimization. The global solution of C3SL can be obtained efficiently with a generalized gradient-based algorithm, and the three concurrent regularization terms reveal the inter-dependence and the clustering effect of the CSI tensor data. Extensive experiments are conducted in multiple real-world indoor environments, showing that AutoID achieves an average human identification accuracy of 91% from a group of 20 people. As a combination of novel sensing and learning platform, AutoID attains substantial progress towards a more accurate, cost-effective and sustainable human identification system for pervasive implementations.

This paper presents a notion of rigidity, Ratio-of-Distance (RoD) rigidity, for studying when a framework can be uniquely determined by a set of RoD constraints up to similar transformations. Unlike the constraints such as distance, bearing and angle used in existing literature, we aim to develop a framework by a set of RoD constraints (the ratio of distances of a pair of edges joining a common vertex). In particular, triangulated Laman graphs are introduced to provide some sufficient conditions for the RoD rigidity. The proposed RoD rigidity theory is further applied to the RoD-based similar formation control. Numerical simulations are provided to support the analysis.

A model-free localization method with noisy distance measurement is proposed for estimating a moving robot in 3D space. Considering that the traditional filter-based sensor network localization algorithms can not provide acceptable estimation accuracy in altitude in 3D space, the proposed method utilizes not only current measurements but also previous measurements to localize a robot. This character adds more constraints to localization to avoid local minimum. In addition, different from the traditional filter-based localization methods which need kinetic model for localization, our proposed method is model-free and converts the localization problem to graph optimization problem. The advantage is that we avoid the possible estimation error caused by inaccurate or simplified kinetic model. Considering that the communication limitation in application makes many graph optimization theories such as distributed localization theory and trilateration difficult to be realized, our method proposes to add constrained equation between adjacent positions to solve this problem. Experiments under a variety of scenarios verify the stability of this method and show that the algorithm achieves better localization accuracy than filter-based methods.

Queue overflow of a dynamic queue system gives rise to the information loss (or packet loss) in the communication buffer or the decrease of throughput in the transportation network. This paper investigates a stochastic optimal control problem for dynamic queue systems when imposing probability constraints on queue overflows. We reformulate this problem as a Markov decision process (MDP) with safety constraints. We prove that both finite-horizon and infinite-horizon stochastic optimal control for MDP with such constraints can be transformed as a linear program (LP), respectively. Feasibility conditions are provided for the finite-horizon constrained control problem. Two implementation algorithms are designed under the assumption that only the state (not the state distribution) can be observed at each time instant. Simulation results compare optimal cost and state distribution among different scenarios, and show the probability constraint satisfaction by the proposed algorithms.

This paper studies a multi-agent formation control problem under a leader-follower framework, where the agents are governed by double-integrator dynamics and the objective is to achieve a formation with desired shape and a specified size. Firstly, a distributed control algorithm is developed for the leaders to achieve a desired distance. Then, by combining the control law for the leaders and a relative position estimation based control law for the followers, an estimation based formation control algorithm is developed for the entire multi-agent system to asymptotically achieve a formation with desired shape and a specified size in the case without initial relative position estimation errors and approximately attain the desired formation in the case with initial relative position initial estimation errors. Simulation results are provided to validate the proposed algorithm.

As recalling an Unmanned Aerial Vehicle (UAV) after completing a mission requires quite a lot of attention and skill from its operator, in this paper we propose a method to empower UAV with the capability to autonomously return to base and perform precision landing after completing a mission. The main challenge being tackled in this work is that while the vision-based landing technique is already mature, due to GPS error, UAV can only return to within several meters of home position after completing a mission and may fail to detect the visual marker. To resolve this problem, we employ Ultra-wideband (UWB) ranging measurements to localize and approach the home station. Once the UAV detects the visual marker, both UWB and visual tracking information are fused with onboard sensor to achieve even more accurate positioning. Real-life experiment is used to demonstrate the efficacy of the proposed scheme.

This paper proposes a low complexity distributed multi-agent coordination algorithm for agents to reach their target positions under dense traffic while accounting for limited communication. The single-integrator model is used and each agent is assumed to be communicating with only one other agent at a time to approximate limited bandwidth in real systems. A unique hybrid algorithm of Voronoi Cells, repulsion fields and the velocity obstacle method is used to guarantee non-collision while enabling agents to pass between each other. We break symmetry to avoid deadlocks and livelocks by having agent pairs make mutual decisions based on comparison of the conditional priority of each agent. The effectiveness of our method is demonstrated in simulations, which show that they are able to converge to their targets in reasonable time.

In the era of Internet of Things (IoT), human activity recognition is becoming the vital underpinning for a myriad of emerging applications in smart home and smart buildings. Existing activity recognition approaches require either the deployment of extra infrastructure or the cooperation of occupants to carry dedicated devices, which are expensive, intrusive and inconvenient for pervasive implementation. In this paper, we propose DeepSense, a device-free human activity recognition scheme that can automatically identify common activities via deep learning using only commodity WiFi-enabled IoT devices. We design a novel OpenWrt-based IoT platform to collect Channel State Information (CSI) measurements from commercial IoT devices. Moreover, an innovative deep learning framework, Autoencoder Long-term Recurrent Convolutional Network (AE-LRCN), is proposed. It consists of an autoencoder module, a convolutional neural network (CNN) module and a long short-term memory (LSTM) module, which aims to sanitize the noise in raw CSI data, extract high-level representative features and reveal the inherent temporal dependencies among data for accurate human activity recognition, respectively. All the hyperparameters in AE-LRCN are fine-tuned end-to-end automatically. Extensive experiments are conducted in typical indoor environments and the experimental results demonstrate that DeepSense outperforms existing methods and achieves a 97.6% activity recognition accuracy without human intervention.

Accurate human gesture recognition is becoming a cornerstone for myriad emerging applications in human-computer interaction. Existing gesture recognition systems either require dedicated extra infrastructure or user's active cooperation. Although some WiFi-enabled gesture recognition systems have been proposed, they are vulnerable to environmental dynamics and rely on the tedious data re-labeling and expert knowledge each time being implemented in a new environment. In this paper, we propose a WiFi- enabled device-free adaptive gesture recognition scheme, WiADG, that is able to identify human gestures accurately and consistently under environmental dynamics via adversarial domain adaptation. Firstly, a novel OpenWrt-based IoT platform is developed, enabling the direct collection of Channel State Information (CSI) measurements from commercial IoT devices. After constructing an accurate source classifier with labeled source CSI data via the proposed convolutional neural network in the source domain (original environment), we design an unsupervised domain adaptation scheme to reduce the domain discrepancy between the source and the target domain (new environment) and thus improve the generalization performance of the source classifier. The domain- adversarial objective is to train a generator (target encoder) to map the unlabeled target data to a domain invariant latent feature space so that a domain discriminator cannot distinguish the domain labels of the data. In the phase of implementation, we utilize the trained target encoder to map the target CSI frame to the latent feature space and use the source classifier to identify various gestures performed by the user. We implement WiADG on commercial WiFi routers and conduct experiments in multiple indoor environments. The results validate that WiADG achieves 98% gesture recognition accuracy in the original environment. Furthermore, the proposed unsupervised adversarial domain adaptation is able to enhance the recognition accuracy of WiADG by 25% on average without the needs of labeled data collection and new classifier generation when implements it in new environments.

In this paper we propose a method to achieve relative positioning and tracking of a target by a quadcopter using Ultra-wideband (UWB) ranging sensors, which are strategically installed to help retrieve both relative position and bearing between the quadcopter and target. To achieve robust localization for autonomous flight even with uncertainty in the speed of the target, two main features are developed. First, an estimator based on Extended Kalman Filter (EKF) is developed to fuse UWB ranging measurements with data from onboard sensors including inertial measurement unit (IMU), altimeters and optical flow. Second, to properly handle the coupling of the target's orientation with the range measurements, UWB based communication capability is utilized to transfer the target's orientation to the quadcopter. Experiments results demonstrate the ability of the quadcopter to control its position relative to the target autonomously in both cases when the target is static and moving.

Robust velocity and position estimation is crucial for autonomous robot navigation. The optical flow based methods for autonomous navigation have been receiving increasing attentions in tandem with the development of micro unmanned aerial vehicles. This paper proposes a kernel cross-correlator (KCC) based algorithm to determine optical flow using a monocular camera, which is named as correlation flow (CF). Correlation flow is able to provide reliable and accurate velocity estimation and is robust to motion blur. In addition, it can also estimate the altitude velocity and yaw rate, which are not available by traditional methods. Autonomous flight tests on a quadcopter show that correlation flow can provide robust trajectory estimation with very low processing power. The source codes are released based on the ROS framework.

Occupant's thermal comfort detection is a significant contributor to building energy efficiency. However, the predictions from the traditional PMV (Predicted Mean Vote) method often deviate from the actual thermal sensation of occupants. This paper proposes two new approaches for TS (thermal state: Discomfort/Comfort) detection, based on personal physiological features extracted using wearable technology. The first approach, CNN-(Tsk)TP, is based on a deep convolutional neural network (CNN) that associates TS with images of hand skin temperature temporal profile (TP). CNN has shown great success in the classification of captured images, however, its application to 2-D sensor data is rather less explored. In this study, the hand skin temperature was observed to show distinct temporal patterns under different TS. Leveraging this high responsiveness of skin temperature, the two-dimensional sensor data was transferred to image domain. A 4-step domain transfer process was adopted to obtain 5-minute TP for the CNN. The second approach, SVMphyis based on a Support Vector Machine (SVM) model with 6 distinct physiological input features. SVM-RBF performed better among four kernel types evaluated (linear, polynomial, radial, sigmoid). Our proposed approaches CNN-(Tsk)TPand SVMphyachieved 93.33% and 90.6% accuracy, outperforming the existing methods (PMV, ePMV, aPMV and PTS models). Additionally, practical advantages of our approaches are discussed.

In this paper we study the distance-based docking problem of unmanned aerial vehicles (UAVs) by using a single landmark placed at an arbitrarily unknown position. To solve the problem, we propose an integrated estimation-control scheme to simultaneously achieve the relative localization and navigation tasks for discrete-time integrators under bounded velocity: a nonlinear adaptive estimation scheme to estimate the relative position to the landmark, and a delicate control scheme to ensure both the convergence of the estimation and the asymptotic docking at the given landmark. A rigorous proof of convergence is provided by invoking the discrete-time LaSalle's invariance principle, and we also validate our theoretical findings on quadcopters equipped with ultra-wideband ranging sensors and optical flow sensors in a GPS-less environment.

Device-free activity recognition is appealing in smart home applications. It not only is convenient, but also causes no privacy concern, as compared to other activity recognition techniques such as the vision based technique. Existing WiFi-based methods have achieved high accuracy in static circumstances but have limitations in adapting changes in environment and activities locations. In this paper, we propose a fine-grained adaptive location-independent activity recognition system (FALAR) which leverages WiFi signals to characterize and recognize common activities regardless of inconsistency of mutative surroundings. FALAR applies fine-grained channel state information (CSI) to achieve accurate recognitions. To address the issue of environmental changes, we present a Kernel Density Estimation (KDE) based motion extraction method and a coarse-to-fine search strategy for speedy processing. After a denoising scheme, we introduce Class Estimated Basis Space Singular Value Decomposition (CSVD) to efface the static path in the background, and use nonnegative matrix factorization to distinguish various activities by looking into the signal profiles. We evaluate FALAR using two commodity WiFi routers in a typical office environment. Our results show that it achieves remarkable performance.

Intelligent occupancy sensing is becoming a vital underpinning for various emerging applications in smart homes, such as security surveillance and human behavior analysis. However, prevailing approaches mainly rely on video camera, ambient sensors, or wearable devices, which either requires arduous deployment or arouses privacy concerns. In this paper, we present a novel real-time, device-free, and privacy-preserving WiFi-enabled Internet of Things platform for occupancy sensing, which can promote a myriad of emerging applications. It is designed to achieve an optimal tradeoff between performance and scalability. Our system empowers commercial off-the-shelf WiFi routers to collect channel state information (CSI) measurements and provides an efficient cloud server for computing via a lightweight communication protocol. To demonstrate the usefulness of our platform, an occupancy detection system is developed by exploiting the CSI curve of human presence. Furthermore, we also design an innovative activity recognition system based on our platform and machine learning techniques with high availability and extensibility. In the evaluation, the experimental results show that our platform enables these applications efficiently, with the accuracy of 96.8% and 90.6% in terms of occupancy detection and recognition, respectively.

Fingerprinting based indoor positioning system is gaining more research interest under the umbrella of location-based services. However, existing works have certain limitations in addressing issues such as noisy measurements, high computational complexity, and poor generalization ability. In this work, a random vector functional link network based approach is introduced to address these issues. In the proposed system, a subset of informative features from many randomized noisy features is selected to both reduce the computational complexity and boost the generalization ability. Moreover, the feature selector and predictor are jointly learned iteratively in a single framework based on an augmented Lagrangian method. The proposed system is appealing as it can be naturally fit into parallel or distributed computing environment. Extensive real-world indoor localization experiments are conducted on users with smartphone devices and results demonstrate the superiority of the proposed method over the existing approaches.

This paper is concerned with solving a large category of convex optimization problems using a group of agents, each only being accessible to its individual convex cost function. The optimization problems are modeled as minimizing the sum of all the agents' cost functions. The communication process between agents is described by a sequence of time-varying yet balanced directed graphs which are assumed to be uniformly strongly connected. Taking into account the fact that the communication channel bandwidth is limited, for each agent we introduce a vector-valued quantizer with finite quantization levels to preprocess the information to be exchanged. We exploit an event-triggered broadcasting technique to guide information exchange, further reducing the communication cost of the network. By jointly designing the dynamic event-triggered encoding-decoding schemes and the event-triggered sampling rules (to analytically determine the sampling time instant sequence for each agent), a distributed subgradient descent algorithm with constrained information exchange is proposed. By selecting the appropriate quantization levels, all the agents' states asymptotically converge to a consensus value which is also the optimal solution to the optimization problem, without committing saturation of all the quantizers. We find that one bit of information exchange across each connected channel can guarantee that the optimiztion problem can be exactly solved. Theoretical analysis shows that the event-triggered subgradient descent algorithm with constrained data rate of networks converges at the rate of O(lnt√t). We supply a numerical simulation experiment to demonstrate the effectiveness of the proposed algorithm and to validate the correctness of theoretical results.

This paper presents an error-constrained line-of-sight (ECLOS) path-following control method for a surface vessel subject to uncertainties, disturbances, and actuator saturation and faults. Based on a cascaded three degrees-of-freedom model of surface vessel, the backstepping technique is adopted as the main control framework. Error constraint of the vessel position is handled by integrating a novel tan-type barrier Lyapunov function. The proposed ECLOS method is in accordance with the classical line-of-sight method where no constraint is imposed. A nonlinear disturbance observer is developed to estimate the lumped disturbance that comprises the effects of parametric uncertainties, external environment disturbances, and actuator saturation and faults. It is proved that under the proposed control, the constrained requirements on the vessel position error are never violated and all closed-loop signals are uniformly ultimately bounded, regardless of fully actuated or under-actuated control configuration. Simulation results and comparisons illustrate the effectiveness and advantages of the proposed ECLOS path-following method.

In the era of Internet of Things, crowd counting, which estimates the number of people within a region, becomes the underpinning for many emerging applications, such as occupancy estimation in smart building and queuing management and product placement in shopping center. Existing vision based crowd counting schemes require favorable lighting conditions and also raise privacy concerns. RF based approaches rely on specialized sensors and require users to carry RF devices. Thus, an accurate, reliable and non-intrusive crowd counting scheme is still desired. In this paper, we propose FreeCount, a device-free crowd counting scheme that is able to precisely estimate the number of people within a region using only commodity WiFi routers. To this end, the channel state information (CSI) data in PHY layer is obtained directly by upgrading the router's software. We propose an information theory based feature selection scheme to select the most representative features that are sensitive to human motion. To build a classifier that is robust to temporal and environmental disparities, we adopt transfer kernel learning, which minimizes the difference between the source and target distributions in the reproducing kernel Hilbert space, is adopted to process the real-time CSI feature data. Experiments were conducted in moderate sized rooms and the results demonstrated that FreeCount is able to accurately estimate the number of people with 96% crowd counting accuracy consistently over temporal and environmental variation.

Human activity recognition is becoming the vital underpinning for a myriad of emerging applications in the field of human-computer interaction, mobile computing, and smart grid. Besides the utilization of up-to-date sensing techniques, modern activity recognition systems also require a machine learning (ML) algorithm that leverages the sensory data for identification purposes. In view of the unique characteristics of the measurement data and the ML challenges thereof, we propose a non-intrusive human activity recognition system that only uses existing commodity WiFi routers. The core of our system is a novel multiple kernel representation learning (MKRL) framework that automatically extracts and combines informative patterns from the Channel State Information (CSI) measurements. The MKRL firstly learns a kernel string representation from time, frequency, wavelet, and shape domains with an efficient greedy algorithm. Then it performs information fusion from diverse perspectives based on multi-view kernel learning. Moreover, different stages of MKRL can be seamlessly integrated into a multiple kernel learning framework to build up a robust and comprehensive activity classifier. Extensive experiments are conducted in typical indoor environments and the experimental results demonstrate that the proposed system outperforms existing methods and achieves a 98\% activity recognition accuracy.

In condition monitoring of electrical machines, wireless sensors enables access to internal signals that are relatively unexplored compared to the conventional external measurements. This paper demonstrates the application of wireless sensor to measure the rotor field winding current in a Brushless Synchronous Generator and compare the fault detection capability with the stator exciter field current. The faults considered are the stator armature and rotor field winding inter-turn short circuit fault. From the experimental investigation, the rotor field current measurements obtained through a wireless sensor is found to be more effective in tracking the evolution of a fault.

This paper proposes an ultra-wideband (UWB) aided localization and mapping system that leverages on inertial sensor and depth camera. Inspired by the fact that visual odometry (VO) system, regardless of its accuracy in the short term, still faces challenges with accumulated errors in the long run or under unfavourable environments, the UWB ranging measurements are fused to remove the visual drift and improve the robustness. A general framework is developed which consists of three parallel threads, two of which carry out the visualinertial odometry (VIO) and UWB localization respectively. The other mapping thread integrates visual tracking constraints into a pose graph with the proposed smooth and virtual range constraints, such that a bundle adjustment is performed to provide robust trajectory estimation. Experiments show that the proposed system is able to create dense drift-free maps in real-time even running on an ultra-low power processor in featureless environments.

In industrial automation, EtherCAT is a real-time communication protocol widely employed in the local computer networks for fast and predictable data communication between system processes. End-to-end delay is a key performance indicator to industrial applications as it reflects the real-time capability of the system. This paper explores end-to-end delays of EtherCAT-based control systems under different schemes of synchronization, including free-running, frame-driven and clock-driven schemes. A simulation study is presented to assess the delays using various schemes. Simulation results demonstrate that free-running and frame-driven approaches could be adopted in traditional automation applications, while clock-driven scheme can be advantageous in networked control systems where deterministic data communication is required.

EtherCAT is one of the real-time Ethernet protocols widely employed in industrial controls. However, its wireless extension is not straightforward due to the unique on-the-fly bus access method based on summation-frame. This paper explores the possibilities of interconnecting EtherCAT with wireless standards such as IEEE 802.11 and 802.15.4. The gateway design is presented and event-driven interconnection schemes using Type 12 process data or mailbox frame are described and discussed. The EtherCAT bus cycle after the wireless extension is formulated for both schemes. A case study of a typical hybrid network is presented to demonstrate the pros and cons of each scheme.

Existing human activity recognition approaches require either the deployment of extra infrastructure or the cooperation of occupants to carry dedicated devices, which are expensive, intrusive and inconvenient for pervasive implementation. In this paper, we propose SmartSense, a device-free human activity recognition system based on a novel machine learning algorithm with existing commercial off-the-shelf (COTS) WiFi routers. By exploiting the prevalence of existing WiFi infrastructure in buildings, we developed a novel OpenWrt based firmware for COTS WiFi routers to collect the CSI measurements from regular data frames. To identify different human activities, an automatic kernel representation learning method, namely auto-HSRL, is established to selection informative Hilbert space patterns from time, frequency, wavelet, and shape domains. A new information fusion tool based on multi-view kernel learning is proposed to combine the representations extracted from diverse perspectives and build up a robust and comprehensive activity classifier. Extensive experiments were conducted in an office and the experimental results demonstrate that SmartSense outperforms existing methods and achieves a 98% activity recognition accuracy.

Accurate Location Based Service (LBS) is one of the fundamental but crucial services in the era of Internet of Things (IoT). WiFi fingerprinting-based Indoor Positioning System (IPS) has become the most promising solution for indoor LBS. However, the offline calibrated received signal strength (RSS) radio map is unable to provide consistent LBS with high localization accuracy under various environmental dynamics. To address this issue, we propose TKL-WinSMS as a systematic strategy, which is able to realize robust and adaptive indoor localization in dynamic indoor environments. We developed a WiFi-based Non-intrusive Sensing and Monitoring System (WinSMS) that enables WiFi routers as online reference points by extracting real-time RSS readings among them. With these online data and labeled source data from the offline calibrated radio map, we further combine the RSS readings from target mobile devices as unlabeled target data, to design a robust localization model using an emerging transfer learning algorithm, namely transfer kernel learning (TKL). It is able to learn a domain-invariant kernel by directly matching the source and target distributions in the reproducing kernel Hilbert space instead of the raw noisy signal space. The resultant kernel can be used as input for the SVR training procedure. In this manner, the trained localization model can inherit the information from online phase to adaptively enhance the offline calibrated radio map. Extensive experiments were conducted and demonstrated that the proposed TKL- WinSMS is able to improve the localization accuracy by at least 26% compared with existing solutions under various environmental interferences.

Air traffic flow management (ATFM) is an important component in an air traffic control system and has significant effects on the safety and efficiency of air transportation. In this paper, we propose a distributed ATFM strategy to minimize the airport departure and arrival schedule deviations. The scheduling problem is formulated based on an en-route air traffic system model consisting of air routes, waypoints, and airports. A cell transmission flow dynamic model is adopted to describe the system dynamics under safety related constraints, such as the capacities of air routes and airports, and the aircraft speed limits. Our ATFM problem is formulated as an integer quadratic programming problem. To overcome the computational complexity associated with this problem, we first solve a relaxed quadratic programming problem by a distributed approach based on Lagrangian relaxation. Then a heuristic forward-backward propagation algorithm is proposed to obtain the final integer solution. Experimental results demonstrate the effectiveness of the proposed scheduling strategy.

This paper develops an off-policy reinforcement learning (RL) algorithm to solve optimal synchronization of multiagent systems. This is accomplished by using the framework of graphical games. In contrast to traditional control protocols, which require complete knowledge of agent dynamics, the proposed off-policy RL algorithm is a model-free approach, in that it solves the optimal synchronization problem without knowing any knowledge of the agent dynamics. A prescribed control policy, called behavior policy, is applied to each agent to generate and collect data for learning. An off-policy Bellman equation is derived for each agent to learn the value function for the policy under evaluation, called target policy, and find an improved policy, simultaneously. Actor and critic neural networks along with least-square approach are employed to approximate target control policies and value functions using the data generated by applying prescribed behavior policies. Finally, an off-policy RL algorithm is presented that is implemented in real time and gives the approximate optimal control policy for each agent using only measured data. It is shown that the optimal distributed policies found by the proposed algorithm satisfy the global Nash equilibrium and synchronize all agents to the leader. Simulation results illustrate the effectiveness of the proposed method.

WiFi fingerprinting-based indoor positioning system (IPS) has become the most promising solution for indoor localization. However, there are two major drawbacks that hamper its large-scale implementation. First, an offline site survey process is required which is extremely time-consuming and labor-intensive. Second, the RSS fingerprint database built offline is vulnerable to environmental dynamics. To address these issues comprehensively, in this paper, we propose WinIPS, a WiFi-based non-intrusive IPS that enables automatic online radio map construction and adaptation, aiming for calibration-free indoor localization. WinIPS can capture data packets transmitted in existing WiFi traffic and extract the RSS and MAC addresses of both WiFi access points (APs) and mobile devices in a non-intrusive manner. APs can be used as online reference points for radio map construction. A novel Gaussian process regression model is proposed to approximate the non-uniform RSS distribution of an indoor environment. Extensive experiments were conducted, which demonstrated that WinIPS outperforms existing solutions in terms of both RSS estimation accuracy and localization accuracy.

In the era of Internet of Things, WiFi fingerprinting based indoor positioning system (IPS) has been recognized as the most promising IPS for indoor location-based service. Fingerprinting-based algorithms critically rely on a fingerprint database built from machine learning methods, and extreme machine learning (ELM) is preferred for its fast training speed. However, traditional WiFi based IPS usually requires a central server to collect and process data, which is tremendously vulnerable to server breakdown and communication link failure. To address this issue, we propose Consensus-based Parallel ELM (CPELM) to enhance the robustness by distributing the data on different computation nodes. Specifically, each node keeps updating the corresponding terms in the ELM regression equation as a weighted average of those from neighboring nodes based on the distributed consensus iterative scheme. Upon the agreement of the regression equation within the network, the output weight of ELM can be calculated on some nodes and propagated to other nodes. Extensive simulation with real data has demonstrated that CPELM is able to produce same level of localization accuracy as centralized ELM without incurring additional computational cost, and in the meanwhile provides more robustness to the entire IPS in case of server breakdown and link failures.

This paper is concerned with cooperative output regulation of heterogeneous multi-agent systems. Agents are allowed to be heterogeneous general linear time-invariant systems and the communication graph is not restricted to the acyclic type. New distributed state-feedback controllers with extra scalar parameters are constructed. A global sufficient solvability condition is first derived and simplified stability conditions for closed-loop poles in a specified region are then obtained in terms of an H∞-type performance of local sub-systems coupled through a matrix associated with the graph. Linear matrix inequality conditions are further presented for allocating the H∞-type performance levels and designing controllers for each sub-system. A numerical example is presented for illustrating the advantages of the proposed design method. Both continuous- and discrete-time multi-agent systems are investigated in a unified framework.

A key problem in event-triggered control of multi-agent systems is to design triggering conditions. We first give an overview of existing triggering conditions used in the literature. However, not all existing triggering conditions can both relax continuous communication between neighboring agents and admit a positive lower bound of inter-event times. Then, we propose two new triggering conditions based on edge information rather than neighbor information, and show that Zeno behavior is ruled out by using a time-dependent threshold and a periodic event detector, respectively. Moreover, we list some open problems which are worth the effort to launch future investigations.

We study a distributed constrained optimal consensus problem of discrete-time multi-agent systems under fixed communication delays. Specifically, the total cost is expressed as the sum of individual cost of each agent, and only part of agents have access to the constraint. The constrained optimal consensus is solved if the final consensus value falls within the constraint, and in the meanwhile minimizes the total cost. Based on consensus method and subgradients, we propose a distributed two-step update scheme in which the state of each agent is firstly averaged with the delayed state information from neighbors, followed by a decaying subgradient descent from the individual cost, together with a movement along the projection direction if the agent can access the constraint. We show that the distributed constrained optimal consensus problem under fixed communication delays can be solved if the fixed network is balanced and contains a spanning tree, the constraint is accessible by at least one agent, and the gain on subgradient is decaying but persistent. Simulation results are provided to verify our conclusion.

In this paper, an improved direction-of-arrival (DOA) estimation algorithm for circular and non-circular signals is proposed. Most state-of-the-art algorithms only deal with the DOA estimation problem for the maximal non-circularity rated and circular signals. However, common non-circularity rated signals are not taken into consideration. The proposed algorithm can estimates not only the maximal non-circularity rated and circular signals, but also the common non-circularity rated signals. Based on the property of the non-circularity phase and rate, the incident signals can be divided into three types as mentioned above, which can be estimated separately. The interrelationship among these signals can be reduced significantly, which means the resolution performance among different types of signals is improved. Simulation results illustrate the effectiveness of the proposed method.

The authors have recently proposed two kinds of gridless sparse methods for direction of arrival (DOA) estimation that exploit joint sparsity among snapshots and completely resolve the grid mismatch issue of previous grid-based sparse methods. One is based on covariance fitting from a statistical perspective and termed as the gridless SPICE (GL-SPICE, GLS); the other uses deterministic atomic norm optimization which extends the recent super-resolution and continuous compressed sensing framework from the single to the multi-snapshot case. In this paper, we unify the two techniques by interpreting GLS as atomic norm methods in various scenarios. As a byproduct, we are able to provide theoretical guarantees of GLS for DOA estimation in the case of limited snapshots.

This paper concerns the line spectral estimation problem within the recent super-resolution framework. The frequencies of interest are assumed to follow a prior probability distribution. To effectively and efficiently exploit the prior information, we devise a weighted atomic norm approach that is physically sound and can be formulated as convex programming like the standard atomic norm method. Numerical simulations are provided to demonstrate the superior performance of the proposed approach in accuracy and speed compared to the state-of-the-art.

Mean Radiant Temperature (MRT) is an important factor of Fanger's PMV model, which is the most popular method to study human thermal comfort. However, it has often been a practice to assume MRT equal to air temperature (Ta) during indoor thermal studies. In this paper, we have studied the consequences of this simplistic assumption on the thermal comfort of occupants in air-conditioned buildings. A worldwide database of about 10000 occupants covering 9 climatic zones and 4 seasons has been studied. The effect on comfort indices-Predicted Mean Vote (PMV), Actual Mean Vote (AMV), Thermal Acceptability Vote (TSA) and Thermal Preference Vote (MCI) are presented. It is observed that even a small difference in Taand MRT can lead to significant error in determination of thermal comfort. A correlation study between AMV and the six Macpherson factors reveals that MRT has the highest positive correlation with the thermal sensation reported by the occupants. Study of TSA and MCI show that the assumption is more likely to affect comfort level determination in the uncomfortable range.

Existing WiFi fingerprinting-based Indoor Positioning System (IPS) suffers from the vulnerability of environmental dynamics. To address this issue, we propose TKL-WinSMS as a systematic strategy, which is able to realize robust and adaptive localization in dynamic indoor environments. We developed a WiFi-based Non-intrusive Sensing and Monitoring System (WinSMS) that enables COTS WiFi routers as online reference points by extracting real-time RSS readings among them. With these online data and labeled source data from the offline calibrated radio map, we further combine the RSS readings from target mobile devices as unlabeled target data, to design a robust localization model using an emerging transfer learning algorithm, namely transfer kernel learning (TKL). It can learn a domain-invariant kernel by directly matching the source and target distributions in the reproducing kernel Hilbert space instead of the raw noisy signal space. By leveraging the resultant kernel as input for the SVR training, the trained localization model can inherit the information from online phase to adaptively enhance the offline calibrated radio map. Extensive experimental results verify the superiority of TKL-WinSMS in terms of localization accuracy compared with existing solutions in dynamic indoor environments.

The explosive proliferation of mobile devices and the popularity of social networks have spurred extensive demands on Location Based Services (LBSs) in recent decades. The IEEE 802.11 (WiFi) based Indoor Positioning Systems (IPSs) are gaining popularity because of the wide and ubiquitous availability of WiFi infrastructures in indoor environments. Most of IPSs are adopting the fingerprinting approach to mitigate pervasive indoor multipath effects. However, the heterogeneity of mobile devices significantly degrades the localization performance of the fingerprinting approach. In this paper, we apply the Procrustes analysis method to transform the WiFi received signal strengths (RSSs) to a new type of standard location fingerprints which are tolerant of the heterogeneity of various devices. Then, a robust indoor positioning algorithm based on the standardized location fingerprints and the weighted k nearest neighbor (WKN-N) method is proposed. Extensive experiments are carried out and show that the standardized location fingerprints and the proposed positioning system address the device heterogeneity issue satisfactorily.

Accurate occupancy information of indoor environments is one of the key prerequisites for many pervasive and context-aware services, e.g. smart building/home systems. Some of the existing occupancy inference systems can achieve impressive accuracy, but they either require labour-intensive calibration phases, or need to install bespoke hardware such as CCTV cameras, which are privacy-intrusive by default. In this paper, we present the design and implementation of a practical end-to-end occupancy inference system, which requires minimum user effort, and is able to infer room-level occupancy accurately with commodity WiFi infrastructure. Depending on the needs of different occupancy information subscribers, our system is flexible enough to switch between snapshot estimation mode and continuous inference mode, to trade estimation accuracy for delay and communication cost. We evaluate the system on a hardware testbed deployed in a 600m2workspace with 25 occupants for 6 weeks. Experimental results show that the proposed system significantly outperforms competing systems in both inference accuracy and robustness.

The increasing demands of location-based services have spurred the rapid development of indoor positioning system and indoor localization system interchangeably (IPSs). However, the performance of IPSs suffers from noisy measurements. In this paper, two kinds of robust extreme learning machines (RELMs), corresponding to the close-to-mean constraint, and the small-residual constraint, have been proposed to address the issue of noisy measurements in IPSs. Based on whether the feature mapping in extreme learning machine is explicit, we respectively provide random-hidden-nodes and kernelized formulations of RELMs by second order cone programming. Furthermore, the computation of the covariance in feature space is discussed. Simulations and real-world indoor localization experiments are extensively carried out and the results demonstrate that the proposed algorithms can not only improve the accuracy and repeatability, but also reduce the deviation and worst case error of IPSs compared with other baseline algorithms.

The Vandermonde decomposition of Toeplitz matrices, discovered by Carathéodory and Fejér in the 1910s and rediscovered by Pisarenko in the 1970s, forms the basis of modern subspace methods for 1-D frequency estimation. Many related numerical tools have also been developed for multidimensional (MD), especially 2-D, frequency estimation; however, a fundamental question has remained unresolved as to whether an analog of the Vandermonde decomposition holds for multilevel Toeplitz matrices in the MD case. In this paper, an affirmative answer to this question and a constructive method for finding the decomposition are provided when the matrix rank is lower than the dimension of each Toeplitz block. A numerical method for searching for a decomposition is also proposed when the matrix rank is higher. The new results are applied to study the MD frequency estimation within the recent super-resolution framework. A precise formulation of the atomic $\ell _{0}$ norm is derived using the Vandermonde decomposition. Practical algorithms for frequency estimation are proposed based on the relaxation techniques. Extensive numerical simulations are provided to demonstrate the effectiveness of these algorithms compared with the existing atomic norm and subspace methods.

Indoor positioning system (IPS) has become one of the most attractive research fields due to the increasing demands on location-based services (LBSs) in indoor environments. Various IPSs have been developed under different circumstances, and most of them adopt the fingerprinting technique to mitigate pervasive indoor multipath effects. However, the performance of the fingerprinting technique severely suffers from device heterogeneity existing across commercial off-the-shelf mobile devices (e.g., smart phones, tablet computers, etc.) and indoor environmental changes (e.g., the number, distribution and activities of people, the placement of furniture, etc.). In this paper, we transform the received signal strength (RSS) to a standardized location fingerprint based on the Procrustes analysis, and introduce a similarity metric, termed signal tendency index (STI), for matching standardized fingerprints. An analysis of the capability of the proposed STI to handle device heterogeneity and environmental changes is presented. We further develop a robust and precise IPS by integrating the merits of both the STI and weighted extreme learning machine (WELM). Finally, extensive experiments are carried out and a performance comparison with existing solutions verifies the superiority of the proposed IPS in terms of robustness to device heterogeneity.

The super-resolution theory developed recently by Candès and Fernandes-Granda aims to recover fine details in a sparse frequency spectrum from coarse scale information. The theory was then extended to the cases of compressive samples and/or multiple measurement vectors. However, the existing atomic norm (or total variation norm) techniques succeed only if the frequencies are sufficiently separated, prohibiting commonly known high resolution. In this paper, a reweighted atomic-norm minimization (RAM) approach is proposed which iteratively carries out atomic norm minimization (ANM) with a sound reweighting strategy that enhances sparsity and resolution. It is demonstrated analytically and via numerical simulations that the proposed method achieves high resolution with application to DOA estimation.

In this paper, we consider the distributed parameter estimation problem over sensor networks in the presence of quantized data and directed communication links. We propose a two-stage algorithm aiming at achieving the centralized sample mean estimate in a distributed manner. The running average technique is utilized in the proposed algorithm to smear out the randomness caused by the probabilistic quantization scheme. It is shown that the centralized estimate can be achieved in the mean square sense, which is not observed in the conventional consensus algorithms. Simulation results are presented to illustrate the effectiveness of the proposed algorithm and highlight the improvements by using running average technique.

Many companies face issues today in locating their assets and inventories in storage racks. In this study, we explored the use of passive UHF RFID for locating relative positions of items on storage racks. The storage rack with UHF RFID localization function is named as smart rack in this study. This paper addresses the technical issue of how to make use of the reference RFID tag signals to improve the accuracy of the position estimation of the target RFID tag. A passive UHF RFID smart rack test bed was built to quantify the UHF RFID signals variance in the test bed. The signals are analyzed for the development of next generation RFID system for smart rack applications.

The Vandermonde decomposition of Toeplitz matrices, discovered by Carathéodory and Fejér in the 1910s and rediscovered by Pisarenko in the 1970s, forms the basis of modern subspace methods for 1D frequency estimation. Many related numerical tools have also been developed for multi-dimensional (MD), especially 2D, frequency estimation; however, a fundamental question has remained unresolved as to whether an analog of the Vandermonde decomposition holds in the MD case. In this paper, an affirmative answer to this question and a constructive method for finding the decomposition are provided under appropriate conditions. The new result is also used to study MD frequency estimation from compressive data within the recent super-resolution framework. A systematic approach is proposed and a numerical simulation is provided to demonstrate its effectiveness compared to the existing atomic norm method.

This paper is concerned with the nonfragile distributed H∞filtering problem for a class of discrete-time Takagi-Sugeno (T-S) systems in sensor networks. Additive filter gain uncertainties that reflect imprecision in filter implementation are considered. Based on the robust control approach, sufficient conditions are obtained to ensure that the filtering error system is asymptotically stable with a prescribed H∞performance level and the eigenvalues of the filtering error system in a given circular region. The filter parameters are determined by solving a set of linear matrix inequalities. A simulation study on the nonlinear tunnel diode circuit system is presented to show the effectiveness of the proposed design method.

This paper focuses on the problem of source localization using time-difference-of-arrival (TDOA) measurements in both 2-D and 3-D spaces. Different from existing studies where the variance of TDOA measurement noises is assumed to be independent of the associated source-to-sensor distances, we consider the more realistic model where the variance is a function of the source-to-sensor distances, which dramatically complicates TDOA-based source localization. After formulating the distance-dependent noise model, we prove that using the extra information about the source location in the functional variance improves the estimation accuracy of TDOA-based source localization, but contributes little under a sufficiently small noise level. Further, we theoretically analyze the problem of optimal sensor placement, and derive the necessary and sufficient conditions for optimizing localization performance under different circumstances. Then, a localization scheme based on the iteratively reweighted generalized least squares (IRGLS) method is proposed to efficiently exploit the extra source location information. Finally, a simulation analysis confirms our theoretical studies, and shows that the performance of the proposed localization scheme is comparable to the Cramer-Rao lower bound (CRLB) given moderate TDOA measurement noises.

In this paper, we investigate source localization for wireless sensor networks based on received signal strength. We first formulate the localization problem as the intersection computation of a group of sensing rings, and then convert this non-convex problem into two weighted convex optimization problems. We next propose a unified distributed alternating projection algorithm to solve the resulting weighted optimization problems, where sensor nodes can communicate only locally with their neighbors over a time-varying jointly-connected topology. We also show that sensor nodes' estimates can achieve consensus on a possible minimizer. Both theoretical analysis and some comparative simulations reveal that the proposed approach has good estimation performance in both the consistent and inconsistent cases.

This article describes a method for indoor positioning of human-carried active Radio Frequency Identification (RFID) tags based on the Sampling Importance Resampling (SIR) particle filtering algorithm. To use particle filtering methods, it is necessary to furnish statistical state transition and observation distributions. The state transition distribution is obstacle-aware and sampled from a precomputed accessibility map. The observation distribution is empirically determined by ground truth RSS measurements while moving the RFID tags along a known trajectory. From this data, we generate estimates of the sensor measurement distributions, grouped by distance, between the tag and sensor. A grid of 24 sensors is deployed in an office environment, measuring Received Signal Strength (RSS) from the tags, and a multithreaded program is written to implement the method. We discuss the accuracy of the method using a verification data set collected during a field-operational test.

Unmanned Aerial Vehicles have a lot of potentials in outdoor applications. However, uncertainties such as wind disturbances and mass change when performing some particular tasks, greatly affect their tracking performance. This paper presents a methodology using L1adaptive control to address some of the robustness issues of the quadcopter in outdoor flight which significantly improves the performance comparing to the baseline controller. Simulation and flight tests verify the potential of the presented controller.

Extreme learning machine (ELM) as an emergent technology has shown its good performance in regression applications as well as in large dataset classification applications. It has been broadly embedded in many applications due to its fast speed of computation and accuracy. How to make good use of machine learning techniques in Indoor Positioning System (IPS) is a hot research topic in recent years. Some existing IPSs have already adopted ELM, but it suffers from signal variation and environmental dynamics in indoor settings. In this paper, extreme learning machine with dead zone (DZ-ELM) is proposed to address this problem. The consistency of this approach should be applied is studied. Simulations are also conducted to compare the performance of DZ-ELM and ELM. Lastly, real-world experimental results show that the proposed algorithm can not only provide higher accuracy but also improve the repeatability of IPSs.

In this paper, we investigate the behavior of agents in mean field games where each agent evolves according to a dynamic equation containing the input average and seeks to minimize its long time average (LTA) cost encompassing a population state average (PSA), which is also known as the mean field term. Due to the informational burden resulting from the PSA coupling to the states of all agents, our idea is to find a deterministic function φ to approximate it. It is shown that φ is an approximation of the PSA as the population size N goes to infinity. The resulting decentralized mean field control laws lead the system to achieve mean-consensus asymptotically as time goes to infinity. Furthermore, the optimal controls generate an almost sure asymptotic Nash equilibrium, which implies that the LTA cost of each agent can reach its minimal value as the number of agents increases to infinity. Finally, we consider the socially optimal case where the basic objective is to minimize the social cost as the sum of the individual LTA cost containing the PSA. In this case, it is shown that the decentralized mean field social control strategies are the same as the mean field Nash controls for infinite population systems.

Advances in microelectronics and communication technologies make networked sensing and control feasible. Well known examples include mobile sensor networks for environment monitoring, unmanned aircraft in search and rescue operations, arrays of micro satellites that form a distributed large aperture radar, and vehicle platooning in intelligent transportation systems. The distributed nature of information processing, sensing and actuation makes these applications a significant departure from the traditional centralized control system paradigm. In this talk, we shall discuss some recent developments in distributed cooperative control and optimization and their applications.

Dual-stage actuation consisting of a PZT Microactuator (MA) and a Voice Coil Motor (VCM) has been used to improve the servo bandwidth and the disturbance rejections of Hard Disk Drivers (HDDs). However, the hysteresis in PZT MA limits the performance that can be achieved. In this paper, a Hammerstein model structure consisting of static hysteresis nonlinear block and dynamic linear block is used to model the PZT MA in HDDs and the identification method is also given. The nonlinear subsystem in the Hammerstein model is represented by Modified Prandtl-Ishlinskii (MPI) hysteresis model. It is proved that the proposed model is equivalent to the physical system. A hysteresis compensator is designed based on the proposed model. By the hysteresis compensation, the effects of hysteresis on the frequency responses of the PZT MA are measured. It is shown that apparent phase lead is achieved by hysteresis compensation, which is useful to improve the performance of control system.

This paper presents the detection and localization methods of entrance and staircase markers for the team E-Mobile in TechX Challenge 2013. Autonomous vehicles are required to detect and locate traffic cones beside the indoor entrance and staircase. One big challenge is from the unpredictable lighting conditions and environment. Different practical techniques such as color space selection, segmentation, shape analysis, distance estimation, and detector training are combined to obtain good detection rate and localization accuracy. The proposed methods can achieve satisfactory performance in real-world experiments.

This paper studies the blind identification of multi-channel FIR systems in the context of sensor networks. Distributed identification algorithms are developed for both noise-free and noise-contaminated networked systems. The proposed algorithms distribute the data storage and computational load among multiple agents connected by a specified topology, and are fulfilled via information exchanges among neighboring agents without the need of fusion centers. In the presence of measurement noises, a stabilized distributed algorithm is provided which can avoid trivial estimations of the multiple channels. In addition, convergence properties of the proposed algorithms are provided, and simulation examples are given to show the performances of the proposed algorithms.

Real-time protocols originated from standard Ethernet technology have been gradually developed over the last decade. Candidates of real-time Ethernet protocols are EtherCAT and PROFINET IRT from Beckhoff and Siemens, respectively. In contrast to standard Ethernet, data-link layers of both Ethernet-like networks are modified from IEEE 802.3 in order to improve real-time capabilities for critical communications. Network delay refers to data transfer time on a network. It is of great importance to assess this parameter for industrial networks as it can possibly make significant impact to the control system performance. This paper gives network delay analysis on EtherCAT and PROFINET IRT in the context of industrial applications. Delays are formulated and then evaluated under both synchronous and asynchronous system configurations. Simulation results demonstrate that both protocols are of realtime guarantees in data transfer. However, each protocol has its own characteristics and strong points due to different MAC layer design.

Communication networks are essentially the backbone of industrial automation systems such as Baggage Handling Systems (BHS) and are experiencing technological transition from traditional fieldbus to Real-time Ethernet (RTE). EtherCAT is evolving into one of the fastest RTE protocols due to its unique “processing on-the-fly” characteristic. This paper presents how EtherCAT can be the key technology for the innovative next generation BHS. Network performance indices such as communication delay and cycle time are analyzed and discussed in comparison with fieldbus to illustrate the applicability of EtherCAT in BHS. A case study of baggage handling application is given to demonstrate the advantages of EtherCAT-based architecture.

In recent years, applying RFID technology to develop an Indoor Positioning System (IPS) has become a hot research topic. The most prominent advantage of active RFID IPS comes from its unique identification of different objects in indoor environment. However, certain drawbacks of existing RFID IPSs, such as high cost of RFID readers and active tags, as well as heavy dependence on the density of reference tags to provide the location based service, largely limit the applications of active RFID IPS. In order to overcome these drawbacks, we develop a cost-efficient RFID IPS by using cheaper active RFID tags, sensors and reader. In addition, one localization algorithm: integrated Weighted Path Loss (WPL) - Extreme Learning Machine (ELM) which combines the fast estimation of WPL and the high localization accuracy of ELM is proposed. According to the algorithm, an indoor environment is divided into small zones firstly and an ELM model is developed for each zone during the offline phase. During the online phase, the WPL approach is used to determine the zone of the target primarily, then the ELM model of that zone is deployed to provide the final estimated location of the target. Based on our experimental result, this integrated algorithm provides a higher localization efficiency and accuracy than existing approaches.

This study considers a networked system in which the measurement suffers from one‐step delay and packet dropouts because of the unreliability of the network. A new model applied to describe the arrival conditions of the measurements is proposed. Based on the new model and using a state augmentation method, optimal linear filter, predictor and smoother are obtained. A sufficient condition for the convergence of the system is given. Finally, the simulation results show the effectiveness of the proposed algorithms.

This letter is concerned with source localization based on time-difference-of-arrival (TDOA) measurements from spatially separated sensors in a wireless sensor network (WSN). Most of the existing works adopt a centralized sensor pairing strategy, where one sensor node is chosen as the common reference. However, due to the bandwidth and power constraints of multihop WSNs, it is well known that this kind of centralized methods is energy consuming due to the need of single and multihop transmissions of raw measurement data. In this letter, we propose a decentralized in-network sensor pairing method to acquire TDOA measurements for source localization. It is proved that the proposed decentralized in-network sensor pairing method can result in the same Cramer-Rao-Bound (CRB) as the centralized one at a far less communication cost.

Source localization is an important application of wireless sensor networks (WSNs). Many types of sensors can be used for source localization, e.g., range sensors, bearing sensors and time-difference-of-arrival (TDOA) based sensors, etc. It is well known that relative sensor-source geometry can significantly affect the performance of any particular localization algorithm. Existing works in the literature mainly deal with geometry analysis for homogeneous sensors. However, in real applications, different types of sensors may be utilized for source localization. Hence, in this paper, we consider the optimal sensor placement problem in heterogeneous sensor networks (HSNs), where two types of sensors are deployed for source localization. Relative optimal sensor-source configurations with the minimum number of sensors for source localization are identified under the Doptimality criterion with potential extensions to the general case. Explicit characterizations of optimal sensor-source geometries are given for hybrid range and bearing sensors, hybrid bearing and TDOA based sensors as well as co-located hybrid range and bearing sensors, respectively. The results of this work can be applied to the sensor path planning problem for optimal source localization.

This paper is concerned with resonance in-phase property of a VCM (voice coil motor) plant system in the sense of control performance in HDDs (hard disk drives). Its relationships with the optimal performance level γopt, the stability margins and the disturbance rejection capability are revealed. It is found that the main resonance being in-phase is particularly beneficial to rejection of the narrow-band disturbances with frequencies near plant resonances. In order to meet the requirement on the inphase property, a partial model matching method is proposed. This model matching problem is solved by an H∞method using an linear matrix inequality approach. The partial model matching method is then applied to the VCM plant system. We especially take into account the in-phase case for the purpose to improve the system ability to attenuate high frequency disturbance. For the new system designed using the proposed model matching method, a feedback controller and a group peak filter are designed to attenuate the disturbance near the plant resonances. The advantages of the in-phase resonances are illustrated, when compared with the original plant.

In this paper, we investigate the effect of voltage and current amplifiers for PZT microactuators on the control performance of dual-stage servo systems in hard disk drives (HDDs), where the PZT microactuator is used as a secondary actuator and works together with the primary actuator of voice coil motor (VCM). First, the PZT microactuator's behavior in terms of motion linearization and frequency responses is experimentally studied and compared when it is driven by a conventional voltage amplifier and a charge or current amplifier. It is found that the PZT microactuator with current amplifier has less hysteresis than with voltage amplifier and its first resonance is relatively smaller. Inspired by this difference, the control performance of the dual-stage servo systems in track-seeking and track-following is then compared between the two driving methods for the PZT microactuator.

This paper addresses vision-based cooperative search for multiple mobile ground targets by a group of unmanned aerial vehicles (UAVs) with limited sensing and communication capabilities. The airborne camera on each UAV has a limited field of view and its target discriminability varies as a function of altitude. First, a general target detection probability model is built based on the physical imaging process of a camera. By dividing the whole surveillance region into cells, a probability map can be formed for each UAV indicating the probability of target existence within each cell. Then, we propose a distributed probability map updating model which includes the fusion of measurement information, information sharing among neighboring agents, information decaying and transmission due to environmental changes such as the target movement. Furthermore, we formulate the target search problem by multiple agents as a cooperative coverage control problem by optimizing the collective coverage area and the detection performance. The proposed map updating model and the cooperative control scheme are distributed, i.e., assuming that each agent only communicates with its neighbors within its communication range. Finally, the effectiveness of the proposed algorithms is illustrated by simulation.

This paper considers the average consensus problem for multi-agent systems with continuous-time first-order dynamics. The communication channels among the agents are constrained in which the exchanged information is quantized. In this paper, logarithmic quantization is considered in the communication channels, and sampled-data based protocol is applied. It is shown that as long as the sampling interval is small enough, the consensus protocol is admissible under arbitrarily coarse quantization. To be specific, the consensus error is uniformly bounded and is proportional to the quantization error and averaged initial value. Numerical examples are given to demonstrate the effectiveness of the protocol.

Source localization is an important application of wireless sensor networks (WSNs). Many types of sensors can be used for source localization, e.g. range-only sensors, bearing-only sensors and time-of-arrival (TOA) sensors, etc. It is well known that the relative sensor-source geometry can significantly affect the performance of any particular localization algorithm. Existing works in the literature mainly deal with the geometry analysis for a single type of sensors. However, in real applications, different types of sensors may be utilized for source localization simultaneously. Hence, in this paper, we consider the optimal sensor placement problem in heterogeneous sensor networks, where two types of sensors are deployed for source localization. Relative optimal sensor-source configurations with the minimum number of sensors for source localization, are identified under the D-optimality criterion with potential extensions to a general case. Explicit characterizations of optimal sensor-source geometries are given for hybrid range-only and bearing-only sensors as well as hybrid bearing-only and TOA sensors, respectively.

A multi-purpose 3D simulator named HNMSim (hybrid networked multi-agent system (MAS) simulator) with application in cooperative control is presented in this paper. Based on USARSim (Unified system for automation and robot simulation) [37], Unreal Engine [33], LabView [17], Matlab [20] and OMNet++ [22], this simulator creates a high-fidelity simulation environment for networked MAS. To provide assistant in research and education, we present three interfaces based on LabView, Matlab and OMNet++, respectively. The system structure (hardware-in-the-loop simulation) and design methodology are also briefly described. In addition, the network simulation is highlighted. Furthermore, we show its wide applications in networked MAS by demonstrating its usage in distributed formation control, distributed coverage control and search problem using multiple UAVs/UGVs.

The sparse signal recovery in standard compressed sensing (CS) requires that the sensing matrix is exactly known. The CS problem subject to perturbation in the sensing matrix is often encountered in practice and has attracted interest of researches. Unlike existing robust signal recoveries with the recovery error growing linearly with the perturbation level, this paper analyzes the CS problem subject to a structured perturbation to provide conditions for stable signal recovery under measurement noise. Under mild conditions on the perturbed sensing matrix, similar to that for the standard CS, it is shown that a sparse signal can be stably recovered by ℓ1minimization. A remarkable result is that the recovery is exact and independent of the perturbation if there is no measurement noise and the signal is sufficiently sparse. In the presence of noise, largest entries (in magnitude) of a compressible signal can be stably recovered. The result is demonstrated by a simulation example.

Acoustic source localization based on time difference of arrival (TDOA) measurements from spatially separated sensors is an important problem in wireless sensor networks (WSNs). While extensive research works have been performed on algorithm development, limited attention has been paid in how to form the sensor pairs. In the literature, most of the works adopt a centralized sensor pairing strategy, where only one common sensor node is chosen as the reference. However, due to the multi-hop nature of WSNs, it is well known that this kind of centralized signal processing method is power consuming since raw measurement data is involved in the transmissions. To reduce the requirements for both network bandwidth and power consumptions, we propose an in-network sensor pairing method to collect the TDOA measurements while guaranteeing the quality of source localization. The solution involves finding a minimal sized dominating set (MSDS) for a graph of the muti-hop network. It has been proved that in-network sensor pairing can result in the same Cramer-Rao-Bound (CRB) as the centralized one but at a far more less communication cost. Furthermore, the structure of the proposed in-network sensor pairing coincides with the decentralized source localization, which is an important application of our method.

The phase transition is a performance measure of the sparsity-undersampling tradeoff in compressed sensing (CS). This letter reports our first observation and evaluation of an empirical phase transition of thel1minimization approach to the complex valued CS (CVCS), which is positioned well above the known phase transition of the real valued CS in the phase plane. This result can be considered as an extension of the existing phase transition theory of the block-sparse CS (BSCS) based on the universality argument, since the CVCS problem does not meet the condition required by the phase transition theory of BSCS but its observed phase transition coincides with that of BSCS. Our result is obtained by applying the recently developed ONE-L1 algorithms to the empirical evaluation of the phase transition of CVCS.

Indoor positioning has emerged as a widely used application of Wi-Fi wireless networks. Fingerprinting techniques can provide a low-cost and high-accuracy localization solution by utilizing in-building communication infrastructures. However, existing fingerprinting localization algorithms are not resistant to outliers, for example, the accidental environment changes, access point (AP) attacks. Another drawback is that traditional K nearest neighbor (KNN) algorithm in the literature may not select the candidate reference points (RPs) correctly. In this paper, we propose a novel environmentally robust and attack resistant probabilistic fingerprinting localization method. In the offline phase, the distribution estimation of the signal strength is performed using probabilistic histogram method. Then in the online phase, a three-step location sensing method is proposed. In the first step, a simple and efficient outlier detection method named non-iterative “RANdom SAmple Consensus” (RANSAC) is run to detect and eliminate part of APs from which the signals measured are severely distorted by unexpected environment effects. In the second step, a novel region-based RP selection method which works like a “family of probability” is proposed to improve the possibility of the correctness of selection of the nearest RPs. In the final step, the location is obtained using a weighted-mean method. In the experiment section, we demonstrate the proposed method in our lab and find that the proposed strategies are resistant to outliers and can improve the localization accuracy effectively compared with existing methods.

This paper proposes filtering methods to cancel the impulsive disturbance contained in position error signal (PES) in hard disk drives (HDDs). The impulsive disturbances may be observed as a few single sudden changes or some consecutive changes in PES. The filtering includes impulsive disturbance identification and estimation. A recursive method is used to determine the dynamic boundaries for identification. Two methods are proposed for estimation: a linear interpolation and an adaptive least mean square (LMS) algorithm. The former is used to estimate the normal PES directly and the later is used to estimate the impulsive disturbance for cancellation. It turns out that these methods are able to effectively cancel the impulsive disturbance and do not affect the servo performance.

In this paper, we propose a systematic solution to the leader-follower consensus of a class of nonholonomic chained form systems. The control design, which is reminiscent of terminal sliding mode and multi-surface sliding mode control methods, is presented to guarantee the convergence of multiple sliding surfaces, which also implies the convergence of the proposed consensus error function. On these sliding surfaces, the desired leader-follower consensus can be reached for multi-agent network formed by the nonholonomic chained form systems.

In this paper, we consider formation control problem for multi-robot systems under an undirected communication network. All the robots will track a leader, while form a desired formation. The leader can be static or dynamic. A distributed formation controller with neighbors' input information is applied. For practical implementation, control input information from neighbors can only be received after some time delays. It is therefore shown that the distributed control protocol using time-delayed control input information from neighbors guarantees the formation of the multi-agent system for any nonnegative delay. We will implement the new protocol on the Amigo robots. The experimental results will demonstrate the effectiveness of the new protocol.

Deploying the sensor nodes at the best locations for random field reconstruction via sensor network is a fundamental task. One-dimensional random field is a stochastic process. In this paper, we first propose an optimal sensor deployment strategy for Wiener process estimation. The optimal locations for the deployed sensors are uniformly distributed in the field. In addition, we propose a suboptimal sensor deployment to estimate the Gaussian random field which is described as an Ornstein-Uhlenbeck process. We show that the suboptimal deployment strategy for Gaussian random field is also uniformly distributed. Several simulations show the performance of our proposed deployment strategies.

The present work fits within the general study of communication and control co-design, where the mean square stabilization of continuous-time networked control systems over stochastic multiplicative input channels is addressed. Motivated by the limited communication capacity and network resource allocation in multiple channel communication systems, we assume that the overall quality of service defined in this paper is fixed and can be assigned among the input channels. We show that there exists a minimal requirement on the overall quality of service for achieving the mean square stabilization of the networked system. For the case of static state feedback, a tight lower bound on the overall quality of service for mean square stabilization is derived in terms of the instability degree of the plant. In the case of output feedback, additional limitations are induced by nonminimum phase zeros, where both stabilization over a single-input channel and stabilization of essentially triangular plants over multi-input channels are studied. The application of the results to vehicle platooning is demonstrated.

In this paper, we extend the blending technique introduced in [2] to a general case. Different from, where all the columns of the distribution matrices of the exogenous inputs should be linearly independent of each other, we only require that part of the columns are linearly independent, and the other matrices have full column ranks. Based on this blending technique, we present a novel approach to the robust fault-tolerant control scheme and robust tracking problem. The novelty lies in the fact that we can separately design the controller gains for different performance indices, while simultaneously achieving all the indices via an elegant construction of an overall controller. The design procedure is based on LMI techniques and basic linear algebra tools.

This paper investigates the joint effects of agent dynamic and network topology on the consensusability of linear discrete-time multi-agent systems via relative output feedback. An observer-based distributed control protocol is proposed. A necessary and sufficient condition for consensusability under this control protocol is given, which explicitly reveals how the intrinsic entropy rate of the agent dynamic and the eigenratio of the undirected communication graph affect consensusability. As a special case, the discrete-time double integrator system is discussed where a simple control protocol directly using the two-step relative position feedback is provided to reach a consensus. The theoretic results are illustrated by a simulation example.

In this paper, rejecting multi-frequency narrow-band disturbances is formulated as a control blending problem according to each disturbance characteristic. Each disturbance rejection is accomplished by using H2optimal control method. Based on all the H2optimal controls, the blending technique is employed to yield a single controller which is capable to achieve the rejection for all disturbances. Rejections for two and three disturbances through the control design for the VCM actuator in a hard disk drive are taken as application examples in the current paper. Simulation and experimental results show that the ultimate controller results in a simultaneous attenuation to disturbances with frequencies higher or lower than the closed-loop system bandwidth. Moreover, the method turns out to be able to lift phase and thus prevent phase margin loss when it is used to deal with disturbances near bandwidth.

Quantized feedback control and quantized estimation have attracted a lot of attention in recent years with many results available on both research topics. In this paper, we investigate connections between quantized feedback control and quantized estimation and try to establish a possible separation principle for quantized output feedback which would allow the control design and state estimation to become independent in a networked control environment. We also consider the use of a variable rate finite-level logarithmic quantizer and show that this may approach the minimum averaged bit rate required for quantized feedback stabilization.

This paper is concerned with leader-follower finite-time consensus control of multi-agent networks with input disturbances. Terminal sliding mode control scheme is used to design the distributed control law. A new terminal slidingmode surface is proposed to guarantee finite-time consensus under fixed topology, with the common assumption that the position and the velocity of the active leader is known to its neighbors only. By using the finite-time Lyapunov stability theorem, it is shown that if the directed graph of the network has a directed spanning tree, then the terminal sliding mode control law can guarantee finite-time consensus even under the assumption that the time-varying control input of the active leader is unknown to any follower.

This paper studies the stabilization problem of networked control systems (NCSs) with bounded random time-varying delays. Both the uncertain time delay of the control channel and the known time-varying delay of the sensor channel are taken into consideration. This networked control system is formulated into a system with polytopic parameter uncertainties and a Markovian jump parameter. A mode-dependent Lyapunov-Krasovskii functional is used to design a mode-dependent static output feedback controller which stabilizes the NCS. A sufficient condition for stabilization is proposed via a parameterized bilinear matrix inequality (BMI) based approach. A numerical example is given to illustrate the effectiveness of the proposed method.

Self-servo track writing (SSTW) process for hard disk drives (HDDs) is described by using a two-dimensional (2D) model, with which the error propagation containment problem of the SSTW is formulated as 2D stabilization problem, and the position error minimization problem is formulated as 2D H2control problem. The adopted 2D controller is designed with easily implemented linear matrix inequality (LMI) approach. With the stored error information of previous track, the 2D control scheme is realizable. The simulation results demonstrate that the error propagation is prevented by the 2D control scheme and the positioning accuracy is remarkably improved by the 2D H2control scheme. Also, from the 2D controller servo performance is evaluated in usually concerned 1D sense

This paper is concerned with vibration control of a flexible spacecraft in the presence of parametric uncertainty/external disturbances as well as control input nonlinearity through distributed piezoelectric sensor/actuator technology. To satisfy pointing requirements and simultaneously suppress vibrations, two separate control loops are adopted. The first uses piezoceramics as sensors and actuators to actively suppress certain flexible modes by designing positive position feedback (PPF) compensators which add damping to the flexible structures in certain critical modes. The second feedback loop is designed based on an output feedback sliding mode control (OFSMC) design where control input nonlinearity is taken into consideration. Simulation studies for the proposed control strategy on a flexible spacecraft demonstrate the effectiveness of the proposed approach

Tracking is an essential capability in many wireless sensor network (WSN) applications. Due to the probabilistic nature of the target movement and the limited detection region and miss detection of sensor, this existing single sensor sensing scheme may result in tracking failure when a scheduled sensor fails to detect the target. We present an adaptive multi-sensor scheduling algorithm for collaborative target tracking in WSNs to improve the tracking reliability and power efficient. This proposed scheme determines the sampling intervals based on the predicted tracking accuracy, and selects a number of sensors to form a temporary tasking group for the next time step, based on a specified detection probability. One of the tasking group members is selected to be the leader node, which acts as a local temporary fusion center

This paper is concerned with the finite horizon output feedback H2control problem for discrete time systems with multiple input delays. A separation principle is applied which converts the H2control problem into an associated linear quadratic regulation (LQR) problem in conjunction with a Kalman filter. The H2output feedback controller is constructed by solving two Riccati difference equations (RDEs) of the same dimension as the plant (ignoring the delays), similar to the solution of the standard H2control problem. The output feedback H2control result is then applied to ATM congestion control. Simulations show that the proposed control technique can achieve desired control performance effectively and is robust to the varying round trip delay of the ATM network to some extent

Sensor scheduling is essential to collaborative target tracking in Wireless Sensor Networks (WSNs). In this paper, we present a Multi-step Adaptive Sensor Scheduling algorithm (MASS) by selecting the next tasking sensor and its associated sampling interval based on the prediction of tracking accuracy and energy cost over a finite horizon of steps. MASS adopts alternative tracking mode for each prediction step, i.e., the fast tracking mode (FTM) or the tracking maintenance mode (TMM) dependent on whether the estimated or predicted tracking accuracy is satisfactory. The Best Sensor Schedule Sequence (BSSS) is found by searching and comparing the Candidate Sensor Schedule Sequences (CSSSs) at two levels, i.e., the logical tracking mode level which is simplely defined on multi-step tracking modes and the physical quantity performance level by considering the tradeoff between tracking accuracy and energy cost. MASS employs the extended Kalman filter (EKF) algorithm to predict the tracking accuracy and an energy consumption model to predict the energy cost. Simulation results show that, compared with the traditional non-adaptive sensor scheduling algorithm and the single-step adaptive sensor scheduling algorithm, MASS can achieve fast tracking speed and superior energy efficiency without degrading the tracking accuracy.

We propose a novel entropy-based method for feature selection in order to reduce the computational burden for real time simultaneous localization and map building (SLAM) for mobile robot navigation. Our approach is based on information (entropy) theory together with a data association method to initialize new features into the map, match measurements to the map features, and remove out-of-date features. The selected features are optimum in the sense that fusion of measurements from those features with existing information would yield the most entropy reduction in estimating the robot location and the map features' locations. Our method has the advantage of selecting a suitable number of features by considering the computational constraint in real time implementations. Simulation results show that the proposed entropy based feature selection strategy is effective in dealing with the map scaling problem in SLAM.

In this paper, we first develop a delay-dependent condition for the stability and Hinfinperformance of systems with time-variant delays in both the state and output equations in terms of an LMI (linear matrix inequality). The analysis result is then applied to derive a Hinfincongestion control where the congestion problem is formulated as the Hinfincontrol of systems with time-variant input delays. Illustrative examples are provided to show excellent performance of the proposed algorithm in achieving an equilibrium in the buffer occupancy in the presence of time-variant delays. To the best of our knowledge, no such congestion control approach that directly accommodates the issue of the uncertain time-variant delays has been reported

The paper addresses the problem of channel estimation for DS-CDMA systems undergoing time-varying multipath fading channels. The multipath fading channels are modeled as AR models. Based on the minimum mean square error (MMSE) criterion, the linear optimal estimator is obtained by a spectral factorization and a Diophantine polynomial matrix equation. The uncertainty of the channel model is taken into consideration to improve the robustness of the estimator. Compared with the Kalman estimator, the proposed algorithm has O(K/sup 2/) computational complexity, where K is the number of users. The simulation results show that the proposed estimator provides good estimation performance and robustness for fast fading channels.

In the present paper, we investigate the H/sub /spl infin// estimation problem for a class of discrete-time switching systems in a piecewise linear form. A type of Luenberger estimator is presented to guarantee an H/sub /spl infin// performance by using arguments from the Lyapunov theory. Two approaches based on linear matrix inequalities (LMIs) and bilinear matrix inequalities (BMIs) respectively are introduced. Our approaches employ S-procedure and partition-dependent slack variables to reduce design conservatism. An example is given to illustrate the proposed estimator design methods.

The linear quadratic Gaussian (LQG) control for one-dimensional (1-D) systems has been known to be one of the fundamental and significant methods in linear system theory. However, the LQG control problem for two-dimensional (2-D) systems has not been satisfactorily solved due to their structural and dynamical complexity. In this paper, sufficient conditions for evaluation of the quadratic performance indices of 2-D systems in terms of the system state and control variables are proposed. Using these conditions, systematic design methods for finite horizon and infinite horizon LQG controls of 2-D systems are developed.

In this paper a feature detection algorithm based on a new curve gradient model is proposed for simultaneous localization and mapping (SLAM) for complex outdoor environments. The curve gradient model is derived for data segmentation and has the advantage of being suitable for segmentation of data from various types of feature such as point feature and circular feature. The real time implementation of SLAM together with this feature extraction algorithm is realized by using a combination of odometry and laser scanner data. The system was tested on a long walk way at Nanyang Technological University. The experimental results show that the feature detection algorithm performs well during SLAM.

In this paper we present a new approach for natural feature extraction using a laser scanner for the purpose of localization in outdoor environments. In semi-structured outdoor environments, naturally predominant features such as trees and edges are considered. The proposed method applies a batch processing which carries out feature extraction after measurements from a full scan are received. The algorithm consists of data segmentation and parameter acquisition. A modified Gauss-Newton method is proposed for fitting circle parameters iteratively. The natural features extracted through this approach are more robust than those obtained by existing methods. In order to reduce the estimation error caused by the linearization in the extended Kalman filtering (EKF), a particle filter is applied to realize the prediction and validation by integrating data from both the laser range sensor and encoder in outdoor environments. The proposed feature extraction and localization algorithms are verified in a real world experiment.

We present an efficient integer programming (IP) based data association approach to simultaneous localization and mapping (SLAM). In this approach, the feature based SLAM data association problem is formulated as a 0-1 IP problem. The IP problem is approached by first solving a relaxed linear programming (LP) problem. Based on the optimal LP solution, a suboptimal solution to the IP problem is then obtained by applying an iterative heuristic greedy rounding (IHGR) procedure. Unlike the traditional nearest-neighbor (NN) algorithm, the proposed algorithm deals with a global matching between existing features and measurements of each scan and is more robust for an environment of high density features which is usually the case in outdoor environments. We provide a simulation study where the NN algorithm fails whereas our proposed algorithm performs satisfactorily. Experimental results also demonstrate the effectiveness and efficiency of our approach.

This paper focuses on decentralized H/sub /spl infin// filtering problem for interconnected linear systems. The problem we address is to find a decentralized filter where each local filter is based only on local available information on its own subsystem and the overall filtering error is totally asymptotically stable and the L/sub 2/-gain from the exogenous noise input to the filtering error less than a prespecified level. This paper shows that the decentralized H/sub /spl infin// filtering problem can be solved by using linear matrix inequality (LMI) techniques, which are numerically efficient due to recent advances in convex optimization.

In this paper, a joint optimal design method for multirate subband coding systems is presented. For a multirate subband coding system with given analysis filters and average bit number of data quantizers, the design is to simultaneously find the synthesis filters and bit number allocation of the subband quantizers such that the variance of the reconstruction error of the system is minimized. We propose an iterative approach for obtaining the synthesis filters and the bit number allocation that minimize the reconstruction error of the system. Our simulation example demonstrates the favorable performance of the proposed method as compared with existing methods.

This paper is concerned with the H/sub 2/ estimation and control problems for uncertain discrete-time systems. We first present an analysis result of H/sub 2/ norm bound for a stable uncertain system in terms of linear matrix inequalities (LMIs). A solution to the robust H/sub 2/ estimation problem is then derived in term of two LMIs. As compared to the existing results, our result on robust H/sub 2/ estimation is more general. In addition, explicit search of appropriate scaling parameters is not needed as the optimization is convex in the scaling parameters. The LMI approach is also extended to solve the robust H/sub 2/ control problem which has been difficult for the tradition Riccati equation approach since no separation principle has been known for uncertain systems. The design approach is demonstrated through a simple example of target tracking.

In many applications, one needs to approximate a filter of very high order with that of lower order. To reduce the order of the filter, some techniques such as the balanced model reduction approach are often applied. In this paper, we introduce a new technique which is based on minimizing the H/sub 2/-norm between the filter of very high order and the reduced one. This technique shows much better performance than other existing model reduction methods and is applied to estimating the vocal tract filter for speech processing systems. A speech processing example is presented to demonstrate the design procedure and the performance of the proposed algorithm.

The problem of passivity analysis finds important applications in many signal processing systems such as digital quantizers, decision feedback equalizers and digital and analog filters. This paper considers the passivity analysis problem for a large class of systems which involve uncertain parameters, time delays, quantization errors, and unmodeled high order dynamics. By characterizing these and many other types of uncertainty using a general tool called integral quadratic constraints (IQCs), we present a solution to the problem of robust passivity analysis. More specifically, we determine if a given uncertain system is robustly passive. The solution is given in terms of the feasibility of a linear matrix inequality (LMI) which can be solved efficiently.