In recent years, intelligent vehicles operating in urban environments have demonstrated the capability to autonomously execute various tasks, such as object detection, lane detection, segmentation, etc. This advancement is facilitated by the extensive datasets accumulated by researchers, alongside advancements in intelligent algorithms, as well as significant breakthroughs in software and hardware. However, within the autonomous driving community, there is a scarcity of data regarding scenarios encountered in mining environments. This scarcity presents challenges and bottlenecks for the advancement of comprehensive autonomous driving systems and autonomousoperations. Although we previously released our dataset, AutoMine, which includes over 18 hours of driving data in open-pit mines, its scope is limited to two specific tasks. This scope limitation impedes the training and validation of the majority of algorithms for different tasks in this particular scenario. To broaden the scope of autonomous driving visual tasks in mining environments, we have curated a diverse collection encompassing multiple tasks, including detection, segmentation, tracking, etc. Additionally, we have established benchmarks and set up baselines for the aforementioned multiple tasks. By comparing the performance differences of visual algorithms between mining areas and other scenarios, we demonstrate the distinctive characteristics of mining regions in an intuitive manner. We have developed a suite of tools for converting annotated data into the standardized format used in existing driving datasets. Our aspiration is to establish data and benchmark foundations, supporting research endeavors in intelligent transportation within mining environments and autonomous driving in comprehensive scenarios. Our project website can be seen in AutoMine, and the dataset can be downloaded via AutoMine-Benchmark.

Flexible spines are critical to the remarkable agility and speed of animals. Translating this biological advantage to quadruped robots presents a significant control challenge, particularly in coordinating the spine and limbs for maximal velocity. In this work, we utilize reinforcement learning (RL) to develop high-speed locomotion for a bioinspired mouse robot with a lateral flexible spine. The resulting controller achieves motor performance that demonstrably surpasses non-spined and model-based methods. More importantly, our analysis reveals the principles behind this performance: the emergence of two distinct motion templates. For high-speed walking, the robot learns a “whip-like” spinal oscillation to increase leg swing frequency, while for agile turning, it adopts a dynamic “bend-and-straighten” pattern. These findings demonstrate the capability of RL to not only generate high-performance controllers but also to produce emergent strategies that, upon analysis, reveal underlying principles of high-speed, spine-driven locomotion.

Large Language Models (LLMs) have gained popularity in task planning for long-horizon manipulation tasks. To enhance the validity of LLM-generated plans, visual demonstrations and online videos have been widely employed to guide the planning process. However, for manipulation tasks involving subtle movements but rich contact interactions, visual perception alone may be insufficient for the LLM to fully interpret the demonstration. Additionally, visual data provides limited information on force-related parameters and conditions, which are crucial for effective execution on real robots. In this paper, we introduce LEMMo-Plan, an in-context learning framework that incorporates tactile and force-torque information from human demonstrations to enhance LLMs' ability to generate plans for new task scenarios. We propose a bootstrapped reasoning pipeline that sequentially integrates each modality into a comprehensive task plan. This task plan is then used as a reference for planning in new task configurations. Real-world experiments on two different sequential manipulation tasks demonstrate the effectiveness of our framework in improving LLMs' understanding of multi-modal demonstrations and enhancing the overall planning performance. More materials are available on our project website: lemmo-plan.github.io/LEMMo-Plan/.

3D Gaussian Splatting (3DGS) allows flexible adjustments to scene representation, enabling continuous optimization of scene quality during dense visual simultaneous localization and mapping (SLAM) in static environments. However, 3DGS faces challenges in handling environmental disturbances from dynamic objects with irregular movement, leading to degradation in both camera tracking accuracy and map reconstruction quality. To address this challenge, we develop an RGB-D dense SLAM which is called Gaussian Splatting SLAM in Dynamic Environments (Gassidy). This approach calculates Gaussians to generate rendering loss flows for each environmental component based on a designed photometricgeometric loss function. To distinguish and filter environmental disturbances, we iteratively analyze rendering loss flows to detect features characterized by changes in loss values between dynamic objects and static components. This process ensures a clean environment for accurate scene reconstruction. Compared to state-of-the-art SLAM methods, experimental results on open datasets show that Gassidy improves camera tracking precision by up to 97.9 % and enhances map quality by up to 6 %. Video of experiments is available here: https://www.wixsite.com.com/wen-Gassidy.

Assembly is a crucial skill for robots in both modern manufacturing and service robotics. However, mastering transferable insertion skills that can handle a variety of high-precision assembly tasks remains a significant challenge. This paper presents a novel framework that utilizes diffusion models to generate 6D wrench for high-precision tactile robotic insertion tasks. It learns from demonstrations performed on a single task and achieves a zero-shot transfer success rate of 95.7% across various novel high-precision tasks. Our method effectively inherits the self-adaptability demonstrated by our previous work. In this framework, we address the frequency misalignment between the diffusion policy and the real-time control loop with a dynamic system-based filter, significantly improving the task success rate by 9.15%. Furthermore, we provide a practical guideline regarding the trade-off between diffusion models' inference ability and speed.

Optimal path planning requires finding a series of feasible states from the starting point to the goal to optimize objectives. Popular path planning algorithms, such as Effort Informed Trees (EIT*), employ effort heuristics to guide the search. Effective heuristics are accurate and computationally efficient, but achieving both can be challenging due to their conflicting nature. This paper proposes Direction Informed Trees (DIT*), a sampling-based planner that focuses on optimizing the search direction for each edge, resulting in goal bias during exploration. We define edges as generalized vectors and integrate similarity indexes to establish a directional filter that selects the nearest neighbors and estimates direction costs. The estimated direction cost heuristics are utilized in edge evaluation. This strategy allows the exploration to share directional information efficiently. DIT* convergence faster than existing single-query, sampling-based planners on tested problems in$\mathbb{R}^{4}$to$\mathbb{R}^{16}$and has been demonstrated in real-world environments with various planning tasks. A video showcasing our experimental results is available at: https://youtu.be/2SX6QT2NOek.

This paper proposes a novel dual-filter architecture utilizing RGB-D camera data and dynamic control barrier functions (D-CBFs) for real-time obstacle avoidance in unstructured environments. The proposed method efficiently handles static, suddenly appearing, and dynamic obstacles, maintaining consistent computational performance across diverse scenarios. To achieve this, two key challenges must be addressed. First, the substantial volume of pixel and depth map data requires robust, real-time processing for efficient D-CBF construction. Second, constructing D-CBFs for each obstacle in multi-obstacle scenarios increases optimization solver time. To address these challenges, we adapt the concept of salient object detection (SOD), proposing an enhanced FastSOD (E-FastSOD) method for rapid risk area identification. This approach rapidly filters out low-risk areas, while high-risk regions are mathematically represented utilizing the proposed enhanced minimal bounding circle (E-MBC) technique. We differentiate static and dynamic obstacles by comparing current and previous MBC states, employing Kalman filtering for obstacle state prediction. This setup enables efficient online D-CBF construction for each MBC, balancing computational speed with accurate obstacle representation. Subsequently, the second filter establishes buffer zones around established D-CBFs, activating only those corresponding to zones the robot actually enters, rather than all D-CBFs to increase real-time performance. We prove the system's safety and asymptotic stabilization under this architecture. Simulated and real-world experiments validate our method, demonstrating an equipped mobile robot's ability to accomplish tasks while ensuring safety across diverse, unknown scenarios.

Industrial assembly of deformable linear objects (DLOs) such as cables offers great potential for many industries. However, DLOs pose several challenges for robot-based automation due to the inherent complexity of deformation and, consequentially, the difficulties in anticipating the behavior of DLOs in dynamic situations. Although existing studies have addressed isolated subproblems like shape tracking, grasping, and shape control, there has been limited exploration of integrated workflows that combine these individual processes.To address this gap, we propose an object-centric perception and planning framework to achieve a comprehensive DLO assembly process throughout the industrial value chain. The framework utilizes visual and tactile information to track the DLO’s shape as well as contact state across different stages, which facilitates effective planning of robot actions. Our approach encompasses robot-based bin picking of DLOs from cluttered environments, followed by a coordinated handover to two additional robots that mount the DLOs onto designated fixtures. Real-World experiments employing a setup with multiple robots demonstrate the effectiveness of the approach and its relevance to industrial scenarios.

The whisker-inspired tactile sensor is advantageous for enhancing robotic perception in proximate range and darkness via non-intrusive contacts. However, localizing contact along the whisker shaft is challenging due to the non-injective mapping between tangential contacts and the resulting bending moments at the whisker base. Previous studies suggest that incorporating axial force measurements can resolve this ambiguity. In this work, we develop a magnetically transduced whisker sensor that integrates axial force sensing as an additional mechanical signal. The sensor features a tapered whisker with a custom slope and a 3-DoF suspension mechanism, enabling axial displacement at the base, which is proportional to the applied axial force. We construct a Penalized Gaussian Process model trained on synthetic data to estimate the whisker’s motion and refine it with real-data constraints. The design is compact, low-cost, and validated through simulations and real-world experiments to differentiate tangential contacts. Furthermore, we propose an optimization-based approach for estimating instantaneous contact locations. Experimental results demonstrate that the proposed method can effectively track contacts in millimeter-level accuracy with a mean error of 7.17 mm, achieving a higher accuracy with only 4.02 mm in large-deflection and close-to-base regions.

The deep learning models has significantly advanced dexterous manipulation techniques for multi-fingered hand grasping. However, the contact information-guided grasping in cluttered environments remains largely underexplored. To address this gap, we have developed ContactDexNet, a method for generating multi-fingered hand grasp samples in cluttered settings through contact semantic map. We introduce a contact semantic conditional variational autoencoder network (CoSe-CVAE) for creating comprehensive contact semantic map from object point cloud. We utilize grasp detection method to estimate hand grasp poses from the contact semantic map. Finally, an unified grasp evaluation model PointNetGPD++ is designed to assess grasp quality and collision probability, substantially improving the reliability of identifying optimal grasps in cluttered scenarios. Our grasp generation method has demonstrated remarkable success, outperforming state-of-the-art (SOTA) methods by at least 4.7%, with 81.0% average grasping success rate in real-world single-object grasping using a known hand, and by at least 9.0% when using an unknown hand. Moreover, in cluttered scenes, our method attains a 76.7% success rate, outperforming the SOTA method by 6.3%. We also proposed the multi-modal multi-fingered grasping dataset generation method. Our multi-fingered hand grasping dataset outperforms previous datasets in scene diversity, modality diversity. More details and supplementary materials can be found at https://sites.google.com/view/contact-dexnet.

Robotic motion planning faces formidable challenges in constrained environments, particularly in rapidly searching for feasible solutions and converging towards optimal. This study introduces Multi-Sets Tree (MST*), a sampling-based planner designed to accelerate path searching and solution optimization. MST* integrates estimated guided incremental local densification (GuILD) sets that are based on prior estimated solution costs before finding the initial solution. For path optimization, MST* integrates novel beacon selectors to define problem subsets, thereby guiding exploration and effectively exploiting high-potential areas. This multi-set strategy ensures balanced exploration and exploitation, enabling MST* to handle sparse free space. Moreover, MST* utilizes adaptive sampling techniques via Lebesgue’s measure of domain subsets for rapid search. MST* improves search efficiency and path optimality, particularly in constrained high-dimensional environments. It extends the informed sampling concept by refining the search region and batch sampling. Experimental results demonstrate that MST* outperforms single-query planners across ℝ4to ℝ16benchmarks and in real-world robotic navigation tasks. A video showcasing our experimental results is available at: https://youtu.be/obftvS0a41M.

This paper introduces Context Informed Trees (CIT*), a sampling-based motion planning algorithm that enhances exploration efficiency by biasing sampling based on uncertainty estimation from local samples and connectivity information obtained during the search process. CIT* is based on Flexible Informed Trees (FIT*) and incorporates three key components: region-based sampling, uncertainty-driven weighting, and connection-greedy prioritization (CGP). It generates regions from sampled states based on local obstacle proximity, assigning weights to these regions using probability uncertainty estimation via kernel density estimation (KDE) classification. To further refine the sampling focus, CGP prioritizes regions that exhibit strong connectivity in previous searches, ensuring that exploration is directed toward unknown and critical areas that have a higher likelihood of contributing to feasible and efficient paths. The sampling process is then guided by a mixture of Gaussian distributions centered on weighted regions, where the weighting biases sampling toward more critical regions, thereby improving search efficiency and accelerating convergence. Benchmark evaluations demonstrate that CIT* improves efficiency by reducing reliance on random sampling, which often leads to slower solution discovery and higher path costs. With biased sampling, CIT* maintains strong performance in solving complex motion planning problems in ${\mathbb{R}^4}$ to ${\mathbb{R}^{16}}$ and has been demonstrated on a real-world manipulation task. A video showcasing our method and experimental results is available at: https://youtu.be/SG2cy9WmjD0.

Path planning in robotics often involves solving continuously valued, high-dimensional problems. Popular informed approaches include graph-based searches, such as A*, and sampling-based methods, such as Informed RRT*, which utilize informed set and anytime strategies to expedite path optimization incrementally. Informed sampling-based planners define informed sets as subsets of the problem domain based on the current best solution cost. However, when no solution is found, these planners re-sample and explore the entire configuration space, which is time-consuming and computationally expensive. This article introduces Multi-Informed Trees (MIT*), a novel planner that constructs estimated informed sets based on prior admissible solution costs before finding the initial solution, thereby accelerating the initial convergence rate. Moreover, MIT* employs an adaptive sampler that dynamically adjusts the sampling strategy based on the exploration process. Furthermore, MIT* utilizes length-related adaptive sparse collision checks to guide lazy reverse search. These features enhance path cost efficiency and computation times while ensuring high success rates in confined scenarios. Through a series of simulations and real-world experiments, it is confirmed that MIT* outperforms existing single-query, sampling-based planners for problems in$\mathbb {R}^{4}$to$\mathbb {R}^{16}$and has been successfully applied to real-world robot manipulation tasks. A video showcasing our experimental results is available at:https://youtu.be/30RsBIdexTUNote to Practitioners—The motivation for this work stems from the challenges faced by existing informed path planners in high-dimensional, continuously valued environments, particularly when an initial feasible solution is difficult to find. Traditional asymmetric bidirectional planners rely on the best current solution to define problem subsets. When a lazy path has been found through lazy reverse search, these planners tend to re-sample and explore the entire problem space, which could hinder the path planning process. Our proposed MIT* algorithm addresses this issue by constructing an estimated informed set based on prior admissible solution costs before finding the initial solution. This estimated set helps to narrow the search area, thereby accelerating the initial convergence rate. MIT* also integrates an adaptive sampling strategy that dynamically adjusts based on the ongoing exploration process, enhancing the planner’s ability to efficiently navigate through challenging spaces. Furthermore, MIT* employs adaptive sparse collision checks, which guide the lazy reverse search that balances computational efficiency with accuracy in pathfinding. The proposed algorithm can be applied to industrial robots, humanoid robots, or service robots to achieve efficient path planning.

This paper proposes a novel adaptive fast variational sparse Gaussian process (AFVSGP) framework to ensure real-time safety for high-order systems under model uncertainties and dynamic obstacle environments. The framework effectively addresses the challenge of maintaining real-time safety guarantees during unknown trajectory transitions in nonstationary environments. To achieve this, the proposed framework incorporates three key innovations. First, a specialized kernel function is embedded within the VSGP algorithm to decouple control inputs from uncertainties while preserving the convexity of posterior-based safety constraints. Second, an adaptive online incremental learning mechanism is introduced, integrating forgetting capabilities with dynamic reconstruction rules for training datasets and inducing sets, thereby accelerating inference convergence and enabling compact uncertainty prediction with reduced computational complexity. Third, a high-order control barrier function (HOCBF)-based safety filter is developed to synthesize safe control inputs by leveraging the proposed learning model, thereby establishing rigorous probabilistic bounds on the satisfaction of safety specifications. The effectiveness of the proposed framework is validated through both simulation and real-world obstacle avoidance experiments on a 7-DOF Franka robot. The video is available at: https://www.youtube.com/watch?v=2tCKYM_79S8.

Autonomous robots operating in diverse scenarios are expected to safely and efficiently adapt to new, unknown, and cluttered environments. In this paper, we introduce a real-time goal-seeking and exploration framework incorporating novel meta-signed distance functions (MetaSDFs) and metabuffer robust control barrier functions (Meta-BRCBFs). To adapt to environmental changes in real time, we employ Bayesian meta-learning to construct MetaSDFs. Deep neural network weights are initially trained offline, followed by efficient online adaptation at the last Bayesian layer, allowing for online updates at linear time complexity. Each MetaSDF is individually trained for its corresponding obstacle class, enhancing online distance estimation accuracy. Subsequently, buffer zones are constructed around the MetaSDFs to establish corresponding Meta-BRCBFs. These Meta-BRCBFs are activated only when the robot enters these zones, substantially reducing the number of CBFs required. Outside these specified buffer zones, the robot remains ingoal-seekingmode, focusing on task completion. After entering a buffer zone, it transitions toexplorationmode, prioritizing safety and exploring safe pathways, effectively balancing task execution with environmental adaptability. We demonstrate that, under this framework, the system achieves both safety and asymptotic stabilization. Extensive simulations and experiments are conducted to demonstrate our framework’s effectiveness in both simulated scenarios and real-world environments. These tests confirm our framework’s real-time capabilities and safety assurances in dynamic settings where state-of-the-art methods fail. The video is available at: https://www.youtube.com/watch?v=C6eshldAMxA.

Nonholonomic vehicles in distributed networks are prone to triggering nested velocity and acceleration saturation during reactive safety formations, exacerbating oscillations. This paper proposes a hybrid secure distributed collaborative frame-work, integrating compound adaptive anti-windup strategies with vehicle kinematics and safe geofences to achieve smooth and effective obstacle and collision avoidance while suppressing saturation-induced oscillations. The vehicle’s safe behavior for bypassing obstacles is formed via acceleration envelopes from safe geofences and input saturation, which generate constraint velocity commands. Additionally, a low-trigger and power-adjustable enhanced artificial potential field is integrated into the safety coordination to fine-tune vehicle maneuvers at extremely close distances to hazardous targets, ensuring high reliability. Safe acceleration envelopes and nested kinematic saturation are utilized to design a compound adaptive auxiliary dynamic system, smoothing oscillations induced by dual command constraints during formation. A distributed formation controller is further designed to enable multitasking collaboration in formations. The overall stability is mathematically analyzed, and the method’s superior smoothness and safety in task coordination are validated through simulations and experiments with vehicle clusters. Note to Practitioners—In response to the severe trajectory oscillations caused by saturation triggered by existing reactive avoidance approaches, this paper proposes a novel hybrid safety collaborative control based on the nonholonomic vehicle kinematics that markedly enhances the smoothness and safeness of formations in obstacle environments. The integration of safety acceleration envelopes, as well as low-trigger and adjustable artificial potential functions, markedly mitigates oscillations from reaction saturation compared with the solitary traditional artificial potential functions, as evidenced by simulations and experiments that demonstrate reduced oscillation amplitudes and shorter recovery times when evading hazardous targets using the proposed method. In addition, existing velocity/acceleration nested windups in actual applications are concurrently considered for the first time, and the corresponding compound adaptive anti-windup method is employed to smooth oscillations caused by control saturation. The security and smoothing strategies outlined allow for collaborative operations in more complicated obstacle environments and enable the deployment of larger vehicle clusters in confined spaces, significantly enhancing multi-vehicle collaboration’s economic viability and efficiency. Furthermore, the safety collaborative control framework designed for kinematics is conveniently structured for engineers as a standalone module, which is easily transferrable to commercial robotic products. The composite approach to safeness and smoothness can also be applied in other unmanned and manned collaborative scenarios.

The variations in camera view and hand spatial pose are the main reasons for the low accuracy and poor robustness of gesture recognition systems. In order to achieve accurate and stable gesture recognition with variable viewpoint, this article carries out research based on 3-D non-Euclidean vector graph features and graph neural networks. First, the 3-D information of hand joints is collected to construct a graph-structured gesture feature dataset, and a joint-based 3-D non-Euclidean vector graph method is proposed to solve the problem that similar gesture features are overly sensitive to spatial position and angle changes. Then, a Multi-Head graph attention network is designed and combined with graph convolutional neural network to explore the optimal hybrid graph neural network gesture recognition model. The experimental results show that, on the dataset processed by the joint-based 3-D non-Euclidean vector graph method, the training, testing, and validation accuracies of the optimal model reach 97.07%, 96.95%, and 87.06%, which are increased by 18.46%, 18.88%, and 44.23% compared to the original dataset, respectively. In conclusion, the method in this article is not only more robust to the variable viewpoint gesture recognition problem, but also has the advantages of low computational resource requirement and high real-time performance.

Controlling the shape of deformable linear objects using robots and constraints provided by environmental fixtures has diverse industrial applications. In order to establish robust contacts with these fixtures, accurate estimation of the contact state is essential for preventing and rectifying potential anomalies. However, this task is challenging due to the small sizes of fixtures, the requirement for real-time performances, and the infinite degrees of freedom of the deformable linear objects. In this paper, we propose a real-time approach for estimating both contact establishment and subsequent changes by leveraging the dependency between the applied and detected contact force on the deformable linear objects. We seamlessly integrate this method into the robot control loop and achieve an adaptive shape control framework which avoids, detects and corrects anomalies automatically. Real-world experiments validate the robustness and effectiveness of our contact estimation approach across various scenarios, significantly increasing the success rate of shape control processes.

This paper presents an adaptive online learning framework for systems with uncertain parameters to ensure safety-critical control in non-stationary environments. Our approach consists of two phases. The initial phase is centered on a novel sparse Gaussian process (GP) framework. We first integrate a forgetting factor to refine a variational sparse GP algorithm, thus enhancing its adaptability. Subsequently, the hyperparameters of the Gaussian model are trained with a specially compound kernel, and the Gaussian model’s online inferential capability and computational efficiency are strengthened by updating a solitary inducing point derived from newly samples, in conjunction with the learned hyperparameters. In the second phase, we propose a safety filter based on high order control barrier functions (HOCBFs), synergized with the previously trained learning model. By leveraging the compound kernel from the first phase, we effectively address the inherent limitations of GPs in handling high-dimensional problems for real-time applications. The derived controller ensures a rigorous lower bound on the probability of satisfying the safety specification. Finally, the efficacy of our proposed algorithm is demonstrated through real-time obstacle avoidance experiments executed using both simulation platform and a real-world 7-DOF robot.

Balancing oneself using the spine is a physiological alignment of the body posture in the most efficient manner by the muscular forces for mammals. For this reason, we can see many disabled quadruped animals can still stand or walk even with three limbs. This paper investigates the optimization of dynamic balance during trot gait based on the spatial relationship between the center of mass (CoM) and support area influenced by spinal flexion. During trotting, the robot balance is significantly influenced by the distance of the CoM to the support area formed by diagonal footholds. In this context, lateral spinal flexion, which is able to modify the position of footholds, holds promise for optimizing balance during trotting. This paper explores this phenomenon using a rat robot equipped with a soft actuated spine. Based on the lateral flexion of the spine, we establish a kinematic model to quantify the impact of spinal flexion on robot balance during trot gait. Subsequently, we develop an optimized controller for spinal flexion, designed to enhance balance without altering the leg locomotion. The effectiveness of our proposed controller is evaluated through extensive simulations and physical experiments conducted on a rat robot. Compared to both a non-spine based trot gait controller and a trot gait controller with lateral spinal flexion, our proposed optimized controller effectively improves the dynamic balance of the robot and retains the desired locomotion during trotting.

Multi-goal robot manipulation tasks with sparse rewards are difficult for reinforcement learning (RL) algorithms due to the inefficiency in collecting successful experiences. Recent algorithms such as Hindsight Experience Replay (HER) expedite learning by taking advantage of failed trajectories and replacing the desired goal with one of the achieved states so that any failed trajectory can be utilized as a contribution to learning. However, HER uniformly chooses failed trajectories, without taking into account which ones might be the most valuable for learning. In this paper, we address this problem and propose a novel approach Contact Energy Based Prioritization (CEBP) to select the samples from the replay buffer based on rich information due to contact, leveraging the touch sensors in the gripper of the robot and object displacement. Our prioritization scheme favors sampling of contact-rich experiences, which are arguably the ones providing the largest amount of information. We evaluate our proposed approach on various sparse reward robotic tasks and compare it with the state-of-the-art methods. We show that our method surpasses or performs on par with those methods on robot manipulation tasks. Finally, we deploy the trained policy from our method to a real Franka robot for a pick-and-place task. We observe that the robot can solve the task successfully. The videos and code are publicly available at: https://erdiphd.github.io/HER_force/.

Insertion is an essential skill for robots in both modern manufacturing and services robotics. In our previous study, we proposed an insertion skill framework based on forcedomain wiggle motion. The main limitation of this method lies in the robot’s inability to adjust its behavior according to changing contact state during interaction. In this paper, we extend the skill formalism by incorporating a behavior tree-based primitive switching mechanism that leverages highfrequency tactile data for the estimation of contact state. The efficacy of our proposed framework is validated with a series of experiments that involve the execution of tightly constrained peg-in-hole tasks. The experiment results demonstrate a significant improvement in performance, characterized by reduced execution time, heightened robustness, and superior adaptability when confronted with unknown tasks. Moreover, in the context of transfer learning, our paper provides empirical evidence indicating that the proposed skill framework contributes to enhanced transferability across distinct operational contexts and tasks.

Soft continuum manipulators are celebrated for their versatility and physical robustness to external forces and perturbations. However, this feature comes at a cost. The many degrees of freedom and compliance pose challenges for accurate pose reconstruction, both in terms of distributed sensing and pose reconstruction algorithms. Moreover, soft arms are inherently susceptible to deformation from external forces or loads, meaning that closed-loop control is essential for robust task performance. In this article, we propose the integration of multiple Inertial Measurement Units (IMUs) of a soft robot arm, Helix, for reconstruction of pose under internal and external forces. Furthermore, we integrate this dynamic pose reconstruction for kinematic-based closed-loop control strategies. By serially integrating sensing in the body of the Helix soft manipulator, we provide the system with high-frequency pose reconstruction and demonstrate improvements in end effector position with comparison to open-loop performance.

In path planning, anytime almost-surely asymptotically optimal planners dominate the benchmark of sampling-based planners. A notable example is Batch Informed Trees (BIT*), where planners iteratively determine paths to batches of vertices within the exploration area. However, utilizing a consistent batch size is inefficient for initial pathfinding and optimal performance, it relies on effective task allocation. This paper introduces Flexible Informed Trees (FIT*), a sampling-based planner that integrates an adaptive batch-size method to enhance the initial path convergence rate. FIT* employs a flexible approach in adjusting batch sizes dynamically based on the inherent dimension of the configuration spaces and the hypervolume of the n-dimensional hyperellipsoid. By applying dense and sparse sampling strategy, FIT* improves convergence rate while finding successful solutions faster with lower initial solution cost. This method enhances the planner’s ability to handle confined, narrow spaces in the initial finding phase and increases batch vertices sampling frequency in the optimization phase. FIT* outperforms existing single-query, sampling-based planners on the tested problems in R2to R8, and was demonstrated on a real-world mobile manipulation task.

Path planning has long been an important and active research area in robotics. To address challenges in high-dimensional motion planning, this study introduces the Force Direction Informed Trees (FDIT*), a sampling-based planner designed to enhance speed and cost-effectiveness in pathfinding. FDIT* builds upon the state-of-the-art informed sampling planner, the Effort Informed Trees (EIT*), by capitalizing on often-overlooked information in invalid vertices. It incorporates principles of physical force, particularly Coulomb’s law. This approach proposes the elliptical k-nearest neighbors search method, enabling fast convergence navigation and avoiding high solution cost or infeasible paths by exploring more problem-specific search-worthy areas. It demonstrates benefits in search efficiency and cost reduction, particularly in confined, high-dimensional environments. It can be viewed as an extension of nearest neighbors search techniques. Fusing invalid vertex data with physical dynamics facilitates force-direction-based search regions, resulting in an improved convergence rate to the optimum. FDIT* outperforms existing single-query, sampling-based planners on the tested problems in ℝ4to ℝ16and has been demonstrated on a real-world mobile manipulation task.

This paper proposes a LiDAR-based goal-seeking and exploration framework, addressing the efficiency of online obstacle avoidance in unstructured environments populated with static and moving obstacles. This framework addresses two significant challenges associated with traditional dynamic control barrier functions (D-CBFs): their online construction and the diminished real-time performance caused by utilizing multiple D-CBFs. To tackle the first challenge, the framework’s perception component begins with clustering point clouds via the DBSCAN algorithm, followed by encapsulating these clusters with the minimum bounding ellipses (MBEs) algorithm to create elliptical representations. By comparing the current state of MBEs with those stored from previous moments, the differentiation between static and dynamic obstacles is realized, and the Kalman filter is utilized to predict the movements of the latter. Such analysis facilitates the D-CBF’s online construction for each MBE. To tackle the second challenge, we introduce buffer zones, generating Type-II D-CBFs online for each identified obstacle. Utilizing these buffer zones as activation areas substantially reduces the number of D-CBFs that need to be activated. Upon entering these buffer zones, the system prioritizes safety, autonomously navigating safe paths, and hence referred to as the exploration mode. Exiting these buffer zones triggers the system’s transition to goal-seeking mode. We demonstrate that the system’s states under this framework achieve safety and asymptotic stabilization. Experimental results in simulated and real-world environments have validated our framework’s capability, allowing a LiDAR-equipped mobile robot to efficiently and safely reach the desired location within dynamic environments containing multiple obstacles. Video and code are available: https://zyzhang4.wixsite.com/iros2024.

The rising demand for customized products necessitates the integration of multiple robotic systems, underscoring the need for advanced production planning and scheduling. This paper introduces an ontology-based, artificial intelligence-enhanced method for dynamic task planning and scheduling, aimed at improving the efficiency of production process, reducing machine downtime, and consequently increasing throughput in assembly operations. Designed to generate and execute feasible production plans dynamically, this method minimizes manual planning and scheduling efforts. We evaluate its effectiveness using two gear assembly use cases with various robot skills, highlighting its flexibility in planning and scheduling and its contributions to the evolution of smart manufacturing. The method’s adaptability suggests its applicability across diverse smart factory environments.

Deep reinforcement learning agents usually need to collect a large number of interactions to solve a single task. In contrast, meta-reinforcement learning (meta-RL) aims to quickly adapt to new tasks using a small amount of experience by leveraging the knowledge from training on a set of similar tasks. State-of-the-art context-based meta-RL algorithms use the context to encode the task information and train a policy conditioned on the inferred latent task encoding. However, most recent works are limited to parametric tasks, where a handful of variables control the full variation in the task distribution, and also failed to work in non-stationary environments due to the few-shot adaptation setting. To address those limitations, we propose MEta-reinforcement Learning with Task Self-discovery (MELTS), which adaptively learns qualitatively different nonparametric tasks and adapts to new tasks in a zero-shot manner. We introduce a novel deep clustering framework (DPMM-VAE) based on an infinite mixture of Gaussians, which combines the Dirichlet process mixture model (DPMM) and the variational autoencoder (VAE), to simultaneously learn task representations and cluster the tasks in a self-adaptive way. Integrating DPMM-VAE into MELTS enables it to adaptively discover the multi-modal structure of the nonparametric task distribution, which previous methods using isotropic Gaussian random variables cannot model. In addition, we propose a zero-shot adaptation mechanism and a recurrence-based context encoding strategy to improve the data efficiency and make our algorithm applicable in non-stationary environments. On various continuous control tasks with both parametric and nonparametric variations, our algorithm produces a more structured and self-adaptive task latent space and also achieves superior sample efficiency and asymptotic performance compared with state-of-the-art meta-RL algorithms.

In this article, we propose a Swin Transformer and multilevel Feature Consistency based Network (STFC-Net), which is a multilevel cascade stereo matching method to predict the disparity in a coarse-to-fine manner. 1) To alleviate the problem of the limited receptive field of existing convolutional neural network (CNN)-based methods, inspired by the capability of modeling the large-scale dependence of transformer, we adopt a multilevel feature extraction module combining CNN and Swin Transformer to capture long-range context information; a multiscale cascaded cost aggregation module is used to cover different image regions with less memory consumption. 2) To make full use of the hierarchical features, we checked the multilevel left-right feature consistency in an unsupervised manner to improve the disparity accuracy. The experimental results show that our method outperforms some previous CNN methods on the Scene Flow and KITTI datasets with lower computational time complexity. Moreover, it generalizes well in some unknown and challenging real-world scenarios.

The main problems in transportation are accidents, increasingly slow traffic flow, and pollution. An intelligent transportation system (ITS) using roadside infrastructure can overcome these problems. For this reason, the number of such systems is increasing dramatically, and therefore requires an adequate overview. To the best of our knowledge, no current systematic review of existing ITS solutions exists. To fill this knowledge gap, our paper provides an overview of existing ITS that use roadside infrastructure worldwide. Accordingly, this paper addresses current questions and challenges. For this purpose, we performed a literature review of documents that describe existing ITS solutions from 2009 until today. We categorized the results according to technology levels and analyzed its hardware system setup and value-added contributions. In doing so, we made the ITS solutions comparable and highlighted past development alongside current trends. We analyzed more than 400 papers, including 53 test bed projects. In summary, current ITSs can deliver accurate information about individuals in traffic situations in real-time. However, further research into ITS should focus on more reliable perception of the traffic using modern sensors, plug-and-play mechanisms, and secure real-time distribution of the digital twins in a decentralized manner. By addressing these topics, the development of intelligent transportation systems will be able to take a step towards its comprehensive roll-out.

To achieve more accurate perception performance, LiDAR and camera are gradually chosen to improve 3D object detection simultaneously. However, it is still a non-trivial task to build an effective fusion mechanism, and this is hindering the development of multi-modal based method. Especially, the mapping relationship construction between two modalities is far from fully explored. Canonical cross-modal mapping suffers from failure when the calibration matrix is incorrect, and it also greatly wastes the amount and density of RGB image information. This paper aims to extend the traditional one-to-one alignment relationship between LiDAR and camera. For all projected point clouds, we enhance their cross-modal mapping relationship through aggregating color-texture related feature and shape-contour related feature. Further, a mapping pyramid is proposed to leverage the semantic representation of the image feature at different stages. Based on the above mapping enhancement strategies, our method increases the engagement rate of image. Finally, we design a fusion module based on an attention mechanism to improve the point cloud feature with the auxiliary image feature. Extensive experiments on the KITTI dataset and SUN-RGBD dataset show that our model achieves satisfactory 3D object detection, especially for categories with sparse point clouds compared with other multi-modal fusion networks.

Recent state-of-the-art artificial agents lack the ability to adapt rapidly to new tasks, as they are trained exclusively for specific objectives and require massive amounts of interaction to learn new skills. Meta-reinforcement learning (meta-RL) addresses this challenge by leveraging knowledge learned from training tasks to perform well in previously unseen tasks. However, current meta-RL approaches limit themselves to narrow parametric and stationary task distributions, ignoring qualitative differences and nonstationary changes between tasks that occur in the real world. In this article, we introduce a Task-Inference-based meta-RL algorithm using explicitly parameterized Gaussian variational autoencoders (VAEs) and gated Recurrent units (TIGR), designed for nonparametric and nonstationary environments. We employ a generative model involving a VAE to capture the multimodality of the tasks. We decouple the policy training from the task-inference learning and efficiently train the inference mechanism on the basis of an unsupervised reconstruction objective. We establish a zero-shot adaptation procedure to enable the agent to adapt to nonstationary task changes. We provide a benchmark with qualitatively distinct tasks based on the half-cheetah environment and demonstrate the superior performance of TIGR compared with state-of-the-art meta-RL approaches in terms of sample efficiency (three to ten times faster), asymptotic performance, and applicability in nonparametric and nonstationary environments with zero-shot adaptation. Videos can be viewed at https://videoviewsite.wixsite.com/tigr.

Six-wheel skid-steer vehicles are widely used in the real world because of their special steering structure and load-carrying capacity. In this article, we aim to study the event-triggered fuzzy yaw control of six-wheel skid-steer vehicles. The nonlinear lateral dynamics of a six-wheel skid-steer vehicle is modeled using the Takagi-Sugeno (T-S) fuzzy system with irregular fuzzy rules, in which more membership function information can be captured. Considering the high-frequency transmission mode used in vehicle motion systems due to high-precision control requirements, an event-triggered scheme with a dynamic threshold is proposed to ensure control accuracy while reducing the data transmission. Then, a flexible yaw control scheme is constructed in which two adjustable weighting factors are introduced to improve the flexibility of the fuzzy yaw controller. By combining the flexible yaw control scheme and dynamic event-triggered scheme, we design a set of event-triggered fuzzy yaw controllers that satisfy the finite-time stability of lateral motion tracking for six-wheel skid-steer vehicles. The advantages and effectiveness of the proposed method are verified by conducting the TruckSim-MATLAB joint simulation.

Meta-reinforcement learning enables artificial agents to learn from related training tasks and adapt to new tasks efficiently with minimal interaction data. However, most existing research is still limited to narrow task distributions that are parametric and stationary, and does not consider out-of-distribution tasks during the evaluation, thus, restricting its application. In this paper, we propose MoSS, a context-based Meta-reinforcement learning algorithm based on Self-Supervised task representation learning to address this challenge. We extend meta-RL to broad non-parametric task distributions which have never been explored before, and also achieve state-of-the-art results in non-stationary and out-of-distribution tasks. Specifically, MoSS consists of a task inference module and a policy module. We utilize the Gaussian mixture model for task representation to imitate the parametric and non-parametric task variations. Additionally, our online adaptation strategy enables the agent to react at the first sight of a task change, thus being applicable in non-stationary tasks. MoSS also exhibits strong generalization robustness in out-of-distributions tasks which benefits from the reliable and robust task representation. The policy is built on top of an off-policy RL algorithm and the entire network is trained completely off-policy to ensure high sample efficiency. On MuJoCo and Meta-World benchmarks, MoSS outperforms prior works in terms of asymptotic performance, sample efficiency (3-50x faster), adaptation efficiency, and generalization robustness on broad and diverse task distributions.

Although deep reinforcement learning has recently been very successful at learning complex behaviors, it requires a tremendous amount of data to learn a task. One of the fundamental reasons causing this limitation lies in the nature of the trial-and-error learning paradigm of reinforcement learning, where the agent communicates with the environment and pro-gresses in the learning only relying on the reward signal. This is implicit and rather insufficient to learn a task well. On the con-trary, humans are usually taught new skills via natural language instructions. Utilizing language instructions for robotic motion control to improve the adaptability is a recently emerged topic and challenging. In this paper, we present a meta-RL algorithm that addresses the challenge of learning skills with language instructions in multiple manipulation tasks. On the one hand, our algorithm utilizes the language instructions to shape its in-terpretation of the task, on the other hand, it still learns to solve task in a trial-and-error process. We evaluate our algorithm on the robotic manipulation benchmark (Meta-World) and it significantly outperforms state-of-the-art methods in terms of training and testing task success rates. Codes are available at https://tumi6robot.wixsite.com/million.

Studying the manipulation of deformable linear objects has significant practical applications in industry, including car manufacturing, textile production, and electronics automation. However, deformable linear object manipulation poses a significant challenge in developing planning and control algorithms, due to the precise and continuous control required to effectively manipulate the deformable nature of these objects. In this paper, we propose a new framework to control and maintain the shape of deformable linear objects with two robot manipulators utilizing environmental contacts. The framework is composed of a shape planning algorithm which automatically generates appropriate positions to place fixtures, and an object-centered skill engine which includes task and motion planning to control the motion and force of both robots based on the object status. The status of the deformable linear object is estimated online utilizing visual as well as force information. The framework manages to handle a cable routing task in real-world experiments with two Panda robots and especially achieves contact-aware and flexible clip fixing with challenging fixtures.

The aim of this research is to investigate the relationship between spinal flexion and quadruped locomotion in a rat robot equipped with a compliant spine, controlled by a central pattern generator (CPG). The study reveals that spinal flexion can enhance limb stride length, but it may also cause significant and unexpected motion disturbances during stride length variations. To address this issue, this paper proposes a CPG model driven by spinal flexion and a novel oscillator that incorporates a circular limit cycle and accounts for the anticipated stride length transition process. This approach effectively matches the torque change with the dynamics of stride length changes, leading to lower energy consumption. Extensive simulations are conducted to evaluate the efficacy of the proposed oscillator and compare it with the original kinetic model and other CPG models. The results demonstrate that the designed CPG model with the proposed oscillator yields smoother gait transitions during stride length variations and reduces energy consumption.

Meta-reinforcement learning (meta-RL) is a promising approach that enables the agent to learn new tasks quickly. However, most meta-RL algorithms show poor generalization in multi-task scenarios due to the insufficient task information provided only by rewards. Language-conditioned meta-RL improves the generalization capability by matching language instructions with the agent's behaviors. While both behaviors and language instructions have symmetry, which can speed up human learning of new knowledge. Thus, combining symmetry and language instructions into meta-RL can help improve the algorithm's generalization and learning efficiency. We propose a dual-MDP meta-reinforcement learning method that enables learning new tasks efficiently with symmetrical behav-iors and language instructions. We evaluate our method in mul-tiple challenging manipulation tasks, and experimental results show that our method can greatly improve the generalization and learning efficiency of meta-reinforcement learning. Videos are available at https://tumi6robot.wixsite.com/symmetry/.

Lane keeping, as a fundamental functionality of autonomous navigation, remains a challenging task for autonomous robots and vehicles. Recently, spiking neural networks (SNNs) have gained attention and research interest due to their biological plausibility and application potential on neuromorphic processors. SNNs have also been successfully deployed on robots to solve autonomous navigation problems. However, lane keeping with a LiDAR sensor is still an open problem for SNNs. In this work, we propose an end-to-end approach based on an SNN to solve the lane-keeping problem using a 3D LiDAR sensor. For the first time, we explore the capability of the proposed SNN controller to perceive the LiDAR input and exploit the features to perform reward-based feedback learning. To ensure the effectiveness of the controller, the proposed method is deployed and evaluated on two high-fidelity simulators. The experimental results demonstrate the high applicability and performance in different scenarios. Furthermore, experiments show that the SNN is capable of performing lane keeping in a simulated urban environment with only 18 control neurons and 32 synapse connections, producing on average only a 17cm deviation from lane center, which is 4.3 % of the lane width.

In recent years, the subject of deep reinforcement learning (DRL) has developed very rapidly, and is now applied in various fields, such as decision making and control tasks. However, artificial agents trained with RL algorithms require great amounts of training data, unlike humans that are able to learn new skills from very few examples. The concept of meta-reinforcement learning (meta-RL) has been recently proposed to enable agents to learn similar but new skills from a small amount of experience by leveraging a set of tasks with a shared structure. Due to the task representation learning strategy with few-shot adaptation, most recent work is limited to narrow task distributions and stationary environments, where tasks do not change within episodes. In this work, we address those limitations and introduce a training strategy that is applicable to non-stationary environments, as well as a task representation based on Gaussian mixture models to model clustered task distributions. We evaluate our method on several continuous robotic control benchmarks. Compared with state-of-the-art literature that is only applicable to stationary environments with few-shot adaption, our algorithm first achieves competitive asymptotic performance and superior sample efficiency in stationary environments with zero-shot adaption. Second, our algorithm learns to perform successfully in non-stationary settings as well as a continual learning setting, while learning well-structured task representations. Last, our algorithm learns basic distinct behaviors and well-structured task representations in task distributions with multiple qualitatively distinct tasks.

The reduced model-based coordinated design of fault detectors and controllers for discrete-time switching fuzzy systems is examined. First, the mean-square exponential stabilization of switching Takagi–Sugeno fuzzy systems is performed using the average dwell time method under an arbitrary switching law. Next, using segmented Lyapunov function techniques, a dynamic full- and reduced-order fault detector and controller is designed to ensure that the overall dynamic residual system is mean-square exponentially stable with a balanced$\mathcal {H}_{\infty }$performance level$(\xi, \beta)$. The solvability conditions for the fault detector and controller are derived using a linearization method, and the relevant parameters can be determined using the mathematical linear matrix solver toolbox. Two examples including a switching Chua’s circuit system are presented to demonstrate the effectiveness of the proposed fault detector and controller.

LiDAR (Light Detection And Ranging) sensors have been widely used in autonomous vehicles as the main sensors. According to the specification details of the widely used 3D LiDAR products in the market, the distribution of vertical beam channels is set according to a uniform angular resolution, which is not ideally efficient for specific autonomous tasks. In this paper, we propose a novel approach to find the optimized angular distribution of the vertical beam channels for different application scenarios and installation configurations. The experimental results in a study case suggest that concerning the vehicle detection task, the optimized LiDARs perform almost two times better than the ones with the same number of channels in terms of the detection range, and have perception performances close to the LiDARs with double channels in the long distance.

By relabeling past experience with heuristic or curriculum goals, state-of-the-art reinforcement learning (RL) algorithms such as hindsight experience replay (HER), hindsight goal generation (HGG), and graph-based HGG (G-HGG) have been able to solve challenging robotic manipulation tasks in multigoal settings with sparse rewards. HGG outperforms HER in challenging tasks in which goals are difficult to explore by learning from a curriculum, in which intermediate goals are selected based on the Euclidean distance to target goals. G-HGG enhances HGG by selecting intermediate goals from a precomputed graph representation of the environment, which enables its applicability in an environment with stationary obstacles. However, G-HGG is not applicable to manipulation tasks with dynamic obstacles, since its graph representation is only valid in static scenarios and fails to provide any correct information to guide the exploration. In this article, we propose bounding-box-based HGG (Bbox-HGG), an extension of G-HGG selecting hindsight goals with the help of image observations of the environment, which makes it applicable to tasks with dynamic obstacles. We evaluate Bbox-HGG on four challenging manipulation tasks, where significant enhancements in both sample efficiency and overall success rate are shown over state-of-the-art algorithms. The videos can be viewed at https://videoviewsite.wixsite.com/bbhgg.

This article focuses on the problem of designing the reduced-order dynamic output feedback (DOF) controller for discrete-time T–S fuzzy plants. Differing from the existing methodologies, the reduced-order approximation technique is applied to simplify the pregiven high-order DOF controller. The key point is to construct a reduced-order closed-loop model to approximate the original high-order closed-loop system. First, a new error auxiliary system between the high-order closed-loop system and the reduced-order closed-loop model is obtained. The sufficient conditions to guarantee that the corresponding error system is asymptotically stable with a prescribed$\mathcal {H}_{\infty }$performance index are developed. Then, the parameters of the desired reduced-order controller are derived by utilizing the projection lemma and the cone complementary linearization algorithm. Finally, the advantages of the proposed technique are illustrated by a series of simulation analysis.

In nature, the movement of quadrupeds is completed under the combined action of the spine and the legs. Inspired by this, this paper explores the effect of a lateral flexing spine on the locomotion of a rat robot. Benefiting from the regular lateral flexion of a soft actuated spine, the rat robot exhibits enhance step length of its hind legs and increased translational velocity by coordinating the opposite movements of the left and right sides. Furthermore, this paper introduces a mathematical model of the effect of the flexible spine on the robot velocity. Finally, extensive experiments are conducted in simulations and on the physical rat robot. Compared with the locomotion without a flexing spine, the simulation results show that the velocity of the robot can be increased up to 218.29%, which is in line with the theoretical results from the proposed mathematical model. Limited by the gap between simulation and the real world, the experiment results of the physical rat robot show a slight performance than the theoretical results. But the physical rat robot can still enhance its translational velocity with the help of a lateral flexing spine.

The wheel-bipedal robot has the advantages of both wheeled robots and legged robots, but as a cost, it is more challenging to perform flexible movements in various surroundings while keeping it balanced. The inaccurate dynamics of the robot makes the balance problem even more intractable. To solve this problem, the robot Ollie is used as a testbed. The whole-body control (WBC) framework is adopted to enhance the dexterity of the robot with multiple degrees of freedom in the task space. Moreover, a learning-based adaptive technique is applied to assist the WBC such that the balance controller can be designed in the absence of the accurate dynamics. Physical experiments demonstrate that the robot can manage various actions, with the help of the combination of the WBC and the learning-based adaptive technique.

Simultaneous localization and mapping (SLAM) is one of the essential techniques and functionalities used by robots to perform autonomous navigation tasks. Inspired by the rodent hippocampus, this paper presents a biologically inspired SLAM system based on a LiDAR sensor using a hippocampal model to build a cognitive map and estimate the robot pose in indoor environments. Based on the biologically inspired models mimicking boundary cells, place cells, and head direction cells, the SLAM system using LiDAR point cloud data is capable of leveraging the self-motion cues from the LiDAR odometry and the boundary cues from the LiDAR boundary cells to build a cognitive map and estimate the robot pose. Experiment results show that with the LiDAR boundary cells the proposed SLAM system greatly outperforms the camera-based brain-inspired method in both simulation and indoor environments, and is competitive with the conventional LiDAR-based SLAM methods.

Localization in the environment is an essential navigational capability for animals and indoor robotic vehicles. In indoor environments, it is still challenging to perfectly solve the global localization problem using probabilistic methods. However, animals are able to instinctively localize themselves with much less effort. Therefore, an intriguing and promising approach is to seek biological inspiration from animals. In this paper, we present a biologically-inspired global localization system using a LiDAR sensor that utilizes a hippocampal model and a landmark-based relocalization approach. The experiment results show that the proposed method is competitive with Monte Carlo Localization, and the results demonstrate the high accuracy, applicability, and reliability of the proposed biologically-inspired localization system in various localization scenarios.

Reinforcement learning algorithms, such as hindsight experience replay (HER) and hindsight goal generation (HGG), have been able to solve challenging robotic manipulation tasks in multigoal settings with sparse rewards. HER achieves its training success through hindsight replays of past experience with heuristic goals but underperforms in challenging tasks in which goals are difficult to explore. HGG enhances HER by selecting intermediate goals that are easy to achieve in the short term and promising to lead to target goals in the long term. This guided exploration makes HGG applicable to tasks in which target goals are far away from the object's initial position. However, the vanilla HGG is not applicable to manipulation tasks with obstacles because the Euclidean metric used for HGG is not an accurate distance metric in such an environment. Although, with the guidance of a handcrafted distance grid, grid-based HGG can solve manipulation tasks with obstacles, a more feasible method that can solve such tasks automatically is still in demand. In this article, we propose graph-based hindsight goal generation (G-HGG), an extension of HGG selecting hindsight goals based on shortest distances in an obstacle-avoiding graph, which is a discrete representation of the environment. We evaluated G-HGG on four challenging manipulation tasks with obstacles, where significant enhancements in both sample efficiency and overall success rate are shown over HGG and HER. Videos can be viewed at https://videoviewsite.wixsite.com/ghgg.

As a vital cognitive function of animals, the navigation skill is first built on the accurate perception of the directional heading in the environment. Head direction cells (HDCs), found in the limbic system of animals, are proven to play an important role in identifying the directional heading allocentrically in the horizontal plane, independent of the animal's location and the ambient conditions of the environment. However, practical HDC models that can be implemented in robotic applications are rarely investigated, especially those that are biologically plausible and yet applicable to the real world. In this article, we propose a computational HDC network that is consistent with several neurophysiological findings concerning biological HDCs and then implement it in robotic navigation tasks. The HDC network keeps a representation of the directional heading only relying on the angular velocity as an input. We examine the proposed HDC model in extensive simulations and real-world experiments and demonstrate its excellent performance in terms of accuracy and real-time capability.

The snake-like robot without wheels is a bio-inspired robot whose high degree of freedom results in a challenge in autonomous locomotion control. The use of a Spiking Neural Network (SNN) which is a biologically plausible artificial neural network can help to achieve the autonomous locomotion behavior of snake robots in an energy-efficient manner. Approaches that use an SNN without hidden layers have been applied in the single-target tracking task. However, due to the complexity of the 3D gaits on a wheel-less snake robot and the imprecision of the pose control while in motion, they have some fluctuation that adversely affects their performances. In this work, we design two multi-layered SNNs with different topology for a wheel-less snake robot to track a certain moving object. The visual signals obtained from a Dynamic Vision Sensor (DVS) are fed into the SNN to drive the locomotion controller. Furthermore, the Reward-modulated Spike-Timing-Dependent Plasticity (R-STDP) learning rule is utilized to train the SNN end-to-end. Compared to the SNN without hidden layers, the proposed multi-layered SNN with a separated hidden layer shows its advantage in terms of robustness.

This paper introduces an end-to-end learning approach based on Reward-modulated Spike-Timing-Dependent Plasticity (R-STDP) for a multi-layered spiking neural network (SNN). As a case study, a snake-like robot is used as an agent to perform target tracking tasks on the basis of our proposed approach. Since the key of R-STDP is to use rewards to modulate synapse strengthens, we first propose a general way to propagate the reward back through a multi-layered SNN. Upon the proposed approach, we build up an SNN controller that drives a snake-like robot for performing target tracking tasks. We demonstrate the practicability and advantage of our approach in terms of lateral tracking accuracy by comparing it to other state-of-the-art learning algorithms for SNNs based on R-STDP.

Pedestrian detection has attracted enormous research attention in the field of Intelligent Transportation System (ITS) due to that pedestrians are the most vulnerable traffic participants. So far, almost all pedestrian detection solutions are based on the conventional frame-based camera. However, they cannot perform very well in scenarios with bad light condition and high-speed motion. In this work, a Dynamic and Active Pixel Sensor (DAVIS), whose two channels concurrently output conventional gray-scale frames and asynchronous low-latency temporal contrast events of light intensity, was first used to detect pedestrians in a traffic monitoring scenario. Data from two camera channels were fed into Convolutional Neural Networks (CNNs) including three YOLOv3 models and three YOLO-tiny models to gather bounding boxes of pedestrians with respective confidence map. Furthermore, a confidence map fusion method combining the CNN-based detection results from both DAVIS channels was proposed to obtain higher accuracy. The experiments were conducted on a custom dataset collected on TUM campus. Benefiting from the high speed, low latency and wide dynamic range of the event channel, our method achieved higher frame rate and lower latency than those only using a conventional camera. Additionally, it reached higher average precision by using the fusion approach.

Similar to their counterparts in nature, the flexible bodies of snake-like robots enhance their movement capability and adaptability in diverse environments. However, this flexibility corresponds to a complex control task involving highly redundant degrees of freedom, where traditional model-based methods usually fail to propel the robots energy-efficiently. In this work, we present a novel approach for designing an energy-efficient slithering gait for a snake-like robot using a model-free reinforcement learning (RL) algorithm. Specifically, we present an RL-based controller for generating locomotion gaits at a wide range of velocities, which is trained using the proximal policy optimization (PPO) algorithm. Meanwhile, a traditional parameterized gait controller is presented and the parameter sets are optimized using the grid search and Bayesian optimization algorithms for the purposes of reasonable comparisons. Based on the analysis of the simulation results, we demonstrate that this RL-based controller exhibits very natural and adaptive movements, which are also substantially more energy-efficient than the gaits generated by the parameterized controller. Videos are shown at https://videoviewsite.wixsite.com/rlsnake .

Learning-based methods have demonstrated clear advantages in controlling robot tasks, such as the information fusion abilities, strong robustness, and high accuracy. Meanwhile, the on-board systems of robots have limited computation and energy resources, which are contradictory with state-of-the-art learning approaches. They are either too lightweight to solve complex problems or too heavyweight to be used for mobile applications. On the other hand, training spiking neural networks (SNNs) with biological plausibility has great potentials of performing fast computation and energy efficiency. However, the lack of effective learning rules for SNNs impedes their wide usage in mobile robot applications. This paper addresses the problem by introducing an end to end learning approach of spiking neural networks for a lane keeping vehicle. We consider the reward-modulated spike-timing-dependent-plasticity (R-STDP) as a promising solution in training SNNs, since it combines the advantages of both reinforcement learning and the well-known STDP. We test our approach in three scenarios that a Pioneer robot is controlled to keep lanes based on an SNN. Specifically, the lane information is encoded by the event data from a neuromorphic vision sensor. The SNN is constructed using R-STDP synapses in an all-to-all fashion. We demonstrate the advantages of our approach in terms of the lateral localization accuracy by comparing with other state-of-the-art learning algorithms based on SNNs.

In this paper, we report on a body state and ground profile estimator for a snake-like robot executing a rolling gait to travel from flat ground to a slope. With the help of the estimator, the snake-like robot can adaptively adjust the body shape and locomotion speed by changing the gait parameters for the purpose of tackling a steep slope. Specifically, we propose a repeating sequence of continuous time dynamical models to fuse kinematic encoder data with on-board Inertial Measurement Unit (IMU) measurements based on extended Kalman filter (EKF). All the sensors are mounted inside each module of the snake-like robot, which measure the joint position, the three-axis acceleration, and the three-axis angular velocity. Further, the robot changes its moving pattern under our policy, judging by the estimated angle of the ground profile. We implement this estimation procedure off-line, using data extracted from repeated runs of the snake-like robot by simulation and evaluate its performance compared to the ground truth.

In this paper, a lightweight central pattern generator(CPG) model is designed for a snake-like robot, to achieve smooth transition of body shape and locomotion speed. First, based on the convergence behavior of the gradient system, a lightweight CPG model with fast computing time is designed and compared with other widely adopted CPG models. Then, the body shape and locomotion speed transitions in rolling gait are simulated based on the proposed CPG model. Compared with the sinusoid-based method, a smooth transition process can be achieved, without generating undesired movement or abnormal torque. Finally, extensive prototype experiments are conducted to demonstrate that the CPG-based control can effectively ensure smooth transition process and avoid abnormal torque, when the body shape and locomotion speed are changed.

In this paper, a biologically inspired 3D slithering gait for a snake-like robot is designed and implemented for the purpose of target tracking. First, by balancing the forward speed and the stability of the robot, a straight slithering gait is modelled, under which the robot can march straight, fast, and stably. Then, for the purpose of steering, the straight slithering gait is modified into a biased slithering gait. The relationship between turning radius and gait parameters is analyzed by the resistive force theory. With the head composition algorithm, we investigate the orientation problem of the snake robot's head module to obtain stable visual information during the locomotion process. Finally, with the guidance of the vision sensor mounted in the head module, target tracking simulations and prototype experiments are conducted to demonstrate the practicality and effectiveness of the slithering gait in autonomous locomotion scenarios.