As labor shortages and safety regulations become more prominent, the need for human-in-the-loop automation of excavators is increasing. To meet this demand, we have developed a comprehensive modeling method for the excavator arm using nonlinear optimization approaches, including a simplified model that maps the task space to the joint space, as well as an equivalent model that maps the joint space to the actuator space. These models were then used to build a feedforward-PID joint velocity controller and a joint trajectory controller combined with position feedback, which forms the core of our proposed semi-automatic control system for the excavator arm. Our deployment scheme is simple and efficient, and has been deployed on two excavators of different makes and sizes. Experiments show that our deployment scheme performs well on both excavators, with an average error of 0.05 rad/s for the velocity controller and less than 5 cm for the trajectory controller. Using our semi-automatic system, we have completed demonstration experiments for precise digging and grading operations. A demonstration video can be found at https://youtu.be/N6I0WZGSF68.

Deep reinforcement learning (DRL) algorithms have proven effective in robot navigation, especially in unknown environments, by directly mapping perception inputs into robot control commands. However, most existing methods ignore the local minimum problem in navigation and thereby cannot handle complex unknown environments. In this paper, we propose the first DRL-based navigation method modeled by a semi-Markov decision process (SMDP) with continuous action space, named Adaptive Forward Simulation Time (AFST), to overcome this problem. Specifically, we reduce the dimensions of the action space and improve the distributed proximal policy optimization (DPPO) algorithm for the specified SMDP problem by modifying its GAE to better estimate the policy gradient in SMDPs. Experiments in various unknown environments demonstrate the effectiveness of AFST.

Existing navigation policies for autonomous robots tend to focus on collision avoidance while ignoring human-robot interactions in social life. For instance, robots can pass along the corridor safer and easier if pedestrians notice them. Sounds have been considered as an efficient way to attract the attention of pedestrians, which can alleviate the freezing robot problem. In this work, we present a new deep reinforcement learning (DRL) based social navigation approach for autonomous robots to move in pedestrian-rich environments with interaction capacity. Most existing DRL based methods intend to train a general policy that outputs both navigation actions, i.e., expected robot's linear and angular velocities, and interaction actions, i.e., the beep action, in the context of reinforcement learning. Different from these methods, we intend to train the policy via both supervised learning and reinforcement learning. In specific, we first train an interaction policy in the context of supervised learning, which provides a better understanding of the social situation, then we use this interaction policy to train the navigation policy via multiple reinforcement learning algorithms. We evaluate our approach in various simulation environments and compare it to other methods. The experimental results show that our approach outperforms others in terms of the success rate. We also deploy the trained policy on a real-world robot, which shows a nice performance in crowded environments.

In this paper, we propose a map-based end-to-end DRL approach for three-dimensional (3D) obstacle avoidance in a partially observed environment, which is applied to achieve autonomous navigation for an indoor mobile robot using a depth camera with a narrow field of view. We first train a neural network with LSTM units in a 3D simulator of mobile robots to approximate the Q-value function in double DRQN. We also use a curriculum learning strategy to accelerate and stabilize the training process. Then we deploy the trained model to a real robot to perform 3D obstacle avoidance in its navigation. We evaluate the proposed approach both in the simulated environment and on a robot in the real world. The experimental results show that the approach is efficient and easy to be deployed, and it performs well for 3D obstacle avoidance with a narrow observation angle, which outperforms other existing DRL-based models by 15.5% on success rate.

It is challenging for a mobile robot to navigate through human crowds. Existing approaches usually assume that pedestrians follow a predefined collision avoidance strategy, like social force model (SFM) or optimal reciprocal collision avoidance (ORCA). However, their performances commonly need to be further improved for practical applications, where pedestrians follow multiple different collision avoidance strategies. In this paper, we propose a map-based deep reinforcement learning approach for crowd-aware robot navigation with various pedestrians. We use the sensor map to represent the environmental information around the robot, including its shape and observable appearances of obstacles. We also introduce the pedestrian map that specifies the movements of pedestrians around the robot. By applying both maps as inputs of the neural network, we show that a navigation policy can be trained to better interact with pedestrians following different collision avoidance strategies. We evaluate our approach under multiple scenarios both in the simulator and on an actual robot. The results show that our approach allows the robot to successfully interact with various pedestrians and outperforms compared methods in terms of the success rate.

Self-play has been shown to be effective to provide a proper training curriculum for a reinforcement learning agent in competitive multi-agent environments without direct supervision. However, its performance is still unstable for problems with sparse rewards, e.g., the scoring task with goalkeeper for robots in RoboCup soccer. It is challenging to solve these tasks in reinforcement learning, especially for those that require combining high-level actions with flexible control. To address these challenges, we introduce a distributed self-play training framework for an extended proximal policy optimization (PPO) algorithm that learns to act in parameterized action space and plays against a group of opponents, i.e., a league. Experiments on the domain of simulated RoboCup soccer show that, the approach is effective and learns more robust policies against various opponents compared to existing reinforcement learning methods. A demonstration video is available online at https://youtu.be/BuLli1vND4.

Multi-robot collision avoidance in a communication-free environment is one of the key issues for mobile robotics and autonomous driving. In this paper, we propose a map-based deep reinforcement learning (DRL) approach for collision avoidance of multiple robots, where robots do not communicate with each other and only sense other robots' positions and the obstacles around them. We use the egocentric grid map of a robot to represent the environmental information around it, which can be easily generated by using multiple sensors or sensor fusion. The learned policy generated from the DRL model directly maps 3 frames of egocentric grid maps and the robot's relative local goal positions into low-level robot control commands. We first train a convolutional neural network for the navigation policy in a simulator of multiple mobile robots using proximal policy optimization (PPO). Then we deploy the trained model to real robots to perform collision avoidance in their navigation. We evaluate the approach with various scenarios both in the simulator and on three differential-drive mobile robots in the real world. Both qualitative and quantitative experiments show that our approach is efficient with a high success rate. The demonstration video can be found at https://youtu.be/jcLKlEXuFuk.

It is meaningful to use data mining technology to predict the type of lawsuit which a company may receive so that enterprises can avoid lawsuit risks. So we propose a corporation lawsuit prediction algorithm based on guiding learning and collaborative filtering recommendation. Firstly, we use the adaptive synthetic sampling approach (ADASYN) to generate more synthetic data for different minority classes according to their different level of difficulty in learning, so that the training would focus on these minority classes that are difficulty to learn and reduce the learning bias introduced by the imbalance of data distribution. Secondly, for the sake of solving the problem that the insufficient samples make it difficult for the model to learn enough knowledge resulting in a large fluctuation of final scores during the training and poor model stability, we use guiding learning to integrate the basic knowledge of all types of lawsuit a company may receive in the future obtained by the multi-label classification model into the training process of TOP-1 and TOP-2 predictive models. Finally, in order to further improve the prediction accuracy, we use the collaborative filtering recommendation algorithm (CFRA) to select the most similar sample with each test sample from the training set, and the lawsuit type of the selected sample is directly used as the predicted lawsuit type of the corresponding test sample, thereby improving the total prediction accuracy. The experimental results show that the proposed algorithm can effectively predict the most probable lawsuit types of the Top2 for corporations.

When employed in systems with higher frequency variability and nonideal supply voltage, the synchronization method used in grid-connected power electronic converter control systems must be considered. A method is presented here that achieves synchronization implicitly through the use of a discrete Fourier transform in combination with the computation of an effective admittance. It is shown that the method is suitable for systems with high frequency variability as well as for systems with nonideal supply voltage including heavy voltage asymmetry.