In goal-conditioned hierarchical reinforcement learning (HRL), a high-level policy specifies a subgoal for the low-level policy to reach. Effective HRL hinges on a suitable subgoal representation function, abstracting state space into latent subgoal space and inducing varied low-level behaviors. Existing methods adopt a subgoal representation that provides a deterministic mapping from state space to latent subgoal space. Instead, this paper utilizes Gaussian Processes (GPs) for the first probabilistic subgoal representation. Our method employs a GP prior on the latent subgoal space to learn a posterior distribution over the subgoal representation functions while exploiting the long-range correlation in the state space through learnable kernels. This enables an adaptive memory that integrates long-range subgoal information from prior planning steps allowing to cope with stochastic uncertainties. Furthermore, we propose a novel learning objective to facilitate the simultaneous learning of probabilistic subgoal representations and policies within a unified framework. In experiments, our approach outperforms state-of-the-art baselines in standard benchmarks but also in environments with stochastic elements and under diverse reward conditions. Additionally, our model shows promising capabilities in transferring low-level policies across different tasks.

Genomic selection (GS) has emerged as an effective technology to accelerate crop hybrid breeding by enabling early selection prior to phenotype collection. Genomic best linear unbiased prediction (GBLUP) is a robust method that has been routinely used in GS breeding programs. However, GBLUP assumes that markers contribute equally to the total genetic variance, which may not be the case. In this study, we developed a novel GS method called GA-GBLUP that leverages the genetic algorithm (GA) to select markers related to the target trait. We defined four fitness functions for optimization, including AIC, BIC, R2, and HAT, to improve the predictability and bin adjacent markers based on the principle of linkage disequilibrium to reduce model dimension. The results demonstrate that the GA-GBLUP model, equipped with R2 and HAT fitness function, produces much higher predictability than GBLUP for most traits in rice and maize datasets, particularly for traits with low heritability. Moreover, we have developed a user-friendly R package, GAGBLUP, for GS, and the package is freely available on CRAN (https://CRAN.R-project.org/package=GAGBLUP).

Robot control for tactile feedback based manip-ulation can be difficult due to modeling of physical contacts, partial observability of the environment, and noise in perception and control. This work focuses on solving partial observability of contact-rich manipulation tasks as a Sequence-to-Sequence (Seq2Seq) Imitation Learning (IL) problem. The proposed Seq2Seq model first produces a robot-environment interaction sequence to estimate the partially observable environment state variables, and then, the observed interaction sequence is transformed to a control sequence for the task itself. The proposed Seq2Seq IL for tactile feedback based manipulation is experimentally validated on a door-open task in a simulated environment and a snap-on insertion task with a real robot. The model is able to learn both tasks from only 50 expert demonstrations while state-of-the-art reinforcement learning and imitation learning methods fail.

As a supplement to synthetic aperture radar interferometry (InSAR), multiple-aperture InSAR (MAI) can measure along-track surface deformation, but it is limited by ionospheric path delays, especially with the L- or P-band data. In this letter, we propose a method to correct an ionospheric error in the MAI measurement based on the along-track gradient and 1-D linear fitting. The method depends on the uniqueness of the spatial variation of the along-track gradient of ionospheric error in MAI measurements, which can be well distinguished from other components, such as deformation by using 1-D linear fitting. The method is first evaluated by employing the L-band ALOS-2 PALSAR-2 dataset of the 2019 Ridgecrest earthquake, U.S., and then applied to estimate glacial movements of Grove Mountain, Antarctica, with the ALOS-2 PALSAR-2 dataset.

We have recently proposed two pile loading controllers that learn from human demonstrations: a neural network (NNet) [1] and a random forest (RF) controller [2]. In the field experiments the RF controller obtained clearly better success rates. In this work, the previous findings are drastically revised by experimenting summer time trained controllers in winter conditions. The winter experiments revealed a need for additional sensors, more training data, and a controller that can take advantage of these. Therefore, we propose a revised neural controller (NNetV2) which has a more expressive structure and uses a neural attention mechanism to focus on important parts of the sensor and control signals. Using the same data and sensors to train and test the three controllers, NNetV2 achieves better robustness against drastically changing conditions and superior success rate. To the best of our knowledge, this is the first work testing a learning-based controller for a heavy-duty machine in drastically varying outdoor conditions and delivering high success rate in winter, being trained in summer.

Simulation to real (Sim-to-Real) is an attractive approach to construct controllers for robotic tasks that are easier to simulate than to analytically solve. Working Sim-to-Real solutions have been demonstrated for tasks with a clear single objective such as "reach the target". Real world applications, however, often consist of multiple simultaneous objectives such as "reach the target" but "avoid obstacles". A straightforward solution in the context of reinforcement learning (RL) is to combine multiple objectives into a multi-term reward function and train a single monolithic controller. Recently, a hybrid solution based on pre-trained single objective controllers and a switching rule between them was proposed. In this work, we compare these two approaches in the multi-objective setting of a robot manipulator to reach a target while avoiding an obstacle. Our findings show that the training of a hybrid controller is easier and obtains a better success-failure trade-off than a monolithic controller. The controllers trained in simulator were verified by a real set-up.

Pedestrian trajectory prediction is essential for collision avoidance in autonomous driving, which can help autonomous vehicles have a better understanding of traffic environment and perform tasks such as risk assessment in advance. In this paper, pedestrian path prediction at a time horizon of 2s for autonomous driving is systematically investigated using waiting/crossing decision model (W/CDM) and modified social force model (MSFM), and the possible conflict between pedestrians and straight-going vehicles at an un-signalized crosswalk is focused on. First of all, a W/CDM is efficiently developed to judge pedestrians' waiting/crossing intentions when a straight-going vehicle is approaching. Then the humanoid micro-dynamic MSFM of pedestrians who have been judged to cross is characterized by taking into account the evasion with conflicting pedestrians, the collision avoidance with straight-going vehicles, and the reaction to crosswalk boundary. The influence of pedestrian heterogeneous characteristics is considered for the first time. Moreover, aerial video data of pedestrians and vehicles at an un-signalized crosswalk is collected and analyzed for model calibration. Maximum likelihood estimation (MLE) is proposed to calibrate the non-measurable parameters of the proposed models. Finally, the model validation is conducted with two cases by comparing with the existing methods. The result reveals that the integrated method (W/CDM-MSFM) outperforms the existing methods and accurately predicts the path of pedestrians, which can give us great confidence to use the current method to predict the path of pedestrian for autonomous driving with significant accuracy and highly improve pedestrian safety.

This work introduces a learning-based pile loading controller for autonomous robotic wheel loaders. Controller parameters are learnt from a small number of demonstrations for which low level sensor (boom angle, bucket angle and hydrostatic driving pressure), egocentric video frames and control signals are recorded. Application specific deep visual features are learnt from demonstrations using a Siamese network architecture and a combination of cross-entropy and contrastive loss. The controller is based on a Random Forest (RF) regressor that provides robustness against changes in field conditions (loading distance, soil type, weather and illumination). The controller is deployed to a real autonomous robotic wheel loader and it outperforms prior art with a clear margin.

We introduce a high-resolution equirectangular panorama (aka 360-degree, virtual reality, VR) dataset for object detection and propose a multi-projection variant of the YOLO detector. The main challenges with equirectangular panorama images are i) the lack of annotated training data, ii) high-resolution imagery and iii) severe geometric distortions of objects near the panorama projection poles. In this work, we solve the challenges by I) using training examples available in the “conventional datasets” (ImageNet and COCO), II) employing only low resolution images that require only moderate GPU computing power and memory, and III) our multi-projection YOLO handles projection distortions by making multiple stereographic sub-projections. In our experiments, YOLO outperforms the other state-of-the-art detector, Faster R-CNN, and our multi-projection YOLO achieves the best accuracy with low-resolution input.

Beauty estimation is a common method for landscape quality estimation, although it has some limitations. With eye tracker, the visual behaviors of the subjects during the estimation can be recorded. Through the analyses of heat maps, path maps and eye movement data, the psychological changes of the subjects and the underlying law of beauty aesthetic can be understood, which will provide supplementation to beauty estimation. This paper studied the beauty estimation of urban waterfront parks and proofed that the landscape quality estimation method focussing on beauty estimation and assisted by eye movement tracking is feasible. It can improve the objectiveness and accuracy of landscape quality estimation to some extent and provide a comprehensive understanding of the effects and combination law of landscape characteristic elements.