Progress in Embodied AI has made it possible for end-to-end-trained agents to navigate in photo-realistic environments with high-level reasoning and zero-shot or language-conditioned behavior, but benchmarks are still dominated by simulation. In this work, we focus on the fine-grained behavior of fast-moving real robots and present a large-scale experimental study involving 262 navigation episodes in a real environment with a physical robot, where we analyze the type of reasoning emerging from end-to-end training. In particular, we study the presence of realistic dynamics which the agent learned for open-loop forecasting, and their interplay with sensing. We analyze the way the agent uses latent memory to hold elements of the scene structure and information gathered during exploration. We probe the planning capabilities of the agent, and find in its memory evidence for somewhat precise plans over a limited horizon. Furthermore, we show in a post-hoc analysis that the value function learned by the agent relates to long-term planning. Put together, our experiments paint a new picture on how using tools from computer vision and sequential decision making have led to new capabilities in robotics and control. An interactive tool is available [here].

In the context of autonomous navigation of terrestrial robots, the creation of realistic models for agent dynamics and sensing is a widespread habit in the robotics literature and in commercial applications, where they are used for model based control and/or for localization and mapping. The more recent Embodied AI literature, on the other hand, focuses on modular or end-to-end agents trained in simulators like Habitat or AI-Thor, where the emphasis is put on photo-realistic rendering and scene diversity, but high-fidelity robot motion is assigned a less privileged role. The resulting sim2real gap significantly impacts transfer of the trained models to real robotic platforms. In this work we explore end-to-end training of agents in simulation in settings which minimize the sim2real gap both, in sensing and in actuation. Our agent directly predicts (discretized) velocity commands, which are maintained through closed-loop control in the real robot. The behavior of the real robot (including the underlying low-level controller) is identified and simulated in a modified Habitat simulator. Noise models for odometry and localization further contribute in lowering the sim2real gap. We evaluate on real navigation scenarios, explore different localization and point goal calculation methods and report significant gains in performance and robustness compared to prior work.

This paper examines the advantages and disadvantages of combining Human-Like and Machine-Like behaviors for a robot taking a shared elevator with a bystander as part of an office delivery service scenario. We present findings of an in-person wizard-of-oz experiment that builds on and implements behavior policies developed in a previous study. In this experiment, we found that the combination of Machine-Like and Human-Like behaviors was perceived as better than Human-Like behaviors alone. We discuss possible reasons and point to key capabilities that a socially competent robot should have to achieve better Human-Like behaviors in order to seamlessly negotiate a social encounter with bystanders in a shared elevator or similar scenario. We found that establishing and maintaining a shared transactional space is one of these key requirements.

We address the problem of large-scale topic classification of web pages based on the minimal text available in the URLs. This problem is challenging because of the sparsity of feature vectors that are derived from the URL text, and the typical asymmetry between the cardinality of train and test sets due to non-availability of sufficient sets of annotated URLs for training and very large test sets (e.g., in the case of large-scale focused crawling). We propose an online incremental learning algorithm which addresses these issues. Our experiments based on large publicly available datasets demonstrate an improvement of 0.11 -- 0.12 in terms of F-measure over the baseline algorithms, like Support Vector Machine, in difficult scenarios where the cardinality of train set is just a fraction of that of the test set.

The problem of large-scale image search has been traditionally addressed with the bag-of-visual-words (BOV). In this article, we propose to use as an alternative the Fisher kernel framework. We first show why the Fisher representation is well-suited to the retrieval problem: it describes an image by what makes it different from other images. One drawback of the Fisher vector is that it is high-dimensional and, as opposed to the BOV, it is dense. The resulting memory and computational costs do not make Fisher vectors directly amenable to large-scale retrieval. Therefore, we compress Fisher vectors to reduce their memory footprint and speed-up the retrieval. We compare three binarization approaches: a simple approach devised for this representation and two standard compression techniques. We show on two publicly available datasets that compressed Fisher vectors perform very well using as little as a few hundreds of bits per image, and significantly better than a very recent compressed BOV approach.