Visual SLAM with thermal imagery remains a difficult problem for many state of the art (SOTA) algorithms. Compared with visible spectrum imagery, thermal imagery generally has lower contrast, higher noise, and tends to have lower resolution, making for challenging front-end data association. Thermal imagery also presents a difficult problem for long term relocalization and map reuse, because the relative temperatures of objects in thermal imagery tend to change dramatically from day to night. Feature descriptors typically used for relocalization in SLAM are unable to maintain consistency over these diurnal changes. We show that learned feature descriptors can be used within existing bag of word based localization schemes to dramatically improve place recognition across large temporal gaps in thermal imagery. In order to demonstrate the effectiveness of our trained vocabulary, we have developed a baseline SLAM system, integrating learned features and matching into a classical SLAM algorithm. Our system demonstrates good local tracking on challenging thermal imagery, and relocalization that overcomes dramatic day to night thermal appearance changes. Our code and datasets are available here: https://github.com/neufieldrobotics/IRSLAM_Baseline

This paper reports on Team Northeastern's Avatar system for telepresence, and our holistic approach to meet the ANA Avatar XPRIZE Final testing task requirements. The system features a dual-arm configuration with hydraulically actuated glove-gripper pair for haptic force feedback. Our proposed Avatar system was evaluated in the ANA Avatar XPRIZE Finals and completed all 10 tasks, scored 14.5 points out of 15.0, and received the 3rd Place Award. We provide the details of improvements over our first generation Avatar, covering manipulation, perception, locomotion, power, network, and controller design. We also extensively discuss the major lessons learned during our participation in the competition.

There has been a drastic shift to remote interaction for professional, industrial and personal interactions. Improving the overall quality of these interactions by removing any sense of distance between the users is the ultimate goal. Video conferencing has been widely adopted as an improvement to audio-only interactions. Having added visuals to audio communication, the next frontier is to add physical interaction to this remote communication. In this paper, we present an avatar system with the aim of tackling these necessities. The proposed system includes both hardware and software designs to ensure a real-time telemanipulation experience with tactile force feedback. We present a coupled hydrostatic actuated gripper and glove with high system bandwidth to reduce the inherent latency of the mechanical system. To account for latency over the network, the wave variable based method is adopted to maintain the stability of the closed-loop gripper control even under hundreds of milliseconds of delay. A bidirectional audiovisual communication system comprised of off-the-shelf hardware and software is incorporated to allow realtime conversation between the operator and the recipient for collaborative tasks. the proposed system has been validated in lab experiments and the global ana avatar xprize challenge semifinal.

Grasp detection of novel objects in unstructured environments is a key capability in robotic manipulation. For 2D grasp detection problems where grasps are assumed to lie in the plane, it is common to design a fully convolutional neural network that predicts grasps over an entire image in one step. However, this is not possible for grasp pose detection where grasp poses are assumed to exist in SE(3). In this case, it is common to approach the problem in two steps: grasp candidate generation and candidate classification [1], [2], [3], [4]. Since grasp candidate classification is typically expensive, the problem becomes one of efficiently identifying high quality candidate grasps. This paper proposes a new grasp candidate generation method that significantly outperforms major 3D grasp detection baselines. Supplementary material is available at this website.

Localizing and tracking the pose of robotic grippers are necessary skills for manipulation tasks. However, the manipulators with imprecise kinematic models (e.g. low-cost arms) or manipulators with unknown world coordinates (e.g. poor camera-arm calibration) cannot locate the gripper with respect to the world. In these circumstances, we can leverage tactile feedback between the gripper and the environment. In this paper, we present learnable Bayes filter models that can localize robotic grippers using tactile feedback. We propose a novel observation model that conditions the tactile feedback on visual maps of the environment along with a motion model to recursively estimate the gripper's location. Our models are trained in simulation with self-supervision and transferred to the real world. Our method is evaluated on a tabletop localization task in which the gripper interacts with objects. We report results in simulation and on a real robot, generalizing over different sizes, shapes, and configurations of the objects.