Compliance plays a crucial role in manipulation, as it balances between the concurrent control of position and force under uncertainties. Yet compliance is often overlooked by today's visuomotor policies that solely focus on position control. This paper introduces Adaptive Compliance Policy (ACP), a novel framework that learns to dynamically adjust system com-pliance both spatially and temporally for given manipulation tasks from human demonstrations, improving upon previous approaches that rely on pre-selected compliance parameters or assume uniform constant stiffness. However, computing full compliance parameters from human demonstrations is an ill- defined problem. Instead, we estimate an approximate compli-ance profile with two useful properties: avoiding large contact forces and encouraging accurate tracking. Our approach en-ables robots to handle complex contact-rich manipulation tasks and achieves over 50% performance improvement compared to state-of-the-art visuomotor policy methods. Project website with result videos: adaptive-compliance.github.io.

Achieving robust dexterous manipulation in un-structured domestic environments remains a significant challenge in robotics. Even with state-of-the-art robot learning methods, haptic-oblivious control strategies (i.e. those relying only on external vision and/or proprioception) often fall short due to occlusions, visual complexities, and the need for precise contact interaction control. To address these limitations, we introduce PolyTouch, a novel robot finger that integrates camera-based tactile sensing, acoustic sensing, and peripheral visual sensing into a single design that is compact and durable. PolyTouch provides high-resolution tactile feedback across multiple temporal scales, which is essential for efficiently learning complex manipulation tasks. Experiments demonstrate an at least 20-fold increase in lifespan over commercial tactile sensors, with a design that is both easy to manufacture and scalable. We then use this multimodal tactile feedback along with visuo-proprioceptive observations to synthesize a tactile-diffusion policy from human demonstrations; the resulting contact-aware control policy significantly outperforms haptic-oblivious policies in multiple contact-aware manipulation policies. This paper highlights how effectively integrating multimodal contact sensing can hasten the development of effective contact-aware manipulation policies, paving the way for more reliable and versatile domestic robots. More information can be found at https://polytouch.alanz.info/.

Automating garment manipulation is challenging due to extremely high variability in object configurations. To reduce this intrinsic variation, we introduce the task of “canonicalized-alignment” that simplifies downstream applications by reducing the possible garment configurations. This task can be considered as “cloth state funnel” that manipulates arbitrarily configured clothing items into a predefined deformable configuration (i.e. canonicalization) at an appropriate rigid pose (i.e. alignment). In the end, the cloth items will result in a compact set of structured and highly visible configurations - which are desirable for downstream manipulation skills. To enable this task, we propose a novel canonicalized-alignment objective that effectively guides learning to avoid adverse local minima during learning. Using this objective, we learn a multi-arm, multi-primitive policy that strategically chooses between dynamic flings and quasi-static pick and place actions to achieve efficient canonicalized-alignment. We evaluate this approach on a real-world ironing and folding system that relies on this learned policy as the common first step. Empirically, we demonstrate that our task-agnostic canonicalized-alignment can enable even simple manually -designed policies to work well where they were pre-viously inadequate, thus bridging the gap between automated non-deformable manufacturing and deformable manipulation.

We introduce a practical robotics solution for the task of heterogeneous bagging, requiring the placement of multiple rigid and deformable objects into a deformable bag. This is a difficult task as it features complex interactions between multiple highly deformable objects under limited observability. To tackle these challenges, we propose a robotic system consisting of two learned policies: a rearrangement policy that learns to place multiple rigid objects and fold deformable objects in order to achieve desirable pre-bagging conditions, and a lifting policy to infer suitable grasp points for bi-manual bag lifting. We evaluate these learned policies on a real-world three-arm robot platform that achieves a 70% heterogeneous bagging success rate with novel objects. To facilitate future research and comparison, we also develop a novel heterogeneous bagging simulation benchmark that will be made publicly available.

Phonemes are defined by their relationship to words: changing a phoneme changes the word.Learning a phoneme inventory with little supervision has been a longstanding challenge with important applications to underresourced speech technology.In this paper, we bridge the gap between the linguistic and statistical definition of phonemes and propose a novel neural discrete representation learning model for self-supervised learning of phoneme inventory with raw speech and word labels.Given the availability of phoneme segmentation and some mild conditions, we prove that the phoneme inventory learned by our approach converges to the true one with an exponentially low error rate.Moreover, in experiments on TIMIT and Mboshi benchmarks, our approach consistently learns a better phonemelevel representation and achieves a lower error rate in a zero-resource phoneme recognition task than previous state-of-the-art selfsupervised representation learning algorithms.

This paper is about improving joint and actuator velocity estimates on a human-sized hydraulic humanoid robot by adding a network of inexpensive microelectromechanical systems (MEMS) gyroscopes. Due to the lack of joint velocity sensors on the majority of humanoid robots, the joint velocity estimates often become a limiting factor on the controller performance. The distributed gyroscopes serve as indirect sensors of the joint velocities that can be estimated by a Kalman filter. Using this framework, we achieve higher velocity gains at the center of mass level on an Atlas hydraulic humanoid robot, which translates into better control performance.

To enable robust dynamic walking on the Atlas robot, we extend our previous work by adding a receding-horizon component. The new controller consists of three hierarchies: a center of mass (CoM) trajectory planner that follows a sequence of desired foot steps, a receding-horizon controller that optimizes the next foot placement to minimize future CoM tracking errors, and an inverse dynamics based full body controller that generates instantaneous joint commands to track these motions while obeying physical constraints. An approximate value function is generated by the CoM planner, and is used to guide the foot placement and inverse dynamics optimizations. The proposed controller is implemented and tested on the Atlas robot. It is capable of walking with strong external perturbations such as recovering from large pushes and traversing unstructured terrain.

We propose a framework to use full-body dynamics for humanoid state estimation. The main idea is to decouple the full body state vector into several independent state vectors. Some decoupled state vectors can be estimated very efficiently with a steady state Kalman Filter. In a steady state Kalman Filter, state covariance is computed only once during initialization. Furthermore, decoupling speeds up numerical linearization of the dynamic model. We demonstrate that these state estimators are capable of handling walking on flat ground and on rough terrain.

We propose a framework for using full-body dynamics for humanoid state estimation. It is formulated as an optimization problem and solved with Quadratic Programming (QP). This formulation provides two main advantages over a nonlinear Kalman filter for dynamic state estimation. QP does not require the dynamic system to be written in the state space form, and it handles equality and inequality constraints naturally. The QP state estimator considers modeling error as part of the optimization vector and includes it in the cost function. The proposed QP state estimator is tested on a Boston Dynamics Atlas humanoid robot.

The ParkourBot climbs in a planar reduced-gravity vertical chute by leaping back and forth between the chute's two parallel walls. The ParkourBot is comprised of a body with two springy legs and its controls consist of leg angles at touchdown and the energy stored in them. During flight, the robot stores elastic potential energy in its springy legs and then converts this potential energy in to kinetic energy at touchdown, when it “kicks off” a wall. This paper describes the ParkourBot's mechanical design, modeling, and open-loop climbing experiments. The mechanical design makes use of the BowLeg, previously used for hopping on a flat ground. We introduce two models of the BowLeg ParkourBot: one is based on a nonzero stance duration using the spring-loaded inverted pendulum model, and the other is a simplified model (the simplest parkour model, or SPM) obtained as the leg stiffness approaches infinity and the stance time approaches zero. The SPM approximation provides the advantage of closed-form calculations. Finally, predictions of the models are validated by experiments in open-loop climbing in a reduced-gravity planar environment provided by an air table.

The ParkourBot is an efficient and dynamic climbing robot. The robot comprises two springy legs connected to a body. Leg angle and spring tension are independently controlled. The robot climbs between two parallel walls by leaping from one wall to the other. During flight, the robot stores elastic energy in its springy legs and automatically releases the energy to "kick off" the wall during touch down. This paper elaborates on the mechanical design of the ParkourBot. We use a simple SLIP model to simulate the ParkourBot motion and stability. Finally, we detail experimental results, from open-loop climbing motions to closed-loop stabilization of climbing height in a planar, reduced gravity environment.

This video shows the investigation of a novel minimalistic, dynamic climbing robot which can climb up tubes of different shapes using a simple dc motor. The motor moves an eccentric mass in a constant velocity. The location of the eccentric mass relative to the contact point determines the stability and the direction of the climbing motion. We present the analysis of this mechanism, simulation and experimental results.