Capturing photo realistic appearance and geometry of scenes is a fundamental problem in computer vision and graphics with a set of mature tools and solutions for content creation [12], [67], large scale scene mapping [5], augmented reality and cinematography [6], [80], [97]. Enthusiast level 3D photogrammetry, especially for small or tabletop scenes, has been supercharged by more capable smartphone cameras and new toolboxes like RealityCapture and NeRF-Studio. A subset of these solutions are geared towards view synthesis where the focus is on photo-realistic view interpolation rather than recovery of accurate scene geometry. These solutions take the “shape-radiance ambiguity”[58] into stride by decoupling the scene transmissivity (related to geometry) from the scene appearance prediction. But without diverse training views, several neural scene representations (e.g. [39], [73], [76]) are prone to poor shape reconstructions while estimating accurate appearance.

This study explores the utility of various internet data sources to select among a set of template robot behaviors to perform skills. Learning contact-rich skills involving tool use from internet data sources has typically been challenging due to the lack of physical information such as contact existence, location, areas, and force in this data. Prior works have generally used internet data and foundation models trained on this data to generate low-level robot behavior. We hypothesize that these data and models may be better suited to selecting among a set of basic robot behaviors to perform these contact-rich skills. We explore three methods of template selection: querying large language models, comparing video of robot execution to retrieved human video using features from a pretrained video encoder common in prior work, and performing the same comparison using features from an optic flow encoder trained on internet data. Our results show that LLMs are surprisingly capable template selectors despite their lack of visual information, optical flow encoding significantly outperforms video encoders trained with an order of magnitude more data, and important synergies exist between various forms of internet data for template selection. By exploiting these synergies, we create a template selector using multiple forms of internet data that achieves a 79% success rate on a set of 16 different cooking skills involving tool-use.

Learning to manipulate dynamic and deformable objects from a single demonstration video holds great promise in terms of scalability. Previous approaches have predominantly focused on either replaying object relationships or actor trajectories. The former often struggles to generalize across diverse tasks, while the latter suffers from data inefficiency. Moreover, both methodologies encounter challenges in capturing invisible physical attributes, such as forces. In this paper, we propose to interpret video demonstrations through a series of Parameterized Symbolic Abstraction Graphs (PSAGs), where nodes represent objects and edges denote relationships between objects. We further ground geometric constraints through simulation to estimate non-geometric, visually imperceptible attributes. The augmented PSAGs are then applied in real robot experiments. Our approach has been validated across a range of tasks, such as Cutting Avocado, Cutting Vegetable, Pouring Liquid, Rolling Dough, and Slicing Pizza. We demonstrate successful generalization to novel objects with distinct visual and physical properties. For visualizations of the learned policies please check: https://www.jianrenw.com/PSAG/

Dynamic in-hand manipulation remains challenging for soft robotic systems, which have demonstrated advantages in safe, compliant interactions but struggle with highspeed dynamic tasks. In this work, we present SWIFT, a system for learning dynamic tasks using a soft and compliant robotic hand. Unlike previous works that rely on simulation, quasistatic actions, and precise object models, SWIFT learns to spin a pen through trial and error using only real-world data and without requiring explicit knowledge of the pen's physical attributes. With self-labeled trials sampled from the real world, SWIFT discovers the set of pen grasping and spinning primitive parameters that enables a soft hand to spin a pen reliably. After 130 sampled actions per object, SWIFT achieves 10/10 success rate across three pens with different weights and weight distributions, demonstrating generalizability and robustness to changes in object properties. The results highlight the potential for soft robotic end-effectors to perform dynamic tasks. We also demonstrate generalization to different shapes and weights, such as a brush and a screwdriver, with 10/10 and 5/10 success rates, respectively. Videos, data, and code are available at https://soft-spin.github.io.

Controlling illumination can generate high quality information about object surface normals and depth discontinuities at a low computational cost. In this work we demonstrate a robot workspace-scaled controlled illumination approach that generates high quality information for table top scale objects for robotic manipulation. With our low angle of incidence directional illumination approach, we can precisely capture surface normals and depth discontinuities of monochromatic Lambertian objects. We show that this approach to shape estimation is 1) valuable for general purpose grasping with a single point vacuum gripper, 2) can measure the deformation of known objects, and 3) can estimate pose of known objects and track unknown objects in the robot’s workspace.

Tasks involving locally unstable or discontinuous dynamics (such as bifurcations and collisions) remain challenging in robotics, because small variations in the environment can have a significant impact on task outcomes. For such tasks, learning a robust deterministic policy is difficult. We focus on structuring exploration with multiple stochastic policies based on a mixture of experts (MoE) policy representation that can be efficiently adapted. The MoE policy is composed of stochastic sub-policies that allow exploration of multiple distinct regions of the action space (or strategies) and a high-level selection policy to guide exploration towards the most promising regions. We develop a robot system to evaluate our approach in a real-world physical problem solving domain. After training the MoE policy in simulation, online learning in the real world demonstrates efficient adaptation within just a few dozen attempts, with a minimal sim2real gap. Our results confirm that representing multiple strategies promotes efficient adaptation in new environments and strategies learned under different dynamics can still provide useful information about where to look for good strategies.

Tasks where the set of possible actions depend discontinuously on the state pose a significant challenge for current reinforcement learning algorithms. For example, a locked door must be first unlocked, and then the handle turned before the door can be opened. The sequential nature of these tasks makes obtaining final rewards difficult, and transferring information between task variants using continuous learned values such as weights rather than discrete symbols can be inefficient. Our key insight is that agents that act and think symbolically are often more effective in dealing with these tasks. We propose a memory-based learning approach that leverages the symbolic nature of constraints and temporal ordering of actions in these tasks to quickly acquire and transfer high-level information. We evaluate the performance of memory-based learning on both real and simulated tasks with approximately discontinuous constraints between states and actions, and show our method learns to solve these tasks an order of magnitude faster than both model-based and model-free deep reinforcement learning methods.

In this work, we investigate a form of dynamic contact-rich locomotion in which a robot pushes off from obstacles in order to move through its environment. We present a reflex-based approach that switches between optimized hand-crafted reflex controllers and produces smooth and predictable motions. In contrast to previous work, our approach does not rely on periodic movements, complex models of robot and contact dynamics, or extensive hand tuning. We demonstrate the effectiveness of our approach and evaluate its performance compared to a standard model-free RL algorithm. We identify continuous clusters of similar behaviours, which allows us to successfully transfer different push-off motions directly from simulation to a physical robot without further retraining.

To Perform reliably and consistently over sustained periods of time, large-scale automation critically relies on computer simulation. Simulation allows us and supervisory AI to effectively design, validate, and continuously improve complex processes, and helps practitioners to gain insight into the operation and justify future investments. While numerous successful applications of simulation in industry exist, such as circuit simulation, finite element methods, and computeraided design (CAD), state-of-the-art simulators fall short of accurately modeling physical phenomena, such as friction, impact, and deformation.

Learning controllers for bipedal robots is a challenging problem, often requiring expert knowledge and extensive tuning of parameters that vary in different situations. Recently, deep reinforcement learning has shown promise at automatically learning controllers for complex systems in simulation. This has been followed by a push towards learning controllers that can be transferred between simulation and hardware, primarily with the use of domain randomization. However, domain randomization can make the problem of finding stable controllers even more challenging, especially for under actuated bipedal robots. In this work, we explore whether policies learned in simulation can be transferred to hardware with the use of high-fidelity simulators and structured controllers. We learn a neural network policy which is a part of a more structured controller. While the neural network is learned in simulation, the rest of the controller stays fixed, and can be tuned by the expert as needed. We show that using this approach can greatly speed up the rate of learning in simulation, as well as enable transfer of policies between simulation and hardware. We present our results on an ATRIAS robot and explore the effect of action spaces and cost functions on the rate of transfer between simulation and hardware. Our results show that structured policies can indeed be learned in simulation and implemented on hardware successfully. This has several advantages, as the structure preserves the intuitive nature of the policy, and the neural network improves the performance of the hand-designed policy. In this way, we propose a way of using neural networks to improve expert designed controllers, while maintaining ease of understanding.

Learning for control can acquire controllers for novel robotic tasks, paving the path for autonomous agents. Such controllers can be expert-designed policies, which typically require tuning of parameters for each task scenario. In this context, Bayesian optimization (BO) has emerged as a promising approach for automatically tuning controllers. However, sample-efficiency can still be an issue for high-dimensional policies on hardware. Here, we develop an approach that utilizes simulation to learn structured feature transforms that map the original parameter space into a domain-informed space. During BO, similarity between controllers is now calculated in this transformed space. Experiments on the ATRIAS robot hardware and simulation show that our approach succeeds at sample-efficiently learning controllers for multiple robots. Another question arises: What if the simulation significantly differs from hardware? To answer this, we create increasingly approximate simulators and study the effect of increasing simulation-hardware mismatch on the performance of Bayesian optimization. We also compare our approach to other approaches from literature, and find it to be more reliable, especially in cases of high mismatch. Our experiments show that our approach succeeds across different controller types, bipedal robot models and simulator fidelity levels, making it applicable to a wide range of bipedal locomotion problems.

Robotics controllers often consist of expert-designed heuristics, which can be hard to tune in higher dimensions. Simulation can aid in optimizing these controllers if parameters learned in simulation transfer to hardware. Unfortunately, this is often not the case in legged locomotion, necessitating learning directly on hardware. This motivates using data-efficient learning techniques like Bayesian Optimization (BO) to minimize collecting expensive data samples. BO is a black-box data-efficient optimization scheme, though its performance typically degrades in higher dimensions. We aim to overcome this problem by incorporating domain knowledge, with a focus on bipedal locomotion. In our previous work, we proposed a feature transformation that projected a 16-dimensional locomotion controller to a 1-dimensional space using knowledge of human walking. When optimizing a human-inspired neuromuscular controller in simulation, this feature transformation enhanced sample efficiency of BO over traditional BO with a Squared Exponential kernel. In this paper, we present a generalized feature transform applicable to non-humanoid robot morphologies and evaluate it on the ATRIAS bipedal robot, in both simulation and hardware. We present three different walking controllers and two are evaluated on the real robot. Our results show that this feature transform captures important aspects of walking and accelerates learning on hardware and simulation, as compared to traditional BO.

This paper is about improving joint and actuator velocity estimates on a human-sized hydraulic humanoid robot by adding a network of inexpensive microelectromechanical systems (MEMS) gyroscopes. Due to the lack of joint velocity sensors on the majority of humanoid robots, the joint velocity estimates often become a limiting factor on the controller performance. The distributed gyroscopes serve as indirect sensors of the joint velocities that can be estimated by a Kalman filter. Using this framework, we achieve higher velocity gains at the center of mass level on an Atlas hydraulic humanoid robot, which translates into better control performance.

We explore a model-based approach to reinforcement learning where partially or totally unknown dynamics are learned and explicit planning is performed. We learn dynamics with neural networks, and plan behaviors with differential dynamic programming (DDP). In order to handle complicated dynamics, such as manipulating liquids (pouring), we consider temporally decomposed dynamics. We start from our recent work [1] where we used locally weighted regression (LWR) to model dynamics. The major contribution of this paper is making use of deep learning in the form of neural networks with stochastic DDP, and showing the advantages of neural networks over LWR. For this purpose, we extend neural networks for: (1) modeling prediction error and output noise, (2) computing an output probability distribution for a given input distribution, and (3) computing gradients of output expectation with respect to an input. Since neural networks have nonlinear activation functions, these extensions were not easy. We provide an analytic solution for these extensions using some simplifying assumptions. We verified this method in pouring simulation experiments. The learning performance with neural networks was better than that of LWR. The amount of spilled materials was reduced. We also present early results of robot experiments using a PR2. Accompanying video: https://youtu.be/aM3hE1J5W98.

To enable robust dynamic walking on the Atlas robot, we extend our previous work by adding a receding-horizon component. The new controller consists of three hierarchies: a center of mass (CoM) trajectory planner that follows a sequence of desired foot steps, a receding-horizon controller that optimizes the next foot placement to minimize future CoM tracking errors, and an inverse dynamics based full body controller that generates instantaneous joint commands to track these motions while obeying physical constraints. An approximate value function is generated by the CoM planner, and is used to guide the foot placement and inverse dynamics optimizations. The proposed controller is implemented and tested on the Atlas robot. It is capable of walking with strong external perturbations such as recovering from large pushes and traversing unstructured terrain.

We introduce CHAMP, an algorithm for online Bayesian changepoint detection in settings where it is difficult or undesirable to integrate over the parameters of candidate models. CHAMP is used in combination with several articulation models to detect changes in articulated motion of objects in the world, allowing a robot to infer physically-grounded task information. We focus on three settings where a changepoint model is appropriate: objects with intrinsic articulation relationships that can change over time, object-object contact that results in quasi-static articulated motion, and assembly tasks where each step changes articulation relationships. We experimentally demonstrate that this system can be used to infer various types of information from demonstration data including causal manipulation models, human-robot grasp correspondences, and skill verification tests.

We present an optimization method to solve coupled redundant inverse kinematics problems and generate trajectories for humanoid robot full-body manipulation. The basic idea of our algorithm is to divide a manipulation task into a series of key postures, generate multiple diverse initial guesses for each key posture, and use optimization to find inverse kinematics solutions based on these initial guesses. We then find an optimal series of key postures and form a continuous trajectory. Our approach is implemented in a Gazebo simulation using the Atlas humanoid robot from Boston Dynamics.

In this paper, we propose a novel walking method for torque controlled robots. The method is able to produce a wide range of speeds without requiring off-line optimizations and re-tuning of parameters. We use a quadratic whole-body optimization method running online which generates joint torques, given desired Cartesian accelerations of center of mass and feet. Using a dynamics model of the robot inside this optimizer, we ensure both compliance and tracking, required for fast locomotion. We have designed a foot-step planner that uses a linear inverted pendulum as simplified robot internal model. This planner is formulated as a quadratic convex problem which optimizes future steps of the robot. Fast libraries help us performing these calculations online. With very few parameters to tune and no perception, our method shows notable robustness against strong external pushes, relatively large terrain variations, internal noises, model errors and also delayed communication.

We propose a framework to use full-body dynamics for humanoid state estimation. The main idea is to decouple the full body state vector into several independent state vectors. Some decoupled state vectors can be estimated very efficiently with a steady state Kalman Filter. In a steady state Kalman Filter, state covariance is computed only once during initialization. Furthermore, decoupling speeds up numerical linearization of the dynamic model. We demonstrate that these state estimators are capable of handling walking on flat ground and on rough terrain.

We propose a framework for using full-body dynamics for humanoid state estimation. It is formulated as an optimization problem and solved with Quadratic Programming (QP). This formulation provides two main advantages over a nonlinear Kalman filter for dynamic state estimation. QP does not require the dynamic system to be written in the state space form, and it handles equality and inequality constraints naturally. The QP state estimator considers modeling error as part of the optimization vector and includes it in the cost function. The proposed QP state estimator is tested on a Boston Dynamics Atlas humanoid robot.

Step planning is becoming an increasingly important research topic for humanoid robots. Most cost functions for step planning in the literature are designed based on terrain information. The energy cost to perform each step action is usually ignored. In walking, energy consumption depends on gait features such as step length and width. In this paper, we use three simple and intuitive energy cost functions for different step lengths, widths, and the turning angle. These functions are inspired by literature on human walking energy analysis, and the function parameters are tuned to match computed costs for optimal humanoid walking motions obtained by simulation. The energy cost and the terrain cost are combined to obtain an optimal step planning sequence using A* search.

This paper compares two approaches to designing Kalman Filters for walking systems. The first design uses Linear Inverted Pendulum Model (LIPM) dynamics, and the other design uses a more complete Planar dynamics. The filter based on the simpler LIPM design is more robust to modeling error. The more complex design estimates center of mass height and joint velocities, and tracks horizontal center of mass translation more accurately. We also investigate different ways of handling contact states and using force sensing in state estimation. In the LIPM filter, force sensing is used to determine contact states and tune filter parameters. In the Planar filter, force sensing is used to select the proper measurement equation.

We present a novel approach to legged locomotion over rough terrain that is thoroughly rooted in optimization. This approach relies on a hierarchy of fast, anytime algorithms to plan a set of footholds, along with the dynamic body motions required to execute them. Components within the planning framework coordinate to exchange plans, cost-to-go estimates, and “certificates” that ensure the output of an abstract high-level planner can be realized by deeper layers of the hierarchy. The burden of careful engineering of cost functions to achieve desired performance is substantially mitigated by a simple inverse optimal control technique. Robustness is achieved by real-time re-planning of the full trajectory, augmented by reflexes and feedback control. We demonstrate the successful application of our approach in guiding the LittleDog quadruped robot over a variety of rough terrains.

This paper presents a model-based method, called Dynamic Balance Force Control (DBFC), for determining full body joint torques based on desired COM motion and contact forces for compliant humanoid robots. The center of mass (COM) dynamics are affected directly through contact force control to achieve stable balance. This idea is used to formulate DBFC considering the full rigid-body dynamics of the robot to produce desired contact forces. To achieve generic force control tasks, a virtual model controller, DBFC-VMC, is presented. Results presented from experiments on a force-controlled humanoid robot and simulation demonstrate the general purpose use of this control.

This paper develops full-state parametric controllers for standing balance of humanoid robots in response to impulsive and constant pushes. We also explore a hypothesis that postural feedback gains in standing balance should change with perturbation size. From an engineering point of view this is known as gain scheduling. We use an optimization approach to see if feedback gains should scale with the perturbation for a simulated robot. We simulate models in the sagittal and lateral plane and in 3-dimensions, use a horizontal push of a given size, direction and location as a perturbation, and optimize parametric controllers for different push sizes, directions and locations. During a simulated perturbation experiment, the appropriate controller is continuously selected based on the current push. For an impulse, the simulated robot recovers back to the initial state; for a constant push, the robot moves to an equilibrium position which leans into the push and has zero joint torques. We show the performance of optimized parametric controllers in response to different external pushes.

This paper presents a standing balance controller that explicitly handles pushes. We employ a library of optimal trajectories and the neighboring optimal control method to generate local approximations to the optimal control. We take advantage of a parametric nonlinear optimization method, SNOPT, to generate initial trajectories and then use Differential Dynamic Programming (DDP) to further refine them and get their neighboring optimal control. A library generation method is proposed, which keeps the trajectory library to a reasonable size. We compare the proposed controller with an optimal controller and an LQR based gain scheduling controller using the same optimization criterion. Simulation results demonstrate the performance of the proposed method.

The use of robots in assistive roles will be an increasingly significant application for robotics. Assistive robots need to physically interact with humans in a safe manner. We propose the use of inflatable robot links as structural members instead of traditional rigid links. We believe such links would allow the development of inherently safe robots. For these robots to be useful in tasks such as assisting humans, it is essential that we be able to control contact forces with these robots. In this paper, we propose a model for force control with a single inflatable link, investigate the dynamics of the model, and present experimental results.

In this video presentation we introduce a 50 degrees of freedom humanoid robot, CB -ComputationalBrain[1]. CB is a humanoid robot created for exploring the underlying processing of the human brain while dealing with the real world. We place our investigations within real world contexts, as humans do. In so doing, we focus on utilising a system that is closer to humans - in sensing, kinematics configuration and performance. We present a full-body compliance controller that was developed for the motion control of our humanoid robot [2]. Our initial experimentation on our system includes: 1) full-body compliant control - physical interactions/balancing/motion control; 2) the integrated visual ocular-motor responses; 3) perception and control - reaching, foveation, and active object recognition; 4) our studies of Central Pattern Generator for walking.

We propose using the kernel dimension reduction (KDR) to extract a low-dimensional feature space for humanoid locomotion tasks. Although humanoids have many degrees of freedom, task relevant feature spaces can be much smaller than the number of dimension of the original state space. We consider an application of the proposed approach to improve the locomotive performance of humanoid robots using an extracted low-dimensional state space. To improve the locomotive performance, we use a reinforcement learning (RL) framework. While RL is a useful non-linear optimizer, it is usually difficult to apply RL to real robotic systems - due to the large number of iterations required to acquire suitable policies. In this study, we use the extracted low-dimensional feature space for RL so that the learning system can improve task performance quickly. The kernel dimension reduction method allows us to extract the feature space even if the task relevant mapping is non-linear. This is an essential property to improve humanoid locomotive performance since stepping or walking dynamics involves highly nonlinear dynamics. We show that we can improve stepping and walking policies by using a RL method on an extracted feature space by using KDR.

We combine three threads of research on approximate dynamic programming: sparse random sampling of states, value function and policy approximation using local models, and using local trajectory optimizers to globally optimize a policy and associated value function. Our focus is on finding steady-state policies for deterministic time-invariant discrete time control problems with continuous states and actions often found in robotics. In this paper, we describe our approach and provide initial results on several simulated robotics problems.

We propose to improve the locomotive performance of humanoid robots by using approximated biped stepping and walking dynamics with reinforcement learning (RL). Although RL is a useful non-linear optimizer, it is usually difficult to apply RL to real robotic systems - due to the large number of iterations required to acquire suitable policies. In this study, we first approximated the dynamics by using data from a real robot, and then applied the estimated dynamics in RL in order to improve stepping and walking policies. Gaussian processes were used to approximate the dynamics. By using Gaussian processes, we could estimate a probability distribution of a target function with a given covariance function. Thus, RL can take the uncertainty of the approximated dynamics into account throughout the learning process. We show that we can improve stepping and walking policies by using a RL method with the approximated models both in simulated and real environments. Experimental validation on a real humanoid robot of the proposed

Libraries of trajectories are a promising way of creating policies for difficult problems. However, often it is not desirable or even possible to create a new library for every task. We present a method for transferring libraries across tasks, which allows us to build libraries by learning from demonstration on one task and apply them to similar tasks. Representing the libraries in a feature-based space is key to supporting transfer. We also search through the library to ensure a complete path to the goal is possible. Results are shown for the Little Dog task. Little Dog is a quadruped robot that has to walk across rough terrain at reasonably fast speeds.

We combine two threads of research on approximate dynamic programming: random sampling of states and using local trajectory optimizers to globally optimize a policy and associated value function. This combination allows us to replace a dense multidimensional grid with a much sparser adaptive sampling of states. Our focus is on finding steady state policies for the deterministic time invariant discrete time control problems with continuous states and actions often found in robotics. In this paper we show that we can now solve problems we couldn't solve previously with regular grid-based approaches.

We show that a humanoid robot can step and walk using simple sinusoidal desired joint trajectories with their phase adjusted by a coupled oscillator model. We use the center of pressure location and velocity to detect the phase of the lateral robot dynamics. This phase information is used to modulate the desired joint trajectories. We applied the proposed control approach to our newly developed human sized humanoid robot and a small size humanoid robot developed by Sony, enabling them to generate successful stepping and walking patterns

We present a control approach that uses a library of trajectories to establish a global control law or policy. This is an alternative to methods for finding global policies based on value functions using dynamic programming and also to using plans based on a single desired trajectory. Our method has the advantage of providing reasonable policies much faster than dynamic programming can provide an initial policy. It also has the advantage of providing more robust and global policies than following a single desired trajectory. Trajectory libraries can be created for robots with many more degrees of freedom than what dynamic programming can be applied to as well as for robots with dynamic model discontinuities. Results are shown for the "Labyrinth" marble maze, both in simulation as well as a real world version. The marble maze is a difficult task which requires both fast control as well as planning ahead

This paper presents a case study of learning to select behavioral primitives and generate subgoals from observation and practice. Our approach uses local features to generalize across tasks and global features to learn from practice. We demonstrate this approach applied to the marble maze task. Our robot uses local features to initially learn primitive selection and subgoal generation policies from observing a teacher maneuver a marble through a maze. The robot then uses this information as it tries to traverse another maze, and refines the information during learning from practice

We propose a model-based reinforcement learning algorithm for biped walking in which the robot learns to appropriately modulate an observed walking pattern. Via-points are detected from the observed walking trajectories using the minimum jerk criterion. The learning algorithm modulates the via-points as control actions to improve walking trajectories. This decision is based on a learned model of the Poincaré map of the periodic walking pattern. The model maps from a state in the single support phase and the control actions to a state in the next single support phase. We applied this approach to both a simulated robot model and an actual biped robot. We show that successful walking policies are acquired.

We explore the use of computational optimal control techniques for automated construction of policies in complex dynamic environments. Our implementation of dynamic programming is performed in a reduced dimensional subspace of a simulated four-DOF biped robot with point feet. We show that a computed solution to this problem can be generated and yield empirically stable walking that can handle various types of disturbances.

A person seeking another person's attention is normally able to quickly assess how interruptible the other person currently is. Such assessments allow behavior that we consider natural, socially appropriate, or simply polite. This is in sharp contrast to current computer and communication systems, which are largely unaware of the social situations surrounding their usage and the impact that their actions have on these situations. If systems could model human interruptibility, they could use this information to negotiate interruptions at appropriate times, thus improving human computer interaction.This article presents a series of studies that quantitatively demonstrate that simple sensors can support the construction of models that estimate human interruptibility as well as people do. These models can be constructed without using complex sensors, such as vision-based techniques, and therefore their use in everyday office environments is both practical and affordable. Although currently based on a demographically limited sample, our results indicate a substantial opportunity for future research to validate these results over larger groups of office workers. Our results also motivate the development of systems that use these models to negotiate interruptions at socially appropriate times.

We propose a model-based reinforcement learning algorithm for biped walking in which the robot learns to appropriately place the swing leg. This decision is based on a learned model of the Poincare map of the periodic walking pattern. The model maps from a state at the middle of a step and foot placement to a state at next middle of a step. We also modify the desired walking cycle frequency based on online measurements. We present simulation results, and are currently implementing this approach on an actual biped robot.

A person seeking someone else's attention is normally able to quickly assess how interruptible they are. This assessment allows for behavior we perceive as natural, socially appropriate, or simply polite. On the other hand, today's computer systems are almost entirely oblivious to the human world they operate in, and typically have no way to take into account the interruptibility of the user. This paper presents a Wizard of Oz study exploring whether, and how, robust sensor-based predictions of interruptibility might be constructed, which sensors might be most useful to such predictions, and how simple such sensors might be.The study simulates a range of possible sensors through human coding of audio and video recordings. Experience sampling is used to simultaneously collect randomly distributed self-reports of interruptibility. Based on these simulated sensors, we construct statistical models predicting human interruptibility and compare their predictions with the collected self-report data. The results of these models, although covering a demographically limited sample, are very promising, with the overall accuracy of several models reaching about 78%. Additionally, a model tuned to avoiding unwanted interruptions does so for 90% of its predictions, while retaining 75% overall accuracy.

We seek intuitive, efficient ways to create and direct human-like behaviors for humanoid robots. Here we present a method to enable humanoid robots to acquire movements by imitation. The robot uses 3D vision to perceive the movements of a human teacher, and then estimates the teacher's body postures using a fast full-body inverse kinematics method that incorporates a kinematic model of the teacher. This solution is then mapped to the robot and reproduced in real-time. The robustness of the method is tested on a 30-degree-of-freedom Sarcos humanoid robot located at ATR using 3D vision data from external cameras and from head-mounted cameras.

This paper focuses on learning to select behavioral primitives and generate sub-goals from practicing a task. We present a novel algorithm that combines Q-learning and a locally weighted learning method to improve primitive selection and sub-goal generation. We demonstrate this approach applied to the tilt maze task. Our robot initially learns to perform this task using learning from observation, and then learns from practice.

We developed a robust control policy design method in high-dimensional state space by using differential dynamic programming with a minimax criterion. As an example, we applied our method to a simulated five link biped robot. The results show lower joint torques from the optimal control policy compared to a hand-tuned PD servo controller. Results also show that the simulated biped robot can successfully walk with unknown disturbances that cause controllers generated by standard differential dynamic programming and the hand-tuned PD servo to fail. Learning to compensate for modeling error and previously unknown disturbances in conjunction with robust control design is also demonstrated. We also applied proposed method to a real biped robot for optimizing swing leg trajectories.

In this paper we present a humanoid system that can integrate information provided by its foveal and peripheral cameras. We use peripheral vision to detect and pursue objects of interest based on simple shape and color models. A detection event triggers the robot to direct its eyes towards the object, thus making a more detailed analysis of the observed objects in higher resolution foveal images feasible. The recognition is based on principal component analysis and is performed while the robot actively pursues the detected object. The classification results are inferred using information from a video stream rather than just a single image. Once the desired object is recognized, the robot reaches for it while ignoring other objects.

Using the pre-recorded human motion and trajectory tracking, we can control the motion of a humanoid robot for free-space, upper body gestures. However, the number of degrees of freedom, range of joint motion, and achievable joint velocities of today's humanoid robots are far more limited than those of the average human subject. In this paper, we explore a set of techniques for limiting human motion of upper body gestures to that achievable by a Sarcos humanoid robot located at ATR. We assess the quality of the results by comparing the motion of the human actor to that of the robot, both visually and quantitatively.

This paper describes humanoid robot learning from observation and game playing using information provided by a real-time PC-based vision system. To cope with extremely fast motions that arise in the environment, a visual system capable of perceiving the motion of several objects at 60 fields per second was developed. We have designed a suitable error recovery scheme for our vision system to ensure successful game playing over longer periods of time. To increase the learning rate of the robot it is given domain knowledge in the form of primitives. The robot learns how to perform primitives from data collected while observing a human. The robot control system and primitive use strategy are also explained.

A longstanding goal of reinforcement learning is to develop non- parametric representations of policies and value functions that support rapid learning without suffering from interference or the curse of di- mensionality. We have developed a trajectory-based approach, in which policies and value functions are represented nonparametrically along tra- jectories. These trajectories, policies, and value functions are updated as the value function becomes more accurate or as a model of the task is up- dated. We have applied this approach to periodic tasks such as hopping and walking, which required handling discount factors and discontinu- ities in the task dynamics, and using function approximation to represent value functions at discontinuities. We also describe extensions of the ap- proach to make the policies more robust to modeling error and sensor noise.

We developed a robust control policy design method in high-dimensional state space by using differential dynamic programming with a minimax criterion. As an example, we applied our method to a simulated five link biped robot. The results show lower joint torques from the optimal con- trol policy compared to a hand-tuned PD servo controller. Results also show that the simulated biped robot can successfully walk with unknown disturbances that cause controllers generated by standard differential dy- namic programming and the hand-tuned PD servo to fail. Learning to compensate for modeling error and previously unknown disturbances in conjunction with robust control design is also demonstrated.

This paper describes the rise of task primitives in robot learning from observation. A framework is developed that uses observed data to initially learn a task and the agent then goes on to increase its performance through repeated task performance (learning from practice). Data that is collected while the human performs a task is parsed into small parts of the task called primitives. Modules are created for each primitive that encode the movements required during the performance of the primitive, and when and where the primitives are performed. The feasibility of this method is currently being tested with agents that learn to play a virtual and an actual air hockey game.

We describe a real-time visual system that enables a humanoid robot to learn from and interact with humans. The core of the visual system is a probabilistic tracker that uses shape and color information to find relevant objects in the scene. Multiscale representations, windowing and masking are employed to accelerate the data processing. The perception system is directly coupled with the motor control system of our humanoid robot DB. We present an example of on-line interaction with a humanoid robot: mimicking of human hand motion. The generation of humanoid robot motion based on the human motion is accomplished in real-time. The study is supported by experimental results on DB.

Locally weighted learning (LWL) is a class of statistical learning techniques that provides useful representations and training algorithms for learning about complex phenomena during autonomous adaptive control of robotic systems. This paper introduces several LWL algorithms that have been tested successfully in real-time learning of complex robot tasks. We discuss two major classes of LWL, memory-based LWL and purely incremental LWL that does not need to remember any data explicitly. In contrast to the traditional beliefs that LWL methods cannot work well in high-dimensional spaces, we provide new algorithms that have been tested in up to 50 dimensional learning problems. The applicability of our LWL algorithms is demonstrated in various robot learning examples, including the learning of devil-sticking, pole-balancing of a humanoid robot arm, and inverse-dynamics learning for a seven degree of-freedom robot.

The formulation and optimization of joint trajectories for humanoid robots is quite different from this same task for standard robots because of the complexity of the humanoid robots' kinematics. We exploit the similarity between the movements of a humanoid robot and human movements to generate joint trajectories for such robots. In particular we show how to transform human motion information captured by an optical tracking device into a high dimensional trajectory of a humanoid robot. We utilize B-spline wavelets to efficiently represent the joint trajectories and to automatically select the density of the basis functions on the time axis. We applied our method to the task of teaching a humanoid robot how to make a dance movement.

Article Investigating the capture, integration and access problem of ubiquitous computing in an educational setting Share on Authors: Gregory D. Abowd GVU Center & College of Computing, Georgia Institute of Technology, Atlanta, GA GVU Center & College of Computing, Georgia Institute of Technology, Atlanta, GAView Profile , Christopher G. Atkeson GVU Center & College of Computing, Georgia Institute of Technology, Atlanta, GA GVU Center & College of Computing, Georgia Institute of Technology, Atlanta, GAView Profile , Jason Brotherton GVU Center & College of Computing, Georgia Institute of Technology, Atlanta, GA GVU Center & College of Computing, Georgia Institute of Technology, Atlanta, GAView Profile , Tommy Enqvist GVU Center & College of Computing, Georgia Institute of Technology, Atlanta, GA GVU Center & College of Computing, Georgia Institute of Technology, Atlanta, GAView Profile , Paul Gulley GVU Center & College of Computing, Georgia Institute of Technology, Atlanta, GA GVU Center & College of Computing, Georgia Institute of Technology, Atlanta, GAView Profile , Johan LeMon GVU Center & College of Computing, Georgia Institute of Technology, Atlanta, GA GVU Center & College of Computing, Georgia Institute of Technology, Atlanta, GAView Profile Authors Info & Claims CHI '98: Proceedings of the SIGCHI Conference on Human Factors in Computing SystemsJanuary 1998 Pages 440–447https://doi.org/10.1145/274644.274704Online:01 January 1998Publication History 73citation1,473DownloadsMetricsTotal Citations73Total Downloads1,473Last 12 Months12Last 6 weeks2 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteGet Access

We introduce a constructive, incremental learning system for regression problems that models data by means of spatially localized linear models. In contrast to other approaches, the size and shape of the receptive field of each locally linear model, as well as the parameters of the locally linear model itself, are learned independently, that is, without the need for competition or any other kind of communication. Independent learning is accomplished by incrementally minimizing a weighted local cross-validation error. As a result, we obtain a learning system that can allocate resources as needed while dealing with the bias-variance dilemma in a principled way. The spatial localization of the linear models increases robustness toward negative interference. Our learning system can be interpreted as a nonparametric adaptive bandwidth smoother, as a mixture of experts where the experts are trained in isolation, and as a learning system that profits from combining independent expert knowledge on the same problem. This article illustrates the potential learning capabilities of purely local learning and offers an interesting and powerful approach to learning with receptive fields.

Learning a complex dynamic robot manoeuvre from a single human demonstration is difficult. This paper explores an approach to learning from demonstration based on learning an optimization criterion from the demonstration and a task model from repeated attempts to perform the task, and using the learned criterion and model to compute an appropriate robot movement. A preliminary version of the approach has been implemented on an anthropomorphic robot arm using a pendulum swing up task as an example.

This paper compares direct reinforcement learning (no explicit model) and model-based reinforcement learning on a simple task: pendulum swing up. We find that in this task model-based approaches support reinforcement learning from smaller amounts of training data and efficient handling of changing goals.

We view college classroom teaching and learning as a multimedia authoring activity. The classroom provides a rich setting in which a number of different forms of communication co-exist, such as speech, writing and projected images. Much of the information in a lecture is poorly recorded or lost currently. Our hypothesis is that tools to aid in the capture and subsequent access of classroom information will enhance both the learning and teaching experience. To test that hypothesis, we initiated the Classroom 2000 project at Georgia Tech. The purpose of the project is to apply ubiquitous computing technology to facilitate automatic capture, integration and access of multimedia information in the educational setting of the university classroom. In this paper, we discuss various prototype tools we have created and used in a variety of courses and provide an initial evaluation of the acceptance and effectiveness of the technology. We also share some lessons learned in applying ubiquitous computing technology in a real setting.

Article Free Access Share on Rapid prototyping of mobile context-aware applications: the Cyberguide case study Authors: Sue Long College of Computing, Georgia Institute of Technology, Atlanta, GA College of Computing, Georgia Institute of Technology, Atlanta, GAView Profile , Rob Kooper College of Computing, Georgia Institute of Technology, Atlanta, GA College of Computing, Georgia Institute of Technology, Atlanta, GAView Profile , Gregory D. Abowd College of Computing, Georgia Institute of Technology, Atlanta, GA College of Computing, Georgia Institute of Technology, Atlanta, GAView Profile , Christopher G. Atkeson College of Computing, Georgia Institute of Technology, Atlanta, GA College of Computing, Georgia Institute of Technology, Atlanta, GAView Profile Authors Info & Claims MobiCom '96: Proceedings of the 2nd annual international conference on Mobile computing and networkingNovember 1996 Pages 97–107https://doi.org/10.1145/236387.236412Published:01 November 1996Publication History 166citation3,011DownloadsMetricsTotal Citations166Total Downloads3,011Last 12 Months139Last 6 weeks16 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteeReaderPDF

This paper examines the implementation of projection pursuit regression (PPR) in the context of machine learning and neural networks. We propose a parametric PPR with direct training which achieves improved training speed and accuracy when compared with nonparametric PPR. Analysis and simulations are done for heuristics to choose good initial projection directions. A comparison of a projection pursuit learning network with a single hidden-layer sigmoidal neural network shows why grouping hidden units in a projection pursuit learning network is useful. Learning robot arm inverse dynamics is used as an example problem.

We present a memory-based local modeling approach to robot learning using a nonparametric regression technique, locally weighted regression. The model of the task to be performed is represented by infinitely many local linear models, the (hyper-) tangent planes at every query point. This is in contrast to other methods using finite set of linear models to accomplish a piecewise linear model. Architectural parameters of our approach, such as distance metrics, are a function of the current query point instead of being global. Statistical tests are presented for when a local model is good enough such that it can be reliably used to build a local controller. These statistical measures also direct the exploration of the robot. We explicitly deal with the case where prediction accuracy requirements exist during exploration: by gradually shifting a center of exploration and controlling the speed of the shift with local prediction accuracy, a goal-directed exploration of state space takes place along the fringes of the current data support until the task goal is achieved. We illustrate this approach by describing how it has been used to enable a robot to learn a juggling task.>

We present an approach to robot learning based on a nonparametric regression technique, locally weighted regression. The model of the task to be performed is represented by infinitely many local linear models, i.e., the (hyper-) tangent planes at every query point. Such a model, however, is only generated when a query performed and is not retained. The architectural parameters of our approach, such as distance metrics, are also a function of the current query point instead of being global. Statistical tests are presented for when a local model is good enough such that it can be reliably used to build a local controller. These statistical measures also direct the exploration of the robot. We explicitly deal with the case where prediction accuracy requirements exist during exploration. By gradually shifting a center of exploration and controlling the speed of the shift with local prediction accuracy, a goal-directed exploration of state space takes place along the fringes of the current data support until the task goal is achieved. We illustrate this approach by describing how it has been used to enable a robot to learn a challenging juggling task.>

The use of locally weighted regression in memory-based robot learning is explored. A local model is formed to answer each query, using a weighted regression in which close points (similar experiences) are weighted more than distant points (less relevant experiences). This approach implements a philosophy of modeling a complex function with many simple local models. The author explains how an appropriate distance metric or measure of similarity can be found, and how the distance metric is used. How irrelevant input variables and terms in the local model are detected is also explained. An example from the control of a robot arm is used to compare this approach with other robot control and learning techniques.>

Results are presented from a preliminary investigation of task-level learning, an approach to learning from practice. The authors programmed a robot to juggle a single ball in three dimensions by batting it upwards with a large paddle. The robot uses a real-time binary vision system to track the ball and measure its performance. Task-level learning consists of building a model of performance errors at the task level during practice, and using that model to refine task-level commands. A polynomial surface was fitted to the errors in the path which the ball took after each hit, and this task model is used to refine how the ball is hit. This application of task-level learning dramatically increased the number of consecutive hits the robot could execute before the ball was hit out of range of the paddle.>

The use of an associative content-addressable memory to model a robot and the world the robot interacts with is discussed. The model can be learned by storing experiences in the memory. To make predictions, the memory is searched for relevant experience. An initial implementation of such a memory-based modeling scheme has been made on a parallel computer, the Connection Machine. The implementation machine was used to model and control a simulated planar two-joint arm and a simulated running machine. The issues and problems that arose in the preliminary work are described. It is found that the use of parallel search in the implementation of an associative content-addressable memory allows quick searching of stored experiences, and reasonable retrieval is obtained using a simple distance metric and a simple generalized scheme. The memory is able to generalize after storing only a small number of relevant experiences. The use of search by parallel processors also avoids many of the problems of previous memory-based or tubular approaches to robot modeling (such as search speed and memory requirements).>

Trajectory tracking errors resulting from the application of various controllers have been experimentally determined on the MIT Serial Link Direct Drive Arm. The controllers range from simple analog PD (proportional-derivative) control applied independently at each joint to feedforward and computed torque methods incorporating full dynamics. It was found that trajectory tracking errors decreased as more dynamic compensation terms were incorporated. There was no significant difference in trajectory tracking performance between the feedforward controller using independent digital servos and the full computed torque controller. Implementing the model-based controller highlights the need for accurate control of joint torque, accurate joint position and velocity sensing, and adequate sampling rates.>

The functionality of robots can be improved by programming them to learn tasks from practice. Task-level learning can compensate for the structural modeling errors of the robot's lower-level control systems and can speed up the learning process by reducing the degrees of freedom of the models to be learned. The authors demonstrate two general learning procedures-fixed-model learning and refined-model learning-on a ball-throwing robot system. Both learning approaches refine the task command based on the performance error of the system, while they ignore the intermediate variables separation the lower-level systems. The authors also provide experimental and theoretical evidence that task-level learning can improve the functionality of robots.>

Work on building robot models to be used in designing model-based controllers is described. Various algorithms are presented estimating the kinematic, link-inertial, and load-inertial parameters. It is shown experimentally that accurate estimates can be obtained automatically using sensor data taken during movement. The algorithms have been implemented on the MIT Serial Link Direct Drive Arm.>

For pt.I see ibid., p.1374-9 (1988). Work on model-based control with the MIT Serial Link Direct Drive Arm is described. It is shown that model-based control leads to performance superior to control not based on carefully constructed models. Trajectory control, trajectory learning, and force control are treated. A new type of trajectory learning is considered in which a robot fine-tunes one particular trajectory through repetition. Various experiments with the direct drive arm are reported to validate the importance of model-based control.>

Trajectory tracking errors resulting from the application of various controllers have been experimentally determined on the MIT Serial Link Direct Drive Arm. The controllers range from simple analog PD control applied independently at each joint to feedforward and computed torque methods incorporating full dynamics. It was found that trajectory tracking errors decreased as more dynamic compensation terms were incorporated. There was no significant difference in trajectory tracking performance between the feedforward controller using independent digital servos and the full computed torque controller.

Trajectory tracking errors resulting from the application of various controllers have been experimentally determined on the MIT Serial Link Direct Drive Arm. The controllers range from simple PD control applied independently at each joint to feedforward control incorporating full dynamics followed by a separate PD control loop. It was found that trajectory tracking errors decreased as more feedforward terms were incorporated.

We present an algorithm that uses trajectory following errors to improve a feedforward command to a robot. This approach to robot learning is based on explicit modeling of the robot; and uses an inverse of the robot model as part of a learning operator which processes the trajectory errors. Results are presented from a successful implementation of this procedure on the MIT Serial Link Direct Drive Arm. The major point of this paper is that more accurate robot models improve trajectory learning performance, and learning algorithms do not reduce the need for good models in robot control.