While commendable progress has been made in user-centric research on mobile assistive systems for blind and low-vision (BLV) individuals, references that directly inform robot navigation design remain rare. To bridge this gap, we conducted a comprehensive human study involving interviews with 26 guide dog handlers, four white cane users, nine guide dog trainers, and one O&M trainer, along with 15+ hours of observing guide dog-assisted walking. After de-identification, we open-sourced the dataset to promote human-centered development and informed decision-making for assistive systems for BLV people. Building on insights from this formative study, we developed GuideNav, a vision-only, teach-and-repeat navigation system. Inspired by how guide dogs are trained and assist their handlers, GuideNav autonomously repeats a path demonstrated by a sighted person using a robot. Specifically, the system constructs a topological representation of the taught route, integrates visual place recognition with temporal filtering, and employs a relative pose estimator to compute navigation actions-all without relying on costly, heavy, power-hungry sensors such as LiDAR. In field tests, GuideNav consistently achieved kilometer-scale route following across five outdoor environments, maintaining reliability despite noticeable scene variations between teach and repeat runs. A user study with 3 guide dog handlers and 1 guide dog trainer further confirmed the system's feasibility, marking, to our knowledge, the first demonstration of a quadruped mobile system guiding a route in a manner comparable to guide dogs.

Bodun Hu, Shuozhe Li, Saurabh Agarwal, Myungjin Lee, Akshay Jajoo, Jiamin Li, Le Xu, Geon-Woo Kim, Donghyun Kim, Hong Xu, Amy Zhang, Aditya Akella. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2025.

Workload traces are essential to understand complex computer systems' behavior and manage processing and memory resources.Since real-world traces are hard to obtain, synthetic trace generation is a promising alternative.This paper proposes a first-of-a-kind approach that relies on training a large language model (LLM) to generate synthetic workload traces, specifically microservice call graphs.To capture complex and arbitrary hierarchical structures and implicit constraints in such traces, we propose to train LLMs to generate recursively, making call graph generation a sequence of more manageable steps.To further enforce learning constraints on the traces and generate uncommon situations, we apply additional instruction tuning steps to align our model with the desired trace features.With this method, we train TraceLLM, an LLM for microservice trace generation, and demonstrate that it produces diverse, realistic traces under varied conditions, outperforming existing approaches in both accuracy and validity.The synthetically generated traces can effectively replace real data to optimize important microservice management tasks.Additionally, TraceLLM adapts to downstream trace-related tasks, such as predicting key trace features and infilling missing data.

Dog guides are favored by blind and low-vision (BLV) individuals for their ability to enhance independence and confidence by reducing safety concerns and increasing navigation efficiency compared to traditional mobility aids. However, only a relatively small proportion of BLV individuals work with dog guides due to their limited availability and associated maintenance responsibilities. There is considerable recent interest in addressing this challenge by developing legged guide dog robots. This study was designed to determine critical aspects of the handler-guide dog interaction and better understand handler needs to inform guide dog robot development. We conducted semi-structured interviews and observation sessions with 23 dog guide handlers and 5 trainers. Thematic analysis revealed critical limitations in guide dog work, desired personalization in handler-guide dog interaction, and important perspectives on future guide dog robots. Grounded on these findings, we discuss pivotal design insights for guide dog robots aimed for adoption within the BLV community.

We introduce StaccaToe, a human-scale, electric motor-powered single-leg robot designed to rival the agility of human locomotion through two distinctive attributes: an actuated toe and a co-actuation configuration inspired by the human leg. Leveraging the foundational design of HyperLeg’s lower leg mechanism, we develop a stand-alone robot by incorporating new link designs, custom-designed power electronics, and a refined control system. Unlike previous jumping robots that rely on either special mechanisms (e.g., springs and clutches) or hydraulic/pneumatic actuators, StaccaToe employs electric motors without energy storage mechanisms. This choice underscores our ultimate goal of developing a practical, high-performance humanoid robot capable of human-like, stable walking as well as explosive dynamic movements. In this paper, we aim to empirically evaluate the balance capability and the exertion of explosive ground reaction forces of our toe and co-actuation mechanisms. Throughout extensive hardware and controller development, StaccaToe showcases its control fidelity by demonstrating a balanced tip-toe stance and dynamic jump. This study is significant for three key reasons: 1) StaccaToe represents the first human-scale, electric motor-driven single-leg robot to execute dynamic maneuvers without relying on specialized mechanisms; 2) our research provides empirical evidence of the benefits of replicating critical human leg attributes in robotic design; and 3) we explain the design process for creating agile legged robots, the details that have been scantily covered in academic literature.

A robot guide dog has compelling advantages over animal guide dogs for its cost-effectiveness, the potential for mass production, and low maintenance burden. However, despite the long history of guide dog robot research, previous studies were conducted with little or no consideration of how the guide dog handler and the guide dog work as a team for navigation. To develop a robotic guiding system that genuinely benefits blind or visually impaired individuals, we performed qualitative research, including interviews with guide dog handlers, trainers, and first-hand blindfold walking experiences with various guide dogs. We build a collaborative indoor navigation scheme for a guide dog robot that includes preferred features such as speed and directional control. For collaborative navigation, we propose a semantic-aware local path planner that enables safe and efficient guiding work by utilizing semantic information about the environment and considering the handler's position and directional cues to determine the collision-free path. We evaluate our integrated robotic system by testing blindfolded walking in indoor settings and demonstrate guide dog-like navigation behavior by avoiding obstacles at typical gait speed (0.7m/s). The following demonstration video link includes an audio description: https://youtu.be/YxlcMeaL7GA

Our paper proposes a direct sparse visual odometry method that combines event and RGBD data to estimate the pose of agile-legged robots during dynamic locomotion and acrobatic behaviors. Event cameras offer high temporal resolution and dynamic range, which can eliminate the issue of blurred RGB images during fast movements. This unique strength holds a potential for accurate pose estimation of agile- legged robots, which has been a challenging problem to tackle. Our framework leverages the benefits of both RGBD and event cameras to achieve robust and accurate pose estimation, even during dynamic maneuvers such as jumping and landing a quadruped robot, the Mini-Cheetah. Our major contributions are threefold: Firstly, we introduce an adaptive time surface (ATS) method that addresses the whiteout and blackout issue in conventional time surfaces by formulating pixel-wise decay rates based on scene complexity and motion speed. Secondly, we develop an effective pixel selection method that directly samples from event data and applies sample filtering through ATS, enabling us to pick pixels on distinct features. Lastly, we propose a nonlinear pose optimization formula that simultaneously performs 3D-2D alignment on both RGB-based and event-based maps and images, allowing the algorithm to fully exploit the benefits of both data streams. We extensively evaluate the performance of our framework on both the public dataset and our own quadruped robot dataset, demonstrating its effectiveness in accurately estimating the pose of agile robots during dynamic movements. Supplemental video: https://youtu.be/-5ieQShOg3M

Technology-mediated dance experiences, as a medium of entertainment, are a key element in both traditional and virtual reality-based gaming platforms. These platforms predominantly depend on unobtrusive and continuous human pose estimation as a means of capturing input. Current solutions primarily employ RGB or RGB-Depth cameras for dance gaming applications; however, the former is hindered by low-light conditions due to motion blur and reduced sensitivity, while the latter exhibits excessive power consumption, diminished frame rates, and restricted operational distance. Boasting ultra-low latency, energy efficiency, and a wide dynamic range, neuromorphic cameras present a viable solution to surmount these limitations. Here, we introduce YeLan, a neuromorphic camera-driven, three-dimensional, high-frequency human pose estimation (HPE) system capable of withstanding low-light environments and dynamic backgrounds. We have compiled the first-ever neuromorphic camera dance HPE dataset and devised a fully adaptable motion-to-event, physics-conscious simulator. YeLan surpasses baseline models under strenuous conditions and exhibits resilience against varying clothing types, background motion, viewing angles, occlusions, and lighting fluctuations.

Quadrupedal landing is a complex process involving large impacts, elaborate contact transitions, and is a crucial recovery behavior observed in many biological animals. This work presents a real-time, optimal landing controller that is free of pre-specified contact schedules. The controller determines optimal touchdown postures and reaction force profiles and is able to recover from a variety of falling configurations. The quadrupedal platform used, the MIT Mini Cheetah, recovered safely from drops of up to 8 m in simulation, as well as from a range of orientations and planar velocities. The controller is also tested on hardware, successfully recovering from drops of up to 2 m.

Enabling mobile robots for solving challenging and diverse shape, texture, and motion related tasks with high fidelity vision requires the integration of novel multimodal imaging sensors and advanced fusion techniques. However, it is associated with high cost, power, hardware modification, and computing requirements which limit its scalability. In this paper, we propose a novel Simultaneously Learned Auto Encoder Domain Adaptation (SAEDA)-based transfer learning technique to empower noisy sensing with advanced sensor suite capabilities. In this regard, SAEDA trains both source and target auto-encoders together on a single graph to obtain the domain invariant feature space between the source and target domains on simultaneously collected data. Then, it uses the domain invariant feature space to transfer knowledge between different signal modalities. The evaluation has been done on two collected datasets (LiDAR and Radar) and one existing dataset (LiDAR, Radar and Video) which provides a significant improvement in quadruped robot-based classification (home floor and human activity recognition) and regression (surface roughness estimation) problems. We also integrate our sensor suite and SAEDA framework on two real-time systems (vacuum cleaning and Mini-Cheetah quadruped robots) for studying the feasibility and usability.

Legged robots have been highlighted as promising mobile platforms for disaster response and rescue scenarios because of their rough terrain locomotion capability. In cluttered environments, small robots are desirable as they can maneuver through small gaps, narrow paths, or tunnels. However small robots have their own set of difficulties such as limited space for sensors, limited obstacle clearance, and scaled-down walking speed. In this paper, we extensively address these difficulties via effective sensor integration and exploitation of dynamic locomotion and jumping. We integrate two Intel RealSense sensors into the MIT Mini-Cheetah, a 0.3 m tall, 9 kg quadruped robot. Simple and effective filtering and evaluation algorithms are used for foothold adjustment and obstacle avoidance. We showcase the exploration of highly irregular terrain using dynamic trotting and jumping with the small-scale, fully sensorized Mini-Cheetah quadruped robot.

Animal-level agility and robustness in robots cannot be accomplished by solely relying on blind locomotion controllers. A significant portion of a robot's ability to traverse terrain comes from reacting to the external world through visual sensing. However, embedding the sensors and compute that provide sufficient accuracy at high speeds is challenging, especially if the robot has significant space limitations. In this paper, we propose a system integration of a small-scale quadruped robot, the MIT Mini-Cheetah Vision, that exteroceptively senses the terrain and dynamically explores the world around it at high velocities. Through extensive hardware and software development, we demonstrate a fully untethered robot with all hardware onboard running a locomotion controller that combines state-of-the-art Regularized Predictive Control (RPC) with Whole-Body Impulse Control (WBIC). We devise a hierarchical state estimator that integrates kinematic, IMU, and localization sensor data to provide state estimates specific to path planning and locomotion tasks. Our integrated system has demonstrated robust autonomous waypoint tracking in dynamic real-world environments at speeds of over 1 m/s with high rates of success.

The ability to measure multi-axis contact forces and contact surface normals in real time is critical to allow robots to improve their dexterous manipulation and locomotion abilities. This paper presents a new fingertip sensor for 3-axis contact force and contact location detection, as well as improvements on an existing footpad sensor through use of a new artificial neural network estimator. The fingertip sensor is intended for use in manipulation, while the footpad sensor is intended for high force use in locomotion. Both sensors consist of pressure sensing elements embedded within a rubber hemisphere, and utilize an artificial neural network to estimate the applied forces (fx, fy, and fz), and contact angles (θ and φ) from the individual sensor element readings. The sensors are inherently robust, and the hemispherical shape allows for easy integration into point feet and fingertips. Both the fingertip and footpad sensors demonstrate the ability to track forces and angles accurately over the surface of the hemisphere (θ=±45° and φ=±45°) and can experience up to 25N and 450N normal force, respectively, without saturating. The performance of the sensor is demonstrated with experimental results of dynamic control of a robotic arm with real-time sensor feedback.

In robotic tasks that require physical interactions such as manipulation and legged locomotion, it is important to simultaneously measure contact forces and contact angles. This paper presents a unified solution for simultaneously measuring three axis contact forces and contact angles for legged locomotion or manipulation. Unlike most tactile sensors, the presented design utilizes the stress field method by sampling pressures over multiple locations within an elastomer, enabling inherently robust operation against impact and abrasive interactions. The presented sensor is designed for point-feet quadrupedal robots and can be easily scaled down for other applications such as grasping. The sampled stress distribution is mapped to output forces fx, fy, and fzand two contact angles, θ and ψ on the hemispherical sensor surface via Gaussian process regression. The prototype sensor is able track normal and shear forces accurately, achieving a normalized root mean (RMS) squared error of only 1.00% - 1.36% for fzacross multiple tests with up to 180N normal force, and a normalized RMS error of 1.71% - 4.67% and 1.82% - 6.68% for fxand fy, respectively, with up to 80N shear force. Additionally, the footpad is able to estimate the contact location coordinates θ and ψ with a normalized RMS error of 2.69% -7.51% over a range of 0-40° and 2.79% - 9.62% over a range of 0-30°, respectively. The footpad can estimate contact location over a maximum range of θ = ±45° and ψ = ±45°, and can withstand over 450N of normal force at location θ = ψ = 0° without reaching saturation. This prototype demonstrates the ability to simultaneously measure force in three axes and contact angles using Gaussian process regression, with the potential to explore other regression methods for embedded computing and miniaturization of the design for finger tip scale sensors.

In this paper, we present a sampling-based kino-dynamic planning framework for a bipedal robot in complex environments. Unlike other footstep planning algorithms which typically plan footstep locations and the biped dynamics in separate steps, we handle both simultaneously. Three primary advantages of this approach are (1) the ability to differentiate alternate routes while selecting footstep locations based on the temporal duration of the route as determined by the Linear Inverted Pendulum Model (LIPM) dynamics, (2) the ability to perform collision checking through time so that collisions with moving obstacles are prevented without avoiding their entire trajectory, and (3) the ability to specify a minimum forward velocity for the biped. To generate a dynamically consistent description of the walking behavior, we exploit the Phase Space Planner (PSP) [1] [2]. To plan a collision-free route toward the goal, we adapt planning strategies from non-holonomic wheeled robots to gather a sequence of inputs for the PSP. This allows us to efficiently approximate dynamic and kinematic constraints on bipedal motion, to apply a sampling-based planning algorithm such as RRT or RRT*, and to use the Dubin's path [3] as the steering method to connect two points in the configuration space. The results of the algorithm are sent to a Whole Body Controller [1] to generate full body dynamic walking behavior. Our planning algorithm is tested in a 3D physics-based simulation of the humanoid robot Valkyrie.

In this paper, we devise methods for the multiobjective control of humanoid robots, a.k.a.prioritized wholebody controllers, that achieve efficiency and robustness in the algorithmic computations.We use a form of whole-body controllers that is very general via incorporating centroidal momentum dynamics, operational task priorities, contact reaction forces, and internal force constraints.First, we achieve efficiency by solving a quadratic program that only involves the floating base dynamics and the reaction forces.Second, we achieve computational robustness by relaxing task accelerations such that they comply with friction cone constraints.Finally, we incorporate methods for smooth contact transitions to enhance the control of dynamic locomotion behaviors.The proposed methods are demonstrated both in simulation and in real experiments using a passive-ankle bipedal robot.

Whole-body operational space controllers (WBOSCs) are versatile and well suited for simultaneously controlling motion and force behaviors, which can enable sophisticated modes of locomotion and balance. In this paper, we formulate a WBOSC for point-foot bipeds with series-elastic actuators (SEA) and experiment with it using a teen-size SEA biped robot. Our main contributions are on devising a WBOSC strategy for point-foot bipedal robots, 2) formulating planning algorithms for achieving unsupported dynamic balancing on our point-foot biped robot and testing them using a WBOSC, and 3) formulating force feedback control of the internal forces-corresponding to the subset of contact forces that do not generate robot motions-to regulate contact interactions with the complex environment. We experimentally validate the efficacy of our new whole-body control and planning strategies via balancing over a disjointed terrain and attaining dynamic balance through continuous stepping without a mechanical support.

Legged robots naturally exhibit continuous and discrete dynamics when maneuvering over level-ground and uneven terrains. In recent years, numerous studies have focused on locomotion hybrid dynamics. However, locomotion on more challenging terrains such as split wedges in Figure 1 has rarely been explored, let alone its hybrid dynamics. In this study, we specifically focus on a two-phase hybrid automaton formulation for this highly steep wedge locomotion. This automaton incorporates both multi-contact and flight single contact phase motions. To dynamically balance and jump upwards on this wedge, an aperiodic phase space planning is used for trajectory generations. Three control strategies are employed simultaneously: internal force control, linear and angular momentum control. Finally, simulation results are shown to verify our strategy's effectiveness.

This paper presents a control scheme for ensuring that a 3D, under-actuated, point-foot biped robot remains balanced while walking. It achieves this by observing the center of mass (COM) position error relative to a reference path and re-planning a new reference trajectory to remove this error at every step. The Prismatic Inverted Pendulum Model (PIPM) is used to simplify behavioral analysis of the robot. We use phase space techniques to plan the COM trajectories and foot placement. While obtaining a stable path using this simplified model is easy, when applied to a real robot, there will usually be deviation from the expected path due to modeling inaccuracies. Although fully-actuated robots can reduce the deviation with relatively simple feedback control loops, when working with under-actuated robots, it is challenging to design such a feedback control loop. Our approach is based on continuous re-planning. By planning the path of the next step based on the observed initial error, we can find the proper landing location of each step. For each step we allocate sufficient time to avoid disturbances from the moment induced by the moving leg, which is not modeled in the PIPM. Our control scheme relies on the PIPM instead of the Linear Inverted Pendulum Model (LIPM) to enable non-planar COM motion, which is essential for rough terrain locomotion. We show simulation results that include full multi-body dynamics, friction, and ground reaction forces.