Demo Data from VLDB 2026 / reviewers in the wild / expert
Shikhar Bahl
dblp:223/4390
· DBLP profile ↗
9ranked-venue papers
2as first author
4since 2021 · last 2024
—ORCID · none
Domains — the database's venue-derived domains; a paper can count in several
Artificial intelligence and machine learning · 9 · 2 first-author · 4 since 2021Systems, architecture and hardware · 4 · 2 since 2021Graphics, computer vision, multimedia, augmented reality and games · 1 · 1 first-author · 1 since 2021
Expertise — from the expertise taxonomy: the topics of the expert's papers under the CCF categories. A weight counts papers with recency: 1 for a paper about the topic, 0.3 when the topic is its context, halved every five years.
| Artificial intelligence
8 papers |
Reinforcement learning · 39% Motion planning and robot control · 30% Representation and self-supervised learning · 14% |
Topics — the 23 heaviest of 25, each with the papers that count most for it
| Topic | Weight | Papers | Last | Evidence papers |
|---|---|---|---|---|
Robotics › Motion planning and robot control
robot learning |
2.1 | 3 | 2024 | Open X-Embodiment: Robotic Learning Datasets and RT-X Models : Open X-Embodiment Collaboration · ICRA 2024 ALAN: Autonomously Exploring Robotic Agents in the Real World · ICRA 2023 Affordances from Human Videos as a Versatile Representation for Robotics · CVPR 2023 |
Machine learning › Reinforcement learning › deep reinforcement learning
visual reinforcement learning |
1.0 | 2 | 2023 | Efficient RL via Disentangled Environment and Agent Representations · ICML 2023 Visual Reinforcement Learning with Imagined Goals · NeurIPS 2018 |
Robotics › Motion planning and robot control › robot learning › robot policy learning
generalist robot policy |
0.8 | 1 | 2024 | Open X-Embodiment: Robotic Learning Datasets and RT-X Models : Open X-Embodiment Collaboration · ICRA 2024 |
Robotics › Robot manipulation
affordance learning |
0.7 | 1 | 2023 | Affordances from Human Videos as a Versatile Representation for Robotics · CVPR 2023 |
Machine learning › Reinforcement learning › exploration
autonomous exploration |
0.7 | 1 | 2023 | ALAN: Autonomously Exploring Robotic Agents in the Real World · ICRA 2023 |
Machine learning › Reinforcement learning › exploration › intrinsic motivation
curiosity-driven exploration |
0.7 | 1 | 2023 | ALAN: Autonomously Exploring Robotic Agents in the Real World · ICRA 2023 |
Machine learning › Representation and self-supervised learning › representation learning
disentangled representation learning |
0.7 | 1 | 2023 | Efficient RL via Disentangled Environment and Agent Representations · ICML 2023 |
Machine learning › Reinforcement learning
exploration |
0.7 | 1 | 2023 | ALAN: Autonomously Exploring Robotic Agents in the Real World · ICRA 2023 |
Machine learning › Reinforcement learning
imitation learning |
0.7 | 1 | 2023 | Affordances from Human Videos as a Versatile Representation for Robotics · CVPR 2023 |
Machine learning › Representation and self-supervised learning
structured representation |
0.7 | 1 | 2023 | Efficient RL via Disentangled Environment and Agent Representations · ICML 2023 |
Robotics › Robot manipulation › robot vision
visual affordance |
0.7 | 1 | 2023 | Affordances from Human Videos as a Versatile Representation for Robotics · CVPR 2023 |
Robotics › Motion planning and robot control
robot control |
0.5 | 2 | 2020 | Residual Reinforcement Learning for Robot Control · ICRA 2019 Neural Dynamic Policies for End-to-End Sensorimotor Learning · NeurIPS 2020 |
Machine learning › Reinforcement learning › goal-conditioned reinforcement learning
goal-conditioned policy |
0.4 | 1 | 2020 | Skew-Fit: State-Covering Self-Supervised Reinforcement Learning · ICML 2020 |
Machine learning › Reinforcement learning
policy learning |
0.4 | 1 | 2020 | Neural Dynamic Policies for End-to-End Sensorimotor Learning · NeurIPS 2020 |
Machine learning › Reinforcement learning
self-supervised reinforcement learning |
0.4 | 1 | 2020 | Skew-Fit: State-Covering Self-Supervised Reinforcement Learning · ICML 2020 |
Robotics › Motion planning and robot control › robot control › sensor-based control
sensorimotor control |
0.4 | 1 | 2020 | Neural Dynamic Policies for End-to-End Sensorimotor Learning · NeurIPS 2020 |
Robotics › Robot manipulation
contact-rich manipulation |
0.4 | 1 | 2019 | Residual Reinforcement Learning for Robot Control · ICRA 2019 |
Robotics › Motion planning and robot control › robot control › learning control
residual reinforcement learning control |
0.4 | 1 | 2019 | Residual Reinforcement Learning for Robot Control · ICRA 2019 |
Machine learning › Reinforcement learning
goal-conditioned reinforcement learning |
0.3 | 1 | 2018 | Visual Reinforcement Learning with Imagined Goals · NeurIPS 2018 |
Machine learning › Representation and self-supervised learning › representation learning
unsupervised representation learning |
0.3 | 1 | 2018 | Visual Reinforcement Learning with Imagined Goals · NeurIPS 2018 |
Machine learning › Representation and self-supervised learning › representation learning
visual representation learning |
0.3 | 1 | 2018 | Visual Reinforcement Learning with Imagined Goals · NeurIPS 2018 |
Computer vision › Video understanding and tracking › human action analysis › action understanding
human action understanding |
0.2 | 1 | 2023 | Affordances from Human Videos as a Versatile Representation for Robotics · CVPR 2023 |
Machine learning › Reinforcement learning
off-policy reinforcement learning |
0.1 | 1 | 2018 | Visual Reinforcement Learning with Imagined Goals · NeurIPS 2018 |
Methods — techniques the papers use, named apart from their topics
reinforcement learning · 1.0transformer policy · 0.8large-scale pretraining · 0.8visual affordance model · 0.7uncertainty maximization · 0.7model-free reinforcement learning · 0.7environment change prediction · 0.7auxiliary loss · 0.7goal-reaching policy learning · 0.4entropy maximization · 0.4
| Year | Publication | Venue | Position |
|---|---|---|---|
| 2024 | Open X-Embodiment: Robotic Learning Datasets and RT-X Models : Open X-Embodiment CollaborationabstractLarge, high-capacity models trained on diverse datasets have shown remarkable successes on efficiently tackling downstream applications. In domains from NLP to Computer Vision, this has led to a consolidation of pretrained models, with general pretrained backbones serving as a starting point for many applications. Can such a consolidation happen in robotics? Conventionally, robotic learning methods train a separate model for every application, every robot, and even every environment. Can we instead train "generalist" X-robot policy that can be adapted efficiently to new robots, tasks, and environments? In this paper, we provide datasets in standardized data formats and models to make it possible to explore this possibility in the context of robotic manipulation, alongside experimental results that provide an example of effective X-robot policies. We assemble a dataset from 22 different robots collected through a collaboration between 21 institutions, demonstrating 527 skills (160266 tasks). We show that a high-capacity model trained on this data, which we call RT-X, exhibits positive transfer and improves the capabilities of multiple robots by leveraging experience from other platforms. The project website is robotics-transformer-x.github.io. Abigail O'Neill, Abhiram Maddukuri, Abhishek Gupta 0004, Abhishek Padalkar, Abraham Lee, Acorn Pooley, Agrim Gupta, Ajay Mandlekar, Ajinkya Jain, Albert Tung, Alex Bewley, Alex Irpan, Alexander Khazatsky, Anant Rai, Anchit Gupta, Andrew E. Wang, Anikait Singh, Animesh Garg, Aniruddha Kembhavi, Annie Xie, Anthony Brohan, Antonin Raffin, Archit Sharma, Arefeh Yavary, Arhan Jain, Ashwin Balakrishna, Ayzaan Wahid, Ben Burgess-Limerick, Bernhard Schölkopf, Blake Wulfe, Brian Ichter, Cewu Lu, Charles Xu 0003, Charlotte Le, Chelsea Finn, Chen Wang 0053, Chenfeng Xu, Cheng Chi 0001, Chenguang Huang, Christine Chan, Christopher Agia, Chuer Pan, Chuyuan Fu, Coline Devin, Danfei Xu, Daniel Morton, Danny Drieß, Daphne Chen, Deepak Pathak, Dhruv Shah, Dieter Büchler, Dinesh Jayaraman, Dmitry Kalashnikov, Dorsa Sadigh, Edward Johns, Ethan Paul Foster, Fangchen Liu, Federico Ceola, Fei Xia 0002, Feiyu Zhao, Freek Stulp, Gaoyue Zhou, Gaurav S. Sukhatme, Gautam Salhotra, Gilbert Feng, Giulio Schiavi, Glen Berseth, Gregory Kahn, Guanzhi Wang, Hao Su 0001, Haoshu Fang, Henghui Bao, Heni Ben Amor, Henrik I. Christensen, Hiroki Furuta, Homer Walke, Hongjie Fang, Huy Ha, Igor Mordatch, Ilija Radosavovic, Isabel Leal, Jacky Liang, Jad Abou-Chakra, Jaehyung Kim 0001, Jaimyn Drake, Jan Peters 0001, Jan Schneider 0007, Jasmine Hsu, Jeannette Bohg, Jeffrey T. Bingham, Jensen Gao, Jiaheng Hu, Jiajun Wu 0001, Jiankai Sun, Jianlan Luo, Jiayuan Gu, Jie Tan 0001, Jihoon Oh, Jimmy Wu, Jingpei Lu, Jitendra Malik, João Silvério, Joey Hejna, Jonathan Booher, Jonathan Tompson, Jonathan Yang, Jordi Salvador, Joseph J. Lim, Junhyek Han, Kanishka Rao, Karl Pertsch, Karol Hausman, Keegan Go, Keerthana Gopalakrishnan, Kenneth Y. Goldberg, Kendra Byrne, Kenneth Oslund, Kento Kawaharazuka, Kevin Black, Kevin Zhang 0002, Kiana Ehsani, Kiran Lekkala, Kirsty Ellis, Krishan Rana, Krishnan Srinivasan, Kuan Fang, Kunal Pratap Singh, Kuo-Hao Zeng, Kyle Hatch, Kyle Hsu, Laurent Itti, Yunliang Chen 0001, Lerrel Pinto, Li Fei-Fei 0001, Liam Tan, Linxi Fan, Lionel Ott, Lisa Lee, Luca Weihs, Magnum Chen, Marion Lepert, Marius Memmel, Masayoshi Tomizuka, Masha Itkina, Mateo Guaman Castro, Max Spero, Maximilian Du, Michael Ahn, Michael C. Yip, Mingtong Zhang 0003, Mingyu Ding, Minho Heo, Mohan Kumar Srirama, Mohit Sharma 0001, Moo Jin Kim, Naoaki Kanazawa, Nicklas Hansen 0001, Nicolas Heess, Nikhil J. Joshi, Niko Sünderhauf, Norman Di Palo, Nur Muhammad Shafiullah, Oier Mees, Oliver Kroemer, Osbert Bastani, Pannag R. Sanketi, Patrick Tree Miller, Patrick Yin, Paul Wohlhart, Peng Xu 0010, Peter David Fagan, Peter Mitrano, Pierre Sermanet, Pieter Abbeel, Priya Sundaresan, Qiuyu Chen, Rafael Rafailov, Ria Doshi, Roberto Martin Martin, Rohan Baijal, Rosario Scalise, Rose Hendrix, Roy Lin, Runjia Qian, Russell Mendonca, Rutav Shah, Ryan Hoque, Ryan Julian, Samuel Bustamante-Gomez, Sean Kirmani, Sergey Levine, Sherry Moore, Shikhar Bahl, Shivin Dass, Shubham D. Sonawani, Shuran Song, Sichun Xu, Siddhant Haldar, Siddharth Karamcheti, Simeon Adebola, Simon Guist, Soroush Nasiriany, Stefan Schaal, Stefan Welker, Stephen Tian, Subramanian Ramamoorthy, Sudeep Dasari, Suneel Belkhale, Sungjae Park, Suraj Nair 0003, Suvir Mirchandani, Takayuki Osa, Tanmay Gupta, Tatsuya Harada, Tatsuya Matsushima, Ted Xiao, Thomas Kollar, Tianhe Yu, Tianli Ding, Todor Davchev, Tony Z. Zhao, Travis Armstrong, Trevor Darrell, Trinity Chung, Vidhi Jain, Vincent Vanhoucke, Wolfram Burgard, Xiaolong Wang 0004, Xinghao Zhu, Xinyang Geng, Liangwei Xu, Yecheng Jason Ma 0001, Yejin Kim 0003, Yevgen Chebotar, Yilin Wu 0003, Yonatan Bisk, Yoonyoung Cho, Youngwoon Lee, Yuchen Cui, Yueh-Hua Wu, Yujin Tang, Yuke Zhu, Yunchu Zhang, Yunfan Jiang 0001, Yunshuang Li, Yunzhu Li, Yusuke Iwasawa, Yutaka Matsuo, Zehan Ma, Zichen Jeff Cui, Zichen Zhang 0016, Zipeng Lin |
ICRA | 208 |
| 2023 | Affordances from Human Videos as a Versatile Representation for RoboticsabstractBuilding a robot that can understand and learn to interact by watching humans has inspired several vision problems. However, despite some successful results on static datasets, it remains unclear how current models can be used on a robot directly. In this paper, we aim to bridge this gap by leveraging videos of human interactions in an environment centric manner. Utilizing internet videos of human behavior, we train a visual affordance model that estimates where and how in the scene a human is likely to interact. The structure of these behavioral affordances directly enables the robot to perform many complex tasks. We show how to seamlessly integrate our affordance model with four robot learning paradigms including offline imitation learning, exploration, goal-conditioned learning, and action parameterization for reinforcement learning. We show the efficacy of our approach, which we call Vision-Robotics Bridge (VRB) across 4 real world environments, over 10 different tasks, and 2 robotic platforms operating in the wild. Shikhar Bahl, Russell Mendonca, Unnat Jain, Deepak Pathak |
CVPR | 1 |
| 2023 | Efficient RL via Disentangled Environment and Agent RepresentationsabstractAgents that are aware of the separation between the environments and themselves can leverage this understanding to form effective representations of visual input. We propose an approach for learning such structured representations for RL algorithms, using visual knowledge of the agent, which is often inexpensive to obtain, such as its shape or mask. This is incorporated into the RL objective using a simple auxiliary loss. We show that our method, SEAR (Structured Environment-Agent Representations), outperforms state-of-the-art model-free approaches over 18 different challenging visual simulation environments spanning 5 different robots. Kevin Gmelin, Shikhar Bahl, Russell Mendonca, Deepak Pathak |
ICML | 2 |
| 2023 | ALAN: Autonomously Exploring Robotic Agents in the Real WorldabstractRobotic agents that operate autonomously in the real world need to continuously explore their environment and learn from the data collected, with minimal human supervision. While it is possible to build agents that can learn in such a manner without supervision, current methods struggle to scale to the real world. Thus, we propose ALAN, an autonomously exploring robotic agent, that can perform tasks in the real world with little training and interaction time. This is enabled by measuring environment change, which reflects object movement and ignores changes in the robot position. We use this metric directly as an environment-centric signal, and also maximize the uncertainty of predicted environment change, which provides agent-centric exploration signal. We evaluate our approach on two different real-world play kitchen settings, enabling a robot to efficiently explore and discover manipulation skills, and perform tasks specified via goal images. Videos can be found at https://robo-explorer.github.io/ Russell Mendonca, Shikhar Bahl, Deepak Pathak |
ICRA | 2 |
| 2020 | Skew-Fit: State-Covering Self-Supervised Reinforcement LearningabstractAutonomous agents that must exhibit flexible and broad capabilities will need to be equipped with large repertoires of skills. Defining each skill with a manually-designed reward function limits this repertoire and imposes a manual engineering burden. Self-supervised agents that set their own goals can automate this process, but designing appropriate goal setting objectives can be difficult, and often involves heuristic design decisions. In this paper, we propose a formal exploration objective for goal-reaching policies that maximizes state coverage. We show that this objective is equivalent to maximizing goal reaching performance together with the entropy of the goal distribution, where goals correspond to full state observations. To instantiate this principle, we present an algorithm called Skew-Fit for learning a maximum-entropy goal distributions. We prove that, under regularity conditions, Skew-Fit converges to a uniform distribution over the set of valid states, even when we do not know this set beforehand. Our experiments show that combining Skew-Fit for learning goal distributions with existing goal-reaching methods outperforms a variety of prior methods on open-sourced visual goal-reaching tasks. Moreover, we demonstrate that Skew-Fit enables a real-world robot to learn to open a door, entirely from scratch, from pixels, and without any manually-designed reward function. Vitchyr Pong, Murtaza Dalal, Steven Lin, Ashvin Nair, Shikhar Bahl, Sergey Levine |
ICML | 5 |
| 2020 | Deep Reinforcement Learning for Industrial Insertion Tasks with Visual Inputs and Natural RewardsabstractConnector insertion and many other tasks commonly found in modern manufacturing settings involve complex contact dynamics and friction. Since it is difficult to capture related physical effects with first-order modeling, traditional control methods often result in brittle and inaccurate controllers, which have to be manually tuned. Reinforcement learning (RL) methods have been demonstrated to be capable of learning controllers in such environments from autonomous interaction with the environment, but running RL algorithms in the real world poses sample efficiency and safety challenges. Moreover, in practical real-world settings, we cannot assume access to perfect state information or dense reward signals. In this paper, we consider a variety of difficult industrial insertion tasks with visual inputs and different natural reward specifications, namely sparse rewards and goal images. We show that methods that combine RL with prior information, such as classical controllers or demonstrations, can solve these tasks from a reasonable amount of real-world interaction. Gerrit Schoettler, Ashvin Nair, Jianlan Luo, Shikhar Bahl, Juan Aparicio Ojea, Eugen Solowjow, Sergey Levine |
IROS | 4 |
| 2020 | Neural Dynamic Policies for End-to-End Sensorimotor LearningabstractThe current dominant paradigm in sensorimotor control, whether imitation or reinforcement learning, is to train policies directly in raw action spaces such as torque, joint angle, or end-effector position. This forces the agent to make decision at each point in training, and hence, limits the scalability to continuous, high-dimensional, and long-horizon tasks. In contrast, research in classical robotics has, for a long time, exploited dynamical systems as a policy representation to learn robot behaviors via demonstrations. These techniques, however, lack the flexibility and generalizability provided by deep learning or deep reinforcement learning and have remained under-explored in such settings. In this work, we begin to close this gap and embed dynamics structure into deep neural network-based policies by reparameterizing action spaces with differential equations. We propose Neural Dynamic Policies (NPDs) that make predictions in trajectory distribution space as opposed to prior policy learning methods where action represents the raw control space. The embedded structure allows us to perform end-to-end policy learning under both reinforcement and imitation learning setups. We show that NDPs achieve better or comparable performance to state-of-the-art approaches on many robotic control tasks using both reward-based training and demonstrations. Project video and code are available at: https://shikharbahl.github.io/neural-dynamic-policies/. Shikhar Bahl, Mustafa Mukadam, Abhinav Gupta 0001, Deepak Pathak |
NeurIPS | 1 |
| 2019 | Residual Reinforcement Learning for Robot ControlabstractConventional feedback control methods can solve various types of robot control problems very efficiently by capturing the structure with explicit models, such as rigid body equations of motion. However, many control problems in modern manufacturing deal with contacts and friction, which are difficult to capture with first-order physical modeling. Hence, applying control design methodologies to these kinds of problems often results in brittle and inaccurate controllers, which have to be manually tuned for deployment. Reinforcement learning (RL) methods have been demonstrated to be capable of learning continuous robot controllers from interactions with the environment, even for problems that include friction and contacts. In this paper, we study how we can solve difficult control problems in the real world by decomposing them into a part that is solved efficiently by conventional feedback control methods, and the residual which is solved with RL. The final control policy is a superposition of both control signals. We demonstrate our approach by training an agent to successfully perform a real-world block assembly task involving contacts and unstable objects. Tobias Johannink, Shikhar Bahl, Ashvin Nair, Jianlan Luo, Avinash Kumar 0005, Matthias Loskyll, Juan Aparicio Ojea, Eugen Solowjow, Sergey Levine |
ICRA | 2 |
| 2018 | Visual Reinforcement Learning with Imagined GoalsabstractFor an autonomous agent to fulfill a wide range of user-specified goals at test time, it must be able to learn broadly applicable and general-purpose skill repertoires. Furthermore, to provide the requisite level of generality, these skills must handle raw sensory input such as images. In this paper, we propose an algorithm that acquires such general-purpose skills by combining unsupervised representation learning and reinforcement learning of goal-conditioned policies. Since the particular goals that might be required at test-time are not known in advance, the agent performs a self-supervised "practice" phase where it imagines goals and attempts to achieve them. We learn a visual representation with three distinct purposes: sampling goals for self-supervised practice, providing a structured transformation of raw sensory inputs, and computing a reward signal for goal reaching. We also propose a retroactive goal relabeling scheme to further improve the sample-efficiency of our method. Our off-policy algorithm is efficient enough to learn policies that operate on raw image observations and goals in a real-world physical system, and substantially outperforms prior techniques. Ashvin Nair, Vitchyr Pong, Murtaza Dalal, Shikhar Bahl, Steven Lin, Sergey Levine |
NeurIPS | 4 |