Demo Data from VLDB 2026 / reviewers in the wild / expert
Suraj Nair 0003
dblp:52/5152-3
· DBLP profile ↗
10ranked-venue papers
3as first author
5since 2021 · last 2025
0000-0002-3999-2436ORCID · conflict
Domains — the database's venue-derived domains; a paper can count in several
Artificial intelligence and machine learning · 10 · 3 first-author · 5 since 2021Systems, architecture and hardware · 4 · 1 first-author · 2 since 2021Graphics, computer vision, multimedia, augmented reality and games · 2 · 1 since 2021
Expertise — from the expertise taxonomy: the topics of the expert's papers under the CCF categories. A weight counts papers with recency: 1 for a paper about the topic, 0.3 when the topic is its context, halved every five years.
| Artificial intelligence
10 papers |
Reinforcement learning · 26% Robot manipulation · 19% Motion planning and robot control · 18% | |
| Software engineering, system software, and programming languages
1 paper |
Program synthesis and code generation · 100% |
Topics — the 28 heaviest of 32, each with the papers that count most for it
| Topic | Weight | Papers | Last | Evidence papers |
|---|---|---|---|---|
Robotics › Motion planning and robot control
robot learning |
1.1 | 2 | 2024 | Open X-Embodiment: Robotic Learning Datasets and RT-X Models : Open X-Embodiment Collaboration · ICRA 2024 Neural Task Programming: Learning to Generalize Across Hierarchical Tasks · ICRA 2018 |
Machine learning › Reinforcement learning
model-based reinforcement learning |
0.9 | 2 | 2021 | Model-Based Visual Planning with Self-Supervised Functional Distances · ICLR 2021 Goal-Aware Prediction: Learning to Model What Matters · ICML 2020 |
Machine learning › Reinforcement learning › goal-conditioned reinforcement learning
goal-conditioned policy |
0.9 | 1 | 2025 | GHIL-Glue: Hierarchical Control with Filtered Subgoal Images · ICRA 2025 |
Robotics › Motion planning and robot control › robot learning › manipulation learning
language-conditioned manipulation |
0.9 | 1 | 2025 | GHIL-Glue: Hierarchical Control with Filtered Subgoal Images · ICRA 2025 |
Robotics › Motion planning and robot control › robot learning › robot policy learning
generalist robot policy |
0.8 | 1 | 2024 | Open X-Embodiment: Robotic Learning Datasets and RT-X Models : Open X-Embodiment Collaboration · ICRA 2024 |
Computer vision › Vision and language
visual question answering |
0.8 | 1 | 2024 | Prismatic VLMs: Investigating the Design Space of Visually-Conditioned Language Models · ICML 2024 |
Robotics › Robot manipulation
subgoal image generation |
0.7 | 2 | 2025 | Hierarchical Foresight: Self-Supervised Learning of Long-Horizon Tasks via Visual Subgoal Generation · ICLR 2020 GHIL-Glue: Hierarchical Control with Filtered Subgoal Images · ICRA 2025 |
Machine learning › Generative modeling › variational autoencoder
hierarchical latent variable model |
0.5 | 1 | 2021 | Greedy Hierarchical Variational Autoencoders for Large-Scale Video Prediction · CVPR 2021 |
Machine learning › Efficient and distributed learning
memory-efficient training |
0.5 | 1 | 2021 | Greedy Hierarchical Variational Autoencoders for Large-Scale Video Prediction · CVPR 2021 |
Machine learning › Generative modeling
variational autoencoder |
0.5 | 1 | 2021 | Greedy Hierarchical Variational Autoencoders for Large-Scale Video Prediction · CVPR 2021 |
Computer vision › Video understanding and tracking
video prediction |
0.5 | 1 | 2021 | Greedy Hierarchical Variational Autoencoders for Large-Scale Video Prediction · CVPR 2021 |
Robotics › Robot navigation and mapping
visual navigation |
0.5 | 1 | 2021 | Model-Based Visual Planning with Self-Supervised Functional Distances · ICLR 2021 |
Knowledge, reasoning and agents › Planning, search and constraint satisfaction › planning
visual planning |
0.5 | 1 | 2021 | Model-Based Visual Planning with Self-Supervised Functional Distances · ICLR 2021 |
Robotics › Robot manipulation
assembly |
0.4 | 1 | 2020 | TRASS: Time Reversal as Self-Supervision · ICRA 2020 |
Machine learning › Reinforcement learning
goal-conditioned reinforcement learning |
0.4 | 1 | 2020 | Goal-Aware Prediction: Learning to Model What Matters · ICML 2020 |
Machine learning › Reinforcement learning
hierarchical reinforcement learning |
0.4 | 1 | 2020 | Hierarchical Foresight: Self-Supervised Learning of Long-Horizon Tasks via Visual Subgoal Generation · ICLR 2020 |
Machine learning › Reinforcement learning › model-based reinforcement learning › world model
learned dynamics models |
0.4 | 1 | 2020 | Goal-Aware Prediction: Learning to Model What Matters · ICML 2020 |
Machine learning › Reinforcement learning
long-horizon tasks |
0.4 | 1 | 2020 | Hierarchical Foresight: Self-Supervised Learning of Long-Horizon Tasks via Visual Subgoal Generation · ICLR 2020 |
Machine learning › Reinforcement learning
imitation learning |
0.4 | 1 | 2019 | Neural Task Graphs: Generalizing to Unseen Tasks From a Single Video Demonstration · CVPR 2019 |
Robotics › Robot manipulation › learning from demonstration
task learning from demonstration |
0.4 | 1 | 2019 | Neural Task Graphs: Generalizing to Unseen Tasks From a Single Video Demonstration · CVPR 2019 |
Robotics › Robot manipulation › learning from demonstration
few-shot learning from demonstration |
0.3 | 1 | 2018 | Neural Task Programming: Learning to Generalize Across Hierarchical Tasks · ICRA 2018 |
Knowledge, reasoning and agents › Planning, search and constraint satisfaction › hierarchical planning
hierarchical task decomposition |
0.3 | 1 | 2018 | Neural Task Programming: Learning to Generalize Across Hierarchical Tasks · ICRA 2018 |
Machine learning › Generative modeling › visual generation
image and video synthesis |
0.3 | 1 | 2025 | GHIL-Glue: Hierarchical Control with Filtered Subgoal Images · ICRA 2025 |
Machine learning › Trustworthy machine learning
hallucination |
0.2 | 1 | 2024 | Prismatic VLMs: Investigating the Design Space of Visually-Conditioned Language Models · ICML 2024 |
Machine learning › Trustworthy machine learning
interpretability |
0.2 | 1 | 2024 | Prismatic VLMs: Investigating the Design Space of Visually-Conditioned Language Models · ICML 2024 |
Computer vision › Image recognition and object detection
object localization |
0.2 | 1 | 2024 | Prismatic VLMs: Investigating the Design Space of Visually-Conditioned Language Models · ICML 2024 |
Machine learning › Reinforcement learning › model-based reinforcement learning
model-based planning |
0.1 | 1 | 2021 | Greedy Hierarchical Variational Autoencoders for Large-Scale Video Prediction · CVPR 2021 |
Program synthesis and code generation › inductive program synthesis
neural program induction |
0.1 | 1 | 2018 | Neural Task Programming: Learning to Generalize Across Hierarchical Tasks · ICRA 2018 |
Methods — techniques the papers use, named apart from their topics
self-supervised learning · 1.4imitation learning · 0.9hierarchical control · 0.9generative model · 0.9visual representation pretraining · 0.8transformer policy · 0.8large-scale pretraining · 0.8instruction tuning · 0.8variational inference · 0.5greedy layer-wise training · 0.5neural program induction · 0.3hierarchical neural program · 0.3few-shot learning · 0.3
| Year | Publication | Venue | Position |
|---|---|---|---|
| 2025 | GHIL-Glue: Hierarchical Control with Filtered Subgoal ImagesabstractImage and video generative models that are pretrained on Internet-scale data can greatly increase the generalization capacity of robot learning systems. These models can function as high-level planners, generating intermediate sub-goals for low-level goal-conditioned policies to reach. However, the performance of these systems can be greatly bottlenecked by the interface between generative models and low-level controllers. For example, generative models may predict photo-realistic yet physically infeasible frames that confuse low-level policies. Low-level policies may also be sensitive to subtle visual artifacts in generated goal images. This paper addresses these two facets of generalization, providing an interface to effectively “glue together” language-conditioned image or video prediction models with low-level goal-conditioned policies. Our method, Generative Hierarchical Imitation Learning-Glue (GHIL-Glue), filters out subgoals that do not lead to task progress and improves the robustness of goal-conditioned policies to generated subgoals with harmful visual artifacts. We find in extensive experiments in both simulated and real environments that GHIL-Glue achieves a 25% improvement across several hierarchical models that leverage generative subgoals, achieving a new state-of-the-art on the CALVIN simulation benchmark for policies using observations from a single RGB camera. GHIL-Glue also outperforms other generalist robot policies across 3/4 language-conditioned manipulation tasks testing zero-shot generalization in physical experiments. Code, model checkpoints, videos, and supplementary materials can be found at https://ghil-glue.github.io. Kyle Hatch, Ashwin Balakrishna, Oier Mees, Suraj Nair 0003, Seohong Park, Blake Wulfe, Masha Itkina, Benjamin Eysenbach, Sergey Levine, Thomas Kollar, Benjamin Burchfiel |
ICRA | 4 |
| 2024 | Prismatic VLMs: Investigating the Design Space of Visually-Conditioned Language ModelsabstractVisually-conditioned language models (VLMs) have seen growing adoption in applications such as visual dialogue, scene understanding, and robotic task planning; adoption that has fueled a wealth of new models such as LLaVa, InstructBLIP, and PaLI-3. Despite the volume of new releases, key design decisions around image preprocessing, architecture, and optimization are under-explored, making it challenging to understand what factors account for model performance – a challenge further complicated by the lack of objective, consistent evaluations. To address these gaps, we first compile a suite of standardized evaluations spanning visual question answering, object localization, and challenge sets that probe properties such as hallucination; evaluations that provide fine-grained insight VLM capabilities. Second, we rigorously investigate VLMs along key design axes, including pretrained visual representations and training from base vs. instruct-tuned language models, amongst others. We couple our analysis with three resource contributions: (1) a unified framework for evaluating VLMs, (2) optimized, flexible training code, and (3) checkpoints for all models, including a family of VLMs at the 7-13B scale that strictly outperform InstructBLIP and LLaVa v1.5, the state-of-the-art in open VLMs. Siddharth Karamcheti, Suraj Nair 0003, Ashwin Balakrishna, Percy Liang, Thomas Kollar, Dorsa Sadigh |
ICML | 2 |
| 2024 | Open X-Embodiment: Robotic Learning Datasets and RT-X Models : Open X-Embodiment CollaborationabstractLarge, high-capacity models trained on diverse datasets have shown remarkable successes on efficiently tackling downstream applications. In domains from NLP to Computer Vision, this has led to a consolidation of pretrained models, with general pretrained backbones serving as a starting point for many applications. Can such a consolidation happen in robotics? Conventionally, robotic learning methods train a separate model for every application, every robot, and even every environment. Can we instead train "generalist" X-robot policy that can be adapted efficiently to new robots, tasks, and environments? In this paper, we provide datasets in standardized data formats and models to make it possible to explore this possibility in the context of robotic manipulation, alongside experimental results that provide an example of effective X-robot policies. We assemble a dataset from 22 different robots collected through a collaboration between 21 institutions, demonstrating 527 skills (160266 tasks). We show that a high-capacity model trained on this data, which we call RT-X, exhibits positive transfer and improves the capabilities of multiple robots by leveraging experience from other platforms. The project website is robotics-transformer-x.github.io. Abigail O'Neill, Abhiram Maddukuri, Abhishek Gupta 0004, Abhishek Padalkar, Abraham Lee, Acorn Pooley, Agrim Gupta, Ajay Mandlekar, Ajinkya Jain, Albert Tung, Alex Bewley, Alex Irpan, Alexander Khazatsky, Anant Rai, Anchit Gupta, Andrew E. Wang, Anikait Singh, Animesh Garg, Aniruddha Kembhavi, Annie Xie, Anthony Brohan, Antonin Raffin, Archit Sharma, Arefeh Yavary, Arhan Jain, Ashwin Balakrishna, Ayzaan Wahid, Ben Burgess-Limerick, Bernhard Schölkopf, Blake Wulfe, Brian Ichter, Cewu Lu, Charles Xu 0003, Charlotte Le, Chelsea Finn, Chen Wang 0053, Chenfeng Xu, Cheng Chi 0001, Chenguang Huang, Christine Chan, Christopher Agia, Chuer Pan, Chuyuan Fu, Coline Devin, Danfei Xu, Daniel Morton, Danny Drieß, Daphne Chen, Deepak Pathak, Dhruv Shah, Dieter Büchler, Dinesh Jayaraman, Dmitry Kalashnikov, Dorsa Sadigh, Edward Johns, Ethan Paul Foster, Fangchen Liu, Federico Ceola, Fei Xia 0002, Feiyu Zhao, Freek Stulp, Gaoyue Zhou, Gaurav S. Sukhatme, Gautam Salhotra, Gilbert Feng, Giulio Schiavi, Glen Berseth, Gregory Kahn, Guanzhi Wang, Hao Su 0001, Haoshu Fang, Henghui Bao, Heni Ben Amor, Henrik I. Christensen, Hiroki Furuta, Homer Walke, Hongjie Fang, Huy Ha, Igor Mordatch, Ilija Radosavovic, Isabel Leal, Jacky Liang, Jad Abou-Chakra, Jaehyung Kim 0001, Jaimyn Drake, Jan Peters 0001, Jan Schneider 0007, Jasmine Hsu, Jeannette Bohg, Jeffrey T. Bingham, Jensen Gao, Jiaheng Hu, Jiajun Wu 0001, Jiankai Sun, Jianlan Luo, Jiayuan Gu, Jie Tan 0001, Jihoon Oh, Jimmy Wu, Jingpei Lu, Jitendra Malik, João Silvério, Joey Hejna, Jonathan Booher, Jonathan Tompson, Jonathan Yang, Jordi Salvador, Joseph J. Lim, Junhyek Han, Kanishka Rao, Karl Pertsch, Karol Hausman, Keegan Go, Keerthana Gopalakrishnan, Kenneth Y. Goldberg, Kendra Byrne, Kenneth Oslund, Kento Kawaharazuka, Kevin Black, Kevin Zhang 0002, Kiana Ehsani, Kiran Lekkala, Kirsty Ellis, Krishan Rana, Krishnan Srinivasan, Kuan Fang, Kunal Pratap Singh, Kuo-Hao Zeng, Kyle Hatch, Kyle Hsu, Laurent Itti, Yunliang Chen 0001, Lerrel Pinto, Li Fei-Fei 0001, Liam Tan, Linxi Fan, Lionel Ott, Lisa Lee, Luca Weihs, Magnum Chen, Marion Lepert, Marius Memmel, Masayoshi Tomizuka, Masha Itkina, Mateo Guaman Castro, Max Spero, Maximilian Du, Michael Ahn, Michael C. Yip, Mingtong Zhang 0003, Mingyu Ding, Minho Heo, Mohan Kumar Srirama, Mohit Sharma 0001, Moo Jin Kim, Naoaki Kanazawa, Nicklas Hansen 0001, Nicolas Heess, Nikhil J. Joshi, Niko Sünderhauf, Norman Di Palo, Nur Muhammad Shafiullah, Oier Mees, Oliver Kroemer, Osbert Bastani, Pannag R. Sanketi, Patrick Tree Miller, Patrick Yin, Paul Wohlhart, Peng Xu 0010, Peter David Fagan, Peter Mitrano, Pierre Sermanet, Pieter Abbeel, Priya Sundaresan, Qiuyu Chen, Rafael Rafailov, Ria Doshi, Roberto Martin Martin, Rohan Baijal, Rosario Scalise, Rose Hendrix, Roy Lin, Runjia Qian, Russell Mendonca, Rutav Shah, Ryan Hoque, Ryan Julian, Samuel Bustamante-Gomez, Sean Kirmani, Sergey Levine, Sherry Moore, Shikhar Bahl, Shivin Dass, Shubham D. Sonawani, Shuran Song, Sichun Xu, Siddhant Haldar, Siddharth Karamcheti, Simeon Adebola, Simon Guist, Soroush Nasiriany, Stefan Schaal, Stefan Welker, Stephen Tian, Subramanian Ramamoorthy, Sudeep Dasari, Suneel Belkhale, Sungjae Park, Suraj Nair 0003, Suvir Mirchandani, Takayuki Osa, Tanmay Gupta, Tatsuya Harada, Tatsuya Matsushima, Ted Xiao, Thomas Kollar, Tianhe Yu, Tianli Ding, Todor Davchev, Tony Z. Zhao, Travis Armstrong, Trevor Darrell, Trinity Chung, Vidhi Jain, Vincent Vanhoucke, Wolfram Burgard, Xiaolong Wang 0004, Xinghao Zhu, Xinyang Geng, Liangwei Xu, Yecheng Jason Ma 0001, Yejin Kim 0003, Yevgen Chebotar, Yilin Wu 0003, Yonatan Bisk, Yoonyoung Cho, Youngwoon Lee, Yuchen Cui, Yueh-Hua Wu, Yujin Tang, Yuke Zhu, Yunchu Zhang, Yunfan Jiang 0001, Yunshuang Li, Yunzhu Li, Yusuke Iwasawa, Yutaka Matsuo, Zehan Ma, Zichen Jeff Cui, Zichen Zhang 0016, Zipeng Lin |
ICRA | 225 |
| 2021 | Greedy Hierarchical Variational Autoencoders for Large-Scale Video PredictionabstractA video prediction model that generalizes to diverse scenes would enable intelligent agents such as robots to perform a variety of tasks via planning with the model. However, while existing video prediction models have produced promising results on small datasets, they suffer from severe underfitting when trained on large and diverse datasets. To address this underfitting challenge, we first observe that the ability to train larger video prediction models is often bottlenecked by the memory constraints of GPUs or TPUs. In parallel, deep hierarchical latent variable models can produce higher quality predictions by capturing the multi-level stochasticity of future observations, but end-to-end optimization of such models is notably difficult. Our key insight is that greedy and modular optimization of hierarchical autoencoders can simultaneously address both the memory constraints and the optimization challenges of large-scale video prediction. We introduce Greedy Hierarchical Variational Autoencoders (GHVAEs), a method that learns highfidelity video predictions by greedily training each level of a hierarchical autoencoder. In comparison to state- of-the-art models, GHVAEs provide 17-55% gains in prediction performance on four video datasets, a 35–40% higher success rate on real robot tasks, and can improve performance monotonically by simply adding more modules. Visualization and more details are at https://sites.google.com/view/ghvae. Bohan Wu, Suraj Nair 0003, Roberto Martin Martin, Li Fei-Fei 0001, Chelsea Finn |
CVPR | 2 |
| 2021 | Model-Based Visual Planning with Self-Supervised Functional Distances
Stephen Tian, Suraj Nair 0003, Frederik Ebert, Sudeep Dasari, Benjamin Eysenbach, Chelsea Finn, Sergey Levine |
ICLR | 2 |
| 2020 | Hierarchical Foresight: Self-Supervised Learning of Long-Horizon Tasks via Visual Subgoal Generation
Suraj Nair 0003, Chelsea Finn |
ICLR | 1 |
| 2020 | Goal-Aware Prediction: Learning to Model What MattersabstractLearned dynamics models combined with both planning and policy learning algorithms have shown promise in enabling artificial agents to learn to perform many diverse tasks with limited supervision. However, one of the fundamental challenges in using a learned forward dynamics model is the mismatch between the objective of the learned model (future state reconstruction), and that of the downstream planner or policy (completing a specified task). This issue is exacerbated by vision-based control tasks in diverse real-world environments, where the complexity of the real world dwarfs model capacity. In this paper, we propose to direct prediction towards task relevant information, enabling the model to be aware of the current task and encouraging it to only model relevant quantities of the state space, resulting in a learning objective that more closely matches the downstream task. Further, we do so in an entirely self-supervised manner, without the need for a reward function or image labels. We find that our method more effectively models the relevant parts of the scene conditioned on the goal, and as a result outperforms standard task-agnostic dynamics models and model-free reinforcement learning. Suraj Nair 0003, Silvio Savarese, Chelsea Finn |
ICML | 1 |
| 2020 | TRASS: Time Reversal as Self-SupervisionabstractA longstanding challenge in robot learning for manipulation tasks has been the ability to generalize to varying initial conditions, diverse objects, and changing objectives. Learning based approaches have shown promise in producing robust policies, but require heavy supervision and large number of environment interactions, especially from visual inputs. We propose a novel self-supervision technique that uses time-reversal to provide high level supervision to reach goals. In particular, we introduce the time-reversal model (TRM), a self-supervised model which explores outward from a set of goal states and learns to predict these trajectories in reverse. This provides a high level plan towards goals, allowing us to learn complex manipulation tasks with no demonstrations or exploration at test time. We test our method on the domain of assembly, specifically the mating of tetris-style block pairs. Using our method operating atop visual model predictive control, we are able to assemble tetris blocks on a KuKa IIWA-7 using only uncalibrated RGB camera input, and generalize to unseen block pairs. Project's-page: https://sites.google.com/view/time-reversal. Suraj Nair 0003, Mohammad Babaeizadeh, Chelsea Finn, Sergey Levine |
ICRA | 1 |
| 2019 | Neural Task Graphs: Generalizing to Unseen Tasks From a Single Video DemonstrationabstractOur goal is to generate a policy to complete an unseen task given just a single video demonstration of the task in a given domain. We hypothesize that to successfully generalize to unseen complex tasks from a single video demonstration, it is necessary to explicitly incorporate the compositional structure of the tasks into the model. To this end, we propose Neural Task Graph (NTG) Networks, which use conjugate task graph as the intermediate representation to modularize both the video demonstration and the derived policy. We empirically show NTG achieves inter-task generalization on two complex tasks: Block Stacking in BulletPhysics and Object Collection in AI2-THOR. NTG improves data efficiency with visual input as well as achieve strong generalization without the need for dense hierarchical supervision. We further show that similar performance trends hold when applied to real-world data. We show that NTG can effectively predict task structure on the JIGSAWS surgical dataset and generalize to unseen tasks. De-An Huang, Suraj Nair 0003, Danfei Xu, Yuke Zhu, Animesh Garg, Li Fei-Fei 0001, Silvio Savarese, Juan Carlos Niebles |
CVPR | 2 |
| 2018 | Neural Task Programming: Learning to Generalize Across Hierarchical TasksabstractIn this work, we propose a novel robot learning framework called Neural Task Programming (NTP), which bridges the idea of few-shot learning from demonstration and neural program induction. NTP takes as input a task specification (e.g., video demonstration of a task) and recursively decomposes it into finer sub-task specifications. These specifications are fed to a hierarchical neural program, where bottom-level programs are callable subroutines that interact with the environment. We validate our method in three robot manipulation tasks. NTP achieves strong generalization across sequential tasks that exhibit hierarchal and compositional structures. The experimental results show that NTP learns to generalize well towards unseen tasks with increasing lengths, variable topologies, and changing objectives.stanfordvl.github.io/ntp/. Danfei Xu, Suraj Nair 0003, Yuke Zhu, Julian Gao, Animesh Garg, Li Fei-Fei 0001, Silvio Savarese |
ICRA | 2 |