Despite transformative advances in generative motion synthesis, real-time interactive motion control remains dominated by traditional techniques. In this work, we identify two key challenges in bridging research and production: 1) Real-time scalability : Industry applications demand real-time generation of a vast repertoire of motion skills, while generative methods exhibit significant degradation in quality and scalability under real-time computation constraints, and 2) Integration : Industry applications demand fine-grained multi-modal control involving velocity commands, style selection, and precise keyframes, a need largely unmet by existing text- or tag-driven models. Moreover, a systematic motion design interface for generative models remains absent. To overcome these limitations, we introduce MotionBricks: a large-scale, real-time generative framework with a two-fold solution. First, we propose a large-scale modular latent generative backbone tailored for robust real-time motion generation, effectively modeling a dataset of over 350,000 motion clips with a single model. Second, we introduce smart primitives that provide a unified, robust, and intuitive interface for authoring both navigation and object interaction. Notably, MotionBricks applies to new downstream tasks in a zero-shot manner, where no fine-tuning or task-specific tagging is required. Applications can be designed in a plug-and-play manner like assembling bricks without expert animation knowledge, enabling an accessible interface for applications in animation and robotics. Quantitatively, we show that MotionBricks produces state-of-the-art motion quality on open-source and proprietary datasets of various scales, while also achieving a real-time throughput of 15,000 FPS with 2ms latency. We demonstrate the flexibility and robustness of MotionBricks in a complete production-level animation demo, covering navigation and object-scene interaction across various styles with a unified model. To showcase our framework's application beyond animation, we deploy MotionBricks on the Unitree G1 humanoid robot to demonstrate its flexibility and generalization for real-time robotic control.

Generating realistic 3D human motions in real-time within interactive applications is key for animation, simulation, and humanoid robotics. While recent offline motion generation approaches offer precise control via text and kinematic constraints, they lack the inference speed required for interactive settings. Conversely, existing online methods enable real-time synthesis but often sacrifice controllability or struggle with complex text semantics and long-horizon goals due to limited context windows. In this work, we introduce ARDY, a streaming generation framework that bridges this gap by enabling high-fidelity motion generation controllable via online text prompts and flexible kinematic constraints. ARDY employs a hybrid representation that combines explicit root features with a latent body embedding, balancing precise trajectory control with efficient generative learning. We propose a two-stage autoregressive transformer denoiser that features variable history context and supports conditioning on flexible, long-horizon kinematic constraints. By training on a large-scale motion capture dataset and being directly conditioned on text labels and kinematic constraints sampled from ground truth poses, ARDY natively learns controllable generation that supports online prompting and flexible long-horizon goals. Extensive evaluations on the HumanML3D benchmark and the large-scale, high-fidelity Bones Rigplay dataset demonstrate ARDY's high motion quality and constraint adherence, validating the efficacy of our key architectural decisions. Finally, we demonstrate the method's practical versatility through an interactive demo featuring dynamic text control, diverse keyframe pose constraints, path following, and interactive locomotion control via mouse and keyboard. Supplementary video results, code, and model releases can be found at https://research.nvidia.com/labs/sil/projects/ardy/.

Human motion synthesis is an important problem with applications in graphics, gaming and simulation environments for robotics. Existing methods require accurate motion capture data for training, which is costly to obtain. Instead, we propose a framework for training generative models of physically plausible human motion directly from monocular RGB videos, which are much more widely available. At the core of our method is a novel optimization formulation that corrects imperfect image-based pose estimations by enforcing physics constraints and reasons about contacts in a differentiable way. This optimization yields corrected 3D poses and motions, as well as their corresponding contact forces. Results show that our physically-corrected motions significantly outperform prior work on pose estimation. We can then use these to train a generative model to synthesize future motion. We demonstrate both qualitatively and quantitatively significantly improved motion estimation, synthesis quality and physical plausibility achieved by our method on the large scale Human3.6m dataset [12] as compared to prior kinematic and physics-based methods. By enabling learning of motion synthesis from video, our method paves the way for large-scale, realistic and diverse motion synthesis.

In this paper, we are interested in modeling complex activities that occur in a typical household. We propose to use programs, i.e., sequences of atomic actions and interactions, as a high level representation of complex tasks. Programs are interesting because they provide a non-ambiguous representation of a task, and allow agents to execute them. However, nowadays, there is no database providing this type of information. Towards this goal, we first crowd-source programs for a variety of activities that happen in people's homes, via a game-like interface used for teaching kids how to code. Using the collected dataset, we show how we can learn to extract programs directly from natural language descriptions or from videos. We then implement the most common atomic (inter)actions in the Unity3D game engine, and use our programs to "drive" an artificial agent to execute tasks in a simulated household environment. Our VirtualHome simulator allows us to create a large activity video dataset with rich ground-truth, enabling training and testing of video understanding models. We further showcase examples of our agent performing tasks in our VirtualHome based on language descriptions.

The recently announced Super Wi-Fi Network proposal in United States is aiming to enable Internet access in a nation-wide area. As traditional cable-connected power supply system becomes impractical or costly for a wide range wireless network, new infrastructure deployment for Super Wi-Fi is required. The fast developing Energy Harvesting (EH) techniques receive global attentions for their potential of solving the above power supply problem. It is a critical issue, from the user's perspective, how to make efficient network selection and access strategies. Unlike traditional wireless networks, the battery charge state and tendency in EH based networks have to be taken into account when making network selection and access, which has not been well investigated. In this paper, we propose a practical and efficient framework for multiple base stations access strategy in an EH powered Super Wi-Fi network. We consider the access strategy from the user's perspective, who exploits downlink transmission opportunities from one base station. To formulate the problem, we used Partially Observable Markov Decision Process (POMDP) to model users' observations on the base stations' battery situation and decisions on the base station selection and access. Simulation results show that our methods are efficacious and significantly outperform the traditional widely used CSMA method.

Super Wi-Fi is expected to enable the Internet access everywhere in a country. Considering the infrastructure deployment issues, energy harvesting technology is a promising solution for power supply. Most of existing works focused on the energy scheduling from the network operator's point of view. In this paper, we study the access point selection strategy from users' perspectives and consider Super WiFi networks powered by solar energy harvesting. Although the network selection problem has been studied, the energy harvesting scenario has not been well investigated and the influence of battery condition has not been taken into account. In our work, we consider the utility of the access users is affected not only by the total number of accessed users, but also the battery condition. In order to formulate the battery states and user states, we incorporate the physical characteristics of solar cell, as well as the dynamic access behaviors of users through Markov Decision Process (MDP)model. By using the value iteration method, the set of optimal network selection strategies is obtained. Simulation results validate that our method has a remarkable performance improvement of utility over the myopic and random access strategies.