Recent advances in code generation and reasoning have enabled the growing use of large language models (LLMs) in robotics. The design of traditional robotic feedback controllers often requires extensive expert knowledge and iterative computational effort, motivating the exploration of LLM-assisted controller development. However, existing efforts are largely restricted to overly simplified systems, provide limited comparison with human-designed controllers, and lack validation on real-world robotic platforms. To address these gaps, this work investigates the use of LLMs in robotic feedback control by focusing on adaptive control and proposing an LLM-guided adaptive compensator. Instead of generating a complete controller from scratch, the LLM is guided to design a compensator that modulates the response of an unknown system to match that of a predefined reference system, thereby achieving adaptivity. The proposed approach is evaluated using five adaptive controllers on a two-degree-of-freedom (DoF) soft robot and a one-DoF shoulder joint of a humanoid robot. Both simulation and real-world experiments demonstrate that the LLM-guided adaptive compensator achieves performance comparable to four conventional adaptive controllers. Lyapunov-based analysis further establishes the stability and generalization capability of the proposed compensator, while analysis of the reasoning process suggests a more structured design approach. This study introduces a novel framework for integrating LLMs into robotic feedback control, with promising practical applicability.

Rearrangement operations form the crux of interactions between humans and their environment. The ability to generate natural, fluid sequences of this operation is of essential value in AR/VR and CG. Bridging a gap in the field, our study introduces FAVOR: a novel dataset for Full-body AR-driven Virtual Object Rearrangement that uniquely employs motion capture systems and AR eyeglasses. Comprising 3k diverse motion rearrangement sequences and 7.17 million interaction data frames, this dataset breaks new ground in research data. We also present a pipeline FAVORITE for producing digital human rearrangement motion sequences guided by instructions. Experimental results, both qualitative and quantitative, suggest that this dataset and pipeline deliver high-quality motion sequences. Our dataset, code, and appendix are available at https://kailinli.github.io/FAVOR.

We present Oakink2, a dataset of bimanual object manipulation tasks for complex daily activities. In pursuit of constructing the complex tasks into a structured representation, Oakink2 introduces three level of abstraction to organize the manipulation tasks: Affordance, Primitive Task, and Complex Task. OAKINK2 features on an object-centric perspective for decoding the complex tasks, treating them as a sequence of object affordance fulfillment. The first level, Affordance, outlines the functionalities that objects in the scene can afford, the second level, Primitive Task, describes the minimal interaction units that humans interact with the object to achieve its affordance, and the third level, Complex Task, illustrates how Primitive Tasks are composed and interdependent. Oakink2 dataset provides multi-view image streams and precise pose annotations for the human body, hands and various interacting objects. This extensive collection supports applications such as interaction reconstruction and motion synthesis. Based on the 3-level abstraction of Oakink2, we explore a task-oriented framework for Complex Task Completion (CTC). CTC aims to generate a sequence of bimanual manipulation to achieve task objectives. Within the CTC framework, we employ Large Language Models (LLMs) to decompose the complex task objectives into sequences of Primitive Tasks and have developed a Motion Fulfillment Model that generates bimanual hand motion for each Primitive Task. Oakink2 datasets and models are available at https://oakink.net/v2.