The rapid growth of e-commerce has intensified the demand for Virtual Try-On (VTO) technologies, enabling customers to realistically visualize products overlaid on their own images. Despite recent advances, existing VTO models face challenges with fine-grained detail preservation, robustness to real-world imagery, efficient sampling, image editing capabilities, and generalization across diverse product categories. In this paper, we present DiT-VTON, a novel VTO framework that leverages an architecture based on a Diffusion Transformer (DiT), renowned for its performance on text-conditioned image generation (text-to-image), adapted here for the image-conditioned VTO task. We systematically explore multiple DiT configurations, including in-context token concatenation, channel concatenation, and ControlNet integration, to determine the best setup for VTO image conditioning. To enhance robustness, we train the model on an expanded dataset encompassing varied backgrounds, unstructured references, and non-garment categories, demonstrating the benefits of data scaling for VTO adaptability. DiT-VTON also redefines the VTO task beyond garment try-on, offering a versatile Virtual Try-All (VTA) solution capable of handling a wide range of product categories and supporting advanced image editing functionalities, such as pose preservation, precise localized region editing and refinement, texture transfer and object-level customization. Experimental results show that our model surpasses state-of-the-art methods on public benchmark tests like VITON-HD, achieving superior detail preservation and robustness without reliance on additional image condition encoders. It also surpasses state-of- the-art models that have VTA and image editing capabilities on a varied dataset composed of thousands of product categories. As a result, DiT-VTON significantly advances VTO applicability in diverse real-world scenarios, enhancing both the realism and personalization of online shopping experiences.

For e-commerce retailers, high-quality product catalogs are vital to customer experience. Yet, despite lots of data cleaning efforts, catalog quality, especially in large catalogs, remains suboptimal. This paper shows how to use unstructured brand knowledge base data as a reference and a large language model agent to automatically enhance an e-commerce retailer's catalog quality. Unlike prior methods that usually repair and match product entries separately, our method does both concurrently. Our evaluation results show its effectiveness.

To give customers good experience, an e-commerce retailer needs high-quality product information in its catalog. Yet, the raw product information often lacks sufficient quality. For a large catalog that can contain billions of products, manually fixing this information is highly labor-intensive. To address this issue, we propose using the tool use functionality of large language models to automatically improve product information. In this talk, we show why existing data cleaning methods are not well suited for this task and how we designed our automated system to improve product information. When evaluated on a random sample of products from an e-commerce catalog, our system improved product information completeness by 78% with no major drop in information accuracy.

We propose a modular BiLSTM / CNN / Transformer deep-learning encoder architecture, together with a data synthesis and training approach, to solve the problem of matching catalog products across different languages, different local catalogs, and different catalog data contributors. The end-to-end model relies solely on raw natural language textual data in the catalog entries and on images of the products, without any feature engineering, and is entirely translation-free, not requiring the translation of the catalog natural-language data to the same base language for inference. We report experiments results on a 4-languages-scope model (English, French, German, Spanish) matching entities from 4 local catalogs (UK, France, Germany, Spain) of a retail website. We demonstrate that the model achieves performance comparable to state-of-the-art existing entity matchers that operate within a single language, and that the model achieves high-performance zero-shot inference on language pairs not seen in training.

We have extended the task-space multiobjective controllers that write as quadratic programs (QPs) to handle multirobot systems as a single centralized control. The idea is to assemble all the “robots” models and their interaction task constraints into a single QP formulation. By multirobot, we mean that whatever entities a given robot will interact with (solid or articulated systems, actuated, partially or not at all, fixed-base or floating-base), we model them as clusters of robots and the controller computes the state of each cluster as an overall system and their interaction forces in a physically consistent way. By doing this, the tasks specification simplifies substantially. At the heart of the interactions between the systems are the contact forces; methodologies are provided to achieve reliable force tracking by our multirobot QP controller. The approach is assessed by a large panel of experiments on real complex robotic platforms (full-size humanoid, dexterous robotic hand, fixed-base anthropomorphic arm) performing whole-body manipulations, dexterous manipulations, and robot-robot comanipulations of rigid floating objects and articulated mechanisms, such as doors, drawers, boxes, or even smaller mechanisms like a spring-loaded click pen.

Human-humanoid collaborative tasks require that the robot take into account the goals of the task, interaction forces with the human, and its own balance. We present a formulation for a real-time humanoid controller which allows the robot to keep itself balanced, while also assisting the human in achieving their shared objectives. We achieve this with a multi-robot quadratic program controller, which solves for human dynamics reconstruction and optimal robot controls in a single optimization problem. Our experiments on a simulated robot platform demonstrate the ability to generate interaction motions and forces that are similar to what a human collaborator would produce.

We address the problem of humanoid falling with a decoupled strategy consisting of a pre-impact and a postimpact stage. In the pre-impact stage, geometrical reasoning allows the robot to choose appropriate impact points in the surrounding environment and to adopt a posture to reach them while avoiding impact-singularities and preparing for the postimpact. The surrounding environment can be unstructured and may contain cluttered obstacles. The post-impact stage uses a quadratic program controller that adapts on-line the joint proportional-derivative (PD) gains to make the robot compliant-to absorb impact and post-impact dynamics, which lowers possible damage risks. This is done by a new approach incorporating the stiffness and damping gains directly as decision variables in the QP along with the usually-considered variables of joint accelerations and contact forces. Constraints of the QP prevent the motors from reaching their torque limits during the fall. Several experiments on the humanoid robot HRP-4 in a full-dynamics simulator are presented and discussed.

We extend the quadratic program (QP)-based task-space character control approach-initially intended for individual character animation-to multiple characters interacting among each other or with mobile/articulated elements of the environment. The interactions between the characters can be either physical interactions, such as contacts that can be established or broken at will between them and for which the forces are subjected to Newton's third law, or behavioral interactions, such as collision avoidance and cooperation that naturally emerge to achieve collaborative tasks from high-level specifications. We take a systematic approach integrating all the equations of motions of the characters, objects, and articulated environment parts in a single QP formulation in order to embrace and solve the most general instance of the problem, where independent individual character controllers would fail to account for the inherent coupling of their respective motions through those physical and behavioral interactions. Various types of motions/behaviors are controlled with only the one single formulation that we propose, and some examples of the original motions the framework allows are presented in the accompanying video.

We propose a generalized framework together with an algorithm to plan a discrete sequence of multi-contact stances that brings a set of collaborating robots and manipulated objects from a specified initial configuration to a desired goal through non-gaited acyclic contacts with their environment or among each other. The broad range of applications of this generic algorithm includes legged locomotion planning, whole-body manipulation planning, dexterous manipulation planning, as well as any multi-contact-based motion planning problem that might combine several of these sub-problems. We demonstrate the versatility of our planner through example scenarios taken from the aforementioned classes of problems in virtual environments.

Our objective in this work is to synthesize dynamically consistent motion for a simulated humanoid robot in acyclic multi-contact locomotion using multi-objective control. We take as an input a planned sequence of static postures that represent the contact configuration transitions; a multi-objective controller then synthesizes the motion between these postures, the objectives of the controller being decided by a finite-state machine. Results of this approach are presented in the attached video in the form of playback motions generated through non-real-time constraint-based dynamic simulations.

We present a motion planning algorithm that computes rough trajectories used by a contact-points planner as a guide to grow its search graph. We adapt collision-free motion planning algorithms to plan a path within the guide space, a submanifold of the configuration space included in the free space in which the configurations are subject to static stability constraint. We first discuss the definition of the guide space. Then we detail the different techniques and ideas involved: relevant C-space sampling for humanoid robot, task-driven projection process, static stability test based on polyhedral convex cones theory's double description method. We finally present results from our implementation of the algorithm.