Visual odometry (VO) is a critical component of autonomous robot systems, enabling precise pose estimation from visual inputs. Learning-based VO methods are increasingly recognized for their robustness in challenging scenarios, including dynamic environments, motion blur, and low-light conditions. However, their performance is constrained by both the diversity of the data and its utilization rate. To overcome these limitations, we propose an end-to-end monocular VO system incorporating a novel learning-based end-to-end VO framework and multiple analogy augmentation strategies. We introduce the Context Attention Uncertainty-aware VO Network (CUVO), which prioritizes semantically rich regions and mitigating interference from high-uncertainty areas to enhance attentional focus and pose estimation accuracy. Furthermore, our analogy augmentation methods—temporal reversal, random rotation, and geometric mirroring—enhance image pairs and compute corresponding true pose transformations, significantly increasing training data quantity and diversity. Simultaneously, an analogous loss is applied to ensure consistency between the original and augmented data. Extensive experiments demonstrate that CUVO significantly enhances VO performance, outperforming previous end-to-end VO methods on TartanAir and KITTI datasets. By leveraging analogy augmentation strategy to expand training data under limited data conditions (27k), zero-shot capability of CUVO degrades by up to 29.5% on TartanAir and 23.3% on KITTI. Our work introduces the first image-to-pose data augmentation method tailored for VO and establishes CUVO as a robust system for advancing learning-based visual odometry.

Sketching is a quick ideation and multimedia tool for effectively expressing design intent. By translating simple strokes into CAD models, it allows non-expert users to create editable designs, reducing the learning curve associated with traditional CAD software. However, current sketch-based CAD modeling methods are often limited to basic shapes and require structured inputs, making them less robust when dealing with varied sketch styles. To overcome these challenges, we propose a novel sketch-based modeling framework DAFU-CAD, that is both efficient and robust. Our approach features a Depth-Assisted and Feature-Unraveling sketch classification module that categorizes sketches into corresponding modeling operations, independent of their drawing style. A parameter regression and optimization module then estimates the modeling parameters, ensuring consistent and stable model reconstruction across different sketch inputs. To support this, we compile a diverse sketch dataset with a range of modeling categories and abstraction levels. Experimental results show that our method outperforms existing approaches in terms of both robustness and versatility.

Current sketch modeling methods face challenges in terms of efficiency and quality due to frequent view changes. To empower users to create models seamlessly and accurately from a single reference drawing, we present WidgetSketcher, a single-view, sketch-based modeling method that incorporates widget manipulations. To ensure seamlessness, we implement a single-view workflow in sketch-based modeling with designed widget manipulations and other interactions. Each user stroke is equipped with a transient axis widget, allowing 3D pose specification and part creation from a single view. To improve accuracy, we develop a two-stage shape generation algorithm, ensuring the precise generation of shapes with detailed features and asymmetry. Our method enables the quick and accurate creation of 3D models consistent with reference drawings, while eliminating frequent view changes. To demonstrate the efficiency of our method, we evaluate the system usability by conducting a user study and compare the results to state-of-the-art techniques.

Recovering a user-special and controllable human model from a single RGB image is a nontrivial challenge. Existing methods usually generate static results with an image consistent subject's pose. Our work aspires to achieve pose-controllable human reconstruction from a single image by learning a dynamic (multi-pose) implicit field. We first construct a feature-embedded human model (FEHM) as a bridge to propagate image features to different pose spaces. Based on FEHM, we then encode three pose-decoupled features. Global image features represent user-specific shapes in images and replace widely used pixel-aligned ways to avoid unwanted shape-pose entanglement. Spatial color features propagate FEHM-embedded image cues into 3D pose space to provide spatial high-frequency guidance. Spatial geometry features improve reconstruction robustness by using the surface shape of the FEHM as the prior. Finally, new implicit functions are designed to predict the dynamic human implicit fields. For effective supervision, a realistic human avatar dataset, SimuSCAN, with 1000+ models is constructed using a low-cost hierarchical mesh registration method. Extensive experiments demonstrate that our method achieves the state-of-the-art reconstruction level.

Single-view clothing reconstruction usually relies on topologically fixed clothing templates to reduce the problem complexity, but this strategy also makes the reconstructed clothing shape contours simple and lack diversity. In this article, we propose a novel clothing reconstruction method to generate complex shape contours and open clothing mesh from a single image. At the heart of our work is an implicit unsigned distance field condition on clothing-oriented and pose-stable spatial shape features to represent the clothing from the image. This feature can provide spatially aligned clothing shape priors to improve the pose robustness. It is based on a type-generic clothing template derived from the mainstream clothing generative model to avoid tedious template design and switching. To output open clothing mesh results from noisy clothing unsigned distance fields, we develop a two-stage clothing mesh extraction method. It takes the point clouds as an intermediate representation and produces smooth, plausible and editable clothing mesh results. To provide effective supervision, we construct a pose-rich and shape-complete clothing scan dataset by enhancing clothing pose diversity and complementing missing clothing geometry caused by occlusion. Extensive experiments demonstrate that our method achieves state-of-the-art levels. More importantly, we provide a simple but effective, and low-cost way to reconstruct complex shape contours clothing from a single image.

Sketch-based modeling studies reconstructing models from sketches automatically, allowing users visualize design concepts rapidly. Generating CAD models based on user sketches helps reduce the learning curve for novice users, which promotes the everyday use of CAD software, and expands its reach to non-professional groups. While various algorithms study automatically generating models from single sketch or line drawing, they often produce non-editable models or editable models limited to simple extrusion operations. To improve this issue, we propose a novel sketch-based modeling system, Sketch2Seq, which generates complex, semantic, and editable CAD models. Our system eliminates the need for additional annotations from users and produces models that support subsequent application in commercial software. The core of our method lies in understanding users' design intent from CAD sketches. We design a novel sketch segmentation network for identifying diverse operation features in CAD sketches, which utilizes geometric features of strokes and different levels of topological connections. Additionally, to tackle the segmentation task, a dataset for CAD sketch segmentation is introduced. Comparative experiments and ablation evaluations prove the effectiveness of the proposed method. Based on segmentation result, coarse CAD sequences are generated and progressively executed. Meanwhile, the orders and parameters of the CAD sequences are optimized with context models and input sketches. All algorithms are integrated into a user interface. Experiments and evaluations validate the feasibility and superiority of our entire system which is able to reconstruct more complex features and achieve better results for longer sequence.

Collaboration with robots through physical contact offers a more intuitive, natural, and engaging operational experience, showcasing vast potential in the field of human-robot interaction. However, current physical interaction devices, such as collaborative robots and haptic feedback mechanisms, are limited by their singular modes of motion and feedback, hindering enhancements in interaction experiences. Herein, we present a multistable soft actuator capable of driving multimodal shape changes and passively conforming to user touch. This actuator can memorize and maintains any deformation with zero power consumption. Its structural mechanical properties can be dynamically adjusted to produce rich haptic feedback for the user, including changes in shape, elasticity, stiffness, and even sensations of rupture and weightlessness. Structurally, the mechanism consists of a network of pneumatic bistable units in series and parallel configurations, which can switch states under air pressure or external force, achieving extension, contraction, and omnidirectional bending. The input of air pressure can either impede or assist deformation, altering structural stiffness and resulting in varied loading curves. With its high safety in physical interactions, robust operability, and rich mechanical tactile feedback, the multistable soft actuator promises new design directions for physical human-robot interaction devices.

We propose a robust and highly realistic clothing modeling method to generate a 3D clothing model with visually consistent clothing style and wrinkles distribution from a single RGB image. Notably, this entire process only takes a few seconds. Our high-quality clothing results benefit from the idea of combining learning and optimization, making it highly robust. First, we use the neural networks to predict the normal map, a clothing mask, and a learning-based clothing model from input images. The predicted normal map can effectively capture high-frequency clothing deformation from image observations. Then, by introducing a normal-guided clothing fitting optimization, the normal maps are used to guide the clothing model to generate realistic wrinkles details. Finally, we utilize a clothing collar adjustment strategy to stylize clothing results using predicted clothing masks. An extended multi-view version of the clothing fitting is naturally developed, which can further improve the realism of the clothing without tedious effort. Extensive experiments have proven that our method achieves state-of-the-art clothing geometric accuracy and visual realism. More importantly, it is highly adaptable and robust to in-the-wild images. Further, our method can be easily extended to multi-view inputs to improve realism. In summary, our method can provide a low-cost and user-friendly solution to achieve realistic clothing modeling.

In many robot operation scenarios, the end-effector’s attitude constraints of movement are indispensable for the task process, such as robotic welding, spraying, handling, and stacking. Meanwhile, the inverse kinematics, collision detection, and space search are involved in the path planning procedure under attitude constraints, making it difficult to achieve satisfactory efficiency and effectiveness in practice. To address these problems, we propose a distributed variable density path planning method with attitude constraints (DVDP-AC) for industrial robots. First, a position–attitude constraints reconstruction (PACR) approach is proposed in the inverse kinematic solution. Then, the distributed signed-distance-field (DSDF) model with single-step safety sphere (SSS) is designed to improve the efficiency of collision detection. Based on this, the variable density path search method is adopted in the Cartesian space. Furthermore, a novel forward sequential path simplification (FSPS) approach is proposed to adaptively eliminate redundant path points considering path accessibility. Finally, experimental results verify the performance and effectiveness of the proposed DVDP-AC method under end-effector’s attitude constraints, and its characteristics and advantages are demonstrated by comparison with current mainstream path planning methods.

Abstract We present an efficient and stable framework, called Unified Intersection Resolver (UIR), for cloth simulation systems where not only impending collisions but also pre‐existing penetrations often arise. These two types of collisions are handled in a unified manner, by detecting edge‐face intersections first and then forming penetration stencils to be resolved iteratively. A stencil is a quadruple of vertices and it reveals either a vertex‐face or an edge‐edge collision event happened. Each quadruple also implicitly defines a collision normal, through which the four stencil vertices can be relocated, so that the corresponding edge‐face intersection disappear. We deduce three different ways, i.e., from predefined surface orientation, from history data and from global intersection analysis, to determine the collision normals of these stencils robustly. Multiple stencils that constitute a penetration region are processed simultaneously to eliminate penetrations. Cloth trapped in pinched environmental objects can be handled easily within our framework. We highlight its robustness by a number of challenging experiments involving collisions.

Abstract The cloth simulation systems often suffer from excessive extension on the polygonal mesh, so an additional strain‐limiting process is typically used as a remedy in the simulation pipeline. A cloth model can be discretized as either a quadrilateral mesh or a triangular mesh, and their strains are measured differently. The edge‐based strain‐limiting method for a quadrilateral mesh creates anisotropic behaviour by nature, as discretization usually aligns the edges along the warp and weft directions. We improve this anisotropic technique by replacing the traditionally used equality constraints with inequality ones in the mathematical optimization, and achieve faster convergence. For a triangular mesh, the state‐of‐the‐art technique measures and constrains the strains along the two principal (and constantly changing) directions in a triangle, resulting in an isotropic behaviour which prohibits shearing. Based on the framework of inequality‐constrained optimization, we propose a warp and weft strain‐limiting formulation. This anisotropic model is more appropriate for textile materials that do not exhibit isotropic strain behaviour.

Abstract Three‐dimensional human model reconstruction has wide applications due to the rapid development of computer vision. The appearance of cheap depth camera, such as Kinect, opens up new horizons for home‐oriented 3D human reconstructions. However, the resolution of Kinect is relatively low, making it difficult to build accurate human models. In this paper, we improve the accuracy of human model reconstruction from two aspects. First, we improve the depth data quality by registering the depth images captured from multi‐views with a single Kinect. The part‐wise registration method and implicit‐surface‐based de‐noising method are proposed. Second, we utilize a statistical human model to iteratively augment and complete the human body information by fitting the statistical human model to the registered depth image. Experimental results and several applications demonstrate the applicability and quality of our system, which can be potentially used in virtual try‐on systems. Copyright © 2015 John Wiley & Sons, Ltd.

We present a new method for resolving some of the most frequently occurring penetration configurations in cloth simulation, with a history-free discrete collision detection (DCD). To be adapted to a wide range of applications, this method is also orientation-free as no inside/outside surface setting is assumed. Our method relies on a heuristic paradigm "small region is illegal" to identify penetrating (illegal) regions whenever possible. First, intersection contours are constructed and classified, followed by a global analysis of the collision configurations. Then for those surfaces having identifiable illegal regions, we use dynamic repulsive normal (DRN) to compute proper displacements to relocate incorrectly configured vertices to be intersection-free. For those configurations that do not clearly define legal/illegal regions, displacements are designed to push one mesh to the boundary of the other. The proposed method can also be used in the context of time-dependent simulation of complex deformable surfaces, making it an competitive alternative to the popular CCD-based approach.

In this paper we present an approach for animating a virtual hand model with the animation data extracting from hand tracking results. We track the hand by using particle filters and deformable contour templates with a Web camera; then we extend the 2D tracking results into 3D animation data using inverse kinematics method; finally, local frame based method is proposed to simulate a 3D virtual hand with the 3D animation data. Our system performs in real time.