Articulated object modeling, which represents interconnected rigid bodies with their geometry, part segmentation, articulation tree, and physical properties, is crucial for robotic perception and manipulation. Recently existing methods like SAGCI leverage Interactive Perception (IP) to refine models through robot interaction. However, SAGCI suffers from prior-dependency (requiring initialization), neglects kinematic/dynamic constraints, and generates non-watertight meshes. To overcome these limitations, we propose SIAM, a novel framework for efficient and generalizable Single-Interaction Articulated Modeling. Given an initial point cloud, SIAM first enables minimal robot interaction to trigger object motion. It then precisely segments parts by analyzing point cloud differences pre- and post-interaction. For joint parameter estimation, we introduce an optimization incorporating novel kinematic energy constraints, enhancing physical consistency. Finally, we reconstruct a high-quality, topologically watertight mesh by learning 3D Gaussian Primitives from multi-view RGB-D observations under deformation. Extensive experiments on the PartNet-Mobility benchmark demonstrate state-of-the-art articulation modeling performance. Successful real-world deployment with an xArm robot further validates the framework's practicality and transferability. SIAM achieves accurate, prior-free modeling with significantly reduced interaction cost.

Articulated objects, as prevalent entities in human life, their 3D representations play crucial roles across various applications. However, achieving both high-fidelity textured surface reconstruction and dynamic generation for articulated objects remains challenging for existing methods. In this paper, we present REArtGS, a novel framework that introduces additional geometric and motion constraints to 3D Gaussian primitives, enabling realistic surface reconstruction and generation for articulated objects. Specifically, given multi-view RGB images of arbitrary two states of articulated objects, we first introduce an unbiased Signed Distance Field (SDF) guidance to regularize Gaussian opacity fields, enhancing geometry constraints and improving surface reconstruction quality. Then we establish deformable fields for 3D Gaussians constrained by the kinematic structures of articulated objects, achieving unsupervised generation of surface meshes in unseen states. Extensive experiments on both synthetic and real datasets demonstrate our approach achieves high-quality textured surface reconstruction for given states, and enables high-fidelity surface generation for unseen states. Project site: https://sites.google.com/view/reartgs/home.

Our life is populated with articulated objects. Current category-level articulation estimation works largely focus on predicting part-level 6D poses on static point cloud observations. In this paper, we tackle the problem of category-level online robust and real-time 6D pose tracking of articulated objects, where we propose KPA-Tracker, a novel 3D KeyPoint based Articulated object pose Tracker. Given an RGB-D image or a partial point cloud at the current frame as well as the estimated per-part 6D poses from the last frame, our KPA-Tracker can effectively update the poses with learned 3D keypoints between the adjacent frames. Specifically, we first canonicalize the input point cloud and formulate the pose tracking as an inter-frame pose increment estimation task. To learn consistent and separate 3D keypoints for every rigid part, we build KPA-Gen that outputs the high-quality ordered 3D keypoints in an unsupervised manner. During pose tracking on the whole video, we further propose a keypoint-based articulation tracking algorithm that mines keyframes as reference for accurate pose updating. We provide extensive experiments on validating our KPA-Tracker on various datasets ranging from synthetic point cloud observation to real-world scenarios, which demonstrates the superior performance and robustness of the KPA-Tracker. We believe that our work has the potential to be applied in many fields including robotics, embodied intelligence and augmented reality. All the datasets and codes are available at https://github.com/hhhhhar/KPA-Tracker.

Recent rapid advances in artificial intelligence methods and computational resources have opened up the possibility of solving complex problems in physics with machine learning. While promising, the training of performant deep learning models remains challenging, especially as the choice of various neural architectures or model hyper-parameters can all greatly impact model performance. Hence, it is now fairly common for researchers to systematically conduct either a neural architecture search (NAS) or hyper-parameter tuning as part of the model training process. However, these usually involve repeated model training and evaluation (typically via a cross-validation framework) and can be very computationally expensive. While there have been training-free heuristics proposed for NAS in recent years, this remains an active area of work, and much remains unresolved. Concurrently, physics-informed neural networks (PINNs) have been proposed in recent years as an attractive means of incorporating physics-derived knowledge into a deep learning framework for the modelling of real-world physical systems. However, recent literature has begun to emerge on the additional complexity of training PINNs due to the addition of these physics-derived equations (typically in the form of partial differential equations, PDEs). The extended optimization trajectories observed further exacerbate the aforementioned computational cost of NAS or hyper-parameter search. Hence, in this work, we propose a first demonstration of the utility of training-free heuristics for accelerating the selection of model parameters. In particular, we assess the correlation between basic concepts espoused in Nyquist-Shannon sampling, and the appropriate selection of mini-batch size for PINN training with stochastic gradient descent (specifically, the state-of-the-art ADAM algorithm). As a proof-of-concept, this paper focuses on training PINNs to predict the solution to a canonical 2D Helmholtz equation with varying complexity (as measured by the frequency of the signal present), and we evaluate and report the influence of different mini-batch size parameters on the effective training of PINNs. The results show that PINN model performance exhibits a step-like phase transition with increased mini-batch size during training. Critically, we note that the minimum mini-batch size used increases proportionally with the presence of higher frequencies in the physical system being modelled, as is intuitively described by the Nyquist-Shannon sampling theorem. This both shows the utility of similar potential heuristics for the training-free selection of hyper-parameters when training PINNs and suggests the potential for insights towards accelerated design and training of PINNs for physical systems from prior theoretical concepts in signal processing and reconstruction.