Procedure step recognition (PSR) aims to identify all correctly completed steps and their sequential order in videos of procedural tasks. The existing state-of-the-art models rely solely on detecting assembly object states in individual video frames. By neglecting temporal features, model robustness and accuracy are limited, especially when objects are partially occluded. To overcome these limitations, we propose Spatio-Temporal Occlusion-Resilient Modeling for Procedure Step Recognition (STORM-PSR), a dual-stream framework for PSR that leverages both spatial and temporal features. The assembly state detection stream operates effectively with unobstructed views of the object, while the spatio-temporal stream captures both spatial and temporal features to recognize step completions even under partial occlusion. This stream includes a spatial encoder, pre-trained using a novel weakly supervised approach to capture meaningful spatial representations, and a transformer-based temporal encoder that learns how these spatial features relate over time. STORM-PSR is evaluated on the MECCANO and IndustReal datasets, reducing the average delay between actual and predicted assembly step completions by 11.2% and 26.1%, respectively, compared to prior methods. We demonstrate that this reduction in delay is driven by the spatio-temporal stream, which does not rely on unobstructed views of the object to infer completed steps. The code for STORM-PSR, along with the newly annotated MECCANO labels, is made publicly available at https://timschoonbeek.github.io/stormpsr . • Spatio-temporal features reduce prediction delay for procedure step recognition (PSR). • Weakly-supervised key-frame selection enables meaningful spatial features. • Key-clip aware sampling significantly improves training of the temporal encoder. • PSR annotations and benchmark on the MECCANO dataset to stimulate further research.

Medial representations have been widely used for many shape analysis and processing tasks. Large and complex 3D shapes are, in this context, a challenging case. Recently, several methods have been proposed that extract point-based medial surfaces with high accuracy and computational scalability. However, the resulting medial clouds are of limited use for shape processing due to the difficulty of computing refined medial features from such clouds. In this paper, we show how to bridge the gap between having a raw medial cloud and enriching this cloud with feature points, medial-point classification, medial axis decomposition into sheets, robust regularization, and Y-network extraction. We further show how such properties can be used to support several shape processing sample applications including edge detection and shape segmentation, for a wide range of complex 3D shapes.

Abstract We present a method for extracting complex manifolds with an arbitrary number of (self‐) intersections from unoriented point clouds containing large amounts of noise. Manifolds are formed in a three‐step process. First, small flat neighbourhoods of all possible orientations are created around all points. Next, neighbourhoods are assembled into larger quasi‐flat patches, whose overlaps give the global connectivity structure of the point cloud. Finally, curved manifolds are extracted from the patch connectivity graph via a multiple‐source flood fill. The manifolds can be reconstructed into meshed surfaces using standard existing surface reconstruction methods. We demonstrate the speed and robustness of our method on several point clouds, with applications in point cloud segmentation, denoising and medial surface reconstruction.

We present a GPU-based framework for extracting surface and curve skeletons of 3D shapes represented as large polygonal meshes. We use an efficient parallel search strategy to compute point-cloud skeletons and their distance and feature transforms (FTs) with user-defined precision. We regularize skeletons by a new GPU-based geodesic tracing technique which is orders of magnitude faster and more accurate than comparable techniques. We reconstruct the input surface from skeleton clouds using a fast and accurate image-based method. We also show how to reconstruct the skeletal manifold structure as a polygon mesh and the curve skeleton as a polyline. Compared to recent skeletonization methods, our approach offers two orders of magnitude speed-up, high-precision, and low-memory footprints. We demonstrate our framework on several complex 3D models.