Generative models for 3D shapes represented by hierarchies of parts can generate realistic and diverse sets of outputs. However, existing models suffer from the key practical limitation of modelling shapes holistically and thus cannot perform conditional sampling, i.e. they are not able to generate variants on individual parts of generated shapes without modifying the rest of the shape. This is limiting for applications such as 3D CAD design that involve adjusting created shapes at multiple levels of detail. To address this, we introduce LSD-StructureNet, an augmentation to the StructureNet architecture that enables re-generation of parts situated at arbitrary positions in the hierarchies of its outputs. We achieve this by learning individual, probabilistic conditional decoders for each hierarchy depth. We evaluate LSD-StructureNet on the PartNet dataset, the largest dataset of 3D shapes represented by hierarchies of parts. Our results show that contrarily to existing methods, LSD-StructureNet can perform conditional sampling without impacting inference speed or the realism and diversity of its outputs.

Our goal is to recognize material categories using images and geometry information. In many applications, such as construction management, coarse geometry information is available. We investigate how 3D geometry (surface normals, camera intrinsic and extrinsic parameters) can be used with 2D features (texture and color) to improve material classification. We introduce a new dataset, GeoMat, which is the first to provide both image and geometry data in the form of: (i) training and testing patches that were extracted at different scales and perspectives from real world examples of each material category, and (ii) a large scale construction site scene that includes 160 images and over 800,000 hand labeled 3D points. Our results show that using 2D and 3D features both jointly and independently to model materials improves classification accuracy across multiple scales and viewing directions for both material patches and images of a large scale construction site scene.

The articles in this special issue focus on the technology and applications supported by augmented reality.

Determining what physical activity the user is performing using mobile devices is an elusive task even though consumer off the shelf (COTS) mobile devices carried by most users contain a vast array of physical sensors. When conducting work improvement studies, construction crew members are typically observed by eye to record information about the physical activities being performed. Another approach to capturing crew member activities is utilizing sensors within COTS mobile devices carried by crew members. This paper presents CTrack: an approach for capturing telemetry from the array of sensors on-board COTS mobile devices. The resulting sensor telemetry is fused together and can be used to make informed decisions when characterizing crew member activities. Additionally, this paper presents results from field testing CTrack and discusses the current benefits and limitations of fusing multiple telemetry sources to characterize crew member activities.