Exploded views are powerful tools for visualizing the assembly and disassembly of complex objects, widely used in technical illustrations, assembly instructions, and product presentations. Previous methods for automating the creation of exploded views are either slow and computationally costly or compromise on accuracy. Therefore, the construction of exploded views is typically a manual process. In this paper, we propose a novel approach for automatically predicting the direction of parts in an exploded view using deep learning. To achieve this, we introduce a new dataset, AssemblyNet, which contains point cloud data sampled from 3D models of real-world assemblies, including water pumps, mixed industrial assemblies, and LEGO models. The AssemblyNet dataset includes a total of 44 assemblies, separated into 495 subassemblies with a total of 5420 parts. We provide ground truth labels for regression and classification, representing the directions in which the parts are moved in the exploded views. We also provide performance benchmarks using various state-of-the-art models for shape classification on point clouds and propose a novel two-path network architecture. Project page available at https://github.com/jgaarsdal/AssemblyNet

A trend of distance underestimations in Virtual Reality (VR) is well documented, but the reason still remains unclear. Therefore, this paper investigates the effect of differently sized Virtual Environments (VEs) on egocentric distance perception in VR as a potential influence. Verbal assessment, blind walking, and our own proposed method: walk and assess, were compared in an experiment, and blind walking was found to be the most accurate. A virtual replica of a real-life location was created as a transitional environment, while small (15m2), medium (35m2) and large (95m2) rooms were created to investigate the effect of VE size on spatial perception in VR. To establish the differences in estimations between the real world and VR when using blind walking, a study was conducted with the virtual replica and its real life counterpart at distances between 1 and 10 meters. Following this, the three distinct room sizes were used in an experiment to investigate the effect of the size of rooms on spatial perception in VR. The findings showed consistent underestimates of distances, and a trend for underestimation to grow as the distance grows was observed. Similarly, underestimates grew with the size of the environment.

The paper investigates the accuracy of scale perception in Virtual Reality (VR) for visualization of urban scale environments. Specifically, we evaluate users’ scale estimation accuracy by subjecting them to an urban scale environment using two different viewing modes: 360° monoscopic panoramas viewed in a VR headset, versus a full stereoscopic 3D VR representation, also in a VR headset, allowing participants to move around. The paper explores various aspects of this, including both ego- and exo-centric distance estimation, perception of own height, and photographic realism of 360° modes. The main experimental result is that, somewhat surprisingly, user distance estimation accuracy is higher in the monoscopic 360° viewing modes than in 3D VR; in 3D VR participants on average underestimate distances by around 20%. Nevertheless, participants on average feel significantly taller than normal in the 360° modes, whereas they feel normal height in 3D VR mode. We conclude that more work is needed in order to properly understand the perceptual and cognitive mechanisms behind scale perception in VR.

Immersive Virtual Reality enables users to experience 3D models and other virtual content in ways that cannot be achieved on a flat screen, and several modern Virtual Reality applications now give users the ability to include or create their own content and objects. With user-generated content however, objects may come in all shapes and sizes. This necessitates the use of object manipulation methods that are effective regardless of object size. In this work we evaluate two methods for manipulating virtual objects of varying sizes. World Pull enables the user to directly manipulate and scale the virtual environment, while Pivot Manipulation enables the user to rotate objects around a set of predefined pivot points. The methods were compared to a traditional 6 degree of freedom manipulation method during a user study and the results showed that World Pull performed better in terms of precision for small and large objects, while Pivot Manipulation performed better for large objects.

Image-based 3D modelling using Structure-form-Motion (SfM) has matured significantly over the last decade. Standard SfM methods create the object's texture from the appearance of the physical object at the time of acquisition. We propose a method for acquiring the diffuse per-point reflectance of the modelled object, as part of the image acquisition work flow, only adding one extra captured image and an irradiance rendering step, making it easy for anyone to digitize physical objects to create 3D content for AR/VR using only consumer grade hardware. Current state of the art of spatially varying reflectance capture requires either large, expensive, and purpose built setups or are optimization based approaches, whereas the proposed approach is model based. This paper proposes adding a render of irradiance with modelled camera and light source, using off the shelf hardware for image capture. The key element is taking two images at each imaging location: one with just the ambient illumination conditions, and one where the light from an on-camera flash is included. It is demonstrated how to get the ambient illumination to cancel out, and by assuming Lambertian materials, render the irradiance corresponding to the flash-only image, enabling computation of spatially varying diffuse reflectance rather than appearance. Qualitative results demonstrate the added realism of the modelled objects when used as assets in renders under varying illumination conditions, including limited outdoor scenarios. Quantitative tests demonstrate that the reflectance can be estimated correctly to within a few percent even in cases with severe un-even ambient illumination.

Temporal coherence of annotations is an important factor in augmented reality user interfaces and for information visualization. In this paper, we empirically evaluate four different techniques for annotation. Based on these findings, we follow up with subjective evaluations in a second experiment. Results show that presenting annotations in object space or image space leads to a significant difference in task performance. Furthermore, there is a significant interaction between rendering space and update frequency of annotations. Participants improve significantly in locating annotations, when annotations are presented in object space, and view management update rate is limited. In a follow-up experiment, participants appear to be more satisfied with limited update rate in comparison to a continuous update rate of the view management system.

When viewing 3-D scatterplots in immersive virtual environments, one commonly encountered problem is the presence of clutter, which obscures the view of any structures of interest in the visualization. In order to solve this problem, we propose to render the 3-D glyphs such that they always cover the the same amount of screen space. For perceptual reasons, we call this approach constant visual angle glyphs, or CVA glyphs. The use of CVA glyphs implies some desirable perceptual consequences, which have not been previously described or discussed in existing literature: CVA glyphs not only have the prospect of dealing with clutter, but also the prospect of allowing for a better perception of the continuous shapes of structures in 3-D scatterplots. In a formal user evaluation of CVA glyphs, the results indicate that such glyphs do allow for better perception of shapes in 3-D scatterplots compared to regular perspective glyphs, especially when a large amount of clutter is present. Furthermore, our evaluation revealed that perception of structures in 3-D scatterplots is significantly affected by the volumetric density of the glyphs in the plot.

Enabling users to shape 3-D boxes in immersive virtual environments is a non-trivial problem. In this paper, a new family of techniques for creating rectangular boxes of arbitrary position, orientation, and size is presented and evaluated. These new techniques are based solely on position data, making them different from typical, existing box shaping techniques. The basis of the proposed techniques is a new algorithm for constructing a full box from just three of its corners. The evaluation of the new techniques compares their precision and completion times in a 9 degree-of-freedom (DoF) docking experiment against an existing technique, which requires the user to perform the rotation and scaling of the box explicitly. The precision of the users' box construction is evaluated by a novel error metric measuring the difference between two boxes. The results of the experiment strongly indicate that for precision docking of 9 DoF boxes, some of the proposed techniques are significantly better than ones with explicit rotation and scaling. Another interesting result is that the number of DoF simultaneously controlled by the user significantly influences the precision of the docking.

This paper presents a method for improving the disparity values obtained with a stereo camera by applying an iconic Kalman filter and known ego-motion. The improvements are demonstrated in an application of determining the free space in a scene as viewed by a vehicle-mounted camera. Using disparity maps from a stereo camera and known camera motion, the disparity maps are first filtered by an iconic Kalman filter, operating on each pixel individually, thereby reducing variance and increasing the density of the filtered disparity map. Then, a stochastic occupancy grid is calculated from the filtered disparity map, providing a top-down view of the scene where the uncertainty of disparity measurements are taken into account. These occupancy grids are segmented to indicate a maximum depth free of obstacles, enabling the marking of free space in the accompanying intensity image. Even without motion of the camera, the quality of the disparity map is increased significantly. Applications of the intermediate results are discussed, enabling features such as motion detection and quantifying the certainty of the measurements. The evaluation shows significant improvement in disparity variance and disparity map density, and consequently an improvement in the application of marking free space.

This paper presents a computer vision based head tracking system for augmented reality. A camera is attached to a head mounted display and used to track markers in the user's field of view. These markers are movable on a table and used as interface with virtual objects. Furthermore, they are used as landmarks to track the user's viewpoint and viewing direction by a homography based camera pose estimation algorithm. By integrating this computer vision tracker with a commercially available InterSense tracker, we take the advantages of the small jitter of the former one without losing the tracking speed of the later one. For static and slow head motion the system has less than 0.3mm RMS of position jitter and 0.165 degrees RMS of orientation jitter.

The dominant architecture for mobile robot perception uses sensors on-board the robot, providing a first-person perspective on the environment. We demonstrate a novel mobile robot architecture that uses an environment-based sensor network, which provides third-person perception. The idea is that a mobile robot working in the area tunes in to broadcasts from the video camera network (or in this case from an environment-based computer processing the video frames) to receive sensor data. This distributed sensing configuration offers several advantages over on-board sensing.

The paper presents a viewpoint planning strategy which automatically guides a movable camera from an arbitrary position to a position where the optical axis is perpendicular to a plane spanned by any two intersecting edges on a polyhedral object, i.e., a junction. In related work it is common to use the changing length of edge segments to control such alignment, but we demonstrate the use of the apparent angle between the edge segments of the junction. Based on the control on the apparent angle we achieve robustness as well as independence of distance to the object and focal length. The strategy is able to determine the true angle of a junction with an accuracy of approximately 1/spl deg/, and align with an accuracy of approximately 6/spl deg/.

The paper presents an analysis of the stability of pose estimation. The investigated pose estimation technique is based on orientations of three edge segments and provides the rotation part of object pose. The specific emphasis of the analysis is on determining how the stability varies with view point relative to an object. The stability investigation propagates the uncertainty in edge segment orientations to the resulting effect on the pose parameters. It is shown that there is a very strong variation in noise sensitivity over the range of viewpoints and that exactly what viewpoints offer highest robustness towards noise can be determined in advance. Experiments on real images verify the theoretical results and show that, dependent on viewpoint, pose parameter variance varies from 0.05 to 20 (degrees squared).

We present a view point planning strategy for determining the true angle between legs of junctions in a polyhedral scene. The strategy results in a reactive visual behaviour guiding the camera towards a canonical view point producing a fronto-parallel projection of object faces. From these views the angle in the image is identical to the true angle, thus the method does not involve reconstruction. Qualitative visual events, e.g., zero-crossings in the rate-of-change of measured, apparent angle, are used to control the motion of the camera, resulting in independence of positional feedback.>

We present a view planning approach to determining angles between two-line junctions on an object. Using view planning we demonstrate an ability to determine such angles without reconstruction of 3D properties, and without using known camera displacement. The core in our approach is a two-step view planning strategy based on two simple motion patterns. The view planning is controlled using local extrema in the apparent (viewed) angle. We demonstrate the approach on real images and experiments show an accuracy in determined angle which is better than /spl plusmn/1 degrees.