Smartphone augmented reality (AR) lets users interact with physical and virtual spaces simultaneously. With 3D hand tracking, smartphones become apparatus to grab and move virtual objects directly. Based on design considerations for interaction, mobility, and object appearance and physics, we implemented a prototype for portable 3D hand tracking using a smartphone, a Leap Motion controller, and a computation unit. Following an experience prototyping procedure, 12 researchers used the prototype to help explore usability issues and define the design space. We identified issues in perception (moving to the object, reaching for the object), manipulation (successfully grabbing and orienting the object), and behavioral understanding (knowing how to use the smartphone as a viewport). To overcome these issues, we designed object-based feedback and accommodation mechanisms and studied their perceptual and behavioral effects via two tasks: picking up distant objects, and assembling a virtual house from blocks. Our mechanisms enabled significantly faster and more successful user interaction than the initial prototype in picking up and manipulating stationary and moving objects, with a lower cognitive load and greater user preference. The resulting system---Portal-ble---improves user intuition and aids free-hand interactions in mobile situations.

We present a method for rendering depth of field (DoF) effects in a ray-tracing based rendering pipeline using very few samples (typically two or three) per pixel, with the ability to refocus at arbitrary depths at a given view point without gathering more samples. To do so, we treat each sample as a proxy for possible nearby samples and calculate its contributions to the final image with a splat-and-gather scheme. The radiance for each pixel in the output image is then obtained via compositing all contributing samples. We optimize the pipeline using mipmap-like techniques so that the running time is independent of the amount of focal blur in the image. Our method approximates the underlying physical image formation process and thus avoids many of the artifacts of other approximation algorithms. With very low budget it provides satisfactory DoF rendering for most purposes, and a quick preview of DoF effects for applications demanding high rendering quality.

We introduce a new method for coloring 3D line fields and show results from its application in visualizing orientation in DTI brain data sets. The method uses Boy's surface, an immersion of RP2 in 3D. This coloring method is smooth and one-to-one except on a set of measure zero, the double curve of Boy's surface.

Touchpad and touchscreen interaction using multiple fingers is emerging as a valuable form of high-degree-of-freedom input. While bimanual interaction has been extensively studied, touchpad interaction using multiple fingers of the same hand is not yet well understood. We describe two experiments on user perception and control of multi-touch interaction using one and two hands. The first experiment addresses how to maintain perceptual-motor compatibility in multi-touch interaction, while the second measures the separability of control of degrees-of-freedom in the hands and fingers. Results indicate that two-touch interaction using two hands is compatible with control of two points, while twotouch interaction using one hand is compatible with control of a position, orientation, and hand-span. A slight advantage is found for two hands in separating the control of two positions.

We describe a GPU-based algorithm for rendering a 3D model as a line drawing, based on the insight that a line drawing can be understood as an abstraction of a shaded image. We thus render lines along tone boundaries or thin dark areas in the shaded image. We extend this notion to the dual: we render highlight lines along thin bright areas and tone boundaries. We combine the lines with toon shading to capture broad regions of tone. The resulting line drawings effectively convey both shape and material cues. The lines produced by the method can include silhouettes. creases, and ridges, along with a generalization of suggestive contours that responds to lighting as well as viewing changes. The method supports automatic level of abstraction, where the size of depicted shape features adjusts appropriately as the camera zooms in or out. Animated models can be rendered in real time because costly mesh curvature calculations are not needed.

We introduce SmoothSketch---a system for inferring plausible 3D free-form shapes from visible-contour sketches. In our system, a user's sketch need not be a simple closed curve as in Igarashi's Teddy [1999], but may have cusps and T-junctions, i.e., endpoints of hidden parts of the contour. We follow a process suggested by Williams [1994] for inferring a smooth solid shape from its visible contours: completion of hidden contours, topological shape reconstruction, and smoothly embedding the shape via relaxation. Our main contribution is a practical method to go from a contour drawing to a fairly smooth surface with that drawing as its visible contour. In doing so, we make several technical contributions: • extending Williams' and Mumford's work [Mumford 1994] on figural completion of hidden contours containing T-junctions to contours containing cusps as well, • characterizing a class of visible-contour drawings for which inflation can be proved possible, • finding a topological embedding of the combinatorial surface that Williams creates from the figural completion, and • creating a fairly smooth solid shape by smoothing the topological embedding using a mass-spring system.We handle many kinds of drawings (including objects with holes), and the generated shapes are plausible interpretations of the sketches. The method can be incorporated into any sketch-based free-form modeling interface like Teddy.

We present an interactive system that lets a user move and deform a two-dimensional shape without manually establishing a skeleton or freeform deformation (FFD) domain beforehand. The shape is represented by a triangle mesh and the user moves several vertices of the mesh as constrained handles. The system then computes the positions of the remaining free vertices by minimizing the distortion of each triangle. While physically based simulation or iterative refinement can also be used for this purpose, they tend to be slow. We present a two-step closed-form algorithm that achieves real-time interaction. The first step finds an appropriate rotation for each triangle and the second step adjusts its scale. The key idea is to use quadratic error metrics so that each minimization problem becomes a system of linear equations. After solving the simultaneous equations at the beginning of interaction, we can quickly find the positions of free vertices during interactive manipulation. Our approach successfully conveys a sense of rigidity of the shape, which is difficult in space-warp approaches. With a multiple-point input device, even beginners can easily move, rotate, and deform shapes at will.

Video matting is the process of pulling a high-quality alpha matte and foreground from a video sequence. Current techniques require either a known background (e.g., a blue screen) or extensive user interaction (e.g., to specify known foreground and background elements). The matting problem is generally under-constrained, since not enough information has been collected at capture time. We propose a novel, fully autonomous method for pulling a matte using multiple synchronized video streams that share a point of view but differ in their plane of focus. The solution is obtained by directly minimizing the error in filter-based image formation equations, which are over-constrained by our rich data stream. Our system solves the fully dynamic video matting problem without user assistance: both the foreground and background may be high frequency and have dynamic content, the foreground may resemble the background, and the scene is lit by natural (as opposed to polarized or collimated) illumination.

We present a technique for scrolling through documents that is simple to implement and requires no special hardware. This is accomplished by simulating a hardware scroll ring--a device that maps circular finger motion into vertical scrolling. The technique performs at least as well as a mouse wheel for medium and long distances, and is preferred by users. It can be particularly useful in portable devices where screen-space and space for peripherals is at a premium.

This paper describes a framework for introducing visually smooth surfaces into sketch-based freeform modeling systems. An existing sketch-based freeform modeling system generates rough polygonal meshes with uneven triangulations after each operation. Our approach generates a dense, visually smooth polygonal mesh by beautifying and refining the original rough mesh. A beautification process generates near-equilateral triangles with a near-uniform distribution of vertices to mask the noise and bad sampling of the uneven mesh. The vertices are distributed on a smoothed surface that approximately interpolates the original mesh. Refinement generates a smooth, dense mesh by subdividing the beautified mesh and moving the vertices to the interpolative surface. The smooth interpolative surface is computed via implicit quadratic surfaces that best fit the mesh locally in a least-squares sense.

No abstract available.

This paper presents interaction techniques (and the underlying implementations) for putting clothes on a 3D character and manipulating them. The user paints freeform marks on the clothes and corresponding marks on the 3D character; the system then puts the clothes around the body so that corresponding marks match. Internally, the system grows the clothes on the body surface around the marks while maintaining basic cloth constraints via simple relaxation steps. The entire computation takes a few seconds. After that, the user can adjust the placement of the clothes by an enhanced dragging operation. Unlike standard dragging where the user moves a set of vertices in a single direction in 3D space, our dragging operation moves the cloth along the body surface to make possible more flexible operations. The user can apply pushpins to fix certain cloth points during dragging. The techniques are ideal for specifying an initial cloth configuration before applying a more sophisticated cloth simulation.

With the advent of sketch-based methods for shape construction, there's a new degree of power available in the rapid creation of approximate shapes. Sketch [Zeleznik, 1996] showed how a gesture-based modeler could be used to simplify conventional CSG-like shape creation. Teddy [Igarashi, 1999] extended this to more free-form models, getting much of its power from its ``inflation'' operation (which converted a simple closed curve in the plane into a 3D shape whose silhouette, from the current point of view, was that curve on the view plane) and from an elegant collection of gestures for attaching additional parts to a shape, cutting a shape, and deforming it. But despite the powerful collection of tools in Teddy, the underlying polygonal representation of shapes intrudes on the results in many places. In this paper, we discuss our preliminary efforts at using variational implicit surfaces [Turk, 2000] as a representation in a free-form modeler. We also discuss the implementation of several operations within this context, and a collection of user-interaction elements that work well together to make modeling interesting hierarchies simple. These include ``stroke inflation'' via implicit functions, blob-merging, automatic hierarchy construction, and local surface modification via silhouette oversketching. We demonstrate our results by creating several models. Categories and Subject Descriptors (according to ACM CCS): I.3.5 [Computer Graphics]: Modeling packages I.3.6 [Computer Graphics]: Interaction techniques

We present a system that lets a designer directly annotate a 3D model with strokes, imparting a personal aesthetic to the non-photorealistic rendering of the object. The artist chooses a "brush" style, then draws strokes over the model from one or more viewpoints. When the system renders the scene from any new viewpoint, it adapts the number and placement of the strokes appropriately to maintain the original look.

We apply simple techniques from traditional artistic composition to the art-based rendering of interactive 3D scenes. A human scene-modeler makes choices about composition in a scene and our sys-tem dynamically adjusts the rendering attributes of objects in the scene to achieve the desired effects for a given view. We can selec-tively group scene elements through shared tone, color, and outline, so as to simplify and structure an image. This can be used, together with controlled level of detail, to emphasize important objects. Fi-nally, we show a technique for adaptively changing color or other attributes to control the contrast of adjacent elements in the pic-ture. We also briefly discuss ideas about larger-scale compositional issues.

We describe the use of non-verbal features in voice for direct control of interactive applications. Traditional speech recognition interfaces are based on an indirect, conversational model. First the user gives a direction and then the system performs certain operation. Our goal is to achieve more direct, immediate interaction like using a button or joystick by using lower-level features of voice such as pitch and volume. We are developing several prototype interaction techniques based on this idea, such as "control by continuous voice", "rate-based parameter control by pitch," and "discrete parameter control by tonguing." We have implemented several prototype systems, and they suggest that voice-as-sound techniques can enhance traditional voice recognition approach.

This paper introduces a new type of interface for 3D drawings that improves the usability of gestural interfaces and augments typical command-based modeling systems. In our suggestive interface, the user gives hints about a desired operation to the system by highlighting related geometric components in the scene. The system then infers possible operations based on the hints and presents the results of these operations as small thumbnails. The user completes the editing operation simply by clicking on the desired thumbnail. The hinting mechanism lets the user specify geometric relations among graphical components in the scene, and the multiple thumbnail suggestions make it possible to define many operations with relatively few distinct hint patterns. The suggestive interface system is implemented as a set of suggestion engines working in parallel, and is easily extended by adding customized engines. Our prototype 3D drawing system, Chateau, shows that a suggestive interface can effectively support construction of various 3D drawings.

The ability to specify non planar 3D curves is of fundamental importance in 3D modelling and animation systems. Effective techniques for specifying such curves using 2D input devices are desirable, but existing methods typically require the user to edit the curve from several viewpoints. We present a novel method for specifying 3D curves with 2D input from a single viewpoint. The user rst draws the curve as it appears from the current viewpoint, and then draws its shadow on the oor plane. The system correlates the curve with its shadow to compute the curve's 3D shape. This method is more natural than existing methods in that it leverages skills that many artists and designers have developed from work with pencil and paper.

Artists and illustrators can evoke the complexity of fur or vegetation with relatively few well-placed strokes. We present an algorithm that uses strokes to render 3D computer graphics scenes in a stylized manner suggesting the complexity of the scene without representing it explicitly. The basic algorithm is customizable to produce a range of effects including fur, grass and trees, as we demonstrate in this paper and accompanying video. The algorithm is implemented within a broader framework that supports procedural stroke-based textures on polyhedral models. It renders moderately complex scenes at multiple frames per second on current graphics workstations, and provides some interframe coherence. CR Categories and Subject Descriptors: I.3.3: Computer Graphics: Picture/Image Generation; line and curve generation; bitmap and frambuffer operations; I.3.5 Computer Graphics: Computational Geometry and Object Modeling: curve, surface, solid, and object representations; I.3.7: Three-Dimens...

We present a new particle-based surface representation with which a user can interactively sculpt free-form surfaces. The particles maintain mesh connectivity and operate under rules that lead them to form triangulations with properties that make them suitable for use in subdivision. A user interactively guides the particles, which we call skin, to grow over a given collection of polyhedral elements (or skeletons), yielding a smooth surface (through subdivision) that approximates the underlying skeletal shapes. Skin resembles blobby modeling in the constructive approach to modeling it supports, but allows a richer vocabulary of skeleton shapes, supports sharp creases where desired, and provides a convenient mechanism for adding multiresolution surface detail. CR

Heuristics that exploit bouning boxes are common in algorithms for rendering, modeling, and animation. While experience has shown that bounding boxes improve the performance of these algorithms in practice, the previous theoretical analysis has concluded that bounding boxes perform poorly in the worst case. This paper reconciles this discrepancy by analyzing intersections among n geometric objects in terms of two parameters: α an upper bound on the aspect ratio or elongatedness of each object; and σ an upper bound on the scale factor or size disparity between the largest and smallest objects. Letting K o and K b be the number of intersecting object pairs and bounding box pairs, respectively, we analyze a ratio measure of the bounding boxes' efficiency, ρ = K b / (n + K 0 ) . The analysis proves that ρ = O(α√σlog 2 σ) and ρ = Ω(α√σ) . One important consequence is that if α and σ are small constants (as is often the case in practice), then K b = O ( K o )+ O ( n , so an algorithm that uses bounding boxes has time complexity proportional to the number of actual object intersections. This theoretical result validates the efficiency that bounding boxes have demonstrated in practice. Another consequence of our analysis is a proof of the output-sensitivity of an algorithm for reporting all intersecting pairs in a set of n convex polyhedra with constant α and σ. The algorithm takes time O ( n log d -1 n + K o log d -1 n ) for dimension d = 2, 3. This running time improves on the performance of previous algorithms, which make no assumptions about α and σ.

Nonphotorealistic rendering (NPR) can help make comprehensible but simple pictures of complicated objects by employing an econ-omy of line. But current nonphotorealistic rendering is primarily a batch process. This paper presents a real-time nonphotorealistic renderer that deliberately trades accuracy and detail for speed. Our renderer uses a method for determining visible lines and surfaces which is a modification of Appel’s hidden-line algorithm, with im-provements which are based on the topology of singular maps of a surface into the plane. The method we describe for determining visibility has the potential to be used in any NPR system that re-quires a description of visible lines or surfaces in the scene. The major contribution of this paper is thus to describe a tool which can significantly improve the performance of these systems. We demon-strate the system with several nonphotorealistic rendering styles, all of which operate on complex models at interactive frame rates.

We present an interactive system for creating pen-and-ink-style line drawings from greyscale images in which the strokes of the rendered illustration follow the features of the original image. The user, via new interaction techniques for editing a direction field, specifies an orientation for each region of the image; the computer draws oriented strokes, based on a user-specified set of example strokes, that achieve the same tone as the image via a new algorithm that compares an adaptively-blurred version of the current illustration to the target tone image. By aligning the direction field with surface orientations of the objects in the image, the user can create textures that appear attached to those objects instead of merely conveying their darkness. The result is a more compelling pen-and-ink illustration than was previously possible from 2D reference imagery. CR Categories and Subject Descriptors: I.3.3 [Computer Graphics ]: Picture/Image Generation --- Display algorithms. I.4.3 [...

We describe a new approach for simulating apparent camera motion through a 3D environment. The approach is motivated by a traditional technique used in 2D cel animation, in which a single background image, which we call a multiperspective panorama,is used to incorporate multiple views of a 3D environment as seen from along a given camera path. When viewed through a small moving window, the panorama produces the illusion of 3D motion. In this paper, we explore how such panoramas can be designed by computer, and we examine their application to cel animation in particular. Multiperspective panoramas should also be useful for any application in which predefined camera moves are applied to 3D scenes, including virtual reality fly-throughs, computer games, and architectural walk-throughs. CR Categories: I.3.3 [Computer Graphics]: Picture/Image Generation. Additional Keywords: CGI production, compositing, illustration, imagebased rendering, mosaics, multiplaning, non-photorealistic renderi...

No abstract available.

We describe an extension of B-splines to surfaces of arbitrary topology, including arbitrary boundaries. The technique inherits many of the properties of B-splines: local control, a compact representation, and guaranteed continuity of arbitrary degree. The surface is specified using a polyhedral control mesh instead of a rectangular one; the resulting surface approximates the polyhedral mesh much as a B-spline approximates its rectangular control mesh. Like a Bspline, the surface is a single, continuousobject. This is achievedby modeling the domain of the surface with a manifold whose topology matches that of the polyhedral mesh, then embedding this domain into 3-space using a basis-function/control-point formulation. We provide a constructive approach to building a manifold. CR Categories: I.3.5 [Computer Graphics]: Computational Geometry and Object Modeling, Curve, Surface, Solid, and Object Representations, Splines 1 Introduction Surfaces of arbitrary topology are currently attrac...

This paper presents the architecture for an extensible toolkit used in construction and rapid prototyping of three dimensional interfaces, interactive illustrations, and three dimensional widgets. The toolkit provides methods for the direct manipulation of 3D primitives which can be linked together through a visual programming language to create complex constrained behavior. Features of the toolkit include the ability to visually build, encapsulate, and parameterize complex models, and impose limits on the models. The toolkit's constraint resolution technique is based on a dynamic object model similar to those in prototype delegation object systems. The toolkit has been used to rapidly prototype tools for mechanical modelling, scientific visualization, construct 3D widgets, and build mathematical illustrations.

Surface-based rendering techniques, particularly those that extract a polygonal approximation of an isosurface, are widely used in volume visualization. As dataset size increases though, the computational demands of these methods can overwhelm typically available computing resources. Recent work on accelerating such techniques has focused on preprocessing the volume data or postprocessing the extracted polygonization. The algorithm presented, concentrates instead on streamlining the surface extraction process itself so as to accelerate the rendering of large volumes. The technique shortens the conventional isosurface visualization pipeline by eliminating the intermediate polygonization. We compute the contribution of the isosurface within a volume cell to the resulting image directly from a simplified numerical description of the cell/surface intersection. The approach also reduces the work in the remaining stages of the visualization process. By quantizing the volume data, we exploit precomputed and cached data at key processing steps to improve rendering efficiency. The resulting implementation provides comparatively fast renderings with reasonable image quality.>

Annotation is a key activity of data analysis. However, current data analysis systems focus almost exclusively on visualization. We propose a system which integrates annotations into a visualization system. Annotations are embedded in 3D data space, using the Post-it metaphor. This embedding allows contextual-based information storage and retrieval, and facilitates information sharing in collaborative environments. We provide a traditional database filter and a Magic Lens filter to create specialized views of the data. The system is customized for fluid flow applications, with features which allow users to store parameters of visualization tools and sketch 3D volumes.>

We describe two instances in which precise mechanical calibration of virtual environments equipment has been replaced by automated algorithmic calibration through software that encapsulates the hardware design and uses a goal-based approach to adjust calibration parameters.We describe a back-projection system for adjusting the assumed locations of beacons in a head-mounted display tracking system; the calculated errors in the navigation system are used to compute adjustments to the beacon positions to reduce such errors.In a second application, a piggyback headtracking/hand-tracking system is calibrated by a similar reduction of computed errors.

Introduction Today's user interfaces for most 3D graphics applications still depend heavily on 2D GUIs and keyboard input. There have been several recent attempts both to extend these user interfaces into 3D and to describe intermediary 3D widgets 1 that control application objects [3; 4; 5; 7; 13; 15]. Even though this style of interaction is a straightforward extension of interaction through intermediary 2D widgets such as dials or sliders, we know of no efforts to develop interactive 3D toolkits akin to UIMX or Garnet [11]. The Brown Graphics Group has had considerable experience using its Unified Graphics Architecture (UGA) system [16] to script 3D widgets such as deformation racks [14], interactive shadows [9], parameterized models, and other constrained 3D geometries. Using this experience, we have developed an interactive toolkit to facilitate the visual programming of the geometry and behavior of such interactive models. The toolkit provides both a co

In this paper we present methods to smoothly interpolate orientations, given N rotational keyframes of an object along a trajectory. The methods allow the user to impose constraints on the rotational path, such as the angular velocity at the endpoints of the trajectory. We convert the rotations to quaternions, and then spline in that non-Euclidean space. Analogous to the mathematical foundations of flat-space spline curves, we minimize the net "tangential acceleration" of the quaternion path. We replace the flat-space quantities with curved-space quantities, and numerically solve the resulting equation with finite difference and optimization methods. 1 Introduction The problem of using spline curves to smoothly interpolate mathematical quantities in flat Euclidean spaces is a well-studied problem in computer graphics [bartels et al 87], [kochanek&bartels 84]. Many quantities important to computer graphics, however, such as rotations, lie in non-Euclidean spaces. In 1985, a method to...

Free-form deformation (FFD) is a powerful modeling tool, but controlling the shape of an object under complex deformations is often difficult. The interface to FFD in most conventional systems simply represents the underlying mathematics directly; users describe deformations by manipulating control points. The difficulty in controlling shape precisely is largely due to the control points being extraneous to the object; the deformed object does not follow the control points exactly. In addition, the number of degrees of freedom presented to the user can be overwhelming. We present a method that allows a user to control a free-form deformation of an object by manipulating the object directly, leading to better control of the deformation and a more intuitive interface. CR Categories: I.3.5 [Computer Graphics]: Computational Geometry and Object Modeling - Curve, Surface, Solid, and Object Representations; I.3.6 [Computer Graphics ]: Methodology and Techniques - Interaction Techniques. Ad...

We describe an easily implemented and computationally feasible method for smoothly transitioning from one sampled volumetric model to another. This induces a transition between isosurfaces of the two models. The technique is based on interpolating smoothly between the Fourier transforms of the two volumetric models and then transforming the results back. A linear interpolation between the transformed datasets yields unsatisfactory results in some cases. We use a schedule for the interpolation in which the high frequencies of the first model are gradually removed, the low frequencies are interpolated to those of the second, and the high frequencies of the second model are gradually added in. Such scheduling yields more satisfactory results. We give several examples and comment briefly on preprocessing models to make the morphing smoother.

We present a new interactive modeling technique based on the notion of sculpting a solid material. A sculpting tool is controlled by a 3D input device and the material is represented by voxel data; the tool acts by modifying the values in the voxel array, much as a "paint" program's "paintbrush" modifies bitmap values. The voxel data is converted to a polygonal surface using a "marching-cubes" algorithm; since the modifications to the voxel data are local, we accelerate this computation by an incremental algorithm and accelerate the display by using a special data structure for determining which polygons must be redrawn in a particular screen region. We provide a variety of tools: one that cuts away material, one that adds material, a "sandpaper" tool, a "heat gun," etc. The technique provides an intuitive direct interaction, as if the user were working with clay or wax. The models created are free-form and may have complex topology; however, they are not precise, so the technique is appropriate for modeling a boulder or a tooth but not for modeling a crankshaft.

We present an interactive modeling and animation system that facilitates the integration of a variety of simulation and animation paradigms. This system permits the modeling of diverse objects that change in shape, appearance, and behaviour over time. Our system thus extends modeling tools to include animation controls. Changes can be effected by various methods of control, including scripted, gestural, and behavioral specification. The system is an extensible testbed that supports research in the interaction of disparate control methods embodied in controller objects. This paper discusses some of the issues involved in modeling such interactions and the mechanisms implemented to provide solutions to some of these issues.The system's object-oriented architecture uses delegation hierarchies to let objects change all of their attributes dynamically. Objects include displayable objects, controllers, cameras, lights, renderers, and user interfaces. Techniques used to obtain interactive performance include the use of data-dependency networks, lazy evaluation, and extensive caching to exploit inter- and intra-frame coherency.

Constructive Solid Geometry (CSG) is a powerful way of describing solid objects for computer graphics and modeling. The surfaces of any primitive object (such as a cube, sphere or cylinder) can be approximated by polygons. Being abile to find the union, intersection or difference of these objects allows more interesting and complicated polygonal objects to be created. The algorithm presented here performs these set operations on objects constructed from convex polygons. These objects must bound a finite volume, but need not be convex. An object that results from one of these operations also contains only convex polygons, and bounds a finite volume; thus, it can be used in later combinations, allowing the generation of quite complicated objects. Our algorithm is robust and is presented in enough detail to be implemented.