In 2012, the authors took responsibility for a CS1 course with 45 students. This experience report reviews the subsequent 10-year learning process of engaging undergraduate students to facilitate small-group teaching and to design and develop an online learning environment to conduct what became our university's flagship CS1 course, currently enrolling 749 students. The course inherited an emphasis on small-group learning from its role model, MIT's 6.001. The size of the learning groups is limited to eight students per group, which currently requires a team of 105 student facilitators. The resulting need for student engagement and scaling motivated the development of a new web-based programming environment and assessment management system custom-made for the course. The system was conceived, designed, and implemented by students of the course, which provided the glue for building a sustainable and scalable community of learners, educators, and student software developers. This experience report describes the pedagogic approach, the course structure, and software system to accommodate the needs of this community. A qualitative and quantitative analysis of the impact of the course over the last four years provides evidence for its efficacy. We hope that this report serves as inspiration for similar large-scale pedagogic efforts that bring learners, educators, and student developers together to form sustainable and scalable learning communities.

Due to the recent availability of different stereoscopic display devices and online 3D media resources (e.g. 3D movies), there is a growing demand for stereoscopic video retargeting that can automatically resize a given stereoscopic video to fit the target display device. In this paper, we propose a warping-based approach that can simultaneously resize and remap the depth of a stereoscopic video to produce a better 3D viewing experience. Firstly, our method computes the significance map for each stereo video frame. It then performs volume warping using non-homogeneous scaling optimization to resize the stereoscopic video. A depth remapping constraint is used to remap the depth and a constraint is applied to preserve the significant content during warping process. Experimental results demonstrate the effectiveness of our method in preserving the significant content, ensuring motion consistency, and enhancing the depth perception of the retargeted video sequences within the comfort depth range.

Due to the availability and affordability of the stereoscopic equipment (e.g., stereo camera, lens, and display devices), stereoscopic image manipulation has been receiving considerable research attention in recent years. In this paper, we present a semiautomatic, aesthetic-driven stereoscopic image recomposition approach, which capacitates the change of the spatial position of the foreground object(s) in a given stereoscopic image to enhance human visual aesthetics. Our algorithm recomposes both the left and right stereo images simultaneously using a global optimization algorithm. To maximize image aesthetics, our algorithm minimizes a set of aesthetic quality errors, which is derived from selected photographic composition rules. In addition, depth adaptation is applied to the resized objects and change in vertical disparity of the resulting stereo image pair is minimized to ensure a pleasant three-dimensional (3-D) viewing experience. Our method can be used to perform stereoscopic image retargeting and recomposition simultaneously by providing the target image scale as the input. Empirical evaluations demonstrate the effectiveness of our approach in enhancing the aesthetics of stereoscopic 3-D images. Notably, depth adaptation is shown to play an important role in aesthetics enhancement.

Abstract Paper pop‐ups are interesting three‐dimensional books that fascinate people of all ages. The design and construction of these pop‐up books however are done manually and require a lot of time and effort. This has led to computer‐assisted or automated tools for designing paper pop‐ups. This paper proposes an approach for automatically converting a 3D model into a multi‐style paper pop‐up. Previous automated approaches have only focused on single‐style pop‐ups, where each is made of a single type of pop‐up mechanisms. In our work, we combine multiple styles in a pop‐up, which is more representative of actual artist's creations. Our method abstracts a 3D model using suitable primitive shapes that both facilitate the formation of the considered pop‐up mechanisms and closely approximate the input model. Each shape is then abstracted using a set of 2D patches that combine to form a valid pop‐up. We define geometric conditions that ensure the validity of the combined pop‐up structures. In addition, our method also employs an image‐based approach for producing the patches to preserve the textures, finer details and important contours of the input model. Finally, our system produces a printable design layout and decides an assembly order for the construction instructions. The feasibility of our results is verified by constructing the actual paper pop‐ups from the designs generated by our system.

Origamic architecture (OA) is a form of papercraft that involves cutting and folding a single sheet of paper to produce a 3D pop-up, and is commonly used to depict architectural structures. Because of the strict geometric and physical constraints, OA design requires considerable skill and effort. In this paper, we present a method to automatically generate an OA design that closely depicts an input 3D model. Our algorithm is guided by a novel set of geometric conditions to guarantee the foldability and stability of the generated pop-ups. The generality of the conditions allows our algorithm to generate valid pop-up structures that are previously not accounted for by other algorithms. Our method takes a novel image-domain approach to convert the input model to an OA design. It performs surface segmentation of the input model in the image domain, and carefully represents each surface with a set of parallel patches. Patches are then modified to make the entire structure foldable and stable. Visual and quantitative comparisons of results have shown our algorithm to be significantly better than the existing methods in the preservation of contours, surfaces, and volume. The designs have also been shown to more closely resemble those created by real artists.

While geometry reconstruction has been extensively studied, several shortcomings still exist. First, traditional geometry reconstruction methods such as geometric or photometric stereo only recover either surface depth or normals. Second, such methods require calibration. Third, such methods cannot recover accurate geometry in the presence of interreflections. In order to address these problems in a single system, we propose an approach to reconstruct geometry from light transport data. Specifically, we investigate the problem of geometry reconstruction from interreflections in a light transport matrix. We show that by solving a system of polynomial equations derived directly from the interreflection matrix, both surface depth and normals can be fully reconstructed. Our system does not require projector-camera calibration, but only make use of a calibration object such as a checkerboard in the scene to pre-determine a few known points to simplify the polynomial solver. Our experimental results show that our system is able to reconstruct accurate geometry from interreflections up to a certain noise level. Our system is easy to set up in practice.

Dual photography is a well-known application of light transport acquired by a projector-camera system. By applying compressive sensing, compressive dual photography [1] is a fast approach to acquire the light transport for dual photography. However, the reconstruction step in compressive dual photography can still take several hours before dual images can be synthesized because the entire light transport needs to be reconstructed from measured data. In this paper, we present a novel reconstruction approach that can directly and progressively synthesize dual images from measured data without the need of first reconstructing the light transport. We show that our approach can produce high-quality dual images in the order of minutes using only a thousand of samples. Our approach is most useful for previewing a few dual images, e.g., during light transport acquisition. As a by-product, our method can also perform low-resolution relighting of dual images. We also hypothesize that our method is applicable to reconstructing dual images in a single projector - multiple cameras system.

This paper addresses the difficult problem of finding dense correspondence across images with large appearance variations. Our method uses multiple feature samples at each pixel to deal with the appearance variations based on our observation that pre-defined single feature sample provides poor results in nearest neighbor matching. We apply the idea in a flow-based matching framework and utilize the best feature sample for each pixel to determine the flow field. We propose a novel energy function and use dual-layer loopy belief propagation to minimize it where the correspondence, the feature scale and rotation parameters are solved simultaneously. Our method is effective and produces generally better results.

A paper sliceform or lattice-style pop-up is a form of papercraft that uses two sets of parallel paper patches slotted together to make a foldable structure. The structure can be folded flat, as well as fully opened (popped-up) to make the two sets of patches orthogonal to each other. Automatic design of paper sliceforms is still not supported by existing computational models and remains a challenge. We propose novel geometric formulations of valid paper sliceform designs that consider the stability, flat-foldability and physical realizability of the designs. Based on a set of sufficient construction conditions, we also present an automatic algorithm for generating valid sliceform designs that closely depict the given 3D solid models. By approximating the input models using a set of generalized cylinders, our method significantly reduces the search space for stable and flat-foldable sliceforms. To ensure the physical realizability of the designs, the algorithm automatically generates slots or slits on the patches such that no two cycles embedded in two different patches are interlocking each other. This guarantees local pairwise assembility between patches, which is empirically shown to lead to global assembility. Our method has been demonstrated on a number of example models, and the output designs have been successfully made into real paper sliceforms.

Aligning image pairs with significant appearance change is a long standing computer vision challenge. Much of this problem stems from the local patch descriptors' instability to appearance variation. In this paper we suggest this instability is due less to descriptor corruption and more the difficulty in utilizing local information to canonically define the orientation (scale and rotation) at which a patch's descriptor should be computed. We address this issue by jointly estimating correspondence and relative patch orientation, within a hierarchical algorithm that utilizes a smoothly varying parameterization of geometric transformations. By collectively estimating the correspondence and orientation of all the features, we can align and orient features that cannot be stably matched with only local information. At the price of smoothing over motion discontinuities (due to independent motion or parallax), this approach can align image pairs that display significant inter-image appearance variations.

We present a semi-automatic photographic recomposition approach that employs a semantics-preserving warp of the input image to enhance the visual dominance of the main subjects. Our method uses the tearable image warping method to shift the subjects against the background (and vice versa), so that their visual dominance is improved, and yet preserve the desired spatial semantics between the subjects and the background. The recomposition is guided by a measure of the resulting visual dominance of the main subjects. Our user experiment shows the effectiveness of the approach.

Sand animation is a performance art technique in which an artist tells stories by creating animated images with sand. Inspired by this medium, we have developed a new multi-touch digital artistic medium named SandCanvas that simplifies the creation of sand animations. SandCanvas also goes beyond traditional sand animation with tools for mixing sand animation with video and replicating recorded free-form hand gestures. In this paper, we analyze common sand animation hand gestures, present SandCanvas's intuitive UI, and describe implementation challenges we encountered. We also present an evaluation with professional and novice artists that shows the importance and unique affordances of this new medium.

We propose a single-image, shift-invariant motion deblurring approach where the blur kernel is directly estimated from light streaks in the blurred image. Combining with the sparsity constraint, the blur kernel can be solved quickly and accurately from a user input region containing a light streak. This kernel can then be applied to state-of-the-art single-image motion deblurring methods to restore the sharp image. As our approach does not require verification of the blur kernel against the blurred image, the deblurring can be performed quickly enough for interactive use. For example, our method can be used for interactively revealing scene details in different image regions when the motion blur is not shift-invariant.

A photograph that has visually dominant subjects in general induces stronger aesthetic interest. Inspired by this, we have developed a new approach to enhance image aesthetics through saliency retargeting. Our method alters low-level image features of the objects in the photograph such that their computed saliency measurements in the modified image become consistent with the intended order of their visual importance. The goal of our approach is to produce an image that can redirect the viewers' attention to the most important objects in the image, and thus making these objects the main subjects. Since many modified images can satisfy the same specified order of visual importance, we trained an aesthetics score prediction model to pick the one with the best aesthetics. Results from our user experiments support the effectiveness of our approach.

We present a new screen-space ambient occlusion algorithm (SSAO), which we call Multi-Resolution Screen-Space Ambient Occlusion (MSSAO). The method computes ambient occlusion by combining occlusion values using multiple mipmap levels of a g-buffer. Our method is based on the observation that occlusion due to further-away occluders are lower frequency and thus can be computed in coarser resolutions. On the other hand, nearby occluders cause higher-frequency occlusion that must be computed in finer resolutions. Our method samples occluders in screen space (similar to [Shanmugam and Arikan 2007]) and combines occlusion values across multiple image resolutions. Compared with two state-of-the-art SSAO methods [Filion and McNaughton 2008; Bavoil et al. 2008], results from our algorithm are closer to ray-traced results, even when our algorithm is running at comparable or higher frame rates. Because no random sampling is used, and occlusion values in all levels are combined using edge-preserving blending, our results are free of noise or excessive blurring which are common in other SSAO methods.

Constructive solid geometry (CSG) is a geometric representation where a complex 3D object is represented by combining simple solid objects (called primitives) using Boolean operators. The primitives can be convex or non-convex solids. The basic Boolean operations are union, intersection and subtraction.

We present an approach to automatically register a large set of color images to a 3D geometric model. The problem arises from the modeling of real-world environments, where surface geometry is acquired using range scanners whereas the color information is separately acquired using untracked and uncalibrated cameras. Our approach constructs a sparse 3D model from the color images using a multiview geometry technique. We project special light patterns onto the scene surfaces to increase the robustness of the multiview geometry reconstruction. The sparse model is then approximately aligned with the detailed model. Planes found in the detailed model are exploited to refine the registration. Finally, the registered color images are mapped to the detailed model using weighted blending, with careful consideration of occlusion and the preservation of image details.

We present a saliency-enhanced method for the classification of professional photos and snapshots. First, we extract the salient regions from an image by utilizing a visual saliency model. We assume that the salient regions contain the photo subject. Then, in addition to a set of discriminative global image features, we extract a set of salient features that characterize the subject and depict the subject-background relationship. Our high-level perceptual approach produces a promising 5-fold cross-validation (5-CV) classification accuracy of 78.8%, significantly higher than existing approaches that concentrate mainly on global features.

We introduce and present preliminary results for a hybrid display system combining head-mounted and projector-based displays. Our work is motivated by a surgical training application, where it is necessary to simultaneously provide both a high-fidelity view of a central close-up task (the surgery) and visual awareness of objects and events in the surrounding environment In particular, for trauma surgeons it would be valuable to learn to work in an environment that is realistically filled with both necessary and distracting objects and events. In this paper, we motivate the use of a hybrid display system, discuss previous work, describe a prototype along with methods for geometric calibration, and present results from a controlled human subject experiment.

Current blending methods in image-based rendering use local information such as "deviations from the closest views" to find blending weights. They include approaches such as the view-dependent texture mapping and blending fields used in unstructured lumigraph rendering. However, in the presence of depth discontinuities, these techniques do not provide smooth transitions in the target image if the intensities of corresponding pixels in the source images are significantly different (e.g. due to specular highlights). In this paper, we present an image blending technique that allows the use of global visibility and occlusion constraints. Each blending weight now has a global component and a local component, which are due to the view-independent and view-dependent contributions of the source images, respectively. Being view-independent, the global components can be computed in a preprocessing stage. Traditional graphics hardware is exploited to accelerate the computation of the global blending weights.

We introduce an idea and some preliminary results for a new projector-based approach to re-creating real and imagined sites. Our goal is to achieve re-creations that are both visually and spatially realistic, providing a small number of relatively unencumbered users with a strong sense of immersion as they jointly walkaround the virtual site.Rather than using head-mounted or general-purpose projector-based displays, our idea builds on previous projector-based work on spatially-augmented realityand shader lamps. Using simple white building blocks we construct a static physical model that approximates the size, shape, and spatial arrangementof the site. We then project dynamic imagery onto the blocks, transforming the lifeless physical model into a visually faithful reproduction of the actual site. Some advantages of this approach include wide field-of-view imagery, real walking around the site, reduced sensitivity to tracking errors, reduced sensitivity to system latency, auto-stereoscopic vision, the natural addition of augmented virtualityand the provision of haptics.In addition to describing the major challenges to (and limitations of) this vision, in this paper we describe some short-term solutions and practical methods, and we present some proof-of-concept results.

Proceedings of the Symposium on Interactive 3D Graphics

Proceedings of the Symposium on Interactive 3D Graphics