A fast and accurate algorithm is presented to register scans from an RGB-D camera, which rotates and scans the entire scene in an automated fashion on a pan-tilt platform. The proposed algorithm, Axis Bound Registration, exploits the movement of the camera that is bound by the two rotation axes of pan-tilt servos, so as to realize fast and accurate registration of acquired point clouds. The rotation parameters, including the rotation axes, pan-tilt transformations and the servo control mechanism, are calibrated beforehand. Subsequently, fast global registration can be performed during online operation with transformation matrices formed by the calibrated rotation axes and angles. In local registration, features are extracted and matched between two scenes. For robust registration, false-positive correspondences are rejected based on the distances of pre-oriented keypoint pairs, namely the circle of deviation constraint. Then, a more accurate registration can be achieved by minimizing the residual distances between correspondence pairs, while estimated transformations are bound to the rotation axes. Results of comparative experiments validate that the proposed method outperforms state-of-the-art algorithms of various approaches based on camera calibration, global registration, and simultaneous-localization-and-mapping in terms of root-mean-square error and computation time.

Since projection augmented reality (AR) robot can provide a lot of information through projector, it can be useful in museums and art galleries that need to provide information to the crowd. Therefore, it is necessary to continue to interact with people, and human-aware path planning is also needed. We prototyped projection AR mobile robot implemented human-aware path planning and wrote about future research direction.

Many kinds of preparations are needed when meeting. For example, projector, laptop, cables and ETC. As such, this video have constructed Meet AR-bot, which helps users to keep meeting going smoothly, based on the projection of Augmented Reality(AR). Our system can easily provide meeting room environment through the movable setting via wheel-based stand. Users do not need to carry a personal laptop and connect them to the projector. Robot reconstructs the 3D geometry information through pan-tilt system and compute projection areas to project information in the space. Users can also control through mobile devices. We offer presentation, table interaction, file sharing and virtual object registration by mobile device.

Children encounter a variety of experiences through play, which can improve their ability to form ideas and undergo multi-faceted development. Using Augmented Reality (AR) technology to integrate various digital learning elements with real environments can lead to increased learning ability. This study proposes a 360° rotatable and portable system specialized for education and development through projection-based AR play. This system allows existing projection-based AR technology, which once could only be experienced at large-scale exhibitions and experience centers, to be used in individual and small-scale spaces. It also promotes the development of multi-sensory abilities through a multi-modality which provides various intuitive and sensory interactions. By experiencing the various educational play applications provided by the proposed system, children can increase their physical, perceptive, and emotional abilities and thinking skills.

Video conferencing, which helps gather opinions and make decisions quickly among employees who are not in the same location, is now a very important communication tool in the workplace. Our research is one of these video conferencing solutions, specifically proposed to address the difficulties of analog materials sharing and feedback, and has added some useful features for the smooth use of conference participants. We conducted a comparative experiment on our proposed method of file sharing and the method that we had previously used in video conferencing. As a result, the proposed system yielded better results in terms of time and usability during a full-scale collaborative situation in which feedback was provided.

We propose a realtime plane detection method for projection-based Augmented Reality (AR) system in an unknown environment. While previous works usually designate space, the plane detection method automatically detects multiple planes based on the proposed constrained sampling strategy in RAndom SAmpling Concensus (RANSAC). For each plane, an area for projection is selected for contents while considering occlusions by other objects. In addition, when the multiple planes are detected, the importance for contents is measured by the score functions based on the properties of planes such as size, color, and position. The proposed method can guide users to select plane for projection by visualizing the importances, or can automatically select a plane according to the users. We achieves a significant improvement in speed (about 260 times faster than the RANSAC) and high precision. These technique has become widely utilized in various AR applications such as AR game, and etc.

We present a method to assist pre-touch on multi-touch surfaces using features extracted from the fingertips of the user's hands. Pixel area a feature obtained from the contour of a fingertip is computed and tracked using commodity cameras to obtain the velocity of approaching fingers. We target sound volume control for digital multi-touch instruments an area where pre-touch is demonstrated with our approach to approximating velocity of finger taps. We explore other areas of application with velocity and how it can help anticipate user actions in mobile touch displays. In order to evaluate the effectiveness of the proposed system, a system prototype was presented to users. We confirmed the satisfactory results by performing the recognition rate experiment for measuring the blob size according to the touch speed.

This research paper proposes a system for interior design prototyping based on portable-type projection-based AR (Augmented Reality). The DIY (do it yourself) interior designing process takes much time, monetary resources and labor. To overcome such difficulties, many research studies in the have been undertaken in the field of AR. Among those studies, projection-based AR technology provides users virtual information that is very real by projecting new digital information onto things and spaces. Most of those studies employing projection-based AR technology have been carried out in predefined spaces. In this paper, a projection-based AR system that can be used to design interiors of non-predefined spaces was proposed. This DesignAR system can effectively design interiors, even in the spaces not predefined by a 3D map of the space. Also, by connecting with mobile devices, the system raises the user's degree of freedom due to the interface usability, mobility and overall convenience. In order to evaluate the effectiveness of the proposed system, we set up a scenario for interior design prototyping. A usability evaluation was conducted by questionnaire after users performed tasks based on a preset scenario.

In this paper, a new object manipulation method is proposed by applying the ray's depth information to the ray-casting-based interface. Ray-casting has been widely used in handheld device interfaces for manipulating objects in three-dimensional (3D) space, such as augmented reality or virtual reality environments. However, the traditional ray-casting interface has limited object manipulation because it cannot designate a specific location in 3D space; it also has poor accuracy in manipulating objects at far distances. In this study, a user could register the virtual object in the real 3D space by specifying the arbitrary position using ray-depth information. We also improved the virtual object manipulation accuracy of the ray-casting interface by compensating the sensitivity according to the distance between the user and the object. To verify the manipulation accuracy of the interface implemented with the proposed technology, this study conducted a comparative experiment with existing interfaces. This verified the performance improvement in 3D space.

Augmented reality (AR) is widely used in various applications of computer vision, such as marker-based AR and markerless-based AR. These AR techniques are used in various fields, including industry, education, and medicine. Using marker-based AR, employees can easily perform step-by-step maintenance and repairs, and they can register parts information for large plants. However, conventional marker-based AR relies on a relatively small number of recognizable IDs compared to barcode markers. In this paper, to address the insufficient identification volume in conventional AR systems, we integrate barcode-based code technology with marker-based AR technology. Based on the results of an experiment, we applied ColorCode to our marker-based AR system. Nevertheless, difficulties arise when applying ColorCode to an AR system, owing to its recognition distance and relatively small size, compared to other AR codes. In this paper, therefore, we complemented quad detection with a tracking technique for various angles and distances, facilitating reliable recognition of the color-code-based AR system, Moreover, we added a tracking module to address the system's failure to detect markers. The experimental results demonstrate that the proposed system offers stable recognition.

The proposed system is a user-friendly interactive Augmented Reality Authoring System, which is for users with basic knowledge of augmented reality or without programming skills. The user can wear the head-mounted display and easily create AR content by using sensors on mobile devices. The user can also directly apply a variety of methods to interact with the AR content. Interaction with the AR content is enabled by using the user's hand, which is recognized by the camera attached to the HMD, and the mobile device's sensor data. In order to evaluate the effectiveness of the proposed system, a system prototype was presented to users. A usability test was conducted with regard to AR content production and interactive tasks.

We present a hybrid architecture, inspired by asynchronous BVH construction [1], for ray tracing animated scenes. Our hybrid architecture utilizes heterogeneous hardware resources: dedicated ray-tracing hardware for BVH updates and ray traversal and a CPU for BVH reconstruction. We also present a traversal scheme using a primitive's axis-aligned bounding box (PrimAABB). This scheme reduces ray-primitive intersection tests by reusing existing BVH traversal units and the primAABB data for tree updates; it enables the use of shallow trees to reduce tree build times, tree sizes, and bus bandwidth requirements. Furthermore, we present a cache scheme that exploits consecutive memory access by reusing data in an L1 cache block. We perform cycle-accurate simulations to verify our architecture, and the simulation results indicate that the proposed architecture can achieve real-time Whitted ray tracing animated scenes at 1,920 × 1,200 resolution. This result comes from our high-performance hardware architecture and minimized resource requirements for tree updates.

This article consists of a collection of slides from the author's conference presentation on the special features, system design and architectures, processing capabilities, and targeted markets for SiliconArts' RayChip, the world's first commercialized chip targeted to realize real-time ray tracing for embedded applications such as TV, media box and game console.

Today, people use a computer almost everywhere. At the same time, they still do their work in the old-fashioned way, such as using a pen and paper. A pen is often used in many fields because it is easy to use and familiar. On the other hand, however, it is a quite inconvenient because the information printed on paper is static. If digital features are added to this paper environment, the users can do their work more easily and efficiently. AR (augmented reality) Lamp is a stand-type projector and camera embedded system with the form factor of a desk lamp. Its users can modify the virtually augmented content on top of the paper with seamlessly combined virtual and physical worlds. AR is quite appealing, but it is difficult to popularize due to the lack of interaction. In this paper, the interaction methods that people can use easily and intuitively are focused on. A high-fidelity prototype of the system is presented, and a set of novel interactions is demonstrated. A pilot evaluation of the system is also reported to explore its usage possibility.

We present RayCore, a mobile ray-tracing hardware architecture. RayCore facilitates high-quality rendering effects, such as reflection, refraction, and shadows, on mobile devices by performing real-time Whitted ray tracing. RayCore consists of two major components: ray-tracing units (RTUs) based on a unified traversal and intersection pipeline and a tree-building unit (TBU) for dynamic scenes. The overall RayCore architecture offers considerable benefits in terms of die area, memory access, and power consumption. We have evaluated our architecture based on FPGA and ASIC evaluations and demonstrate its performance on different benchmarks. According to the results, our architecture demonstrates high performance per unit area and unit energy, making it highly suitable for use in mobile devices.

The parallel scan is a basic tool that is used to parallelize algorithms which appear to have serial dependencies. The performance of these algorithms relies heavily on the efficiency of the parallel scan that is being used. To maintain work efficiency, current parallelization methods either sacrifice the overall depth or limit the scalability. In this study, we present a parallel scan method that is derived from the Han-Carlson parallel prefix graph and is both a work-efficient and a depth-optimal process. In this method, the depth is increased by a small constant value above the lower bound; therefore, the amount of computation and/or memory access is effectively reduced. We also employ a novel cascaded thread-block execution method to exploit the single-program-multiple-data (SPMD) nature of the compute unified device architecture (CUDA) environment developed by NVIDIA. The proposed method facilitates the low-latency interthread accessible shared memory and the single-instruction-multiple-thread (SIMT) characteristics of the graphics hardware to reduce high-latency global memory access and costly barrier synchronization. Our experimental results demonstrate an average speed up of approximately 40 and 10 percent over the CUDA data parallel primitives (CUDPP) library derivation of the Kogge-Stone prefix tree and an implementation of Merrill and Grimshaw's method with coarser combination of the Kogge-Stone graph and the Brent-Kung prefix graph, respectively.

The residential environment is getting smarter, and home appliances have become connected each other automatically. Accordingly, user interface also has become increasingly complicated. In this paper, we found the main causes of making the users uncomfortable at home by task analysis, and then suggest a new interactive system suitable to the smart home. Ambient Wall interface enables users to monitor what's happening in their house at a glance, and control their surroundings by simple gesture without any physical interface device.

Ray tracing naturally supports high-quality global illumination effects, but it is computationally costly. Traversal and intersection operations dominate the computation of ray tracing. To accelerate these two operations, we propose a hardware architecture integrating three novel approaches. First, we present an ordered depth-first layout and a traversal architecture using this layout to reduce the required memory bandwidth. Second, we propose a three-phase ray-triangle intersection architecture that takes advantage of early exit. Third, we propose a latency hiding architecture defined as the ray accumulation unit. Cycle-accurate simulation results indicate our architecture can achieve interactive distributed ray tracing.

Three-dimensional user interfaces on mobile devices are increasingly important. For more realistic three-dimensional visualization on mobile devices, we present the implementation of an OpenGL ES-based CPU-GPU hybrid ray tracer. This ray tracer exploits the availability of CPU and GPU architectures to fully support reflection, refraction, hard shadows, and dynamic scenes. To the best of our knowledge, our ray tracer is the first to demonstrate full Whitted ray tracing of dynamic scenes using OpenGL ES.

We present an ordered depth-first tree layout for ray tracing. Among two child nodes, a child node with the larger surface area is stored next to its parent node. Hence, the probabilities that a ray accesses to the same cache line increase. Our approach can be easily and widely used for various ray tracing systems with very small overheads, as it is based on existing depth-first layouts.

Detection technology is a requirement for an Augmented Reality system. One of the problems with detection technology is the instability problem, which occurs when an obstacle occludes a tag while detecting the tag, and the augmented object suddenly disappears. We have proposed a corner detection algorithm to solve this instability problem. The key feature is that if the tag can recognize its position using its four corner cells despite the obstacle being present, then it can maintain its augmented object. We defined the corner case for all types of cases where the instability problem occurs in ARToolkit or ARTag. We have adapted our proposed algorithm to the corner case in ARToolkit, ARTag and ColorCode vision systems and have compared their false detection rates.

As the complexity of 3D scenes is on the increase, the search for an effective visibility culling method has become one of the most important issues to be addressed in the design of 3D rendering processors. Here, we propose a new rasterization pipeline with visibility culling; the proposed architecture performs the visibility culling at an early stage of the rasterization pipeline (especially at the traversal stage) by retrieving data in a pixel cache without any significant hardware logics such as the hierarchical z-buffer. If the data to be retrieved does not exist in the pixel cache, the proposed architecture performs a prefetch operation in order to reduce the miss penalty of the pixel cache. That is, the cache miss penalty can be reduced as the transfer of a missed cache block from the frame memory into the pixel cache can be handled simultaneously with the rasterization pipeline executions. Simulation results show that the proposed architecture can achieve a performance gain of about 32% compared with the conventional pretexturing architecture and about 7% compared to the hierarchical z-buffer visibility scheme.

Current pipelinable dividers require very large lookup tables. We propose a cost-effective pipelinable divider that uses a modified Taylor-series expansion and has a smaller lookup table than other pipelinable dividers. The proposed divider requires about 27 percent less area than the pipelinable divider based on normal Taylor-series expansion in single precision.

As a 3D scene becomes increasingly complex and the screen resolution increases, the design of an effective memory architecture is one of the most important issues for 3D rendering processors. We propose a pixel rasterization architecture that performs the depth test twice, before and after texture mapping. The proposed architecture eliminates memory bandwidth waste due to fetching unnecessary obscured texture data by performing the depth test before texture mapping. It also reduces the miss penalties of the pixel cache by using a prefetch scheme-that is, a frame memory access, due to a cache miss at the first depth test, is done simultaneously with texture mapping. We have built a trace-driven simulator for the proposed architecture. To validate the proposed architecture, the results of various simulations are provided. The proposed pixel rasterization architecture achieves memory bandwidth effectiveness and reduces power consumption while producing high-performance gains.

As a 3D scene becomes increasingly complex and the screen resolution increases, the design of effective memory architecture is one of the most important issues for 3D rendering processors. We propose a pixel rasterization architecture, which performs a depth test operation twice, before and after texture mapping. The proposed architecture eliminates memory bandwidth waste caused by fetching unnecessary obscured texture data, by performing the depth test before texture mapping. The proposed architecture reduces the miss penalties of the pixel cache by using a pre-fetch scheme - that is, a frame memory access, due to a cache miss at the first depth test, is done simultaneously with texture mapping. The proposed pixel rasterization architecture achieves memory bandwidth effectiveness and reduces power consumption, producing high-performance gains.

The CalmRISC32 FPU (Floating-Point Unit) is a RISC coprocessor for embedded system applications. It supports IEEE-754 standard single precision floating-point addition, floating-point subtraction, floating-point multiplication, floating-point division, format conversion, comparison, rounding, load, store, etc. It also supports four rounding modes, and precise exception. It can execute and complete instructions out of order, if constraints such as data dependency, resource conflict, and exception prediction are resolved. Standard cell-base design techniques were used to reduce design time and expense. The first prototype operated at approximately 70 MHz with the worst-case delay in gate level simulation.

An approach for designing a hybrid parallel system that can perform different levels of parallelism adaptively is presented. An adaptive parallel computer vision system (APVIS) is proposed to attain this goal. The APVIS is constructed by integrating two different types of parallel architectures, i.e. a multiprocessor based system (MBS) and a memory based processor array (MPA), tightly into a single machine. One important feature in the APVIS is that the programming interface to execute data parallel code onto the MPA is the same as the usual subroutine calling mechanism. Thus the existence of the MPA is transparent to the programmers. This research is to design an underlying base architecture that can be optimally executed for a broad range of vision tasks. A performance model is provided to show the effectiveness of the APVIS. It turns out that the proposed APVIS can provide significant performance improvement and cost effectiveness for highly parallel applications having a mixed set of parallelisms. Also an example application composed of a series of vision algorithms, from low-level and medium-level processing steps, is mapped onto the MPA. Consequently, the APVIS with a few or tens of MPA modules can perform the chosen example application in real time when multiple images are incoming successively with a few seconds inter-arrival time.

In this paper an effective memory-processor integrated architecture, called memory based processor array (MPA), for computer vision is proposed. The MPA can be easily attached into any host system via memory interface. In order to measure the impact of the memory interface structure an analytical model is derived. The performance improvement on the proposed model for the memory interface architecture of the MPA system can be 6%/spl sim/40% for vision tasks consisting of sequential and data parallel tasks. The asymptotic time complexities of the mapping algorithms are evaluated to verify the cost-effectiveness and the efficiency of the MPA system.

In this paper, we study area-time tradeoffs in VLSI for prefix computation using graph representations of this problem. Since the problem is intimately related to binary addition, the results we obtain lead to the design of area-time efficient VLSI adders. This is a major goal of our work: to design very low latency addition circuitry that is also area efficient. To this end, we present a new graph representation for prefix computation that leads to the design of a fast, area-efficient binary adder. The new graph is a combination of previously known graph representations for prefix computation, and its area is close to known lower bounds on the VLSI area of parallel prefix graphs. Using it, we are able to design VLSI adders having area A = 0(n log n) whose delay time is the lowest possible value, i. e. the fastest possible area-efficient VLSI adder.