This paper introduces a novel method for detailed 3D shape reconstruction utilizing thermal polarization cues. Unlike state-of-the-art methods, the proposed approach is independent of illumination and material properties. In this paper, we formulate a general theory of polarization observation and show that long-wave infrared (LWIR) polarimetric imaging is free from the ambiguities that affect visible polarization analyses. Subsequently, we propose a method for recovering detailed 3D shapes using multi-view thermal polarimetric images. Experimental results demonstrate that our approach effectively reconstructs fine details in transparent, translucent, and heterogeneous objects, outperforming existing techniques.

In this paper, we propose a new cue of depth sensing using thermal radiation. Our method realizes passive, texture independent, far range, and dark scene applicability, which can broaden the depth sensing subjects. A key ob-servation is that thermal radiation is attenuated by the air and is wavelength dependent. By modeling the wavelength-dependent attenuation by the air and building a multi-spectral LWIR measurement system, we can jointly estimate the depth, temperature, and emissivity of the target. We analytically show the capability of the thermal radiation cue and show the effectiveness of the method in real-world scenes using an imaging system with a few bandpass filters.

Abstract We propose a method for eliminating the temporal illumination variations in whisk-broom (point-scan) hyperspectral imaging. Whisk-broom scanning is useful for acquiring a spatial measurement using a pixel-based hyperspectral sensor. However, when it is applied to outdoor cultural heritages, temporal illumination variations become an issue due to the lengthy measurement time. As a result, the incoming illumination spectra vary across the measured image locations because different locations are measured at different times. To overcome this problem, in addition to the standard raster scan, we propose an additional perpendicular scan that traverses the raster scan. We show that this additional scan allows us to infer the illumination variations over the raster scan. Furthermore, the sparse structure in the illumination spectrum is exploited to robustly eliminate these variations. We quantitatively show that a hyperspectral image captured under sunlight is indeed affected by temporal illumination variations, that a Naïve mitigation method suffers from severe artifacts, and that the proposed method can robustly eliminate the illumination variations. Finally, we demonstrate the usefulness of the proposed method by capturing historic stained-glass windows of a French cathedral.

We present a novel time-resolved light transport decomposition method using thermal imaging. Because the speed of heat propagation is much slower than the speed of light propagation, the transient transport of far infrared light can be observed at a video frame rate. A key observation is that the thermal image looks similar to the visible light image in an appropriately controlled environment. This implies that conventional computer vision techniques can be straightforwardly applied to the thermal image. We show that the diffuse component in the thermal image can be separated, and therefore, the surface normals of objects can be estimated by the Lambertian photometric stereo. The effectiveness of our method is evaluated by conducting real-world experiments, and its applicability to black body, transparent, and translucent objects is shown.

The recent development of 3D printing technology has brought concerns about its potential misuse, such as in copyright infringement and crimes. Although there have been many studies on blind 3D mesh watermarking for the copyright protection of digital objects, methods applicable to 3D printed objects are rare. In this paper, we propose a novel blind watermarking algorithm for 3D printed objects with applications for copyright protection, traitor tracing, object identification, and crime investigation. Our method allows us to embed a few bits of data into a 3D-printed object and retrieve it by 3D scanning without requiring any information about the original mesh. The payload is embedded on the object's surface by slightly modifying the distribution of surface norms, that is, the distance between the surface and the center of gravity. It is robust to resampling and can work with any 3D printer and scanner technology. In addition, our method increases the capacity and resistance by subdividing the mesh into a set of bins and spreading the data over the entire surface to negate the effect of local printing artifacts. The method's novelties include extending the vertex norm histogram to a continuous surface and the use of 3D moments to synchronize a watermark signal in a 3D-printing context. In the experiments, our method was evaluated using a public dataset against center, orientation, minimum and maximum norm misalignments; a printing simulation; and actual print/scan experiments using a standard 3D printer and scanner.

This paper presents a method of passive non-line-of-sight (NLOS) imaging using polarization cues. A key observation is that the oblique light has a different polarimetric signal. It turns out this effect is due to the polarization axis rotation, a phenomena which can be used to better condition the light transport matrix for non-line-of-sight imaging. Our analysis and results show that the use of a polarization for NLOS is both a standalone technique, as well as an enhancement technique to boost the results of other forms of passive NLOS imaging. We make a surprising finding that, despite 50% light attenuation from polarization optics, the gains from polarized NLOS are overall superior to unpolarized NLOS.

We propose a new blind watermarking algorithm for 3D printed objects that has applications in metadata embedding, robotic grasping, counterfeit prevention, and crime investigation. Our method can be used on fused deposition modeling (FDM) 3D printers and works by modifying the printed layer thickness on small patches of the surface of an object. These patches can be applied to multiple regions of the object, thereby making it resistant to various attacks such as cropping, local deformation, local surface degradation, or printing errors. The novelties of our method are the use of the thickness of printed layers as a one-dimensional carrier signal to embed data, the minimization of distortion by only modifying the layers locally, and one-shot detection using a common paper scanner. To correct encoding or decoding errors, our method combines multiple patches and uses a 2D parity check to estimate the error probability of each bit to obtain a higher correction rate than a naive majority vote. The parity bits included in the patches have a double purpose because, in addition to error detection, they are also used to identify the orientation of the patches. In our experiments, we successfully embedded a watermark into flat surfaces of 3D objects with various filament colors using a standard FDM 3D printer, extracted it using a common 2D paper scanner and evaluated the sensitivity to surface degradation and signal amplitude.

Phase ambiguity is a major problem in the depth measurement in either time-of-flight or phase shifting. Resolving the ambiguity using a low frequency pattern sacrifices the depth precision, and using multiple frequencies requires a number of observations. In this paper, we propose a phase disambiguation method that combines temporal and spatial modulation so that the high depth precision is preserved while the number of observation is small. A key observation is that the phase ambiguities of temporal and spatial domains appear differently with respect to the depth. Using this difference, the phase can disambiguate for a wider range of interest. We develop a prototype to show the effectiveness of our method through real-world experiments.

In this paper, we present a practical method for reconstructing the bidirectional reflectance distribution function (BRDF) from multiple images of a real object composed of a homogeneous material. The key idea is that the BRDF can be sampled after geometry estimation using multi-view stereo (MVS) techniques. Our contribution is selection of reliable samples of lighting, surface normal, and viewing directions for robustness against estimation errors of MVS. Our method is quantitatively evaluated using synthesized images and its effectiveness is shown via real-world experiments.

This paper presents a material classification method using an off-the-shelf Time-of-Flight (ToF) camera. The proposed method is built upon a key observation that the depth measurement by a ToF camera is distorted for objects with certain materials, especially with translucent materials. We show that this distortion is due to the variation of time domain impulse responses across materials and also due to the measurement mechanism of the ToF cameras. Specifically, we reveal that the amount of distortion varies according to the modulation frequency of the ToF camera, the object material, and the distance between the camera and object. Our method uses the depth distortion of ToF measurements as a feature for classification and achieves material classification of a scene. Effectiveness of the proposed method is demonstrated by numerical evaluations and real-world experiments, showing its capability of material classification, even for visually indistinguishable objects.

We present a novel time-resolved light transport decomposition method using thermal imaging. Because the speed of heat propagation is much slower than the speed of light propagation, transient transport of far infrared light can be observed at a video frame rate. A key observation is that the thermal image looks similar to the visible light image in an appropriately controlled environment. This implies that conventional computer vision techniques can be straightforwardly applied to the thermal image. We show that the diffuse component in the thermal image can be separated and, therefore, the surface normals of objects can be estimated by the Lambertian photometric stereo. The effectiveness of our method is evaluated by conducting real-world experiments, and its applicability to black body, transparent, and translucent objects is shown.

This paper presents a material classification method using an off-the-shelf Time-of-Flight (ToF) camera. We use a key observation that the depth measurement by a ToF camera is distorted in objects with certain materials, especially with translucent materials. We show that this distortion is caused by the variations of time domain impulse responses across materials and also by the measurement mechanism of the existing ToF cameras. Specifically, we reveal that the amount of distortion varies according to the modulation frequency of the ToF camera, the material of the object, and the distance between the camera and object. Our method uses the depth distortion of ToF measurements as features and achieves material classification of a scene. Effectiveness of the proposed method is demonstrated by numerical evaluation and real-world experiments, showing its capability of even classifying visually similar objects.

This paper describes a method for recovering appearance of inner slices of translucent objects. The appearance of a layered translucent object is the summed appearance of all layers, where each layer is blurred by a depth-dependent point spread function (PSF). By exploiting the difference of low-pass characteristics of depth-dependent PSFs, we develop a multi-frequency illumination method for obtaining the appearance of individual inner slices. Specifically, by observing the target object with varying the spatial frequency of checker-pattern illumination, our method recovers the appearance of inner slices via computation. We study the effect of non-uniform transmission due to inhomogeneity of translucent objects and develop a method for recovering clear inner slices based on the pixel-wise PSF estimates under the assumption of spatial smoothness of inner slice appearances. We quantitatively evaluate the accuracy of the proposed method by simulations and qualitatively show faithful recovery using real-world scenes.

This paper presents a method for recovering shape and normal of a transparent object from a single viewpoint using a Time-of-Flight (ToF) camera. Our method is built upon the fact that the speed of light varies with the refractive index of the medium and therefore the depth measurement of a transparent object with a ToF camera may be distorted. We show that, from this ToF distortion, the refractive light path can be uniquely determined by estimating a single parameter. We estimate this parameter by introducing a surface normal consistency between the one determined by a light path candidate and the other computed from the corresponding shape. The proposed method is evaluated by both simulation and real-world experiments and shows faithful transparent shape recovery.

In this paper, we describe a supervised four-dimensional (4D) light field segmentation method that uses a graph-cut algorithm. Since 4D light field data has implicit depth information and contains redundancy, it differs from simple 4D hyper-volume. In order to preserve redundancy, we define two neighboring ray types (spatial and angular) in light field data. To obtain higher segmentation accuracy, we also design a learning-based likelihood, called objectness, which utilizes appearance and disparity cues. We show the effectiveness of our method via numerical evaluation and some light field editing applications using both synthetic and real-world light fields.

This paper describes a method for recovering appearance of inner slices of translucent objects. The outer appearance of translucent objects is a summation of the appearance of slices at all depths, where each slice is blurred by depth-dependent point spread functions (PSFs). By exploiting the difference of low-pass characteristics of depth-dependent PSFs, we develop a multi-frequency illumination method for obtaining the appearance of individual inner slices using a coaxial projector-camera setup. Specifically, by measuring the target object with varying the spatial frequency of checker patterns emitted from a projector, our method recovers inner slices via a simple linear solution method. We quantitatively evaluate accuracy of the proposed method by simulations and show qualitative recovery results using real-world scenes.

The inner structures of an object can be measured by capturing transmissive images. However, the recorded images of a translucent object tend to be unclear due to strong scattering of light inside the object. In this paper, we propose a descattering approach based on Parallel High-frequency Illumination. We show in this paper that the original high-frequency illumination method and the various extended techniques can be uniformly defined as a separation of overlapped and non-overlapped light rays. Also, we show that transmissive light rays do not overlap each other by constructing a parallel projection/measurement system for performing both illumination and observation. We have developed a measurement system that consists of a camera and projector with telecentric lenses and have evaluated descattering effects by extracting transmissive light rays.

As network technologies evolve in two areas-peer-to-peer technology and nomadic technology-new types of services provided through a ubiquitous environment will be created. To allow users to enjoy services in a ubiquitous environment, services that are appropriate to users' needs and desires must be created and discovered. In this paper, we describe the ubiquitous service-oriented network (USON). The USON is a network architecture for providing future networking services based on ubiquitous technologies. It is based on three technologies-service composition core technology, state-acquisition technology, and network reflective technology. Services provided through the USON will consist of autonomous distributed service elements (SEs). We describe the architecture, implementation, and an example of the USON, and explain how it will provide user services through a ubiquitous environment.

Most current interface designs require that the user focus their attention on them in order to be of value. However, as the price of computation falls, and computational capabilities make their way into many everyday objects, the demand for attention from many different directions may begin to seriously reduce the usefulness of these computational objects. Ambient information displays are intended to fit in a part of the interface design space that does not have this property. They are designed to convey background or context information that the user may or may not wish to attend to at any given time. Ambient Displays are designed to work primarily in the periphery of a user's awareness, moving to the center of attention only when appropriate and desirable. This paper describes a new ambient information display that is designed to give a rich medium of expression placed within an aesthetically pleasing decorative object. This display — the Information Percolator — is formed by air bubbles rising up tubes of water. By properly controlling the release of air, a set of pixels which scroll up the display is created. This allows a rendition of any (small, black and white) image to be displayed. The detailed design and construction of this display device will be considered, along with several applications.

It is well known that paper is a very fluid, natural, and easy to use medium for manipulating some kinds of information. It is familiar, portable, flexible, inexpensive, and offers good readability properties. Paper also has well known limitations when compared with electronic media. Work in hybrid paper electronic interfaces seeks to bring electronic capabilities to real paper in order to obtain the best properties of each. This paper describes a hybrid paper electronic system — the Paper PDA — which is designed to allow electronic capabilities to be employed within a conventional paper notebook, calendar, or organizer. The Paper PDA is based on a simple observation: a paper notebook can be synchronized with a body of electronic information much like an electronic PDA can be synchronized with information hosted on a personal computer. This can be accomplished by scanning, recognizing and processing its contents, then printing a new copy. This paper introduces the Paper PDA concept and considers interaction techniques and applications designed to work within the Paper PDA. The StickerLink technique supports on-paper hyperlinking using removable paper stickers. Two applications are also considered which look at aspects of electronic communications via the Paper PDA.

This paper describes and discusses the modeling of multimedia communication networking architecture suitable to high speed networks such as LAN, B-ISDN, etc. for the flexible management and rapid introduction of services. In this model, the applications are deployed in units of software building blocks called group call server, service control server, resource management server and client agent. Each building block provides a layered view for the effective management of the multimedia network resources and services. In this paper, we propose a service scenario scheme and multimedia logical devices model for the flexible generation and effective management of services, and abstracting the network resources, respectively. Finally, this paper presents an architecture evolution model toward interoperability in a distributed processing environment (DPE). This can be applied for interoperability between multimedia network resources provided by different vendors.>