Many studies of material property estimation and material recognition have been conducted. Previous approaches evaluate the validity or usefulness of hand-designed image features. Thus, we propose a method to directly and naturally acquire image features for material perception using convolutional neural networks. Using a fine-tuned network, we achieved approximately the same recognition accuracy as recent results on the Flickr Material Database. We found that neurons in deep layers are trained to respond to textures or materials.

Much research in augmented reality (AR) technology attempts to match the textures of virtual objects with real world. However, the textures of real objects must also be rendered consistent with virtual information. This paper proposes a method for representing the degradation of real objects in virtual time. Real-world depth information, used to build three-dimensional models of real objects, is captured by a RGB-D camera. The degradation of real objects is then represented by superimposing the degradation texture onto the real object.

This paper proposes an augmented reality marker that can be robustly detected even on a cylindrical surface. The marker enables the surface normal estimation of a cylindrical object to realize the presentation of appropriate virtual information on the object. Conventional markers have difficulty detecting and obtaining accurate surface normal in the presence of occlusion or distortion of the marker in the image. Furthermore, it is difficult to identify a feature on a cylindrical object on which to position a marker. These problems are resolved by relying on the characteristic that a line parallel to the central axis of the cylinder maintains linearity. In addition, surface normal is calculated by estimating the object's shape by using transformation matrices.

This paper describes a novel game system which provides multiple players with a sense of touching fire with their own hands. Players in this game are divided into two teams in front of a wall-type flat display and try to score points by grounding a fireball on the other team's court like volleyball. The players can recognize their contacts with the fireball from mirrored image of their own appearance and the superimposed fireball in the display. Computer detects those contacts by using a real-time time-of-flight camera and renders flame of the fireball based on fluid simulation. The reason why we choose fire as a ball is that several characteristics of fire are advantages in interactions between users and virtual objects. This game can provide players with enough enjoyment and reality due to these advantages even if the implemented human detection algorithm is quite simple.

This paper proposes a method for rendering an inter-reflection between a marker and a glossy floor in Augmented Reality (AR). At first, a reflectance ratio of the floor is estimated from the reflection on the floor and from the marker box directly. Then the roughness of the floor is estimated based on the sharpness of the reflected marker box image on the floor. Lastly, the marker box reflection is eliminated based on the surrounding colors of the marker box reflection. Rendered images show that natural inter-reflection can be achieved by using the proposed method.

Colors in nature can be classified into two categories: pigment colors and structural colors. Structural colors, which are caused by optical path differences of reflected rays in microstructures, change depending on viewing angle and lighting conditions. In the present paper, we propose a generic approach for rendering structural colors in real-time. The proposed method uses optical path differences as common parameters to allow unified treatment of various types of microstructures, such as thin films, multilayer films, and diffraction gratings. To achieve real-time rendering, we store pre-computed information related to specific microstructure and lighting conditions on several kinds of textures. The textures are used as a look-up table in the rendering process. The proposed method can be applied to objects that have arbitrary shape and enables rendering from any viewing direction and under any lighting conditions.

We propose a measurement system of spectral distribution and shape of an object with the use of an imaging spectrograph. Color and shape information are very important for us to recognize objects. A RGB color camera is usually used to measure color, but a real image is a projection of a complex optical phenomenon with spectral distribution. However, there are only few systems measuring spectral distribution as an image. Moreover there is no system which can acquire both spectral distribution and shape information. We propose a system which is able to measure spectral distribution and shape simultaneously with the use of the imaging spectrograph, a turntable and gray code patterns projection.

In this paper we propose a new method of shadowing in image based rendering (IBR). A model-based rendering is the principal method of computer graphics (CG). It needs a perspective transform or light source from three-dimensional models. On the other hand, the method called IBR generates images from multiple real images. It can easily represent objects that are very complex or vague form. And it is less costly for IBR to calculate than such a ray tracing. In this paper, we focus the representation of a shadow that is a weak point in the IBR. Images under arbitrary position of light source can be interpolated from multiple light source images. Moreover, we can synthesize the IBR image to a virtual space and apply to arbitrary view point of IBR object with shadowing. Our shadowing method uses shadow image extracted from a real scene, so it can represent transparent objects.

In this paper, we propose a method for estimation of illuminant spectral distribution using geometrical information. To realize color constancy, it is important to separate the illuminant light color from the reflected light. The light reflected from an object consists of the surface reflection component that is equal to the light-source color, and the body reflection that corresponds the object color. Their intensities vary depending upon the light source, the camera, and the object's geometrical shape information. We estimate the illuminant spectral distribution based on the difference between these intensities, and show effectiveness of this method by experiments.

It is important to express the realism and quality of human skin in computer graphics. We obtain spectral distributions of skin in vivo and attempt to extract the features that show the characteristics of skin by multivariate analysis. Furthermore, we introduce what we learned from the analyses into our proposal model to compute the spectral distributions of virtual skin.

Visual simulation using computer graphics has attracted wide attention in many fields. In this paper, we propose a method of modeling and visualizing pearls to implement a pearl visual simulator. Pearls manifest a very specific optical phenomenon. To investigate this feature, we propose a physical model of blurring and pearl interference. The experimental results show that the physics-based-modeling of internal blurring in the multilayer of a pearl and the partial coherent interference model are effective for high quality pearl visualization. This paper finally describes the possibility that this simulator can be practically used in jewellery shops.

Visual simulation using CG and VR has attracted wide attention in the machine vision field. This paper proposes a method of modeling and visualizing pearls that will be the central technique of a pearl-quality evaluation simulator. Pearls manifest a very specific optical phenomenon that is not dependent on the direction of the light source. To investigate this feature, we propose a physical model, called an illuminant model for multilayer film interference considering the multiple reflection in spherical bodies. The rendering algorithm has been configured from such representations of physical characteristics as interference, mirroring, and texture, which correspond, respectively, to the sense of depth, brightness, and grain that are the main evaluation factors obtained from psychological experiments. Further, portions of photos of real pearls and the images generated by the present method were evaluated based on a scale of psychological evaluations of pearl-like quality demonstrating, thereby, that not merely the generated images as a whole, but the respective parts of images can present such a pearl-like quality.

A new method of recognition of material types is described. To materialize this idea, this paper proposes a simple color reflection model based on a dichromatic reflection model which classifies colored opaque objects into three categories: "metallic", "matte nonmetallic" and "glossy nonmetallic". In addition processing of a spectral image for indoor object recognition is described. An experimental image acquisition system with 2D scanning of a spectrometer is able to generate 2D spectral images, like a multi-band image. As applications of the spectral images, a spectral image is used to recognise material types, and the validity of this method is shown by experiments.

A new color image acquisition system which can measure continuous spectral pixel data in order to apply continuous spectral images to object recognition. For segmentation of continuous spectral image, object contours can be detected by checking the correlation coefficient between neighboring pixels. Also the continuous spectral images can be used as reference image when selecting the most suitable imaging sensor, such as a gray scale camera, a RGB camera or a spectral camera with narrow band-pass filter. When investigating features in spectral distribution, object recognition should be simplified for color objects.