Visualizing objects as they are perceived in the real world is often critical in our daily experiences. We previously focused on objects' surface glossiness visualized with a 3D display and found that a multi-view 3D display reproduces perceived glossiness more accurately than a 2D display. This improvement of glossiness reproduction can be explained by the fact that a glossy surface visualized by a multi-view 3D display appropriately provides luminance differences between the two eyes and luminance changes accompanying the viewer's lateral head motion. In the present study, to determine the requirements of a multi-view 3D display for the accurate reproduction of perceived glossiness, we developed a simulator of a multi-view 3D display to independently and simultaneously manipulate the viewpoint interval and the magnitude of the optical inter-view crosstalk. Using the simulator, we conducted a psychophysical experiment and found that glossiness reproduction is most accurate when the viewpoint interval is small and there is just a small (but not too small) amount of crosstalk. We proposed a simple yet perceptually valid model that quantitatively predicts the reproduction accuracy of perceived glossiness.

Music is known to be a powerful elicitor of emotions. Here, we examined the possibility of decoding music-induced experienced emotions - affective responses induced in the listener by music stimuli - using functional magnetic resonance imaging (fMRI) data collected while participants listened to various movie soundtrack excerpts. Using a machine learning based classification framework, preliminary results indicate that it is possible to, under certain conditions, decode the arousal (High/Low) and valence (Positive/Negative) of the emotional state experienced during listening. Future work will be aimed at improving methodological issues in order to establish whether these results extend to larger populations and more general types of music stimuli.

Implantable brain machine interfaces (BMI) enable severely disabled people high-performance real-time robot control and communication, utilizing high-quality intracranial neural signals. Electrocorticograms (ECoG) are useful for implantable BMIs because of not only their zero time-lag property but their high spatiotemporal resolution with long term stability also. Fully implantable devices for ECoG recording offer long-term home-use with 24/7 supports. This will help not only patients with restoring motor and communication control but also help their caregivers with reducing burdens of caregiving day and night. Until now, we established ECoG-based robot control and communication. High gamma activity (80-150 Hz) was a good decoding feature for ECoG-based real time decoding and control. Independent component analyses effectively extract neural information with dimensional reduction and contribute to improving decoding accuracy. Also, we are developing a 128-channel fully-implantable BMI device (WHERBS) for long-term home-use with 24/7 supports. We completed GLP tests and non-clinical long-term implantation. The next step is a clinical trial to confirm safety and efficacy of the implantable BMI.

The aim of this study is to comparatively review and evaluate three variants of the glottal inverse filtering algorithm based on iterative adaptive inverse filtering (IAIF): the Standard algorithm, and two recently proposed variants that use iterative optimal preemphasis (IOP) and a glottal flow model (GFM), respectively. To enable an objective evaluation, a computational physical model of voice production is used to generate time-domain signals pertaining to both the input glottal flow and the output speech pressure, for a wide range of vowels, fundamental frequencies, and voice qualities (involving co-variation of phonation type and loudness). Furthermore, for a fair comparison, the three key parameters of IAIF are selected by an exhaustive search to minimize the root-mean-square error between the estimated and reference glottal flow derivative in each analyzed frame and performance is assessed with two time-domain and two frequency-domain error measures. A conventional evaluation is also carried out with fixed parameter values determined by cross-validation. Results indicate that IOP tends to yield the lowest errors for nonback vowels (reducing errors by 31% on average compared with Standard), especially for not too high fundamental frequencies and not too pressed voice qualities; GFM becomes competitive for normal phonations when fixed parameter values are used; and in other cases, Standard IAIF is still recommended. In addition, the results suggest that not only the overall spectral tilt (as controlled by IOP and GFM) but also the balance between the levels of different spectral regions, can be important for accurate estimation of the glottal flow.

We present a demo of low-cost mixed reality telepresence system that performs real-time 3D reconstruction of a person or an object and wirelessly transmits the reconstructions to Microsoft's HoloLens head mounted display at frame rates perceived as smooth. A reconstructed frame is represented as a polygonal mesh with polygons textured with high definition data obtained from RGB cameras. Each frame is compressed and sent to HoloLens, so that it can be locally rendered by its GPU, minimizing latency of reacting to head movement. Owing to HoloLens half-translucent displays, the system creates an appearance of remote object being part of user's physical environment. The system can run on a relatively low-cost commodity hardware, such as Kinect sensors and in the most basic scenario can produce smooth frame rates of a single multicore laptop.

This paper presents an unsupervised learning scheme for categorizing 3D objects from their 2D projected images. The scheme exploits an auto-associative network's ability to encode each view of a single object into a representation that indicates its view direction. We propose two models that employ different classification mechanisms; the first model selects an auto-associative network whose recovered view best matches the input view, and the second model is based on a modular architecture whose additional network classifies the views by splitting the input space nonlinearly. We demonstrate the effectiveness of the proposed classification models through simulations using 3D wire-frame objects.