Touch Emotion Recognition (TER) is an emerging field in affective computing that focuses on interpreting human emotions through touch interactions. This survey provides a comprehensive review of the haptic technologies, methodologies, and applications of TER, emphasizing its role as a rich but underexplored modality for emotion recognition. A new taxonomy for TER is proposed, categorizing research based on recognition entities, recognition approaches, and touch gesture types, offering a structured perspective for advancing TER studies. The survey highlights the importance of affective touch gesture datasets, evaluates the hardware technologies underpinning TER systems, and reviews key methods for data processing and classification. Applications across domains such as emotional AI, social robotics, virtual reality, and healthcare are examined, showcasing the potential of TER in enhancing the emotional intelligence of artificial agents and robotic systems. Finally, challenges including dataset scarcity, device standardization, and cultural variability are discussed, alongside future directions for creating more robust and inclusive TER systems.

Nighttime driving presents significant challenges, including detecting pedestrians, cyclists, and potholes in low-visibility conditions, leading to frequent road accidents. In Sri Lanka, there is a lack of automated systems capable of addressing these challenges. This research proposes a system leveraging AI and real-time video processing to enhance driver safety by improving pedestrian and cyclist visibility and developing a reliable pothole detection system for night time. These enhancements rely on extensive data collection and preprocessing for training specialized computer vision and machine learning models tailored for real-time, low-light scenarios. The application provides real-time alerts and visual cues to reduce nighttime driving accidents. Additionally, a software framework is proposed to offer real-time feedback for drivers leveraging available hardware capabilities, enhancing awareness of vulnerable road users. Through rigorous data collection, testing, and development, this approach significantly helps to enhance nighttime driving safety in Sri Lanka.

English as Second Language (ESL) learning widely implemented in the global education system as English is essential for workplace, diplomate communication, and contractual matters. The effective teaching and learning of ESL since early age provide the students competitive advantages. Deployment of multimedia such as Augmented Reality (AR) in ESL had been implemented for this purpose. This study used multimedia learning environment AR English Vocabulary Acquisition (ARenVA) to test its effectiveness of learning among 37 Malaysia primary school students from 4th, 5th, and 6thgrade. Explanatory research was conducted with instruments namely visual text and AR 2D (ARvt), spoken text and AR 2D (ARst), visual text, spoken text and AR 2D (ARvtst). Interview, observation, pre-test/post-test questionnaire and English test were used to identify students' perceived motivation and English performance. The perceived motivation is in relation with Keller's ARCS model of Attention, Relevance, Confidence and Satisfaction. SPSS was used to analyze the quantitative data; while NVivo and content analysis were utilized to analyze the qualitative datasets of semi-structured interview. Findings indicated that the ARenVA's ARstvt treatment mode significantly improves the vocabulary acquisition of ESL learning by 30.76% and perceived motivation by 11.50%; more effective in motivating students by 32 times as compared to ARvt treatment mode. The novelty of this study situated on the effects of AR English learning towards improving motivation and English vocabulary acquisition of ESL learning among 4th, 5thand 6thgrade Malaysia student, significantly identified its benefit through the numerical percentages. Nevertheless, the instructional design of AR English learning controls the success of learning results. The paper reviews on the success systematic implementation of different ARenVA treatment modes.

This research full paper describes our work on the development, and evaluation of a physically-embodied robot teacher (PERT) as a facilitator for Peer Instruction (PI) in a high school science classroom. Embodied pedagogical agents are particularly useful in teaching young learners because of the engagement, and enjoyment they engender. Teaching robots are among possibilities in the future classroom; they may also represent solutions to the looming, acute global teacher scarcity. However, their design creates unique computational challenges. Moreover, robot-human systems involving a single robot in interaction with multiple humans, are not common. PI has been reported in many studies as a learner-centered approach that supports conceptual learning, metacognition, cognitive load sharing, and higher order learning, through peer scaffolding. However, existing studies have reported only applications by human teachers in regular classrooms. This study highlights an approach for enabling a PERT as a PI facilitator. Based on conflicting reports of the significance of agent embodiment for learning, we present the report of two experimental studies from our assessment of the system, employing the mixed method approach. We compared participants' perception of the robot's social presence and teaching quality when the robot is physically present and when it is absent (voice-only). Our findings are based on data from 14 participants in each study. The Social Presence indeX (SPiX) and the teacher quality index (TQI) were used for quantitative data collection for social presence and teaching quality, respectively. Qualitative data were based on live observation and participants' open-ended feedback. Quantitative findings show no significant difference in participants' perceptions in both conditions but qualitative feedback provided more information. Our findings show: i) students are willing to accept non-human teachers, and ii) their expectations of them are similar to those for human teachers; iii) an improved affective system will promote increased presence; iv) participants noted the robot as `neutral', with important implications for culturally-responsive education; v) robot teachers could be potential technology-inspired solution to global teacher scarcity and vi) in addition to promoting deeper learning, PI can be a useful pedagogical approach for non-human teachers. This study is in line with anticipated ability of future technological advancements to support machines that are as effective as intelligent human tutors.

Today's virtual reality (VR) applications such as gaming, multisensory entertainment, remote dining, and online shopping are mainly based on audio, visual, and touch interactions between humans and virtual worlds. Integrating the sense of taste into VR is difficult since humans are dependent on chemical-based taste delivery systems. This paper presents the 'Thermal Taste Machine', a new digital taste actuation technology that can effectively produce and modify thermal taste sensations on the tongue. It modifies the temperature of the surface of the tongue within a short period of time (from 25°C to 40 °C while heating, and from 25°C to 10 °C while cooling). We tested this device on human subjects and described the experience of thermal taste using 20 known (taste and non-taste) sensations. Our results suggested that rapidly heating the tongue produces sweetness, fatty/oiliness, electric taste, warmness, and reduces the sensibility for metallic taste. Similarly, cooling the tongue produced mint taste, pleasantness, and coldness. By conducting another user study on the perceived sweetness of sucrose solutions after the thermal stimulation, we found that heating the tongue significantly enhances the intensity of sweetness for both thermal tasters and non-thermal tasters. Also, we found that faster temperature rises on the tongue produce more intense sweet sensations for thermal tasters. This technology will be useful in two ways: First, it can produce taste sensations without using chemicals for the individuals who are sensitive to thermal taste. Second, the temperature rise of the device can be used as a way to enhance the intensity of sweetness. We believe that this technology can be used to digitally produce and enhance taste sensations in future virtual reality applications. The key novelties of this paper are as follows: 1. Development of a thermal taste actuation technology for stimulating the human taste receptors, 2. Characterization of the thermal taste produced by the device using taste-related sensations and non-taste related sensations, 3. Research on enhancing the intensity for sucrose solutions using thermal stimulation, 4. Research on how different speeds of heating affect the intensity of sweetness produced by thermal stimulation.

This paper provides an overview on side-channel attacks with emphasis on vulnerabilities in the smart home. Smart homes are enabled by the latest developments in sensors, communication technologies, internet protocols, and cloud services. The goal of a smart home is to have smart household devices collaborate without involvement of residents to deliver the variety of services needed for a higher quality of life. However, security and privacy challenges of smart homes have to be overcome in order to fully realize the smart home. Side channel attacks assume data is always leaking, and leakage of data from a smart home reveals sensitive information. This paper starts by reviewing side-channel attack categories, then it gives an overview on recent attack studies on different layers of a smart home and their malicious goals.

Olfactometers are kind of machines specially designed to deliver different odorants to the users, based on a timely protocol. Purchasing off the shelf systems are expensive and require additional customizations according to the type of the experiments they will be used for. In order to facilitate smell, taste, psychology, and neuroscience experiments we have developed our own laboratory olfactometer from the scratch. This paper provide a complete overview of the design and development process of our laboratory olfactometer. The key novelty of our work is the development of low cost, flexible, and easy maintenance olfactometer. Further, this olfactometer provides rich functionalities such as seven odor channels and a control air channel, flow controlling for the overall system and individual channels, air pressure control, air filtration, back flow prevention, and continuous operation of more than 8 hours.

Creativity and imagination are the most interesting and peculiar of all properties of human consciousness that are not yet fully understandable for scientists. “Could creativity or imagination ever be defined?”, “Will humans lose their creativity in the rapid development of technology?” or “Could machines ever be creative and how this will effect on humanity?”. In this paper, we will address these questions from the point of view of different disciplines. We will describe and analyze state-of-the-art literature of existing methodologies and techniques that somehow describe the basics of creativity. At the end, we will come up with the new concept of Creativity Assistant System (CAS) that should be able to replicate “creativity”. We will provide the mathematical model of the CAS concept that is based on the motivation, preference and past experience of specific user along with available knowledge for further developing her/his “psychological creativity”.

With advances in robotics and artificial intelligence, and increasing prevalence of commercial social robots and chatbots, humans are communicating with artificially intelligent machines for various applications in a wide range of settings. Intimate touch is crucial to relationships forming and emotional bonding, however it remains as a missing element in digital communication between humans, as well as humans and machines. A kissing machine, Kissenger, is built to transmit haptic kissing sensations in a communication network. In order to investigate the role of robotic kissing using the Kissenger device in digital communication, we conducted a modified version of the Imitation Game described by Alan Turing by including the use of the kissing machine. The study consists of 4 rounds, in which participants communicate with two players, a male and a female, using online chatrooms with the goal to identify the female player. In the last two rounds, the male player is replaced with a chatbot with female personas. Participants are told to remotely kiss the players using the kissing device at least once during the chat in the second and fourth round, but not in the first and third round. Results show that robotic kissing has no effect on the winning rates of the male and female players during human-human communication, but it increases the winning rate of the female player when a chatbot is involved in the game.

Confucianism is commonly defined as “... a system of philosophical, ethical and political thought based on the teachings of Confucius,” which originated through the teachings of Confucius during the sixth-century BCE. It is a way of life or a philosophy of human nature that considers human relationships as the foundation of the society. Confucius teachings had highly influenced the development of several cultures in Asia, making Confucianism an intangible cultural heritage. In this paper, we are re-acquainting users with an intangible heritage that is part of their everyday, by developing a system that permits experiencing Confucius teachings virtually and interactively. The system can measure philosophical intent of the human and generate meaningful philosophical answers. It is also aimed for intergenerational sharing of Confucius heritage through a simple interactive process with the virtual sage making the experience enjoyable and entertaining. Previous research in natural language processing (NLP) mainly focused on the understanding and delivering of human natural language accurately. In this research, we explored how to apply NLP to model the knowledge and teachings of Confucius, through the natural conversation between human and computer. This virtual Confucius, a chat agent that generates outputs based on Confucius teachings, using a series of algorithms and techniques to improve the matching accuracy between user input and computer output, introduces a novel way of interacting with intangible cultures. Our user evaluation results revealed that there is a positive correlation between relevance and enjoyment, finding their experiences interacting with virtual Confucius very encouraging. Adults who participated in experiencing the virtual Confucius together with their children believed that this system has the potential to improve intergenerational interactions through shared play.

Picture books are considered beneficial for children in many ways (e.g., psychological development and language acquisition), yet some children have less opportunity to receive such benefits because of their visual impairment. Considering such a situation, we designed Alice and Her Friend, an interactive book for visually-impaired children. While we put a significant focus on multisensory interaction, the book is designed entirely in black, intending both to emphasize that visually-impaired people also enjoy rich multisensory experiences and to raises awareness of the needs that visually-impaired children have. We used the book to interview experts in related fields to investigate practical needs among visually-impaired children. Based on their insights, we also propose a novel research topic, "creative technologies for the disabled"; it is necessary to support people with disabilities to create what they need by themselves, since the degrees and types of visual impairment significantly differ among visually-impaired people.

In this poster, we present our concept of magnetic dining table interface and magnetic foods. This interface allows new interactions for the food and utensils such as modifying weight, levitation, movement, and dynamic textures. We implement these interactions by manipulating a strong magnetic field formed by an array of Bitter electromagnets which placed under the surface of the table. Then we add edible magnetic materials to the foods, and make them interactive and controllable.

As human communication is rapidly migrating from the physical world to the digital world, it is crucial to develop affective communication systems to enable the expression of intimacy, emotion and empathy over the Internet, as these are essential elements in forming social relationships. A multisensory, real-time kissing device, Kissenger, is developed to transmit multisensory kissing sensations remotely using mobile phones. The aim is to provide an intimate communication channel for families and friends to physically interact with each other remotely, in order to effectively convey deep emotions and intimacy through a multisensory internet communication experience.

This poster paper discuss the concept of magnetic table interface and magnetic foods. This interface introduces new human-food interactions such as modifying weight, levitation, movement, and dynamic textures for our daily used food and utensils using a strong magnetic field formed by an array of Bitter electromagnets. To make foods and cutleries interactive, we add edible magnetic material on the food such as iron and iron oxides, and make cutleries with magnetic materials. We expect that this system will alter our food consumption behaviours and make the whole experience much more interactive and enjoyable.

This paper presents a new taste interface for multisensory communication called "Thermal Sweet Taste Machine". We developed this interface in order to create sweet sensations, by manipulating the temperature on the tongue, without using chemicals. This device device changes the temperature on the surface of the tongue (from 20°C to 40°C) within a short period of time using a computer controlled circuit. Our preliminary user studies suggested that this device would be effective in two ways; producing the sweet sensations without the aid of chemicals, and enhancing the sweetness of the food and drinks. Here we discuss our concept, development of the interface, and some preliminary studies that has been carried out. We believe our technology would enhance the experiences and capabilities in future multisensory communication in different disciplines such as Human-Computer Interaction, human robot interactions, gaming and interacting with artificial agents.

Robots are increasingly becoming involved in people's lives as social companions or even romantic partners rather than mere productivity tools. To facilitate intimacy in human-robot relationships, technologies should enable human to have intimate physical interactions with robots, such as kissing. This paper presents a kissing machine that reproduces and transmits the haptic sensations of kissing. It provides a physical interface for human to form emotional and intimate connections with robots or virtual characters through kissing, and also acts as a remote agent for human to transmit kisses remotely through a communication network.

The sense of smell (olfaction) has been recognised as an important factor governing our emotions and memory. Nevertheless, it remains the most underexplored sense and is still a relatively new modality in the domain of marketing. While researchers have begun to explore the merits of olfaction as a powerful trigger of memories, its relation with emotions has been largely disregarded and its interplay with other modalities remains largely unknown. In this paper, we explore the use of scent for mobile user experiences and devices with a hardware tele-smell device "Scentee". We provide two examples of how Scentee has successfully been used in marketing campaigns.

THIS tutorial paper describes the development of a new interface for multisensory communications called "Thermal Sweet Taste Interface". By rapidly changing the temperature of the surface of the tongue, this interface is capable of generating virtual sweet sensations. This device may enhance new user experiences in different disciplines such as Human-Computer Interaction, New Media Arts, Internet Communication and Medicine.

This workshop aims to increase awareness of the different ways in which new technologies (e.g. social media, virtual reality, digital, and neuroimaging technologies) can be used as input or means to develop multisensory experiences that improve consumer satisfaction. Consumers experience products and services via their senses (i.e. sight, hearing, smell, taste, touch). The specific multisensory content of those products and services can influence people's respective attitudes, purchase intentions, and consumption. Today, consumers experiment the world through new technologies, for example, by sharing their feelings via social media, or by creating avatar in a virtual world. The challenge for marketing consists of monitoring new technologies to better understand consumer sensory experiences and to develop innovative actions that will help to improve these experiences. We will begin with the definition of sensory marketing and the role of multisensory processes in consumer experience. Second, we will address the possibility of better understanding the consumer through social media and neuroimaging technologies. To finish, we will present some technologies that are already enriching sensory experiences of consumers.

The development of neuroimaging techniques such as functional magnetic resonance imaging (fMRI) has provided marketers with the possibility of studying changes in brain activity in relation to marketing information (packaging, pricing, promotions, etc.) and decision-making. When consumers interact with a product, the brain captures and stores the related sensory experience in a multisensory representation in memory. Such memories serve consumers when making their future decisions. Thus, the senses play an important role in consumer behavior and the results of fMRI studies provide an interesting way for sensory marketers to analyze how the brain processes sensory information affecting product perception. In this article, we show how multisensory information, as well as mental simulation, impact taste expectations and subsequently taste, or better-said, flavor, perception in the light of neuroimaging studies. First, we discuss how neural analysis can help to understand the effect of marketing information on taste perception. Second, we highlight the role of gustatory inference on food perception in light of the latest fMRI findings. Third, we suggest some directions to improve the understanding of crossmodal correspondences in the context of consumer sensory experiences.

The sense of touch has a significant role in establishing effective communication with customers and improving brand awareness and product experience. With mobile and wearable devices at the core of communication today, mobile marketing is crucial for advertisers to appeal to a larger audience and new haptic technologies should be developed to deliver more realistic and meaningful haptic messages to these devices. We introduce RingU and Kissenger, two haptic devices for mobile devices that enhance the vibration haptic feedback using force-variable vibrotactile and linear force actuation, allowing users to hug and kiss each other over the Internet in real-time. Businesses can use these devices to communicate with their customers using haptic stimulations to enhance customer experience and advertise their services. Artist agencies or game producers could also increase the popularity of their idols and virtual characters by allowing fans from worldwide to come in "close contact" with them.

Memory and emotion are an ever-important aspect of Human Computer Interaction. While we are researching new interfaces for emotional and experience communication through multisensory communication, there is one aspect of our emotional perception that has been largely ignored in the process: dreaming. We propose to take a new direction by studying the perception of emotions during sleep and how they can be affected through external stimuli via novel user interfaces. In this position poster we take results from studies that have shown that smell and emotions are closely interlinked as a basis for an olfactory computer-dream interface.

In this paper we describe Ring*U which is a ring-shaped, wearable system for sharing intimate, interpersonal interactions remotely through subtle colored lighting and tactile expressions. We present an overview of the Ring*U system. We also describe our next objectives for future work.

Crying is the language of babies. Inability of parents in understanding the needs of babies causes stress and anxiety. The aim of this study is to develop a rapid prototype of infant-parent communication system. The system was tested on real babies and parents, while they were apart. The Results pointed to the overall acceptance and effectiveness of the system in cry translation and reducing the parents' anxiety with regards to baby cries.

In this paper, we present a synergy of technology and traditional craft made possible with AmbiKraf Byobu. AmbiKraf is a non-emissive, color-changing fabric technology that allows subtle animations on fabrics. Byobu is a traditional Japanese room-divider screen, usually painted by artisans and crafts people. We first discuss the organic qualities of AmbiKraf technology that particularly lend itself to traditional art and craft practice. These qualities include the animation of ambient, non-intrusive and calming motifs using non-emissive display characteristics. We then discuss in detail how AmbiKraf technology is implemented and paired with the art form of Byobu. Interaction with the system is also briefly touched upon, as well as discussion regarding the cultural implications and mutual benefit of combining technology with traditional craft. Through the use of AmbiKraf technology embodied by the Byobu, we hope to provide a convincing example that sensitively combines traditional textile crafts with new technology in order to help redefine the meaning and role of traditional textiles.

Paper, as a traditional material for art and communication, shows great potential as a medium for organic user interfaces, with its ubiquity and flexibility. However, controlling and powering the sensors and actuators that enable interactive paper-crafts has not been fully explored. We present a method of selective inductive power transmission (SIPT) to support interactive paper-crafts. The novelty of this method is that the power transmitter can be controlled to selectively activate one specific receiver at a time through inductive power transferring with multiple receivers. This was achieved by changing the output frequency of the power transmitter to match the impedance of the receivers. The receivers could be embedded or printed to drive paper-crafts. Based on inductor–capacitor oscillating circuit and a function generator with a power amplifier, we developed two different prototypes of SIPT. By comparing the performance of both prototypes, we discussed the advantages and disadvantages of the two systems, and their applications in different contexts of paper-crafts. In addition, we proposed the instructions for using SIPT in developing interactive paper-crafts. With this technology and instructions, we hope to facilitate users to easily design new types of paper-craft systems without being concerned about the arrangement of wire connections to power supply on a massive scale.

In this paper, we present Kissenger (Kiss Messenger), an interactive device that provides a physical interface for transmitting a kiss between two remotely connected people. Each device is paired to another and can sense and transmit the amount of force that a user applies to a pair of lips which is recreated on the other device using motors. Kissenger was designed to augment already existing remote communication technologies such as video chat. The goal of this work is to promote intimacy between humans in long distance relationships. After presenting the background and motivation for the need of such a device, we describe the design process that consisted of three iteration stages, each with its own focus and evaluation. We then present a preliminary user study performed with seven couples that compare Kissenger to current video chat technology.

This paper investigates whether and how digitally mediated social touch (remote touch) may influence the sense of connectedness toward a speaker and the emotional experience of what is being communicated. We employ an 'augmented' storytelling methodology where we manipulate the modality of an 'emotive' channel that accompanies the speech, and the contextual expectation of the listener. Comparing a remote upper-arm touch against a similarly timed flashing light, we explore the importance of the touch modality in affect conveyance. Our second manipulation involves two cover stories where the listener is told that the touch or flashing light is triggered either by the storyteller expressively squeezing a touch input device while speaking, or by measured 'high points' in the mental state of the storyteller. Our results show that the story accompanied by communicative touch resulted in a significant increase in the sense of connectedness with the storyteller over the speech-only condition, and a trend toward greater affective conveyance.

In this paper, we present a new methodology for integrating the sense of taste with the existing digital communication domain. First, we discuss existing problems and limitations for integrating the sense of taste as a digital communication media. Then, to assess this gap, we present a solution with three core modules: the transmitter, form of communication, and receiver. The transmitter is a mobile application, where the sender formulates a taste message to send. Then, for communication, we present a new extensible markup language (XML) format, the TasteXML (TXML) to specify the format of taste messages. As the receiver (actuator), we introduce Digital Taste Stimulator, a novel method for stimulating taste sensations on human. Initial user experiments and qualitative feedbacks were discussed mainly focusing on the Digital Taste Stimulator. We conclude with a brief overview of future aspects of this technology and possibilities on other application domains.

Having dinner together is undoubtedly a kind of pleasurable experience, which involves various channels for mutual interactions, not only audio, vision and touch, but also smell and taste. With the aim to extend this rich experience to remote situations, we developed the Co-dining system to support a range of mealtime interactions to enhance the feeling of social togetherness. This paper describes the preliminary study with this interactive multisensory system. It aims to investigate the actual effectiveness of the working prototype on enhancing the social presence and engagement during telepresent dining, and also to get a comprehensive understanding about users' perception. This evaluation focused on three main aspects: the overall Co-dining feeling, cultural awareness, and engagement. The study results revealed that this system positively achieved the sense of "being together" among users, through the interactive activities touching upon tableware, tablecloth and real edible food, and each interaction module contributed differently to the overall experience. In this paper, we report the evaluation process, present and interpret the data, and discuss the initial insights to enhance the sense of co-presence through multi-channel interactions.

We describe a real time authoring tool that is useful to supervise group of users in an augmented reality environment. The overall system is composed of a server; the Command Center (CC) and several clients; Wearable Systems (WS). The server is used to visualize the virtual representation of the real environment, 3d features map where augmentation are to occur, augmented content that will be rendered and users avatar that evolves on the field. The user's WS is equipped of a Head Mounted Display (HMD) along with sensors (GPS/DRM/Compass) to provide constant pose estimation (position and orientation) to the CC and a vision based tracking that uses natural features and pose estimation allgorithms to render virtual content accordingly. Alternatively a sensor based renderer is used to display AR guidance on the HMD based on the path defined and sent from the CC. The natural features system handles a large number of small 3D features maps. The overall system offers a complete system to train and supervise remote users through an AR environment.

In this paper we outline a genealogy of Mixed Reality Lab (MXR) projects, their influencing factors both from the Asian region and also within the lab, the employment of "Blue-Sky Innovation" by the lab for ideation, collaboration and project generation, as well as discuss some major points of inspiration for MXR from various sources.

With this paper we present AmbiKraf, a none missive fast color changing fabric display. Our main motivation is to investigate the possibility of an actual fabric technology that can change the color in the form of a display. The core novelty of this paper addresses in presenting a base line technology that enables such a ubiquitous fabric display that lets the user interact with the actual fabric itself. Here, we use thermo chromic inks and Peltier semiconductor elements to achieve a rapid color changing display. The implemented controller is able to control the temperature with a high accuracy and thereby accurately control the color of the thermo chromic ink display. In addition the controller has fast heating/cooling ability which presents the ability for a relatively high speed display on the fabric. The calming nature of this animated fabric display is presented through few prototypes that ubiquitously combine the technology with daily fabrics. The paper describes the complete implementation of the system and presents a detailed analysis of the technical results.

In this paper, we present a system, which could digitally stimulate the sense of taste (gustation) on human. The system utilizes electrical stimulation on human tongue to produce taste sensations. The initial experiments reveal that the method is viable and deserves further developments. This requires further analyses of the properties of electric pulses (current, frequency, and voltage) on tongue along with the stimulating material. The experimental results suggested that sourness, bitterness, and saltiness are the main sensations that could be evoked at present.

The pervasiveness of computing has extended into domestic realms, including the dining room. Beyond simply a place to consume food, the dining room is a social hub where family members meet and share experiences. Yet busy lifestyles can make it difficult to spend social time with your family. To provide a new solution for family bonding, this paper presents the CoDine system, a dining table embedded with interactive subsystems that augment and transport the experience of communal family dining to create a sense of coexistence among remote family members. CoDine connects people in different locations through shared dining activities: gesture-based screen interaction, mutual food serving, ambient pictures on an animated tablecloth, and the transportation of edible messages. Rather than focusing on functionality or efficiency, CoDine aims to provide people with an engaging interactive dining experience through enriched multi-sensory communication.

This talk outlines new facilities within human media spaces supporting embodied interaction between humans, animals, and computation both socially and physically, with the aim of novel interactive communication and entertainment. We aim to develop new types of human communications and entertainment environments using all the senses, including touch, taste, and smell, which can increase support for multi-person multi-modal interaction and remote presence. In this talk, we present an alternative ubiquitous computing environment based on an integrated design of real and virtual worlds. We discuss some different research prototype systems for interactive communication, culture, and play.

Although animals and pets are so important for families and society, in modern urban lifestyles, we can only spend little time with our animal friends. Interactive media should be aimed to enhance not only human-to-human communication but also human-to-animal communication. Thus, we promote a new type of interspecies media interaction which allows human users to interact and play with their small pet friends (in this case, hamsters) remotely via the Internet through a mixed-reality-based game system “Metazoa Ludens.” We used a two-pronged approach to scientifically examine the system. First, and most importantly, the body condition score study was conducted to evaluate the positive effects to the hamsters. Second, the method of Duncan was used to assess the strength of preference of the hamsters toward Metazoa Ludens. Lastly, the effectiveness of this remote interaction with respect to the human users as an interactive gaming system with their pets/friends (hamster) was examined based on Csikszentmihalyi's Flow theory. Results of both studies inform of positive remote interaction between human users and their pet friends using our research system. This research is not only just aimed at providing specific experimental results on the implemented research system but is also aimed as a wider lesson for human-to-animal interactive media. Therefore, as an addition, we present a detailed framework suited in general for human-to-animal interaction systems inferred from the lessons learned.

In order to develop a close relationship between humans and robots, we proposed a multi-modal sentimental model which considers both long and short term affective parameters of interaction. Our model is inspired from scientific studies of love in humans and aims to generate a bi-directional love between humans and robots. We refer to this sentimental connection as “Lovotics”. We have formulated probabilistic mathematical models for identified factors of love, and aim to provide a clear, distinct and discrete interpretation of the intimacy between humans and robots. Such mathematical models are assembled by a Bayesian Network depicting the relationship between intimacy and the causal factors for love. Furthermore, a novel affective state transition system is proposed which takes into account not only the current state caused by interactions, but also the effects of the previous states and internal factors of the robot. Hence, the robot is capable of acting consistently and naturally. The behavior of the robot is controlled by the above two modules via an Artificial Neural Network to develop a realistic affective communication between a human and a robot.

This paper introduces a system that can recognize different type of paper-folding by users. The system allows users to register and use their desired paper in the interaction, and detect the folding by using Speed Up Robust Feature (SURF) algorithm. The paper also describes a paper-based tower defense game which has been developed as a proof of concept of our method. This method can be considered as the initial step for seamlessly migrating meaningful traditional art of origami into the digital world as a part of the interactive media.

In this demonstration we introduce a system that allows users to register and use their desired paper for the interaction, and recognize the folding during the interaction in real time. This method can be considered as an initial step for seamlessly migrating meaningful traditional art of origami into the digital world and as part of the interactive media.

Lovotics is a novel field of research that strives to achieve a close relationship between humans and robots by utilizing fundamental concepts from psychology, biology, neuroscience, and robotics to develop a model that effectively imitates human affection process to create an emotionally engaging robotic system with high level of intimacy. We have designed and developed a robotics system that is an active participant in a bi-directional non verbal communication process with humans, and uses audio and touch channels along with internal state parameters in order to establish long standing bonds with individuals. The novel Affective Intelligence system of Lovotics includes an Artificial Endocrine System and an Intimacy Module. These two modules work in tandem to develop intimate interactions between the robot and humans.

Myglobe is a user generated navigation service that enables users to share each cognitive map with one another. Cognitive map is a personalized map, shape of which is emphasized according to user's preference and activity in the city. It facilitates users to look back on their own city and have a new understanding by using an application in smart phones and physically interacting with a globe shaped device. In this paper, we present Myglobe service for users to achieve a new city experience with cognitive maps.

As social networking widely spreads among the community, especially among the younger generation, the negative influence created on children has become a serious social concern. "Petimo" is an interactive robotic toy designed to protect children from potential risks in social networks and the virtual world and it helps them to make a safely connected social networking environment. It adds a new physical dimension to social computing through enabling a second authentication mode, providing extra safety in making friends by physically touching each others robot. Petimo can be connected to any social network and it provides safety and security for children. As a proof of concept, we have developed a 3D virtual world, "Petimo-World" which demonstrates all of the realizable basic features with traditional online social networks. Petimo-World stands out from all other virtual worlds with its interesting and sophisticated interactions such as the visualization of a friends' relationships through spatial distribution in the 3D space to clearly understand the closeness of the friendship, personalized avatars and sending of special gifts/emoticons.

Huggy Pajama is a novel wearable system aimed at promoting physical interaction in remote communication between parent and child. This system enables parents and children to hug one another through a novel hugging interface device and a wearable, hug reproducing pajama connected through the Internet. The hugging device is a small, mobile doll with an embedded pressure sensing circuit that is able to accurately sense varying levels of the range of human force produced from natural touch. This device sends hug signals to a haptic jacket that simulates the feeling of being hugged to the wearer. It features air pressure actuation to reproduce hug.

Paruresis is an object critical design that addresses the phobia of an inability to urinate in the presence of others. It combines machine vision and sensors with a camera recording the gaze triggered for playback by urination.

Present computer games and digital entertainment do not usually facilitate intergenerational family interactions. According to recent survey results in Japan, there is a high percentage of older people who own and play electronic or computer games, but rarely do they play the games with their family members. It is a positive sign that more older people are participating in the digital games arena, but it would be even more beneficial if they could interact actively with the young family members through gaming activities. This could possibly strengthen family bonding and bridge the gap between older people and youth culture. This paper presents steps for designing an intergenerational family entertainment system which focuses on physical and social interactions using a mixed reality floor system. The main design goals include: facilitating interactions between users with varied levels of skill in utilizing technology, utilizing the familiar physical motions from other activities to make an intuitive physical interface, and encouraging social interactions among families and friends. Detailed implementation of these steps is presented in the design of our intergenerational entertainment system, Age Invaders. Four main prototype iterations for the system is presented. Our design process is based on User Centered Design and relies on constant involvement of users to understand the key issues and to help make effective design decisions. The results of the study help to focus the refinements of the existing platform from a usability standpoint and also aids in the development of new physical entertainment and interactive applications. This study provides insights into user issues including how users interact in a complex mixed reality experience, which is heavily based in physicality. The use of one portion of the user base which is most proficient with technology before involving the novice users was found to empower novice users to begin to use digital technology.

We describe a new form of interactive living media used to communicate social or ecological information in the form of an empathetic ambient media. In the fast paced modern world people are generally too busy to monitor various significant social or human aspects of their lives, such as time spent with their family, their overall health, state of the ecology, etc. By quantifying such information digitally, information is semantically coupled into living microorganisms, E. coli. Through the use of transformed DNA, the E. coli will then glow or dim according to the data. The core technical innovation of this system is the development of an information system based on a closed-loop control system through which digital input is able to control input fluids to the E. coli, and thereby control the output glow of the E. coli in real time. Thus, social or ecological based information is coupled into a living and organic media through this control system capsule and provides a living media which promotes empathy. We provide user design and feedback results to verify the validity of our hypothesis, and provide not only system results but generalized design frameworks for empathetic living media in general.

Huggy Pajama is a novel wearable system aimed at promoting physical interaction in remote communication between parent and child. This system enables parents and children to hug one another through a novel hugging interface device and a wearable, hug reproducing pajama connected through the Internet. The hugging device is a small, mobile doll with an embedded pressure sensing circuit that is able to accurately sense varying levels of the range of human force produced from natural touch. This device sends hug signals to a haptic jacket that simulates the feeling of being hugged to the wearer. It features air pockets actuating to reproduce hug, heating elements to produce warmth that accompanies hug, and color changing pattern and accessory to indicate distance of separation and communicate expressions. In this paper, we present the system design of Huggy Pajama.

Short message service (SMS) is extremely popular today. Currently it is being mainly used for peer-to-peer communication. BlogWall extends the SMS to a new level of self-expression and public communication by combining art and poetry. The application can simply display SMS, gather data using polls, and create poetry or even Japanese "Haiku" all based on the SMS.

10.1145/1377999.1378058

For an increasing amount of people (especially young people), the mobile phone, and not the computer, is becoming their main media device and portal to the virtual world. Mobile phones have become a ubiquitous technology and for many people an important tool for communication and information access. Mobile telephony offers an interesting platform for building multipresence-enabled applications that utilize the phone as a social or commercial assistant. The main objective of this research is to develop multipresence-enabled audio windowing systems for visualization, attention, and privacy awareness of narrowcasting (selection) functions in collaborative virtual environments (CVEs) for mobile devices such as 3rd- and 4th-generation mobile phones. Mobile audio windowing system enhances auditory information on mobile phones and encourages modernization of office- and mobile-based conferencing.

Three-dimensional sound's effectiveness in virtual reality (VR) environments has been widely studied. However, due to the big differences between VR and augmented reality (AR) systems in registration, calibration, perceptual difference of immersiveness, navigation, and localization, it is important to develop new approaches to seamlessly register virtual 3-D sound in AR environments and conduct studies on 3-D sound's effectiveness in AR context. In this paper, we design two experimental AR environments to study the effectiveness of 3-D sound both quantitatively and qualitatively. Two different tracking methods are applied to retrieve the 3-D position of virtual sound sources in each experiment. We examine the impacts of 3-D sound on improving depth perception and shortening task completion time. We also investigate its impacts on immersive and realistic perception, different spatial objects identification, and subjective feeling of "human presence and collaboration". Our studies show that applying 3-D sound is an effective way to complement visual AR environments. It helps depth perception and task performance, and facilitates collaborations between users. Moreover, it enables a more realistic environment and more immersive feeling of being inside the AR environment by both visual and auditory means. In order to make full use of the intensity cues provided by 3-D sound, a process to scale the intensity difference of 3-D sound at different depths is designed to cater small AR environments. The user study results show that the scaled 3-D sound significantly increases the accuracy of depth judgments and shortens the searching task completion time. This method provides a necessary foundation for implementing 3-D sound in small AR environments. Our user study results also show that this process does not degrade the intuitiveness and realism of an augmented audio reality environment

This paper outlines new facilities within ubiquitous human media spaces supporting embodied interaction between humans and computation both socially and physically. We believe that the current approach to developing electronic based design environments is fundamentally defective with regard to support for multi-person multi-modal design interactions. In this paper, we present an alternative ubiquitous computing environment based on an integrated design of real and virtual worlds. We implement three different research prototype systems: the Virtual Kyoto Garden, touchy Internet, and the human pacman. The functional capabilities implemented in these systems include spatially-aware 3D navigation, tangible interaction, and ubiquitous human media spaces. Some of its details, benefits, and issues regarding design support are discussed

For mobile computing system such as wearable computer, one of the most critical hardware issues is the provision of electric power. Various different sources of power for wearable computer have been investigated; however, we are interested only in power that can be generated in-situ, such as human power. This paper describes a novel mean of multiple source human power generation for small wearable electronic devices, and then demonstrates the digital information transfer between wearable computing devices by using human skin as "antenna". There are a wide range of peripheral devices in a mobile computing system, such as sensors and identification memory tags. As the amount of such devices increases in a mobile computing design, there is a need for these devices to communicate efficiently with the central processing unit. Also, it is highly desirable that these devices could be conveniently connected to the wearable computer without many dangling wires. We developed a personal area network (PAN) system which attempts to interconnect such devices, and at the same time uses human body as communication channel. This system is in a way novel because it is totally powered by human motion.

We describe an augmented reality system for superimposing three-dimensional (3-D) live content onto two-dimensional fiducial markers in the scene. In each frame, the Euclidean transformation between the marker and the camera is estimated. The equivalent virtual view of the live model is then generated and rendered into the scene at interactive speeds. The 3-D structure of the model is calculated using a fast shape-from-silhouette algorithm based on the outputs of 15 cameras surrounding the subject. The novel view is generated by projecting rays through each pixel of the desired image and intersecting them with the 3-D structure. Pixel color is estimated by taking a weighted sum of the colors of the projections of this 3-D point in nearby real camera images. Using this system, we capture live human models and present them via the augmented reality interface at a remote location. We can generate 384/spl times/288 pixel images of the models at 25 fps, with a latency of <100 ms. The result gives the strong impression that the model is a real 3-D part of the scene.

A real-time system for capturing humans in 3D and placing them into a mixed reality environment is presented in this paper. The subject is captured by nine cameras surrounding her. Looking through a head-mounted-display with a camera in front pointing at a marker, the user can see the 3D image of this subject overlaid onto a mixed reality scene. The 3D images of the subject viewed from this viewpoint are constructed using a robust and fast shape-from-silhouette algorithm. The paper also presents several techniques to produce good quality and speed up the whole system. The frame rate of our system is around 25 fps using only standard Intel processor-based personal computers. Besides a remote live 3D conferencing and collaborating system, we also describe an application of the system in art and entertainment, named Magic Land, which is a mixed reality environment where captured avatars of human and 3D computer generated virtual animations can form an interactive story and play with each other. This system demonstrates many technologies in human computer interaction: mixed reality, tangible interaction, and 3D communication. The result of the user study not only emphasizes the benefits, but also addresses some issues of these technologies.

In this paper we describe two Augmented Reality (AR) applications for children using mobile phones as the user interface. We make use of standard mobile phones readily available in the consumer market without making any hardware modifications to them. The first AR application we describe is the AR Comic Book which allows children to view their favorite cartoon characters in full 3D appearing on books or magazines (or any paper). These 3D virtual characters are rendered into the actual scene captured by the mobile phone's camera. The second AR application is the AR Post-It, which combines the speed of traditional electronic messaging with the tangibility of paper based messages. The key concept of the AR Post-It system is that the messages are displayed only when the intended receiver is within the relevant spatial context. For both these applications a server is used to do the image processing tasks and the phone connects to the server using Bluetooth.

Storytelling is an effective and important educational means for children. With the augmented reality (AR) technology, storytelling becomes more and more interactive and intuitive in the sense of human computer interaction. Although AR technology is not new, it's potential in education is just beginning to be explored. In this paper, we present a 3D mixed media story cube which uses a foldable cube as the tangible and interactive storytelling interface. Here, we embed both the concept of AR and the concept of tangible interaction. Multiple modalities including speech, 3D audio, 3D graphics and touch are used to provide the user (especially children) with multi-sensory experiences in the process of storytelling. Our research explores a new interface for children education.

Investigation of augmented reality (AR) environments has become a popular research topic for engineers, computer and cognitive scientists. Although application oriented studies focused on audio AR environments have been published, little work has been done to vigorously study and evaluate the important research questions of the effectiveness of 3D sound in the AR context, and to what extent the addition of 3D sound would contribute to the AR experience. Thus, we have developed two AR environments and performed vigorous experiments with human subjects to study the effects of 3D sound in the AR context. The study concerns two scenarios. In the first scenario, one participant must use vision only and vision with 3D sound to judge the relative depth of augmented virtual objects. In the second scenario, two participants must co-operate to perform a joint task in a game-based AR environment. Hence, the goals of this study are (1) to access the impact of 3D sound on depth perception in a single-camera AR environment, (2) to study the impact of 3D sound on task performance and the feeling of ‘human presence and collaboration’, (3) to better understand the role of 3D sound in human-computer and human–human interactions, (4) to investigate if gender can affect the impact of 3D sound in AR environments. The outcomes of this research can have a useful impact on the development of audio AR systems which provide more immersive, realistic and entertaining experiences by introducing 3D sound. Our results suggest that 3D sound in AR environment significantly improves the accuracy of depth judgment and improves task performance. Our results also suggest that 3D sound contributes significantly to the feeling of ‘human presence and collaboration’ and helps the subjects to ‘identify spatial objects’.

Investigation of augmented reality (AR) environments has become a popular research topic for engineers, computer and cognitive scientists. Although application oriented studies focused on audio AR environments have been published, little work has been done to vigorously study and evaluate the important research questions of the effectiveness of three-dimensional (3D) sound in the AR context, and to what extent the addition of 3D sound would contribute to the AR experience. Thus, we have developed two AR environments and performed vigorous experiments with human subjects to study the effects of 3D sound in the AR context. The study concerns two scenarios. In the first scenario, one participant must use vision only and vision with 3D sound to judge the relative depth of augmented virtual objects. In the second scenario, two participants must cooperate to perform a joint task in a game-based AR environment. Hence, the goals of this study are (1) to access the impact of 3D sound on depth perception in a single-camera AR environment, (2) to study the impact of 3D sound on task performance and the feeling of ‘human presence and collaboration’, (3) to better understand the role of 3D sound in human–computer and human–human interactions, (4) to investigate if gender can affect the impact of 3D sound in AR environments. The outcomes of this research can have a useful impact on the development of audio AR systems, which provide more immersive, realistic and entertaining experiences by introducing 3D sound. Our results suggest that 3D sound in AR environment significantly improves the accuracy of depth judgment and improves task performance. Our results also suggest that 3D sound contributes significantly to the feeling of human presence and collaboration and helps the subjects to ‘identify spatial objects’.

Audio-visual speech recognition employing both acoustic and visual speech information is a novel extension of acoustic speech recognition and it significantly improves the recognition accuracy in noisy environments. Although various audio-visual speech-recognition systems have been developed, a rigorous and detailed comparison of the potential geometric visual features from speakers' faces is essential. Thus, in this paper the geometric visual features are compared and analyzed rigorously for their importance in audio-visual speech recognition. Experimental results show that among the geometric visual features analyzed, lip vertical aperture is the most relevant; and the visual feature vector formed by vertical and horizontal lip apertures and the first-order derivative of the lip corner angle leads to the best recognition results. Speech signals are modeled by hidden Markov models (HMMs) and using the optimized HMMs and geometric visual features the accuracy of acoustic-only, visual-only, and audio-visual speech recognition methods are compared. The audio-visual speech recognition scheme has a much improved recognition accuracy compared to acoustic-only and visual-only speech recognition especially at high noise levels. The experimental results showed that a set of as few as three labial geometric features are sufficient to improve the recognition rate by as much as 20% (from 62%, with acoustic-only information, to 82%, with audio-visual information at a signal-to-noise ratio of 0 dB).

No abstract available.

We describe a real-time 3-D augmented reality video- conferencing system. With this technology, an observer sees the real world from his viewpoint, but modified so that the image of a remote collaborator is rendered into the scene. We register the image of the collaborator with the world by estimating the 3-D transformation between the camera and a fiducial marker. We describe a novel shape- from-silhouette algorithm, which generates the appropriate view of the collaborator and the associated depth map at 30 fps. When this view is superimposed upon the real world, it gives the strong impression that the collaborator is a real part of the scene. We also demonstrate interaction in virtual environments with a live fully 3-D collaborator. Finally, we consider interaction between users in the real world and collaborators in a virtual space, using a tangible AR interface.

Previous work in modeling curves and surfaces in augmented reality (AR) space has used expensive sensors such as magnetic sensors. In this work, we propose an augmented reality system where a user can model interesting surfaces with her hands, without expensive sensing systems. The system uses computer vision based methods for tracking the user's head and hand position. Using a glove and the tracking system, the user can draw smooth lines or surfaces with her hands in a physical space. The user can also intuitively modify the lines or surface created by pushing or pulling at the control points of lines or curves in a tangible manner.

This paper presents an interactive theatre based on an embodied mixed reality space and wearable computers. Embodied computing mixed reality spaces integrate ubiquitous computing, tangible interaction and social computing within a mixed reality space, which enables intuitive interaction with physical world and virtual world. We believe it has potential advantages to support novel interactive theatre experiences. Therefore, we explored the novel interactive theatre experience supported in the embodied mixed reality space, and implemented live 3D characters to interact with user in such a system.

We present a complete scalable system for 6 DOF camera tracking based on natural features. Crucially, the calculation is based only on pre-captured reference images and previous estimates of the camera pose and is hence suitable for online applications. We match natural features in the current frame to two spatially separated reference images. We overcome the wide baseline matching problem by matching to the previous frame and transferring point positions to the reference images. We then minimize deviations from the two-view and three-view constraints between the reference images and the current frame as a function of camera position parameters. We stabilize this calculation using a recursive form of temporal regularization that is similar in spirit to the Kalman filter. We can track camera pose over hundreds of frames and realistically integrate virtual objects with only slight jitter.

We present a complete system for live capture of 3D content and simultaneous presentation in augmented reality. The user sees the real world from his viewpoint, but modified so that the image of a remote collaborator is rendered into the scene. Fifteen cameras surround the collaborator, and the resulting video streams are used to construct a three-dimensional model of the subject using a shape-from-silhouette algorithm. Users view a two-dimensional fiducial marker using a video-see-through augmented reality interface. The geometric relationship between the marker and head-mounted camera is calculated, and the equivalent view of the subject is computed and drawn into the scene. Our system can generate 384 /spl times/ 288 pixel images of the models at 25 fps, with a latency of < 100 ms. The result gives the strong impression that the subject is a real part of the 3D scene. We demonstrate applications of this system in 3D videoconferencing and entertainment.

Both soft computing (SC) and hard computing (HC) techniques are often successful for solving real-world control problems. In cases where problems could be solved by either or both methodologies, an important research problem is to find what are the advantages for fusing SC methods together with HC methods, rather than using the HC method alone. Hence, in this paper, a performance comparison is detailed for a fused soft control/hard observer type controller (where a classical or HC type observer is fused with an adaptive fuzzy or SC type controller) and a hard control/hard observer type controller (where both the observer and feedback linearization controller are classical HC types). The domain in which this comparison is made is for the sensorless speed control of switched reluctance motors (SRMs). This is because this type of motor has highly nonlinear characteristics, and the HC type controller can often be detrimentally affected by modeling inaccuracies, as well as noise. Simulation and experimental results are illustrated to show the performance comparison of the soft control/hard observer type controller and the hard control/hard observer type controller under a wide range of identical operation conditions including transient speed and torque, SRM model parameter variations, and measurement noise. It can be seen from the results that the soft control/hard observer type exhibits a better performance than the hard control/hard observer type controller.

We discuss a type of hidden Markov model (HMM) based on fuzzy sets and fuzzy integral theory which generalizes the classical stochastic HMM. The Choquet integral is used as a fuzzy integral which relaxes one of the two independence assumptions that we had with the classical HMM. We apply this new model to speech recognition and compare the performance with the classical HMM. In this research, the main innovation is that this new generalized fuzzy HMM is applied for the first time to speech recognition. Due to the fuzziness of the model, an interesting gain can be observed in terms of a lower computation time.

The use of soft computing for virtual object implementation is a novel technique that has received a lot of attention in the last few years. This field is very useful especially for applications related to computer games and virtual environments. One of the application areas is the use of real time lip tracking information for the implementation of the virtual lips in animated characters. The increased computing power of computers has made it possible to do real time lip tracking. This real time lip tracking information can be used to implement and control a virtual lip. On the other hand, the use of soft computing to represent the real time lip parameters enables us to have a more robust and flexible system which can compensate for the potential errors of lip tracking. In this paper we present a system which generates real time lip information for soft computing based virtual animated lips.

This paper describes a system built for wearable computer and mixed reality applications that applies natural human computer interaction using a pen interface. The pen is fitted with hardware micro-accelerometers and interacts wirelessly with the host computer. This system is used to draw on a digital paper with digital ink and virtual objects. The tracking problem is solved using both inertial and visual methods.

Bimodal speech recognition is a robust technique for automated speech analysis, and has received a lot of attention in the last few decades. In this paper, we analyze the effect of the HMM models on the performance of the bimodal speech recognizer, present a comparative analysis of the different HMM models that can be used in bimodal speech recognition, and finally propose a novel model, which has been experimentally verified to perform better than others. One of the unique characteristic of our HMM model is the novel fusion strategy of the acoustic and the visual features, that takes into account the different sampling rates of these two signals. Compared to audio only, the bimodal speech recognition scheme has a much more improved recognition accuracy, especially in presence of noise.

Audio-visual speech recognition is a novel extension of acoustic speech recognition and has received a lot of attention in the last few decades. The main motivation behind bimodal speech recognition is the bimodal characteristics of speech perception and production systems of human beings. The effect of the modeling parameters of hidden Markov models (HMM) on the recognition accuracy of the bimodal speech recognizer is analyzed, a comparative analysis of the different HMMs that can be used in bimodal speech recognition is presented, and finally a novel model, which has been experimentally verified to perform better than others is proposed. Also, the geometric visual features are compared and analyzed for their importance in bimodal speech recognition. One of the unique characteristics of our bimodal speech recognition system is the novel fusion strategy of the acoustic and the visual features, which takes into account the different sampling rates of these two signals. Compared to acoustic only, the audio-visual speech recognition scheme has a much more improved recognition accuracy, especially in the presence of noise.

In modern railway applications, the prevention of wheel skid is very important. This is because wheel skid can lead to an increase in noise and vibration from wheels with flat points, as well as an increased braking distance. However, conventional antiskid control has problems because the train wheel adhesion and skid characteristics are difficult and time consuming to accurately model. In addition, adequate measured numerical data describing wheel skid is difficult and expensive to obtain from actual railway systems. Therefore, a fuzzy logic based antiskid controller was implemented, where both linguistic and numerical system information could be used. In this paper, the design and implementation of the fuzzy logic controller is described. Results show that the antiskid controller has a very good performance, and performs better than a conventional controller. The described controller is currently running in Mitsubishi Electric railway brake sets in both Japan and overseas.

In modern railway applications the prevention of wheel-skid is very important. This is because wheel skid can lead to an increase in noise and vibration from wheels with flat points, as well as an increased braking distance. However conventional anti-skid control has problems because the train wheel adhesion and skid characteristics are difficult and time consuming to accurately model. Therefore a fuzzy logic based anti-skid controller was implemented where linguistic control information could be used. In this paper the design and implementation of the fuzzy logic controller is described. Results show that the anti-skid controller has a very good performance, performing better than the previous conventional controller. The described controller is currently running in railway brake sets in both Japan and other countries.