Roasting coffee beans is the most important process for developing desirable aroma and flavor. A roasting curve is a temperature graph that illustrates the relationship between temperature and time during the roasting process. Roasting curves are typically established by experts with many years of experience. Consequently, it is challenging for novice learners to acquire roasting skills in a short period. To address this, we conducted multiple roasting sessions using various roasting curves to collect training data. We trained a deep learning model using the collected data. Furthermore, we integrated the deep learning model with a custom-developed interface. The resulting prototype system can predict the degree of roast and ratio of moisture loss by inputting a roasting curve and certain parameters, eliminating the need for actual roasting. Thus, it functions as a roasting simulator. We evaluated the roasting simulator by comparing an experimental group that learned roasting using the simulator with a control group that learned through actual roasting. The results indicate no significant difference in roasting knowledge between the two groups; however, the learning performance differed significantly. The roasting simulator has the potential to accelerate the acquisition of roasting knowledge.

Ryuteki is one of the wind instruments in gagaku, a traditional music of Japan since ancient times. This instrument is difficult for beginners to acquire playing skills. The best way to learn how to play ryuteki is to learn from an expert, but the opportunity is limited due to time or cost constraints in gagaku. Therefore, in this study, we developed a system that automatically judges whether the pitch and timing of the sound are correct when a beginner plays ryuteki using sound processing technology, and presents feedback on the results.

As a developmental task for learning phases of the Moon, students are sometimes given the opportunity to study the phases of the Earth from the perspective on the Moon as well as Moon phases from the perspective on the Earth. We proposed a cognitive model of understanding of Moon phases, and we developed an active learning environment in which two students work in pairs: one playing the role of the Moon and the other playing the role of the Earth. We used a technology by which two people can share the same AR space. We verified the effect of the learning environment by an evaluation test.

In general, we recognize an object, determine the next action to choose, and execute the optimal action. Then, the state of the object changes, and the cycle of recognizing it is repeated. The repetition of this cycle of interaction between recognition, judgment, and action can be called an experience. Memories that involve experiences are called episodic memories and are more likely to be retained than mere semantic memories. Therefore, in this study, we propose a learning support environment that allows a single learner to learn with experience. As a prototype system, we use language word learning as the learning content. We developed and evaluated a word learning system using VR that incorporates this kind of cycle in learning. As a result, to change the state of the object were useful for deepening the understanding of word meanings through the cycle and also useful for adding fun to learning.

Currently, there are many contents related to celestial bodies in the world. However, their systems introduce constellations mainly. Most of them don’t introduce information about each star. Existing systems read aloud names of stars and constellations and show their color and brightness only visually. Therefore, in this research, we developed a system that can quickly present information of the color and brightness of stars by associating them with pitches and volumes of sounds. In the evaluation experiment, we compared experimental group using this system with control group using the voice reading system. We verified the speed and accuracy of grasping the information of stars in each system.

We have been developing learning support systems for sketching human body. A learning support system compares an artwork by learners with a model of the human body figure in regard to a skeleton or an outline and presents the difference between them and scores. The system in the previous research had the problem that learners could not freely change the poses they wanted to draw. This research proposed a method to change the pose of a virtual humanoid model by 3D pose estimation of a real human using Motion Capture with a depth camera. We developed a prototype system based on this method in order to improve the pose change. In the evaluation experiment, we tested the usefulness of this system by using the improvement values of the scores of the experimental and control groups before and after learning.

Coffee beans are roasted to bring out their unique aroma and taste. When a user tries to roast coffee beans by a roasting machine, he/she can use a roasting curve in which temperature is set to the vertical axis and time is set to the horizontal axis. In the roasting of coffee beans, the user needs to select an appropriate roasting curve depending on the desired characteristics. However, it is difficult for him/her to know what characteristics he/she wants to produce and what roasting curve he/she should use to roast the coffee beans to produce the characteristics. Therefore, roasting is a skilful technique performed by experts. In this study, we collected various data related to roasting. Furthermore, we developed a learning model that outputs color information of the resulting beans from the input information on roasting using multiple machine learning algorithms. Finally, we compared and verified the accuracy of the model.

In this research, it is verified whether the speed of human task processing can be accelerated by utilizing the knowledge of ICT technology and psychology. In a previous study, it was reported that listening to fast-paced sounds accelerates the pace of human action. Furthermore, it has been reported that the physiological arousal level is improved by viewing a stereoscopic image synchronized with the sound. Based on these findings, it is considered that the action pace is accelerated by combining visual and auditory stimuli, compared with the stimulation alone. In this report, we set reading as a task and conducted an experiment to see if the reading speed of characters could be increased.

In this research, we propose a system which can freely set navigation without using GPS location information and AR marker at any location. The system can be shared with multiple terminals via server. Moreover, in order to verify the usefulness of the system, a verification experiment was carried out at the library in Wakayama University. We divided 20 subjects into 10 subjects for experimental group and 10 subjects for control group. The control group searched the target book using information printed by a system for book search in the Library. The experimental group searched the target book using the information and also using the developed system in this research. The result showed that this system has significant effect.

The purpose of this research is to propose a new drawing learning support environment to replace the conventional learning environment. In conventional drawing learning, it is customary to sketch 3D object on a plane. Therefore, there are many research cases in which a system assists a learner to learn drawing skill, when the learner draw a sketch on a plane. On the other hand, in this research, we aimed at design and development of the environment that can support the acquisition of the drawing skill when the learner is drawing in three dimensions. A virtual 3D object is superimposed on a real 3D object using augmented reality, and the positional relationship between the real 3D object and the virtual 3D object is linked. As a result, we have realized a system that allows students to easily compare pictures in real 3D object with virtual 3D object when painting.

In thinking about human-computer interface, it is important to know how human beings recognize, judge and act. Therefore, Card et al. devised a model of human cognitive processing process called Model Human Processor (MHP). However, this MHP predicts processing time from empirical rules, and is not much considered from the aspect of brain activity. In this study, we focused on readiness potential (RP), which is a characteristic brain wave, and verified MHP from the aspect of brain activity by measuring Electroencephalography (EEG) when performing basic user performance. Experimental results show that MHP can be roughly explained from the aspect of RP, which is a characteristic brain activity.

There are a lot of people who are not good at learning history, since they are simply trying to memory terms. Certainly, memorization of terms is also necessary, but just reading and remembering words does not mean that they understand history, and it is difficult to keep in memories. The most important thing in learning history is to know the flow of what caused an event, what process it took and what kind of result was caused. However, it is difficult for those who do not know this to notice on their own, and a new teaching material that can learn historical facts is necessary. In this research, we set up a system to learn by tracing the flow of historical facts, using Sekigahara battle in Japanese history as a test case, and remove the weak consciousness in learning in Japanese history. In addition, we aim to increase learner’s interest in Japanese history and improve learning motivation. In addition, we aimed to be able to deepen understanding by introducing AR marker into the system, seeing it and moving it by hand. In order to confirm that the system is useful, 12 subjects used this system, and they answered a questionnaire about their feelings.

Virtual human figures have been used in the conventional human figure sketch learning support system. A learner selects one of the virtual figures for a motif. However, the learner could not intuitively manipulate the virtual human figure in real time to change the orientation of the figure for learner’s favorite viewpoint. Therefore, in this research, we developed a system to change the orientation of the virtual model of human figure intuitively by using the drawing doll as a tangible interface. Specifically, it uses PCL which can acquire point cloud data by RGB-D camera of KINECT, then it acquires and tracks the three-dimensional coordinates of the real object and superimposes and displays the virtual human figure on the real drawing doll. In the verification experiment tracking accuracy was verified and the improvement of the system was discussed.

Distance recognition is the skill to estimate the distance to the target object. Such a distance recognition skill is one of the most important skills in everyday life. In various situations, such as walking, driving a car, communicating with people and so on, humans perceive the distance between the targets, and do the adequate action. Generally, the precision of distance recognition is different according to difference of the ambient surroundings, the objects, and especially the individual’s ability. The distance recognition is achieved by capturing the target object visually and processing its visual information in their brains. Therefore, it is important to investigate how the difference of distance recognition skills affects the brain activity. In the paper, we measure the cerebral blood flow as a brain activity by using NIRS (Near-Infrared Spectroscopy) in the distance recognition.

We developed a learning support environment in which learner’s motion in real-time is synchronized with archived target expert’s motion. It is an effective method for a learner to compare his/her motion with a target expert’s motion, and many existing learning environment make a learner follow target expert’s motions. However, it is not always easy for a learner to follow a target motion, since the target motion is sometimes fast. To solve this problem, we developed a method to synchronize target expert’s motion with learner’s motion in real-time by using dynamic programming method (DP method). Even if the learner moves backward on the timeline of archived expert’s motion, the archived expert’s motion is also replayed backward. When a learner tries to compare his/her posture with the target posture, it is easier for the learner to find his/her defects by using the function of motion synchronization. We verified the effect of the prototype system by questionnaire survey.

We repeat the interaction cycle of recognition, judgment and action while practicing skills. The interaction cycle was modeled by Card et al. from the viewpoint of information processing, which is called as Model Human Processor. Model Human Processor can estimate the reaction time of the cycle. However, Model Human Processor does not estimate processing time from the aspect of brain science, but estimates it by empirical model. In this study, we performed four experiments according to Model Human Processor and measured brain activity by Electroencephalography (EEG). As a result of experiments, characteristic EEG was detected. Also, it was found that characteristic EEG was appearing late as well as models delayed cognitive processors of reaction decisions.

According to social rank, high rank people are located “Up” and low rank people are located “Low”. “Up” and “Low” are concept of spatial arrangement. However, they are used as metaphors to represent the social rank. This study describes psychophysical experiments to confirm whether there is a link between the “Up and Low” of the spatial arrangement and “High and Low” of the social rank. In the experiments, two randomly words from three words that have social ranks were shown to subjects vertically on the display. We taught the subjects a target word at the beginning of the experiments. We asked the subjects to check whether the target word appears. If the target word appeared, subjects pressed the button as soon as possible. We obtained reaction times of the button press by psychophysical experiments. We assumed a linear model that contains three elements: “Time which differs between the target word”, “Time which differs between the spatial arrangement of the words to be presented”, “Time which differs between the social rank”. The results show that the reaction time in case that social rank coincides with spatial arrangement is significantly shorter than the reaction time in case that social rank does not coincide with spatial arrangement.

Japanese traditional art works, like ceramics painting, need some skills of painting onto three-dimensional objects. Training is needed for a long time to get their skills. Therefore, it is difficult for novices to draw pictures they want. This is the reason that we developed the drawing learning support system using AR. There are two barriers for novice learners to learn drawing, difficulty of fixing their mistakes, and a lot of cost for learning. Using the system, the learner is able to break through these barriers. Our final goal is to develop a skill learning environment for drawing onto 3D objects. The environment assists a learner to get skills for drawing lines keeping uniform width. This skill is available to draw lines with various widths that the learner wants. To accomplish it, the system overlays strokes by AR, and gives advice using text or voices. First, the learner traces strokes overlaid on an object. Next, the system diagnoses the learner's stroke. Finally, the system gives advice to the learner by text or voices. In this paper, we propose and explain the design of the skill learning environment for drawing onto 3D objects using AR.

Augmented Reality (AR) in the traditional systems have a problem that the drawn objects are always displayed in the foreground because 3D models by AR are superimposed later than the picture of the actual world. This paper proposed a system to produce a realistic picture of AR in accordance with every depth. We developed a prototype system to verify the effect of the method. The prototype system was developed by focusing on a human hand. This paper utilized a Leap Motion Controller as a motion capture device to acquire the depth data of the hand and fingers.

Our laboratory has developed a skill learning support environment “Motion Navigator”, by which a learner can train himself/herself for a motion just by following model motion displayed on his/her HMD with AR. we have developed a golf learning support environment by improving “Motion Navigator”, which takes the necessary elements into account for a learner to mimic and to practice golf swing. By displaying learner's operation superimposed with the model motion simultaneously, the system can give the learner awareness of mistake of his/her motion.

Traditional motion learning support systems have a single piece of motion data as a model motion although the data is not always the best one for learners. There is also a fear that some learners regard the model motion as the absolute one. To solve this problem, we proposed a system showing various model motions to the learners. They can select one of the following model motion; arbitrary coach's motion, the coach's motion whose physique is the most similar to learner's one, the average motion of all coaches and the nearest motion to the average. On the system, learners are able to choose one from these four types as a model motion if the system has some coaches’ motion data. Through an evaluation experiment, we concluded that the system was helpful for motion learning.

Recently, more and more hearing person starts learning sign language. For finger motion skill learning such as sign language, it is important for an expert to give objective advice to a learner, because the learner is not able to find any motion errors in his/her sign language. In other words the learner needs the self-education tool of sign language which gives good objective advice. However, there are few studies about such a system. In this paper, we developed a finger motion skill learning support system using data gloves. This system helps a learner to recognize motion errors intuitively by himself/herself by overlaying skilled person's 3D models with learner's 3D models of hands and fingers.

This paper describes the development of a gesture learning environment for a novices’ erhu bow strokes, which diagnoses bow strokes by using magnetic position sensors and feedback errors by showing bow strokes on virtual 3D space and coloring. The system has two windows: bow strokes window for visualization of bow strokes and feedback errors and score window for diagnosing bow strokes along score. In addition, we evaluated the system by Likert scale questionnaire survey. The result of questionnaire indicates that the novice learner was able to recognize and improved errors of bow strokes. It was difficult for novice learners to do that by using conventional method. Moreover, our experiment shows the system useful to learn gesture of bow strokes.

Abstract In recent years, it is difficult to inherit high level sensory skill, because the number of experts is not so much or the experts are too busy to teach their skill to the beginners. Therefore, many learners do the experiential learning through visual and haptic digital teaching materials. In such a system, however, it is difficult to evaluate whether the learner could recognize the sensation and obtain the sensory skill. In the paper, we investigate whether the biological signal such as EEG can be used for the evaluation of the haptic task skill level.

In this study, we build a new system “Motion Navigator II”, which displays finger motion integrated with whole body motion with bone CG animation by AR. The function to display whole body motion is inherited from an existing motion skill learning support system “Motion Navigator”. The existing Motion Navigator can’t display fingers’ motion but shows limited variation of motion in precedent research. Therefore we acquired finger motion data using data gloves and improved Motion Navigator integrating the finger motion data with whole body motion data. We verified learning effect of the experimental group using Motion Navigator II in comparison with the control group using conventional video.

This paper describes the method for classification of brain state by the measured electroencephalogram (EEG) frequency in directions (up, down, left, and right) imagination. Recently, Brain-Machine Interface (BMI) has been studied in a variety of ways due to the development of brain measurement technology. Therefore, we have used the BMI to identify the human selection of directions. Our method consists of data normalization, principal component analysis and neural network. The maximum value of the identification rate was 46% by using 3 electrodes (F4, F8 and T8) in the previous study. In this study, we improved the learning method of neural network for the improvement of identification rate of brain state. For that purpose, the measurement points of EEG and the number of subjects are increased. As a result, the maximum value of the identification rate was improved.

Existing application software for touch typing training cannot diagnose fingering errors. Given this fact, we developed a skill learning environment for touch typing training that can diagnose fingering errors by recognizing fingers with color markers using image recognition technique. This study developed two systems: a learning support environment for an experimental group and a learning environment for a control group. We evaluated the effect of the learning environment that can diagnose fingering errors for the experimental group, by comparison with the other learning environment for the control group.

In late years, the ITS which aims at improvement of convenience and the comfort in the road traffic using an information technology attracts attention, and service to deliver useful information quickly by communication among vehicles is flourishing research area. However, vehicles may be distributed sparsely by traffic conditions in VANET, and it is difficult to maintain stability of the communication. Therefore, DTN technology that communication is enabled by permitting a delay even if vehicles are distributed sparsely can be applied in such an environment. However, a delay occurs for the delivery of data when I apply DTN environment. Therefore, it becomes difficult to deliver a reply message to the requesting vehicle due to movement of the vehicle before message arrival. For such a problem, we aim at the arrival rate improvement of the reply message by while the requesting vehicle waits for a reply, advertising the own position on neighboring appropriate vehicles. Then, we evaluated our method which distribute an advertisement to vehicles nearby and suggested the method using the flooding and the method using geocast. We investigated that an arrival rate of the reply message is improved by our proposed method by computer simulation.

In this paper, we propose two new methods to learn and practice non-dominant motion using Motion Converter and Motion Navigator. One is a method of non-dominant motion practice by following reversed dominant motion animation. The other is awareness of non-dominant motion error by dominant body site. We think that both methods are complementary each other. Therefore, a learner can practice his/her non-dominant motion and get awareness of non-dominant motion error by trying both methods.

This paper presents a proposal of two new modes of collaborative human navigation. Both are used in a situation in which two users depart from different points simultaneously, but they want to reach common goal. One mode of navigation is used in a situation in which both users want to reach the goal independently. The navigation system calculates each route and arrival time for each user. If the two arrival times differ, then the system suggests another route with a dropping point such as a store or park to the user who will reach the goal earlier. The other navigation is used in a situation in which two users want to join somewhere before reaching a common goal. The navigation system suggests not only a juncture point, but also suggests routes from each departure point to the juncture point, and a route from the juncture point to the goal. We developed a prototype system using the city center of Wakayama as a test area. Using it, we evaluated the proposed navigations.

We developed a real-world oriented mobile constellation learning environment. Learners point at a target constellation by gazing through a cylinder with a gyro-sensor, which can display information related to the constellation. The system has a learning function. Through experimentation, we evaluated the learning environment to assess its learning effects.

We constructed a sketch learning Environment. Instead of a real bowl and a real glass, virtual objects represented as 3D CG objects in a computer are used as motifs and are displayed in real space using augmented reality techniques. Consequently learners are able to compose freely. We conducted an evaluation experiment. The experimental results obtained from an experimental group and a control group were compared, revealing that the learning effect was more pronounced in the experimental group.

Through experimentation and analyses of apple peeling, we examined the implicit knowledge necessary to perform study support for exercise and performance. A pressure sensor, an eye-mark recorder, a miniature wireless camera, and a surface electromyography were used to assess actions of 10 examinees (6 men, 4 women) during an experiment, which revealed three main differences between experts and non-experts: holding of the kitchen knife using the index finger to the little finger, use of the right thumb, and holding the apple with the left hand. Results show that securely holding the apple and the kitchen knife is important. Finally, we designed the skill learning support system.

We developed star learning environment with magnetic sensors. This environment has three functions. First function is star name teller. When a learner point out a star with the sensor on his/her finger top, the system tells him/her name of the star and constellation with voice. Then, second function is star navigator. If the learner know the name of a star, but he/she does not know where it is in the real sky, the system shows the path from current pointing position to the target star. Third function is constellation tutor. We developed guidance and diagnosis function of constellationspsila shapes with Wii remote controller.

A picture is one of important research subjects in order to make our life spiritually rich. Most studies on pictures, however, only propose some substitute functions of actual drawing or painting materials. There is no system that evaluates pictures drawn by users and gives advice about them. We propose a learning support system for basic techniques in beginner's pencil drawing. The proposed system receives a subject (motif) data set and an image of user's sketch and returns advice to the user. The system is composed of the four subsystems: feature extraction of motifs, feature extraction of sketches, error identification, and generation and presentation of advice. We developed and experimented a prototype system limited to treat a basic motif and some principal advice. As a result, the validity of the proposed system was confirmed.

In a pencil drawing class, a teacher explains often the differences between a drawing and a motif by many ways; even so, it is difficult for most learners to understand the existence of errors in his/her drawing. When a learner watches the 3D model of his/her drawing from appropriate viewpoints, he/she is immediately aware of the errors. This paper describes the technique of error visualization from a learner's erroneous drawing. We developed a drawing advice system that consists of a 3D model converter from a 2D image and an evaluation function to estimate the effect of the 3D model. We designed a geometrical formula that expresses the relation between the geometrical feature and the error expression in the teacher's advice.

We developed a remote meteorological observation system and interactive data publishing system on the web. Meteorological sensors collect data and send it to the DBMS. When students retrieve past data on the web, the data are brought with XML. Therefore the data can be transformed by XSLT with XSL style format in accordance with students' demand. The system has flexible GUI by using XML. Also the system can generate some pairs of problem and answer automatically to learn meteorology. This system will contribute to science education.