Abstract This paper describes research into the development of an artificial intelligence (AI) based music therapist (AI music therapist) for elderly people with dementia. Music therapy is expected to be effective for the ‘behavioural and psychological symptoms’ of dementia and thus contribute to reducing the burden of caregiving. However, the number of music therapists in Japan is very small compared to the number of elderly people with dementia, and not all elderly people with dementia in Japan have easy access to music therapy. Therefore, in order to enable elderly people with dementia to receive music therapy anytime, anywhere and as many times as possible, we are developing a virtual music therapist (AI music therapist) called ‘Satsuki-san’, who appears on a computer screen. In this paper, we report on the latest version of ‘Satsuki-san’ and collect opinions from caregivers actually engaged in the care of elderly people with dementia and music therapists with extensive clinical practice experience with elderly people with dementia. The opinions of the caregivers and music therapists showed a high level of expectation for the AI music therapist, but they also pointed out a number of problems, for example about Satsuki’s speech, such as the need for more intonation. The music therapists also pointed out a number of areas for improvement with regard to singing accompaniment. In the future, we would like to collect a variety of data from the subjects of music therapy through the use of the AI music therapists and establish a music digital therapeutics that can objectively evaluate the subjects’ condition and predict the progression of dementia.

In this paper, we describe our research on the possibility of automatically grouping textual information on behavioral and psychological symptoms of dementia stored in the Ninchisho Chienowa-net using the latest deep learning models for natural language processing. Ninchisho Chienowa-net is a web system that publishes the probability that coping methods for various symptoms occurring in patients with dementia will “go well” (probability of success) by collecting care information in the form of the posting from dementia caregivers and by grouping the posted care information according to the similarity of the care information. This grouping has been done manually by clinicians, but since the workload of clinicians has been increasing, we will verify the possibility of automating this process using deep learning models through experiments and investigate practical feasibility in Ninchisho Chienowa-net web system. As a result of the experiment, we constructed a learning model for automatic grouping with 20 epochs after multiple types of data preprocessing, and confirmed that the model can output the correct group within the top five with a probability of more than 90%. Furthermore, for the purpose of improving the performance of the automatic grouping model, we conducted data augmentation of text data using a translation service, and confirmed that we could output the correct groups within the top five with a probability of 98%.

This paper introduces the newest features and analysis of access logs of Ninchisho Chienowa-net, a web system that publishes the probability that coping methods for various symptoms occurring in patients with dementia will "go well" (probability of success). With an aim of reducing the burden on dementia caregivers, Ninchisho Chienowa-net not only publishes the probability of success of coping methods and information on dementia care, but is also equipped with features to collect experiences of coping methods for specific symptoms and to support the discovery of various coping methods. Since its launch in 2015, the site has seen a steady increase in the number of registered users and amount of access. Analysis of the access logs of Ninchisho Chienowa-net carried out in this study revealed that the access rate for the list of individual care topics is high and that it is necessary to prepare for large-scale access in the future.

This paper proposes a grouping technique of sentences that contain dementia care information and describes analytical results of statistical information which influences the grouping. This paper focuses on the grouping of the "incidents" in sentences. One of the problems to group the incidents is that the sentences input by caregivers are redundant. In other words, the sentences contain care information which were related to the incidents, but were unnecessary for the grouping. In order to extract the incident information from the sentences, we divide them into sub-sentences for every verbs and auxiliary verbs. Similarities among sub-sentences are used for the grouping and the accuracy is evaluated with the precision. As a result, we found that a sub-sentence containing seven words is good for the grouping.

This paper presents our web system to collect and publish good dementia care techniques for caregivers. The system collects a lot of dementia care information from caregivers and, in the future, it will extract good dementia care methods from the information by using the text mining technology. The web system was released for about 40 users in July, 2015 to validate the system functions and to get feedbacks for the system. The system was well received by the users. We also found that the caregivers spend about 20 minutes on average for inputting care information in a single session. In addition, the number of input data of the choice was almost double of those of the blank input. We will improve the web system based on these results and the feedbacks from the users in the future.

To maintain and uplift the motivation for doing exercise which is tend to be too simple in rehabilitation and ergotherapy, doing exercise with music is one of the good solutions. In this paper, we confirm that doing exercise with music is fun. Next, we design five types of musical features which relate to exercises. These features are evaluated by doing five types of exercises with listening to the tunes. Based on the results obtained from the experiments, we found many new things, for example, tempo has the adjustment limitation based on how well the user knows the tune. In this paper, we report the latest results from the experiments and explain the way to progress of this research.

This paper describes a prototype system of remote music therapy for elderly people with dementia using the latest communication technology in Japan. The system uses the latest high-speed optical network, called NGN (Next Generation Network) and the latest terminal especially for audio data communication called Hikari DUETTO NY1, which was released at the end of the last year. We did an experiment of the system using the real communication environment in this January between Tokyo and Osaka. The communication time of the system between Tokyo and Osaka was about 25.5 msec. Thus, we confirmed that the system was able to provide almost real-time audio data communication. Based on the result, the feasibility of music therapy activities, such as singing and exercises with music, using the system was examined. We confirmed that the singing activity which is the most important one in the music therapy was feasible.

This paper describes a new method of music genre recognition. Even for people it is difficult to define musical genres, because a genre is something more than a set of rules. Automation of this task could improve the work of multiple audio-related WEB portals, such as audio-libraries, and could simplify human activity in other music-related areas. For music genre recognition, we introduce the instantaneous frequency spectrum (IFS) whose calculation is based on the Hilbert-Huang transform. In our method, IFSs of audio signals are generated from their instantaneous frequencies and used to calculate music genre templates. The experimental results for three test music pieces show that the method can accurately detect and differentiate genres of tunes. Slicing test music into frames and recognizing genres for short fragments of a whole music piece gives a precise description of a piece's internal structure, which could help to enhance people's understanding of the music. Presentation of this information also is an advance in music visualization.

This paper proposes a method to identify and visualize repetitive structures in music in PCM format using a similarity matrix. The Hough transform is used on the matrix to identify repetitions in a tune. Identified structures are visualized on an image called misual. Misual is a colored cylinder of a varying diameter, where colors represent repetitions and the diameter represents volume changes. The performance of the identification and visualization is evaluated with well-known classical tunes. Misual shows not only the dominant repetitions but also repeated patterns which can not be easily recognized while listening to a tune or reading its score. By revealing these patterns, misual gives a new point of view for musical structure analysis and at the same time gives people a new way to appreciate music.

This paper discusses how people feel the speed of tunes and proposes a way to express it with images. In Misual Project, the research is done to provide people with a way to learn/know music other than by listening to it. This paper focuses on the speed feeling of music, since it is an important musical characteristic for listeners. Although tempo itself is an important indicator of the speed of music, we found that the number of notes per time unit and rhythms (combination of regular and dotted notes) are also important. How and how much these factors contribute to the speed feeling is investigated in listening tests with simple tunes. The factors are visualized so that people can intuitively imagine how fast a tune is. A selection of famous tunes is visualized and the advantage of music visualization is discussed. The visualization allows us to look at a tune as a whole and to compare tunes.

This paper describes a music visualization method called Misual. The goal of this research to provide objective and static information regarding musical impressions that will enable people to efficiently categorize a huge number of music pieces and select favorites from them. Features are directly extracted from music and then visualized in an image that suits human intuition. As a first step in this research, volume transitions and repetitions are focued on as the visualized objects. For the volume transition, power information is extracted and then smoothed by a moving average. This information is continuously visualized with 3D-like images, in which radii of circles reflect the volume information. Repetitions are visualized with color. These are detected on the basis of the frame-to-frame similarity measured by the Mel-frequency cepstrum. These two kinds of information were visualized for three pieces of real music. The visualizations will be useful for roughly grasping the musical features in a very short time and imagining the music intuitively.

In this paper, we describe music monitoring in TV broadcasting based on content-based retrieval. A part of audio signals is sequentially extracted from TV broadcasting as a retrieval key, and a music DB that stores a great number of musical pieces is retrieved by this key based on content-based retrieval, and a musical piece is identified sequentially. In this way, we are able to carry out music monitoring. There are three necessary requirements important for realization of the music monitoring. They are robustness against non-stationary noise, real-time processing of large-scale music DB retrieval, and high granularity of the retrieval key. As a method of realizing robustness against non-stationary noise, we propose a partially similar retrieval method which improves retrieval accuracy by using the moment in which no superfluous noise is produced during the existence of non-stationary noise. In order to realize real-time processing of large-scale music DB retrieval, we adopt a coarse-to-fine strategy, and propose a spectral peaks hashing method which performs high-speed refining by using hashing. To calculate a hash value in this hashing, frequency channel numbers of the spectral peaks are used. In order to realize high granularity of the retrieval key, it is necessary to solve the problem of retrieval accuracy degradation associated with heightening the granularity. To improve this accuracy, we propose a detection-by-continuity method which uses music continuity. Moreover, by using music continuity to correct the starting point and the terminal point of a musical piece in TV broadcasting, the retrieval accuracy is improved further. In order to evaluate the effectiveness of the proposed methods, we performed experiments using a music DB which stores over 28,000 musical pieces (over 1800 hours) and TV broadcasting audio signals containing music and background music (BGM). The granularity of the retrieval key was set at about 0.5 seconds. Through these experiments, We verified that music monitoring was possible for over 90% of the total time of music and BGM used in TV broadcasting, and that real-time processing was possible.

This paper describes our practical query-by-humming system, SoundCompass, which is being used as a karaoke song selection system in Japan. First, we describe the fundamental techniques employed by SoundCompass such as normalization in a time-wise sense of music data, time-scalable and tone-shiftable time-series data, and making subsequences for efficient matching. Second, we describe techniques to make effective feature vectors based on real music data and do matching with them to develop accurate query-by-humming. Third, we share valuable knowledge that has been obtained through month's of practical use of Sound Compass. Fourth, we describe the latest version of the SoundCompass system that incorporates these new techniques and knowledge, as well as describe quantitative evaluations that prove the practicality of SoundCompass. The new system provides flexible and accurate similarity retrieval based on k-nearest neighbor searches with multi-dimensional spatial indices structured with multi-dimensional features vectors.

A music retrieval system that accepts hummed tunes as queries is described in this paper. This system uses similarity retrieval because a hummed tune may contain errors. The retrieval result is a list of song names ranked according to the closeness of the match. Our ultimate goal is that the correct song should be first on the list. This means that eventually our system's similarity retrieval should allow for only one correct answer.

No abstract available.