Some elderly care homes attempt to remedy the shortage of skilled caregivers and provide long-term care for the elderly residents, by enhancing the management of the care support system with the aid of smart devices such as mobile phones and tablets. Since mobile phones and tablets lack the flexibility required for laborious elderly care work, smart AR glasses have already been considered. Although lightweight smart AR devices with a transparent display are more convenient and responsive in an elderly care workplace, fetching data from the server through the Internet results in network congestion not to mention the limited display area. To devise portable smart AR devices that operate smoothly, we first present a no-keepalive-Internet and privacy-compliant required smart hands-free elderly care support system that employs smart glasses with facial recognition and text-to-speech synthesis technologies. Our support system utilizes automatic lightweight facial recognition to identify residents, and information about each resident in question can be obtained hands-free link with a local database. Moreover, a resident information can be displayed immediately on just a portion of the AR smart glasses on the spot. Due to the limited size of the display area, it cannot show all the necessary information. We therefore exploit synthesized voice in the system to read out the elderly care-related information in its entirety. By using the support system, caregivers can gain an understanding of each resident condition immediately, instead of having to devote considerable time in advance in obtaining the complete information of all elderly residents, especially of new residents. Our experiments on this support system were conducted at an elderly care home in Tokyo. Our lightweight facial recognition model achieved high accuracy with fewer model parameters than current state-of-the-art methods. The validation rate of our facial recognition system in the practical use scenario was 99.3% or higher with the false accept rate of 0.001, and caregivers rated the acceptability at 3.6 (5 levels) or higher.

In this paper, we utilize facial action units (AUs) detection to construct an end-to-end deep learning framework for the macro- and micro-expressions spotting task in long video sequences. The proposed framework focuses on individual components of facial muscle movement rather than processing the whole image, which eliminates the influence of image change caused by noises, such as body or head movement. Compared with existing models deploying deep learning methods with classical Convolutional Neural Network (CNN) models, the proposed framework utilizes Gated Recurrent Unit (GRU) or Long Short-term Memory (LSTM) or our proposed Concat-CNN models to learn the characteristic correlation between AUs of distinctive frames. The Concat-CNN uses three convolutional kernels with different sizes to observe features of different duration and emphasizes both local and global mutation features by changing dimensionality (max-pooling size) of the output space. Our proposal achieves state-of-the-art performance from the aspect of overall F1-scores: 0.2019 on CAS(ME)2-cropped, 0.2736 on SAMM Long Video, and 0.2118 on CAS(ME)2, which not only outperforms the baseline but is also ranked the 3rd of FME challenge 2021 for combined datasets of CAS(ME)2-cropped and SAMM-LV.

Watching TV once encouraged generations of families and friends [11] to communicate and share empathy. However, the Internet is changing how we watch TV and reducing interaction, leading to problems such as lack of self-control and inadequate communication skills [17]. To understand the conversations while watching TV, we design a scheme based on human conversational behavior [2], and then develop a prototype of TV-watching companion robot supported by the chatbot “KACTUS” [20]. The robot generates a disclosure utterance (e.g., ”I like elephants”) with extracted keywords from the TV program in “TV-watching mode” and uses a cross-topic dialogue management method from “KACTUS” with question utterance to respond with rich conversations in ”Conversation mode”. The robot switches between these two modes at a preset ratio (TV-watching:3, Conversation:1) and behaves like a human enjoying TV-watching. The result of initial experiment shows that three groups of participants enjoyed talking with the robot and the question about their interests in the robot were rated 6.5 (7-levels: ascending from ”extremely disagree” to ”extremely agree”).

Group cohesiveness reflects the level of intimacy that people feel with each other, and the development of a dialogue robot that can understand group cohesiveness will lead to the promotion of human communication. However, group cohesiveness is a complex concept that is difficult to predict based only on image pixels. Inspired by the fact that humans intuitively associate linguistic knowledge accumulated in the brain with the visual images they see, we propose a linguistic knowledge injectable deep neural network (LDNN) that builds a visual model (visual LDNN) for predicting group cohesiveness that can automatically associate the linguistic knowledge hidden behind images. LDNN consists of a visual encoder and a language encoder, and applies domain adaptation and linguistic knowledge transition mechanisms to transform linguistic knowledge from a language model to the visual LDNN. We train LDNN by adding descriptions to the training and validation sets of the Group AFfect Dataset 3.0 (GAF 3.0), and test the visual LDNN without any description. Comparing visual LDNN with various fine-tuned DNN models and three state-of-the-art models in the test set, the results demonstrate that the visual LDNN not only improves the performance of the fine-tuned DNN model leading to an MSE very similar to the state-of-the-art model, but is also a practical and efficient method that requires relatively little preprocessing. Furthermore, ablation studies confirm that LDNN is an effective method to inject linguistic knowledge into visual models.

We present an arbitrarily angled audio-visual VR experience app called OtonoVR for 360-degree panoramic videos using our selective synthesis sound field technique. This technique can synthesize two-channel stereo sound with scaled stereo width having an arbitrary angle range from 0 to 360 degrees centering on an arbitrary direction from multi-channel surround sound based on spectral modification. In the app, users can enjoy arbitrarily angled videos that they choose themselves by manipulating the touchscreen, and the stereo sound changes in terms of its spatial synchronization depending on the view. The app has been released for iOS and has been officially endorsed by Japanese idol groups.

Utilization of web images with social tags as training data has been a major trend for the development of automatic image tagging/classification systems. While the amount of information available on web sites such as Flickr is abundant, the majority of information obtained from such sites is in English, the dominant language on the web. This linguistic unbalance is expected to affect auto-tagging results in a negative way for non-English users, who demand image tags in their native languages. The objective of this research is to develop an image auto-tagging system which can generate tags in languages other than English. This paper examines the effect of linguistic unbalance in training data to construct auto-tagging systems, which aim to generate tags in minor languages. Furthermore, we propose methods which utilize an auto-tagging model generated on English training data, and convert the auto-tagging results to the target language. Subjective evaluations show that the proposed method is capable of generating auto-tagging results with better quality than conventional approaches.

Scalability is one of the main challenges for kernel-based methods and support vector machines (SVMs). The quadratic demand in memory for storing kernel matrices makes it impossible for training on million-size data. Sophisticated decomposition algorithms have been proposed to efficiently train SVMs using only important examples, which ideally are the final support vectors (SVs). However, the ability of the decomposition method is limited to large-scale applications where the number of SVs is still too large for a computer's capacity. From another perspective, the large number of SVs slows down SVMs in the testing phase, making it impractical for many applications. In this paper, we introduce the integration of a vector combination scheme to simplify the SVM solution into an incremental working set selection for SVM training. The main objective of the integration is to maintain a minimal number of final SVs, bringing a minimum resource demand and faster training time. Consequently, the learning machines are more compact and run faster thanks to the small number of vectors included in their solution. Experimental results on large benchmark datasets shows that the proposed condensed SVMs achieve both training and testing efficiency while maintaining a generalization ability equivalent to that of normal SVMs.

In this research, we propose a method to automatically generate a landmark identification system for geo-tagged photographs, based on analysis of various data collected from the Web. The method first conducts Web analysis based on three major procedures: (1) Automatic extraction of points-of-interest (POIs) based on geographical clustering of geo-tagged images, (2) Retrieval of landmark candidates for each extracted POI from search results of map search API, and (3) Collection and feature extraction of Web images of the landmark candidates. The system then identifies the landmark which appears in the query geo-tagged photograph, by comparing the location and content-based features of the query with the information accumulated by the previous procedures. Experimental results show that the proposed method is capable to construct a highly accurate landmark identification system by leveraging Web information.

Automatic beat estimation is an essential technology not only for fundamental analysis of music, but also for the development of advanced music applications such as DJ mixing. In this paper, we propose an autocorrelation based beat estimation method, which, unlike existing methods, is capable of accurately detecting the position of beats, even from songs with drastic change of tempo. Our proposal consists of two major steps. First, the approximate positions of the beats are pre-estimated by a long-sized analysis window. Next, the preestimated beats are verified by using a short analysis window. These steps are repetitively executed until the beat estimation results converge. Experiments conducted with audio signals and songs with artificially applied tempo changes have proved that our proposal can detect beats with high accuracy.

This paper proposes a high quality coding mode decision method for MPEG-2 to H.264 transcoding. Here, we evaluate the coding modes of MPEG-2 stream in order to specify whether they should be inherited or changed in H.264 coding since many new coding schemes such as intra prediction have been introduced in H.264. By adaptively determining suitable intra/inter modes according to DCT coefficients, motion vectors (MVs), and neighboring macroblock (MB) modes of input MPEG-2 stream, high quality transcoding has been realized. In the experiment, we observed that the proposed method can improve a peak signal to noise ratio (PSNR) up to 0.29 dB at the cost of 22% more processing time compared with the conventional transcoding method.

This paper proposes a novel visual cryptography method that embeds a natural secret image into two natural carrier images with high quality at low computational complexity. The secret image is decoded visually by superposing the two carrier images. For some applications such as entertainment use, all three images should be high quality natural images and low computational complexity is also required. However, in the conventional methods, picture quality and processing speed have been tradeoff relations. To solve this problem, we propose a method which is based on conventional visual cryptography using error diffusion halftoning and introduce an additional feedback mechanism to embed the high quality natural image. Experimental results show that the proposed method can achieve higher quality natural images of a superposed image and carrier images while computational complexity is maintained low when compared with the conventional methods.

This paper proposes a high speed bit rate reduction technique on the compressed domain for AAC audio. There are many kinds of AAC audio devices such as PCs and mobile terminals. In order to adapt to various requirements on audio quality and storage capacity, a fast audio transcoding method from a higher to a lower bit rate would be beneficial for the efficient use of master AAC contents. Although the scalability function of AAC or baseband transcoding, which converts a bitstream by decoding and re-encoding, can be employed for this purpose, lower coding efficiency or higher computation load is induced. To resolve these problems, this paper derives the relationship among MDCT, quantization and scaling, and achieves a fast and high quality bit rate reduction by introducing (1) an adaptive scalefactor control, (2) MDCT gain control and 3) the reuse of side information. Experimental results show that the proposed method is about 1.4 times faster than baseband transcoding while achieving almost the equivalent quality.

We propose a fast transcoder from digital video (DV) to MPEG-4 in the coded domain. Since DV is interlaced sequence whereas MPEG-4 (SIF) is progressive sequence and different discrete cosine transform (DCT) mode $({\hbox {2}}\ast{\hbox {4}}\ast{\hbox {8}}{\hbox {DCT}})$ is used in DV, different compressed domain transcoding method from that of MPEG to MPEG conversion is required. We have exploited matrix conversion reflecting these properties and introduce approximation and integration of resolution conversion and quantization process. Simulation results of DV to MPEG-4 conversion show that the proposed method can achieve very fast conversion while maintaining high transcoding performance when compared with base-band transcoding and siginificant improvement over conventional method is also realized.

This paper proposes a fast MPEG-2 to H.264 intra frame transcoding method. It deploys a novel technology in the refering process of MPEG-2 discrete cosine transform (DCT) coefficients to determine the block size and the prediction mode of intra frame prediction for H.264. The proposed method reduces the coding complexity by approximately 60% while maintaining a peak signal to noise ratio (PSNR) compared with a typical baseband conversion.

In this paper, visual cryptography for halftone images is proposed, which uses threshold arrays generated by a void-and-cluster method. The proposed halftone visual cryptography (HFC) using threshold arrays shows great flexibility for a combination of share images in order to decode a secret image. Moreover, it also achieves high quality share images, high speed processing and extensibility to (k,n) the visual cryptography of three share image categories, such as (2,3) or (3,3). Simulation results show that all the above features are achieved, while a secret image is clearly decoded, and the secret image cannot be seen on each share image.

A novel video watermarking method is proposed, which can directly embed a watermark into a bit stream generated by an H.264 video encoder. Watermarking to encoded data provides a content provider of flexible capability to attach secret information; however it brings a drift problem. A drift compensation method has been proposed, however it requires many computations such as DCT and motion compensation. Therefore a new drift compensation method with spatiotemporal watermarking is proposed that only cancels the effect of the embedding on blocks with little motion. It achieves great reduction of computation and realizes fast and high picture quality watermarking. The computer simulation shows that degradation of picture quality is negligible compared to noises due to H.264 compression, and that the proposed method is highly resilient to frequency domain attacks

In this paper, we propose an efficient algorithm for conversion of an MP3 stream into AAC. Generally, this kind of conversion, transcoding, requires full decoding and re-encoding. However, the re-encoding based transcoding process may result in degradation of quality and take longer than encoding from a PCM signal. This paper proposes a method to inherit the frame structure and quantization scale from MP3 to AAC. This enables a reduction in the iteration process, which requires the most time in the AAC encoding process, without incurring degradation in quality. Experimental results show that the proposed method can perform bitstream domain transcoding at high speed while maintaining a high level of audio quality

We propose a new algorithm for the fast conversion of MPEG-4 from DV. DV, which is a video compression coding format, is adopted by commercial and non-commercial digital video cameras and achieves high video quality using a fixed high bitrate of 25 Mbps. However, compared with MPEG-4, which is used extensively in IP or mobile phone applications, DV has a higher storage and transmission cost. This is due to the extremely low coding efficiency of DV required to maintain high quality for frame-accurate editing. In this paper, we analyze the difference between DV and MPEG-4, and propose a new algorithm for the fast conversion of MPEG-4 from DV. Our algorithm accelerates the conversion process in the coded domain using the property of conversion matrices, where resolution conversion, quantization, and inverse quantization are integrated. The experimental results indicate that, while the integration and approximation of a conversion matrix do not affect the video quality, this could greatly reduce the number of operands.

This paper proposes a H.264 rate control method considering picture characteristics and buffer constraints. Selecting the optimum buffer size corresponding to each application's requirement gives better compression quality and functionality. However, conventional methods allocate the number of bits equally to each GOP and do not explicitly consider the buffer size, where picture quality might not be optimal and the buffer size is not optimally determined for applications. This paper provides a bit allocation method that reflects the picture characteristic of each frame while keeping the buffer transition compliant within a given size. In the proposed method, the number of bits to be generated is estimated from the complexity of each frame, and it is adjusted by a scaling coefficient newly introduced in this paper. Thus, the allocated number of bits reflects picture characteristics and fully complies with buffer constraints. Experimental results show that the proposed method improves picture quality and suppresses quality fluctuation.

H.264/MPEG-4 AVC has become widely used in various applications and services because of its high coding efficiency. With the spread of H.264/MPEG-4 AVC contents, it has been strongly required to edit the H.264/MPEG-4 AVC coded contents with less computational complexity since the compression of H.264/MPEG-4 AVC requires far more processing power than other existing formats such as MPEG-4, MPEG-2. In this paper, we propose a fast frame-accurate editing method for H.264/MPEG-4 AVC contents. In the proposed method, we employed "adaptive bit allocation and GOP-length modification" and "Re-quantization scheme with drift noise management" The proposed editing method can realize frame-accurate cut and splice editing regardless of picture coding types I, P, and B. In the experiment, we could also confirm the effectiveness of the proposed method in terms of processing power and the picture quality of the edited contents when compared with conventional editing

Multimedia contents transmission and management system "VAST-web" is proposed. It employs store-and-forward transmission technique to maintain the contents quality. It also has a central management function of contents stored at remote locations so that contents query and contents pull model as well as contents push model are supported. A novel transmission protocol "SVFTP" is also introduced, which is a core technique in VAST-web. Its throughput performance is evaluated theoretically and experimentally to show it is well-suited to large volume multimedia contents transmission. Besides, its pipeline transmission capability provides VAST-web system with great flexibility for media playback.

Proposes some reversible variable length codes (RVLCs) which can be decoded instantaneously both in the forward and backward directions and have high transmission efficiency. These codes can be used, for example, in the backward reconstruction of video signals from the data last received when some signal is lost midway in the transmission. Schemes for a symmetrical RVLC requiring only a single code table and for an asymmetrical RVLC having short average code length are introduced. They compare favorably with other reversible codes such as B2 codes in several aspects.>

An efficient technique for obtaining a variable length code (VLC) resistant to synchronization loss due to channel errors is proposed. Bit errors in the channel lead to incorrect decoding of a VLC and result in error states for continuous codewords. In this paper first a precise value is found for assessing the ability of a VLC to recover synchronization from the error states. Next a new transformation for converting a code into another of the same efficiency (equivalent transformation) is introduced, and several VLCs with an identical efficiency are compared through estimation of synchronization recovery in order to find an error-resistant VLC. Based on the equivalent transformation, an efficient method for finding an error-resistant VLC with high recovery performance from among multiple VLCs is described.>

Integration of video services are addressed. The view that various types of end-to-end quality of service (QOS) for video communications, ranging from videophone to high-definition television (HDTV), should be supported to utilize broadband integrated services digital network (B-ISDN) services and video data is expressed. End-to-end QOS is recognized as a function of terminal capability and network performance. The key feature of B-ISDN is that both terminal capability and network performance are selectable and controllable. In order to capture the basic characteristics of end-to-end QOS, it is necessary to conduct experiments using a prototype system. A prototype system comprising an asynchronous transfer mode (ATM) switch and video codecs is presented. The ATM switch is based on distributed-memory-switch architecture and it will provide for nonblocking cell-switching capability with a very low cell-transfer delay. Developed codecs conform to H.261 with an additional function of controlling average coding bit rate. The coding bit rate is chosen as a parameter of terminal capability and the cell-loss ratio is chosen as a parameter of network performance. From experiments using a prototype system, a relationship between coding bit rate and cell-loss ratio is obtained.>