In on-demand e-learning environments, the lack of direct intervention can lead to a decline in learners' engagement. To address this issue, systems that estimate the learners' attitudes and provide feedback have been proposed. However, the acceptability of such systems has not been sufficiently researched. In this study, we investigated the acceptability by people to an e-learning system with internal state feedback, for future personalized learning support. To this end, we developed a system that estimates and visualizes the learner's internal state in real-time. The system was exhibited in a public space for free use, and users' impressions were analyzed. To estimate the learners' internal state, we developed a machine-learning model that recognizes learners' alertness from facial videos. The system was deployed in an exhibition space, and 131 responses were collected. These responses were coded and analyzed using a co-occurrence network. The result indicated that learners tend to dislike the system due to feelings of being observed by supervisors. In contrast, instructors expressed favorable options toward the introduction of the system.

Long-term desk work with lousy posture causes physical disabilities. To solve this problem, we have developed a smart chair with a controllable seat angle in our previous research. In order to control the seat angle and guide the user to the correct posture using this smart chair, it is necessary to investigate how the posture changes in response to the seat angle change. In this study, we first constructed a dataset by capturing posture data from 30 subjects while changing the seat angle of the smart chair. This dataset trained a model based on a graph neural network to predict posture in response to seat angle changes. We compared the model's prediction performance in the evaluation experiment depending on the training data and the body part.

Dynamic facial expression recognition (DFER) is an important task in the field of computer vision. To apply automatic DFER in practice, it is necessary to accurately recognize ambiguous facial expressions, which often appear in data in the wild. In this paper, we propose MIDAS, a data augmentation method for DFER, which augments ambiguous facial expression data with soft labels consisting of probabilities for multiple emotion classes. In MIDAS, the training data are augmented by convexly combining pairs of video frames and their corresponding emotion class labels, which can also be regarded as an extension of mixup to soft-labeled video data. This simple extension is remarkably effective in DFER with ambiguous facial expression data. To evaluate MIDAS, we conducted experiments on the DFEW dataset. The results demonstrate that the model trained on the data augmented by MIDAS outperforms the existing state-of-the-art method trained on the original dataset.

We annotate 35,000 SNS posts with both the writer’s subjective sentiment polarity labels and the reader’s objective ones to construct a Japanese sentiment analysis dataset. Our dataset includes intensity labels (none, weak, medium, and strong) for each of the eight basic emotions by Plutchik (joy, sadness, anticipation, surprise, anger, fear, disgust, and trust) as well as sentiment polarity labels (strong positive, positive, neutral, negative, and strong negative). Previous studies on emotion analysis have studied the analysis of basic emotions and sentiment polarity independently. In other words, there are few corpora that are annotated with both basic emotions and sentiment polarity. Our dataset is the first large-scale corpus to annotate both of these emotion labels, and from both the writer’s and reader’s perspectives. In this paper, we analyze the relationship between basic emotion intensity and sentiment polarity on our dataset and report the results of benchmarking sentiment polarity classification.

In the e-learning context, how much the learner is concentrated and engaged, or the learners’ efficiency, is essential for providing adaptive and flexible materials, timely suggestions, etc., which can lead to efficient learning. In this work, we explore to predict learners’ efficiency with a realistic configuration, in which we use a webcam or a laptop PC’s built-in camera. Specifically, we first provide a feasible definition of the learners’ efficiency, and based on this definition, we predict one’s efficiency from facial behavior. We predict the learners’ efficiency using various convolutional neural networks. Results are discussed using different evaluation metrics.

Tomoyuki Kajiwara, Chenhui Chu, Noriko Takemura, Yuta Nakashima, Hajime Nagahara. Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2021.

Gait-based age estimation is one of key techniques for many applications (e.g., finding lost children/aged wanders). It is well known that the age estimation uncertainty is highly dependent on ages (i.e., it is generally small for children while is large for adults/the elderly), and it is important to know the uncertainty for the above-mentioned applications. We therefore propose a method of uncertainty-aware gait-based age estimation by introducing a label distribution learning framework. More specifically, we design a network which takes an appearance-based gait feature as an input and outputs discrete label distributions in the integer age domain. Experiments with the world-largest gait database OULP-Age show that the proposed method can successfully represent the uncertainty of age estimation and also outperforms or is comparable to the state-of-the-art methods.

In this paper, we propose a full pipeline of analysis of a large corpus about a century of public meeting in historical Australian news papers, from construction to visual exploration. The corpus construction method is based on image processing and OCR. We digitize and transcribe texts of the specific topic of public meeting. Experiments show that our proposed method achieves a F-score of 87.8% for corpus construction. As a result, we built a content search tool for temporal and semantic content analysis.

Deep convolutional neural networks (CNNs) have established their feet in the ground of computer vision and machine learning, used in various applications. In this work, an attempt is made to learn a CNN for a task of facial expression recognition (FER). Our network has convolutional layers linked with an FC layer with a skip-connection to the classification layer. Motivation behind this design is that lower layers of a CNN are responsible for lower level features, and facial expressions can be mainly encoded in low-to-mid level features. Hence, in order to leverage the responses from lower layers, all convo-lutional layers are integrated via FC layers. Moreover, a network with shared parameters is used to extract landmark motion trajectory features. These visual and landmark features are fused to improve the performance. Our method is evaluated on the CK+ and Oulu-CASIA facial expression datasets.

In this paper, we discuss input/output architectures for convolutional neural network (CNN)-based cross-view gait recognition. For this purpose, we consider two aspects: verification versus identification and the tradeoff between spatial displacements caused by subject difference and view difference. More specifically, we use the Siamese network with a pair of inputs and contrastive loss for verification and a triplet network with a triplet of inputs and triplet ranking loss for identification. The aforementioned CNN architectures are insensitive to spatial displacement, because the difference between a matching pair is calculated at the last layer after passing through the convolution and max pooling layers; hence, they are expected to work relatively well under large view differences. By contrast, because it is better to use the spatial displacement to its best advantage because of the subject difference under small view differences, we also use CNN architectures where the difference between a matching pair is calculated at the input level to make them more sensitive to spatial displacement. We conducted experiments for cross-view gait recognition and confirmed that the proposed architectures outperformed the state-of-the-art benchmarks in accordance with their suitable situations of verification/identification tasks and view differences.

This paper deals with a multiagent path-planning problem where several robots track humans to obtain detailed information on human behaviors and characteristics. For this, agents' paths are planned on the basis of the similarity between the predicted positions of humans and the agents' field of view. The long-horizon path planned on the basis of an accurate long-horizon prediction improves the tracking performance. However, it requires heavy computation and is less useful if the prediction is inaccurate. Since the accuracy of the prediction depends on the situation, the prediction term is determined by the similarity between the current and previous predictions. The results of computer simulation showed that our path-planning method works well for trajectories of humans in a dynamic environment by changing the horizon length of the path planning.

This paper deals with a multi-agent path planning problem for tracking humans in order to obtain detail information such like human behavior and characteristics. To achieve this, paths of agents are planned based on similarity between the predicted positions of humans and agents' field of views, and the path length in the path planning is determined according to the consistency between the current prediction and the previous prediction of the future human positions. We conducted computer simulations and results showed that our path planning method works well for trajectories of human in a real environment.

This paper describes a view planning of multiple cameras for tracking multiple persons for surveillance purposes. When only a few active cameras are used to cover a wide area, planning their views is an important issue in realizing a competent surveillance system. We develop a multi-start local search (MLS)-based planning method which iteratively selects fixation points of the cameras by which the expected number of tracked persons is maximized. Considering the fact that a person's motion can be estimated with its intermittent observations, we set a criterion which encourages frequent shifts of fixation points and develop a procedure for generating promising initial solutions for MLS. The method is shown to outperform the other approaches. We then modify the method such that it dynamically divides the cameras into mutually independent groups and determines fixation points within each group. The modified method is comparable to the original one with a much lower planning cost.