Student engagement is a central construct in Learning Analytics, yet it is often operationalized through persistence indicators derived from logs, overlooking affective–cognitive states. Focusing on the analysis of reading logs, this study examines how trait-level flow—operationalized as the tendency to experience Deep Effortless Concentration (DEC)—and traces of reading strategies derived from e-book interaction data can extend traditional engagement indicators in explaining learning outcomes. We collected data from 100 students across two engineering courses, combining questionnaire measures of DEC with fine-grained reading logs. Correlation and regression analyses show that (1) DEC and traces of reading strategies explain substantial additional variance in grades beyond log-traced engagement (ΔR2 = 21.3% over the baseline 25.5%), and (2) DEC moderates the relationship between reading behaviors and outcomes, indicating trait-sensitive differences in how log-derived indicators translate into performance. These findings suggest that, to support more equitable and personalized interventions, the analysis of reading logs should move beyond a one-size-fits-all interpretation and integrate personal traits with metrics that include behavioral and strategic measures of reading.

Background and context : Open datasets play a crucial role in three prominent research domains that intersect data science and education: learning analytics, educational data mining, and AI in education. Researchers in these domains apply computational methods to analyze data from educational contexts, aiming to better understand and improve teaching and learning. Research scope and gap : Providing open datasets alongside research papers supports research reproducibility, fosters collaboration, and increases trust in research findings. It also provides individual benefits for authors, such as greater visibility, credibility, and citation potential. However, despite these advantages, the availability of open datasets and the associated practices within the learning analytics research communities, especially at their flagship conference venues, remain unclear. Goal and method : To address this gap, we conducted a systematic survey of publicly available datasets published alongside research papers in learning analytics domains. We manually examined 1,125 papers from three respected flagship conferences (LAK, EDM, and AIED) over the past five years (2020–2024). We discovered, categorized, and analyzed 172 unique datasets used in 204 publications. Results and contributions : Our study presents the most comprehensive collection and analysis of open educational datasets to date, along with the most detailed categorization. Of the 172 datasets identified, 143 were not captured in any prior survey of open data in learning analytics. We provide insights into the datasets’ context, analytical methods, use, and other properties. Based on this survey, we summarize the current gaps in the field. Furthermore, we list practical recommendations, advice, and 8-item guidelines under the acronym PRACTICE with a checklist to help researchers publish their data. Lastly, we share our original dataset: an annotated inventory detailing the discovered datasets and the corresponding publications. We hope these findings will support further adoption of open data practices in learning analytics communities and beyond.

This paper proposes Attention-Seeker, an unsupervised keyphrase extraction method that leverages self-attention maps from a Large Language Model to estimate the importance of candidate phrases. Our approach identifies specific components – such as layers, heads, and attention vectors – where the model pays significant attention to the key topics of the text. The attention weights provided by these components are then used to score the candidate phrases. Unlike previous models that require manual tuning of parameters (e.g., selection of heads, prompts, hyperparameters), Attention-Seeker dynamically adapts to the input text without any manual adjustments, enhancing its practical applicability. We evaluate Attention-Seeker on four publicly available datasets: Inspec, SemEval2010, SemEval2017, and Krapivin. Our results demonstrate that, even without parameter tuning, Attention-Seeker outperforms most baseline models, achieving state-of-the-art performance on three out of four datasets, particularly excelling in extracting keyphrases from long documents.

Traditional knowledge graph (KG) approach often rely on static textbook content and overlook the dynamic, collaborative interactions in collaborative problem-solving (CPS). This study introduced a three-step integrated KG approach designed to support CPS in STEM education and examined the effective KG features that influence CPS learning outcomes. KGs were generated by combining learning materials and student dialogue data. Two types of features, graph structural features and knowledge activation features, were identified to classify knowledge nodes and analyze how students activated knowledge during CPS. Clustering analysis revealed three types of knowledge nodes: Peripheral Nodes, Core Nodes, and Degree Hubs. Furthermore, key features such as depth, branch, and activated paths showed positive correlations with group discussion performance and CPS skills but had limited influence on test scores. These findings highlight the potential of integrated KGs to support both individual and group learning in STEM education.

In supervised machine learning (SML) research, large training datasets are essential for valid results. However, obtaining primary data in learning analytics (LA) is challenging. Data augmentation can address this by expanding and diversifying data, though its use in LA remains underexplored. This paper systematically compares data augmentation techniques and their impact on prediction performance in a typical LA task: prediction of academic outcomes. Augmentation is demonstrated on four SML models, which we successfully replicated from a previous LAK study based on AUC values. Among 21 augmentation techniques, SMOTE-ENN sampling performed the best, improving the average AUC by 0.01 and approximately halving the training time compared to the baseline models. In addition, we compared 99 combinations of chaining 21 techniques, and found minor, although statistically significant, improvements across models when adding noise to SMOTE-ENN (+0.014). Notably, some augmentation techniques significantly lowered predictive performance or increased performance fluctuation related to random chance. This paper's contribution is twofold. Primarily, our empirical findings show that sampling techniques provide the most statistically reliable performance improvements for LA applications of SML, and are computationally more efficient than deep generation methods with complex hyperparameter settings. Second, the LA community may benefit from validating a recent study through independent replication.

Time series modeling serves as the cornerstone of real-world applications, such as weather forecasting and transportation management. Recently, Mamba has become a promising model that combines near-linear computational complexity with high prediction accuracy in time series modeling, while facing challenges such as insufficient modeling of nonlinear dependencies in attention and restricted receptive fields caused by convolutions. To overcome these limitations, this paper introduces an innovative framework, Attention Mamba, featuring a novel Adaptive Pooling block that accelerates attention computation and incorporates global information, effectively overcoming the constraints of limited receptive fields. Furthermore, Attention Mamba integrates a bidirectional Mamba block, efficiently capturing long-short features and transforming inputs into the Value representations for attention mechanisms. Extensive experiments conducted on diverse datasets underscore the effectiveness of Attention Mamba in extracting nonlinear dependencies and enhancing receptive fields, establishing superior performance among leading counterparts. Our codes will be available on GitHub.

The use of digital textbooks in education provides valuable data on student reading behavior that can help educators refine their course materials and instructional design for future iterations. Previous studies have explored methods for extracting important evidence from this data, but they require manual intervention. By automating these methods, this paper introduces an end-to-end system capable of extracting evidence from e-book data and providing recommendations for slides' content review based on this evidence. Our system incorporates information about reading preferences into the evidence-extraction process and implements Large Language Models (LLMs) for automatic interpretation. Six teachers evaluated our proposed system indicating a promising level of effectiveness, while also highlighting areas for future improvement to ensure a successful classroom implementation. These include considerations for improving the actionability of recommendations, improving the identification of content that needs refinement, and improving the performance of LLMs.

A novel coronavirus discovered in late 2019 (COVID-19) quickly spread into a global epidemic and, thankfully, was brought under control by 2022. Because of the virus's unknown mutations and the vaccine's waning potency, forecasting is still essential for resurgence prevention and medical resource management. Computational efficiency and long-term accuracy are two bottlenecks for national-level forecasting. This study develops a novel multivariate time series forecasting model, the densely connected highly flexible dendritic neuron model (DFDNM) to predict daily and weekly positive COVID-19 cases. DFDNM's high flexibility mechanism improves its capacity to deal with nonlinear challenges. The dense introduction of shortcut connections alleviates the vanishing and exploding gradient problems, encourages feature reuse, and improves feature extraction. To deal with the rapidly growing parameters, an improved variation of the adaptive moment estimation (AdamW) algorithm is employed as the learning algorithm for the DFDNM because of its strong optimization ability. The experimental results and statistical analysis conducted across three Japanese prefectures confirm the efficacy and feasibility of the DFDNM while outperforming various state-of-the-art machine learning models. To the best of our knowledge, the proposed DFDNM is the first to restructure the dendritic neuron model's neural architecture, demonstrating promising use in multivariate time series prediction. Because of its optimal performance, the DFDNM may serve as an important reference for national and regional government decision-makers aiming to optimize pandemic prevention and medical resource management. We also verify that DFDMN is efficiently applicable not only to COVID-19 transmission prediction, but also to more general multivariate prediction tasks. It leads us to believe that it might be applied as a promising prediction model in other fields.

The main objective of this paper is to improve the image segmentation model for handwritten notebook analytics. We conducted a considerable amount of research in this area to increase the accuracy and efficiency of segmentation. To address the issues with traditional methods, we introduced attention mechanism and recursive residual convolutional neural network in the multi-task U-Net model. Through training and testing the model on handwritten notebook dataset and compared it with other existing technologies, we demonstrated the effectiveness of this method. The results showed that the model had a significant improvement in accuracy. Therefore, the research findings in this paper are important for improving the technology of handwritten notebook analytics.

Background subtraction networks outperform traditional hand-craft background subtraction methods. The main advantage of background subtraction networks is their ability to automatically learn background features for training scenes. When applying the trained network to new target scenes, adapting the network to the new scenes is crucial. However, few studies have focused on reusing multiple trained models for new target scenes. Considering background changes have several categories, such as illumination changes, a model trained for each background scene can work effectively for the target scene similar to the training scene. In this study, we propose a method to ensemble the module networks trained for each background scene. Experimental results show that the proposed method is significantly more accurate compared with the conventional methods in the target scene by tuning with only a few frames.

Handwritten notes are one important component of students’ learning process, which is used to record what they have learned in class or tease out knowledge after class for reflection and further strengthen the learning effect. It also helps a lot during review. We hope to divide handwritten notes (Japanese) into different parts, such as text, mathematical expressions, charts, etc., and quantify them to evaluate the condition of the notes and compare them among students. At the same time, data on students’ learning behaviors in the course are collected through the online education platform, such as the use time of textbook and attendance, as well as the scores of the online quiz and course grade. In this paper, the analysis of the relationship between the segmentation results of handwritten notes and learning behavior are reported, as well as the research on automatic page segmentation based on deep learning.

Summer vacation is considered a cause of loss in students' learning performance. In this study, we investigated the differences in learning behaviors in reading learning materials and time-related behavior patterns regarding summer assignments among three classes under different instructional styles. The results showed that students' learning behaviors in summer were correlated with instructional styles in school.

Previous research has established that motivation has a positive impact on the learning processes and behaviors [1] [2]. Learning analytics (LA) can play an important role in addressing the issue of collecting learning behaviors. In this study, we observed the teaching activities of three classes and examined the relationships among learning motivation, learning performance, and learning behaviors of students in digital learning material readers.

Recently, deep neural networks have demonstrated excellent performance in foreground segmentation tasks such as moving object detection and change detection tasks. Various types of neural networks have been proposed, however, the previous works mainly discuss the accuracy. Analytics of the neural networks is important to utilize them effectively and improve their performance. In this paper, we investigate a foreground segmentation network and background subtraction network. In our analysis, we discuss differences of behaviors of the two networks in specific scenes and feature distributions in each layer of a background subtraction network to investigate feature learning. In addition, we provide suggestions about the comparison with these networks.

Calibration of relative poses between cameras is a challenging problem, known as extrinsic calibration, for nonoverlapping cameras that do not share the field of view. We propose a method for calibrating non-overlapping in-vehicle cameras placed at front, back, left and right positions by using visual SLAM(vSLAM). Our proposal is to calibrate the cameras during the motion of 90-degree backing-up parking on the fly, without using any dedicated calibration equipment. With this motion, the adjacent cameras are able to have the close field of view at different moments. The relative poses can be computed if the maps computed with vSLAM on each camera are merged by using the common structures. Therefore, we propose an efficient calibration framework with this feature. The proposed method is divided into three steps: map reconstruction with vSLAM on each camera, map merging for all the cameras, and extrinsic calibration. Especially, we propose to separately utilize the frames for vSLAM and the ones for the calibration so that the accuracy of vSLAM can be maximized for the calibration. In the evaluation, the calibration was performed in a practical environment to investigate the performance in comparison with the ground truth acquired by using a calibration equipment.

Recently, background subtraction based on deep convolutional neural networks has demonstrated excellent performance in change detection tasks. However, most of the reported approaches require pixel-level label images for training the networks. To reduce the cost of rendering pixel-level annotation data, weakly supervised learning approaches using frame-level labels have been proposed. These labels indicate if a target class is present. Frame-level supervised learning is challenging because we cannot use location information for training the networks. Therefore, some constraints are introduced for guiding foreground locations. Previous works exploit prior information on foreground sizes and shapes. In this work, we propose two constraints for weakly supervised background subtraction networks. Our constraints use binary mask images generated by simple background subtraction. Unlike previous works, our approach does not require prior information on foreground sizes and shapes. Moreover, our constraints are more suitable for change detection tasks. We also present an experiment verifying that our constraints can improve foreground detection accuracy compared to other methods, which do not include them.

Learning log is defined as a digital record of what learners have learned in their daily lives using ubiquitous technologies. By using the ubiquitous learning system named SCROLL(System for Capturing and Remining Of Learning Logs), learners can save what they have learned in their daily lives with photo, such as location (latitude and longitude), learning place, and date and time of creation as a learning log. Although learners have many opportunities to learn words and meanings of objects with taking a photo in their daily lives, SCROLL is not implemented functions for supporting language learning with object and text detection. Therefore, this paper proposes a ubiquitous learning system to support language learning with object and text detection technologies.

In this paper, we present an ontology-based visualization support system for e-book learners, which provides not only a meaningful receptive learning environment but also a meaningful discovery learning environment. Those two environments are developed to help e-book learners to effectively construct their knowledge frameworks. A series of experiments were conducted on four undergraduate classes instructed by two professors (A and B): two classes(one guided by A and the other guided by B) were assigned as control groups and studied with one e-book chapter in receptive learning environment while another two classes (one guided by A and the other guided by B) were assigned as experimental groups and studied with the same e-book chapter in discovery learning environment. For analyzing the learner behavior, K-means clustering algorithm is performed not only by considering the number of total command actions and the cumulative duration of stay on target pages as learner features, but also by considering the duration of stay on each target page (in total 15 pages) as learner features. Learners’ behavior differences in e-book system are examined and discussed.

Extended learning support systems for all-age education requires inclusive user interface design, especially for elderly users. A dual-tablet user interface with simplified visual layers and more intuitive operations was proposed aiming to reduce the physical and mental loads of elderly learners. An initial prototype with basic functions of viewing learning material was developed based on a cross-platform framework. Two preliminary user experiments participated by elderly volunteers were carried out for formative evaluations, in order to improve the usability of the interface design iteratively. The prototype was modified based on the participants' comments and observation of their operations during the experiments. Additional findings of the elderly users' preference and tendency were discussed for further development.

To improve designs of e-learning materials, it is necessary to know which word or figure a learner felt "difficult" in the materials. In this pilot study, we measured electroencephalography (EEG) and eye gaze data of learners and analyzed to estimate which area they had difficulty to learn. The developed system realized simultaneous measurements of physiological data and subjective evaluations during learning. Using this system, we observed specific EEG activity in difficult pages. Integrating of eye gaze and EEG measurements raised a possibility to determine where a learner felt "difficult" in a page of learning materials. From these results, we could suggest that the multimodal measurements of EEG and eye gaze would lead to effective improvement of learning materials. For future study, more data collection using various materials and learners with different backgrounds is necessary. This study could lead to establishing a method to improve e-learning materials based on learners' mental states.

Online learning environments presently accumulate large amounts of log data. Analysis of learning behaviors from these log data is expected to benefit instructors and learners. This study was intended to identify effective measures from e-book materials used at Kyushu University and to employ these measures for analyzing learning behavioral patterns. In an evaluation, students were grouped into four clusters using k-means clustering, and their learning behavioral patterns were analyzed. We examined whether the learning behavioral patterns exhibited relations with the learning outcomes. The results reveal that the learning behavior of “backtrack” style reading exerts a significant positive influence on learning effectiveness, which can aid students to learn more efficiently.

We propose a new change detection method using clickstream data collected through an e-Book system. Most of the prior work has focused on the batch processing of clickstream data. In contrast, the proposed method is designed for online processing, with the model parameters for change detection updated sequentially based on observations of new click events. More specifically, our method generates a model for an individual student and performs minute-by-minute change detection based on click events during a classroom lecture. We collected clickstream data from four face-to-face lectures, and conducted experiments to demonstrate how the proposed method discovered change points and how such change points correlated with the students' performances.

Sensor pose estimation is an essential technology for various applications. For instance, it can be used not only to display immersive contents according user movements in Virtual Reality (VR) and but also to superimpose computer-generated objects onto images from a camera in Augmented Reality (AR). As a technical term definition, camera localization with respect to a pre-created map database is specifically referred to as image based localization, memory based localization, or camera relocalization.

Recently, deep neural networks (DNNs) have demonstrated excellent performance for change detection. The DNN-based background subtraction automatically discovers background features from datasets and outperforms traditional background modeling based on handcraft features and/or subtraction strategies. Most researchers mainly discuss the accuracy of foreground detection and do not analyze how and why the DNN works well for change detection tasks. It is necessary to understand what the DNN learns as background features in order to discuss the potential of the DNN in background subtraction. In this paper, we focus on the filters in the first convolution layer and the activations of neurons in the last fully connected layer to understand the behavior of the DNN. From the experiment, we found that 1) the first layer performs the role of background subtraction using several filters, and 2) the last layer categorizes some background changes into a group without supervised signals. These findings suggest the possibility of a new background modeling strategy based on data-driven extracted features.

In this paper, we present a system framework making use of e-book logs for visualization learning support systems intended to provide meaningful learning environment for e-book learners. An ontology-based visualization support system, which supports not only meaningful reception learning but also meaningful discovery learning, is designed and developed to help e-book learners to effectively construct their knowledge frameworks. Three main functions are provided in this personalized visualization support system: (1) for any knowledge of E-books, the learner can get its e-book location information and a relation map including its relevance knowledge and their relations; (2) for any page range of any e-book, the learner can get a relation map including knowledge shown in those pages, their relevant KPs and their upper concepts; (3) for any learning period, the learner can check the knowledge map including the knowledge they had read, the relevance knowledge and their relations. Compared to the passive reception environment, in meaningful discovery learning environment learners are encouraged to actively locate new knowledge in their own knowledge framework and restructure existing knowledge by detecting hidden relations between relevant KPs through reflecting the attributes of acquired knowledge visually. Meanwhile the iterative procedure of confirmation and modification in their own relation map ensures that they check the logical consistency of their ideas and clear up misunderstandings.

A real-time learning analytics system is proposed for in-classroom use. We used an e-learning system and an e-book system to collect real-time learning activities during lectures. The collected logs were analyzed and presented visually on a web-based system for the teacher. The teacher can monitor how many students are viewing the same page as the teacher, whether they are following the explanation, or if they are reading previous or subsequent pages. Through a case study, we confirmed the effectiveness of the real-time learning analytics system, in terms of high synchronization between the teacher and the students, i.e., that the majority of students followed the teacher's explanation and added more bookmarks, highlights, or notes on the e-book, compared with the control group where the teacher did not use our system.

To discover teaching knowledge efficiently, we must extract the various teaching activities from educational data. In this paper, through the use of e-book logs and techniques of time-series analysis, we describe a method of practicing teaching analytics in face-to-face classes, one which enable us to extract the teaching activity efficiently and accurately.

Students' reflective writings are useful not only for students themselves but also teachers. It is important for teachers to know which concepts were understood well by students and which concepts were not, to continuously improve their classes. However, it is difficult for teachers to thoroughly read the journals of more than one hundred students. In this paper, we propose a novel method to extract common topics and students' common impressions against them from students' journals. Weekly keywords are discovered from journals by scoring noun words with a measure based on TF-IDF term weighting scheme, and then we analyze co-occurrence relationships between extracted keywords and adjectives. We employs nonnegative matrix factorization, one of the topic modeling techniques, to discover the hidden impression topics from the co-occurrence relationships. As a case study, we applied our method on students' journals of the course "Information Science" held in our university. Our experimental results show that conceptual keywords are successfully extracted, and four significant impression topics are identified. We conclude that our analysis method can be used to collectively understand the impressions of students from journal texts.

Many universities choose the C programming language (C) as the first one they teach their students, early on in their program. However, students often consider programming courses difficult, and these courses often have among the highest dropout rates of computer science courses offered. It is therefore critical to provide more effective instruction to help students understand the syntax of C and prevent them losing interest in programming. In addition, homework and paper-based exams are still the main assessment methods in the majority of classrooms. It is difficult for teachers to grasp students' learning situation due to the large amount of evaluation work. To facilitate teaching and learning of C, in this article we propose a system---LAPLE (Learning Analytics in Programming Language Education)---that provides a learning dashboard to capture the behavior of students in the classroom and identify the different difficulties faced by different students looking at different knowledge. With LAPLE, teachers may better grasp students' learning situation in real time and better improve educational materials using analysis results. For their part, novice undergraduate programmers may use LAPLE to locate syntax errors in C and get recommendations from educational materials on how to fix them.

In this paper, we examined whether previous findings on educational big data consisting of e-book logs from a given academic course can be reproduced with different data from other academic courses. The previous findings showed that (1) students who attained consistently good achievement more frequently browsed different e-books and their pages than low achievers and that (2) this difference was found only for logs of preparation for course sessions (preview), not for reviewing material (review). Preliminarily, we analyzed e-book logs from four courses. The results were reproduced in only one course and only partially, that is, (1) high achievers more frequently changed e-books than low achievers (2) for preview. This finding suggests that to allow effective usage of learning and teaching analyses, we need to carefully construct an educational environment to ensure reproducibility.

In this paper, we propose a method for predicting final grades of students by a Recurrent Neural Network (RNN) from the log data stored in the educational systems. We applied this method to the log data from 108 students and examined the accuracy of prediction. From the experimental results, comparing with multiple regression analysis, it is confirmed that an RNN is effective to early prediction of final grades.

This paper describes visualization and analysis methods using educational big data collected by research project at Kyushu University in Japan. The project uses an e-book system called BookLooper, Moodle, and Mahara. Logs for this analytics were collected from 99 first-year students in an information science course at Kyushu University. The number of logs are collected approximately 330,000, and this paper visualize and analyze the collected logs. The purpose of this study is to predict students' final grade and to profile visualization and analysis results. The prediction of this study shows that it leads to discoveries of students who fail to make the grade.

This research aims to investigate the relationship between self-regulated learning awareness, learning behaviors, and learning performance in ubiquitous learning environments. In order to do so, psychometric data about self-regulated learning and log data such as marker, annotation, accessing device types that stored the learning management system were collected and analyzed using multiple regression analysis with stepwise method. The results indicated that self-efficacy, internal value, and the number of read slides had a significant influence on the final score, and the awareness of cognitive learning strategy use has slightly significant power to predict the final score.

Background modeling and subtraction are essential to video surveillance applications. There are two main issues related to background modeling: how to initialize the background model, and how to update the model based on observations. In this paper, we consider the first issue with the aim of generating a clear background image that does not contain foreground objects or noise. We used a bidirectional analysis and consensus voting strategy to achieve this goal. We demonstrated the effectiveness of our technique using open access datasets.

As smartwatches are becoming more widely used in society, gesture recognition, as an important aspect of interaction with smartwatches, is attracting attention. An accelerometer that is incorporated in a device is often used to recognize gestures. However, a gesture is often detected falsely when a similar pattern of action occurs in daily life. In this paper, we present a novel method of designing a new gesture that reduces false detection. We refer to such a gesture as a low-false-positive (LFP) gesture. The proposed method enables a gesture design system to suggest LFP motion gestures automatically. The user of the system can design LFP gestures more easily and quickly than what has been possible in previous work. Our method combines primitive gestures to create an LFP gesture. The combination of primitive gestures is recognized quickly and accurately by a random forest algorithm using our method. We experimentally demonstrate the good recognition performance of our method for a designed gesture with a high recognition rate and without false detection.

In this paper, we present a visualization tool for person re-identification when tracking objects across non-overlapping cameras. Tracking objects across non-overlapping cameras is challenging because the observations from different cameras are widely separated in both time and space. Hence, these systems need a large amount of labeled training data. Commonly, this training data is constructed manually at significant human cost. We support this process efficiently by visualizing the correspondences of objects across multiple cameras. Our tool facilitates the construction of a database for person re-identification with ease. Moreover, the accuracy of person re-identification can be increased using the generated database because the amount of training data is increased. In the experiments, we apply the proposed tool to real world situations to verify the validity of the proposed system.

Existing background model based change detection methods have difficulty in distinguishing between foreground and background changes when both changes are caused by the same factors. We explore the possibility of using a light field camera to resolve the problem of existing single-view camera-based approaches. We present a new change detection strategy that processes light rays captured by the light field camera. The light rays are used for three purposes: 1) generating an active surveillance field (ASF) to determine in-focus and out-focus areas, 2) evaluating focusness to determine whether the light rays come from the ASF, and 3) creating and updating light-ray background models to capture temporal changes in light rays. To investigate the effectiveness of the proposed approach, we evaluated several video sequences captured by a light field camera. Experimental results show that our change detection scheme can robustly handle challenging situations that cannot be resolved by existing single-view approaches.

This paper discusses learning behavior analysis using a learning management system (LMS) and an e-textbook system. We collected a large number of operation logs from e-textbooks to analyze the process of learning. In addition, we conducted a quiz to check the level of understanding. In our study, we especially focus on an analysis of the relationship between learning behavior in informal learning and its effectiveness in the corresponding quiz. We apply a machine learning and classification methodology for behavior analysis. Our experimental results demonstrate that students who undertake good informal learning achieve better scores in quizzes.

This preliminary research investigates the relationship between psychometric data and learning behaviors in the learning analytics research field, specifically, the relationship between self-regulated learning and learning behavior. The results of this limited research show that marker and annotation use have a weak significant relationship with self-efficacy and the intrinsic value of learning materials.

Analyses on students’ learning behaviors comprise an important thrust in education research. This study focused on e-books system used in the classroom and this system recorded students’ learning logs in their daily academic life. These learning logs can be used to analysis students’ learning behaviors. By performing partial correlation analysis, the study found that a number of learning behaviors have a significant relation with students’ test scores.

The segmentation of transparent objects can be very useful in computer vision applications. However, because they borrow texture from their background and have a similar appearance to their surroundings, transparent objects are not handled well by regular image segmentation methods. We propose a method that overcomes these problems using the consistency and distortion properties of a light-field image. Graph-cut optimization is applied for the pixel labeling problem. The light-field linearity is used to estimate the likelihood of a pixel belonging to the transparent object or Lambertian background, and the occlusion detector is used to find the occlusion boundary. We acquire a light field dataset for the transparent object, and use this dataset to evaluate our method. The results demonstrate that the proposed method successfully segments transparent objects from the background.

We propose a strategy of background subtraction for an image sequence captured by a moving camera. To adapt for camera motion, it is necessary to estimate the relation between consecutive frames in background subtraction. However, simple background subtraction using the relation between consecutive frames results in many false detections. We use re-projection error to handle this problem. The re-projection error has a low value in a background region. According to re-projection error, our method searches neighboring background models and tunes a threshold value for detection in order to reduce false detections. We evaluated the accuracy of detection of our method in experiments. Our method provided better detection than a method that does not search neighboring background models. Our method thus reduced the number of false detections.

In this paper, we investigate the problem of modeling images and associated text for cross-modal retrieval tasks such as text-to-image search and image-to-text search. To make the data from image and text modalities comparable, previous cross-modal retrieval methods directly learn two projection matrices to map the raw features of the two modalities into a common subspace, in which cross-modal data matching can be performed. However, the different feature representations and correlation structures of different modalities inhibit these methods from efficiently modeling the relationships across modalities through a common subspace. To handle the diversities of different modalities, we first leverage the coupled dictionary learning method to generate homogeneous sparse representations for different modalities by associating and jointly updating their dictionaries. We then use a coupled feature mapping scheme to project the derived sparse representations from different modalities into a common subspace in which cross-modal retrieval can be performed. Experiments on a variety of cross-modal retrieval tasks demonstrate that the proposed method outperforms the state-of-the-art approaches.

Nowadays massive amount of images and texts has been emerging on the Internet, arousing the demand of effective cross-modal retrieval such as text-to-image search and image-to-text search. To eliminate the heterogeneity between the modalities of images and texts, the existing subspace learning methods try to learn a common latent subspace under which cross-modal matching can be performed. However, these methods usually require fully paired samples (images with corresponding texts) and also ignore the class label information along with the paired samples. This may inhibit these methods from learning an effective subspace since the correlations between two modalities are implicitly incorporated. Indeed, the class label information can reduce the semantic gap between different modalities and explicitly guide the subspace learning procedure. In addition, the large quantities of unpaired samples (images or texts) may provide useful side information to enrich the representations from learned subspace. Thus, in this paper we propose a novel model for cross-modal retrieval problem. It consists of 1) a semi-supervised coupled dictionary learning step to generate homogeneously sparse representations for different modalities based on both paired and unpaired samples; 2) a coupled feature mapping step to project the sparse representations of different modalities into a common subspace defined by class label information to perform cross-modal matching. Experiments on a large scale web image dataset MIRFlickr-1M with both fully paired and unpaired settings show the effectiveness of the proposed model on the cross-modal retrieval task.

Nowadays, it is a hot topic to analyze the huge amount of data in the world. This issue also exists in the learning during students’ life. The learning data are collected only to record students’ learning status. As a result, most learning data are not used to improve the quality of learning for students. In this paper, we propose an order made education system, which can recommend students to select the courses they want to learn. In order to analyze students’ learning behaviors, we collect students’ learning data by using mobile devices.

User-provided image tags are usually incomplete or noisy to describe the visual content of corresponding images. In this paper, we consider defective tagging which covers both incomplete and noisy situations, and address the problem of tag completion where tag assignments of training images are defective. While previous studies on tag completion usually assign equal penalty to empirical loss when processing each missing or noisy tag for each image, we show that this may be suboptimal as the relatedness of each tag to each image varies due to the defective setting. Thus, we introduce an image-tag re-weighting scheme to re-weight the penalty term of each tag to each image considering both image similarities and tag associations, and formulate a unified re-weighted empirical loss function. Experimental evaluations show that embedding proposed re-weighted empirical loss function in state-of-the-art tag completion algorithms achieves significant improvement in dealing with defective tag assignments.

Current object-recognition algorithms use local features, such as scale-invariant feature transform (SIFT) and speeded-up robust features (SURF), for visually learning to recognize objects. These approaches though cannot apply to transparent objects made of glass or plastic, as such objects take on the visual features of background objects, and the appearance of such objects dramatically varies with changes in scene background. Indeed, in transmitting light, transparent objects have the unique characteristic of distorting the background by refraction. In this paper, we use a single-shot light field image as an input and model the distortion of the light field caused by the refractive property of a transparent object. We propose a new feature, called the light field distortion (LFD) feature, for identifying a transparent object. The proposal incorporates this LFD feature into the bag-of-features approach for recognizing transparent objects. We evaluated its performance in laboratory and real settings.

Background modeling and subtraction is an essential task in video surveillance applications. Most traditional studies use information observed in past frames to create and update a background model. To adapt to background changes, the background model has been enhanced by introducing various forms of information including spatial consistency and temporal tendency. In this paper, we propose a new framework that leverages information from a future period. Our proposed approach realizes a low-cost and highly accurate background model. The proposed framework is called bidirectional background modeling, and performs background subtraction based on bidirectional analysis, i.e., analysis from past to present and analysis from future to present. Although a result will be output with some delay because information is taken from a future period, our proposed approach improves the accuracy by about 30% if only a 33-millisecond of delay is acceptable. Furthermore, the memory cost can be reduced by about 65% relative to typical background modeling.

For the task of image annotation, traditional probabilistic topic models based on Latent Dirichlet Allocation (LDA) [1], assume that an image is a mixture of latent topics. An inevitable limitation of LDA is the inability to model topic correlation since topic proportions of an image are generated independently. Motivated by Correlated Topic Model (CTM) [2] which derives from natural language processing to model topic correlation of a document, we extend the popular LDA based models (corrLDA [3], sLDA-bin [4], trmmLDA [5]) to CTM based models (corrCTM, sCTM-bin, trmmCTM). We present a comprehensive comparison between CTM based and LDA based models on three benchmark datasets, illustrating the superior annotation performance of proposed CTM based models, by means of propagating topic correlation among image features and annotation words.

Gabor-based face representation has achieved enormous success in face recognition. However, one drawback of Gabor-based face representation is the huge amount of data that must be stored. Due to the nonlinear structure of the data obtained from Gabor response, classical linear projection methods like principal component analysis fail to learn the distribution of the data. A nonlinear projection method based on a set of self-organizing maps is employed to capture this nonlinearity and to represent face in a new reduced feature space. The Multiple Self-Organized Gabor Features (MSOGF) algorithm is used to represent the input image using all winner indices from each SOM map. A new local matching algorithm based on the similarity between local features is also proposed to classify unlabeled data. Experimental results on FERET database prove that the proposed method is robust to expression variations.

Background modeling has been widely researched to detect moving objects from image sequences. It is necessary to adapt the background model various changes of illumination condition. A hybrid type of background model which consists of more than one background model has been used for object detection since it is very robust for illumination changes. In this paper, we also propose a new hybrid type of background model named "hybrid spatial-temporal background model". Our model consists of two different kinds of background models. One is pixel-level background model which is robust for long-term illumination changes. The other is spatial-temporal background model which is robust for short-term illumination changes. Our experimental results demonstrate superiority of our method to some related works.

We propose an approach to recognize time-series gesture partems wilh Hierarchical Self-Organizing Map(HSOM). Dlle of the key issue of the time-series ponem recognition is to absorb the time Variant appropriately and to make clusters which indude the same gesture class. In our approach. we arrange the SOM hierarchically. In each layer of the SOM the time-series pattenrs divided into some periods; postures, gesture elements and gestures. They are learned in each layer of HSOM. For example, postures and learned in the first layer, gesture elements are learned in the second layer and so on. Using the sparse code in the bottom layer, the SOM can perform time invariant recognition of the gesture elemems and gestures.

Hypercolumn model (HCM) is a neural network model previously proposed to solve image recognition problem. In this paper, we propose an improved version of HCM network and demonstrate its ability to solve face recognition problem. HCM network is a hierarchical model based on self-organizing map (SOM) that closely follows the organization of visual cortex and builds an increasingly complex and invariant feature representation. This invariance achieved by alternating between feature extraction and feature integration operation. To improve the recognition rate of HCM, we propose a variable dimension for each map in the feature extraction layer. The number of neurons in each map-side is decided automatically from training data. We demonstrate the performance of the approach using ORL face database.

We propose an approach to improve the accuracy of estimating feature points of human body on a vision-based motion capture system (MCS) by using the Variable-density Self-Organizing Map (VDSOM). The VDSOM is a kind of Self-Organizing Map (SOM) and has an ability to learn training samples incrementally. We let VDSOM learn 3-D feature points of human body when the MCS succeeded in estimating them correctly. On the other hand, one or more 3-D feature point could not be estimated correctly, we use the VDSOM for the other purpose. The SOM including VDSOM has an ability to recall a part of weight vector which have learned in the learning process. We use this ability to recall correct patterns and complement such incorrect feature points by replacing such incorrect feature points with them.

Non-parametric representation of pixel intensity distribution is quite effective to construct proper background model and to detect foreground objects accurately. However, from the viewpoint of practical application, the computation cost of the distribution estimation should be reduced. In this paper, we present fast estimation of the probability density function (PDF) of pixel value using Parzen density estimation and foreground object detection based on the estimated PDF. Here, the PDF is computed by partially updating the PDF estimated at the previous frame, and it greatly reduces the computation cost of the PDF estimation. Thus, the background model adapts quickly to changes in the scene and, therefore, foreground objects can be robustly detected. Several experiments show the effectiveness of our approach.

We propose a method for create a background model in non-stationary scenes. Each pixel has a dynamic Gaussian mixture model. Our approach can automatically change the number of Gaussians in each pixel. The number of Gaussians increases when pixel values often change because of Illumination change, object moving and so on. On the other hand, when pixel values are constant in a while, some Gaussians are eliminated or integrated. This process helps reduce computational time. We conducted experiments to investigate the effectiveness of our approach.