Deep learning has been successfully applied to many tasks such as image classification, feature learning, Text classification (sentiments analysis or opinion mining) etc. However, little research has focused on extracting polarity of sentiments expressed in text using a multimodal architecture. In other words, no researches take in consideration the multimodal nature of human behaviors before classifying sentiments. The representation of a person (also call User Modeling in some domains such as Intelligent Tutoring Systems) is an important feature to take in consideration if one wants to extract subjective information such as the polarity of sentiments expressed by the person. To design an effective representation of a user, it is important to consider all sources of data informing about its current state. We present a usersensitive deep multimodal architecture which takes advantage of deep learning and user data to extract a rich latent representation of a user. This rich latent representation mainly helps in text classification tasks. The architecture consists of the combination of a Long Short-Term Memory (LSTM), LSTM-Auto-Encoder, Convolutional Neural Networks and multiple Deep Neural Networks, in order to support the multimodality of data. The resulting model has been tested on a public multimodal dataset and is able to achieve best results compared to state-of-the-art algorithms for a similar task: detection of opinion polarity. The results suggest that the latent representation learnt from multimodal data helps in the discrimination of polarity of opinion.

Prediction of emotions is important for understanding human be-havior and modeling users in learning environments. In this paper,we present a deep multi-modal architecture for emotions predic-tion, which takes advantage of deep learning, user multimodal dataand the hierarchy of human memory. The architecture consists ofthe combination of Long Short-Term memory (LSTMs). One of thenovelty of our approach is that, we enhance the LSTM with anexplicit memory since in brain studies, the memory is often dividedinto two further main types: explicit (or declarative) memory andimplicit (or procedural) memory, the last one being the main pur-pose of LSTMs architectures. The resulting model has been testedon a public multi-modal dataset.

In this paper we aim to assess students' visual behavior during problem solving. We propose to use the scan path metric to model the students' analytical reasoning process while solving medical cases. Fifteen participants were recruited for the experiment using an eye tracker to record their eye movements. Our preliminary results have implications for assessing novice clinicians' reasoning processe that can lead us in the future to enhance students' analytical skills.

The use of Video Games as learning tool is becoming increasingly widespread. Indeed, these games are well known as educational games or serious games. They mainly aim at providing to the learner an interactive, motivational and educational environment at the same time. In order to better study the necessary characteristics for the development of an effective serious game (both motivational and educational), we evaluated the physiological responses of participants during their interaction with our serious game, called HeapMotiv. We essentially measured a physiological index of engagement through an EEG wifi headset and studied the evolution of this index with the different missions and motivational strategies of HeapMotiv. Focusing on the gaming aspects, the analysis of this engagement index behavior showed the significant impact of motivational strategies on skills acquisition and motivational experience.

In this paper we seek to model the users' experience within an interactive learning environment. More precisely, we are interested in assessing three extreme trends in the interaction experience, namely flow (a perfect immersion within the task), stuck (a difficulty to maintain focused attention) and off-task (a drop out from the task). We propose a hierarchical probabilistic framework using a dynamic Bayesian network to simultaneously assess the probability of experiencing each trend, as well as the emotional responses occurring subsequently. The framework combines three-modality diagnostic variables that sense the learner's experience including physiology, behavior and performance, predictive variables that represent the current context and the learner's profile, and a dynamic structure that tracks the temporal evolution of the learner's experience. We describe the experimental study conducted to validate our approach. A protocol was established to elicit the three target trends as 44 participants interacted with three learning environments involving different cognitive tasks. Physiological activities (electroencephalography, skin conductance and blood volume pulse), patterns of the interaction, and performance during the task were recorded. We demonstrate that the proposed framework outperforms conventional non-dynamic modeling approaches such as static Bayesian networks, as well as three non-hierarchical formalisms including naive Bayes classifiers, decision trees and support vector machines.

This study investigated players’ motivation during serious game play. It is based on a theoretical model of motivation. Statistical analysis showed a significant increase of motivation during the game. This study tried to dissect predictors of Players’ Motivational States. Multiple linear regression showed statistical significance of specific electrophysiological data. The theta wave in the frontal regions and motivation were positively correlated. High-beta wave in the left-center region was also a significant predictor for high level of motivation. Skin conductance was also a significant predictor for motivation. However, we could not find a significant correlation between players’ motivation and their heart rate responses.

Emotions have a crucial role in our social life in general and in our professional life and education in particular. Besides, they proved essential in information processing systems, more specifically in Intelligent Tutoring Systems (ITS). In this article, we present new strategies integrated into an ITS' tutor module and intended to teach children the vocabulary of a foreign language, namely English. These strategies can generate emotions for students making words memorization easier. We describe the experimentation carried out to validate these emotional strategies as well as the results obtained.

In this paper, we propose to introduce the self-esteem component within learning process. More precisely, we explore the effects of learner self-esteem conditioning in a tutoring system. Our approach is based on a subliminal priming method aiming at enhancing implicit self-esteem. An experiment was conducted while participants were outfitted with biofeedback device. Three physiological sensors were used to continuously monitor learners' affective reactions namely electroencephalogram, skin conductance and blood volume pulse sensors. The purpose of this work is to analyze the impact of self-esteem conditioning on learning performance on one hand and learners' emotional and mental states on the other hand.

This paper explores the feasibility of equipping computers with the ability to predict, in a context of a human computer interaction, the probable user's emotion and its intensity for a given emotion-eliciting situation. More specifically, an online framework, the Emotional Machine, is developed enabling machines to “understand” situations using the Ortony, Clore and Collins (OCC) model of emotion and to predict user's reaction by combining refined versions of Artificial Neural Network and k Nearest Neighbors algorithms. An empirical procedure including a web-based anonymous questionnaire for data acquisition was established to provide the chosen machine learning algorithms with a consistent knowledge and to test the application's recognition performance. Results from the empirical investigation show that the proposed Emotional Machine is capable of producing accurate predictions. Such an achievement may encourage future using of our framework for automated emotion recognition in various application fields.

Cultural profiling involves a complex interplay of multiple dimensions that are virtually impossible to wholly address. This paper explores several cultural (nationality and its associated Hofstede dimensions, religious beliefs), and demographic (gender, age) variables to evaluate if each of these are good candidates for predicting behavioural as well as cognitive attitudes related to computer use and learning activities. Results indicate that each variable taken individually will lead to limited success in attitudinal predictions. Several combinations of variables however could allow an interesting degree of prediction.

The emotional factor has been never taken into account in Intelligent Tutoring Systems (ITS) until recently. However, emotions play a crucial role in cognitive processes particularly in learning tasks [4]. Thus, our purpose in this research study was to analyse the effect of some tutoring actions on the learner's emotional state in order to ascertain the feasibility of positive emotions inducing in ITSs. To achieve this aim, we developed a data structure web course and a virtual tutor using different pedagogical actions to induce positive emotions in the learner. We have conducted an experiment to collect participants' physiological responses after the tutoring actions. The results of this experimental study showed that certain actions have significant positive effects on the learner's emotional state.

We want to explore the relation between affective states, brainwaves and the learner answers during a multi-choice test questions. 24 participants were used in our experiment. While we were measuring their brainwaves, we asked them to answer 35 questions related to the 7 texts they read, for the first time, the day before. During the experiment, the participants can rate, at any time, their emotional dimensions (pleasure, arousal and dominance) on the Self-Assessment Manikin scale (SAM). Measuring the brainwaves determines the learner mental state and the emotional dimensions indicate the learner affective state. When a participant answers, he mentions if he knows the answer or not. Each answer can be either Right or False. The hypothesis of this paper is: “We can predict the learner's answers from his emotional dimensions and his brainwaves”. By using some machine learning techniques, we reached 90.49% accuracy. In a future work, these results will be implemented in an agent to improve the pedagogical strategies and the adaptation of the content within an Intelligent Tutoring System (STI).

It is very common that students fail their exam because of an excessive stress. However, an attitude with no stress at all can cause the same thing as stress can be beneficial in some learning cases. Studying the stress level variation can then be very useful in a learning environment. In this paper, the aim is predicting the stress level variation of the learner in relation to his electrical brain activity within an experiment over two days. To attain that goal, three personal and non-personal characteristics were used: gender, usual mode of study and dominant activity between the first and second day. 21 participants were recruited for our experiment. Results were very encouraging: an accuracy of 71% was obtained by using the ID3 machine learning algorithm.

In this paper, we discuss the use of physiological data for quasi real-time adaptation in ITS. We present preliminary results where we analyze learners' reactions while using a game-like virtual learning environment. We also discuss the relevance of adding cerebral data to adaptation by the means of an electroencephalogram.

The tutor tries, by using feedbacks, to keep the learner's attention and to increase his motivation and then his performance. However, the effectiveness of the tutor's feedbacks on the learner varies from one learner to another. It depends essentially on the learner's traits and it is reflected in the learner's emotional reaction. Consequently, it is important to detect the learner's emotional reaction following the tutor's intervention in order to personalize the feedback type. Our proposal consists of a methodology, based on machine learning technique, able to predict the learner's emotional reaction starting from the analysis of descriptive variables of the learner's current situation, such as: personality, motivation and tutor's feedback type.

Intelligent Tutoring Systems (ITS) learner model has progressively evolved. Initially composed of a cognitive module it was extended with a psychological module and an emotional module. The learner model still remains non-exhaustive. Methods of data collection on the cognitive and emotional state of the learner often lack precision and objectivity. In this paper we introduce an emomental agent. It interacts with an ITS to communicate the emotional state of the learner based upon his mental state. The mental state is obtained from the learner's brainwaves. The agent learns to predict the learner's emotions by using machine learning techniques.

Competence-based learning models have great importance for learning resources: they constitute a meaningful structure for just-in-time and just-enough learning. In this paper, we present an ontology-based competence model that allows the on-the-fly generation of learning knowledge objects (LKOs). The automatic aggregation process relies on knowledge objects and ontologies created through text mining and natural language processing. It is guided by instructional theories encoded declaratively through SWRL. Our framework offers a constructivist learning approach through the presentation of the LKO's context to the learner based on domain ontology. Finally, it allows the standardization of the generated learning objects in SCORM and IMS-LD.

This article presents an autonomous guide agent that can observe a community of learners on the web, interpret the learners' inputs, and then assess their sharing. The goal of this agent is to find a reliable helper (tutor or other learner) to assist a learner in solving his task. Despite the growing number of Internet users, the ability to find helpers is still a challenging and important problem. Although helpers could have much useful information about courses to be taught, many learners fail to understand their presentations. For that, the agent must be able to deal autonomously with the following challenges: Do helpers have information that the learners need? Will helpers present information that learners can understand? And can we guarantee that these helpers will collaborate effectively with learners? We have developed a new filtering framework, called a pyramid collaborative filtering model, to whittle the number of helpers down to just one. We have proposed four levels for the pyramid. Moving from one level to another depends on three filtering techniques: domain model filtering, user model filtering, and credibility model filtering. A new technique is filtering according to helpers' credibilities. Our experiments show that this method greatly improves filtering effectiveness. © 2007 Wiley Periodicals, Inc. Int J Int Syst 22: 1065–1082, 2007.

In this paper, we present The Knowledge Puzzle, an ontology-based platform designed to facilitate domain knowledge acquisition for knowledge-based systems and especially for intelligent tutoring systems. We present a new content model, the Knowledge Puzzle Content Model, that aims to create Learning Knowledge Objects (LKOs) from annotated content. Annotations are performed semi-automatically using natural language processing algorithms. These LKOs are then aggregated in an Organizational memory (OM) which serves as a knowledge base for an intelligent tutoring system (ITS)

Existing multilearner systems support three areas of group collaborative e-learning environments: communication, coordination, and collaboration. This distinction reflects the description of the functional aspects of the strict idea WYSIWIS (what you see is what i see). Yet, this method is only of limited use because it does not take into consideration individual requirements. In other words, it lacks adaptation. Therefore, adding adaptation features to this method personalizes the course presentation to the learner's needs through learning sessions. Accordingly this leads to a new trend that we could call "what you see is adapted to what i need to see" (WYSIA WINS).

We describe our approach to building an automatically adaptive course presentation on the Web. We aim to combine adaptability with the learner-driven course. To achieve this goal, we design and implement an agent called a confidence agent. This agent analyzes the learner's behavior during a learner-driven stage and then not only adapts the next presentation but also updates the domain model accordingly.

Reading is still the main tool for knowledge communication, especially in higher education. Lack of an effective support during this activity, induces many problems of comprehension and understanding. We present the LEKC framework (learning by explicit knowledge construction), which aims to assist readers by an explicit construction of key understanding elements. It introduces knowledge models for the discourse, the epistemic tasks and the instruction.

The promise of digital libraries is an easy and rapid access to digital documents using information technology. However, in an educational context, helping the students access the right document is not sufficient. Systems must be able to help during the reading phase as well. We present the LEKC framework (learning by explicit knowledge construction) [K. Rouane, et al. (2000)] which aims to take advantage of the natural annotation activity to increase the reader's engagement with the document and enhance his understanding, by externalizing and making explicit some of the implicit knowledge construction processes taking place.

Improvements both in technology and in user-oriented software have shown the feasibility of new kinds of non-procedural languages. However, interaction between end-user and data should rely more and more on graphical languages and, particularly, on 'iconic” languages. In the following we review and analyze the forces which are at the origin of changes in the user environment. We give the main specifications of an iconic interface and a command language based on icons. Examples are given in a medical environment.