We investigate the potential of gaze as a predictor for the quality of dialogue and the level of understanding in collaborative problem-solving. We unveil a triangular relation among collaborators' dialogue, their gaze pattern, and their performance in the context of a pair programming task. Pairs of participants were asked to understand a JAVA program while their gaze was synchronously recorded. The performance was measured as the level of understanding attained by the pair at the end of the program comprehension task. Gaze patterns were analyzed based on probabilistic hit based areas of interest called gaze tokens. A novel dialogue coding scheme was developed to capture the program description as well as collaboration management aspect of pair program understanding. Both the areas of interest and the dialogue codes reflect top-down and bottom-up program comprehension strategies. Results show that it is possible to relate gaze to the level of abstraction in dialogue and to the level of understanding.

In the context of MOOCs, "With-me-ness" refers to the extent to which the learner succeeds in following the teacher, specifically in terms of looking at the area in the video that the teacher is explaining. In our previous works, we employed eye-tracking methods to quantify learners' With-me-ness and showed that it is positively correlated with their learning gains. In this contribution, we describe a tool that is designed to improve With-me-ness by providing a visual-aid superimposed on the video. The position of the visual-aid is suggested by the teachers' dialogue and deixis, and it is displayed when the learner's With-me-ness is under the average value, which is computed from the other students' gaze behavior. We report on a user-study that examines the effectiveness of the proposed tool. The results show that it significantly improves the learning gain and it significantly increases the extent to which the students follow the teacher. Finally, we demonstrate how With-me-ness can create a complete theoretical framework for conducting gaze-based learning analytics in the context of MOOCs.

We present an eye-tracking study where we augment a Massive Open Online Course (MOOC) video with the gaze information of the teacher. We tracked the gaze of a teacher while he was recording the content for a MOOC lecture. Our working hypothesis is that displaying the gaze of the teacher will act as cues in crucial moments of dyadic conversation, the teacher-student dyad, such as reference disambiguation. We collected data about students’ video interaction behaviour within a MOOC. The results show that the showing the teacher’s gaze made the content easier to follow for the students even when complex visual stimulus present in the video lecture.

In co-located meetings, participants create and share content to establish a common understanding. In this paper, we present a collaborative environment that enables group members to create and share content simultaneously by providing them with different kinds of individual input devices and a shared workspace. We also report on an exploratory study to investigate the influence of the input device used on the shared knowledge produced by the group. The results suggest that driven by the affordances, various input devices complement each other. We thus recommend groups to be equipped with multitude of them to support diverse meeting task demands. Additionally, we observed that groupware usage differs across various phases of the problem-solving activity. This provides implications for the design of collaborative environments to assist each of the respective phases of the task, in order to extend their usefulness for the group.

We present a dual eye-tracking study that demonstrates the effect of sharing selection among collaborators in a remote pair-programming scenario. Forty pairs of engineering students completed several program understanding tasks while their gaze was synchronously recorded. The coupling of the programmers' focus of attention was measured by a cross-recurrence analysis of gaze that captures how much programmers look at the same sequence of spots within a short time span. A high level of gaze cross-recurrence is typical for pairs who actively engage in grounding efforts to build and maintain shared understanding. As part of their grounding efforts, programmers may use text selection to perform collaborative references. Broadcast selections serve as indexing sites for the selector as they attract non-selector's gaze shortly after they become visible. Gaze cross-recurrence is highest when selectors accompany their selections with speech to produce a multimodal reference.

We present the results of group dynamics and their effect on success in problem solving/decision making meetings. We use a novel multiple input environment for collaboration and data collection, and a hidden profile task given to groups, whose goals are to find the correct solution. We observe that groups elect 0, 1 or 2 leaders, and the best results are obtained by the groups with a single leader. Prior acquaintance (familiarity), does not show any effect on the success or on the group strategies. Groups with a single leader tend to be more successful, and leaders expressed their authority verbally rather than by through the collaborative system.

We present the design and evaluation of an iPad application that will be used to operate the modular robots “Roombots”. Roombots are the building blocks for adaptive pieces of furniture. The application allows a user to arrange adaptive furniture within a room. We conducted a user study with 24 participants to evaluate our approach and to freely explore people's interaction. Data suggests that the ability to move with the tablet leads to a better precision of the furniture arrangement. No significant difference has been observed between using the application through a virtual representation of the room in contrast to an augmented reality environment, even if participants mentioned in a post-study questionnaire their preference for the augmented condition. Users described the interface as intuitive and easy to use.

This paper exploits the personal aspects of an individual's eye-movements in dynamic Tetris-playing environments. Effective features representing the players' eye-moving characteristics are extracted, and they are shown to be different across difference players. Delta features are also calculated to present the dynamic changes of the static features. A series of personal identification experiments are performed by using a hidden Markov models (HMM). Our experimental results show that compared with local information, modeling and tracking the dynamic temporal information (i.e., delta features) is of more importance in distinguishing different players' eye-movement. Given a 10-zoid consecutive playing signals (about 30 seconds) we can achieve an identification rate of 82.1% by combining them both.

Tabletop systems have become quite popular in recent years, during which there was considerable enthusiasm for the development of new interfaces. In this paper, we establish a comparison between touch and tangible interfaces. We set up an experiment involving several actions like translation and rotation. We recruited 40 participants to take part in a user study and we present our results with a discussion on the design of touch and tangible interfaces. Our contribution is an empirical study showing that overall, the tangible interface is much faster but under certain conditions, the touch interface could gain the upper hand.

This paper describes TinkerSheets, a paper-based interface to tangible simulations. The proposed interface combines the advantages of form-based input and paper. Form-based input allows to set an arbitrary number of parameters. Using paper as a medium for the interface keeps the interaction modality consistently physical. TinkerSheets are also used as an output screen to display summarized information about the simulation. A user study conducted in an authentic context shows how the characteristics of the interface shape real world usage. We also describe how the affordances of this control and visualization interface support the co-design of interaction with end-users.