Operators of critical systems (e.g., air traffic controllers, pilots in commercial aircraft, operators managing space vehicle launch…) must acquire specific knowledge before being granted permission to perform their tasks on the job. They acquire such knowledge by following a training program developed for the specific tasks they will be doing. The preparation of their training program requires identifying the knowledge they have to acquire, and in particular, the two main types of knowledge that are declarative knowledge (a set of concepts required to operate a system) and procedural knowledge (a set of tasks required to operate a system). Task analysis techniques were originally proposed for the purpose of identifying and describing the knowledge required by users to interact with a system. These techniques provide support to describe procedural knowledge in detail, and a few of them provide limited support to describe declarative knowledge. In this paper, we propose a model-based technique to exhaustively identify and describe declarative and procedural knowledge, as well as to couple the different types of models in order to ensure the full coverage and consistency of the identification of knowledge required to operate an interactive system. We present the results from the application of the technique on a case study in the space domain.

Designing new input devices and associated interaction techniques is a key contribution in order to increase the bandwidth between users and interactive applications. In the field of Human-Computer Interaction, research and development services in industry and research laboratories in universities have been, since the invention of the mouse and the graphical user interface, proposing multiple contributions including the integration of multiple input devices in multimodal interaction technique. Those contributions, most of the time, are presented as prototypes or demonstrators providing evidence of the bandwidth increase through user studies. Such contributions however, do not provide any support to software developers for integrating them in real-life systems. When done, this integration is performed in a craft manner, outside required software engineering good practice. This paper proposes a systematic process to integrate novel input devices and associated interaction techniques to better support users' work. It exploits most recent interactive systems architectural model and formal model-based approaches for interactive systems supporting verification and validation when required. The paper focusses on Frame-based input devices which support gesture-based interactions and movements recognition but also addresses their multimodal use. This engineering approach is demonstrated on an interactive application in the area of rehabilitation in healthcare where dependability of interactions and applications is as critical as their usability.

Post-stroke or post-brain injuries rehabilitation is a long-term process defined by therapists and tuned to the specific damages and their consequences for each patient. This process requires the definition of well-defined multiple exercises to be performed multiple times by each patient over a long period of time. One of the key findings in the rehabilitation domain is that exercises should be tasks and goals oriented to increase functional independence, enhance development and prevent disability [24]. Patients performing the exercises must be carefully monitored to identify progress, possible regression and prevent damages due to the inadequate performance of the exercises. The SIVIRE tool [20] provides interactive graphical feedback to patients about the performance of their exercises and deviations from nominal, planed practice but fails at providing high-level descriptions of exercises and at connecting them to high-level rehabilitation goals (e.g. gain autonomy in cleaning a room). The HAMSTERS-XLE tool [16] provides a tuneable editor and a simulator to describe goals and task models including low-level tasks covering perception of the various senses, cognition and motoric actions. This paper presents the integration of these two tools and the benefits it brings both to the patients and to the therapists. The resulting tool and its engineering are presented in detail highlighting some generic integration mechanisms. On a case study, we show how exercises can be defined in HAMSTERS-XLE, transferred to SIVRE for fine tuning, monitored by SIVRE during the multiple practices of the patient and transferred back again to HAMSTERS-XLE to monitor progress, identify issues with the exercises and provide therapists with goal and task-based data about patients' evolutions

Training operators of complex interactive systems is a difficult task that involves multiple actors, requires specific competencies and may be extremely costly in terms of time and resources. For instance, an initial training has to be performed in order to bring operators to a desired level of declarative and procedural knowledge. A successful operator will be granted a qualification to operate the system that was used for (or targeted by) the training. However, depending on the nature and the diversity of the operations performed on a daily basis, this initial training may decay and some knowledge may become deprecated. Recurrent training needs to be organised on a regular basis in order to keep operators qualified for the work. Such training differs from initial training and requires taking into account information about how humans learn and forget, what has been performed by operators since last training and the expected required level of knowledge. Trainers need to encompass all this generic information (to all operators) and specific information (to each individual) to define both initial and recurrent training programs. One particularly cumbersome task is the gathering of what happened during operations for each operator in order to minimize recurrent training program to knowledge that might have been forgotten as it was not used in recent operations. We propose a tool-supported task-model approach fed by information of the real work of operators in order to identify complete, relevant and efficient training for operators. These contributions have been applied to the civil aviation domain demonstrating multiple induced benefits.

Along the design process of interactive system multiple intermediate artefacts (such as user interface prototypes, task models, dialog models?) are created, tested, revised, and improved until the development team produces a full-fledged system. However, relevant information for describing the design solution and/or supporting design decisions (such as rational about the design, decisions made, recommendations, etc.) are not explicitly captured in the models/artefacts, hence the need for annotations. Many approaches argue against information duplication to increase maintainability of the artefacts. Nonetheless, annotations created on one artefact are usually relevant to other artefacts/models. So that, there is a need for tools and techniques to coordinate annotations across artefacts/models which is the contribution of the present work. In this paper, we propose a model-based approach that was conceived to handle annotations in a systematic way along the development process of interactive systems. As part of the solution, we propose an annotation model built upon the W3C's Web Annotation Data Model. The feasibility of the approach is demonstrated by means of a tool suite featuring a plugin, which has been deployed and tested over the multi-artefacts. The overall approach is illustrated on the design of an interactive cockpit application performing two design iterations. The contribution brings two main benefits for interactive systems engineering: i) it presents a generic pattern for integrating information in multiple usually heterogenous artefacts throughout the design process of interactive systems; and ii) it highlights the need for tools helping to rationalize and to document the various artefacts and the related decisions made during interactive systems design.

The goal of software testing is to detect defects with the objective of removing them at a later stage in the development process. Interactive software development follows the User Centered Design approach that promotes continuous involvement of users both at design and evaluation phases. This process is meant to produce usable interactive software by gathering functional and non-functional requirements related to both user needs and context of use. However, taking into account these potentially very-complex-to-implement requirements increases the complexity of the software that is likely, without appropriate methods and tools, to encompass a large number of defects. One of the limitations of UCD approaches is that it provides no guidance on the engineering of the interactive application, which thus usually embeds numerous defects resulting in failures at the origin of user frustrations and performance drops. Even though a classification of interactive application defects has been proposed, interactive application testers remain only superficially supported in detecting them. This paper defines a model-based approach to engineer the testing activity for interactive applications. It proposes a process that bridges the gap between UCD artefacts and interactive software implementation by the production of a dedicated formal model exploited for testing purposes only. The application of the process is demonstrated on an interactive cockpit WIMP application. Finally, threats to validity (capability of the approach to detect defects and to ensure an acceptable coverage testing of the interactive application) are addressed by a longitudinal study on 61 variants of a simple application developed by 61 different developers. ?

When task descriptions are precise they can be analysed to yield a variety of insights about interaction, such as the quantity of actions performed, the amount of information that must be perceived, and the cognitive workload involved. Task modelling notations and associated tools provide support for precise task description, but they generally provide a fixed set of constructs, which can limit their ability to model new and evolving application domains and technologies. This article describes challenges involved in using fixed notations for describing tasks. We use examples of recognized tasks analysis processes and their phases to show the need for customization of task notations, and through a series of illustrative examples, we demonstrate the benefits using our extensible task notation and tool (HAMSTERS-XL).

Designing systems in such a way that as much functions as possible are automated has been the driving direction of research and engineering in aviation, space and more generally in computer science for many years. In the 90's many studies (e.g. [12] related to the notion of mode confusion) have demonstrated that fully automated systems are out of the grasp of current technologies and that additionally migrating functions [2] from the operator to the system might have disastrous impact on operations both in terms of safety and usability. In order to be able to design automation with a hedonic view of the involved factors (safety, usability, reliability, ...) a complete understanding of operator's tasks is required prior to considering migrating them to the system side. This paper proposes a contribution for reasoning about automation designs using a model-based approach exploiting refined task models. These models describe operations with enough details in order to reason about automation and to rationalize automation designs. In this paper we present how such representations can support the assessment of alternative design options for automation. The proposed approach is applied to satellite ground segments.

The design of real-life complex systems calls for advanced software engineering models, methods, and tools in order to meet critical requirements such as reliability, dependability, safety, or resilience that will avoid putting the company, the mission, or even human life at stake. When such systems encompass a substantial interactive component, the same level of confidence is required towards the human-computer interface. Conventional empirical or semiformal techniques, although very fruitful, do not provide sufficient insight on the reliability of the human-system cooperation, and offer no easy way to, for example, quantitatively and qualitatively compare two design options with respect to that reliability. The aim of this article is to present a user interface description language (called ICOs) for the engineering and development of usable and reliable user interfaces. The CASE tool supporting the ICOs notation (called Petshop) is a Petri nets-based-tool for the design, specification, prototyping, and validation of interactive software. In that environment models (built with the formal description technique ICOs) of the interactive application can be interactively modified and executed. This is used to support prototyping phases (when the models and the interactive application evolve significantly to meet late user requirements, for instance) as well as the operation phase (after the system is deployed). The use of ICOs and PetShop is presented on several large-scale systems such as a multimodal ground segment application for satellite control, an air traffic control interactive application, and an application for new generation of interactive cockpits in large civil aircraft such as Airbus A380 or Boeing 787. The article emphasizes the demonstration of the expressive power of the notation and how it can support the description of various aspects of user interfaces, namely interaction techniques (both WIMP and post-WIMP), interactive components (such as widgets), and the behavioral part of interactive applications such as the dialog and the functional core. It also demonstrates that PetShop provides dedicated support for prototyping activities of behavioral aspects at the various levels of the architecture of interactive systems. While the focus is on past work done on various large-scale applications, the article also highlights why and how ICOs and Petshop are able to address challenges raised by next-generation user interfaces.