International audience

International audience

Machines, e.g. empowered by AI and based on virtual models, can help to improve the quality of life. To exploit this potential and also integrate this with the real world and society, cooperation and teaming of these machines with humans, and with societies is crucial. Human-Machine Patterns can be a key concept to analyze, understand, design, engineer and evaluate the delicate interplay of humans and machines. Key issues here are to understand in which situations which agents should do, want to do, can do, will do and finally actually do which actions, and who then is accountable. This overview article is intended as an introduction into the special session on Shared and Cooperative Control, especially on patterns and models for controllability and resilience. It is a direct follow-up on the 2022's special session and overview article (which is also available on IEEE Xplore). It introduces the topic of shared and cooperative control of human-machine and human-AI systems, especially in the light of the new advances in AI technology. This paper gives a short overview on the state of research on interaction patterns, controllability and resilience, before it focuses on the fundamental aspects of which actor should do, wants to do, can do, does and will be accountable for a pattern, sub-pattern or action within a pattern. Examples of what can be achieved with this basic architecture are given, e.g. for the recognition of intent or for the support by assistant systems, using the automotive domain as a first application example.

This paper proposes to model the dynamic management of dialogue in human-machine cooperation. It seeks to articulate a generic model of human-machine cooperation, based on two complementary abilities of agents (the Know-How-to-Operate and the Know-How-to-Cooperate), with 2 prominent models of agent transparency (SAT and HRT model). Modeling is applied to a specific Human-Machine Cooperation, a cyber-physical system assisting a human for planning decisions of maintenance operations. Dialogue management mechanisms filter the information displayed on the interface, from the agent's analysis of the supervised process to the team's performance and cooperation activities. This dynamic dialogue management contributes to the calibration of trust and optimisation of cooperation.

With increasing autonomous capabilities and intelligence of machines and interconnectivity of humans and machines, interaction and cooperation become even more crucial for socio-cyber-physical systems. How can we organize the exploding number of design options in a way that it is easy for humans to understand and control these systems? Patterns for the design of interaction and cooperation could be the key for describing problems and generalizable solution patterns, which can be instantiated again in individual solutions. Modeling, matching and instantiating could be the fundamental use cases for those patterns. Understanding, designing, engineering and using are the essential activities in an interplay of researchers, designers, engineers and users of human-machine systems. A realistic dream is to have global databases of human-machine patterns and their effects so that the wheel does not have to be re-invented several times and common knowledge about patterns can be shared in our community.

Brain-Computer Interface technology will be soon available to the general public. It would be to replace or compensate physical disability in the case of handicap. However, it would also be for all people looking for peaceful consciousness, to control environment from no other types of command but directly from the brain. Such a device that could be easily and quickly handled by untrained and novice users has been proposed by Mentalista. The company designed and trained a Brain-Computer Interface and in this paper, we propose a method to identify how such a device may be used to control a robot. The method proposes to identify the cooperative patterns that suit users’ states and robot’s abilities regarding the environment constraints. The Human-Machine Cooperation principles support the methodological approach.

Since the first principles of Human-Machine System proposed by Thomas Sheridan in 1974, Human-Machine System designers are fighting to find what could be the best organization between humans and machines. Several criteria can be used to evaluate such organizations and find the best combination of functions completed by human and machine. A good combination of functions adapted to specific situations may be identified as cooperative patterns used by agents to succeed in their cooperative tasks. The objective of this paper is firstly to introduce the concept of pattern and the way to adapt this concept in the context of cooperation. Then, a second part addresses methodological aspects and proposes methods to identify cooperative patterns. The third and fourth parts apply this method to the design of a Human-Cyber-Physical System studied within the framework of the HUMANISM project, and preliminary results of an experiment are presented.

With technological progresses, machines such as robots are increasingly used to work on hazardous and dynamic environments. In general, robots work with humans in a team. For this reason, our research aims to improve human-robot cooperation using human-machine cooperation principles. An important aspect of cooperation is function allocation. The function allocation between human and robot is mainly managed through Levels of Automation. This current study focuses on the implementation of proposed Levels of Automation in a real application for crisis management with firefighters, where a human operator is in a command post and robots operate in a hostile environment. Results regarding the human operator situation awareness, workload and task performance during an experiment are presented.

Better human machine cooperation is based on human trust in the machine. It has not been established, however, the system design which assures the machine is trusted by human operators in an appropriate manner. This is because the notion of trust is still vague since it has several aspects. The paper proposes to adopt a human-machine cooperation framework that the authors have developed. Based on the human-machine cooperation framework, each aspect of trust is identified. Examples are given to help readers understand each aspect.

Emerging technologies in the field of automatization meanwhile enable partially and highly automated vehicles in the aviation and automotive domains, where the human operators are assisted or (partially/temporarily) replaced in their tasks by advanced automation systems. Nevertheless due to technological limitations and ethical reasons, full autonomous vehicles in both domains might not be realizable in the near future. Instead system designs, which enable shared and cooperative guidance and control of vehicles by human operators and automation systems could be more feasible solutions. This paper sketches a common framework of shared and cooperative control that describes the two concepts not as different but as coinciding concepts for the shared intentionality, control and cooperation between humans and machines. A brief overview off developed shared and cooperative control designs in the aviation and ground vehicle domain is given.

In highly dynamic domains like mobile robotics, it is essential to keep human in the control loop of the process and to maintain his/her situation awareness through adequate levels of automation. Our paper is focused on how to better match human and machine capabilities defining adaptive levels of automation. A method based on human-machine cooperation concepts which allow designing different possible human-machine interactions is used in order to identify different adaptive levels of automation. We present our approach to control the cooperation among the agents on use cases in the crisis management domain with firefighters and robots. Three types of agents take place in the scenario, human supervisor, firefighter with smart garment and robots. At the operational level, the robots and firefighter have to cooperate together and with the human supervisor in order to achieve a common goal. The human supervisor is at the tactical level; he/she can observe the situation, make decision about the process and send instructions to the robots using a visual support in which large amounts of information are presented. The proposed approach helps the agents to better organize their cooperative actions.

Since the start of industrialization, machine capabilities have increased in such a way that the control of processes by humans is becoming increasingly complex. This is especially the case in Intelligent Manufacturing Systems for which processes tend to be so autonomous that humans are more and more unaware of processes running, particularly when humans may need to intervene to update the manufacturing plan or to modify the process configuration if machines or intelligent entities need assistance. The present paper proposes solutions based on the use of Human(s)-Machine(s) Cooperation (HMC) principles to support humans in the process control. The aim of these principles is to adopt a human-centered approach for the design and evaluation of assistance systems and processes, as well as their interaction with humans. Two main complementary features of HMC, the know-how and the know-how-to-cooperate, are detailed. They provide a very useful approach to design task allocation, support for mutual understanding and communication between one human operator and one Artificial Self Organizing system. An assistance system resulting from this approach was evaluated and first results highlighted the improvement of global performance and acceptability.

The current study focuses on how the levels of automation and the number of simultaneous functions that complete the task demands affect the human operator performance. The research objective is to assess the effects of different levels of automation of vehicles (teleoperated VS. autonomous) and the complexity of the task on human operator performance. An experiment is realized using the simulation environment “Mixed Initiative Experimental” (MIX) which is designed for research and training with unmanned systems. The MIX platform aims to respond to the need for warfighters support and training in the domain of robotic military equipment and unmanned systems in general. The complexity of the task is simulated by increasing the number of autonomous vehicles that a human operator has to supervise or to teleoperate.

Driving with shared control systems has been shown to be efficient in reducing collisions during a lane change. However, the ability of the current shared control systems is limited and their absolute reliability is not assured. The goal of this study is to determine how changes in system performance affect human-machine cooperation and safety. An experiment using a driving simulator was designed to compare two types of support systems, (1) haptic shared control that provides control guidance to resists hazardous lane changes; and (2) mixed-input control that autonomously controls vehicle directions to prevent lane change collisions. Both systems were examined under three conditions: (i) 100% reliability, (ii) full reliability but limited system ability, and (iii) system failure. While both systems were judged trustworthy and accepted by the drivers when fully reliable, the mixed-input was more efficient at maintaining safety. In the second condition, drivers overestimating the capabilities of the system led to more collisions for both systems. The results illustrate that how machine failure impacts surprises and distrust in automation is highly dependent on driver's ability to regain the control. Overall, the haptic guidance was more efficient and accepted more by the drivers.

Goals of shared control usually focus on the sharing of the action. Several examples such as shared control in robotics, in car driving, in wheelchair control highlight the interest for Human operator to be able to take part in the control of a process in the same way as assistance systems. Moreover, with the haptic shared control, Human operator can feel what could be or what is the current action of the assistance system. But if shared control is so useful today in some domains and will also be very interesting for other domains, it is because action is not the only function that Human operator can share with an assistance system. In fact, we would even like to suppose that the more Human operator and assistance systems share problem solving process functions, the more their cooperation would be safe, efficient, and comfortable. The objective of this paper is to propose an extension of the use of Shared control concept by giving other opportunities for Human operator and assistance system to share activity. The horizontal extension relates the integration of all the functions, from information gathering to action implementation. The vertical extension relates the integration of the levels of activity usually represented by the strategic, tactical and operational levels. Human machine cooperation principles are used to attempt such an objective.

Manufacturing companies have to be more and more competitive. They have to improve continuously products quality, as well as reducing cost and shorting production time. The paper focuses on the design of a system that is able to ensure product quality and process protection against damages. The system, so-called “robust filter”, has to stop the process when the command, firstly prepared by process engineer, does not cope anymore with process state. Indeed, some modifications would have been introduced by other agents according to new orders or process' components. But when the process is stopped, all information that could provide clues to explain problem are removed. So, Human-Machine interfaces have been proposed in order to support maintenance agents' activity when they have to understand errors in command. The design of such interfaces has been performed with the principles stemming from the Human-Machine Cooperation approaches and assessed with students.

The interest of cooperation is more and more highlighted. When competition has been presented as the main driving force of species evolution, cooperation is today presented as the mean for everything or everyone to reach the necessary symbiosis with another one to exist and to evolve. Cooperation has been central to our studies now for over twenty years, with the conviction that automation has to take part in evolution supporting individual activity with an assistance system or supporting interactions. Model of cooperative activity is presented in order to help to define such supports, as well as Levels of Cooperation (LoC) which are combinations of parts of the model. Nevertheless, each one is always assessing the interest of cooperation by the valuation of the risk to loose or the chance to earn something. In this paper, we propose to use BCD (Benefit/Cost/Deficit) model to assess such a risk or chance to cooperate. Model of cooperative activity in terms of know-how and know-how-to-cooperate and model of BCD are first presented and used together in order to define a way to design and to evaluate new or existing human(s)-machine(s) systems.

Previous works in human engineering and particularly in the air traffic control domain show three basic conditions for designing a cooperative decision support system: 1) sufficient know-how for solving problems in an autonomous way; 2) know how to cooperate and 3) adequate organizational structure integrating human and machine. Through a multi-disciplinary approach including cognitive psychology and human engineering research we discuss some basic concepts. The paper gives a definition for cooperation from works by psychologists:"two agents are cooperating if 1) each one strives towards goals and can interfere with the other (on goals, resources, procedures, etc.) and 2) each agent tries to detect and process such interferences to make the other's activities easier". Two generic structures, respectively called vertical (hierarchical) and horizontal (heterarchical), are presented. When evaluating these structures, we show that the choice of a cooperative structure is not sufficient for allowing a real cooperation between human and machine. Therefore, deriving K. Schmidt's (1991) typology of the cooperative forms and this definition, we specify the capabilities needed for managing interferences between goals (MI) and the capabilities for making other agents' goals easier (FG). We then transpose MI and FG to human-machine cooperative systems, taking into account technical limitations of artificial agents and constraints imposed by the natural structure of the human-machine organization.