Scientists of the mind have always sought to understand how human beings grow and analyse the visual and acoustic information, move their body, reason, learn, memorise, forget, develop emotion, and build consciousness. There are functions that neuroscience examine. Scientists have also used materials to try to simulate life artificially. There is no disruption but continuity between these approaches. The aim of the talk is to discuss the relationship between neuroscience and robotics and reflect on the brain-mind-machine concept, on its internalised and externalised expressions. The ambition is to understand the enactment from the real to artificial world is rendered possible via robots. Robots are therefore considered oeuvres de l'esprit (i.e. creations of the mind). Just as importantly, what has been internalised and symbolised has been simulated and externalised. The result is that life reproduction via robots is the result of the interaction between external information (i.e. objects including their shadows) and internal interpretations: enrobotment. The construction of animate robots and humanoid robots in particular signifies that they have been incorporated into the mind. This may be the continuum from Deus ex Machina to Conscientia ex Machina.

The Ventral Pathway (VP) and Dorsal Pathway (DP) are responsible for identifying (what) and localising (where) objects in 3D space. These pathways project integrated information into the frontal (FC) and the prefrontal (PF) cortices making possible for humans not only to interact with objects but also express motivations and emotions directly or indirectly associated with the actions they perform with objects. The neural implications of these pathways were examined in a sample of healthy participants ( n = 30) aged 22 years old while invited to create an installation using a set of common (CO) and uncommon objects (UO) in augmented reality (AR). All participants were equipped with a 32 channels EEG, a frequency analysis of brain waves has been performed. The results have shown an alpha power increase in the ventral pathway when UO were involved in the creative task. This finding is attributed to a greater need to attend the UO as no representation of them exists in the VP. Increased theta and delta powers were observed in the FC when creating an installation with UO, indicating more higher-order functioning involving emotion and motivation. Engaging multimodal complex brain areas, it seems that creativity in augmented reality requires similar brain patterns as in real world.

ASD children are characterised by a lack of intentionality. We analysed nonverbal and verbal information, associated with heart rate and emotional feeling, respectively, in ASD and neurotypical children. Analogies of heart rate between ASD and neurotypical children were expressed when the human was ‘passive’ actor and the robot was ‘active’ actor; disanalogies were released when the human was ‘active’ actor. Only ASD children reported better emotional feeling ‘after’ than ‘before’ the interaction with the robot. These results suggest that ASD children might be more reliable to low-level intentionality represented by robots, than to high-level intentionality associated with humans.

We developed the new therapy device named “Digital Play Therapy Device”, which combines house-shaped device with sensors attached and CG. The device was targeted for children with learning disabilities. We obtained the result from routine developed children, and obtained the result of increasing the number of speaking and decreasing the number of miscommunication, comparing to normal dollhouse. However, it’s difficult to compare these results objectively. This paper analyzes the effect of “Digital Play Therapy Device (DPTD)” with more detailed method by using biological information. We compared the effect between DPTD and just operating house-shaped device, DPTD and just operating CG. Analysis of the experiment indicates that participants who experienced DPTD exerts higher concentration than operating house-shaped device or CG individually.

Children suffering with learning and developmental disorders require daily training to develop their social skills. However, such daily training is sometimes not provided because it requires interactive help from therapists, and lots of special programs required for the training. We proposed an immersive digital dollhouse that enhances traditional psychological play therapy with sensors installed, while provides feedback from the sensors to the computer graphics (CG). The immersive digital dollhouse provides an immersive space for children, which develops their communication skills through their imaginary play with the complement of CG for enhancing the understanding of their situation, and it is easy to use in their home as daily training. We conducted an experiment with a 6-year old child with a similar level as the older children with developmental disorders. We found that the number of reactions was twice as many as with a normal dollhouse, and the number of miscommunications was less than one-tenth.

In the present interdisciplinary study, we have combined cognitive neuroscience knowledge, psychiatry and engineering knowledge with the aim to analyze emotion, language and un/consciousness in children aged 6 (n=20) and 9 (n=20) years via a listener-speaker communication. The speaker was always a child; the listener was a Human InterActor or a Robot InterActor, i.e.,. a small robot which reacts to speech expression by nodding only. Unconscious nonverbal emotional expression associated with physiological data (heart rate) as well as conscious process related to behavioral data (number of nouns and verbs in addition reported feelings) were considered. The results showed that 1) the heart rate was higher for children aged 6 years old than for children aged 9 years old when the InterActor was the robot; 2) the number of words (nouns and verbs) expressed by both age groups was higher when the InterActor was a human. It was lower for the children aged 6 years than for the children aged 9 years. Even if a difference of consciousness exists amongst the two groups, everything happens as if the InterActor Robot would allow children to elaborate a multivariate equation encoding and conceptualizing within their brain, and externalizing into unconscious nonverbal emotional behavior i.e., automatic activity. The Human InterActor would allow children to externalize the elaborated equation into conscious verbal behavior (words), i.e., controlled activity. Unconscious and conscious processes would not only depend on natural environments but also on artificial environments such as robots.

Using a "speaker-listener" situation, we have compared the verbal and the emotional expressions of neurotypical and autistic children aged 6 years. The speaker was always a child (neurotypical or autistic); the listener was a human or an InterActor robot named Pekoppa, i.e., a small toy robot which reacts to speech expression by nodding only. The results suggest that an InterActor robot characterized by small variance nonverbal behavior better facilitate verbal and nonverbal emotional expressions of autistic children than a human. This might be related to autistic children preferences: autistic children are rather interested in minimalist objects to which they can assign mental states of their own or of others. Everything happens as if the InterActor robot could allow autistic children to elaborate a multivariate equation encoding and conceptualizing within his/her brain, and externalizing into unconscious emotion (heart rate) and conscious verbal speech (words).

Language offers the possibility to transfer information between speaker and listener who both possess the ability to use it. Using a “speaker-listener” situation, we have compared the verbal and the emotional expressions of neurotypical and autistic children aged 6 to 7 years. The speaker was always a child (neurotypical or autistic); the listener was a human InterActor or an InterActor robot, i.e., a small toy robot which reacts to speech expression by nodding only. The results suggest that a robot characterized by predictable reactions is better able to facilitate autistic children in expression than a human. When comparing to the performance of neurotypical children, the data would indicate that minimalistic artificial environments have the potential to open the way for neuronal organization and reorganization with the ability to support the embrainment of verbal and emotional information processing among autistic children.

Under the hypothesis that multimodal cognitive processes could be though as a building block from which language could emerge in autistic children, we used a toy robot with and without free game play. A range of cognitive nonverbal criteria including eye contact, touching, manipulation, and posture were analyzed; the frequency of the words and verbs was calculated. Only during game play the children interact quasi-continually and multimodally with the robot and express language. A positive correlation exists between expressive language and multimodal cognitive processes only when free game play with the robot is possible. The data suggest that a mobile toy robot could be used as a neural orthesis in order to improve autistic children's brain multimodal activity and incite them to express language.