Multi-task perception networks hold great potential as they can improve performance and computational efficiency compared to their single-task counterparts, facilitating online deployment. However, current multi-task architectures in point cloud perception combine multiple task-specific point cloud representations, each requiring a separate feature encoder, making the network significantly large and slow. In this work, we propose PAttFormer, an efficient multi-task learning architecture for joint semantic segmentation and object detection in point clouds, only relying on a point-based representation. The network builds on transformer-based feature encoders using neighborhood attention and grid-pooling, complemented with a query-based detection decoder using a novel 3D deformable-attention detection head topology. Unlike other LiDAR-based multi-task architectures, our proposed PAttFormer does not require separate feature encoders for multiple task-specific point cloud representations, resulting in a network that is 3× smaller and 1.4× faster while achieving competitive performance on the nuScenes and KITTI benchmarks for autonomous driving perception. We perform extensive evaluations that show substantial improvement from multi-task learning, achieving +1.7% in mIoU for LiDAR semantic segmentation and +1.7% in mAP for 3D object detection on the nuScenes benchmark compared to the single-task models.

Interactive storytelling is a social situation that places a number of demands on a system when realized by an artificial agent. It can be used as a means of teaching or entertainment by a social robot or agent. To be successful, the storytelling has to be interesting and responsive. An agent needs to be aware of the user's state and coordinate the course of the story with the user input. Thus, this is an attractive scenario for the examination of HAI topics by user studies. We implemented an interactive storytelling system using the anthropomorphic robot head Flobi that presents the story with multimodal output and, at the same time, is responsive to multimodal input. The system is designed to serve as a basis for future experiments.

Nodding is an important factor in human communication, providing a physical cue for socially communicative acts such as turn taking, backchanneling, and confirmation. In this article, we describe a vision-based online head nodding detector that works with monocular camera images. Using SVM regression, our system estimates the head pose based on facial landmarks. Subsequence dynamic time-warping is then used to compare head pose features against nod templates. In contrast to many other previous implementations, our system was evaluated with study participants who were not instructed to reply by nodding, and shows good results while maintaining a low false positive rate.

It is generally accepted that a robot should exhibit a contingent behavior, adaptable to the needs of each individual user, to achieve a more natural and pleasant interaction. In this paper we have evaluated whether this general rule applies also when the robot plays a leading role and needs to motivate the human partner to keep a certain pace, as during training or teaching. Also among humans, in schools or factories, structured interaction is often guided by a predefined rhythm, which facilitates the coordination of the partners involved and is thought to maximize their efficiency. On the other hand, a pre-established timing forces all participants to adjust their natural speed to the external, sometimes not appropriate, timing requirement. Where does the optimal trade-off between these two paradigms lie? We have addressed this question in a dictation scenario where the humanoid robot iCub plays the role of a teacher and dictates brief English or Italian sentences to the participants. In particular we compare a condition in which the dictation is performed at a fixed timing with a condition in which iCub monitors subjects' gaze to adjust its dictation speed. The results are discussed both in terms of participants' subjective evaluation and their objective performance, by highlighting the advantages and drawbacks of the choice of contingent robot behavior.

Robots are not human. They might in some cases have a similar appearance but different behavioral and cognitive strengths and limitations. In this sense, an interaction with a robot is asymmetric. When interacting with a robot one is unsure what behavior to expect as the appearance does not necessarily make the abilities of the robot transparent. In human-human interaction, we can also find asymmetric interactions to occur. For example, in an interaction with a child, adults have to adapt to the learner's capabilities and understanding. Similarly, in interactions with special populations such as persons with autistic spectrum disorders (ASD), asymmetry occurs as specific information seems to be processed differently.

Human learning strongly depends on the ability to structure the actions of teachers in order to identify relevant parts. We propose that this is also true for learning in robots. Therefore, we apply a method for multimodal action segmentation called Acoustic Packaging to a corpus of pairs of users teaching object names to a robot. Going beyond previous use cases, we analyze how the structure of human actions changes if the robot is learning quickly or slowly. Our results reveal differences between action structuring in the conditions such as longer utterances and more motion when the robot learns slowly. We also evaluate how the partners in the pair influence each other's action structuring. The results show a strong correlation between the participants in the pairs, even more so in the trials where the robot is learning slowly. We conclude that the action structuring based on Acoustic Packaging allows robots to differentiate how well the interaction with multiple users is going and is, thus, a vehicle for feedback generation.

Robots interacting with humans need to understand actions and make use of language in social interactions. Research on infant development has shown that language helps the learner to structure visual observations of action. This acoustic information typically in the form of narration overlaps with action sequences and provides infants with a bottom-up guide to find structure within them. This concept has been introduced as acoustic packaging by Hirsh-Pasek and Golinkoff. We developed and integrated a prominence detection module in our acoustic packaging system to detect semantically relevant information linguistically\nhighlighted by the tutor. Evaluation results on speech data from adult-infant interactions show a significant agreement with human raters. Furthermore a first approach based on acoustic packages which uses the prominence detection results to generate acoustic feedback is presented.\n\nIndex Terms: prominence, multimodal action segmentation,\nhuman robot interaction, feedback

If robots are to succeed in novel tasks, they must be able to learn from humans. To improve such human-robot interaction, a system is presented that provides dialog structure and engages the human in an exploratory teaching scenario. Thereby, we specifically target untrained users, who are supported by mixed-initiative interaction using verbal and non-verbal modalities. We present the principles of dialog structuring based on an object learning and manipulation scenario. System development is following an interactive evaluation approach and we will present both an extensible, event-based interaction architecture to realize mixed-initiative and evaluation results based on a video-study of the system. We show that users benefit from the provided dialog structure to result in predictable and successful human-robot interaction.