For autonomous driving of construction machinery such as construction crane, it is necessary to detect objects (e.g. person) in a construction site using the camera hung from the crane’s boom. Since the camera is attached to the boom, it is in motion as the boom moves from one place to another. In this research, we propose a camera motion compensation and person detection technique in a construction site for moving camera. The motion parameters are estimated using point-to-point features correspondences with RANSAC-like algorithm. The algorithm selects the best model with few randomly selected sample points. After compensating the motion effect, a Bayesian interface along with the Yolo model (called Yollo-Bayes model) is used for person detection. Two types of Bayes models are proposed to detect missing person by the Yolo. Bayes model for moving person is used to detect the person when he/she is in motion and Yolo fail to detect him/her. On the other hand, Bayes model for nonmoving person is used when the person is not in motion and Yolo miss to detect him/her. The motion estimation accuracy is confirmed by creating ground truth images with translation and rotation changes. To evaluate the performance of the Yolo-Bayes model, extensive experimental evaluations have been done with a lot of video images collected from the real construction site. It reveals that the proposed Yolo-Bayes model outperforms Yolo model. The proposed model is more effective when the number of training sample is small (Fβ– score is 5.77% higher than Yolo) or the model is tested on a completely new dataset (Fβ– score is 13.75% higher than Yolo).

We propose a human-robot interaction approach for social robots that attracts and controls the attention of a target person depending on her/his current visual focus of attention. The system detects the person's current task (attention) and estimates the level by using the “task-related contextual cues” and “gaze pattern.” The attention level is used to determine the suitable time to attract the target person's attention toward the robot. The robot detects the interest or willingness of the target person to interact with it. Then, depending on the level of interest of the target person, the robot generates awareness and establishes a communication channel with her/him. To evaluate the performance, we conducted an experiment using our static robot to attract the target human's attention when she/he is involved in four different tasks: reading, writing, browsing, and viewing paintings. The proposed robot determines the level of attention of the current task and considers the situation of the target person. Questionnaire measures confirmed that the proposed robot outperforms a simple attention control robot in attracting participants' attention in an acceptable way. It also causes less disturbance and establishes effective eye contact. We implemented the system into a commercial robotic platform (Robovie-R3) to initiate interaction between visitors and the robot in a museum scenario. The robot determined the visitors' gaze points and established a successful interaction with a success rate of 91.7%.

In this paper, we propose a human-robot interaction system in which the robot detects and classifies the target human's gaze pattern into either spontaneous looking or scene-relevant looking. If the gaze pattern is detected as the spontaneous looking, the robot waits for the target human without disturbing his/her attention. However, if the gaze pattern is detected as the scene-relevant looking, the robot establishes a communication channel with him/her in order to explain about the scene. We have implemented the proposed system into a robot, Robovie-R3 as a museum guide robot and tested the system to confirm its effectiveness.

Recent research in HRI has emphasized the need to design affective interaction systems equipped with social intelligence. A robot's awareness of its social role encompasses the ability to behave in a socially acceptable manner, the ability to communicate appropriately according to the situation, and the ability to detect the feelings of interactive partners, as humans do with one another. In this paper, we propose an intelligent robotic method of attracting a target person's attention in a way congruent to satisfying these social requirements. If the robot needs to initiate communication urgently, such as in the case of reporting an emergency, it does not need to consider the current situation of the person it is addressing. Otherwise, the robot should observe the person to ascertain who or what s/he is looking at (VFOA), and how attentively s/he is doing so (VFOA level). Moreover, the robot must identify an appropriate time at which to attract the target person's attention so as to not interfere with his/her work. We have realized just such a robotic system by developing computer vision methods to detect a target person's VFOA and its level, and testing the system's effectiveness in a series of experiments.

A major challenge is to design a robot that can attract and control human attention in various social situations. If a robot would like to communicate a person, it may turn its gaze to him/her for eye contact. However, it is not an easy task for the robot to make eye contact because such a turning action alone may not be enough in all situations, especially when the robot and the human are not facing each other. In this paper, we present an attention control approach through robot's behaviors that can attract a person's attention by three actions: head turning, head shaking, and uttering reference terms corresponding to three viewing situations in which the human vision senses the robot (near peripheral field of view, far peripheral field of view, and out of field of view). After gaining attention, the robot makes eye contact through showing gaze awareness by blinking its eyes, and directs the human attention by eye and head turning behaviors to share an object.

A major challenge in HRI is to design a social robot that can attract a target human's attention to control his/her attention toward a particular direction in various social situations. If a robot would like to initiate an interaction with a person, it may turn its gaze to him/her for eye contact. However, it is not an easy task for the robot to make eye contact because such a turning action alone may not be enough to initiate an interaction in all situations, especially when the robot and the human are not facing each other or the human intensely attends to his/her task. In this paper, we propose a conceptual model of attention control with four phases: attention attraction, eye contact, attention avoidance, and attention shift. In order to initiate an attention control process, the robot first tries to gain the target participant's attention toward it through head turning, or head shaking action depending on the three viewing situations where the robot is captured in his/her field of view (central field of view, near peripheral field of view, and far peripheral field of view). After gaining her/his attention, the robot makes eye contact only with the target person through showing gaze awareness by blinking its eyes, and directs her/his attention toward an object by turning its eyes and head cues. Moreover, the robot can show attention to aversion behaviors if non-target persons look at it. We design a robot based on the proposed approach, and it is confirmed as effective to control the target participant's attention in experimental evaluation.

A social robot needs to attract the attention of a target human and shift it from his/her current focus to what is sought by the robot. The robot should recognize the current target's attention level to smoothly perform this attention control. In this paper, we propose a vision-based system to detect the level of attention or willingness of the target person towards the robot and to control his/her attention. The system estimates the attention level from rich visual cues of human's face and head. Then, by timing target's attention to determine the appropriate attention level, it generates aware signals and makes eye contact with the target. Finally, the robot shifts the target's attention to an intended direction. The experimental results reveal that the proposed system is effective in controlling the target's attention.

It is a major challenge in HRI to design a social robot that is able to direct a target human's attention towards an intended direction. For this purpose, the robot may first turn its gaze to him/her in order to establish eye contact. However, such a turning action of the robot may not in itself be sufficient to make eye contact with the target person in all situations, especially when the robot and the person are not facing each other or the human is intensely engaged in a task. In this paper, we propose a conceptual model of attention control with five phases: attention attraction, eye contact, attention avoidance, gaze back, and attention shift. We conducted two experiments to validate our model in human-robot interaction scenarios.

Object category detection with large appearance variation is a fundamental problem in computer vision. The appearance of object categories can change due to intra-class variability, viewpoint, and illumination. For object categories with large appearance change a sub-categorization based approach is necessary. This paper proposes a sub-category optimization approach that automatically divides an object category into an appropriate number of sub-categories based on appearance variation. Instead of using a predefined intra-category sub-categorization based on domain knowledge or validation datasets, we divide the sample space by unsupervised clustering based on discriminative image features. Then the clustering performance is verified using a sub-category discriminant analysis. Based on the clustering performance of the unsupervised approach and sub-category discriminant analysis results we determine an optimal number of sub-categories per object category. Extensive experimental results are shown using two standard and the authors' own databases. The comparison results show that our approach outperforms the state-of-the-art methods.