Current research related to human-robot interaction (HRI) of bipedal humanoid robots often assumes that the robot is in a standing stationary state, i.e., the relative position of the robot does not change, and rarely considers the effect of lower limb movement on interaction. However, HRI in the real world does not assume a moving or stationary state of the robot, and the equilibrium perturbations caused by movement can prevent HRI from functioning properly. In this paper, we propose a movement supported humanoid robot interaction method that empowers the robot to move stably while achieving HRI. First, a reinforcement learning-based neural network is run offline to generate interaction actions that satisfy the equilibrium constraint and support movement, and then an intention recognition network is introduced to run the movement-supported HRI framework online. It is demonstrated that the training method proposed in this paper can enable a bipedal robot to achieve a variety of interactive actions while moving stably.

Semantic image synthesis aims to synthesize photo-realistic images through the given semantic segmentation masks. Most existing models use conditional batch normalization (CBN) to regulate normalization activation by spatially varying modulation parameters. It can prevent semantic information from being eliminated during normalization. But the modulation parameters in CBN lack location constraint, resulting in the lack of structural information in the synthetic image. And CBN is highly dependent on the batch size. To address these limitations, we propose location aware conditional group normalization (LACGN) and construct a location aware generative adversarial network (LAGAN) based on this method. LACGN can learn spatial location aware information in a weakly supervised manner that relies on the current image synthesis process to guide transformations spatially. It allows the synthetic image to have more structural information and detailed features. At the same time, group normalization(GN) replace the traditional BN to eliminate the dependence on batch size. Extensive experiments show that LAGAN is better than other methods.

Although human pose estimation technology based on RGB images is becoming more and more mature, most of the current mainstream methods rely on depth camera to obtain human joints information. These interaction frameworks are affected by the infrared detection distance so that they cannot well adapt to the interaction scene of different distance. Therefore, the purpose of this paper is to build a modular interactive framework based on RGB images, which aims to alleviate the problem of high dependence on depth camera and low adaptability to distance in the current human-robot interaction (HRI) framework based on human body by using advanced human pose estimation technology. To enhance the adaptability of the HRI framework to different distances, we adopt optical cameras instead of depth cameras as acquisition equipment. Firstly, the human joints information is extracted by a human pose estimation network. Then, a joints sequence filter is designed in the intermediate stage to reduce the influence of unreasonable skeletons on the interaction results. Finally, a human intention recognition model is built to recognize the human intention from reasonable joints information, and drive the robot to respond according to the predicted intention. The experimental results show that our interactive framework is more robust in the distance than the framework based on depth camera and is able to achieve effective interaction under different distances, illuminations, costumes, customers, and scenes.

Deep reinforcement learning (DRL) has been widely used in the field of visual navigation. However, due to the lack of adaptability of DRL to the new tasks, the generalization ability of current visual navigation model using DRL is not desired. In order to improve this deficiency, we introduce the memory-prediction mechanism. By enhancing the memory of the scene, and combining the past experience of navigation to predict the next state, a more reasonable action can be obtained. First, we pass the image features extracted during the navigation process to an LSTM, and use LSTM to memorize the scene information in the image features. Then, we combine all the information (including state, target, and action) of each time step in the navigation process, and pass the historical information of multiple time steps to another LSTM to predict the next state. The action performed by the robot is determined by the predicted state. We use the AI2-THOR framework to carry out experiments. The results show that the proposed method can improve the navigation performance of the DRL visual navigation model and improve its adaptability to new tasks.

With the breakthrough performance in a variety of computer vision and medical image analysis problems, convolutional neural networks (CNNs) have been successfully introduced for the classification task of breast cancer histopathological images in recent years. Nevertheless, existing breast cancer histopathological image classification networks mainly utilize the first-order statistic information of deep features to represent histopathological images, failing to characterize the complex global feature distribution of breast cancer histopathological images. To address the problem, this work makes a first attempt to explore global second-order statistics of deep features for the above task. More specifically, we propose a novel deep second-order pooling network (DSoPN) for breast cancer histopatho-logical image classification, in which a robust global covariance pooling module based on matrix power normalization (MPN) is embedded into a simple yet effective CNN architecture. The given DSoPN model can capture richer second-order statistical information of deep convolutional features and produce more informative global representations for breast cancer histopatho-logical images. Experimental results on the public BreakHis dataset illuminate the promising performance of the second-order pooling for breast cancer histopathological image classification. Besides, our DSoPN achieves very competitive performance compared to the state-of-the-art methods.