Transformer-based and interaction point-based methods have demonstrated promising performance and potential in human-object interaction detection. However, due to differences in structure and properties, direct integration of these two types of models is not feasible. Recent Transformer-based methods divide the decoder into two branches: an instance decoder for human-object pair detection and a classification decoder for interaction recognition. While the attention mechanism within the Transformer enhances the connection between localization and classification, this paper focuses on further improving HOI detection performance by increasing the intrinsic correlation between instance and action features. To address these challenges, this paper proposes a novel Transformer-based HOI Detection framework. In the proposed method, the decoder contains three parts: learnable query generator, instance decoder, and interaction classifier. The learnable query generator aims to build an effective query to guide the instance decoder and interaction classifier to learn more accurate instance and interaction features. These features are then applied to update the query generator for the next layer. Especially, inspired by the interaction point-based HOI and object detection methods, this paper introduces the prior bounding boxes, keypoints detection and spatial relation feature to build the novel learnable query generator. Finally, the proposed method is verified on HICO-DET and V-COCO datasets. The experimental results show that the proposed method has the better performance compared with the state-of-the-art methods.

Current methods for human-object interaction detection often use the spatial relation between a human and an object as an interaction pattern. However, this strategy is relatively simple and has low discrimination in similar interactions. To solve this drawback, the spatial relation between skeletons and objects is proposed to model the interaction pattern and improve the detection accuracy. First, the skeleton-object interaction pattern image is extracted for each interaction proposal. Second, a deep neural network is applied to learn the interaction features from these images. Finally, the interaction feature is added to the human-object interaction detection network by a multistream structure. In the experiments, we evaluate the proposed method on the HICO-DET and V-COCO datasets. Experimental results show that the proposed method can achieve the best performance compared with state-of-art methods.

Abstract Generally, it is regarded as challenge work to grasp the drawing style of an ancient masterpiece in Chinese painting learning. This paper presents a novel approach to the generation of a Chinese ink painting in a certain style and animating its brushwork process with expert skills. In order to demonstrate the techniques of brush and ink inside a stroke, a serials of geometric properties of a brush stroke, are first extracted, then through rational deformation calculation, the best stroke source is mapped onto the stroking path, which is sketched by the user, and finally a new Chinese painting can be synthesized by style migration and natural stroke composition. So with the generated strokes, the lifelike brushwork process of the new painting can be represented dramatically. Actually, by showing the authentic painting process, the tool we implemented helps the learners, who have no profound skills and knowledge in domain of Chinese painting, master the essence of a great painting style, and also provides an easy way to art creation and the comprehension of mysterious Chinese traditional art.