In recent years, Transformers have been widely applied in human pose estimation by converting image features into token forms as inputs. However, redundant information in images burdens the network and can even negatively impact training as noise. Thus, we propose a coarse-to-fine Transformer called CFPose for efficient human pose estimation. We first extract visual features through a backbone network, then remove redundancy and coarsely crop the human figure via a coarse-grained decision network. Coarse-Grained token (CG token) and keypoint token are fed into a two-stage Transformer, where after the CG encoder, keypoint token sufficiently incorporate coarse features. Only keypoint token and Fine-Grained token (FG token) from a fine-grained decision network that further segments features are input to the FG encoder for training. Finally, keypoint token are mapped to 2D heatmaps for keypoint prediction. Impressively, CFPose reduces computational complexity by 43% while improving accuracy to 76.2% on COCO. It also achieves competitive results on MPII.

Abstract Human keypoints detection is different from general detection tasks and requires networks that can learn visual information and anatomical constraints. Since CNN is excellent in extracting texture features of images and transformer can learn the correlation among keypoints well, many CTPNets (CNN+transformer type human pose estimation networks) have emerged. However, these networks are unconcerned with the processing of the features extracted from the CNN and naturally expand only from the channel dimension, ignoring the spatial features in the visual information that are essential for complex detection tasks like pose estimation. So the channel spatial integrated transformer for human pose estimation, termed CSIT, is proposed. The visual information are summarized as texture and spatial information, and a parallel network is used to expand the feature maps in the channel and spatial dimensions to learn texture features and spatial features respectively. In addition, anatomically constrained information is learned by keypoint embeddings. At the end of the network, the 1D vector representation method with more advanced performance and more compatible with transformer's characteristics is used to predict keypoints. Experiments show that CSIT outperforms the mainstream CTPNets on the COCO test‐dev dataset, and also show satisfactory results on the MPII dataset.

In this paper, a multi-hypothesis square-root cubature Kalman particle filter (MH-SRCKPF) is proposed for speaker tracking in noisy and reverberant environments with distributed microphone arrays. The conventional cubature Kalman particle filter (CKPF) uses the cubature Kalman filter (CKF) to generate its proposal for particle sampling. Such a proposal incorporates only one observation from a certain localization function for the state estimation, which is vulnerable to noise or reverberation, yielding the degraded tracking performance. To tackle the problem, by incorporating multiple possible observations into CKF for the proposal, a multi-hypothesis CKPF (MH-CKPF) algorithm is first developed. Furthermore, to improve the numerical stability, an MH-SRCKPF algorithm is developed, where the state estimate and the square root of the error covariance are propagated at each time. Finally, the MH-SRCKPF is applied to the speaker tracking problems in distributed microphone arrays. Experimental results demonstrate that the proposed MH-SRCKPF outperforms the competing methods in the presence of noise and reverberation. Meanwhile, by propagating the square root of the state covariance, the proposed method exhibits attractive numerical characteristics.

Multiview video coding (MVC) plays an important role in 3D video system, while the huge computational complexity blocks its applications. This paper proposes an adaptive Inter mode decision algorithm to reduce the complexity of MVC. First, the selection of Inter modes is determined by using the textural region type of macro block (MB). Then, the estimation of small size Inter modes (Inter16×8, Inter8×16, and Inter8×8) is decided based on the motion homogenization of MB, which is predicted by utilizing the motion estimation results of Inter16×16 mode. Finally, the complexity of Inter8×8 mode estimation is progressively reduced by employing rate-distortion (RD) costs of estimated modes. As compared to the full mode decision in MVC reference software, the proposed algorithm achieved 71% encoding time saving on average with 0.026 dB peak signal-to-noise ratio loss and 0.74% bit rate increase.