It is a common paradigm in object detection frameworks to perform multi-scale detection. However, each scale is treated equally during training. In this paper, we carefully study the objective imbalance of multi-scale detector training. We argue that the loss in each scale is neither equally important nor independent. Different from the existing solutions of setting fixed multi-task weights, we dynamically optimize the loss weight of each scale in the training process. Specifically, we propose an Adaptive Variance Weighting (AVW) to balance multi-scale loss according to the statistical variance. Then we develop a novel Reinforcement Learning Optimization (RLO) to decide the weighting scheme probabilistically during training. The proposed dynamic methods make better utilization of multi-scale training loss without extra computational complexity and learnable parameters for backpropagation. Experiments on Pascal VOC and MS COCO benchmark validate the effectiveness of our proposed methods.

Feature Pyramid Network (FPN) has become a common detection paradigm by improving multi-scale features with strong semantics. However, most FPN-based methods typically treat each feature map equally and sum the loss without distinction, which might lead to suboptimal overall performance. In this paper, we propose a Multi-scale Reinforcement Learning Strategy (MRLS) for balanced multi-scale training. First, we design Dynamic Feature Fusion (DFF) to dynamically magnify the impact of more important feature maps in FPN. Second, we introduce Compensatory Scale Training (CST) to enhance the supervision of the under-training scale. We regard the whole detector as a reinforcement learning system while the state bases on multi-scale loss. And we develop the corresponding action, reward, and policy. Compared with adding more rich model architectures, MRLS would not add any extra modules and computational burdens on the baselines. Experiments on MS COCO and PASCAL VOC benchmark demonstrate that our method significantly improves the performance of commonly used object detectors.

Modern efficient Convolutional Neural Networks(CNNs) always use Depthwise Separable Convolutions(DSCs) and Neural Architecture Search(NAS) to reduce the number of parameters and the computational complexity. But some inherent characteristics of networks are overlooked. Inspired by visualizing feature maps and N×N(N>1) convolution kernels, several guidelines are introduced in this paper to further improve parameter efficiency and inference speed. Based on these guidelines, our parameter-efficient CNN architecture, called VGNetG, achieves better accuracy and lower latency than previous networks with about 30%~50% parameters reduction. Our VGNetG-1.0MP achieves 67.7% top-1 accuracy with 0.99M parameters and 69.2% top-1 accuracy with 1.14M parameters on ImageNet classification dataset. Furthermore, we demonstrate that edge detectors can replace learnable depthwise convolution layers to mix features by replacing the N×N kernels with fixed edge detection ker-nels. And our VGNetF-1.5MP archives 64.4%(-3.2%) top-1 accuracy and 66.2%(-1.4%) top-1 accuracy with additional Gaussian kernels.

Extensive neurophysiological studies have shown that the receptive field plays a significant role in the human visual system. It has various kinds of properties such as orientation-selectivity, correlativity, etc. Motivated by these structural and functional properties, we propose in this paper a novel local image descriptor namely the Discriminative Transform of Receptive Fields (DTRF). Specifically, Receptive Field Patterns (RFP) are defined around each sample pixel and then divided into two kinds of components: RFP-Surround and RFP-Center. The RFP-Surround serves as the basic feature structure, which is extracted based on Local Annular Discrete Cosine Transform (LADCT) algorithm. The RFP-Center is used to pool these local features to simulate the correlative property of receptive field. Experimental results on the standard Oxford data set demonstrate the superiority of DTR-F over the state-of-the-art descriptors under various types of image transformations such as rotation and scaling changes, viewpoint changes, image blurring, JPEG compression, illumination changes, and image noise.

Accurate head pose estimation is significant for many applications such as face recognition and human-computer interaction. In this paper, we treat the head pose estimation as a classification problem and employ the Lie Algebrized Gaussians (LAG) feature as the representation approach for head image. The LAG feature, which is built on Gausssian Mixture Model (GMM), has the capability to preserve the structure of Gaussian components in the original Lie group manifold. Moreover, to keep more spatial structure information of the image, LAG is operated on many subregions of the image. As a result, these properties of LAG enable it to reflect the pose characteristic of the head image well and possess powerful discriminative ability in pose classification. Experiments on CMU Pose, Illumination, and Expression (PIE) and Pointing'04 benchmarks show state-of-the-art performance and demonstrate that LAG represents the head pose characteristic well.