Many works have been proposed to automatically recognize facial human emotions, yet distinguishing subtle emotions still remains a challenge. However, just amplifying these facial movements as a whole does not accurately reflect the actual expressions as action intensity is different for each facial part. We propose an attributionaware amplification of facial descriptors that considers gender, facial components, and emotions to alleviate this issue. The amplifications are also continuously updated in an evolutionary manner during training using Population Based Augmentation (PBA). We found that applying different amplifications for each facial region/unit is crucial for the success of our method, outperforming the competing approaches on both the RAVDESS and THAI-SER datasets.

Set prediction tasks require the matching between predicted set and ground truth set in order to propagate the gradient signal. Recent works have performed this matching in the original feature space thus requiring predefined distance functions. We propose a method for learning the distance function by performing the matching in the latent space learned from encoding networks. This method enables the use of teacher forcing which was not possible previously since matching in the feature space must be computed after the entire output sequence is generated. Nonetheless, a naive implementation of latent set prediction might not converge due to permutation instability. To address this problem, we provide sufficient conditions for permutation stability which begets an algorithm to improve the overall model convergence. Experiments on several set prediction tasks, including image captioning and object detection, demonstrate the effectiveness of our method.