In recent years, there has been increasing interest in using deep learning for binary analysis tasks. Particularly, Transformer-based pre-trained language models have attracted enormous attention and obtained encouraging results. Numerous research attempts modified the Transformer network architecture and designed new pre-training tasks explicitly tailored for individual downstream binary analysis tasks, and positive results were reported. However, it remains unclear whether these architectural changes and their associated pre-training tasks are beneficial to other downstream binary analysis tasks, and whether a vanilla Transformer model can perform equally well via fine-tuning.In order to provide guidance for future explorations in this direction, in this paper, we evaluate four custom Transformer-based models (i.e. jTrans, PalmTree, StateFormer, and Trex) and their pre-training tasks on four downstream applications. According to our evaluation results, we have the following surprising observations: aside from MLM (Masked Language Model), many existing pre-training tasks seem either too noisy or too challenging for the Transformer model to learn effectively; the vanilla BERT model is comparable or superior to these custom Transformers in all the four downstream applications. Moreover, our evaluation suggests that improvements in fine-tuning are generally more beneficial than introducing new pre-training tasks or making architectural modifications. Consequently, we conclude that recent architectural modifications and additional pre-training tasks for Transformer models may offer limited impact that does not sufficiently justify their associated costs.

Pedestrian Attribute Recognition (PAR) is an important task in surveillance video analysis. In this paper, we propose a novel end-to-end hierarchical deep learning approach to PAR. The proposed network introduces semantic segmentation into PAR and formulates it as a multi-task learning problem, which brings in pixel-level supervision in feature learning for attribute localization. According to the spatial properties of local and global attributes, we present a two stage learning mechanism to decouple coarse attribute localization and fine attribute recognition into successive phases within a single model, which strengthens feature learning. Besides, we design an attribute relationship attention module to efficiently capture and emphasize the latent relations among different attributes, further enhancing the discriminative power of the feature. Extensive experiments are conducted and very competitive results are reached on the RAP and PETA databases, indicating the effectiveness and superiority of the proposed approach.

Facial Attractiveness Prediction (FAP) is a useful yet challenging problem in the domain of computer vision. In this paper, we propose a deep learning based approach. Different from the existing deep methods, the proposed one models both the texture and shape clues within a multi-task learning framework consisting of attractiveness score prediction and fiducial landmark localization, thus highlighting both of their roles in assessing attractiveness of faces. Considering that the training data are not extensive, a lightweight CNN is designed to jointly learn the facial representation, landmark location, and facial attractiveness score. The proposed method is evaluated on the SCUT-FBP database, and a prediction correlation 0.92, is delivered, which shows the effectiveness of our method. Furthermore, two additional experiments in terms of comparison between facial images before and after make-up or beautification are conducted. The results also prove the advantage of the proposed method.