Lifelong user behavior sequences are crucial for capturing user interests and predicting user responses in modern recommendation systems. A two-stage paradigm is typically adopted to handle these long sequences: a subset of relevant behaviors is first searched from the original long sequences via an attention mechanism in the first stage and then aggregated with the target item to construct a discriminative representation for prediction in the second stage. In this work, we identify and characterize, for the first time, a neglected deficiency in existing long-sequence recommendation models: a single set of embeddings struggles with learning both attention and representation, leading to interference between these two processes. Initial attempts to address this issue with some common methods (e.g., linear projections---a technique borrowed from language processing) proved ineffective, shedding light on the unique challenges of recommendation models. To overcome this, we propose the Decoupled Attention and Representation Embeddings (DARE) model, where two distinct embedding tables are initialized and learned separately to fully decouple attention and representation. Extensive experiments and analysis demonstrate that DARE provides more accurate searches of correlated behaviors and outperforms baselines with AUC gains up to 9‰ on public datasets and notable improvements on Tencent's advertising platform. Furthermore, decoupling embedding spaces allows us to reduce the attention embedding dimension and accelerate the search procedure by 50\% without significant performance impact, enabling more efficient, high-performance online serving. Code in PyTorch for experiments, including model analysis, is available at https://github.com/thuml/DARE.

Automatic object detection is a fundamental but challenging problem in very high-resolution (VHR) satellite images' interpretation. Recently, context-based feature pyramid detection architecture has been proposed, which generates high-quality fusion feature levels for multiscale object detection and significantly improves the accuracy of traditional detection frameworks. However, in feature pyramid architecture, small objects are easily lost in deep feature levels, and the context cues will be weakened simultaneously. Moreover, each feature level in the pyramids is simply constructed from single-level layers of the backbone, which are not representative enough for multiscale object detection. In this letter, an end-to-end multiscale convolutional neural network (MSCNN) is proposed, which is based on a unified multiscale backbone named EssNet for extracting features of diverse-scale objects in HRS images. The proposed EssNet backbone maintains the resolution of deep feature levels, which improves the capability of multiscale objects' feature expression. Meanwhile, a dilated bottleneck structure is introduced into the backbone, which generates high-quality semantic features and improves the prediction capability of multiscale objects. Finally, the whole network can be optimized end-to-end by minimizing a multitask loss. Experiments on publicly available NWPU VHR-10 benchmark and demonstrate that the proposed method outperforms several state-of-the-art detection approaches.

Automatic object detection is a basic but challenging problem in remote sensing images (RSIs) interpretation. Recently, a context-based top-down detection architecture has been proposed, which generates high-quality fusion features at all scales for object detection and significantly improves the accuracy of traditional detection framework. However, in the top-down architecture, small objects are easily lost in deep layers and the context cues will be weakened simultaneously. In this paper, to tackle these problems mentioned above, a novel Multi-scale Detection Network (MSDN) is proposed. The proposed method maintains the resolution of deep features, which enhances the capability of multi-scale objects feature expression. Meanwhile, a dilated bottleneck structure is introduced, which effectively enlarges the receptive filed and improves the regression ability of multi-scale objects. The proposed method is evaluated on NWPU VHR-10 benchmarks and achieves impressive improvement over the comparable state-of-the-art detection framworks.

Convolutional Neural Network (CNN) achieved satisfying performance in click-through rate (CTR) prediction in recent studies. Since features used in CTR prediction have no meaningful sequence in nature, the features can be arranged in any order. As CNN learns the local information of a sample, the feature sequence may influence its performance significantly. However, this problem has not been fully investigated. This paper firstly investigates whether and how the feature sequence affects the performance of the CNN-based CTR prediction method. As the data distribution of CTR prediction changes with time, the best current sequence may not be suitable for future data. Two multi-sequence models are proposed to learn the information provided by different sequences. The first model learns all sequences using a single feature learning module, while each sequence is learnt individually by a feature learning module in the second one. Moreover, a method of generating a set of embedding sequences which aims to consider the combined influence of all feature pairs on feature learning is also introduced. The experiments are conducted to demonstrate the effectiveness and stability of our proposed models in the offline and online environment on both the benchmark Avazu dataset and a real commercial dataset.