Discriminative recommendation tasks, such as CTR (click-through rate) and CVR (conversion rate) prediction, play critical roles in the ranking stage of large-scale industrial recommender systems. However, training a discriminative model encounters a significant overfitting issue induced by data sparsity. Moreover, this overfitting issue worsens with larger models, causing them to underperform smaller ones. To address the overfitting issue and enhance model scalability, we propose a framework named GPSD (Generative Pretraining for Scalable Discriminative Recommendation), drawing inspiration from generative training, which exhibits no evident signs of overfitting. GPSD leverages the parameters learned from a pretrained generative model to initialize a discriminative model, and subsequently applies a sparse parameter freezing strategy. Extensive experiments conducted on both industrial-scale and publicly available datasets demonstrate the superior performance of GPSD. Moreover, it delivers remarkable improvements in online A/B tests. GPSD offers two primary advantages: 1) it substantially narrows the generalization gap in model training, resulting in better test performance; and 2) it leverages the scalability of Transformers, delivering consistent performance gains as models are scaled up. Specifically, we observe consistent performance improvements as the model dense parameters scale from 13K to 0.3B, closely adhering to power laws. These findings pave the way for unifying the architectures of recommendation models and language models, enabling the direct application of techniques well-established in large language models to recommendation models.

Traditional satellite-to-vessel (S2V) models struggle to handle wave movement and random obstacles in complex maritime environments. These models are designed for urban scenarios and fail to account for the unique dynamics of sea surface fluctuations, obstacle distribution, and satellite posture, limiting their applicability in maritime communications. We propose a novel empirical model that integrates multiple maritime environmental factors to overcome these challenges. A realistic virtual sea surface scenario is constructed by extracting oceanic geometric features and statistically modelling waves and obstacles. A large volume of line-of-sight (LoS) data is obtained through ray tracing (RT) simulations. A multi-input graph convolutional network (GCN) is trained for parameter estimation, and a new complexity parameter is introduced to characterize environmental impact. Simulation results demonstrate that the proposed method achieves higher prediction accuracy in maritime communication scenarios and significantly outperforms baseline models. Even with simplified inputs, the model performs well in urban scenarios, demonstrating strong generalization and adaptability.

Vision-and-language pretraining (VLP) aims to learn generic multimodal representations from massive image-text pairs. While various successful attempts have been proposed, learning fine-grained semantic alignments between image-text pairs plays a key role in their approaches. Nevertheless, most existing VLP approaches have not fully utilized the intrinsic knowledge within the image-text pairs, which limits the effectiveness of the learned alignments and further restricts the performance of their models. To this end, we introduce a new VLP method called ROSITA, which integrates the cross- and intra-modal knowledge in a unified scene graph to enhance the semantic alignments. Specifically, we introduce a novel structural knowledge masking (SKM) strategy to use the scene graph structure as a priori to perform masked language (region) modeling, which enhances the semantic alignments by eliminating the interference information within and across modalities. Extensive ablation studies and comprehensive analysis verifies the effectiveness of ROSITA in semantic alignments. Pretrained with both in-domain and out-of-domain datasets, ROSITA significantly outperforms existing state-of-the-art VLP methods on three typical vision-and-language tasks over six benchmark datasets.

The emerging Internet of Things (IoT) applications, such as smart manufacturing and smart home, lead to a huge demand on the provisioning of low-cost and high-accuracy positioning and navigation solutions. Inertial measurement unit (IMU) can provide an accurate inertial navigation solution in a short time but its positioning error increases fast with time due to the cumulative error of accelerometer measurement. On the other hand, ultrawideband (UWB) positioning and navigation accuracy will be affected by the actual environment and may lead to uncertain jumps even under line-of-sight (LOS) conditions. Therefore, it is hard to use a standalone positioning and navigation system to achieve high accuracy in indoor environments. In this article, we propose an integrated indoor positioning system (IPS) combining IMU and UWB through the extended Kalman filter (EKF) and unscented Kalman filter (UKF) to improve the robustness and accuracy. We also discuss the relationship between the geometric distribution of the base stations (BSs) and the dilution of precision (DOP) to reasonably deploy the BSs. The simulation results show that the prior information provided by IMU can significantly suppress the observation error of UWB. It is also shown that the integrated positioning and navigation accuracy of IPS significantly improves that of the least squares (LSs) algorithm, which only depends on UWB measurements. Moreover, the proposed algorithm has high computational efficiency and can realize real-time computation on general embedded devices. In addition, two random motion approximation model algorithms are proposed and evaluated in the real environment. The experimental results show that the two algorithms can achieve certain robustness and continuous tracking ability in the actual IPS.

Existing approaches to neural machine translation are typically autoregressive models.While these models attain state-of-the-art translation quality, they are suffering from low parallelizability and thus slow at decoding long sequences.In this paper, we propose a novel model for fast sequence generationthe semi-autoregressive Transformer (SAT).The SAT keeps the autoregressive property in global but relieves in local and thus are able to produce multiple successive words in parallel at each time step.Experiments conducted on English-German and Chinese-English translation tasks show that the SAT achieves a good balance between translation quality and decoding speed.On WMT'14 English-German translation, the SAT achieves 5.58× speedup while maintaining 88% translation quality, significantly better than the previous non-autoregressive methods.When produces two words at each time step, the SAT is almost lossless (only 1% degeneration in BLEU score).

Character-based sequence labeling framework is flexible and efficient for Chinese word segmentation (CWS). Recently, many character-based neural models have been applied to CWS. While they obtain good performance, they have two obvious weaknesses. The first is that they heavily rely on manually designed bigram feature, i.e. they are not good at capturing n-gram features automatically. The second is that they make no use of full word information. For the first weakness, we propose a convolutional neural model, which is able to capture rich n-gram features without any feature engineering. For the second one, we propose an effective approach to integrate the proposed model with word embeddings. We evaluate the model on two benchmark datasets: PKU and MSR. Without any feature engineering, the model obtains competitive performance — 95.7% on PKU and 97.3% on MSR. Armed with word embeddings, the model achieves state-of-the-art performance on both datasets — 96.5% on PKU and 98.0% on MSR, without using any external labeled resource.