This paper explores how to enhance existing masked time-series modeling by randomly dropping sub-sequence level patches of time series. On this basis, a simple yet effective method named DropPatch is proposed, which has two remarkable advantages: 1) It improves the pre-training efficiency by a square-level advantage; 2) It provides additional advantages for modeling in scenarios such as in-domain, cross-domain, few-shot learning and cold start. This paper conducts comprehensive experiments to verify the effectiveness of the method and analyze its internal mechanism. Empirically, DropPatch strengthens the attention mechanism, reduces information redundancy and serves as an efficient means of data augmentation. Theoretically, it is proved that DropPatch slows down the rate at which the Transformer representations collapse into the rank-1 linear subspace by randomly dropping patches, thus optimizing the quality of the learned representations.

As online transactions rapidly increase, money laundering has become more difficult to detect, rendering traditional rule-based algorithms inadequate for the current severe laundering landscape. Although efforts have been made to model user behavior sequences for detecting money laundering, these approaches still fall short in scenarios with extremely low anomaly rates. In our anti-money laundering practices, we have identified the following three challenges: weak perception of intensity, scarce labels, poor representation robustness. In this paper, we present CLeAR, a novel robust sequence-based self-supervised Representation Learning framework for Anti-Money Laundering. To address the weak perception of intensity, we devise an Intensity-Aware Transformer to better capture the nuances of user behavior sequences. By introducing sequence-based Contrastive Learning into this task, we effectively tackle the issue of scarce labels and enhance sequence modeling. Additionally, we developed two self-supervised learning tasks-next behavior matching and sub-sequence matching-that significantly enhance the overall robustness of representation. After rigorous experiments across datasets of various scales, CLeAR consistently delivers exceptional performance, even under the extremely low anomaly rates that closely mimic real-world conditions.

Urban public nuisance events, like garbage exposure, illegal parking, facilities damage, and etc., impair the quality of life for city residents. Predicting and controlling these nuisances is crucial but complicated due to their ties to subjective and psychological factors. In this study, we reveal a significant correlation between such nuisances and meteorological indicators, influenced by the impact of climate on people's psychological states. We employ meteorology predictions that are integrated in Hawkes processes to enhance the accuracy of predicting the category and timing of these nuisances. To this end, we propose Spatial-Temporal Two-Tower Transformer (ST-T3), which simultaneously considers spatial data and further improves the prediction accuracy. Evaluated by about three-year data from both downtown and suburban Shanghai, our method outperforms both traditional and advanced prediction systems. We share a portion of the de-identified dataset for open research.