Deploying language models often requires navigating accuracy vs. performance trade-offs to meet latency constraints while preserving utility. Traditional model distillation reduces size but incurs substantial costs through training separate models. We introduce ModularStarEncoder (MoSE), a 1-billion-parameter multi-exit encoder for code retrieval and classification that employs a novel Self-Distillation mechanism. This approach significantly enhances lower-layer representations, enabling flexible deployment of different model portions with favorable performance trade-offs. Our architecture improves text-to-code and code-to-code search by targeting specific encoder layers as exit heads, where higher layers guide earlier ones during training, thereby improving intermediate representations at minimal additional cost. We further enhance MoSE with a repository-level contextual loss that maximizes training context window utilization. Additionally, we release a new dataset created through code translation that extends text-to code benchmarks with cross-language code-to-code pairs. Evaluations demonstrate the effectiveness of Self-Distillation as a principled approach to trading inference cost for accuracy across various code understanding tasks.

This paper introduces Trajectory-Embedded Matryoshka Representation Learning (TE-MRL).This novel framework synergies the capabilities of trajectory representation learning with the adaptability and efficiency of Matryoshka Representation Learning (MRL).TE-MRL is engineered to generate adaptive, multi-granular embeddings that efficiently capture the spatial-temporal dynamics inherent in trajectory data.We evaluate TE-MRL on the Porto dataset, focusing on trajectory similarity and k-nearest trajectory similarity tasks.Our findings demonstrate that TE-MRL preserves critical features such as travel semantics and temporal regularities while it can significantly reduce computational time and memory footprint.The proposed approach matches existing methods' accuracy and efficiency but demonstrates robust adaptability under varying computational constraints.Furthermore, we proposed a two-stage retrieval pipeline to enhance computational time while maintaining the same precision.We reduced the computation time by 8× while maintaining state-of-the-art precision.The effectiveness of TE-MRL in handling the complexity of the Porto dataset underlines its potential for broader applications in urban computing and mobility analytics.

With the increasing popularity of LLM-based code completers, like GitHub Copilot, the interest in automatically detecting AI-generated code is also increasing-in particular in contexts where the use of LLMs to program is forbidden by policy due to security, intellectual property, or ethical concerns. We introduce a novel technique for AI code stylometry, i.e., the ability to distinguish code generated by LLMs from code written by humans, based on a transformer-based encoder classifier. Differently from previous work, our classifier is capable of detecting AI-written code across 10 different programming languages with a single machine learning model, maintaining high average accuracy across all languages (84.1% ± 3.8%). Together with the classifier we also release H-AIRosettaMP, a novel open dataset for AI code stylometry tasks, consisting of 121 247 code snippets in 10 popular programming languages, labeled as either human-written or AI-generated. The experimental pipeline (dataset, training code, resulting models) is the first fully reproducible one for the AI code stylometry task. Most notably our experiments rely only on open LLMs, rather than on proprietary/closed ones like ChatGPT.