As fine-tuning becomes the dominant paradigm for improving large language models (LLMs), understanding what changes during this process is increasingly important.Traditional benchmarking often fails to explain why one model outperforms another.In this work, we use model diffing, a mechanistic interpretability approach, to analyze the specific capability differences between Gemma-2-9b-it and a SimPO-enhanced variant.Using crosscoders, we identify and categorize latent representations that differentiate the two models.We find that SimPO acquired latent concepts predominantly enhance safety mechanisms (+32.8%),multilingual capabilities (+43.8%), and instruction-following (+151.7%),while its additional training also reduces emphasis on model self-reference (-44.1%) and hallucination management (-68.5%).Our analysis shows that model diffing can yield fine-grained insights beyond leaderboard metrics, attributing performance gaps to concrete mechanistic capabilities.This approach offers a transparent and targeted framework for comparing LLMs.

We introduce a novel approach for mapping attack behaviors described in threat analysis reports to entries in an adversarial techniques knowledge base. Our method leverages a multi-stage ranking architecture to efficiently rank the most related techniques based on their semantic relevance to the input text. Each ranker in our pipeline uses a distinct design for text representation. To enhance relevance modeling, we leverage pretrained language models, which we fine-tune for the technique annotation task. While generic large language models are not yet capable of fully addressing this challenge, we obtain very promising results. We achieve a recall rate improvement of +35% compared to the previous state-of-the-art results. We further create new public benchmark datasets for training and validating methods in this domain, which we release to the research community aiming to promote future research in this important direction.

Ahmed Abdelali, Hamdy Mubarak, Shammur Chowdhury, Maram Hasanain, Basel Mousi, Sabri Boughorbel, Samir Abdaljalil, Yassine El Kheir, Daniel Izham, Fahim Dalvi, Majd Hawasly, Nizi Nazar, Youssef Elshahawy, Ahmed Ali, Nadir Durrani, Natasa Milic-Frayling, Firoj Alam. Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers). 2024.

Effective modeling of patient representation from electronic health records (EHRs) is increasingly becoming a vital research topic. Yet, modeling the non-stationarity in EHR data has received less attention. Most existing studies follow a strong assumption of stationarity in patient representation from EHRs. However, in practice, a patient's visits are irregularly spaced over a relatively long period of time, and disease progression patterns exhibit non-stationarity. Furthermore, the time gaps between patient visits often encapsulate significant domain knowledge, potentially revealing undiscovered patterns that characterize specific medical conditions. To address these challenges, we introduce a new method which combines the self-attention mechanism with non-stationary kernel approximation to capture both contextual information and temporal relationships between patient visits in EHRs. To assess the effectiveness of our proposed approach, we use two real-world EHR datasets, comprising a total of 76,925 patients, for the task of predicting the next diagnosis code for a patient, given their EHR history. The first dataset is a general EHR cohort and consists of 11,451 patients with a total of 3,485 unique diagnosis codes. The second dataset is a disease-specific cohort that includes 65,474 pregnant patients and encompasses a total of 9,782 unique diagnosis codes. Our experimental evaluation involved nine prediction models, categorized into three distinct groups. Group 1 comprises the baselines: original self-attention with positional encoding model, RETAIN model, and LSTM model. Group 2 includes models employing self-attention with stationary kernel approximations, specifically incorporating three variations of Bochner's feature maps. Lastly, Group 3 consists of models utilizing self-attention with non-stationary kernel approximations, including quadratic, cubic, and bi-quadratic polynomials. The experimental results demonstrate that non-stationary kernels significantly outperformed baseline methods for NDCG@10 and Hit@10 metrics in both datasets. The performance boost was more substantial in dataset 1 for the NDCG@10 metric. On the other hand, stationary Kernels showed significant but smaller gains over baselines and were nearly as effective as Non-stationary Kernels for Hit@10 in dataset 2. These findings robustly validate the efficacy of employing non-stationary kernels for temporal modeling of EHR data, and emphasize the importance of modeling non-stationary temporal information in healthcare prediction tasks.

The activity of babies and more specifically the posture of babies is an important aspect in their safety and development. In this paper, we studied the automatic classification of baby posture using a pressure-sensitive mat. The posture classification problem is formulated as the design of features that describe the pressure patterns induced by the child in combination with generic classifiers. Novel rotation invariant features constructed from high order statistics obtained from the concentric rings around the center of gravity. Non-constant ring radii are used in order to ensure uniform cell areas and therefore equal importance of features. A vote fusion of various generic classifiers is used for classification. Temporal information was shown to improve the classification performance. The obtained results are promising and open new opportunities for applications and further research in the area of baby safety and development.

In this paper we address the problem of eye localization in low and standard definition content, such as webcam-generated and TV images. We present a probabilistic eye localization method based on well-known multiscale local binary patterns (LBPs), which provide a simple and powerful spatial description of texture, and are robust to the noise typical to low and standard definition content. Our primary contribution is in the proposed method of combining the LBPs that is targeted towards achieving spatial accuracy under mentioned conditions. Evaluation performed on a standard dataset of webcam-quality images shows that our approach has superior performance with respect to the state of the art, while having a reasonable complexity and a low memory footprint.

This technical demo is build around a system we developed to automatically match persons within a video sequence. The persons are clustered by face similarity and shot information. The demo is a video editing tool with a "person query" option to show only that part of the sequence that features one of the selected actors. The demo is targeted for narrative content: series and movies, and shows the ability of the underlying person matching algorithm to operate well in a general (unconstrained) case.

Histogram intersection (HI) kernel has been recently introduced for image recognition tasks. The HI kernel is proved to be positive definite and thus can be used in support vector machine (SVM) based recognition. Experimentally, it also leads to good recognition performances. However, its derivation applies only for binary strings such as color histograms computed on equally sized images. In this paper, we propose a new kernel, which we named generalized histogram intersection (GHI) kernel, since it applies in a much larger variety of contexts. First, an original derivation of the positive definiteness of the GHI kernel is proposed in the general case. As a consequence, vectors of real values can be used, and the images no longer need to have the same size. Second, a hyper-parameter is added, compared to the HI kernel, which allows us to better tune the kernel model to particular databases. We present experiments which prove that the GHI kernel outperforms the simple HI kernel in a simple recognition task. Comparisons with other well-known kernels are also provided.

Kernel based methods such as support vector machine (SVM) has provided successful tools for solving many recognition problems. One of the reasons of this success is the use of kernels. Positive definiteness has to be checked for kernels to be suitable for most of these methods. For instance for SVM, the use of a positive definite kernel insures that the optimized problem is convex and thus the obtained solution is unique. Alternative class of kernels called conditionally positive definite have been studied for a long time from the theoretical point of view and have drawn attention from the community only in the last decade. We propose a new kernel, named log kernel, which seems particularly interesting for images. Moreover, we prove that this new kernel is a conditionally positive definite kernel as well as the power kernel. Finally, we show from experimentations that using conditionally positive definite kernels allows us to outperform classical positive definite kernels

We introduce the intermediate matching (IM) kernel for SVM-hased object recognition. The IM kernel operates on a feature space of vector sets where each image is represented by a set of local features. Matching algorithms have proved to be efficient for such types of features. Nevertheless, kernelizing the matching for SVM does not lead to positive definite kernels. The IM kernel overcomes this drawback, as it mimics matching algorithms while being positive definite. The IM kernel introduces an intermediary set of so-called virtual local features. These select the pairs of local features to be matched. Comparisons with the matching kernel shows that the IM kernels leads to similar performances.

On the one hand, Support Vector Machines have met with significant success in solving difficult pattern recognition problems with global features representation. On the other hand, local features in images have shown to be suitable representations for efficient object recognition. Therefore, it is natural to try to combine SVM approach with local features representation to gain advantages on both sides. We study in this paper the Mercer property of matching kernels which mimic classical matching algorithms used in techniques based on points of interest. We introduce a new statistical approach of kernel positiveness. We show that despite the absence of an analytical proof of the Mercer property, we can provide bounds on the probability that the Gram matrix is actually positive definite for kernels in large class of functions, under reasonable assumptions. A few experiments validate those on object recognition tasks.

In image retrieval systems, a variety of simple similarity measures are used. The choice for one similarity measure or another is generally driven by an experimental comparison on a labeled database. The drawback of such an approach is that, while a large number of possible similarity measures can be tested, we do not know how to extend from the obtained results. However, the choice of a good similarity measure leads to noticeable better results. It is known that this choice is related to the variability of the images within the same class. Therefore, we propose a model of image retrieval systems and deduce a scheme for deriving the best similarity measure in a set of similarity measures, assuming a parametric model of the variability of feature vectors within the same class. An experimental validation of the model and the derived similarity measures is performed on synthetic ground-truth databases. Finally, from our experiments, we give several rules to follow for the design of ground-truth databases allowing reliable conclusions on the search of better similarity measures.