Multimodal embedding models have been widely adopted in text-toimage retrieval, enabling direct comparison between text and image modalities.However, how well they handle long text is poorly understood.For instance, Long-CLIP found that OpenAI's CLIP model, despite having a 77-token input limit, maintains optimal performance for only 20 tokens-its effective token length.In this paper, we build on the Long-CLIP study, and extend the analysis to other widely used multimodal models and find their effective token length.Unlike Long-CLIP, we examine how domain-specific language influences changes in effective token length and explore its implications on different domains.Based on our findings, we create a comprehensive reference of various models' effective token length across different domains; offering deeper insights into the true limitations of multimodal models used in text-to-image retrieval.Finally, we introduce a systematic benchmark that determines the effective token length of any multimodal model using a given dataset.Our results show that the effective token length is consistently lower than the input token limit for all models, meaning that these models cannot utilize all the text that can be given to them.We also find that the effective token length varies by dataset, with domain-specific language influencing how much text a model can use before retrieval performance plateaus.Our code is available for reproducibility at https://github.com/aiforsec/EffectiveTokenLength-MModels

Large Language Models (LLMs) have demonstrated potential in cybersecurity applications but have also caused lower confidence due to problems like hallucinations and a lack of truthfulness. Existing benchmarks provide general evaluations but do not sufficiently address the practical and applied aspects of LLM performance in cybersecurity-specific tasks. To address this gap, we introduce the SECURE (Security Extraction, Understanding & Reasoning Evaluation), a benchmark designed to assess LLMs performance in realistic cybersecurity scenarios. SECURE includes six datasets focused on the Industrial Control System sector to evaluate knowledge extraction, understanding, and reasoning based on industry-standard sources. Our study evaluates seven state-of-the-art models on these tasks, providing insights into their strengths and weaknesses in cybersecurity contexts. We also offer recommendations for improving LLMs reliability as cyber advisory tools and release our benchmark datasets and framework for community use at https://github.com/aiforsec/SECURE.

Exercise-induced fatigue resulting from physical activity can be an early indicator of overtraining, illness, or other health issues. In this paper, we present an automated method for estimating exercise-induced fatigue levels through the use of thermal imaging and facial analysis techniques utilizing deep learning models. Leveraging a novel dataset comprising over 400,000 thermal facial images of rested and fatigued users, our results suggest that exercise-induced fatigue levels could be predicted with only one static thermal frame with an average error smaller than 15%. The results emphasize the viability of using thermal imaging in conjunction with deep learning for reliable exercise-induced fatigue estimation.

Cyber threat intelligence (CTI) is crucial in today's cybersecurity landscape, providing essential insights to understand and mitigate the ever-evolving cyber threats. The recent rise of Large Language Models (LLMs) have shown potential in this domain, but concerns about their reliability, accuracy, and hallucinations persist. While existing benchmarks provide general evaluations of LLMs, there are no benchmarks that address the practical and applied aspects of CTI-specific tasks. To bridge this gap, we introduce CTIBench, a benchmark designed to assess LLMs' performance in CTI applications. CTIBench includes multiple datasets focused on evaluating knowledge acquired by LLMs in the cyber-threat landscape. Our evaluation of several state-of-the-art models on these tasks provides insights into their strengths and weaknesses in CTI contexts, contributing to a better understanding of LLM capabilities in CTI.

We developed a plugin for IntelliJ IDEA called AntiCopyPaster, which tracks the pasting of code fragments inside the IDE and suggests the appropriate Extract Method refactoring to combat the propagation of duplicates. Unlike the existing approaches, our tool is integrated with the developer’s workflow, and pro-actively recommends refactorings. Since not all code fragments need to be extracted, we develop a classification model to make this decision. When a developer copies and pastes a code fragment, the plugin searches for duplicates in the currently opened file, waits for a short period of time to allow the developer to edit the code, and finally inferences the refactoring decision based on a number of features.