Encoder transformer models compress information from all tokens in a sequence into a single [CLS] token to represent global context.This approach risks diluting fine-grained or hierarchical features, leading to information loss in downstream tasks where local patterns are important.To remedy this, we propose a lightweight architectural enhancement: an inception-style 1-D convolution module that sits on top of the transformer layer and augments token representations with multi-scale local features.This enriched feature space is then processed by a self-attention layer that dynamically weights tokens based on their task relevance.Experiments on five diverse tasks show that our framework consistently improves general-purpose, domain-specific, and multilingual models, outperforming baselines by 1% to 14% while maintaining efficiency.Ablation studies show that multi-scale convolution performs better than any single kernel and that the self-attention layer is critical for performance.

Identifying vulnerabilities in complex 5G network protocols is a challenging task. Manual analysis is time-consuming and often inadequate. Modern ML and NLP methods, though effective, are resource-intensive and struggle to find implicit vulnerabilities. In this research, we utilize GPT-4’s advanced language understanding to detect vulnerabilities directly from 5G specifications. To assess GPT-4’s fundamental capabilities in this domain, we first adopt a zero-shot approach that relies solely on the specification text without external guidance. For detecting more sophisticated vulnerabilities that require deep contextual understanding, we introduce a novel domain-aware strategy, where we explicitly teach GPT-4 about security properties and hazard indicators from related works using few-shot learning. We further employ chain-of-thought prompting to guide the model through structured reasoning steps to identify violations or exploitations that may lead to vulnerabilities. A two-tier filtering process ensures that only promising test-cases are retained. Our method has identified 47 potential vulnerabilities in 5G mobility management procedures, including 27 previously unreported issues, and generated corresponding test-cases. Simulating 14 of them, we have found 9 vulnerabilities, five of which are new. The zero-shot approach is effective in detecting procedural and validation flaws, while the domain-aware method excels in finding protocol violations and advanced attack scenarios. These findings validate our methodology and demonstrate its strength in discovering both known and novel vulnerabilities in 5G protocols.

Coronavirus disease 2019 (COVID-19), an infectious disease caused by the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), has been declared a global pandemic by the World Health Organization, and the situation worsens daily, associated with acute increases in case fatality rates. The main protease (Mpro) enzyme produced by SARS-CoV-2 was recently demonstrated to be responsible for not only viral reproduction but also impeding host immune responses. The element selenium (Se) plays a vital role in immune functions, both directly and indirectly. Thus, we hypothesised that Se-containing heterocyclic compounds might curb the activity of SARS-CoV-2 Mpro. We performed a molecular docking analysis and found that several of the selected selenocompounds showed potential binding affinities for SARS-CoV-2 Mpro, especially ethaselen (49), which exhibited a docking score of -6.7 kcal/mol compared with the -6.5 kcal/mol score for GC376 (positive control). Drug-likeness calculations suggested that these compounds are biologically active and possess the characteristics of ideal drug candidates. Based on the binding affinity and drug-likeness results, we selected the 16 most effective selenocompounds as potential anti-COVID-19 drug candidates. We also validated the structural integrity and stability of the drug candidate through molecular dynamics simulation. Using further in vitro and in vivo experiments, we believe that the targeted compound identified in this study (ethaselen) could pave the way for the development of prospective drugs to combat SARS-CoV-2 infections and trigger specific host immune responses.