Knowledge Distillation (KD) serves as an effective approach to addressing heterogeneity issues in Federated Learning (FL), leveraging additional datasets to align local and global models better. There are two primary distillation paradigms: feature-based distillation, which utilizes intermediate-layer features of the network, and logit-based distillation, which employs the final layer's logit outputs. However, existing studies often select distillation methods based on intuitive and empirical evidence when facing different heterogeneous settings, neglecting the intrinsic relationship between distillation paradigms and heterogeneity. This oversight may result in suboptimal federated knowledge distillation performance under heterogeneous conditions. In this paper, we propose the Consolidated Distillation for Heterogeneous Federated Learning - FedCD that balances knowledge representations from both feature-based and logit-based distillation to enhance performance. Specifically, to address the misalignment between knowledge conveyed by features and logits, we aggregate features from different layers via cross-layer attention to preserve semantic knowledge, followed by distribution modeling using Gaussian Mixture Models. This process strengthens knowledge distillation by constraining the transformation of different network layers' features under a consolidated distribution, thereby mitigating impacts from both data and model heterogeneity. Extensive experiments demonstrate that FedCD outperforms state-of-the-art methods by over 10.72% and validate the effectiveness of our approach.

Data-Centric Sequential Recommendation (DaCSR) has emerged as a promising technique that enhances dataset quality to better capture user preferences without increasing training complexity. However, mining item relations to improve data quality remains challenging due to the intricate nature of interaction sequences. Existing methods predominantly either: 1) optimize models to learn such item relations from fixed datasets at significant training cost, or 2) employ generative models to adaptively learn only interaction patterns, which lack interpretability and cannot guarantee effective data quality enhancement. In this paper, we pioneer a relation-guided dataset augmentation and regeneration framework for sequential recommendation called \textbf{RaSR}. This framework can significantly improve model performance on original datasets while maintaining training efficiency without modifying the model architecture. Specifically, we first preprocess user interactions to construct standardized sequential data and extract semantic representations via a Large Language Model (LLM). We then build a multi-relation graph with manually predefined metrics and semantic representations to generate augmented datasets. Finally, a relation-aware generator can produce regenerated datasets with both the multi-relation graph and the augmented dataset. To verify the effectiveness of RaSR, we conduct experiments on various backbone models and datasets, and achieve significant performance improvement compared to training the model only on the original dataset.

Adapting Large Multimodal Models (LMMs) to real-world scenarios poses the dual challenges of learning from sequential data streams while handling frequent modality incompleteness, a task known as Continual Missing Modality Learning (CMML). However, existing works on CMML have predominantly relied on prompt tuning, a technique that struggles with this task due to cross-task interference between its learnable prompts in their shared embedding space. A naive application of Low-Rank Adaptation (LoRA) with modality-shared module will also suffer modality interference from competing gradients. To this end, we propose DeLo, the first framework to leverage a novel dual-decomposed low-rank expert architecture for CMML. Specifically, this architecture resolves modality interference through decomposed LoRA expert, dynamically composing LoRA update matrix with rank-one factors from disentangled modality-specific factor pools. Embedded within a task-partitioned framework that structurally prevents catastrophic forgetting, this expert system is supported by two key mechanisms: a Cross-Modal Guided Routing strategy to handle incomplete data and a Task-Key Memory for efficient, task-agnostic inference. Extensive experiments on established CMML benchmarks demonstrate that our method significantly outperforms state-of-the-art approaches. This highlights the value of a principled, architecturally-aware LoRA design for real-world multimodal challenges.

The deployment of large, black-box foundation models for medical image classification is often hindered by the high cost of acquiring large, task-specific labeled datasets for fine-tuning. While active learning (AL) presents a promising solution, many state-of-the-art AL methods are computationally expensive or require full access to internal model parameters. We present VALIANT (Visual Adaptation and Learning Integration for Active learNing Tasks), a new active learning framework designed to efficiently adapt black-box foundation models by overcoming these limitations. VALIANT introduces a lightweight Visual Prompt Decoder (VIPD), trained via unsupervised Zero-Order Optimization (ZOO), to generate task-specific visual prompts without internal model access. Our core contribution is a perturbation-based ranking strategy that leverages this VIPD to formulate a computationally efficient, gradient-aware informativeness metric. This metric, which we term prompt instability, identifies the most impactful samples for the labeling budget. VALIANT further enhances this process by incorporating anatomical information from unsupervised segmentation maps to generate more discriminative visual prompts. Extensive evaluations on multiple medical datasets demonstrate VALIANT’s superior performance and significant reduction in labeling costs compared to a range of existing active learning techniques, positioning it as a scalable and practical solution for medical image analysis.

Ghadir Alselwi, Hao Xue, Shoaib Jameel, Basem Suleiman, Flora D. Salim, Imran Razzak. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2026.

Sign language is the primary communication mode for 72 million hearing-impaired individuals worldwide, necessitating effective bidirectional Sign Language Production and Sign Language Translation systems. However, functional bidirectional systems require a unified linguistic environment, hindered by the lack of suitable unified datasets, particularly those providing the necessary pose information for accurate Sign Language Production (SLP) evaluation. Concurrently, current SLP evaluation methods like back-translation ignore pose accuracy, and high-quality coordinated generation remains challenging. To create this crucial environment and overcome these challenges, we introduce CNText2Sign and CNSign, which together constitute the first unified dataset aimed at supporting bidirectional accessibility systems for Chinese sign language; CNText2Sign provides 15,000 natural language-to-sign mappings and standardized skeletal keypoints for 8,643 vocabulary items supporting pose assessment. Building upon this foundation, we propose the AuraLLM model, which leverages a decoupled architecture with CNText2Sign's pose data for novel direct gesture accuracy assessment. The model employs retrieval augmentation and Cascading Vocabulary Resolution to handle semantic mapping and out-of-vocabulary words, and achieves all-scenario production with controllable coordination of gestures and facial expressions via pose-conditioned video synthesis. Concurrently, our Sign Language Translation model SignMST-C employs targeted self-supervised pretraining for dynamic feature capture, achieving new SOTA results on PHOENIX2014-T with BLEU-4 scores up to 32.08. AuraLLM establishes a strong performance baseline on CNText2Sign with a BLEU-4 score of 50.41 under direct evaluation.

Explaining the decision-making process of Graph Neural Networks (GNNs) is essential for improving their transparency and reliability. However, real-world graphs are often heterogeneous and subject to structural noise, posing severe challenges to the robustness of existing explanation methods. To address these issues, we propose RoHeX, a Robust Heterogeneous GNN Explainer that enhances explanation quality under noisy conditions. RoHeX begins with a theoretical analysis revealing how different heterogeneous GNN architectures amplify structural perturbations through message passing. Building on this insight, we design a denoising variational inference framework that filters noisy structures and learns robust latent graph representations. Furthermore, we incorporate relation-aware heterogeneous semantics into the explanation generation process, formulating explanation as an optimization problem under the graph information bottleneck principle. This formulation enables RoHeX to balance fidelity and compactness, producing explanations that are both semantically meaningful and structurally stable. Comprehensive experiments on multiple real-world heterogeneous graphs demonstrate that RoHeX consistently surpasses state-of-the-art baselines in explanation fidelity, robustness to structural perturbations, and explainability.

Retrieval-augmented generation (RAG) has improved large language models (LLMs) on knowledge-intensive tasks, yet most systems assume static facts and struggle when answers depend on serialized and dynamic data, like time--e.g., ordering events, aligning facts to valid intervals, or planning actions under evolving conditions. This paper presents Knowledge-Graph Iterative Retrieval-Augmented Generation (KG-iRAG), a framework specialized for temporal reasoning. KG-iRAG couples a time-aware planner with a knowledge graph (KG) to iteratively fetch and compose evidence along a temporal axis. Concretely, it (i) represents events and facts with explicit timestamps and validity intervals; (ii) propagates temporal constraints through iterative retrieval using operators; and (iii) verifies temporal consistency while refining intermediate hypotheses, enabling step-by-step deduction for queries that mix knowledge retrieval with inference. Across public temporal QA benchmarks, KG-iRAG consistently improves accuracy and calibration over strong RAG baselines while reducing unnecessary retrieval through targeted, constraint-guided steps. To stress-test real-time decision queries, three application-oriented datasets (weatherQA-Irish, ~ weatherQA-Sydney, and~ trafficQA-TFNSW) are additionally constructed and tested alongside existing temporal benchmarks. The results demonstrate that injecting temporal structure into KG-driven RAG yields robust gains on multi-step, time-dependent queries, advancing the state of temporal reasoning with LLMs.

Single-cell RNA sequencing (scRNA-seq) enables high-resolution characterization of cellular heterogeneity, but its rich, complementary structure across cells and genes remains underexploited, especially in the presence of technical noise and sparsity. Effectively leveraging this multi-scale structure is essentially an information fusion problem that requires integrating heterogeneous graph-based views of cells and genes into robust low-dimensional representations. In this paper, we introduce GatorSC, a unified representation learning framework that models scRNA-seq data through multi-scale cell and gene graphs and fuses them with a mixture-of-experts architecture. GatorSC constructs a global cell-cell graph, a global gene-gene graph, and a local gene-gene graph derived from neighborhood-specific subgraphs, and learns graph neural network embeddings that are adaptively fused by a gating network. To learn noise-robust and structure-preserving embeddings without labels, we couple graph reconstruction and graph contrastive learning in a unified self-supervised objective applied to both cell- and gene-level graphs. We evaluate GatorSC on 19 publicly available scRNA-seq datasets covering diverse tissues, species, and sequencing platforms. Experiments showed that GatorSC consistently outperforms state-of-the-art deep generative, graph-based, and contrastive methods for cell clustering, gene expression imputation, and cell-type annotation. The learned embeddings are used for accurate trajectory inference, recovery of canonical marker gene programs, and cell-type-specific pathway signatures in an Alzheimer's disease single-nucleus dataset. GatorSC provides a flexible foundation for comprehensive single-cell transcriptomic analysis and can be readily extended to multi-omic and spatial modalities.

The availability of large, high-quality annotated datasets in the medical domain poses a substantial challenge in segmentation tasks. To mitigate the reliance on annotated training data, self-supervised pre-training strategies have emerged, particularly employing contrastive learning methods on dense pixel-level representations. In this work, we proposed to capitalize on intrinsic anatomical similarities within medical image data and develop a semantic segmentation framework through a self-supervised fusion network, where the availability of annotated volumes is limited. In a unified training phase, we combine segmentation loss with contrastive loss, enhancing the distinction between significant anatomical regions that adhere to the available annotations. To further improve the segmentation performance, we introduce an efficient parallel transformer module that leverages Multiview multiscale feature fusion and depth-wise features. The proposed transformer architecture, based on multiple encoders, is trained in a self-supervised manner using contrastive loss. Initially, the transformer is trained using an unlabeled dataset. We then fine-tune one encoder using data from the first stage and another encoder using a small set of annotated segmentation masks. These encoder features are subsequently concatenated for the purpose of brain tumor segmentation. The multiencoder-based transformer model yields significantly better outcomes across three medical image segmentation tasks. We validated our proposed solution by fusing images across diverse medical image segmentation challenge datasets, demonstrating its efficacy by outperforming state-of-the-art methodologies.

The metaverse, which integrates physical and virtual realities through technologies such as high-speed internet, virtual and augmented reality, and artificial intelligence (AI), offers transformative prospects across various fields, particularly healthcare. This integration introduces a new paradigm in AI-driven medical imaging, particularly in assessing brain age-a crucial marker for detecting age-related neuropathologies such as Alzheimer's disease (AD) using magnetic resonance imaging (MRI). Despite advances in deep learning for estimating brain age from structural MRI (sMRI), incorporating functional MRI (fMRI) data presents significant challenges due to its complex data structure and the noisy nature of functional connectivity measurements. To address these challenges, we present the Multitask Adversarial Variational Autoencoder (M-AVAE), a bespoke deep learning framework designed to enhance brain age predictions through multimodal MRI data integration. The M-AVAE uniquely separates latent variables into generic and unique codes, effectively isolating shared and modality-specific features. Additionally, integrating multitask learning with sex classification as a supplementary task enables the model to account for sex-specific aging nuances. Evaluated on the OpenBHB dataset-a comprehensive multisite brain MRI aggregation-the M-AVAE demonstrates exceptional performance, achieving a mean absolute error of 2.77 years, surpassing conventional methodologies. This success positions M-AVAE as a powerful tool for metaverse-based healthcare applications in brain age estimation.

Abstract Language models often underperform when fine tuned with limited labelled data. To reduce dependence on costly annotation, recent studies have explored clustering and pseudo labelling to leverage unlabeled text with only a few labelled examples for fine tuning. However, these methods face persistent challenges: clustering errors and pseudo-label mismatches can degrade fine-tuning performance. We propose Cluster-SCP , a novel framework that improves intra-cluster coherence and reduces cluster quantity to generate more reliable pseudo labels. The framework begins with K-Means initialization and applies an iterative refinement process, Intermediate Pseudo Labels , comprising two stages: embedding reassignment to minimize clustering errors, and cluster merging via contrastive learning with graph readouts. The refined pseudo labels are first used to fine tune language model, which is subsequently refined using a small set of labels. Experiments on three benchmark datasets show that Cluster-SCP consistently outperforms baseline and state-of-the-art methods, achieving at least a 5.3% accuracy improvement while reducing the number of clusters by 22%.

Recent multimodal large language models (MLLMs) have demonstrated significant potential in open-ended conversation, generating more accurate and personalized responses. However, their abilities to memorize, recall, and reason in sustained interactions within real-world scenarios remain underexplored. This paper introduces MMRC, a Multi-Modal Real-world Conversation benchmark for evaluating six core open-ended abilities of MLLMs: information extraction, multi-turn reasoning, information update, image management, memory recall, and answer refusal. With data collected from real-world scenarios, MMRC comprises 5,120 conversations and 28,720 corresponding manually labeled questions, posing a significant challenge to existing MLLMs. Evaluations on 20 MLLMs in MMRC indicate an accuracy drop during open-ended interactions. We identify four common failure patterns: long-term memory degradation, inadequacies in updating factual knowledge, accumulated assumption of error propagation, and reluctance to “say no.” To mitigate these issues, we propose a simple yet effective NOTE-TAKING strategy, which can record key information from the conversation and remind the model during its responses, enhancing conversational capabilities. Experiments across six MLLMs demonstrate significant performance improvements.

Long-sequence causal reasoning seeks to uncover causal relationships within extended time series data but is hindered by complex dependencies and the challenges of validating causal links. To address the limitations of large-scale language models (e.g., GPT-4) in capturing intricate emotional causality within extended dialogues, we propose CauseMotion, an innovative framework combining emotional causal dynamic mapping with multimodal feature fusion. CauseMotion implements dynamic mapping through a sliding window mechanism and fusion strategies, while integrating audio features—vocal emotion, intensity, and speech rate—to enrich semantic representations. This design enables efficient retrieval of contextually relevant information and precise inference of emotional causal chains spanning multiple conversational turns. We constructed the first benchmark dataset for long-sequence emotional causal reasoning, featuring dialogues with over 70 turns. Experimental results show that CauseMotion significantly enhances emotional understanding and causal inference capabilities in large language models. A GLM-4 integrated with CauseMotion achieves an 8.7% improvement in causal accuracy over the original model and surpasses GPT-4o by 1.2%. On the DiaASQ dataset, CauseMotion-GLM-4 achieves state-of-the-art results in accuracy, F1 score, and causal reasoning accuracy.

Cardiovascular disease (CVD) prediction remains a tremendous challenge due to its multifactorial etiology and global burden of morbidity and mortality. Despite the growing availability of genomic and electrophysiological data, extracting biologically meaningful insights from such high-dimensional, noisy, and sparsely annotated datasets remains a non-trivial task. Recently, LLMs has been applied effectively to predict structural variations in biological sequences. In this work, we explore the potential of fine-tuned LLMs to predict cardiac diseases and SNPs potentially leading to CVD risk using genetic markers derived from high-throughput genomic profiling. We investigate the effect of genetic patterns associated with cardiac conditions and evaluate how LLMs can learn latent biological relationships from structured and semi-structured genomic data obtained by mapping genetic aspects that are inherited from the family tree. By framing the problem as a Chain of Thought (CoT) reasoning task, the models are prompted to generate disease labels and articulate informed clinical deductions across diverse patient profiles and phenotypes. The findings highlight the promise of LLMs in contributing to early detection, risk assessment, and ultimately, the advancement of personalized medicine in cardiac care.

Segment Anything Model (SAM) demonstrates powerful zero-shot capabilities; however, its accuracy and robustness significantly decrease when applied to medical image segmentation. Existing methods address this issue through modality fusion, integrating textual and image information to provide more detailed priors. In this study, we argue that the granularity of text and the domain gap affect the accuracy of the priors. Furthermore, the discrepancy between high-level abstract semantics and pixel-level boundary details in images can introduce noise into the fusion process. To address this, we propose Prior-Guided SAM (PG-SAM), which employs a fine-grained modality prior aligner to leverage specialized medical knowledge for better modality alignment. The core of our method lies in efficiently addressing the domain gap with fine-grained text from a medical large language model (LLM). Meanwhile, it also enhances the priors' quality after modality alignment, ensuring more accurate segmentation. In addition, our decoder enhances the model's expressive capabilities through multi-level feature fusion and iterative mask optimizer operations, supporting unprompted learning. We also propose a unified pipeline that effectively supplies high-quality semantic information to SAM. Extensive experiments on the datasets demonstrate that the proposed PG-SAM achieves state-of-the-art performance. Our anonymous code is released at https://github.com/logan-0623/PG-SAM.

The integration of conversational agents into daily life has become increasingly common. However, sustaining deeply engaging and natural interactions remains challenging due to a lack of multimodal datasets capturing personal and emotional nuances. In this paper, we introduce MERCI (Multimodal dataset for Emotionally-aware peRsonalised Conversational In-teractions), a dataset derived from user-robot dialogues involving thirty participants who completed user profile questionnaires covering ten personal topics (e.g., hobbies, music). A conver-sational system called PERCY then engaged with each partici-pant in open-domain conversations, leveraging GPT-4, real-time facial-expression and sentiment analysis to generate contextu-ally appropriate, empathetic responses. MERCI contains 1860 utterances, equating to about 12.5 hours of aligned audio, three-view video, transcripts with timestamps, emotion labels, and sentiment scores. This dataset serves as a reproducible test-bed for tasks such as emotion-aware response generation, multimodal affect recognition, and personalised policy learning. Baseline performance results have been established using advanced models such as BERT, T5, BART, and GPT-3.5/4/4o-mini across gener-ation, regression, and classification. Evaluations through human and automated methods have demonstrated strong naturalness, relevance, and consistency in responses while indicating areas for enhanced personalisation and empathic depth. We expect that MERCI will enhance the development of emotionally intelligent, user-centric conversational AI applications, potentially ranging from social robotics to mental health support.

Personality detection from text is commonly performed by analysing users' social media posts. However, existing methods heavily rely on large-scale annotated datasets, making it challenging to obtain high-quality personality labels. Moreover, most studies treat emotion and personality as independent variables, overlooking their interactions. In this paper, we propose a novel self-supervised framework, EmoPerso, which improves personality detection through emotion-aware modelling. EmoPerso first leverages generative mechanisms for synthetic data augmentation and rich representation learning. It then extracts pseudo-labeled emotion features and jointly optimizes them with personality prediction via multi-task learning. A cross-attention module is employed to capture fine-grained interactions between personality traits and the inferred emotional representations. To further refine relational reasoning, EmoPerso adopts a self-taught strategy to enhance the model's reasoning capabilities iteratively. Extensive experiments on two benchmark datasets demonstrate that EmoPerso surpasses state-of-the-art models. The source code is available at https://github.com/slz0925/EmoPerso.

NLP models often face challenges with under-represented languages due to a lack of sufficient training data and language complexities. This can result in inaccurate predictions and a failure to capture the inherent uncertainties within these languages. This paper introduces a new method for modelling uncertainty in under-represented languages by employing deep Bayesian Gaussian Processes. We develop a novel framework that integrates prior knowledge and leverages kernel functions. This helps enable the quantification of uncertainty in predictions to overcome the data limitations in under-represented languages. The efficacy of our approach is validated through various experiments, and the results are benchmarked against existing methods to highlight the enhancements in prediction accuracy and measurement of uncertainty.

Multi-level Hierarchical Classification (MLHC) tackles the challenge of categorizing items within a complex, multi-layered class structure. However, traditional MLHC classifiers often rely on a backbone model with n independent output layers, which tend to ignore the hierarchical relationships between classes. This oversight can lead to inconsistent predictions that violate the underlying taxonomy. Leveraging Large Language Models (LLMs), we propose novel taxonomy-embedded transitional LLM-agnostic framework for multimodality classification. The cornerstone of this advancement is the ability of models to enforce consistency across hierarchical levels. Our evaluations on the MEP-3M dataset - a Multi-modal E-commerce Product dataset with various hierarchical levels- demonstrated a significant performance improvement compared to conventional LLMs structure.

Recent advancements in multimodal large language models (MLLMs) have significantly improved performance in visual question answering. However, they often suffer from hallucinations. In this work, hallucinations are categorized into two main types: initial hallucinations and snowball hallucinations. We argue that adequate contextual information can be extracted directly from the token interaction process. Inspired by causal inference in the decoding strategy, we propose to leverage causal masks to establish information propagation between multimodal tokens. The hypothesis is that insufficient interaction between those tokens may lead the model to rely on outlier tokens, overlooking dense and rich contextual cues. Therefore, we propose to intervene in the propagation process by tackling outlier tokens to enhance in-context inference. With this goal, we present FarSight, a versatile plug-and-play decoding strategy to reduce attention interference from outlier tokens merely by optimizing the causal mask. The heart of our method is effective token propagation. We design an attention register structure within the upper triangular matrix of the causal mask, dynamically allocating attention to capture attention diverted to outlier tokens. Moreover, a positional awareness encoding method with a diminishing masking rate is proposed, allowing the model to attend to further preceding tokens, especially for video sequence tasks. With extensive experiments, FarSight demonstrates significant hallucination-mitigating performance across different MLLMs on both image and video benchmarks, proving its effectiveness.

Medical decision-making often involves integrating knowledge from multiple clinical specialties, typically achieved through multidisciplinary teams.Inspired by this collaborative process, recent work has leveraged large language models (LLMs) in multi-agent collaboration frameworks to emulate expert teamwork.While these approaches improve reasoning through agent interaction, they are limited by static, pre-assigned roles, which hinder adaptability and dynamic knowledge integration.To address these limitations, we propose KAMAC, a Knowledge-driven Adaptive Multi-Agent Collaboration framework that enables LLM agents to dynamically form and expand expert teams based on the evolving diagnostic context.KAMAC begins with one or more expert agents and then conducts a knowledge-driven discussion to identify and fill knowledge gaps by recruiting additional specialists as needed.This supports flexible, scalable collaboration in complex clinical scenarios, with decisions finalized through reviewing updated agent comments.Experiments on two real-world medical benchmarks demonstrate that KAMAC significantly outperforms both single-agent and advanced multi-agent methods, particularly in complex clinical scenarios (i.e., cancer prognosis) requiring dynamic, cross-specialty expertise.

Graph-based personality detection constructs graph structures from textual data, particularly social media posts. Current methods often struggle with sparse or noisy data and rely on static graphs, limiting their ability to capture dynamic changes between nodes and relationships. This paper introduces LL4G, a self-supervised framework leveraging large language models (LLMs) to optimize graph neural networks (GNNs). LLMs extract rich semantic features to generate node representations and to infer explicit and implicit relationships. The graph structure adaptively adds nodes and edges based on input data, continuously optimizing itself. The GNN then uses these optimized representations for joint training on node reconstruction, edge prediction, and contrastive learning tasks. This integration of semantic and structural information generates robust personality profiles. Experimental results on Kaggle and Pandora datasets show LL4G outperforms state-of-the-art models.

Rip currents are a major hazard on beaches worldwide, and their strong, offshore-directed currents can place even experienced beachgoers at risk of drowning. While it is intuitive to consider developing an automated rip current detection system to assist lifeguards in protecting beachgoers, rip current detection is in its infancy due to the lack of high-quality large-scale annotated rip current datasets. Also, the collection and annotation of rip current images require expert knowledge, which makes it more difficult to build datasets. So, this paper proposes a GAN-based rip current data augmentation method, RipGAN, to improve the performance of rip current detectors by increasing representative training data. To create new training images, RipGAN, has two branches. One is a texture generator that enriches the pattern and texture details of waves, making the image more realistic. The other is a rip generator based on FFFM-Unet. FFFM (Fast Fourier Fusion Module) uses Fast Fourier convolution to fuse the features from the low and the high layers, so as to further optimise the generated image. Furthermore, we trained Yolov8, YOLOv10, DINO and RT-DETR as rip current detectors to prove the effectiveness of RipGAN. The detectors' rnAP50:95improved by 2.67% on the test set and AP50by 4.93% on real-scene videos, outperforming other data augmentation methods. Besides, abundant ablation studies have been conducted to further evaluate each component of RipGAN.

Personality detection automatically identifies an individual’s personality from various data sources, such as social media texts. However, as the parameter scale of language models continues to grow, the computational cost becomes increasingly difficult to manage. Fine-tuning also grows more complex, making it harder to justify the effort and reliably predict outcomes. We introduce a novel parameter-efficient fine-tuning framework, PersLLM, to address these challenges. In PersLLM, a large language model (LLM) extracts high-dimensional representations from raw data and stores them in a dynamic memory layer. PersLLM then updates the downstream layers with a replaceable output network, enabling flexible adaptation to various personality detection scenarios. By storing the features in the memory layer, we eliminate the need for repeated complex computations by the LLM. Meanwhile, the lightweight output network serves as a proxy for evaluating the overall effectiveness of the framework, improving the predictability of results. Experimental results on key benchmark datasets like Kaggle and Pandora show that PersLLM significantly reduces computational cost while maintaining competitive performance and strong adaptability.

Humans understand visual scenes by first capturing a global impression and then refining this understanding into distinct, object-like components. Inspired by this process, we introduce \textbf{C}oncept-\textbf{G}uided \textbf{S}elf-\textbf{S}upervised \textbf{L}earning (CG-SSL), a novel framework that brings structure and interpretability to representation learning through a curriculum of three training phases: (1) global scene encoding, (2) discovery of visual concepts via tokenised cross-attention, and (3) alignment of these concepts across views. Unlike traditional SSL methods, which simply enforce similarity between multiple augmented views of the same image, CG-SSL accounts for the fact that these views may highlight different parts of an object or scene. To address this, our method establishes explicit correspondences between views and aligns the representations of meaningful image regions. At its core, CG-SSL augments standard SSL with a lightweight decoder that learns and refines concept tokens via cross-attention with patch features. The concept tokens are trained using masked concept distillation and a feature-space reconstruction objective. A final alignment stage enforces view consistency by geometrically matching concept regions under heavy augmentation, enabling more compact, robust, and disentangled representations of scene regions. Across multiple backbone sizes, CG-SSL achieves state-of-the-art results on image segmentation benchmarks using $k$-NN and linear probes, substantially outperforming prior methods and approaching, or even surpassing, the performance of leading SSL models trained on over $100\times$ more data. Code and pretrained models will be released.

Despite significant advances in inference-time search for vision–language models (VLMs), existing approaches remain both computationally expensive and prone to unpenalized, low-confidence generations which often lead to persistent hallucinations. We introduce \textbf{Value-guided Inference with Margin-based Reward (ViMaR)}, a two-stage inference framework that improves both efficiency and output fidelity by combining a temporal-difference value model with a margin-aware reward adjustment. In the first stage, we perform a single pass to identify the highest-value caption among diverse candidates. In the second stage, we selectively refine only those segments that were overlooked or exhibit weak visual grounding, thereby eliminating frequently rewarded evaluations. A calibrated margin-based penalty discourages low-confidence continuations while preserving descriptive richness. Extensive experiments across multiple VLM architectures demonstrate that ViMaR generates captions that are significantly more reliable, factually accurate, detailed, and explanatory, while achieving over 4$\times$ speedup compared to existing value-guided methods. Specifically, we show that ViMaR trained solely on LLaVA Mistral-7B \textit{generalizes effectively to guide decoding in stronger unseen models}. To further validate this, we adapt ViMaR to steer generation in both LLaVA-OneVision-Qwen2-7B and Qwen2.5-VL-3B, leading to consistent improvements in caption quality and demonstrating robust cross-model guidance. This cross-model generalization highlights ViMaR's flexibility and modularity, positioning it as a scalable and transferable inference-time decoding strategy. Furthermore, when ViMaR-generated captions are used for self-training, the underlying models achieve substantial gains across a broad suite of visual comprehension benchmarks, underscoring the potential of fast, accurate, and self-improving VLM pipelines. Code: https://github.com/ankan8145/ViMaR

Accurate 3D reconstruction of hands and instruments is critical for vision-based analysis of ophthalmic microsurgery, yet progress has been hampered by the lack of realistic, large-scale datasets and reliable annotation tools. In this work, we introduce OphNet-3D, the first extensive RGB-D dynamic 3D reconstruction dataset for ophthalmic surgery, comprising 41 sequences from 40 surgeons and totaling 7.1 million frames, with fine-grained annotations of 12 surgical phases, 10 instrument categories, dense MANO hand meshes, and full 6-DoF instrument poses. To scalably produce high-fidelity labels, we design a multi-stage automatic annotation pipeline that integrates multi-view data observation, data-driven motion prior with cross-view geometric consistency and biomechanical constraints, along with a combination of collision-aware interaction constraints for instrument interactions. Building upon OphNet-3D, we establish two challenging benchmarks—bimanual hand pose estimation and hand–instrument interaction reconstruction—and propose two dedicated architectures: H-Net for dual-hand mesh recovery and OH-Net for joint reconstruction of two-hand–two-instrument interactions. These models leverage a novel spatial reasoning module with weak-perspective camera modeling and collision-aware center-based representation. Both architectures outperform existing methods by substantial margins, achieving improvements of over 2mm in Mean Per Joint Position Error (MPJPE) and up to 23\% in ADD-S metrics for hand and instrument reconstruction, respectively.

Causal mediation analysis is crucial for deconstructing complex mechanisms of action. However, in current mediation analysis, complex structures derived from causal discovery lack direct interpretation of mediation pathways, while traditional mediation analysis and effect estimation are limited by the reliance on pre-specified pathways, leading to a disconnection between structure discovery and causal mechanism understanding. Therefore, a unified framework integrating structure discovery, pathway identification, and effect estimation systematically quantifies mediation pathways under structural uncertainty, enabling automated identification and inference of mediation pathways. To this end, we propose Structure-Informed Guided Mediation Analysis (SIGMA), which guides automated mediation pathway identification through probabilistic causal structure discovery and uncertainty quantification, enabling end-to-end propagation of structural uncertainty from structure learning to effect estimation. Specifically, SIGMA employs differentiable Flow-Structural Equation Models to learn structural posteriors, generating diverse Directed Acyclic Graphs (DAGs) to quantify structural uncertainty. Based on these DAGs, we introduce the Path Stability Score to evaluate the marginal probability of pathways, identifying high-confidence mediation paths. For identified mediation pathways, we integrate Efficient Influence Functions with Bayesian model averaging to fuse within-structure estimation uncertainty and between-structure effect variation, propagating uncertainty to the final effect estimates. In synthetic data experiments, SIGMA achieves state-of-the-art performance in pathway identification accuracy and effect quantification precision under structures uncertainty, concurrent multiple pathways, and nonlinear scenarios. In real-world applications using Human Phenotype Project data, SIGMA identifies mediation effects of sleep quality on cardiovascular health through inflammatory and metabolic pathways, uncovering previously unspecified multiple mediation paths.

Cultural awareness in language models is the capacity to understand and adapt to diverse cultural contexts. However, most existing approaches treat culture as static background knowledge, overlooking its dynamic and evolving nature. This limitation reduces their reliability in downstream tasks that demand genuine cultural sensitivity. In this work, we introduce CALM, a novel framework designed to endow language models with cultural self-awareness. CALM disentangles task semantics from explicit cultural concepts and latent cultural signals, shaping them into structured cultural clusters through contrastive learning. These clusters are then aligned via cross-attention to establish fine-grained interactions among related cultural features and are adaptively integrated through a Mixture-of-Experts mechanism along culture-specific dimensions. The resulting unified representation is fused with the model's original knowledge to construct a culturally grounded internal identity state, which is further enhanced through self-prompted reflective learning, enabling continual adaptation and self-correction. Extensive experiments conducted on multiple cross-cultural benchmark datasets demonstrate that CALM consistently outperforms state-of-the-art methods.

Despite the impressive progress of recent pretraining methods on multimodal tasks, existing methods are inherently biased towards either spatial modeling (e.g., CLIP) or temporal modeling (e.g., V-JEPA), limiting their joint capture of spatial details and temporal dynamics. To this end, we propose UniViT, a cluster-driven unified self-supervised learning framework that effectively captures the structured semantics of both image spatial content and video temporal dynamics through event-level and object-level clustering and discrimination. Specifically, we leverage offline clustering to generate semantic clusters across both modalities. For videos, multi-granularity event-level clustering progressively expands from single-event to structured multi-event segments, capturing coarse-to-fine temporal semantics; for images, object-level clustering captures fine-grained spatial semantics. However, while global clustering provides semantically consistent clusters, it lacks modeling of structured semantic relations (e.g., temporal event structures). To address this, we introduce a contrastive objective that leverages these semantic clusters as pseudo-label supervision to explicitly enforce structural constraints, including temporal event relations and spatial object co-occurrences, capturing structured semantics beyond categories. Meanwhile, UniViT jointly embeds structured object-level and event-level semantics into a unified representation space. Furthermore, UniViT introduces two key components: (i) Unified Rotary Position Embedding integrates relative positional embedding with frequency-aware dimension allocation to support position-invariant semantic learning and enhance the stability of structured semantics in the discrimination stage; and (ii) Variable Spatiotemporal Streams adapt to inputs of varying frame lengths, addressing the rigidity of conventional fixed-input approaches. Extensive experiments across varying model scales demonstrate that UniViT achieves state-of-the-art performance on linear probing, attentive probing, question answering, and spatial understanding tasks.

Enabling multiple autonomous machines to perform reliably requires the development of efficient cooperative control algorithms. This paper presents a survey of algorithms that have been developed for controlling and coordinating autonomous machines in complex environments. We especially focus on task allocation methods using computational intelligence (CI) and deep reinforcement learning (RL). The advantages and disadvantages of the surveyed methods are analysed thoroughly. We also propose and discuss in detail various future research directions that shed light on how to improve existing algorithms or create new methods to enhance the employability and performance of autonomous machines in real-world applications. The findings indicate that CI and deep RL methods provide viable approaches to addressing complex task allocation problems in dynamic and uncertain environments. The recent development of deep RL has greatly contributed to the literature on controlling and coordinating autonomous machines, and it has become a growing trend in this area. It is envisaged that this paper will provide researchers and engineers with a comprehensive overview of progress in machine learning research related to autonomous machines. It also highlights underexplored areas, identifies emerging methodologies, and suggests new avenues for exploration in future research within this domain.

The increasing demand for autonomous systems in complex and dynamic environments has driven significant research into intelligent path planning methodologies. For decades, graph-based search algorithms, linear programming techniques, and evolutionary computation methods have served as foundational approaches in this domain. Recently, deep reinforcement learning (DRL) has emerged as a powerful method for enabling autonomous agents to learn optimal navigation strategies through interaction with their environments. This survey provides a comprehensive overview of traditional approaches as well as the recent advancements in DRL applied to path planning tasks, focusing on autonomous vehicles, drones, and robotic platforms. Key algorithms across both conventional and learning-based paradigms are categorized, with their innovations and practical implementations highlighted. This is followed by a thorough discussion of their respective strengths and limitations in terms of computational efficiency, scalability, adaptability, and robustness. The survey concludes by identifying key open challenges and outlining promising avenues for future research. Special attention is given to hybrid approaches that integrate DRL with classical planning techniques to leverage the benefits of both learning-based adaptability and deterministic reliability, offering promising directions for robust and resilient autonomous navigation.

In critical domains such as healthcare and law, accurately modelling the uncertainty of automatic computational models is essential. For instance, healthcare models must produce reliable estimates to guide human decision-making. However, modelling uncertainty remains challenging, particularly for models handling low-resource datasets and complex, domain-specific vocabulary. Most existing predictive models model point estimates rather than probability distributions, limiting our ability to quantify model uncertainty. This paper introduces a novel model, BAKER, designed to address these limitations. BAKER combines the strengths of Bayesian inference, known for its effectiveness in modelling uncertainty, and kernel methods, which excel at capturing complex data relationships. Incorporating kernel functions enhances model performance, particularly by reducing overfitting in data-limited scenarios. Our experimental analysis shows that BAKER significantly improves uncertainty reasoning compared to existing models.

Multimodal fake news detection often involves modelling heterogeneous data sources, such as vision and language. Existing detection methods typically rely on fusion effectiveness and cross-modal consistency to model the content, complicating understanding how each modality affects prediction accuracy. Additionally, these methods are primarily based on static feature modelling, making it difficult to adapt to the dynamic changes and relationships between different data modalities. This paper develops a significantly novel approach, GAMED, for multimodal modelling, which focuses on generating distinctive and discriminative features through modal decoupling to enhance cross-modal synergies, thereby optimizing overall performance in the detection process. GAMED leverages multiple parallel expert networks to refine features and pre-embed semantic knowledge to improve the experts' ability in information selection and viewpoint sharing. Subsequently, the feature distribution of each modality is adaptively adjusted based on the respective experts' opinions. GAMED also introduces a novel classification technique to dynamically manage contributions from different modalities, while improving the explainability of decisions. Experimental results on the Fakeddit and Yang datasets demonstrate that GAMED performs better than recently developed state-of-the-art models. The source code can be accessed at https://github.com/slz0925/GAMED.

Sequential recommendation is becoming more critical in a variety of e-commerce platforms. The aim of sequential recommender systems is to model the dynamic preferences of users based on their previous actions and predict what they will do next. The collected user activity logs on real-world platforms could be quite long. This wealth of information provides options to follow users’ actual interests. Prior efforts primarily aimed at providing recommendations following recent behaviors. Meanwhile, the entire sequential data may not be used efficiently since early actions may influence users’ decisions at present. Furthermore, scanning the whole behavior sequence while doing inference for every user is unbearable due to the need for prompt reaction time in real-world applications. To this end, we propose the DELIGHT Sequential Recommender System (DSRS), which takes the above properties into account to recommend the next item the user might be interested in. DSRS divides the entire user behavior sequence into long- and short-term segments and models them through independent networks before integrating their learned representations. In particular, the first network learns user long-term, whereas the second one learns short-term preferences and then combines them for an efficient joint recommendation. Experimental findings across four datasets show that our model outperforms other state-of-the-art sequential models in apprehending long-term dependence.

The field of computer vision applied to videos of minimally invasive surgery is ever-growing. Workflow recognition pertains to the automated recognition of various aspects of a surgery, including: which surgical steps are performed; and which surgical instruments are used. This information can later be used to assist clinicians when learning the surgery or during live surgery. The Pituitary Vision (PitVis) 2023 Challenge tasks the community to step and instrument recognition in videos of endoscopic pituitary surgery. This is a particularly challenging task when compared to other minimally invasive surgeries due to: the smaller working space, which limits and distorts vision; and higher frequency of instrument and step switching, which requires more precise model predictions. Participants were provided with 25-videos, with results presented at the MICCAI-2023 conference as part of the Endoscopic Vision 2023 Challenge in Vancouver, Canada, on 08-Oct-2023. There were 18-submissions from 9-teams across 6-countries, using a variety of deep learning models. The top performing model for step recognition utilised a transformer based architecture, uniquely using an autoregressive decoder with a positional encoding input. The top performing model for instrument recognition utilised a spatial encoder followed by a temporal encoder, which uniquely used a 2-layer temporal architecture. In both cases, these models outperformed purely spatial based models, illustrating the importance of sequential and temporal information. This PitVis-2023 therefore demonstrates state-of-the-art computer vision models in minimally invasive surgery are transferable to a new dataset. Benchmark results are provided in the paper, and the dataset is publicly available at: https://doi.org/10.5522/04/26531686.

Abstract Visual Question Answering (VQA) is a multimodality research domain that intersects the fields of computer vision and natural language processing for visual-textual data processing and understanding. Traditional VQA methods extract visual and textual features from pre-trained architectures, respectively, then combine the features from both modalities in a common feature space. The traditional methods perform well on high-level perception questions. However, attaining high accuracy on low-level perception questions still remains challenging. The difficulties include detecting relevant visual and textual information, building meaningful associations, and extracting insights from the multimodal data. To address these challenges, unlike existing approaches, we propose a novel multi-modality guided cross self-attention mechanism for building semantic relationships within individual modalities as well as between them. Specifically, we examine visual-guided cross-attention (VGCA), textual-guided cross-attention (TGCA), and multi-modality-guided cross-attention (MMGCA). We utilise convolutional neural networks (CNNs) for visual feature learning, and LSTM and FNET for textual feature learning. We evaluate our method on two benchmark datasets, including VQA 1.0 and VQA 2.0. Experimental results demonstrate the superiority of our method over existing baselines by improving the performance on various types of questions in both datasets.

Recent works attempt to extend Graph Convolution Networks (GCNs) to point clouds for classification and segmentation tasks. These works tend to sample and group points to create smaller point sets locally and mainly focus on extracting local features through GCNs, while ignoring the relationship between point sets. In this paper, we propose the Dynamic Hop Graph Convolution Network (DHGCN) for explicitly learning the contextual relationships between the voxelized point parts, which are treated as graph nodes. Motivated by the intuition that the contextual information between point parts lies in the pairwise adjacent relationship, which can be depicted by the hop distance of the graph quantitatively, we devise a novel self-supervised part-level hop distance reconstruction task and design a novel loss function accordingly to facilitate training. In addition, we propose the Hop Graph Attention (HGA), which takes the learned hop distance as input for producing attention weights to allow edge features to contribute distinctively in aggregation. Eventually, the proposed DHGCN is a plug-and-play module that is compatible with point-based backbone networks. Comprehensive experiments on different backbones and tasks demonstrate that our self-supervised method achieves state-of-the-art performance. Our source codes are available at: https://github.com/Jinec98/DHGCN.

Federated learning facilitates collaborative training of machine learning models on data distributed across multiple locations, effectively addressing the privacy concerns by eliminating the need for data centralization—a critical consideration in medical image analysis. In healthcare applications like cardiovascular segmentation, datasets from individual sites often feature annotations for specific heart regions, leading to partial overlaps. To address this challenge, we present a two-step partial annotation framework for federated learning, featuring a hybrid 3D multi-encoding UNet enhanced with focal modulation layers in the second stage. This architecture enables specialized subnetworks to act as experts, extracting features tailored to specific regions of interest based on each client’s data. To further improve feature extraction and differentiation, we incorporate focal modulation blocks and apply regularization by introducing an auxiliary generic decoder during training. Comprehensive experiments on diverse cardiac MRI datasets demonstrate that our approach significantly outperforms centralized learning models.

Mental health concerns are a critical issue in today’s digital age, posing a threat to both individual and societal well-being and making the identification of at-risk individuals crucial. Analyzing an individual’s social media post history can offer insights into their mental health state and help identify the presence of suicidal ideation. However, the complexity of linguistic and temporal data, along with sparsity and time irregularities, poses a formidable challenge in machine learning. Previous methods in this domain either rely on Euclidean space for processing which does not adequately model the power-law properties of social media posts, or lose information due to the discretization of the time axis. To address these challenges, we propose a novel framework, THYMES, which leverages pre-trained encoders and a rich representation learning paradigm with hyperbolic learning to model power-law features for enhanced sequence modeling. We perform experiments on two datasets and demonstrate that THYMES outperforms previously proposed methods while maintaining classification fairness under heavy data imbalances. Additionally, we qualitatively analyze commonly misclassified samples to reveal the shortcomings of models in this domain.

The rising rate of mental health issues in the digital age underscores the critical need for proactive interventions to assess an individual’s well-being. This problem is further exacerbated by the social stigma surrounding the subject, which suppresses the willingness of victims to seek help. Social media can serve as an outlet for such individuals to express their negative emotions or thoughts of self-harm. The social media account of an individual can offer a plethora of valuable information that can be used to predict their mental health. By unifying principles of robust classifier training and selective classification, we propose a novel framework, SAFENet, to predict the suicide risk of users by using their historical social media posts. When the confidence of prediction is low or the individual is classified as a high-risk user, SAFENet delegates the analysis of the posts to a human evaluator for further intervention. Our experiments show that SAFENet outperforms existing state-of-the-art frameworks. We further qualitatively analyze predictions from SAFENet and demonstrate that it performs robustly on difficult samples that may cause contemporary methods to make errors. Our system addresses the urgent need for efficient and effective mental health intervention in the digital era.

Retinal diseases can cause irreversible vision loss in both eyes if not diagnosed and treated early. Since retinal diseases are so complicated, retinal imaging is likely to show two or more abnormalities. Current deep learning techniques for segmenting retinal images with many labels and attributes have poor detection accuracy and generalisability. This paper presents a multipath convolutional neural network for multifeature segmentation. The proposed network is lightweight and spatially sensitive to information. A patch-based implementation is used to extract local image features, and focal modulation attention blocks are incorporated between the encoder and the decoder for improved segmentation. Filter optimisation is used to prevent filter overlaps and speed up model convergence. A combination of convolution operations and group convolution operations is used to reduce computational costs. This is the first robust and generalisable network capable of segmenting multiple features of fundus images (including retinal vessels, microaneurysms, optic discs, haemorrhages, hard exudates, and soft exudates). The results of our experimental evaluation on more than ten publicly available datasets with multiple features show that the proposed network outperforms recent networks despite having a small number of learnable parameters.

The identification of suspects based on partial and smudged fingerprints, commonly referred to as fingermarks or latent finger-prints, presents a significant challenge in the field of fingerprint recognition. While fixed-length embeddings have shown effectiveness in recognizing rolled and slap fingerprints, the methods for matching latent fingerprints have primarily centered around local minutiae-based embeddings, failing to fully exploit global representations for matching purposes. Consequently, enhancing latent fingerprints becomes critical to ensuring robust identification for forensic investigations. Current approaches often prioritize restoring ridge patterns, overlooking the fine minutiae details crucial for accurate fingerprint recognition. To tackle this, we propose a novel approach that utilizes Generative Adversarial Networks (GANs) to redefine Latent Fingerprint Enhancement (LFE) through a structured approach to fingerprint generation. By directly optimizing minutiae information during the generation process, the model produces enhanced latent fingerprints exhibiting exceptional fidelity to ground truth instances. This leads to a significant improvement in identification performance. Our framework integrates minutiae locations and orientation fields, ensuring the preservation of both local and structural fingerprint features. Extensive evaluations conducted on two publicly available datasets demonstrate our method's dominance over existing state-of-the-art techniques, highlighting its potential to significantly enhance latent fingerprint recognition accuracy in forensic applications.

Buildings play a crucial role in human well-being, influencing occupant comfort, health, and safety.Additionally, they contribute significantly to global energy consumption, accounting for one-third of total energy usage, and carbon emissions.Optimizing building performance presents a vital opportunity to combat climate change and promote human flourishing.However, research in building analytics has been hampered by the lack of accessible, available, and comprehensive real-world datasets on multiple building operations.In this paper, we introduce the Building TimeSeries (BTS) dataset.Our dataset covers three buildings over a three-year period, comprising more than ten thousand timeseries data points with hundreds of unique ontologies.Moreover, the metadata is standardized using the Brick schema.To demonstrate the utility of this dataset, we performed benchmarks on two tasks: timeseries ontology classification and zero-shot forecasting.These tasks represent an essential initial step in addressing challenges related to interoperability in building analytics.Access to the dataset and the code used for benchmarking are available here: https://github.com/cruiseresearchgroup/DIEF_BTS

Knowledge in materials science is widely dispersed across extensive scientific literature, posing significant challenges for efficient discovery and integration of new materials. Traditional methods, often reliant on costly and time-consuming experimental approaches, further complicate rapid innovation. Addressing these challenges, the integration of artificial intelligence with materials science has opened avenues for accelerating the discovery process, though it also demands precise annotation, data extraction, and traceability of information. To tackle these issues, this article introduces the Materials Knowledge Graph (MKG), which utilizes advanced natural language processing techniques, integrated with large language models to extract and systematically organize a decade's worth of high-quality research into structured triples, contains 162,605 nodes and 731,772 edges. MKG categorizes information into comprehensive labels such as Name, Formula, and Application, structured around a meticulously designed ontology, thus enhancing data usability and integration. By implementing network-based algorithms, MKG not only facilitates efficient link prediction but also significantly reduces reliance on traditional experimental methods. This structured approach not only streamlines materials research but also lays the groundwork for more sophisticated materials knowledge graphs.

Language models such as Bidirectional Encoder Representations from Transformers (BERT) have been very effective in various Natural Language Processing (NLP) and text mining tasks including text classification. However, some tasks still pose challenges for these models, including text classification with limited labels. This can result in a cold-start problem. Although some approaches have attempted to address this problem through single-stage clustering as an intermediate training step coupled with a pre-trained language model, which generates pseudo-labels to improve classification, these methods are often error-prone due to the limitations of the clustering algorithms. To overcome this, we have developed a novel two-stage intermediate clustering with subsequent fine-tuning that models the pseudo-labels reliably, resulting in reduced prediction errors. The key novelty in our model, IDoFew, is that the two-stage clustering coupled with two different clustering algorithms helps exploit the advantages of the complementary algorithms that reduce the errors in generating reliable pseudo-labels for fine-tuning. Our approach has shown significant improvements compared to strong comparative models.

Challenges drive the state-of-the-art of automated medical image analysis. The quantity of public training data that they provide can limit the performance of their solutions. Public access to the training methodology for these solutions remains absent. This study implements the Type Three (T3) challenge format, which allows for training solutions on private data and guarantees reusable training methodologies. With T3, challenge organizers train a codebase provided by the participants on sequestered training data. T3 was implemented in the STOIC2021 challenge, with the goal of predicting from a computed tomography (CT) scan whether subjects had a severe COVID-19 infection, defined as intubation or death within one month. STOIC2021 consisted of a Qualification phase, where participants developed challenge solutions using 2000 publicly available CT scans, and a Final phase, where participants submitted their training methodologies with which solutions were trained on CT scans of 9724 subjects. The organizers successfully trained six of the eight Final phase submissions. The submitted codebases for training and running inference were released publicly. The winning solution obtained an area under the receiver operating characteristic curve for discerning between severe and non-severe COVID-19 of 0.815. The Final phase solutions of all finalists improved upon their Qualification phase solutions.

Alzheimer's is progressive and irreversible type of dementia, which causes degeneration and death of cells and their connections in the brain. AD worsens over time and greatly impacts patients' life and affects their important mental functions, including thinking, the ability to carry on a conversation, and judgment and response to environment. Clinically, there is no single test to effectively diagnose Alzheimer disease. However, computed tomography (CT) and magnetic resonance imaging (MRI) scans can be used to help in AD diagnosis by observing critical changes in the size of different brain areas, typically parietal and temporal lobes areas. In this work, an integrative mulitresolutional ensemble deep learning-based framework is proposed to achieve better predictive performance for the diagnosis of Alzheimer disease. Unlike ResNet, DenseNet and their variants proposed pipeline utilizes PartialNet in a hierarchical design tailored to AD detection using brain MRIs. The advantage of the proposed analysis system is that PartialNet diversified the depth and deep supervision. Additionally, it also incorporates the properties of identity mappings which makes it powerful in better learning due to feature reuse. Besides, the proposed ensemble PartialNet is better in vanishing gradient, diminishing forward-flow with low number of parameters and better training time in comparison to its counter network. The proposed analysis pipeline has been tested and evaluated on benchmark ADNI dataset collected from 379 subjects patients. Quantitative validation of the obtained results documented our framework's capability, outperforming state-of-the-art learning approaches for both multi-and binary-class AD detection.

Guest Editorial: Special Issue on Generating Human Readable Explanations in NLP

In this study, we propose LDMRes-Net, a lightweight dual-multiscale residual block-based convolutional neural network tailored for medical image segmentation on IoT and edge platforms. Conventional U-Net-based models face challenges in meeting the speed and efficiency demands of real-time clinical applications, such as disease monitoring, radiation therapy, and image-guided surgery. In this study, we present the Lightweight Dual Multiscale Residual Block-based Convolutional Neural Network (LDMRes-Net), which is specifically designed to overcome these difficulties. LDMRes-Net overcomes these limitations with its remarkably low number of learnable parameters (0.072 M), making it highly suitable for resource-constrained devices. The model's key innovation lies in its dual multiscale residual block architecture, which enables the extraction of refined features on multiple scales, enhancing overall segmentation performance. To further optimize efficiency, the number of filters is carefully selected to prevent overlap, reduce training time, and improve computational efficiency. The study includes comprehensive evaluations, focusing on the segmentation of the retinal image of vessels and hard exudates crucial for the diagnosis and treatment of ophthalmology. The results demonstrate the robustness, generalizability, and high segmentation accuracy of LDMRes-Net, positioning it as an efficient tool for accurate and rapid medical image segmentation in diverse clinical applications, particularly on IoT and edge platforms. Such advances hold significant promise for improving healthcare outcomes and enabling real-time medical image analysis in resource-limited settings.

In recent years, the management and analysis of biological data have experienced exponential growth propelled by the relentless advancement of machine learning (ML) and artificial intelligence (AI) technologies. This is driven mainly by the remarkable ability and potentials of AI-based systems to craft sophisticated, yet effective, algorithms and analytical models tailored for the interpretation of biological information; thus, assist in making accurate predictions and/or decisions [1]. The surge in AI adoption is not unfounded; it's a response to the overwhelming increase in both the volume and acquisition rates of biological data.

Leakage and tampering problems in collection and transmission of biomedical data have attracted much attention as these concerns instigates negative impression regarding privacy, security, and reputation of medical networks. This article presents a novel security model that establishes a threat-vector database based on the dynamic behaviours of smart healthcare systems. Then, an improved and privacy-preserved SRU network is designed that aims to alleviate fading gradient issue and enhance the learning process by reducing computational cost. Then, an intelligent federated learning algorithm is deployed to enable multiple healthcare networks to form a collaborative security model in a personalized manner without the loss of privacy. The proposed security method is both parallelizable and computationally effective since the dynamic behaviour aggregation strategy empowers the model to work collaboratively and reduce communication overhead by dynamically adjusting the number of participating clients. Additionally, the visualization of the decision process based on the explainability of features enhances the understanding of security experts by enabling them to comprehend the underlying data evidence and causal reasoning. Compared to existing methods, the proposed security method is capable of thoroughly analyzing and detecting severe security threats with high accuracy, reduce overhead and lower computation cost along with enhanced privacy of biomedical data.

Cancer begins when healthy cells change and grow out of control, forming a mass called a tumor. Head and neck (H&N) cancers usually develop in or around the head and neck, including the mouth (oral cavity), nose and sinuses, throat (pharynx), and voice box (larynx). 4% of all cancers are H&N cancers with a very low survival rate (a five-year survival rate of 64.7%). FDG-PET/CT imaging is often used for early diagnosis and staging of H&N tumors, thus improving these patients' survival rates. This work presents a novel 3D-Inception-Residual aided with 3D depth-wise convolution and squeeze and excitation block. We introduce a 3D depth-wise convolution-inception encoder consisting of an additional 3D squeeze and excitation block and a 3D depth-wise convolution-based residual learning decoder (3D-IncNet), which not only helps to recalibrate the channel-wise features but adaptively through explicit inter-dependencies modeling but also integrate the coarse and fine features resulting in accurate tumor segmentation. We further demonstrate the effectiveness of inception-residual encoder-decoder architecture in achieving better dice scores and the impact of depth-wise convolution in lowering the computational cost. We applied random forest for survival prediction on deep, clinical, and radiomics features. Experiments are conducted on the benchmark HECKTOR21 challenge, which showed significantly better performance by surpassing the state-of-the-artwork and achieved 0.836 and 0.811 concordance index and dice scores, respectively. We made the model and code publicly available.

People across the globe have felt and are still going through the impact of COVID-19. Some of them share their feelings and suffering online via different online social media networks such as Twitter. Due to strict restrictions to reduce the spread of the novel virus, many people are forced to stay at home, which significantly impacts people's mental health. It is mainly because the pandemic has directly affected the lives of the people who were not allowed to leave home due to strict government restrictions. Researchers must mine the related human-generated data and get insights from it to influence government policies and address people's needs. In this paper, we study social media data to understand how COVID-19 has impacted people's depression. We share a large-scale COVID-19 dataset that can be used to analyze depression. We also have modeled the tweets of depressed and non-depressed users before and after the start of the COVID-19 pandemic. To this end, we developed a new approach based on Hierarchical Convolutional Neural Network (HCN) that extracts fine-grained and relevant content on user historical posts. HCN considers the hierarchical structure of user tweets and contains an attention mechanism that can locate the crucial words and tweets in a user document while also considering the context. Our new approach is capable of detecting depressed users occurring within the COVID-19 time frame. Our results on benchmark datasets show that many non-depressed people became depressed during the COVID-19 pandemic.

The early detection of glaucoma is essential in preventing visual impairment. Artificial intelligence (AI) can be used to analyze color fundus photographs (CFPs) in a cost-effective manner, making glaucoma screening more accessible. While AI models for glaucoma screening from CFPs have shown promising results in laboratory settings, their performance decreases significantly in real-world scenarios due to the presence of out-of-distribution and low-quality images. To address this issue, we propose the Artificial Intelligence for Robust Glaucoma Screening (AIROGS) challenge. This challenge includes a large dataset of around 113,000 images from about 60,000 patients and 500 different screening centers, and encourages the development of algorithms that are robust to ungradable and unexpected input data. We evaluated solutions from 14 teams in this paper and found that the best teams performed similarly to a set of 20 expert ophthalmologists and optometrists. The highest-scoring team achieved an area under the receiver operating characteristic curve of 0.99 (95% CI: 0.98-0.99) for detecting ungradable images on-the-fly. Additionally, many of the algorithms showed robust performance when tested on three other publicly available datasets. These results demonstrate the feasibility of robust AI-enabled glaucoma screening.

Graph convolutional networks (GCNs) have emerged as a powerful tool for action recognition, leveraging skeletal graphs to encapsulate human motion. Despite their efficacy, a significant challenge remains the dependency on huge labeled datasets. Acquiring such datasets is often prohibitive, and the frequent occurrence of incomplete skeleton data, typified by absent joints and frames, complicates the testing phase. To tackle these issues, we present graph representation alignment (GRA), a novel approach with two main contributions: 1) a self-training (ST) paradigm that substantially reduces the need for labeled data by generating high-quality pseudo-labels, ensuring model stability even with minimal labeled inputs and 2) a representation alignment (RA) technique that utilizes consistency regularization to effectively reduce the impact of missing data components. Our extensive evaluations on the NTU RGB+D and Northwestern-UCLA (N-UCLA) benchmarks demonstrate that GRA not only improves GCN performance in data-constrained environments but also retains impressive performance in the face of data incompleteness.

Traditional support vector machines (SVMs) are fragile in the presence of outliers; even a single corrupt data point can arbitrarily alter the quality of the approximation. If even a small fraction of columns is corrupted, then classification performance will inevitably deteriorate. This article considers the problem of high-dimensional data classification, where a number of the columns are arbitrarily corrupted. An efficientSupportMatrixMachine that simultaneously performs matrixRecovery (SSMRe) is proposed, i.e. feature selection and classification through joint minimization of$\ell_{2,1}$(the nuclear norm of$L$). The data are assumed to consist of a low-rank clean matrix plus a sparse noisy matrix. SSMRe works under incoherence and ambiguity conditions and is able to recover an intrinsic matrix of higher rank in the presence of data densely corrupted. The objective function is a spectral extension of the conventional elastic net; it combines the property of matrix recovery along with low rank and joint sparsity to deal with complex high-dimensional noisy data. Furthermore, SSMRe leverages structural information, as well as the intrinsic structure of data, avoiding the inevitable upper bound. Experimental results on different real-time applications, supported by the theoretical analysis and statistical testing, show significant gain for BCI, face recognition, and person identification datasets, especially in the presence of outliers, while preserving a reasonable number of support vectors.

In recent years, distributed graph convolutional networks (GCNs) training frameworks have achieved great success in learning the representation of graph-structured data with large sizes. However, existing distributed GCN training frameworks require enormous communication costs since a multitude of dependent graph data need to be transmitted from other processors. To address this issue, we propose a graph augmentation-based distributed GCN framework (GAD). In particular, GAD has two main components: GAD-Partition and GAD-Optimizer. We first propose an augmentation-based graph partition (GAD-Partition) that can divide the input graph into augmented subgraphs to reduce communication by selecting and storing as few significant vertices of other processors as possible. To further speed up distributed GCN training and improve the quality of the training result, we design a subgraph variance-based importance calculation formula and propose a novel weighted global consensus method, collectively referred to as GAD-Optimizer. This optimizer adaptively adjusts the importance of subgraphs to reduce the effect of extra variance introduced by GAD-Partition on distributed GCN training. Extensive experiments on four large-scale real-world datasets demonstrate that our framework significantly reduces the communication overhead ( ≈ 50% ), improves the convergence speed ( ≈ 2 × ) of distributed GCN training, and obtains a slight gain in accuracy ( ≈ 0.45% ) based on minimal redundancy compared to the state-of-the-art methods.

A fascinating challenge in robotics-human interaction is imitating the emotion recognition capability of humans to robots with the aim to make human-robotics interaction natural, genuine and intuitive. To achieve the natural interaction in affective robots, human-machine interfaces, and autonomous vehicles, understanding our attitudes and opinions is very important, and it provides a practical and feasible path to realize the connection between machine and human. Multimodal interface that includes voice along with facial expression can manifest a large range of nuanced emotions compared to purely textual interfaces and provide a great value to improve the intelligence level of effective communication. Interfaces that fail to manifest or ignore user emotions may significantly impact the performance and risk being perceived as cold, socially inept, untrustworthy, and incompetent. To equip a child well for life, we need to help our children identify their feelings, manage them well, and express their needs in healthy, respectful, and direct ways. Early identification of emotional deficits can help to prevent low social functioning in children. In this work, we analyzed the child’s spontaneous behavior using multimodal facial expression and voice signal presenting multimodal transformer-based last feature fusion for facial behavior analysis in children to extract contextualized representations from RGB video sequence and Hematoxylin and eosin video sequence and then using these representations followed by pairwise concatenations of contextualized representations using cross-feature fusion technique to predict users emotions. To validate the performance of the proposed framework, we have performed experiments with the different pairwise concatenations of contextualized representations that showed significantly better performance than state-of-the-art method. Besides, we perform t-distributed stochastic neighbor embedding visualization to visualize the discriminative feature in lower dimension space and probability density estimation to visualize the prediction capability of our proposed model.

Retinal disorders diagnosis is of immense importance for appropriate treatment, i.e., accurate personalized medicine. In this work, a multi-resolutional feature ensemble approach is developed for retinal image classification using optical coherence tomography (OCT) images. Particularly, feature-rich pipeline using pyramidal architecture is designed to extract features from multi-scale inputs using partially-connected networks (PCNet). In addition, higher-order reflectivity features are extracted from the input images and are fused with pyramidal features for classification. The advantage of the hierarchical PCNet structure is that it allowed our system to extract multi-scale information to help in such task, all-at-once classification of the normal and abnormal retina. Namely, the larger input sizes give more global information, while the small inputs focus on local details. Evaluation on public OCT data set of four classes (normal, diabetic macular edema (DME), choroidal neovascularization (CNV), and drusen) and comparison against recent networks demonstrates not only the advantages of the proposed architecture’s ability to produce feature-rich classification, but also highlights tangible advantages, such as network parameter reduction, enhanced feature learning and information flow, while reducing the risk of over fitting.

Edge computing (EC) offers cloud-like services at the edge of mobile networks to satisfy the delay-sensitive and rapid computation applications in meeting the demands of rapidly increasing mobile devices and other Internet of Things. EC is known to be constrained with limited resources that its efficacy greatly depends on an effective and efficient resource allocation to provide optimal resource utilization. Focusing on the fact, this article presents an adaptive resource allocation mechanism, abbreviated as A-PBRA, for effective resources utilization in the EC paradigm. To realize optimal utilization, the available resources are allocated dynamically (adaptability) by considering the nature of the incoming requests. The proposed scheme shall adapt to the resource demands and priorities of the incoming requests. After identifying the received request which can be either the priority-based or normal request, each of them is processed with three possibilities. The available resources are thus allocated as per the priorities of the incoming requests to satisfy the constraints accordingly. The proposed mechanism is adaptable to a maximum number of incoming requests along with optimizing the utilization of limited resources at the edge node. Extensive simulations were performed through ifogsim to evaluate the performance of the proposed method. Critical comparisons were made against closely related algorithms and techniques, i.e., the novel bioinspired hybrid algorithm and the CORA-GT. The simulation results from the proposed scheme optimistically showing that it performed better in terms of resources utilization, average response time, task execution time, and energy consumption.

A multimodal biometric recognition system on Internet of Things (IoT) with Blockchain environments has been proposed in this article. This system distributes a decentralized biometric authentication process mechanism and improves security in the IoT environment. The implementation of this system consists of five components: 1) image preprocessing; 2) feature representation; 3) cancelable biometrics; 4) classification; and 5) encryption–decryption of multimodal biometrics templates. A region of interest is segmented from each biometric trait during image preprocessing. Then, a discriminant feature extraction technique has been employed for feature computation. A cancellable biometric system (CBS) is introduced to secure and preserve the original biometric features from external hazards and misuse. The extracted cancelable features undergo classification to perform the subjects’ authentication. Then, a method of encryption–decryption of templates is performed to handle the various online authentication attacks and improve IoT-enabled authentication. Finally, the recognition scores due to iris, periocular, palmprint, and face biometrics are fused to increase the performance of the proposed IoT-enabled multimodal biometric system. The proposed system obtains identification performance 99.92%, 100% for CASIA-V4-distance (CASIA-DIST), UBIRIS-v2 iris, 100% for periocular (CASIA-DIST, UBIRIS-v2), 100% for Bosphorus palmprint, and 100% for FERET face databases using 30-D cancelable features that show the superiority of the proposed system as compared with state-of-the-art methods.

Abstract Skin cancer, primarily resulting from the abnormal growth of skin cells, is among the most common cancer types. In recent decades, the incidence of skin cancer cases worldwide has risen significantly (one in every three newly diagnosed cancer cases is a skin cancer). Such an increase can be attributed to changes in our social and lifestyle habits coupled with devastating man-made alterations to the global ecosystem. Despite such a notable increase, diagnosis of skin cancer is still challenging, which becomes critical as its early detection is crucial for increasing the overall survival rate. This calls for advancements of innovative computer-aided systems to assist medical experts with their decision making. In this context, there has been a recent surge of interest in machine learning (ML), in particular, deep neural networks (DNNs), to provide complementary assistance to expert physicians. While DNNs have a high processing capacity far beyond that of human experts, their outputs are deterministic, i.e., providing estimates without prediction confidence. Therefore, it is of paramount importance to develop DNNs with uncertainty-awareness to provide confidence in their predictions. Monte Carlo dropout (MCD) is vastly used for uncertainty quantification; however, MCD suffers from overconfidence and being miss calibrated. In this paper, we use MCD algorithm to develop an uncertainty-aware DNN that assigns high predictive entropy to erroneous predictions and enable the model to optimize the hyper-parameters during training, which leads to more accurate uncertainty quantification. We use two synthetic (two moons and blobs) and a real dataset (skin cancer) to validate our algorithm. Our experiments on these datasets prove effectiveness of our approach in quantifying reliable uncertainty. Our method achieved 85.65 ± 0.18 prediction accuracy, 83.03 ± 0.25 uncertainty accuracy, and 1.93 ± 0.3 expected calibration error outperforming vanilla MCD and MCD with loss enhanced based on predicted entropy.

Achieving accurate retinal vessel segmentation is critical in the progression and diagnosis of vision-threatening diseases such as diabetic retinopathy and age-related macular degeneration. Existing vessel segmentation methods are based on encoder-decoder architectures, which frequently fail to take into account the retinal vessel structure's context in their analysis. As a result, such methods have difficulty bridging the semantic gap between encoder and decoder characteristics. This paper proposes a Prompt Deep Light-weight Vessel Segmentation Network (PLVS-Net) to address these issues by using prompt blocks. Each prompt block use combination of asymmetric kernel convolutions, depth-wise separable convolutions, and ordinary convolutions to extract useful features. This novel strategy improves the performance of the segmentation network while simultaneously decreasing the number of trainable parameters. Our method outperformed competing approaches in the literature on three benchmark datasets, including DRIVE, STARE, and CHASE.

Depression is a mental disorder characterized by persistent depressed mood or loss of interest in performing activities, causing significant impairment in daily routine. Possible causes include psychological, biological, and social sources of distress. Clinical depression is the more-severe form of depression, also known as major depression or major depressive disorder. Recently, electroencephalography and speech signals have been used for early diagnosis of depression; however, they focus on moderate or severe depression. We have combined audio spectrogram and multiple frequencies of EEG signals to improve diagnostic performance. To do so, we have fused different levels of speech and EEG features to generate descriptive features and applied vision transformers and various pre-trained networks on the speech and EEG spectrum. We have conducted extensive experiments on Multimodal Open Dataset for Mental-disorder Analysis (MODMA) dataset, which showed significant improvement in performance in depression diagnosis (0.972,0.973and0.973precision, recall and F1 score respectively) for patients at the mild stage. Besides, we provided a web-based framework using Flask and provided the source code publicly.1

Identifying rising stars is very useful for faster growth of any organization. Rising entities has been explored in academics, sports, and blogs in the recent past, but business side is ignored. However, predicting rising business managers (RBMs) can result in significant business growth of any business. In order to maintain a competitive edge, machine learning techniques should be adopted to devise intelligent business strategies and perform predictions. In this article, RBMs are classified by exploring features of co-business managers (Co-BMs), rather than their own work history. Since ignoring their work history enables such prediction in a cold-start scenario, where work history is not available. After formulating features of Co-BM, the dataset is classified into two different evaluation setups. One is average revenue (AR) and the other one is average relative increase in revenue (ARIR)—class labels. All instances for both labels are randomly sorted into multisize (10, 20, and 30–100) datasets. Later on, these datasets are explored through machine learning classifiers using fivefold cross validation. In order to compare the prediction results with baseline and to measure the effectiveness of the proposed methods, the candidates’ business scores are used, which are calculated by the business definition and formulation. In terms of precision, recall, and f-measure, the feature, category, and model-based experimental results show that the generative models, particularly Bayesian networks, produce better results for an AR-based dataset. Also, overall results show the effective the proposed method.

Users can interact with one another through social networks (SNs) by exchanging information, delivering comments, finding new information, and engaging in discussions that result in the production of vast volumes of data daily. These data, available in various forms, such as images, text, and videos, may be interpreted to reflect the user’s activities, including their mental state regarding depression. For example, depression is a chronic disease from which the vast majority of users suffer, and it has emerged as a significant issue relating to mental health on a global scale. However, because these data are scant, unfinished, and sometimes given inaccurately, it is challenging to make an accurate automated diagnosis from them. Even though several procedures have been utilized over the past few decades to diagnose depression, machine learning (ML) and deep learning (DL) techniques supply superior insights. Thus, in this study, we review several state-of-the-art ML and DL techniques in terms of the systematic literature review (SLR) approach for depression detection. We also highlight some critical challenges from the existing literature that may help to explore for future study. Finally, we believe this survey will help readers and researchers in ML and DL to understand critical solutions in diagnosing depression.

Health mention classification classifies a given piece of text as a health mention or not. However, figurative usage of disease words makes the classification task challenging. To address this challenge, consideration of emojis and surrounding words of the disease names in the text can be helpful. Transformer-based methods are better at capturing the meaning of a word based on its surrounding words compared to traditional methods. However, there are numerous transformer-based methods available and pretrained on natural language processing (NLP) data that are inherently different from Twitter data. Moreover, the size of these models varies in terms of the number of parameters. Hence, it is challenging to decide and choose one of these methods for fine-tuning it on the downstream tasks such as tweet classification. In this work, we experiment with nine widely used transformer methods and compare their performance on the personal health mention classification of tweet data. Furthermore, we analyze the impact of model size on the classification task and provide a brief interpretation of the classification decision made by the best performing classifier. Experimental results show that RoBERTa outperforms all other models by achieving an F1 score of 93%, while two other models perform similarly by achieving an F1 score of 92.5%.

A fundamental expectation of the stakeholders from the Industrial Internet of Things (IIoT) is its trustworthiness and sustainability to avoid the loss of human lives in performing a critical task. A trustworthy IIoT-enabled network encompasses fundamental security characteristics such as trust, privacy, security, reliability, resilience and safety. The traditional security mechanisms and procedures are insufficient to protect these networks owing to protocol differences, limited update options, and older adaptations of the security mechanisms. As a result, these networks require novel approaches to increase trust-level and enhance security and privacy mechanisms. Therefore, in this paper, we propose a novel approach to improve the trustworthiness of IIoT-enabled networks. We propose an accurate and reliable supervisory control and data acquisition (SCADA) network-based cyberattack detection in these networks. The proposed scheme combines the deep learning-based Pyramidal Recurrent Units (PRU) and Decision Tree (DT) with SCADA-based IIoT networks. We also use an ensemble-learning method to detect cyberattacks in SCADA-based IIoT networks. The non-linear learning ability of PRU and the ensemble DT address the sensitivity of irrelevant features, allowing high detection rates. The proposed scheme is evaluated on fifteen datasets generated from SCADA-based networks. The experimental results show that the proposed scheme outperforms traditional methods and machine learning-based detection approaches. The proposed scheme improves the security and associated measure of trustworthiness in IIoT-enabled networks.

Federated learning (FL) has recently emerged as a striking framework for allowing machine and deep learning models with thousands of participants to have distributed training to preserve the privacy of users' data. Federated learning comes with the pros of allowing all participants the possibility of creating robust models even in the absence of sufficient training data. Recently, smartphone usage has increased significantly due to its portability and ability to perform many daily life tasks. Typing on a smartphone's soft keyboard generates vibrations that could be abused to detect the typed keys, aiding side-channel attacks. Such data can be collected using smartphone hardware sensors during the entry of sensitive information such as clinical notes, personal medical information, username, and passwords. This study proposes a novel framework based on federated learning for side-channel attack detection to secure this information. We collected a dataset from 10 Android smartphone users who were asked to type on the smartphone soft keyboard. We convert this dataset into two windows of five users to make two clients training local models. The federated learning-based framework aggregates model updates contributed by two clients and trained the Deep Neural Network (DNN) model individually on the dataset. To reduce the over-fitting factor, each client examines the findings three times. Experiments reveal that the DNN model achieves an accuracy of 80.09%, showing that the proposed framework has the potential to detect side-channel attacks.

The Internet of things (IoT) is a network of technologies that support a wide variety of healthcare workflow applications to facilitate users' obtaining real-time healthcare services. Many patients and doctors' hospitals use different healthcare services to monitor their healthcare and save their records on the servers. Healthcare sensors are widely linked to the outside world for different disease classifications and questions. These applications are extraordinarily dynamic and use mobile devices to roam several locales. However, healthcare apps confront two significant challenges: data privacy and the cost of application execution services. This work presents the mobility-aware security dynamic service composition (MSDSC) algorithmic framework for workflow healthcare based on serverless, serverless, and restricted Boltzmann machine mechanisms. The study suggests the stochastic deep neural network trains probabilistic models at each phase of the process, including service composition, task sequencing, security, and scheduling. The experimental setup and findings revealed that the developed system-based methods outperform traditional methods by 25% in terms of safety and 35% in application cost.

The thirteen papers in this special issue focus on open and interoperable artificial intelligence (AI) in computational pathology. Recent years have seen exponential advances in the quality of AI techniques in the medical filed. Particularly, AI/ML tools have been widely exploited pathological image analysis to examine and assess the function of human organs and/or to provide trustworthy prediction of diseases.

As per the latest statistics, Alzheimer's disease (AD) has become a global burden over the following decades. Identifying AD at the intermediate stage became challenging, with mild cognitive impairment (MCI) utilizing credible biomarkers and robust learning approaches. Neuroimaging techniques like magnetic resonance imaging (MRI) and positron emission tomography (PET) are practical research approaches that provide structural atrophies and metabolic variations. With the help of MRI and PET scans, metabolic and structural changes in AD patients can be visible even ten years before the disease's onset. This paper proposes a novel wavelet packet transform-based structural and metabolic image fusion approach using MRI and PET scans. An eight-layer trained CNN extracts features from multiple layers and these features are fed to an ensemble of non-iterative random vector functional link (RVFL) models. The RVFL network incorporates the s-membership fuzzy function as an activation function that helps overcome outliers. Lastly, outputs of all the customized RVFL classifiers are averaged and fed to the RVFL classifier to make the final decision. Experiments are performed over Alzheimer's Disease Neuroimaging Initiative (ADNI) dataset, and classification is made over CN vs. AD vs. MCI. The model performance obtained is decent enough to prove the effectiveness of the fusion-based ensemble approach.

The convergence of Maritime Transportation Systems (MTS) and Internet of Things (IoT) has led to the promising IoT-empowered MTS (IoT-MTS). However, abnormal trajectories of maritime transportation ships can have highly negative impacts on the management of IoT-MTS. Therefore, anomaly detection of trajectories is important for the successful deployment of IoT-MTS. In this paper, we propose a Transfer Learning based Trajectory Anomaly Detection strategy, named TLTAD, for IoT-MTS. Specifically, a variational autoencoder is used to discover the potential connections between each dimension of the normal trajectory, while a graph variational autoencoder is used to explore the spatial similarity between normal trajectories. Based on internal connection of trajectories, a deep reinforcement learning algorithm, Twin Delayed Deep Deterministic policy gradient (TD3), is employed to train the trajectory anomaly detection model. To reduce the model training time, transfer learning is used to migrate the trained anomaly detection model between different regions of an ocean area or between similar ocean areas. Moreover, an efficient data transformation module is designed to improve the efficiency of model transfer. The experiments were conducted on a real-world automatic identification system (AIS) dataset. The results indicate that the proposed TLTAD can provide accurate anomaly detection on ships’ trajectories in IoT-MTS with reduced model training times.

The proliferation of the Internet of Things (IoT) has led to the design and incorporation of innovative user control mechanisms, one category based on brain-derived biometric data and known as Brain Control Interface (BCI). BCI devices measure brain signals in EEG and allow users to interact with computerised systems, such as the Industrial Internet of Things (IIoT), intuitively. However, the utilisation of EEG data in the IIoT for actuator control and the collection of such biometric data as evidence for policing the IIoT introduces considerable implications for the users’ cognitive privacy. Thus, considering the importance of cognitive privacy in an evolving era of smart environments, it becomes imperative to develop methods that can ensure the cognitive privacy of users. Cognitive privacy also protects the anonymity of corporations and law enforcement. Furthermore, it maintains the inherent information found in EEG measurements, used by various machine and deep learning models. This paper proposes a novel deep learning-based human cognitive privacy framework, named DeepCog, that ensures users’ privacy through the application of feature transforming normalisation. A deep MLP model then processes the encoded data to classify samples according to an integer-based subject ID, enabling the framework to select the correct secondary deep MLP model (one for each subject) to identify eye activity. Our experiments indicate high accuracy of 93.4%, with precision 93.2% and recall 93.8%, outperforming compelling techniques.

This paper proposes a novel mask-based output layer for multi-level hierarchical classification, addressing the limitations of existing methods which (i) often do not embed the taxonomy structure being used, (ii) use a complex backbone neural network with n disjoint output layers that do not constraint each other, (iii) may output predictions that are often inconsistent with the taxonomy in place, and (iv) have often a fixed value of n. Specifically, we propose a model agnostic output layer that embeds the taxonomy and that can be combined with any model. Our proposed output layer implements a top-down divide-and-conquer strategy through a masking mechanism to enforce that predictions comply with the embedded hierarchy structure. Focusing on image classification, we evaluate the performance of our proposed output layer on three different datasets, each with a three-level hierarchical structure. Experiments on these datasets show that our proposed mask-based output layer allows to improve several multi-level hierarchical classification models using various performance metrics.

This paper presents Marine-tree, a large-scale hierarchical annotated dataset for marine organism classification. Marine-tree contains more than 160k annotated images divided into 60 classes organised in a hierarchy-tree structure using an adapted CATAMI (Collaborative and Automated Tools for the Analysis of Marine Imagery and video) classification scheme. Images were meticulously collected by scuba divers using the RLS (Reef Life Survey) methodology and later annotated by experts in the field. We also propose a hierarchical loss function that can be applied to any multi-level hierarchical classification model, which takes into account the parent-child relationship between predictions and uses it to penalize inconsistent predictions. Experimental results demonstrate thatMarine-tree and the proposed hierarchical loss function are a good contribution for both research in underwater imagery and hierarchical classification.

In this paper we introduce Jarvis, a context-as-a-service mobile tool, which enables context-aware data collection, service discovery, and computer-aided situational awareness through a conversational User Interface (UI). At the core of Jarvis are two main components: (i) a voice-based UI to translate speech to Context Definition and Query Language (Speech-to-CDQL), and (ii) an operational component called Context-as-a-Service (CoaaS), which enables smart things and IoT silos to discover, validate and share relevant and dependable context. The UI is based on two machine learning models: a Speech-to-Text model and a Text-to-CDQL model based on an encoder-decoder architecture. Jarvis is developed as a mobile application that allows people with different backgrounds to interact with various IoT devices. Our demo shows how easy Jarvis can be used for context-aware data collection and to interact with diverse objects in a smart home environment through voice.

The modern industrial sector generates enormous amounts of high-dimensional heterogeneous data daily. However, mostly the vectored data (rank-one tensor) have been considered for anomaly detection, whereas the data in real-life is high dimensional. The expressive power of methods based on vector data is restrictive as they may destroy the structural information embedded in data and lead to the curse-of-dimensionality and overfitting. In this paper, we present a novel anomaly detection approach for large-scale tensor data. We first present novel one-class support tensor machines (OCSTM) with bounded loss function. We further extend it by leveraging the randomness to design a scalable approach that can also be used for large-scale anomaly detection. To solve the corresponding optimization of the objective function, we utilize half-quadratic optimization followed by solving it like a traditional OCSTM optimization at each iteration. We demonstrate the proposed randomized OCSTM with bounded hinge loss through experiments on 14 benchmark data sets. Experimental results demonstrate the effectiveness of the proposed approach against anomalies and a significant reduction in the computational complexity.

Alzheimer's disease (AD) is an irreversible and progressive disorder where a large number of brain cells and their connections degenerate and die, eventually destroy the memory and other important mental functions that affect memory, thinking, language, judgment, and behavior. Not a single test can effectively determine AD; however, CT and magnetic resonance imaging (MRI) can be used to observe the decrease in size of different areas (mainly temporal and parietal lobes). This paper proposes an integrative deep ensemble learning framework to obtain better predictive performance for AD diagnosis. Unlike DenseNet, we present a multiresolutional ensemble PartialNet tailored to Alzheimer detection using brain MRIs. PartialNet incorporates the properties of identity mappings, diversified depth as well as deep supervision, thus, considers feature reuse that in turn results in better learning. Additionally, the proposed ensemble PartialNet demonstrates better characteristics in terms of vanishing gradient, diminishing forward flow with better training time, and a low number of parameters compared with DenseNet. Experiments performed on benchmark AD neuroimaging initiative data set that showed considerable performance gain (2 + % ↑ $\uparrow $ ) and (1.2 + % ↑ $\uparrow $ ) for multiclass and binary class in AD detection in comparison to state-of-the-art methods.

Many modern neural network architectures with over parameterized regime have been used for identification of skin cancer. Recent work showed that network, where the hidden units are polynomially smaller in size, showed better performance than overparameterized models. Hence, in this paper, we present multistage unit-vise deep dense residual network with transition and additional supervision blocks that enforces the shorter connections resulting in better feature representation. Unlike ResNet, We divided the network into several stages, and each stage consists of several dense connected residual units that support residual learning with dense connectivity and limited the skip connectivity. Thus, each stage can consider the features from its earlier layers locally as well as less complicated in comparison to its counter network. Evaluation results on ISIC-2018 challenge consisting of 10,015 training images show considerable improvement over other approaches achieving 98.05 percent accuracy and improving on the best results achieved in comparison to state of the art methods. The code of Unit-vise network is publicly available.1

The industrial Internet of Things (IIoTs) is a key component of the fourth industrial revolution (Industry 4.0) which is faced with privacy issues as the scale and sensitivity of user and system data constantly increases. Eavesdropping attack is one of such privacy issue of the IIoT system especially when the number of transmitting antennas is increased. Thus, the focus of this article is on establishing efficient privacy in an IIoT-multiple-input–multiple-output–multiple-antenna eavesdropping communications scenario. To achieve this, a closed-form derivation for asymptotic regularized prompt privacy rate is first formulated for IIoT network system. Then, the study further examines the design of optimal jamming parameters by proposing a model referred as optimal counter-eavesdropping channel approximation technique for tackling eavesdropping attack in IIoT. The simulated performance of the proposed model clearly shows that provided that the channel coherence time is less than two times the number of transmitting nodes, a high privacy precision is achieved even without deploying any artificial noise.

Critical energy infrastructure (CEI) is specific engineering information about proposed or existing critical infrastructure. Modern critical infrastructures are increasingly turning into distributed, complex cyber-physical systems that need proactive protection and fast restoration to mitigate physical or cyber incidents or attacks. Most importantly, combined cyber-physical attacks are much more challenging and are expected to become the most intrusive attack. This is particularly true for the CEIs. During 2015, the Industrial Control Systems Cyber Emergency Response Team in the Unites States responded to more than 245 incidents; the energy sector tops the list with 32% incidents. Considering the importance of energy in our daily lives and its influence on other critical infrastructures, CEI requires significant attention comparatively. For example, the wind-turbine system is considered one of the most complex cyber-physical infrastructures, causing huge cascading effects to other CEIs, such as electrical power and energy systems and transportation, healthcare sector, communications, industry, and finance. Wind turbines are mainly composed of condition monitoring and operational data (i.e., supervisory command and data acquisition), including air temperature, air pressure, voltage, and power with multiple parameters and periodic characteristics.

Federated learning methods offer secured monitor services and privacy-preserving paradigms to end-users and organisations in the Internet of Things networks such as smart healthcare systems. Federated learning has been coined to safeguard sensitive data, and its global aggregation is often based on a centralised server. This design is vulnerable to malicious attacks and could be breached by privacy attacks such as inference and free-riding, leading to inefficient training models. Besides, uploaded analysing parameters by patients can reveal private information and the threat of direct manipulation by the central server. To address these issues, we present a three-fold Federated Edge Aggregator, the so-called Edge Intelligence, a federated learning-based privacy protection framework for safeguarding Smart Healthcare Systems at the edge against such privacy attacks. We employ an iteration-based Conventional Neural Network (CNN) model and artificial noise functions to balance privacy protection and model performance. A theoretical convergence bound of Edge Intelligence on the trained federated learning model's loss function is also introduced here. We evaluate and compare the proposed framework with the recently established methods using model performance and privacy budget on popular and recent datasets: MNIST, CIFAR10, STL10, and COVID19 chest x-ray. Finally, the proposed framework achieves 90% accuracy and a high privacy rate demonstrating better performance than the baseline technique.

Machine learning (ML) algorithms play a vital role in brain age estimation frameworks. The impact of regression algorithms on prediction accuracy in the brain age estimation frameworks have not been comprehensively evaluated. Here, we sought to assess the efficiency of different regression algorithms on brain age estimation. To this end, we built a brain age estimation framework based on a large set of cognitively healthy (CH) individuals (N = 788) as a training set followed by different regression algorithms (18 different algorithms in total). We then quantified each regression-algorithm on independent test sets composed of 88 CH individuals, 70 mild cognitive impairment patients as well as 30 Alzheimers disease patients. The prediction accuracy in the independent test set (i.e., CH set) varied in regression algorithms (mean absolute error (MAE) from 4.63 to 7.14 yrs, R2 from 0.76 to 0.88). The highest and lowest prediction accuracies were achieved by Quadratic Support Vector Regression algorithm (MAE = 4.63 yrs, R2 = 0.88, 95% CI = [-1.26, 1.42]) and Binary Decision Tree algorithm (MAE = 7.14 yrs, R2 = 0.76, 95% CI = [-1.50, 2.62]), respectively. Our experimental results demonstrate that prediction accuracy in brain age frameworks is affected by regression algorithms, indicating that advanced machine learning algorithms can lead to more accurate brain age predictions in clinical settings.

Online healthcare applications have grown more popular over the years. For instance, telehealth is an online healthcare application that allows patients and doctors to schedule consultations, prescribe medication, share medical documents, and monitor health conditions conveniently. Apart from this, telehealth can also be used to store a patient's personal and medical information. With its rise in usage due to COVID-19, given the amount of sensitive data it stores, security measures are necessary. A simple way of making these applications more secure is through user authentication. One of the most common and often used authentications is face recognition. It is convenient and easy to use. However, face recognition systems are not foolproof. They are prone to malicious attacks like printed photos, paper cutouts, replayed videos, and 3D masks. The goal of face anti-spoofing is to differentiate real users (live) from attackers (spoof). Although effective in terms of performance, existing methods use a significant amount of parameters, making them resource-heavy and unsuitable for handheld devices. Apart from this, they fail to generalize well to new environments like changes in lighting or background. This paper proposes a lightweight face anti-spoofing framework that does not compromise on performance. Our proposed method achieves good performance with the help of an ArcFace Classifier (AC). The AC encourages differentiation between spoof and live samples by making clear boundaries between them. With clear boundaries, classification becomes more accurate. We further demonstrate our model's capabilities by comparing the number of parameters, FLOPS, and performance with other state-of-the-art methods.

Alzheimer's disease (AD) is one of the deadliest neurodegenerative diseases ailing the elderly population all over the world. An ensemble of Deep learning (DL) models can learn highly complicated patterns from MRI scans for the detection of AD by utilizing diverse solutions. In this work, we propose a computationally efficient, DL-architecture agnostic, ensemble of deep neural networks, named 'Deep Transfer Ensemble (DTE)' trained using transfer learning for the classification of AD. DTE leverages the complementary feature views and diversity introduced by many different locally optimum solutions reached by individual networks through the randomization of hyper-parameters. DTE achieves an accuracy of 99.05% and 85.27% on two independent splits of the large dataset for cognitively normal (NC) vs AD classification task. For the task of mild cognitive impairment (MCI) vs AD classification, DTE achieves 98.71% and 83.11% respectively on the two independent splits. It also performs reasonable on a small dataset consisting of only 50 samples per class. It achieved a maximum accuracy of 85% for NC vs AD on the small dataset. It also outperformed snapshot ensembles along with several other existing deep models from similar kind of previous works by other researchers.

With the integration of the Internet of Things (IoT) in the field of transportation, the Internet of Vehicles (IoV) turned to be a vital method for designing Smart Transportation Systems (STS). STS consist of various interconnected vehicles and transportation infrastructure exposed to cyber intrusion due to the broad usage of software and the initiation of wireless interfaces. This study proposes a federated deep learning-based intrusion detection framework (FED-IDS) to efficiently detect attacks by offloading the learning process from servers to distributed vehicular edge nodes. FED-IDS introduces a context-aware transformer network to learn spatial-temporal representations of vehicular traffic flows necessary for classifying different categories of attacks. Blockchain-managed federated training is presented to enable multiple edge nodes to offer secure, distributed, and reliable training without the need for centralized authority. In the blockchain, miners confirm the distributed local updates from participating vehicles to stop unreliable updates from being deposited on the blockchain. The experiments on two public datasets (i.e., Car-Hacking, TON_IoT) demonstrated the efficiency of FED-IDS against state-of-the-art approaches. It reveals the credibility of securing networks of intelligent transportation systems against cyber-attacks.

The Industrial Internet of Things (IIoT) is creating a massive impact in a wide range of applications. In addition, with the forthcoming 5G and 6G technologies, vehicular ad-hoc networks will have pioneer advancements. However, security concerns are not well addressed, as vehicular networks should be deployed at a large scale. To address the security concerns, especially to ensure secure emergency message transmission, a blockchain-based protocol is proposed in this paper, where one of the blockchains is to store the authentication information of the vehicle, and another one to store and distribute blockchain services. Experimental analysis revealed that the proposed blockchain-based protocols are superior than the existing ones in terms of several metrics.

While recent works on investigating renewable energy sources for powering the highway offer promising solutions for sustainable environments, they are often impeded by unequal distribution of sources across the region due to variations in solar exposure and road intensity that electromagnetically and mechanically generate the energy. By exploiting viable gathering of massive renewable energy data using the Internet of Things (IoT), this paper proposes a framework for improved highway-energy management based on the unmanned aerial vehicle-assisted wireless energy re-distribution of the harvested renewable energy. Combining both massive low-rate sensing with high-speed 6G-envisioned transmission for data aggregation, the IoT architecture is of multi-scale, consisting of: i) global data exchange and analytics for energy mapping, re-distribution planning and forecasting, and ii) local data sensing and processing at individual highway lampposts for micro-energy management. The feasibility of the networked energy system is analyzed via analytical cost-reliability analyses. The cost analysis demonstrates the cost-effectiveness through the lowest Requirement of Energy and Cost of Energy for the setup and maintenance. The reliability analysis reveals the energy plus (E+) feature of the system in certain conditions with enhanced reliability in adverse weathers that impact energy generation. With multi-scale data connectivity to intelligently manage standalone renewable energy, this work puts forward a viable idea of 6G use cases with massively networked energy sensors with a vision of achieving super-connected and intelligence-equipped highways.

The dependency on vehicles is increasing tremendously due to its excellent transport capacity, fast, efficient, flexible, pleasant journey, minimal physical effort, and substantial economic impact. As a result, the demand for smart and intelligent feature enhancement is growing and becoming a prime concern for maximum productivity based on the current perspective. In this case, the Internet of Everything (IoE) is an emerging concept that can play an essential role in the automotive industry by integrating the stakeholders, process, data, and things via networked connections. But the unavailability of intelligent features leads to negligence about proper maintenance of vehicle vulnerable parts, reckless driving and severe accident, lack of instructive driving, and improper decision, which incurred extra expenses for maintenance besides hindering national economic growth. For this, we proposed a conceptual framework for a central VHMS exploiting IoE-driven Multi-Layer Heterogeneous Networks (HetNet) and a machine learning technique to oversee individual vehicle health conditions, notify the respective owner-driver real-timely and store the information for further necessary action. This article transparently portrayed an overview of central VHMS and proposed the taxonomy to achieve such an objective. Subsequently, we unveiled the framework for central VHMS, IoE-driven Multi-tire HetNet, with a secure and trustworthy data collection and analytics system. Finally, anticipating this proposition’s outcome is immense in the automotive sector. It may motivate the researcher to develop a central intelligent and secure vehicular condition diagnostic system to move this sector towards Industry 4.0.

The area of intelligent transport systems (ITS) is attracting growing attention because of the integration of the smart IoT with vehicles that improve user safety and overall travel experience. Vehicular ad hoc network (VANET) is the part of ITS; that deals with the routing protocols and security of smart vehicles. However, due to the rapid increase in the number of smart vehicles, the existing network technology’s resources unable to handle the traffic load. It expects that the 6G communication system has the ability to fulfill the requirements of VANETs. Only a few studies explore this area, but they also overlooked the security aspect of VANETs in 6G communications networks. In this paper, we present an approach to address authentication and security issues for vehicles in VANET. By authenticating cars in the VANET and identifying various cyber assaults such as DDoS, our method significantly contributes to the intelligent transport communication network. Our approach uses the concepts of identity-based encryption to provide access control to the vehicles and deep learning-based techniques for filtering malicious packets. Our identity-based encryption technique is IND-sID-CCA secure, and a state-of-the-art deep learning algorithm detects malicious packets with an accuracy of 99.72%. These results emphasize the validity of our proposed approach for VANETs in 6G communication systems.

Federated Learning (FL), as an emerging form of distributed machine learning (ML), can protect participants’ private data from being substantially disclosed to cyber adversaries. It has potential uses in many large-scale, data-rich environments, such as the Internet of Things (IoT), Industrial IoT, Social Media (SM), and the emerging SM 3.0. However, federated learning is susceptible to some forms of data leakage through model inversion attacks. Such attacks occur through the analysis of participants’ uploaded model updates. Model inversion attacks can reveal private data and potentially undermine some critical reasons for employing federated learning paradigms. This article proposes novel differential privacy (DP)-based deep federated learning framework. We theoretically prove that our framework can fulfill DP’s requirements under distinct privacy levels by appropriately adjusting scaled variances of Gaussian noise. We then develop a Differentially Private Data-Level Perturbation (DP-DLP) mechanism to conceal any single data point’s impact on the training phase. Experiments on real-world datasets, specifically the social media 3.0, Iris, and Human Activity Recognition (HAR) datasets, demonstrate that the proposed mechanism can offer high privacy, enhanced utility, and elevated efficiency. Consequently, it simplifies the development of various DP-based FL models with different tradeoff preferences on data utility and privacy levels.

Abstract The ability to explain why the model produced results in such a way is an important problem, especially in the medical domain. Model explainability is important for building trust by providing insight into the model prediction. However, most existing machine learning methods provide no explainability, which is worrying. For instance, in the task of automatic depression prediction, most machine learning models lead to predictions that are obscure to humans. In this work, we propose explainable Multi-Aspect Depression Detection with Hierarchical Attention Network MDHAN, for automatic detection of depressed users on social media and explain the model prediction. We have considered user posts augmented with additional features from Twitter. Specifically, we encode user posts using two levels of attention mechanisms applied at the tweet-level and word-level, calculate each tweet and words’ importance, and capture semantic sequence features from the user timelines (posts). Our hierarchical attention model is developed in such a way that it can capture patterns that leads to explainable results. Our experiments show that MDHAN outperforms several popular and robust baseline methods, demonstrating the effectiveness of combining deep learning with multi-aspect features. We also show that our model helps improve predictive performance when detecting depression in users who are posting messages publicly on social media. MDHAN achieves excellent performance and ensures adequate evidence to explain the prediction.

In today's world, databases are growing rapidly. Fast automatic detection of anomalous records in these massive databases is a challenging task. Traditional distance-based anomaly detectors are limited to small datasets because of their high time complexities. The univariate histogram-based method is arguably the fastest anomaly detection method. The anomaly score of a data instance is computed as the product of the probability mass of histograms in each dimension. Recent studies proved that such a simple method is comparable with many state-of-the-art methods on several datasets. However, as data features are assumed to be independent, it results in poor performance when features are correlated. Such an issue can be taken care of by using Principal Component (PC) features, which is the primary element of this paper. Our results show that integrating PCs with the original input features improves the performance of histogram-based anomaly detector with no real compromise in computational complexity.

Authorship attribution identifies the true author of an unknown document. Authorship attribution plays a crucial role in plagiarism detection and blackmailer identification, however, the existing studies on authorship attribution in Bengali are limited. In this paper, we propose an instance-based deep authorship attribution model, called DAAB, to identify authors in Bengali. Our DAAB model fuses features from convolutional neural networks and another set of features from an artificial neural network to learn the stylometry of an author for authorship attribution. Extensive experiments with three real benchmark datasets such as Bengali-Quora and two online Bengali Corpus demonstrate the superiority of our authorship attribution model.

Understanding customer behaviour (UCB) sequences with the multi-dimensional and temporal of the data is necessary for any competitive and global business aiming to provide interesting insights and to improve business strategies. While existing researchers have applied various data analytics approaches to understand and analyze behaviors of customer, they often failed to allow the analysts including business management, product marketing and development, and decision making, etc. Achieving these goals in collaboration with domain experts, we conducted a design study contributes to address a known problem with a novel solution and to provide data-driven visual decision support in collective policy data. We determine core study demands and then use a Visual Interactive System for Understanding Customer Behavior, named UCBVis that enables decision makers to gain detail insights into customer activities. In this study, we present customer behaviour pattern of multidimensional relationship through the visualisation system based on interweaving the pattern mining and querying with a designed encoding scheme. We use a large number of customer claim records and present visual outcomes to facilitate the exploration of customer behavior. Furthermore, we provide a concise set of insights and challenges associated with the use of UCBVis in the life insurance industry. We show the robustness of UCBVis through a user study with five participants shows that UCBVis is perceived to be more useful and provides actionable insights.

Sensitive and personal information theft is one of the biggest threats faces by enterprise networks. DNS is frequently used by sophisticated attackers to exfiltrate data over DNS queries, or facilitate command and control communications for malware in networks (i.e., tunneling). Commercial firewalls and intrusion detection systems (IDSs) seemingly have some capabilities to detect evolving attack vectors, but they are expensive and inflexible hardware solutions, yet incapable of offering advanced security features at high throughput. This paper develops and evaluates novel deep neural network-based DNS exfiltration using 1 Million benign domains and 1 Million DNS exfiltrated domains. We then compare its performance with state-of-the-art methods that show our deep learning-based framework outperforms with an accuracy of more than 99.9%.

Knowledge of the underlying anatomy of the left atrium can promote improved diagnostic protocols and clinical interventions; therefore, automatic segmentation of the left atrium on magnetic resonance imaging (MRI) can support diagnosis, treatment and surgery planning of the heart. Due to the small size of the left atrium with respect to the whole MRI volume, most of the current deep learning approaches are based on cropping or cascading networks. Dense V-Network is an encoder-decoder model designed for volumetric images by introducing a specialised dense feature stack to the standard V-Net model. A minor manipulation in parameters of the Dense V-Network can make it suitable for the fast and efficient segmentation of the left atrium. We present a brief review showing the ability of the Dense V-Network to deal with the issue of class imbalance and the unavailability of a large dataset of left atrium segmentation. We conclude that Dense V-Network can be tailored to left atrium MRI segmentation showing the accuracy that can surpass current methods, potentially supporting cardiac diagnosis and surgery.

In recent years, with the growing amount of biomedical documents, coupled with advancement in natural language processing algorithms, the research on biomedical named entity recognition (BioNER) has increased exponentially. However, BioNER research is challenging as NER in the biomedical domain are: (i) often restricted due to limited amount of training data, (ii) an entity can refer to multiple types and concepts depending on its context and, (iii) heavy reliance on acronyms that are sub-domain specific. Existing BioNER approaches often neglect these issues and directly adopt the state-of-the-art (SOTA) models trained in general corpora, which often yields unsatisfactory results. We propose biomedical ALBERT (A Lite Bidirectional Encoder Representations from Transformers for Biomedical Text Mining) - bioALBERT - an effective domain-specific pre-trained language model trained on a huge biomedical corpus designed to capture biomedical context-dependent NER. We adopted a self-supervised loss function used in ALBERT that targets modelling inter-sentence coherence to better learn context-dependent representations and incorporated parameter reduction strategies to minimise memory usage and enhance the training time in BioNER. In our experiments, BioALBERT outperformed comparative SOTA BioNER models on 8 biomedical NER benchmark datasets with 4 different entity types. The performance is increased for; (i) disease type corpora by 7.47% (NCBI- disease) and 10.63% (BC5CDR-disease); (ii) drug-chem type corpora by 4.61 % (BC5CDR-Chem) and 3.89% (BC4CHEMD); (iii) gene-protein type corpora by 12.25% (BC2GM) and 6.42% (JNLPBA); and (iv) species type corpora by 6.19% (LINNAEUS) and 23.71 % (Species-800) is observed which leads to a state-of-the-art results. The performance of a proposed model on four different biomedical entity types shows that our model is robust and generalisable in recognising biomedical entities in text.

Rip currents are intense and localized seaward-flowing water from shore through the surf zone, cutting through the lines of breaking ocean waves. It generally pulls a person out to sea very fast, with speeds up to eight feet per second. It is one of the most important reasons for an average of 21 confirmed fatalities each year. Fully automated Rip current detection using state-of-the-art deep learning techniques can help to monitor the coast. However, there are several challenges involved, like dealing with a small training sample size and unavailability of pretrained models in the domain of Rip current. In this work, we address these issues and propose a novel fast and lightweight RipDet framework for an efficient and accurate automated Rip current detection. The proposed model is aided by careful data augmentation, fine-tuning, and a custom learning rate schedule that helps the model adapt to Rip currents' distribution with the low number of training samples. Extensive experiments on benchmark data show that the proposed model outperforms other state-of-the-art methods for Rip current detection with mean average precision (mAP) score of 98.131 %.

Twitter is currently a popular online social media platform which allows users to share their user-generated content. This publicly-generated user data is also crucial to healthcare technologies because the discovered patterns would hugely benefit them in several ways. One of the applications is in automatically discovering mental health problems, e.g., depression. Previous studies to automatically detect a depressed user on online social media have largely relied upon the user behaviour and their linguistic patterns including user's social interactions. The downside is that these models are trained on several irrelevant content which might not be crucial towards detecting a depressed user. Besides, these content have a negative impact on the overall efficiency and effectiveness of the model. To overcome the shortcomings in the existing automatic depression detection methods, we propose a novel computational framework for automatic depression detection that initially selects relevant content through a hybrid extractive and abstractive summarization strategy on the sequence of all user tweets leading to a more fine-grained and relevant content. The content then goes to our novel deep learning framework comprising of a unified learning machinery comprising of Convolutional Neural Network (CNN) coupled with attention-enhanced Gated Recurrent Units (GRU) models leading to better empirical performance than existing strong baselines.

Abstract Data stored in physical storage or transferred over a communication channel includes substantial redundancy. Compression techniques cut down the data redundancy to reduce space and communication time. Nevertheless, compression techniques lack proper security measures, e.g., secret key control, leaving the data susceptible to attack. Data encryption is therefore needed to achieve data security in keeping the data unreadable and unaltered through a secret key. This work concentrates on the problems of data compression and encryption collectively without negatively affecting each other. Towards this end, an efficient, secure data compression technique is introduced, which provides cryptographic capabilities for use in combination with an adaptive Huffman coding, pseudorandom keystream generator, and S-Box to achieve confusion and diffusion properties of cryptography into the compression process and overcome the performance issues. Thus, compression is carried out according to a secret key such that the output will be both encrypted and compressed in a single step. The proposed work demonstrated a congruent fit for real-time implementation, providing robust encryption quality and acceptable compression capability. Experiment results are provided to show that the proposed technique is efficient and produces similar space-saving (%) to standard techniques. Security analysis discloses that the proposed technique is susceptible to the secret key and plaintext. Moreover, the ciphertexts produced by the proposed technique successfully passed all NIST tests, which confirm that the 99% confidence level on the randomness of the ciphertext.

Word representation has always been an important research area in the history of natural language processing (NLP). Understanding such complex text data is imperative, given that it is rich in information and can be used widely across various applications. In this survey, we explore different word representation models and its power of expression, from the classical to modern-day state-of-the-art word representation language models (LMS). We describe a variety of text representation methods, and model designs have blossomed in the context of NLP, including SOTA LMs. These models can transform large volumes of text into effective vector representations capturing the same semantic information. Further, such representations can be utilized by various machine learning (ML) algorithms for a variety of NLP-related tasks. In the end, this survey briefly discusses the commonly used ML- and DL-based classifiers, evaluation metrics, and the applications of these word embeddings in different NLP tasks.

Social media (and the world at large) have been awash with news of the COVID-19 pandemic. With the passage of time, news and awareness about COVID-19 spread like the pandemic itself, with an explosion of messages, updates, videos, and posts. Mass hysteria manifest as another concern in addition to the health risk that COVID-19 presented. Predictably, public panic soon followed, mostly due to misconceptions, a lack of information, or sometimes outright misinformation about COVID-19 and its impacts. It is thus timely and important to conduct anex post factoassessment of the early information flows during the pandemic on social media, as well as a case study of evolving public opinion on social media which is of general interest. This study aims to inform policy that can be applied to social media platforms; for example, determining what degree of moderation is necessary to curtail misinformation on social media. This study also analyzes views concerning COVID-19 by focusing on people who interact and share social media on Twitter. As a platform for our experiments, we present a new large-scale sentiment data set COVIDSENTI, which consists of 90 000 COVID-19-related tweets collected in the early stages of the pandemic, from February to March 2020. The tweets have been labeled into positive, negative, and neutral sentiment classes. We analyzed the collected tweets for sentiment classification using different sets of features and classifiers. Negative opinion played an important role in conditioning public sentiment, for instance, we observed that people favored lockdown earlier in the pandemic; however, as expected, sentiment shifted by mid-March. Our study supports the view that there is a need to develop a proactive and agile public health presence to combat the spread of negative sentiment on social media following a pandemic.

Since early 2020, the COVID-19 global pandemic has significantly impacted almost every aspect of the human society throughout the world. Until now, middle of 2021, although with all the efforts on pandemic intervention and vaccination, COVID-19 is still hovering around the world, resulting in more than 177 million confirmed cases and 3.8 million deaths.

Text in natural scene image portrays rich semantic information that plays an important role in content analysis. However, apart from Arabic text in documents, the text in natural scene images exhibit much higher diversity and variability, especially in uncontrolled circumstances. In this paper, a hybrid feature extraction approach is presented to detect extremal region of Arabic scene text. The binary image and image mask are considered as a variant of input image and look for concurrent extremal regions in both images. After determination of conjoined extremal points, the scale invariant technique is applied to consider those invariant points which are common in both images based on their coordinate positions. To evaluate the performance, a multidimensional long short term memory (LSTM) network is adapted and obtained 94.21% accuracy for word recognition on unconstrained Arabic scene text recognition (ASTR) dataset.

Right from the very beginning, the text has vital importance in human life. As compared to the vision-based applications, preference is always given to the precise and productive information embodied in the text. Considering the importance of text, recognition, and detection of text is also equally important in human life. This paper presents a deep analysis of recent development on scene text and compare their performance and bring into light the real modern applications. Future potential directions of scene text detection and recognition are also discussed.

Humans use tonal stress and gestural cues to reveal negative feelings that are expressed ironically using positive or intensified positive words when communicating vocally. However, in textual data, like posts on social media, cues on sentiment valence are absent, thus making it challenging to identify the true meaning of utterances, even for the human reader. For a given post, an intelligent natural language processing system should be able to identify whether a post is ironic/sarcastic or not. Recent work confirms the difficulty of detecting sarcastic/ironic posts. To overcome challenges involved in the identification of sentiment valence, this paper presents the identification of irony and sarcasm in social media posts through transformer-based deep, intelligent contextual embedding - T-DICE - which improves noise within contexts. It solves the language ambiguities such as polysemy, semantics, syntax, and words sentiments by integrating embeddings. T-DICE is then forwarded to attention-based Bidirectional Long Short Term Memory (BiLSTM) to find out the sentiment of a post. We report the classification performance of the proposed network on benchmark datasets for #irony & #sarcasm. Results demonstrate that our approach outperforms existing state-of-the-art methods.

Support Matrix Machines are one the efficient learning approach for the classification of complex nature data. However, either it can only deal with binary class problem or can deal with multi-class classification problem by breaking the problem into number of binary class problem and solving them individually or through solving larger optimization. Aiming to improve performance of support matrix machines, in this paper, we present Multi-class Support Matrix Machine based on evolutionary optimization (MSMM-CE) by breaking down the original multi-class problem of support matrix into sub-problems in cooperative fashion. The proposed objective function is a combination of binary hinge loss function for specific class, Frobenius and nuclear norms as a penalty that promote low rank and sparsity as well as an additional penalty term to penalize the multiclass classification error. The additional penalty term allow us to decompose the problem into sub-problems and solving them in simultaneously in cooperative fashion. The proposed objective function learns for each class and consider the information from other classes, that results in solving the problem in parallel. A comprehensive experimental study on publicly available benchmark EEG dataset is carried out to investigate the proposed approach that confirms the superiority of MSMM-CE for accurate classification of EEG signal associated with motor imagery in BCI applications. MSMM-CE provides a generalized solution to investigate the complex and nonlinear high dimensional data for various real-world applications.

Traditional one class support tensor machine (OCSTM) is a popular classifier that is widely adopted for one class classification, however, outliers in the data negatively affects its performance. To improve the robustness of OCSTM against outliers, in this paper, we present OCSTM with bounded loss function rather than finding optimized support vectors with unbounded loss function. To solve the corresponding optimization problem, we have presented half quadratic optimization to drive the problem to traditional OCSTM, followed by solving a typical OCSTM optimization problem iteratively. We further demonstrate our algorithms through experiments on eight real-world benchmark datasets. Experimental results show that the proposed approach separates well most of the samples of interested class from origin even in the presence of outliers.

Parkinson's is not just an older person's disease. Although, it is typically diagnosed around the age of 60 or later, however, usually the symptoms of Parkinson's may start at 50 years old or even earlier. Usually, the symptoms of Parkinson's appears when approximately 70% of the dopamine-producing cells have stopped working normally. Evidence suggests that early diagnosis of Parkinson's disease leads to an effective path to manage its symptoms efficiently. However, the diagnosis of Parkinson's is challenging, especially in its early stage. Handwriting abnormalities are majorly used for diagnosis of Parkinson's disease (PD) have handwriting abnormalities. Selecting a writing task can affect the diagnosis performance. In this paper, we analyzed different writing task for Parkinson patient to select an optimal writing task that could help to improve the diagnosis performance. We have used transfer learning based strategies and used different methods to select an optimal writing task. Extensive evaluation of different writing styles showed that complex task are much better in diagnosis of Parkinson disease in comparison to simple task, however, further research is warranted to gain better insight into analysis of complex task.

Automatic classification of skin disease is a challenging task due to high similarity among them as well as complex nature of skin images i.e. hair, skin and nails and illumination conditions. It is one of the most common malignancy that becomes more difficult to treat and potential disfigurement or even death, if not diagnosed early. In this paper, we present a fully automatic skin lesion classification by leveraging three stage deep residual neural networks that is trained end-to-end and does not rely on prior knowledge of the data. Extensive experiment on ISIC-2018 challenges shows that our methods showed considerable improvement (96.07%) in the classification of skin melanoma by not only super-passing the ISIC-2018 challenges as well state of the art methods.

One out of five plants are threatened as evident from the IUCN Red List data. Such high rates of loss in plant species triggered to protect and conserve biodiversity. It needs extremely high identification skills obtained via intensive training and experience, even for experienced botanists it is sometimes impossible to provide a definite identification based on a single image. Automatic identification of plant species in natural scene images is one of the important however challenging research problem with various applications in the field of agriculture and botany. Recently, state-of-the-art Deep Convolutional Neural Networks have been applied to classify different species of plants however, they still suffer due to the complexity of the plant images. In this paper, we present multi-path multi deep convolutional network for the identification of plant species which feeds different versions of plant images, thus resultant model has better image presentation than traditional CNN. Comprehensive experimental evaluation on benchmark plant datasets showed that without using any pre-trained models, our proposed shallow network demonstrate very competitive performance for plant species identification. The experimental results proved that the multi-path multi CNN are highly effective for learning discriminative features.

Abstract Developing cursive script recognition systems have always been a challenging task for researchers. This article proposes a ligature‐based recognition system for the cursive Pashto script using four pre‐trained CNN models using a fine‐tuned approach. The SqueezeNet, ResNet, MobileNet and DenseNet models have been observed for the classification and the recognition of Pashto sub‐word (ligature). Overall, the proposed system is divided into two domains (Source and Target). The source domain contains the pre‐trained models used on the ImageNet Dataset. These models are later fine‐tuned using the transfer learning approach to be used for the Pashto ligature recognition. The data augmentation techniques of negative and contour are used to increase the representation of ligature images and the dataset size. The CNN models have been evaluated on the benchmarks Pashto ligatures FAST‐NU dataset. The proposed system achieved the highest recognition rate of up to 99.31% using the DenseNet architecture of Convolutional Neural Network for Pashto ligature.

Manual segmentation of the brain tumors for cancer diagnosis from MRI images is a difficult, tedious, and time-consuming task. The accuracy and the robustness of brain tumor segmentation, therefore, are crucial for the diagnosis, treatment planning, and treatment outcome evaluation. Mostly, the automatic brain tumor segmentation methods use hand designed features. Similarly, traditional methods of deep learning such as convolutional neural networks require a large amount of annotated data to learn from, which is often difficult to obtain in the medical domain. Here, we describe a new model two-pathway-group CNN architecture for brain tumor segmentation, which exploits local features and global contextual features simultaneously. This model enforces equivariance in the two-pathway CNN model to reduce instabilities and overfitting parameter sharing. Finally, we embed the cascade architecture into two-pathway-group CNN in which the output of a basic CNN is treated as an additional source and concatenated at the last layer. Validation of the model on BRATS2013 and BRATS2015 data sets revealed that embedding of a group CNN into a two pathway architecture improved the overall performance over the currently published state-of-the-art while computational complexity remains attractive.

Principal component analysis (PCA) is widely used methods for dimensionality reduction and Lots of variants have been proposed to improve the robustness of algorithm, however, these methods suffer from the fact that PCA is linear combination which makes it difficult to interpret complex nonlinear data, and sensitive to outliers or cannot extract features consistently, i.e., collectively; PCA may still require measuring all input features. 2DPCA based on ℓ1-normhas been recently used for robust dimensionality reduction in the image domain but still sensitive to noise. In this paper, we introduce robust formation of 2DPCA by centering the data using the optimized mean for two-dimensional joint sparse as well as effectively combining the robustness of 2DPCA and the sparsity-inducing lasso regularization. Optimal mean helps to improve the robustness of joint sparse PCA further. The distance in spatial dimension is measure in F-norm and sum of different datapoint uses 1-norm. 2DR-JSPCA imposes joint sparse constraints on its objective function whereas additional plenty term help to deal with outliers efficiently. Both theoretical and empirical results on six publicly available benchmark datasets shows that Optimal mean 2DR-JSPCA provides better performance for dimensionality reduction as compare to non-sparse (2DPCA and 2DPCA-L1) and sparse (SPCA, JSPCA).

Recognition of Urdu cursive script is a challenging task due to the implicit complexities associated with it. The performance of a recognition system is immensely dependent on extracted features. There are various features extraction approaches proposed in recent years. Among many, an approach based on zoning features proved to be efficient and popular. Such zoning features represent significant information with low complexity and high speed. In this paper, we used zoning features for the classification of Urdu Nasta’liq text lines, with a combination of 2-Dimensional Long Short Term Memory networks (2DLSTM) as learning classifier. The proposed model is evaluated on publicly available UPTI dataset and character recognition rate of 93.39% is obtained.

Health Information technology (HIT) played important role in wide range of healthcare settings. The strategy of HIT is to reduce cost and increase efficiency. The aim of cardiac implantable devices is to reduce the effect and risk of cardiovascular diseases. The number of patients that benefits from these devices are growing rapidly. Cardiac rehabilitation program improves the ability of cardiac patient to restore his health to an optimal level and reduce his stress and depression, especially after the cardiovascular surgeries. The need for the use of ontologies in cardiac rehabilitation domain has emerged over the last years, in order to improve the standardization of cardiac rehabilitation's practice worldwide. In this study, ontology-based system for managing the cardiac rehabilitation program has been developed. The ontology presents the required functionalities to implement the program management that helps to enhance the quality of cardiac rehabilitation program. Moreover, proposed ontology contributes to enhance the understanding, reusability, sharing and transferring knowledge in the domain of cardiac rehabilitation.