Abstract Accurate brain tumor classification via MRI is essential for diagnosis and treatment. This study introduces RViT-FusionNet, a hybrid deep learning model that integrates convolutional and transformer architectures for enhanced tumor detection. The model utilizes ResNet-50 to capture textural details and a Vision Transformer for extracting global context. A Local Cross-Attention (LCA) module is proposed to align and merge these features, allowing the network to model local structures and long-range dependencies concurrently. To enhance generalization across varied imaging conditions and tumor types, a domain discriminator is included to discern spatial and domain-specific patterns, fostering the learning of domain-invariant representations. The approach is validated on four public MRI datasets, yielding classification accuracies of 99.08% ± 0.16%, 99.56% ± 0.17%, 96.20% ± 0.25% and 94.76% ± 0.35% for glioma, meningioma, pituitary tumors, and healthy cases, respectively. For interpretability, Grad-CAM is utilized to create saliency maps highlighting tumor regions, confirming the model’s focus on clinically relevant areas. These findings demonstrate that RViT-FusionNet achieves exceptional performance while maintaining high interpretability, positioning it as an effective tool for computer-assisted brain tumor diagnosis. This framework is posited as a robust and scalable solution for multi-class brain tumor classification in clinical practice. The code is available at https://github.com/naimaislam-ice/RViT-FusionNet-LCA-based-brain-tumor-classification .

Abstract Integrating machine learning techniques into medical diagnostic systems holds great promise for enhancing disease identification and treatment. Among the various options for training such systems, the extreme learning machine (ELM) stands out due to its rapid learning capability and computational efficiency. However, the random selection of input weights and hidden neuron biases in the ELM can lead to suboptimal performance. To address this issue, our study introduces a novel approach called modified Harris hawks optimizer (MHHO) to optimize these parameters in ELM for medical classification tasks. By applying the MHHO‐based method to seven medical datasets, our experimental results demonstrate its superiority over seven other evolutionary‐based ELM trainer models. The findings strongly suggest that the MHHO approach can serve as a valuable tool for enhancing the performance of ELM in medical diagnosis.

Abstract Multimodal Human Action Recognition (MHAR) is an important research topic in computer vision and event recognition fields. In this work, we address the problem of MHAR by developing a novel audio-image and video fusion-based deep learning framework that we call Multimodal Audio-Image and Video Action Recognizer (MAiVAR). We extract temporal information using image representations of audio signals and spatial information from video modality with the help of Convolutional Neutral Networks (CNN)-based feature extractors and fuse these features to recognize respective action classes. We apply a high-level weights assignment algorithm for improving audio-visual interaction and convergence. This proposed fusion-based framework utilizes the influence of audio and video feature maps and uses them to classify an action. Compared with state-of-the-art audio-visual MHAR techniques, the proposed approach features a simpler yet more accurate and more generalizable architecture, one that performs better with different audio-image representations. The system achieves an accuracy 87.9% and 79.0% on UCF51 and Kinetics Sounds datasets, respectively. All code and models for this paper will be available at https://tinyurl.com/4ps2ux6n .

Due to its widespread applications, human action recognition is one of the most widely studied research problems in Computer Vision. Recent studies have shown that addressing it using multimodal data leads to superior performance as compared to relying on a single data modality. During the adoption of deep learning for visual modelling in the past decade, action recognition approaches have mainly relied on Convolutional Neural Networks (CNNs). However, the recent rise of Transformers in visual modelling is now also causing a paradigm shift for the action recognition task. This survey captures this transition while focusing on Multimodal Human Action Recognition (MHAR). Unique to the induction of multimodal computational models is the process of ‘fusing’ the features of the individual data modalities. Hence, we specifically focus on the fusion design aspects of the MHAR approaches. We analyze the classic and emerging techniques in this regard, while also highlighting the popular trends in the adaption of CNN and Transformer building blocks for the overall problem. In particular, we emphasize on recent design choices that have led to more efficient MHAR models. Unlike existing reviews, which discuss Human Action Recognition from a broad perspective, this survey is specifically aimed at pushing the boundaries of MHAR research by identifying promising architectural and fusion design choices to train practicable models. We also provide an outlook of the multimodal datasets from their scale and evaluation viewpoint. Finally, building on the reviewed literature, we discuss the challenges and future avenues for MHAR.

Technological advances in medical imaging are enabling us to understand healthcare datasets in great detail. Machine Learning enabled methods, specifically, deep neural networks are continuously achieving benchmark performances in terms of accuracy and computational efficiency. However, the lack of agreed-upon standard procedures, variations in the devices by different vendors, and artifacts induced by the physical phenomenon in the sensors make the data inconsistent and noisy. These variations in the data are detrimental to the performance of learning-based methods. In this study, we analyze the behavior of traditional and deep learning-based image registration methods and explore strategies to handle the problem of intensity distributional shifts without compromising the performance. To achieve this, we propose an intensity-based loss function and demonstrate that the models trained with our proposed loss function are better at handling unseen data from different sites using machines from different vendors. In addition, our trained model is superior in preserving the boundaries of anatomical regions after registration.

Currently, action recognition is predominately performed on video data as processed by CNNs. We investigate if the representation process of CNN s can also be leveraged for multimodal action recognition by incorporating image-based audio representations of actions in a task. To this end, we propose Multimodal Audio-Image and Video Action Recognizer (MAiVAR), a CNN-based audio-image to video fusion model that accounts for video and audio modalities to achieve superior action recognition performance. MAiVAR extracts meaningful image representations of audio and fuses it with video representation to achieve better performance as compared to both modalities individually on a large-scale action recognition dataset.

COVID-19 classification using chest Computed Tomography (CT) has been found pragmatically useful by several studies. Due to the lack of annotated samples, these studies recommend transfer learning and explore the choices of pre-trained models and data augmentation. However, it is still unknown if there are better strategies than vanilla transfer learning for more accurate COVID-19 classification with limited CT data. This paper provides an affirmative answer, devising a novel ‘model’ augmentation technique that allows a considerable performance boost to transfer learning for the task. Our method systematically reduces the distributional shift between the source and target domains and considers augmenting deep learning with complementary representation learning techniques. We establish the efficacy of our method with publicly available datasets and models, along with identifying contrasting observations in the previous studies.

Deep learning has provided numerous breakthroughs in natural imaging tasks. However, its successful application to medical images is severely handicapped with the limited amount of annotated training data. Transfer learning is commonly adopted for the medical imaging tasks. However, a large covariant shift between the source domain of natural images and target domain of medical images results in poor transfer learning. Moreover, scarcity of annotated data for the medical imaging tasks causes further problems for effective transfer learning. To address these problems, we develop an augmented ensemble transfer learning technique that leads to significant performance gain over the conventional transfer learning. Our technique uses an ensemble of deep learning models, where the architecture of each network is modified with extra layers to account for dimensionality change between the images of source and target data domains. Moreover, the model is hierarchically tuned to the target domain with augmented training data. Along with the network ensemble, we also utilize an ensemble of dictionaries that are based on features extracted from the augmented models. The dictionary ensemble provides an additional performance boost to our method. We first establish the effectiveness of our technique with the challenging ChestXray-14 radiography data set. Our experimental results show more than 50% reduction in the error rate with our method as compared to the baseline transfer learning technique. We then apply our technique to a recent COVID-19 data set for binary and multi-class classification tasks. Our technique achieves 99.49% accuracy for the binary classification, and 99.24% for multi-class classification.

Effective business planning requires seamless access and intelligent analysis of information in its totality to allow the business planner to gain enhanced critical business insights for decision support. Current business planning tools provide insights from structured business data (i.e. sales forecasts, customers and products data, inventory details) only and fail to take into account unstructured complementary information residing in contracts, reports, user's comments, emails etc. In this article, a planning support system is designed and developed that empower business planners to develop and revise business plans utilizing both structured data and unstructured information conjointly. This planning system activity model comprises of two steps. Firstly, a business planner develops a candidate plan using planning template. Secondly, the candidate plan is put forward to collaborating partners for its revision interleaving deliberation. Planning interleaving deliberation activity in the proposed framework enables collaborating planners to challenge both a decision and the thinking that underpins the decision in the candidate plan. The planning system is modeled using situation calculus and is validated through a prototype development.

Ear detection from a profile face image is an important step in many applications including biometric recognition. But accurate and rapid detection of the ear for real-time applications is a challenging task, particularly in the presence of occlusions. In this work, a cascaded AdaBoost based ear detection approach is proposed. In an experiment with a test set of 203 profile face images, all the ears were accurately detected by the proposed detector with a very low (5 × 10-6) false positive rate. It is also very fast and relatively robust to the presence of occlusions and degradation of the ear images (e.g. motion blur). The detection process is fully automatic and does not require any manual intervention.