Advancements in Computer-Aided Screening (CAS) systems are essential for improving the detection of security threats in X-ray baggage scans. However, current datasets are limited in representing real-world, sophisticated threats and concealment tactics, and existing approaches are constrained by a closed-set paradigm with predefined labels. To address these challenges, we introduce STCray, the first multimodal X-ray baggage security dataset, comprising 46,642 image-caption paired scans across 21 threat categories, generated using an X-ray scanner for airport security. STCray is meticulously developed with our specialized protocol that ensures domain-aware, coherent captions, that lead to the multi-modal instruction following data in X-ray baggage security. This allows us to train a domain-aware visual AI assistant named STING-BEE that supports a range of vision-language tasks, including scene comprehension, referring threat localization, visual grounding, and visual question answering (VQA), establishing novel baselines for multi-modal learning in X-ray baggage security. Further, STING-BEE shows state-of-the-art generalization in cross-domain settings. Code, data, and models are available at https://divs1159.github.io/STING-BEE/.

Retinopathy is a group of retinal disabilities that causes severe visual impairments or complete blindness. Due to the capability of optical coherence tomography to reveal early retinal abnormalities, many researchers have utilized it to develop autonomous retinal screening systems. However, to the best of our knowledge, most of these systems rely only on mathematical features, which might not be helpful to clinicians since they do not encompass the clinical manifestations of screening the underlying diseases. Such clinical manifestations are critically important to be considered within the autonomous screening systems to match the grading of ophthalmologists within the clinical settings. To overcome these limitations, we present a novel framework that exploits the fusion of vision language correlation between the retinal imagery and the set of clinical prompts to recognize the different types of retinal disabilities. The proposed framework is rigorously tested on six public datasets, where, across each dataset, the proposed framework outperformed state-of-the-art methods in various metrics. Moreover, the clinical significance of the proposed framework is also tested under strict blind testing experiments, where the proposed system achieved a statistically significant correlation coefficient of 0.9185 and 0.9529 with the two expert clinicians. These blind test experiments highlight the potential of the proposed framework to be deployed in the real world for accurate screening of retinal diseases.

This article addresses the crucial aspect of safety in collaborative robotics by introducing a new continuous wavelet transform-convolutional neural network (CWT-CNN) for efficient robot collision detection. Unlike conventional methods, CWT-CNN exhibits superior data efficiency, requiring minimal collision data for robust training without relying on a dynamic model. The network’s adaptability extends to varying internal stiffness levels, offering robustness to changes in robotic system characteristics. Through comprehensive experimental studies, we investigate the impact of input signal types, wavelet types, wavelet scale ranges, and time-moving window sizes on collision detection performance, offering critical insights for optimal CWT parameter selection. Additionally, our transferability analysis demonstrates that the CWT-CNN can seamlessly adapt from one joint to another, requiring only minimal free-motion data from the new joint. This adaptability is validated through extensive experiments on an industrial robot and the robot equipped with variable stiffness actuators. In conclusion, the CWT-CNN is highly generalizable and data-efficient, making it a reliable solution for real-time collision detection in human-robot interactions, addressing a key aspect of safety in collaborative environments.

Human activity recognition is a crucial area of research that involves understanding human movements using computer and machine vision technology. Deep learning has emerged as a powerful tool for this task, with models such as Convolutional Neural Networks (CNNs) and Transformers being employed to capture various aspects of human motion. One of the key contributions of this work is the demonstration of the effectiveness of feature fusion in improving human activity recognition accuracy, which has important implications for the development of more accurate and robust activity recognition systems. This approach addresses a limitation in the field, where the performance of existing models is often limited by their inability to capture both spatial and temporal features effectively. This work presents an approach for human activity recognition using sensory data extracted from four distinct datasets: HuGaDB, PKU-MMD, LARa, and TUG. Two models, the Parameter-Optimized Multi-Stage Graph Convolutional Network (PO-MS-GCN) and a Transformer, were trained and evaluated on each dataset to calculate accuracy and F1-score. Subsequently, the features from the last layer of each model were combined and fed into a classifier. The findings prove that PO MS-GCN outperforms state-of-the-art models in human activity recognition. Specifically, HuGaDB achieved an accuracy of 92.7% and f1-score of 95.2%, TUG achieved an accuracy of 93.2% and f1-score of 98.3%, while LARa and PKU-MMD achieved lower accuracies of 64.31% and 69%, respectively, with corresponding f1-scores of 40.63% and 48.16%. Moreover, feature fusion exceeded the PO-MS-GCN’s results in PKU-MMD, LARa, and TUG datasets.

This study presents a performance comparison of different deep machine learning models for the intelligent segregation of date fruit bunches of numerous varieties. The shape, size, and color of the various types of dates with different maturity conditions require experienced farmers to estimate the accrual class. This study has used a transfer learning approach on various machine learning models (VGG-19, MobileNetV2, DenseNet, and NASNet) to classify five different kinds of date bunches (Barhi, Khalas, Menifi, Nabout saif, and Sullaj). Each model is trained on a dataset of 11,944 images of five different date classes. The comparison deduces that DenseNet produces the best accuracy, precision, and recall when compared with the performance of other models. The maximum achieved accuracy of the trained DenseNet model is $100 \%$ for the validation dataset and 98.4% for the test dataset.

This paper presents a thorough review of recent advancements in screening heart and lung diseases via auscultation using artificial intelligence (AI) methods. Auscultation has historically been fundamental in diagnosing cardiopulmonary conditions; however, conventional techniques depend significantly on clinician proficiency, rendering diagnosis vulnerable to human error. Recent advancements in digital stethoscopes and AI-based sound analysis algorithms have transformed the conventional analysis, facilitating more precise, real-time identification of anomalies such as murmurs, arrhythmias, wheezes, and crackles. This paper delineates the principal methodologies employed in sound acquisition, feature extraction, and disease classification, while assessing the reliability of diverse models of machine learning and deep learning. Moreover, the paper addresses the obstacles in implementing these technologies in clinical practice, including data standardization, computational constraints, and integration with current healthcare systems. The results indicate that AI-augmented stethoscope systems have significant potential to enhance early diagnosis and patient outcomes for cardiac and pulmonary conditions.

Baggage screening in airports is a cornerstone in airport security measures. The advent of computer vision technologies in recent years has led to the development of several automated systems for identifying security threats in baggage scans. However, existing methods struggle to adapt to new threat categories when faced with a scarcity of data samples, and the rapid emergence of new threats. Hence, in this paper, we propose a novel CLIP-driven few-shot framework (CLIFS) to explore the potential of multi-modality using text-image fusion through contrastive learning to learn relevant contextual features for recognizing security threats with limited samples. By integrating features from GPT-4 generated captions with image features, CLIFS leverages both visual and textual data to significantly improve threat classification performance with limited samples in a few-shot learning context. Our proposed CLIFS was rigorously tested on the SIXray public available baggage X-ray dataset, where it outperformed state-of-the-art by 31.3% in accuracy and 28.40% in F1-score for the challenging 5-shots scenario, demonstrating its robustness and effectiveness in classifying threats from limited data samples.

Recent advancements in artificial intelligence algorithms for medical imaging show significant potential in automating the detection of lung infections from chest radiograph scans. However, current approaches often focus solely on either 2-D or 3-D scans, failing to leverage the combined advantages of both modalities. Moreover, conventional slice-based methods place a manual burden on radiologists for slice selection. To overcome these challenges, we propose the Recurrent 3-D Multi-level Vision Transformer (R3DM-ViT) model, capable of handling multimodal data to enhance diagnostic accuracy. Our quantitative evaluations demonstrate that R3DM-ViT surpasses existing methods, achieving an impressive accuracy of $96.67 \%$, F1-score of $96.88 \%$, mean average precision of $96.75 \%$, and mean average recall of $97.02 \%$. This research signifies a significant stride forward in the automated detection of lung infections through multimodal imaging.

Autonomous X-ray baggage security screening has shown significant strides recently, proving itself a viable solution to the flaws in manual screening, thanks to advancements in deep learning. However, these data-hungry techniques feed on extensively annotated data involving strenuous labor, impeding their advances in baggage screening. Consequently, we present a context-aware transformer for weakly supervised localization to relieve the annotation burden and provide visual interpretability that aids screeners in threat recognition and researchers in identifying the pitfalls of existing systems. The proposed approach can generalize and localize different types of contraband with only cost-effective binary labels without explicit training on item detection. Context extraction block, integrated into the dual-token framework, generates threat-aware context maps, while the token scoring block focuses on minimizing partial activations. Experimental results surpass state of the art (SOTA) methods in terms of classification and localization accuracies. Furthermore, we analyze failures to determine current vulnerabilities and provide new insights for future research.

Accurate classification of nuclei communities is an important step towards timely treating the cancer spread. Graph theory provides an elegant way to represent and analyze nuclei communities within the histopathological landscape in order to perform tissue phenotyping and tumor profiling tasks. Many researchers have worked on recognizing nuclei regions within the histology images in order to grade cancerous progression. However, due to the high structural similarities between nuclei communities, defining a model that can accurately differentiate between nuclei pathological patterns still needs to be solved. To surmount this challenge, we present a novel approach, dubbed neural graph refinement, that enhances the capabilities of existing models to perform nuclei recognition tasks by employing graph representational learning and broadcasting processes. Based on the physical interaction of the nuclei, we first construct a fully connected graph in which nodes represent nuclei and adjacent nodes are connected to each other via an undirected edge. For each edge and node pair, appearance and geometric features are computed and are then utilized for generating the neural graph embeddings. These embeddings are used for diffusing contextual information to the neighboring nodes, all along a path traversing the whole graph to infer global information over an entire nuclei network and predict pathologically meaningful communities. Through rigorous evaluation of the proposed scheme across four public datasets, we showcase that learning such communities through neural graph refinement produces better results that outperform state-of-the-art methods.

Early-stage cancer diagnosis potentially improves the chances of survival for many cancer patients worldwide. Manual examination of Whole Slide Images (WSIs) is a time-consuming task for analyzing tumor-microenvironment. To overcome this limitation, the conjunction of deep learning with computational pathology has been proposed to assist pathologists in efficiently prognosing the cancerous spread. Nevertheless, the existing deep learning methods are ill-equipped to handle fine-grained histopathology datasets. This is because these models are constrained via conventional softmax loss function, which cannot expose them to learn distinct representational embeddings of the similarly textured WSIs containing an imbalanced data distribution. To address this problem, we propose a novel center-focused affinity loss (CFAL) function that exhibits 1) constructing uniformly distributed class prototypes in the feature space, 2) penalizing difficult samples, 3) minimizing intra-class variations, and 4) placing greater emphasis on learning minority class features. We evaluated the performance of the proposed CFAL loss function on two publicly available breast and colon cancer datasets having varying levels of imbalanced classes. The proposed CFAL function shows better discrimination abilities as compared to the popular loss functions such as ArcFace, CosFace, and Focal loss. Moreover, it outperforms several SOTA methods for histology image classification across both datasets.

Classifying nuclei communities in histology images is vital for early cancer treatment, but it remains challenging due to the similar structure of nuclei communities. To address this, we propose an iterative neural graph improvement and broadcasting approach. A fully connected graph is constructed with nuclei as nodes starting with a baseline classification. Node and edge features are updated and exchanged along a Hamiltonian path, removing weak connections. This process filters communities by disconnecting weakly connected nodes and iterates until stability is reached. Loose nodes from this refining stage are then assigned to their closest community clusters. Experimental results on two public datasets demonstrate the superiority of the proposed approach over state-of-the-art methods.

The emerging deep learning algorithms have shown significant potential in the development of efficient computer-aided diagnosis tools for automated detection of lung infections using chest radiographs. However, many existing methods are slice-based and require manual selection of appropriate slices from the entire CT scan, which is tedious and requires expert radiologists. To overcome these limitations, we propose a recurrent 3D Inception network (R3DI-Net) that sequentially exploits spatial and 3D structural features of the entire CT scan, ultimately leading to improved diagnostic performance. Additionally, the proposed method flexibly handles input CT scans with a variable number of slices without incurring performance degradation. A quantitative evaluation of R3DI-Net was made using a combined collection of three publicly accessible datasets containing a sufficient number of data samples. Our method outperforms various existing methods by achieving remarkable performances of 98.39%, 98.36%, 98.1%, and 98.64% in terms of accuracy, F1-score, sensitivity, and average precision, respectively.

Recent advances in deep learning have facilitated significant progress in the autonomous detection of concealed security threats from baggage X-ray scans, a plausible solution to overcome the pitfalls of manual screening. However, these data-hungry schemes rely on extensive instance-level annotations that involve strenuous skilled labor. Hence, this paper proposes a context-aware transformer for weakly supervised baggage threat localization, exploiting their inherent capacity to learn long-range semantic relations to capture the object-level context of the illegal items. Unlike the conventional single-class token transformers, the proposed dual-token architecture can generalize well to different threat categories by learning the threat-specific semantics from the token-wise attention to generate context maps. The framework has been evaluated on two public datasets, Compass-XP and SIXray, and surpassed other SOTA approaches.

Visual perception is an important component for autonomous navigation of unmanned surface vessels (USV), particularly for the tasks related to autonomous inspection and tracking. These tasks involve vision-based navigation techniques to identify the target for navigation. Reduced visibility under extreme weather conditions in marine environments makes it difficult for vision-based approaches to work properly. To overcome these issues, this paper presents an autonomous vision-based navigation framework for tracking target objects in extreme marine conditions. The proposed framework consists of an integrated perception pipeline that uses a generative adversarial network (GAN) to remove noise and highlight the object features before passing them to the object detector (i.e., YOLOv5). The detected visual features are then used by the USV to track the target. The proposed framework has been thoroughly tested in simulation under extremely reduced visibility due to sandstorms and fog. The results are compared with state-of-the-art de-hazing methods across the benchmarked MBZIRC simulation dataset, on which the proposed scheme has outperformed the existing methods across various metrics.

Autonomous detection of threat items from baggage X-ray imagery is one of the most vital and challenging tasks. Manual detection of these items is a cumbersome, slow, and error-ridden process which is also limited by the examination capacity of the security inspector. To overcome these limitations, many researchers have proposed deep learning-driven approaches to recognize suspicious objects from the baggage X-ray scans. However, threat items are rarely seen in the real world compared to innocuous baggage content. Therefore, when trained with imbalanced data, the performance of the conventional threat detection models drastically decreases. This paper addresses these issues with a contour-driven instance segmentation model optimized with a novel combined loss function, dubbed balanced affinity loss function. In addition to mitigating the class imbalance, this function best handles the fine-grained classification aspect inferred by contours and the instance segmentation. We validated the proposed system on three public baggage X-ray datasets, where it outperformed state-of-the-art methods by 7.76%, 25.81%, and 8.78% in terms of intersection-over-union score.

Unlicensed cognitive-radio (CR)-assisted Internet of Vehicles (IoV) users can access licensed providers’ radio spectrum and concurrently utilize the dedicated channel for data transmission in vehicular communication. Optimizing channel access in cognitive IoV networks can help maximize available spectrum resources. This article proposes a novel sensing and communication integrated framework, dubbed as the CR-assisted IoV network (CRAV-Net), using a cluster-based hybrid optimization approach with adaptive congestion-aware modeling for dynamic high-mobility vehicular networks in an urban city context. In CRAV-Net, intelligent hybrid learning spectrum agents are introduced, which perform spectrum sensing (SS) using a deep learning (DL) model. It dynamically learns the multilevel spatial and temporal graphical features from input spectrograms through layer-by-layer propagation. It efficiently predicts the spectrum occupancy in the primary spectrum, without a priori knowledge of the radio environment. Then, to assign the vacant channels to the secondary vehicles, a support vector machine classifier is trained based on several learning features, including the vehicle stay time, vehicle density, and network capacity, to select the optimal resource route. The proposed framework achieves an overall accuracy of 99.74% in SS using the custom data set, outperforming state of the art by 12.60% at −25-dB signal-to-noise ratio. In addition, it brings a performance gain of 0.81% in SS accuracy when evaluated on real-world signals. Furthermore, in optimal network node allocation, the proposed framework achieves a mean accuracy of 98.45%, outperforming the existing methods by 0.63% and 18.32% in terms of accuracy and allocation time, respectively.

Screening cluttered and occluded contraband items from baggage X-ray scans is a cumbersome task even for the expert security staff. This article presents a novel strategy that extends a conventional encoder-decoder architecture to perform instance-aware segmentation and extract merged instances of contraband items without using any additional subnetwork or an object detector. The encoder-decoder network first performs conventional semantic segmentation and retrieves cluttered baggage items. The model then incrementally evolves during training to recognize individual instances using significantly reduced training batches. To avoid catastrophic forgetting, a novel objective function minimizes the network loss in each iteration by retaining the previously acquired knowledge while learning new class representations and resolving their complex structural interdependencies through Bayesian inference. A thorough evaluation of our framework on two publicly available X-ray datasets shows that it outperforms state-of-the-art methods, especially within the challenging cluttered scenarios, while achieving an optimal tradeoff between detection accuracy and efficiency.

Detection of illegal and threatening items in bag-gage is one of the utmost security concern nowadays. Even for experienced security personnel, manual detection is a time-consuming and stressful task.Many academics have created automated frameworks for detecting suspicious and contraband data from X-ray scans of luggage. However, to our knowledge, no framework exists that utilizes temporal baggage X-ray imagery to effectively screen highly concealed and occluded objects which are barely visible even to the naked eye. To address this, we present a novel temporal fusion driven multi-scale residual fashioned encoder-decoder that takes series of consecutive scans as input and fuses them to generate distinct feature representations of the suspicious and non-suspicious baggage content, leading towards a more accurate extraction of the contraband data. The proposed methodology has been thoroughly tested using the publicly accessible GDXray dataset, which is the only dataset containing temporally linked grayscale X-ray scans showcasing extremely concealed contraband data. The proposed framework outperforms its competitors on the GDXray dataset on various metrics.

The identification of retinal lesions plays a vital role in accurately classifying and grading retinopathy. Many researchers have presented studies on optical coherence tomography (OCT) based retinal image analysis over the past. However, to the best of our knowledge, there is no framework yet available that can extract retinal lesions from multi-vendor OCT scans and utilize them for the intuitive severity grading of the human retina. To cater this lack, we propose a deep retinal analysis and grading framework (RAG-FW). RAG-FW is a hybrid convolutional framework that extracts multiple retinal lesions from OCT scans and utilizes them for lesion-influenced grading of retinopathy as per the clinical standards. RAG-FW has been rigorously tested on 43,613 scans from five highly complex publicly available datasets, containing multi-vendor scans, where it achieved the mean intersection-over-union score of 0.8055 for extracting the retinal lesions and the accuracy of 98.70% for the correct severity grading of retinopathy.

Retinal lesions play a vital role in the accurate classification of retinal abnormalities. Many researchers have proposed deep lesion-aware screening systems that analyze and grade the progression of retinopathy. However, to the best of our knowledge, no literature exploits the tendency of these systems to generalize across multiple scanner specifications and multi-modal imagery. Towards this end, this paper presents a detailed evaluation of semantic segmentation, scene parsing and hybrid deep learning systems for extracting the retinal lesions such as intra-retinal fluid, sub-retinal fluid, hard exudates, drusen, and other chorioretinal anomalies from fused fundus and optical coherence tomography (OCT) imagery. Furthermore, we present a novel strategy exploiting the transferability of these models across multiple retinal scanner specifications. A total of 363 fundus and 173,915 OCT scans from seven publicly available datasets were used in this research (from which 297 fundus and 59,593 OCT scans were used for testing purposes). Overall, a hybrid retinal analysis and grading network (RAGNet), backboned through ResNet50, stood first for extracting the retinal lesions, achieving a mean dice coefficient score of 0.822. Moreover, the complete source code and its documentation are released at http://biomisa.org/index.php/downloads/.

X-ray baggage screening plays a vital role in aviation security. Manual inspection of potentially anomalous items is challenging due to the clutter and occlusion within Xray scans. Here, we address this issue by presenting an object-boundaries driven framework for the automated detection of suspicious items from X-ray baggage scans. Rather than recognizing objects directly from the X-ray images, our two-stage detection approach first extracts contour-based proposals using a novel cascaded structure tensor technique and subsequently passes the candidate proposals to a single feed-forward convolutional neural network for recognition. Thorough experimentation on GDXray and SIXray datasets demonstrates that the proposed model achieves a mean area under the curve of 0.9878, outperforming the existing renown state-of-the-art object detection frameworks.

Extraction of retinal layers from optical coherence tomography (OCT) scans is critical for analyzing retinal anomalies and manual segmentation of these retinal layers is a very cumbersome task. Recently, deep learning has gained much popularity in medical image analysis due to its underlying precision and robustness. Many researchers have utilized deep learning for extracting retinal layers from OCT images. However, to the best of our knowledge, there is no literature available that presents a robust segmentation framework that is able to extract retinal layers from OCT scans having different retinal pathological syndromes. Therefore, this paper presents a deep convolutional neural network and structure tensor-based segmentation framework (CNN-STSF) for the fully automated segmentation of up to eight retinal layers from normal as well as diseased OCT scans. First of all, the proposed framework computes coherent tensor from the candidate scan through which retinal layers are extracted. Afterwards, the pixels representing the layers are further classified using cloud based deep convolutional neural network (CNN) model trained on 1,200 retinal layers patches. CNN model in the proposed framework computes the probability of each layer pixels and assign it to be part of that layer for which it has the highest probability. The proposed framework was tested and validated on more than 39,000 retinal OCT scans from different publicly available datasets and from local Armed Forces Institute of Ophthalmology (AFIO) dataset where it outperformed all the existing solutions by achieving the overall layer segmentation accuracy of 0.9375.