Abstract Glaucoma is a critical eye condition that causes permanent blindness by damaging the Optic Nerve Head (ONH). Ophthalmologists diagnose glaucoma in patients by conducting morphological analysis of Optic Cup (OC) and Optic Disc (OD) regions in retinal fundus images. The implementation of lightweight and robust Artificial-Intelligence-enabled tools to deliver prompt glaucoma diagnostics is paramount in biomedical engineering. This article proposes CSP-SegNet, a novel Channel-Spatial-Pixel (CSP) attention-integrated lightweight encoder-decoder architecture for joint semantic segmentation of OC and OD in retinal fundus images. The crux of this work lies in the novel Channel-Spatial-Pixel (CSP) attention module which facilitates enhanced feature representation from different levels of abstraction with faster convergence. The efficacy of novel CSP attention is analyzed using Grad-CAM–based attention map evolution for explainable interpretation. CSP-SegNet is a novel depthwise separable convolutional neural network comprising approximately 1.54M trainable parameters with 13.3G FLOPS and hence it is highly lightweight compared to other methods. This paper has rigidly analyzed the robustness and generalization ability of CSP-SegNet in contrast to state-of-the-art segmentation networks, across the REFUGE and ORIGA datasets with cross-dataset evaluation on Drishti-GS. The proposed CSP-SegNet has obtained statistically significant results in outperforming many competing methods for joint semantic segmentation of OC and OD across Dice Similarity Coefficient (DSC) and Intersection over Union (IoU) metrics. The quantitative and qualitative evaluation results justify the superiority of CSP-SegNet and effectiveness of CSP attention in terms of generalization ability, segmentation performance, robustness across distribution shift and compactness. The code is available at https://github.com/AIAnkitDas/CSP-SegNet/tree/main .

Automatic analysis of colonoscopy images has been an active field of research motivated by the importance of early detection of precancerous polyps. However, detecting polyps during the live examination can be challenging due to various factors such as variation of skills and experience among the endoscopists, lack of attentiveness, and fatigue leading to a high polyp miss-rate. Therefore, there is a need for an automated system that can flag missed polyps during the examination and improve patient care. Deep learning has emerged as a promising solution to this challenge as it can assist endoscopists in detecting and classifying overlooked polyps and abnormalities in real time, improving the accuracy of diagnosis and enhancing treatment. In addition to the algorithm’s accuracy, transparency and interpretability are crucial to explaining the whys and hows of the algorithm’s prediction. Further, conclusions based on incorrect decisions may be fatal, especially in medicine. Despite these pitfalls, most algorithms are developed in private data, closed source, or proprietary software, and methods lack reproducibility. Therefore, to promote the development of efficient and transparent methods, we have organized the “Medico automatic polyp segmentation (Medico 2020)” and “MedAI: Transparency in Medical Image Segmentation (MedAI 2021)” competitions. The Medico 2020 challenge received submissions from 17 teams, while the MedAI 2021 challenge also gathered submissions from another 17 distinct teams in the following year. We present a comprehensive summary and analyze each contribution, highlight the strength of the best-performing methods, and discuss the possibility of clinical translations of such methods into the clinic. Our analysis revealed that the participants improved dice coefficient metrics from 0.8607 in 2020 to 0.8993 in 2021 despite adding diverse and challenging frames (containing irregular, smaller, sessile, or flat polyps), which are frequently missed during a routine clinical examination. For the instrument segmentation task, the best team obtained a mean Intersection over union metric of 0.9364. For the transparency task, a multi-disciplinary team, including expert gastroenterologists, accessed each submission and evaluated the team based on open-source practices, failure case analysis, ablation studies, usability and understandability of evaluations to gain a deeper understanding of the models’ credibility for clinical deployment. The best team obtained a final transparency score of 21 out of 25. Through the comprehensive analysis of the challenge, we not only highlight the advancements in polyp and surgical instrument segmentation but also encourage subjective evaluation for building more transparent and understandable AI-based colonoscopy systems. Moreover, we discuss the need for multi-center and out-of-distribution testing to address the current limitations of the methods to reduce the cancer burden and improve patient care. • We present a detailed analysis of the Medico 2020 and MedAI 2021 challenges that are aimed at advancing automated polyp and instrument segmentation in colonoscopy for early colorectal cancer diagnosis by using novel deep learning methods. • To the best of our knowledge, MedAI 2021 is the first challenge to evaluate the transparency in both GI endoscopy and colonoscopy. Through the challenge, we invited the participants to list package dependencies and architecture code (with instructions for building, compiling, and training) and share trained model weights in a standardized format. Additionally, we invited participants to include the code for model evaluation and provide repository licensing information to enable others to use the code and the trained model responsibly. Moreover, we asked the participants to explain model predictions using intermediate heatmaps, perform ablation studies, conduct a thorough failure analysis, and share their code for reproducing the results. Finally, we performed a subjective evaluation by including an expert gastroenterologist in the group and gave the final transparency score based on the usefulness and understandability of the results. Our initiative aims to promote transparency in AI research and foster the development of reliable, interpretable, and trustworthy algorithms for use in medical image segmentation. • We provide a comparative analysis of the 34 proposed methods in both challenges (3 subtasks), covering small details of each team in the form of Tables, qualitative and quantitative results (failure analysis), and an in-depth analysis of the findings. • We explore trust, safety, interpretability, transparency, and generalizability issues and provide future strategies to overcome the current limitations of developed algorithms.

Visibility issues in intelligent transportation systems are exacerbated by bad weather conditions such as fog and haze. It has been observed from recent studies that major road accidents have occurred in the world due to low visibility and inclement weather conditions. Single image dehazing attempts to restore a haze-free image from an unconstrained hazy image. We proposed a dehazing method by cascading two models utilizing a novel parameter-adaptive dual-channel modified simplified pulse coupled neural network (PA-DC-MSPCNN). The first model uses a new color channel for removing haze from images. The second model is the improved brightness preserving model (I-GIHE), which retains the brightness of the image while improving the gradient strength. To integrate the results from these two models and provide a pleasing haze-free image, a PA-DC-MSPCNN-based fusion is used. Furthermore, the proposed approach is deployed on a Xilinx Zynq SoC by exploiting the recently released PYNQ platform. The dehazing system runs on a PYNQ-Z2 all-programmable SoC platform, where it will input the camera feed through the FPGA unit and carry out the dehazing algorithm in the ARM core. This configuration has allowed reaching real-time processing speed for image dehazing. The results of dehazing are analyzed using both synthetic and real-world hazy images. Synthetic hazy images are acquired from the O-HAZE, I-HAZE, SOTS, and FRIDA datasets, while real-world hazy images are taken from the RailSem19, E-TUVD dataset, and the internet. For evaluation, twelve cutting-edge approaches are chosen. The proposed method is also analyzed on underwater and low-light images. Extensive experiments indicate that the proposed method outperforms state-of-the-art methods of qualitative and quantitative performances.

Heterogeneous Face Recognition (HFR) is a challenging task due to the significant intra-class variation between the query and gallery images. The reason behind this vast intra-class variation is the varying image capturing sensors and the varying image representation techniques. Visual, Infrared, thermal images are the output of different sensors and viewed sketches, and composite sketches are the output of different image representation techniques. Conventional deep learning models are trying to solve the problem. Still, progress is impeded due to small HFR data samples, task-specific models (one model trained for face sketch-photo matching can’t perform well for NIR-VIS face matching), joint learning of two different HFR scenarios are not possible by one single deep network, and models are not interpretable. In this paper, to solve these major problems, we presented a novel interpretable Local Frequency Binary Pattern (LFrBP) based continual learning shallow network for HFR. The model is divided into two parts. A modality-invariant CNN model using the LFrBP feature, fine-tuned with CNN, is presented in the first part. The second part is based on continual learning to jointly learn the two HFR scenarios (face sketch-photo and NIR-VIS face matching) using a single network. Recognition results on different challenging HFR databases depict the superiority of the proposed model over other state-of-the-art deep learning-based methods.

This paper presents a novel local image descriptor called Pattern of Local Gravitational Force (PLGF). It is inspired by Law of Universal Gravitation. PLGF is a hybrid descriptor, which is a combination of two feature components: one is the Pattern of Local Gravitational Force Magnitude (PLGFM), and another is Pattern of Local Gravitational Force Angle (PLGFA). PLGFM encodes the local gravitational force magnitude, and PLGFA encodes the local gravitational force angle that the center pixel exerts on all other pixels within a local neighborhood. We propose a novel noise resistance and the edge-preserving binary pattern called neighbors to center difference binary pattern (NCDBP) for gravitational force magnitude encoding. Finally, the histograms of the two components are concatenated to construct the PLGF descriptor. Experimental results on the existing face recognition databases, texture database, and biomedical image database show that PLGF is an effective image descriptor, and it outperforms other widely used existing descriptors. Even if in complicated variations like noise, and illumination with smaller databases, a combination of PLGF and convolutional neural network (CNN) performs consistently better than other state-of-the-art techniques.

In this work, we proposed a novel method, called Local Modified Zernike Moment per unit Mass (LMZMPM), for face recognition, which is invariant to illumination, scaling, noise, in-plane rotation, and translation, along with other orthogonal and inherent properties of the Zernike Moments (ZMs). The proposed LMZMPM is computed for each pixel in a neighborhood of size 3 × 3 , and then considers the complex tuple that contains both the phase and magnitude coefficients of LMZMPM as the extracted features. As it contains both the phase and the magnitude components of the complex feature, it has more information about the image and thus preserves both the edge and structural information. We also propose a hybrid similarity measure, combining the Jaccard Similarity with the L1 distance, and applied to the extracted feature set for classification. The feasibility of the proposed LMZMPM technique on varying illumination has been evaluated on the CMU-PIE and the extended Yale B databases with an average Rank-1 Recognition (R1R) accuracy of 99.8% and 98.66% respectively. To assess the reliability of the method with variations in noise, rotation, scaling, and translation, we evaluate it on the AR database and obtain an average R1R higher than that of recent state-of-the-art methods. The proposed method shows a very high recognition rate on Heterogeneous Face Recognition as well, with 100% on CUFS, and 98.80% on CASIA-HFB.

This paper proposes a novel local feature descriptor coined as a local instant-and-center-symmetric neighbor-based pattern of the extrema-images (LINPE) to detect breast abnormalities in thermal breast images. It is a hybrid descriptor that combines two different feature descriptors: one is the inverse-probability difference extrema (IpDE), and another is the local instant and center-symmetric neighbor-based pattern (LICsNP). IpDE is developed to compute the intensity-inhomogeneity-invariant feature-based image of the breast thermogram. Besides, the LICsNP is intended to capture the local microstructure pattern information in the IpDE image. A new paradigm, named Broad Learning (BL) network, is introduced here as a classifier to differentiate the healthy and sick breast thermograms efficiently. The efficacy of the proposed system is quantitatively validated on the images of DMR-IR and DBT-TU-JU databases. Extensive experimentation on these databases with an average accuracy of 96.90% and 94%, respectively, justifies proposed system's superiority in the differentiation of healthy and sick breast thermograms over the other related existing state-of-the-art methods. The proposed system also performs consistently in the presence of noise and rotational changes.

Generalized nucleus segmentation techniques can contribute greatly to reducing the time to develop and validate visual biomarkers for new digital pathology datasets. We summarize the results of MoNuSeg 2018 Challenge whose objective was to develop generalizable nuclei segmentation techniques in digital pathology. The challenge was an official satellite event of the MICCAI 2018 conference in which 32 teams with more than 80 participants from geographically diverse institutes participated. Contestants were given a training set with 30 images from seven organs with annotations of 21,623 individual nuclei. A test dataset with 14 images taken from seven organs, including two organs that did not appear in the training set was released without annotations. Entries were evaluated based on average aggregated Jaccard index (AJI) on the test set to prioritize accurate instance segmentation as opposed to mere semantic segmentation. More than half the teams that completed the challenge outperformed a previous baseline. Among the trends observed that contributed to increased accuracy were the use of color normalization as well as heavy data augmentation. Additionally, fully convolutional networks inspired by variants of U-Net, FCN, and Mask-RCNN were popularly used, typically based on ResNet or VGG base architectures. Watershed segmentation on predicted semantic segmentation maps was a popular post-processing strategy. Several of the top techniques compared favorably to an individual human annotator and can be used with confidence for nuclear morphometrics.

A finger biometric system at an unconstrained environment is presented in this paper. A technique for hand image normalization is implemented at the preprocessing stage that decomposes the main hand contour into finger-level shape representation. This normalization technique follows subtraction of transformed binary image from binary hand contour image to generate the left-side of finger profiles (LSFPs). Then, XOR is applied to LSFP image and hand contour image to produce the right side of finger profiles. During feature extraction, initially, 30 geometric features are computed from every normalized finger. The rank-based forward-backward greedy algorithm is followed to select relevant features and to enhance classification accuracy. Two different subsets of features containing 9 and 12 discriminative features per finger are selected for two separate experimentations those use the k-nearest neighbor and the random forest (RF) for classification on the Bosphorus hand database. The experiments with the selected features of four fingers except the thumb have obtained improved performances compared to features extracted from five fingers and also other existing methods evaluated on the Bosphorus database. The best identification accuracies of 96.56% and 95.92% using the RF classifier have been achieved for the rightand left-hand images of 638 subjects, respectively. An equal error rate of 0.078 is obtained for both types of the hand images.

Segmentation of suspicious regions (SRs) of a thermal breast image (TBI) is a very significant and challenging problem for the identification of breast cancer. Therefore, in this work, we have proposed an active contour model for the segmentation of the SRs in TBI. The proposed segmentation method combines three significant steps. First, a novel method, called smaller-peaks corresponding to the high-intensity-pixels and the centroid-knowledge of SRs (SCH-CS), is proposed to approximately locate the SRs, whose contours are later used as the initial evolving curves of the level set method (LSM). Second, a new energy functional, called different local priorities embedded (DLPE), is proposed regarding the level set function. DLPE is then minimized using the interleaved level set evolution to segment the potential SRs in TBI more accurately. Finally, a new stopping criterion is incorporated into the proposed LSM. The proposed LSM not only increases the segmentation speed but also ameliorates the segmentation accuracy. The performance of our SR segmentation method was evaluated on two TBI databases, namely, DMR-IR and DBT-TU-JU, and the average segmentation accuracies obtained on these databases are 72.18% and 71.26% respectively, which are better than the other state-of-the-art methods. Beside this, a novel framework to analyze TBIs is proposed for differentiating abnormal and normal breasts on the basis of the segmented SRs. We have also shown experimentally that investigating only the SRs instead of the whole breast is more effective in differentiating abnormal and normal breasts.

Abstract This paper presents a new hybrid fusion framework based on thermal and visible face images. Fusion of information is done here in two phases, first at the pixel level and then at the decision level. For the pixel level fusion process, à‐trous wavelet transform is applied on both the thermal and visible face images. In decision level fusion, 34 region classifiers, each concentrating on a specified region of the face image, are tested individually for their ability to identify a person from the face image. The region classifiers, which contribute significantly in recognizing the face image, are considered for decision level fusion using majority voting. All experiments have been conducted on the UGC‐JU face database and IRIS benchmark face database. The maximum recognition rate is about 97.22% for both the databases whereas decisions of 17 region classifiers among 34 are considered. Experimental results and comparative study show that the proposed fusion method provides a framework for recognition of face images in uncontrolled environments such as variations in illumination conditions, pose, and facial expressions.

The advancement of research in a specific area of clinical diagnosis crucially depends on the availability and quality of the radiology and other test related databases accompanied by ground truth and additional necessary medical findings. This paper describes the creation of the Department of Biotechnology-Tripura University-Jadavpur University (DBT-TU-JU) breast thermogram database. The objective of creating the DBT-TU-JU database is to provide a breast thermogram database that is annotated with the ground-truth images of the suspicious regions. Along with the result of breast thermography, the database comprises of the results of other breast imaging methodologies. A standard breast thermogram acquisition protocol suite comprising of several critical factors has been designed for the collection of breast thermograms. Currently, the DBT-TU-JU database contains 1100 breast thermograms of 100 subjects. Due to the necessity of evaluating any breast abnormality detection system, this study emphasizes the generation of the ground-truth images of the hotspot areas, whose presence in a breast thermogram signifies the presence of breast abnormality. With the generated ground-truth images, we compared the results of six state-of-the-art image segmentation methods using five supervised evaluation metrics to identify the proficient segmentation methods for hotspot extraction. Based on the evaluation results, the fractional-order Darwinian particle swarm optimization, region growing, mean shift, and fuzzy c-means clustering are found to be more efficient in comparison to k-means clustering and threshold-based segmentation methods.

A new image fusion algorithm based on the visible and thermal images for face recognition is presented in this paper. The new fusion algorithm derives the benefit from both the modalities images. The proposed fusion process is the weighted sum of thermal and visible face information with two weighting factors [Formula: see text] and [Formula: see text], respectively. The weighting factors are calculated using a directed search algorithm automatically. The proposed fusion framework is evaluated through extensive experiments using UGC-JU face database. Experiments are of three fold. Firstly, individual modalities images are used separately for human face recognition. Secondly, fused face images using the proposed method are used for recognition purpose. The highest level of accuracy achieved by using the proposed method is about 98.42%. Lastly, the three existing fusion methods are applied on the same face database for comparison with the results of the proposed method. All the results demonstrate significant performance improvements in recognition over individual modalities and some of the existing fusion approaches, suggesting that fusion is a viable approach that deserves further study and consideration.

This paper proposes a novel method called local-gravity-face (LG-face) for illumination-invariant and heterogeneous face recognition (HFR). LG-face employs a concept called the local gravitational force angle (LGFA). The LGFA is the direction of the gravitational force that the center pixel exerts on the other pixels within a local neighborhood. A theoretical analysis shows that the LGFA is an illumination-invariant feature, considering only the reflectance part of the local texture effect of the neighboring pixels. It also preserves edge information. Rank 1 recognition rates of 97.78% on the CMU-PIE database and 97.31% on the Extended Yale B database are achieved under varying illumination, demonstrating that LG-face is an effective method of illumination-invariant face recognition. For HFR, when faces appear in different modalities, LG-face produces a common feature representation. Rank 1 recognition rates of 99.96% on the CUFS database, 98.67% on the CUFSF database, and 99.78% on the CASIA-HFB database show that the LG-face is also an effective method for HFR. The proposed method also performs consistently in the presence of complicated variations and noise.

This paper presents a new technique for user identification and recognition based on the fusion of hand geometric features of both hands without any pose restrictions. All the features are extracted from normalized left and right hand images. Fusion is applied at feature and also at decision level. Two probability-based algorithms are proposed for classification. The first algorithm computes the maximum probability for nearest three neighbors. The second algorithm determines the maximum probability of the number of matched features with respect to a thresholding on distances. Based on these two highest probabilities initial decisions are made. The final decision is considered according to the highest probability as calculated by the Dempster–Shafer theory of evidence. Depending on the various combinations of the initial decisions, three schemes are experimented with 201 subjects for identification and verification. The correct identification rate is found to be 99.5%, and the false acceptance rate (FAR) of 0.625% has been found during verification.

This paper presents a robust approach for recognition of thermal face images based on decision level fusion of 34 different region classifiers. The region classifiers concentrate on local variations. They use singular value decomposition (SVD) for feature extraction. Fusion of decisions of the region classifier is done by using majority voting technique. The algorithm is tolerant against false exclusion of thermal information produced by the presence of inconsistent distribution of temperature statistics which generally make the identification process difficult. The algorithm is extensively evaluated on UGC-JU thermal face database, and Terravic facial infrared database and the recognition performance are found to be 95.83% and 100%, respectively. A comparative study has also been made with the existing works in the literature.

In this paper one investigation has been done to find the optimum level of fusion to find a fused image from visual as well as thermal images. Because of the use of face recognition system in critical areas like, authenticating an authorized person in highly secured areas, investigation of criminals, online monitoring etc, face recognition system should be very robust and accurate one. This work is an attempt to fuse visual and thermal face images at optimum level to extract the advantages of visual as well as thermal images. In our work, Object Tracking and Classification Beyond Visible Spectrum (OTCBVS) database has been used for the visual and thermal images. Among all the experiments a maximum recognition result obtained is 93%.