Action recognition and localization in videos pose significant challenges, as they require identifying temporal boundaries and classifying actions within long video sequences. This paper introduces an anchor-free, single-stage framework that predicts both the start and end times of actions while simultaneously classifying the actions, framing the task as a sequence labeling problem. The proposed model utilizes an encoder-decoder architecture with LSTM projections and 1D convolutions to capture rich temporal dependencies. This is followed by hierarchical feature pyramid generation, which is refined using Pyramidal Pooling Aggregation (PPA) and enhanced through a Progressive Feature Enhancement Unit (PFEU) with dilated convolutions to preserve the contextual relationships present in complex video frames. We also propose a novel Temporal Granularity Convolution (TGC) Layer, which refines temporal features. The TGC Layer captures various temporal details using a multi-branch structure consisting of Fine-Grained and Coarse-Grained Temporal Branches, designed to efficiently handle different temporal granularities. Finally, a decoder utilizes these multi-scale features to predict action instances at multiple levels of granularity. The model simultaneously integrates classification and regression heads to predict action labels and temporal boundaries, achieving improved localization and recognition. We have demonstrated the effectiveness of the proposed scheme, using mean average precision (mAP) across various Intersection over Union (IoU) thresholds on the Thumos14, ActivityNet-1.3, and MultiTHUMOS datasets over multiple state-of-the-art (SOTA) methods.

This article presents a new deep-learning architecture based on an encoder-decoder framework that retains contrast while performing background subtraction (BS) on thermal videos. The proposed scheme consists of three consecutive blocks: the encoder, the Multi-Scale Contrast Preservation (MSCP) block, and the decoder. The encoder network employs a hybrid of convolution and atrous convolution blocks to preserve both sparse and dense features, with a skip connection. The encoder, combined with the MSCP block, maintains multi-scale contrast features with reduced training loss. Furthermore, the decoder network accurately projects the extracted features at different layers into pixel-level detail. The proposed end-to-end model efficiently provides a binary map for the corresponding thermal video scene. The efficiency of the proposed algorithm is validated on two large-scale datasets, namely CDnet 2014 and the Tripura University Video Dataset at Night Time (TU-VDN). Both qualitative and quantitative results demonstrate that MSCP outperforms thirty-eight existing BS schemes.

The background subtraction (BGS) technique is popularly used for many surveillance systems, segmenting the foreground by subtracting the modeled background from the image sequences. The effectiveness of any BGS technique depends on the robustness of the constructed background model. It is to be noted that many BGS schemes are affected by the inclusion of either noisy pixels in background construction or parameters of generative models. In this regard, we propound an idea of a kernel-induced possibilistic fuzzy associated BGS scheme for local change detection from a fixed camera captured sequence. The proposed scheme follows two stages: background training and foreground segmentation. In the background construction stage, each pixel is modeled using a possibilistic fuzzy cost function in kernel-induced space. The use of the induced kernel function will project the low-dimensional data into a higher dimensional space and the use of the possibilistic function will construct a robust background model based on the density of the data in the temporal domain avoiding the noisy and outlier points. The performance of the proposed scheme is tested on three benchmark databases. The effectiveness of the proposed scheme is evaluated on different performance evaluation measures: precision, recall, F-measure, and average similarity. We corroborate our findings by comparing them against 19 state-of-the-art existing BGS techniques.

Local change detection for moving object detection is an essential step in any computer vision task. The most well-known technique is background subtraction BGS. However, the performance of BGS is strongly dependent on the background construction. The background construction to be robust in the presence of various challenges: dynamic backgrounds, illumination changes, camera jitter, etc. In this paper, we propose a novel encoder-decoder-based end-to-end deep learning framework for BGS. Thus, we explore a VGG-19 deep network with a transfer learning strategy as an encoder that deeply learned and extracted the features at different levels. We herewith propose a Multi-scale Feature Pulling MFP block which can retain the features at the various scales of the challenging video scenes. We also design a decoder network which is a stack of several transposed convolutional layers which precisely predict that each pixel of the target frame belongs to the background or foreground. The efficiency of the proposed algorithm is validated on the CDNet-2014 dataset by comparing its results against seventeen state-of-the-art techniques.

Multi-object tracking involves maintaining several trajectories of different objects moving in the scene throughout the video. With this objective, in this article, a fully automatic and real-time tracking algorithm to track multiple vehicles in a video is proposed. The proposed method specifically tries to address the challenges of occlusion and fast-motion in traffic surveillance. The algorithm begins with automatic detection of the moving targets by an adaptive GMM-based background subtraction method. The trajectories of these detected targets are then built using a three-level multi-motion modeled particle filter framework which allows to deal with the challenges of occlusion and fast-motions of the target. The likelihood model for targets is based on their color distribution and edge oriented histogram features. It is contended that the color distribution feature, which can represent the target appearance and the edge oriented histogram, which can describe the target structure are sufficient to represent it in an unique feature space. Based on the similarity of target likelihood, the locations of the targets are filtered. These filtered locations are then associated with the most likely detections using the proposed low-cost and fast data association algorithm based on Euclidean distance and prevailing motion vector. The performance evaluation of the proposed scheme is carried out based on the six measures: Multi Object Tracking Accuracy, Multi Object Tracking Precision, Mostly Tracked trajectories, Mostly Lost trajectories, Identity Switches and Frames Per Second. The results evaluated on stationary camera shot sequences from benchmark datasets as well as real-time shot videos indicate that the proposed algorithm ensures robust tracking and can be used effectively for real-time surveillance of highways.

Abstract Early stage of diagnosis of eye diseases through automatic analysis in the retinal image is the emerging technology in the area of retinopathy. Glaucoma is the primary reason for the loss of visibility in people around the world. The separation of the disc and the cup in the optic region is the technique used to identify glaucoma in the human retinal image. In this paper, superpixel segmentation, followed by Modified Kernel Fuzzy C‐Means (MKFCM) algorithm is used to segment the optic disc and optic cup. The proposed segmentation method achieves a maximum average of F‐score as 0.979, an average boundary distance as 10.016 pixels, and an average correlation coefficient of 0.949. To train convolutional neural networks (CNN), the segmented images obtained by the MKFCM segmentation algorithm is given as the input for the identification of glaucoma. This CNN uses the gray level co‐occurrence matrix features calculated from the segmented image. The experiment used for this study demonstrates that CNN gives superior categorization correctness and requires fewer figures of knowledge iterations than the original CNN. The accuracy obtained by this proposed method is 94.2%. The model will help to identify the proper class of severity of glaucoma in retinal images.

Abstract Glaucoma and diabetic retinopathy are the most common eye diseases and the leading cause of blindness around the world. The prime objective of this study is to devise and develop an experimental computer‐aided diagnosis system to provide an efficient way for assisting the ophthalmologist in early detection of ocular diseases such as glaucoma and diabetic retinopathy. The proposed technique follows three stages: Pre‐processing, feature selection and classification. Initially, the fundus image is pre‐processed to extract the green channel image, and the obtained green channel image is further enhanced using contrast limited adaptive histogram equalisation technique. Three different kinds of features: Clinical features, transform domain features and structural features are utilised to extract the relevant information from the enhanced fundus images. To avoid redundant information, an improved feature selection mechanism is used to select the optimum set of features from the extracted features. Subsequently, the selected features are used to train the support vector machine classifier for the classification of the retinal diseases with 10‐fold cross‐validation. The performance of the proposed method is assessed using eight different quantitative evaluation measures. The experimental results demonstrate the effectiveness of the proposed work over prior works for the early detection of ocular diseases.

Abstract Fluorescence microscopy is an important investigation tool of discoveries in the field of biological sciences where the imaging phenomena are limited by the noise. This paper introduces the integration of biorthogonal wavelet filters along with mixed Poisson‐Gaussian unbiased risk estimate (MPGURE) based subband adaptive thresholding function for the restoration of low photon count microscopy images. The proposed algorithm consists of four steps. In the first step, variance stabilization transform along with a multi‐scale Wiener filtering approach is used to filter out the noise and blurring effect. In the second step, deconvolved images are further decomposed by the biorthogonal wavelet filters. The modified wavelet subband structure is used for the identification of noisy and noise‐free subbands. In the next stage, different noisy coefficients are thresholded using the MPGURE‐based thresholding operation. The different thresholded images are combined along with different optimum coefficients. Finally, inverse variance stabilization transformation is applied to obtain the final restored output. Performance of the proposed algorithm is tested on 14 different benchmark image data sets with performance evaluation measures like signal‐to‐noise ratio, peak signal‐to‐noise ratio, mean structural similarity index measur, and correlation coefficient. Simulation results of the proposed algorithm claim better results than other state‐of‐the‐art techniques.

In this article, a novel edge preserving image fusion method is proposed by merging multiple images captured from different imaging sensors. The objective of this paper is to highlight the informative contents of multiple images into a single fused image. Fusion of data from multiple sensors is a difficult task as the imaging modality are different and sensors capturing the data may be affected by sensors noise. As the images captured from multiple sensors possess uncertainty within a pixel due to the multi-valued level of brightness. It is obvious that a deterministic method of fusion may not give a better results. Hence, it is required to explore the use of fuzzy sets theoretic approaches in this regard. The proposed scheme follow three stages. In the first stage of the algorithm, a resultant image is obtained by setting the maximum value between the pixel intensity of visible and infrared sub-images considered within a small spatial neighborhood. The edges of the visible image are preserved in the second stage of the algorithm using a Fuzzy edge technique. Finally the fused image is obtained by combining the obtained resultant image and the edges of the visible image. In order to evaluate the performance of the proposed method quantitatively, and qualitatively experiments were carried out on publicly available benchmark database, "TNO-database". The proposed method is compared with those of eight state-of-the-arts techniques. The experimental results of the proposed method attained state-of-the-art performance in objective assessment and visual quality assessment.

One of the well-established research domains among computer vision scientists is object tracking. However, not much work has been done in underwater scenarios. This article addresses the problem of visual tracking in the underwater environment with the stationary and nonstationary camera setups. In order to deal with the underwater optical dynamics, a dominant color component-based scene representation is employed in the YCbCr color space. An adaptive approach is devised to select the Walsh-Hadamard (WH) kernels for the efficient extraction of color, edge, and texture strengths, whereas a new feature called range strength is proposed to extract the variation of intensity from underwater sequences in the local neighborhood using the WH kernel. The likelihood of these feature strengths is integrated in a particle filter framework to track the object of interest in underwater sequences. The reference feature strengths used in assigning weights to the particles are updated based on the S$\phi$rensen distance. The coefficients of feature strengths are calculated in such a way that if one feature fails, then its coefficient become insignificant, whereas the more suitable features get higher feature coefficients. The effectiveness of the proposed scheme is evaluated using the underwater video datasets: reefVid, fish4knowledge (F4K), underwaterchangedetection (UWCD), and National Oceanic and Atmospheric Administration (NOAA). The performance evaluation is performed by comparing the scheme with five recent state-of-the-art tracking schemes. The quantitative analysis of the proposed scheme is carried out using three evaluation measures: overall intersection over union, centroid location error, and average tracking error. The performance of the proposed scheme is quite encouraging in the case of sequences with hazy and degraded, partially occluded, and camouflaged challenges.

Background subtraction (BGS) is a popular scheme epitomized in the state-of-the-art literature on video processing. In this context, a novel online kernelized fuzzy modal variation based background subtraction scheme for detecting local changes from the sequences of image frames is proposed. In the proposed scheme, the time varying background at different instances of time are modeled using fuzzy set theory. The proposed background subtraction scheme, utilizes the fuzzy modal variation as the cost function for fitting the pixel values of the image frames. The use of kernel based modal variation helps in projecting the pixel values in a higher dimensional space, linearly separating them into object and background classes. The results of the proposed technique is verified on different challenging sequences including dynamic background, camera jitter, noise, blurred scene, etc. The proposed technique is successfully tested over several test sequences with two major databases (all sequences) and it provides better results compared to the twenty one existing state-of-the-art techniques.

Two‐dimensional transforms are extensively used for speckle noise reduction (despeckling) in ultrasound images. This work proposes a double filter bank structure consisting of a discrete wavelet packet transform (DWPT) and directional filter bank (DFB) along with a fuzzy‐based clustering technique to despeckle ultrasound images. Wavelet packet transform can efficiently reject noises based on grey scale relational thresholding and DFB can efficiently preserve edge information. In this study, instead of conventional thresholding methods, fuzzy‐based clustering techniques are applied for noise rejection. The algorithm provides a consistent improvement over the competing state‐of‐the‐art speckle reduction algorithms due to the improved ability to preserve geometric features while rejecting speckle noise adaptively. The authors claim is validated by applying a number of clinical images with performance indices such as peak signal to noise ratio, mean structural similarity, signal to mean square error, speckle signal to noise ratio, speckle suppression index and edge preservation index.

The study attempts to despeckle ultrasound images by the application of a direction sensitive wavelet packet transform. In the proposed method, wavelet packet decomposition is performed on the image and best subbands are selected using singular value decomposition. The low frequency subband is preserved, as it has the maximum information content. Iterated directional filter bank (IDFB) is applied on all other selected subbands. The vertical cells of the IDFB in the horizontal subbands and horizontal cells of IDFB in vertical subbands are eliminated because of their minimum edge information and maximum noise. Shrinkages are also applied on the remaining cells to be evaluated. The performance of the proposed algorithm is evaluated in terms of mean square error (MSE), peak signal to noise ratio, mean structural SIMilarity index, speckle suppression index, signal to MSE and speckle signal to noise ratio. It is found after the evaluation that the obtained experimental results are better than the existing state of the art despeckling techniques.

A modified decision based unsymmetrical trimmed median filter algorithm for the restoration of gray scale, and color images that are highly corrupted by salt and pepper noise is proposed in this paper. The proposed algorithm replaces the noisy pixel by trimmed median value when other pixel values, 0's and 255's are present in the selected window and when all the pixel values are 0's and 255's then the noise pixel is replaced by mean value of all the elements present in the selected window. This proposed algorithm shows better results than the Standard Median Filter (MF), Decision Based Algorithm (DBA), Modified Decision Based Algorithm (MDBA), and Progressive Switched Median Filter (PSMF). The proposed algorithm is tested against different grayscale and color images and it gives better Peak Signal-to-Noise Ratio (PSNR) and Image Enhancement Factor (IEF).