In this paper, we introduce a benchmark dataset for the diagnosis and treatment planning of Class III malocclusion, a condition that requires precise evaluation to determine the necessity of surgical intervention. Our dataset comprises paired lateral cephalometric X-rays and profile photographs, each annotated with a treatment plan mainly indicating whether surgery is required or not. We assess state-of-the-art deep learning models on both imaging modalities to explore their potential for automating diagnosis and treatment decisions. Notably, the dataset facilitates research into non-radiographic diagnostic approaches, potentially enabling treatment planning based solely on profile photographs. We release this dataset as a resource to advance automated applications in orthodontics and maxillofacial surgery.

Air leakage significantly contributes to energy loss in commercial and residential building envelopes. For the next generation of sustainable building construction, enhancing energy efficiency through effective air leak detection is paramount. Traditional air leak tests rely on intrusive methods and manual detection, which lack precision and speed in locating leaks. The rapid advancements of artificial intelligence offer tremendous opportunities to revolutionize these detection methods. This paper introduces an automatic real-time detection method based on the integration of an edge detection Sobel block with the standard real-time object detector YOLO (You Only Look Once) frame-work. It transforms single-channel infrared (IR) images into three channels that provide additional edge information while retaining the original data. It has been implemented in various YOLO versions to demonstrate its effectiveness with training on a custom dataset primarily collected from a test chamber experiment. The results show that the proposed enhanced models achieve fast and accurate detection with compact model size, making it easily implementable in handheld IR cameras or drone-mounted systems for both indoor and outdoor applications. This allows a significant advancement in the non-intrusive and precise detection of air leaks.

Bearing data compression is vital to manage the large volumes of data generated during condition monitoring. In this paper, a novel asymmetrical autoencoder with a lifting wavelet transform (LWT) layer is developed to compress bearing sensor data. The encoder part of the network consists of a convolutional layer followed by a wavelet filterbank layer. Specifically, a dual-channel convolutional block with diverse convolutional kernel sizes and varying processing depths is integrated into the wavelet filterbank layer to enable comprehensive feature extraction from the wavelet domain. Additionally, the adaptive hard-thresholding nonlinearity is applied to remove redundant components while denoising the primary wavelet coefficients. On the decoder side, inverse LWT, along with multiple linear layers and activation functions, is employed to reconstruct the original signals. Furthermore, to enhance compression efficiency, a sparsity constraint is introduced during training to impose sparsity on the latent representations. The experimental results demonstrate that the proposed approach achieves superior data compression performance compared to state-of-the-art methods.

Deep learning models have great potential in medical imaging, including orthodontics and skeletal maturity assessment. However, using a model on data different from its training set can lead to unreliable predictions that may impact patient care. To address this, we introduce a Gradient Attention Map (GAM)-based framework that evaluates a model’s suitability for new data by examining its attention patterns. Using Grad-CAM, we generate attention maps and compare them with metrics such as IoU, Dice Similarity, SSIM, Cosine Similarity, Pearson Correlation, KL Divergence, and Wasserstein Distance. A Random Forest classifier then distinguishes between models that are well-suited and those that are misapplied. Experimental results show that our method effectively filters out unsuitable models, promoting safer and more reliable use of deep learning in medical imaging.

Data-driven intelligent fault diagnosis methods have emerged as powerful tools for monitoring and maintaining the operating conditions of mechanical equipment. However, in real-world engineering scenarios, mechanical equipment typically operates under normal conditions, resulting in limited and imbalanced (L&I) data. This situation gives rise to label bias and biased training. Meanwhile, the current multi-source information fault diagnosis research to date has tended to focus on fault identification rather than effective feature fusion strategies. To solve these issues, a novel end-to-end mechanical fault diagnosis framework under limited & imbalanced data using multi-source information fusion is proposed to model data-level and algorithm-level ideas in a unified deep network for achieving effective multi-source information fusion under the L&I working conditions. From a data-level perspective, a data preprocessing operation is first employed to capture time–frequency information simultaneously. Subsequently, multi-source time–frequency information is fed into feature extractors with information discriminators to construct local and information-invariant feature maps with different scales to eliminate multi-source information domain shift. Then, the multi-source feature vectors are modeled by a multi-source information transformer-based neural network to achieve effective multi-source information fusion through cross-attention mechanism. Next, the global max pooling and global average pooling layers are leveraged to obtain the more representative features. Finally, from an algorithm-level perspective, a dual-stream diagnosis predictor with a binary diagnosis predictor and a multi-class diagnosis predictor is designed to synthesize the diagnostic results through a reweighing activation mechanism for addressing the L&I problems. Extensive experiments on four different multi-source information datasets show the superiority and promising performance of our method compared to the state-of-the-art methods, as evidenced by indicators from various aspects. • Proposing a novel end-to-end fault diagnosis framework. • A new information discriminator is constructed to facilitate fusion. • A novel multi-source transformer-based is introduced to achieve multi-source information exchange and fusion. • A novel dual-stream diagnosis predictor is proposed to address L&I problems.

The lack of an efficient preamble detection algorithm remains a challenge for solving preamble collision problems in intelligent random access (RA) in the cellular Internet of Things (IoT). To address this problem, we present an early preamble detection scheme based on a maximum likelihood estimation (MLE) model at the first step of the grant-based RA procedure. A novel unsupervised normalized Stein variational gradient descent (NSVGD)-based detector is proposed to obtain an approximate solution to the MLE model. First, by exploring the relationship between the Hadamard transform and wavelet packet transform, a new modified Hadamard transform (MHT) is developed to separate high-frequency components from signals using the second-order derivative filter. Next, to eliminate noise and mitigate the vanishing gradients problem in the Stein variational gradient descent (SVGD)-based detectors, the block MHT layer is designed based on the MHT, scaling layer, soft-thresholding layer, inverse MHT, and sparsity penalty. Then, the unsupervised NSVGD algorithm is derived to perform preamble detection without prior knowledge of noise power and the number of active IoT devices. The experimental results show the proposed block MHT layer outperforms other transform-based methods in terms of computation costs and denoising performance. Furthermore, with the assistance of the block MHT layer, the proposed unsupervised NSVGD algorithm achieves a higher preamble detection accuracy and throughput than other state-of-the-art detection methods.

The large volume of electroencephalograph (EEG) data produced by brain-computer interface (BCI) systems presents challenges for rapid transmission over bandwidth-limited channels in Internet of Things (IoT) networks. To address the issue, we propose a novel multichannel asymmetrical variational discrete cosine transform (DCT) network for EEG data compression within an edge-fog computing framework. At the edge level, low-complexity DCT compression units are designed using parallel trainable hard-thresholding and scaling operators to remove redundant data and extract the effective latent space representation. At the fog level, an adaptive filter bank is applied to merge important features from adjacent channels into each individual channel by leveraging interchannel correlations. Then, the inverse DCT reconstructed multihead attention is developed to capture both local and global dependencies and reconstruct the original signals. Furthermore, by applying the principles of variational inference, a new evidence lower bound is formulated as the loss function, driving the model to balance compression efficiency and reconstruction accuracy. Experimental results on two public datasets demonstrate that the proposed method achieves superior compression performance without sacrificing any useful information for BCI detection compared with state-of-the-art techniques, indicating a feasible solution for EEG data compression.

In this article, we propose a set of transform-based neural network layers as an alternative to the Conv2D layers in convolutional neural networks (CNNs). The proposed layers can be implemented based on orthogonal transforms, such as the discrete cosine transform (DCT), Hadamard transform (HT), and biorthogonal block wavelet transform (BWT). Furthermore, by taking advantage of the convolution theorems, convolutional filtering operations are performed in the transform domain using elementwise multiplications. Trainable soft-thresholding layers, that remove noise in the transform domain, bring nonlinearity to the transform domain layers. Compared with the Conv2D layer, which is spatial-agnostic and channel-specific, the proposed layers are location-specific and channel-specific. Moreover, these proposed layers reduce the number of parameters and multiplications significantly while improving the accuracy results of regular ResNets on the ImageNet-1K classification task. Furthermore, they can be inserted with a batch normalization (BN) layer before the global average pooling layer in the conventional ResNets as an additional layer to improve classification accuracy.

Domain generalization aims to train models on multiple source domains so that they can generalize well to unseen target domains. Among many domain generalization methods, Fourier-transformbased domain generalization methods have gained popularity primarily because they exploit the power of Fourier transformation to capture essential patterns and regularities in the data, making the model more robust to domain shifts. The mainstream Fouriertransform-based domain generalization swaps the Fourier amplitude spectrum while preserving the phase spectrum between the source and the target images. However, it neglects background interference in the amplitude spectrum. To overcome this limitation, we introduce a soft-thresholding function in the Fourier domain. We apply this newly designed algorithm to retinal fundus image segmentation, which is important for diagnosing ocular diseases but the neural network’s performance can degrade across different sources due to domain shifts. The proposed technique basically enhances fundus image augmentation by eliminating small values in the Fourier domain and providing better generalization. The innovative nature of the soft thresholding fused with Fourier-transform-based domain generalization improves neural network models’ performance by reducing the target images’ background interference significantly. Experiments on public data validate our approach’s effectiveness over conventional and state-of-the-art methods with superior segmentation metrics.

Traditional preamble detection algorithms have low accuracy in the grant-based random access scheme in massive machine-type communication (mMTC). We present a novel preamble detection algorithm based on Stein variational gradient descent (SVGD) at the second step of the random access procedure. It efficiently leverages deterministic updates of particles for continuous inference. To further enhance the performance of the SVGD detector, especially in a dense user scenario, we propose a normalized SVGD detector with momentum. It utilizes the momentum and a bias correction term to reduce the preamble estimation errors during the gradient descent process. Simulation results show that the proposed algorithm performs better than Markov Chain Monte Carlo-based approaches in terms of detection accuracy.

Electroencephalogram (EEG) data compression is necessary for wireless recording applications to reduce the amount of data that needs to be transmitted. In this paper, an asymmetrical sparse autoencoder with a discrete cosine transform (DCT) layer is proposed to compress EEG signals. The encoder module of the autoencoder has a combination of a fully connected linear layer and the DCT layer to reduce redundant data using hard-thresholding nonlinearity. Furthermore, the DCT layer includes trainable hard-thresholding parameters and scaling layers to give emphasis or de-emphasis on individual DCT coefficients. Finally, the one-by-one convolutional layer generates the latent space. The sparsity penalty-based cost function is employed to keep the feature map as sparse as possible in the latent space. The latent space data is transmitted to the receiver. The decoder module of the autoencoder is designed using the inverse DCT and two fully connected linear layers to improve the accuracy of data reconstruction. In comparison to other state-of-the-art methods, the proposed method significantly improves the average quality score in various data compression experiments.

In this paper, we present a method for estimating the deposition parameters of the thin film material Aluminum Nitride (AIN) using a deep neural network. The neural network predicts the AIN orientations, which are critical for micromachining Micro-Electro-Mechanical Systems (MEMS) transducers such as accelerometers and acoustic emission sensors. The network features three parallel channels, each equipped with a Discrete Cosine Transform (DCT) based layer that encodes the input parameters into a latent space. This DCT layer applies a hard-thresholding nonlinearity to eliminate noise from the input parameters, resulting in a sparse representation of the latent space. Trained with a dataset comprising AlN orientations parameters and their optimal values, our model is adept at simultaneously extracting and integrating various essential frequency components. Experimental results underscore the effectiveness of our proposed approach in achieving accurate and comprehensive estimation of AlN orientations and MEMS design parameters, thereby providing a promising path for advanced optimization.

Graph convolutional networks (GCNs) as the emerging neural networks have shown great success in Prognostics and Health Management because they can not only extract node features but can also mine relationship between nodes in the graph data. However, the most existing GCNs-based methods are still limited by graph quality, variable working conditions, and limited data, making them difficult to obtain remarkable performance. Therefore, it is proposed in this paper a two stage importance-aware subgraph convolutional network based on multi-source sensors named I2SGCN to address the above-mentioned limitations. In the real-world scenarios, it is found that the diagnostic performance of the most existing GCNs is commonly bounded by the graph quality because it is hard to get high quality through a single sensor. Therefore, we leveraged multi-source sensors to construct graphs that contain more fault-based information of mechanical equipment. Then, we discovered that unsupervised domain adaptation (UDA) methods only use single stage to achieve cross-domain fault diagnosis and ignore more refined feature extraction, which can make the representations contained in the features inadequate. Hence, it is proposed the two-stage fault diagnosis in the whole framework to achieve UDA. In the first stage, the multiple-instance learning is adopted to obtain the importance factor of each sensor towards preliminary fault diagnosis. In the second stage, it is proposed I2SGCN to achieve refined cross-domain fault diagnosis. Moreover, we observed that deficient and limited data may cause label bias and biased training, leading to reduced generalization capacity of the proposed method. Therefore, we constructed the feature-based graph and importance-based graph to jointly mine more effective relationship and then presented a subgraph learning strategy, which not only enriches sufficient and complementary features but also regularizes the training. Comprehensive experiments conducted on four case studies demonstrate the effectiveness and superiority of the proposed method for cross-domain fault diagnosis, which outperforms the state-of-the art methods.

The edge processing of deep neural networks (DNNs) is becoming increasingly important due to its ability to extract valuable information directly at the data source to minimize latency and energy consumption. Although pruning techniques are commonly used to reduce model size for edge computing, they have certain limitations. Frequency-domain model compression, such as with the Walsh–Hadamard transform (WHT), has been identified as an efficient alternative. However, the benefits of frequency-domain processing are often offset by the increased multiply-accumulate (MAC) operations required. This article proposes a novel approach to an energy-efficient acceleration of frequency-domain neural networks by utilizing analog-domain frequency-based tensor transformations. Our approach offers unique opportunities to enhance computational efficiency, resulting in several high-level advantages, including array microarchitecture with parallelism, analog-to-digital converter (ADC)/digital-to-analog converter (DAC)-free analog computations, and increased output sparsity. Our approach achieves more compact cells by eliminating the need for trainable parameters in the transformation matrix. Moreover, our novel array microarchitecture enablesadaptive stitchingof cells column-wise and row-wise, thereby facilitating perfect parallelism in computations. Additionally, our scheme enables ADC/DAC-free computations by training against highly quantized matrix-vector products, leveraging the parameter-free nature of matrix multiplications. Another crucial aspect of our design is its ability to handle signed-bit processing for frequency-based transformations. This leads to increased output sparsity and reduced digitization workload. On a$16 \ttimes 16$crossbars, for 8-bit input processing, the proposed approach achieves the energy efficiency of 801 tera operations per second per Watt (TOPS/W) without early termination strategy and 2655 TOPS/W with early termination strategy at VDD$=$0.85 V for 16-nm predictive technology models (PTM).

We present a novel deep learning method for fully automated detection and classification of the Cervical Vertebrae Maturation (CVM) stages. The deep convolutional neural network consists of three parallel networks (TriPodNet) independently trained with different initialization parameters. They also have a built-in set of novel directional filters that highlight the Cervical Vertebrae edges in X-ray images. Outputs of the three parallel networks are combined using a fully connected layer. 1018 cephalometric radiographs were labeled, divided by gender, and classified according to the CVM stages. Resulting images, using different training techniques and patches, were used to train TripodNet together with a set of tunable directional edge enhancers. Data augmentation is implemented to avoid overfitting. TripodNet achieves the state-of-the-art accuracy of 81.18% in female patients and 75.32% in male patients. The proposed TripodNet achieves a higher accuracy in our dataset than the Swin Transformers and the previous network models that we investigated for CVM stage estimation.

In this paper, we present a wireless ECG-derived Respiration Rate (RR) estimation using an autoencoder with a DCT Layer. The wireless wearable system records the ECG data of the subject and the respiration rate is determined from the variations in the baseline level of the ECG data. A straightforward Fourier analysis of the ECG data obtained using the wireless wearable system may lead to incorrect results due to uneven breathing. To improve the estimation precision, we propose a neural network that uses a novel Discrete Cosine Transform (DCT) layer to denoise and decorrelates the data. The DCT layer has trainable weights and soft-thresholds in the transform domain. In our dataset, we improve the Mean Squared Error (MSE) and Mean Absolute Error (MAE) of the Fourier analysis-based approach using our novel neural network with the DCT layer.

In this paper, we propose a novel Hadamard Transform (HT)-based neural network layer for hybrid quantum-classical computing. It implements the regular convolutional layers in the Hadamard transform domain. The idea is based on the HT convolution theorem which states that the dyadic convolution between two vectors is equivalent to the element-wise multiplication of their HT representation. Computing the HT is simply the application of a Hadamard gate to each qubit individually, so the HT computations of our proposed layer can be implemented on a quantum computer. Compared to the regular Conv2D layer, the proposed HT-perceptron layer is computationally more efficient. Compared to a CNN with the same number of trainable parameters and 99.26% test accuracy, our HT network reaches 99.31% test accuracy with 57.1% MACs reduced in the MNIST dataset; and in our ImageNet-1K experiments, our HT-based ResNet-50 exceeds the accuracy of the baseline ResNet-50 by 0.59% center-crop top-1 accuracy using 11.5% fewer parameters with 12.6% fewer MACs.

In this article, we review neural networks which have neurons with binary operations or networks that use binary transforms such as the Walsh-Hadamard transform (WHT). Neural networks with binary neurons or binary layers can be used in edge applications and/or applications requiring energy-efficient decision-making. WHT-based network is as accurate as the regular neural networks in the CIFAR-10 and the Tiny ImageNet image databases.

In this work, we present a method for detecting anomalous chemical sensors using contrastive learning-based framework. In many practical systems, an array of multiple chemical sensors are used. Some of the sensors may malfunction due to sensor drift and chemical poisoning. In standard contrastive learning, the aim is to learn representations that will have maximum agreement among data samples of the same concept while having a minimal agreement with data samples from other concepts. In this work, we adapt standard contrastive learning to learning useful representations for out-of-distribution sample detection. Furthermore, we compare the proposed framework with the cosine similarity measure and a novel similarity measure based on the ℓ1norm. Our experimental results show that our approach achieves higher AUC scores (93.6%) than baseline methods (90.1%).

Power iteration is a fundamental algorithm in data analysis. It extracts the eigenvector corresponding to the largest eigenvalue of a given matrix. Applications include ranking algorithms, principal component analysis (PCA), among many others. Certain use cases may benefit from alternate, non-linear power methods with low complexity. In this paper, we introduce multiplication-avoiding power iteration (MAPI). MAPI replaces the standard ℓ2inner products that appear at the regular power iteration (RPI) with multiplication-free vector products, which are Mercer-type kernels that induce the ℓ1norm. For an n × n matrix, MAPI requires n multiplications, while RPI needs n2multiplications per iteration. Therefore, MAPI provides a significant reduction of the number of multiplication operations, which are known to be costly in terms of energy consumption. We provide applications of MAPI to PCA-based image reconstruction as well as to graph-based ranking algorithms. When compared to RPI, MAPI not only typically converges much faster, but also provides superior performance.

Convolution has been the core operation of modern deep neural networks. It is well known that convolutions can be implemented in the Fourier Transform domain. In this article, we propose to use binary block Walsh–Hadamard transform (WHT) instead of the Fourier transform. We use WHT-based binary layers to replace some of the regular convolution layers in deep neural networks. We utilize both one-dimensional (1D) and 2D binary WHTs in this article. In both 1D and 2D layers, we compute the binary WHT of the input feature map and denoise the WHT domain coefficients using a nonlinearity that is obtained by combining soft-thresholding with the tanh function. After denoising, we compute the inverse WHT. We use 1D-WHT to replace the 1 × 1 convolutional layers, and 2D-WHT layers can replace the 3 × 3 convolution layers and Squeeze-and-Excite layers. 2D-WHT layers with trainable weights can be also inserted before the Global Average Pooling layers to assist the dense layers. In this way, we can reduce the number of trainable parameters significantly with a slight decrease in trainable parameters. In this article, we implement the WHT layers into MobileNet-V2, MobileNet-V3-Large, and ResNet to reduce the number of parameters significantly with negligible accuracy loss. Moreover, according to our speed test, the 2D-FWHT layer runs about 24 times as fast as the regular 3 × 3 convolution with 19.51% less RAM usage in an NVIDIA Jetson Nano experiment.

Sensor drift is a major problem in chemical sensors that requires addressing for reliable and accurate detection of chemical analytes. In this paper, we develop a causal convolutional neural network (CNN) with a Discrete Cosine Transform (DCT) layer to estimate the drift signal. In the DCT module, we apply soft-thresholding nonlinearity in the transform domain to denoise the data and obtain a sparse representation of the drift signal. The soft-threshold values are learned during training. Our results show that DCT layer-based CNNs are able to produce a slowly varying baseline drift signal. We train the CNN on synthetic data and test it on real chemical sensor data. Our results show that we can have an accurate and smooth drift estimate even when the observed sensor signal is very noisy.

We propose a co-design approach for compute-in-memory inference for deep neural networks (DNN). We use multiplication-free function approximators based on l1norm along with a co-adapted processing array and compute flow. Using the approach, we overcame many deficiencies in the current art of in-SRAM DNN processing such as the need for digital-to-analog converters (DACs) at each operating SRAM row/column, the need for high precision analog-to-digital converters (ADCs), limited support for multi-bit precision weights, and limited vector-scale parallelism. Our co-adapted implementation seamlessly extends to multi-bit precision weights, it doesn't require DACs, and it easily extends to higher vector-scale parallelism. We also propose an SRAM-immersed successive approximation ADC (SA-ADC), where we exploit the parasitic capacitance of bit lines of SRAM array as a capacitive DAC. Since the dominant area overhead in SA-ADC comes due to its capacitive DAC, by exploiting the intrinsic parasitic of SRAM array, our approach allows low area implementation of within-SRAM SA-ADC. Our 8×62 SRAM macro, which requires a 5-bit ADC, achieves ~105 tera operations per second per Watt (TOPS/W) with 8-bit input/weight processing at 45 nm CMOS. Our 8×30 SRAM macro, which requires a 4-bit ADC, achieves ~84 TOPS/W. SRAM macros that require lower ADC precision are more tolerant of process variability, however, have lower TOPS/W as well. We evaluated the accuracy and performance of our proposed network for MNIST, CIFAR10, and CIFAR100 datasets. We chose a network configuration which adaptively mixes multiplication-free and regular operators. The network configurations utilize the multiplication-free operator for more than 85% operations from the total. The selected configurations are 98.6% accurate for MNIST, 90.2% for CIFAR10, and 66.9% for CIFAR100. Since most of the operations in the considered configurations are based on proposed SRAM macros, our compute-in-memory's efficiency benefits broadly translate to the system-level.

Recent developments in the field of neural networks, adaptive learning, and control will enable autonomous vehicles to operate in complex environments including urban, rural, and dangerous environments.

Electrocardiography (ECG) is a widely used tool for studying and diagnosing the heart diseases. Atrial fibrillation (AF) is an irregular and often rapid heart rate that can increase the risk of strokes, heart failure and other heart-related complications. In this study, we develop a novel and effective method to predict the potential AF risk of patients using our ECG signal dataset collected in the University of Illinois Hospital and Health Sciences System. We use a convolutional neural network (CNN) structure to process both the ECG signals and the related health data of patients. Our experimental results indicate that the model with patients' health data can predict the AF with 79.9% accuracy), and which is better than a CNN trained without related health data 72.2% accuracy), which implies that patients' health data play an important role in predicting AF risk. Very high sensitivity and specificity of the class of normal sinus rhythm (NSR) cases also verify that the model works well for distinguishing between NSR and ECG signals with potential AF risk.

Robust and computationally efficient anomaly detection in videos is a problem in video surveillance systems. We propose a technique to increase robustness and reduce computational complexity in a Convolutional Neural Network (CNN) based anomaly detector that utilizes the optical flow information of video data. We reduce the complexity of the network by denoising the intermediate layer outputs of the CNN and by using powers-of-two weights, which replaces the computationally expensive multiplication operations with bit-shift operations. Denoising operation during inference forces small valued intermediate layer outputs to zero. The number of zeros in the network significantly increases as a result of denoising, we can implement the CNN about 10% faster than a comparable network while detecting all the anomalies in the testing set. It turns out that denoising operation also provides robustness because the contribution of small intermediate values to the final result is negligible. During training we also generate motion vector images by a Generative Adversarial Network (GAN) to improve the robustness of the overall system. We experimentally observe that the resulting system is robust to background motion.

Large, open, public events, such as marathons and festivals, have always presented a unique safety challenge. These sprawling events, which can take up entire city blocks or stretch for many miles, can draw tens to hundreds of thousands of spectators and in some cases have open admission. As it is impracticable to guarantee the subjection of every event-goer to a security screening, we propose a crowd-based explosive detection system that uses a multitude of low-cost ChemFET sensors which are distributed to attendees. As the sensors offer limited accuracy, we further propose a server-based decision-making framework that utilizes a two-level feedback loop between the sensors and the server and explores spatial and temporal locality of the collected data to overcome the inherent low-accuracy of individual sensors. We thoroughly explore two distinct detection schemes, stressing their performance under a myriad of conditions, thus showing that such a crowd-based detection system comprised of low-cost and low-accuracy sensors can deliver high detection accuracy with minimal false positives.

In this paper, we propose a deep convolutional neural network for camera based wildfire detection. We train the neural network via transfer learning and use window based analysis strategy to increase the fire detection rate. To achieve computational efficiency, we calculate frequency response of the kernels in convolutional and dense layers and eliminate those filters with low energy impulse response. Moreover, to reduce the storage for edge devices, we compare the convolutional kernels in Fourier domain and discard similar filters using the cosine similarity measure in the frequency domain. We test the performance of the neural network with a variety of wildfire video clips and prune system performs as good as the regular network in daytime wild fire detection, and it also works well on some night wild fire video clips.

Early detection of wildfire smoke in real-time is essentially important in forest surveillance and monitoring systems. We propose a vision-based method to detect smoke using Deep Convolutional Generative Adversarial Neural Networks (DC-GANs). Many existing supervised learning approaches using convolutional neural networks require substantial amount of labeled data. In order to have a robust representation of sequences with and without smoke, we propose a two-stage training of a DCGAN. Our training framework includes, the regular training of a DCGAN with real images and noise vectors, and training the discriminator separately using the smoke images without the generator. Before training the networks, the temporal evolution of smoke is also integrated with a motion-based transformation of images as a pre-processing step. Experimental results show that the proposed method effectively detects the smoke images with negligible false positive rates in real-time.

While Volatile Organic Compounds (VOC) and ammonia have a place in our daily lives, their leakage into the environment is harmful to human health. In order to prevent and detect gaseous leaks of harmful VOCs, a cyber-physical system (CPS) comprised of ordinary people or first responders is proposed. This CPS uses small, low-cost sensors coupled to smart phones or mobile devices with the necessary computation and communication capabilities. The efficacy of such a CPS hinges on its ability to address technical challenges stemming from the fact that identically produced sensors may produce different results under the same conditions due to sensor drift, noise, or resolution errors. The proposed system makes use of time-varying signals produced by sensors to detect gas leaks. Sensors sample the gas vapor level in a continuous manner and time-varying sensor data is processed using deep neural networks. One of the neural networks (NN) is an energy efficient Additive Neural Network (AddNet) which can be implemented in host devices. The second NN is the discriminator of a GAN and the third a regular convolutional NN. AddNet produces comparable VOC gas leak detection results to regular convolutional networks while reducing area requirements by two thirds.

The compressed sensing (CS) framework leverages the sparsity of MR images to reconstruct from the undersampled acquisitions. CS reconstructions involve one or more regularization parameters that weigh sparsity in transform domains against fidelity to acquired data. While parameter selection is critical for reconstruction quality, the optimal parameters are subject and dataset specific. Thus, commonly practiced heuristic parameter selection generalizes poorly to independent datasets. Recent studies have proposed to tune parameters by estimating the risk of removing significant image coefficients. Line searches are performed across the parameter space to identify the parameter value that minimizes this risk. Although effective, these line searches yield prolonged reconstruction times. Here, we propose a new self-tuning CS method that uses computationally efficient projections onto epigraph sets of the${\ell }_{{1}}$and total-variation norms to simultaneously achieve parameter selection and regularization. In vivo demonstrations are provided for balanced steady-state free precession, time-of-flight, and T1-weighted imaging. The proposed method achieves an order of magnitude improvement in computational efficiency over line-search methods while maintaining near-optimal parameter selection.

We present a non-Euclidean vector product for artificial neural networks. The vector product operator does not require any multiplications while providing correlation information between two vectors. Ordinary neurons require inner product of two vectors. We propose a class of neural networks with the universal approximation property over the space of Lebesgue integrable functions based on the proposed non-Euclidean vector product. In this new network, the “product” of two real numbers is defined as the sum of their absolute values, with the sign determined by the sign of the product of the numbers. This “product” is used to construct a vector product in RN. The vector product induces the l1norm. The additive neural network successfully solves the XOR problem. Experiments on MNIST and CIFAR datasets show that the classification performance of the proposed additive neural network is comparable to the corresponding multi-layer perceptron and convolutional neural networks.

Contrast enhancement is an important preprocessing step for the analysis of images. The main aim of contrast enhancement techniques is to increase the visibility of the objects by modifying the spatial characteristics of the image. In this paper, phase transition based contrast enhancement framework is proposed to overcome the limitations of existing image enhancement techniques. The proposed image enhancement framework transforms the changes in image phase into the variations of magnitude to enhance the structural details of the image and to improve visibility. In addition, the concept of Selective Variation (SV) technique is introduced and enhancement parameters are optimized using SV. The experimental studies that were carried out on TID2008 dataset, show that the proposed scheme obtains promising results on contrast enhancement.

Accurate localization of mobile devices based on camera-acquired visual media information usually requires a search over a very large GPS-referenced image database collected from social sharing websites like Flickr or services such as Google Street View. This paper proposes a new method for reliable estimation of the actual query camera location by optimally utilizing structure from motion (SFM) for three-dimensional (3-D) camera position reconstruction, and introducing a new approach for applying a linear transformation between two different 3-D Cartesian coordinate systems. Since the success of SFM hinges on effectively selecting among the multiple retrieved images, we propose an optimization framework to do this using the criterion of the highest intraclass similarity among images returned from retrieval pipeline to increase SFM convergence rate. The selected images along with the query are then used to reconstruct a 3-D scene and find the relative camera positions by employing SFM. In the last processing step, an effective camera coordinate transformation algorithm is introduced to estimate the query's geo-tag. The influence of the number of images involved in SFM on the ultimate position error is investigated by examining the use of three and four dataset images with different solution for calculating the query world coordinates. We have evaluated our proposed method on query images with known accurate ground truth. Experimental results are presented to demonstrate that our method outperforms other reported methods in terms of average error.

Time-frequency techniques have difficulties in yielding efficient online algorithms for almost periodic signals. We describe a new topological method to find the period of signals that have an almost periodic waveform. Proposed method is applied to signals received from a pyro-electric infrared sensor array for the online estimation of the respiratory rate (RR) of a person. Time-varying analog signals captured from the sensors exhibit an almost periodic behavior due to repetitive nature of breathing activity. Sensor signals are transformed into two-dimensional point clouds with a technique that allows preserving the period information. Features, which represent the harmonic structures in the sensor signals, are detected by applying persistent homology and the RR is estimated based on the persistence barcode of the first Betti number. Experiments have been carried out to show that our method makes reliable estimates of the RR.

This paper presents a novel infrared (IR) object tracking algorithm based on the co-difference matrix. Extraction of co-difference features is similar to the well known covariance method except that the vector product operator is redefined in a multiplication-free manner. The new operator yields a computationally efficient implementation for real time object tracking applications. Experiments on an extensive set of IR image sequences indicate that the new method performs better than covariance tracking and other tracking algorithms without requiring any multiplication operations.

Kernel density estimation (KDE) is widely‐used for non‐parametric estimation of an underlying density from data. The performance of KDE is mainly dependent on the bandwidth parameter of the kernel. This study presents an alternative method of estimating the bandwidth by incorporating sparsity priors in the Fourier transform domain. By using cross‐validation (CV) together with an l 1 constraint, the proposed method significantly reduces the under‐smoothing effect of traditional CV methods. A solution for all free parameters in the minimisation is proposed, such that the algorithm does not need any additional parameter tuning. Simulation results indicate that the new approach is able to outperform classical and more recent approaches over a set of distributions of interest.

In this paper, we propose a real-time dynamic texture recognition method using projections onto random hyperplanes and deep neural network filters. We divide dynamic texture videos into spatio-temporal blocks and extract features using local binary patterns (LBP). We reduce the computational cost of the exhaustive LBP method by using randomly sampled subset of pixels in a given spatio-temporal block. We use random hyperplanes and deep neural network filters to reduce the dimensionality of the final feature vectors. We test the performance of the proposed method in a dynamic texture database. We also propose an application of the proposed method to real-time detection of flames in infrared videos. We observe that the approach based on random hyperplanes produces the best results.

A new multi-resolution super-pixel based algorithm is proposed to track cell size, count and motion in Mesenchymal Stem Cells (MSCs) images. Multi-resolution super-pixels are obtained by placing varying density seeds on the image. The density of the seeds are determined according to the local high frequency components of the MSCs image. In this way a multi-resolution super-pixels decomposition of the image is obtained. A second contribution of the paper is novel decision rule for merging similar neighboring super-pixels. One-dimensional version of the well known scale invariant feature transform (SIFT) is developed and applied to the histograms of the neighboring super-pixels to determine similar regions. The proposed algorithm is experimentally shown to be successful in segmenting and tracking cells in MSCs images.

A new denoising algorithm based on orthogonal projections onto the epigraph set of a convex cost function is presented. In this algorithm, the dimension of the minimization problem is lifted by one and feasibility sets corresponding to the cost function using the epigraph concept are defined. As the utilized cost function is a convex function in RN, the corresponding epigraph set is also a convex set in RN+1. The denoising algorithm starts with an arbitrary initial estimate in RN+1. At each step of the iterative denoising, an orthogonal projection is performed onto one of the constraint sets associated with the cost function in a sequential manner. The method provides globally optimal solutions for total-variation, ℓ1, ℓ2, and entropic cost functions.

The common spatial pattern (CSP) method has large number of applications in brain machine interfaces (BMI) to extract features from the multichannel neural activity through a set of linear spatial projections. These spatial projections minimize the Rayleigh quotient (RQ) as the objective function, which is the variance ratio of the classes. The CSP method easily overfits the data when the number of training trials is not sufficiently large and it is sensitive to daily variation of multichannel electrode placement, which limits its applicability for everyday use in BMI systems. To overcome these problems, the amount of channels that is used in projections, should be limited to some adequate number. We introduce a spatially sparse projection (SSP) method that renders unconstrained minimization possible via a new objective function with an approximated ℓ1penalty. We apply our new algorithm with a baseline regularization to the ECoG data involving finger movements to gain stability with respect to the number of sparse channels.

A new framework for signal processing is introduced based on a novel vector product definition that permits a multiplier-free implementation. First a new product of two real numbers is defined as the sum of their absolute values, with the sign determined by product of the hard-limited numbers. This new product of real numbers is used to define a similar product of vectors in RN. The new vector product of two identical vectors reduces to a scaled version of the l1norm of the vector. The main advantage of this framework is that it yields multiplication-free computationally efficient algorithms for performing some important tasks in signal processing. An application to the problem of cancer cell line image classification is presented that uses the notion of a co-difference matrix that is analogous to a covariance matrix except that the vector products are based on our new proposed framework. Results show the effectiveness of this approach when the proposed co-difference matrix is compared with a covariance matrix.

We propose a new framework, called Filtered Variation (FV), for denoising and sparse signal processing applications. These problems are inherently ill-posed. Hence, we provide regularization to overcome this challenge by using discrete time filters that are widely used in signal processing. We mathematically define the FV problem, and solve it using alternating projections in space and transform domains. We provide a globally convergent algorithm based on the projections onto convex sets approach. We apply to our algorithm to real denoising problems and compare it with the total variation recovery.

In this paper, a Content Based Image Retrieval (CBIR) algorithm employing relevance feedback is developed. After each round of user feedback Biased Discriminant Analysis (BDA) is utilized to find a transformation that best separates the positive samples from negative samples. The algorithm determines a sparse set of eigenvectors by L1 based optimization of the generalized eigenvalue problem arising in BDA for each feedback round. In this way, a transformation matrix is constructed using the sparse set of eigenvectors and a new feature space is formed by projecting the current features using the transformation matrix. Transformations developed using the sparse signal processing method provide better CBIR results and computational efficiency. Experimental results are presented.

In this paper, we present a novel method for classification of cancer cell line images using complex wavelet-based region covariance matrix descriptors. Microscopic images containing irregular carcinoma cell patterns are represented by randomly selected subwindows which possibly correspond to foreground pixels. For each subwindow, a new region descriptor utilizing the dual-tree complex wavelet transform coefficients as pixel features is computed. ℂWT as a feature extraction tool is preferred primarily because of its ability to characterize singularities at multiple orientations, which often arise in carcinoma cell lines, and approximate shift invariance property. We propose new dissimilarity measures between covariance matrices based on Kullback-Leibler (KL) divergence and L2-norm, which turn out to be as successful as the classical KL divergence, but with much less computational complexity. Experimental results demonstrate the effectiveness of the proposed image classification framework. The proposed algorithm outperforms the recently published eigenvalue-based Bayesian classification method.

In this paper, an entropy-functional-based online adaptive decision fusion (EADF) framework is developed for image analysis and computer vision applications. In this framework, it is assumed that the compound algorithm consists of several subalgorithms, each of which yields its own decision as a real number centered around zero, representing the confidence level of that particular subalgorithm. Decision values are linearly combined with weights that are updated online according to an active fusion method based on performing entropic projections onto convex sets describing subalgorithms. It is assumed that there is an oracle, who is usually a human operator, providing feedback to the decision fusion method. A video-based wildfire detection system was developed to evaluate the performance of the decision fusion algorithm. In this case, image data arrive sequentially, and the oracle is the security guard of the forest lookout tower, verifying the decision of the combined algorithm. The simulation results are presented.

Video fire detection system which uses a spatio-temporal covariance matrix of video data is proposed. This system divides the video into spatio-temporal blocks and computes covariance features extracted from these blocks to detect fire. Feature vectors taking advantage of both the spatial and the temporal characteristics of flame colored regions are classified using an SVM classifier which is trained and tested using video data containing flames and flame colored objects. Experimental results are presented.

Spectrum sensing is one of the most essential characteristics of cognitive radios (CRs). Robustness and adaptation to varying wireless propagation scenarios without compromising the sensing accuracy are desirable features of any spectrum sensing method to be deployed in CR systems. In this study, an online adaptive cooperation technique for spectrum sensing is proposed in order to maintain the level of reliability and performance. Cooperation is achieved by sensors which employ energy detection. These sensors send their output to a center where data fusion operation is carried out in an online and adaptive manner. Adaptation is realized by the use of orthogonal projections onto convex sets (POCS). In conjunction with the proposed method, an end-to-end methodology for a flexible experimental setup is also proposed in this study. This setup is arranged to emulate the proposed adaptive cooperation scheme for spectrum sensing and validate its practical use in cognitive radio systems. Comparative performance results for both inphase and quadrature branches are presented.

In this article, an image feature extraction method based on two-dimensional (2D) Mellin cepstrum is introduced. The concept of one-dimensional (1D) mel-cepstrum that is widely used in speech recognition is extended to two-dimensions using both the ordinary 2D Fourier transform and the Mellin transform. The resultant feature matrices are applied to two different classifiers such as common matrix approach and support vector machine to test the performance of the mel-cepstrum- and Mellin-cepstrum-based features. The AR face image database, ORL database, Yale database and FRGC database are used in experimental studies, which indicate that recognition rates obtained by the 2D mel-cepstrum-based method are superior to that obtained using 2D principal component analysis, 2D Fourier-Mellin transform and ordinary image matrix-based face recognition in both classifiers. Experimental results indicate that 2D cepstral analysis can also be used in other image feature extraction problems.

In this paper, we propose a novel method for detecting and monitoring Volatile Organic Compounds (VOC) gas leaks by using a Pyro-electric (or Passive) Infrared (PIR) sensor whose spectral range intersects with the absorption bands of VOC gases. A continuous time analog signal is obtained from the PIR sensor. This signal is discretized and analyzed in real time. Feature parameters are extracted in wavelet domain and classified using a Markov Model (MM) based classifier. Experimental results are presented.

A feature extraction method based on two-dimensional (2D) mel-cepstrum is introduced. The concept of one-dimensional (1D) mel-cepstrum which is widely used in speech recognition is extended to 2D in this article. Feature matrices resulting from the 2D mel-cepstrum, Fourier LDA, 2D PCA and original image matrices are converted to feature vectors and individually applied to a Support Vector Machine (SVM) classification engine for comparison. The AR face database, ORL database, Yale database and FRGC version 2 database are used in experimental studies, which indicate that recognition rates obtained by the 2D mel-cepstrum method is superior to the recognition rates obtained using Fourier LDA, 2D PCA and ordinary image matrix based face recognition. This indicates that 2D mel-cepstral analysis can be used in image feature extraction problems.

In this paper, an adaptive color transform for image compression is introduced. In each block of the image coefficients of the color transform are determined from the previously compressed neighboring blocks using weighted sums of the RGB pixel values, making the transform block-specific. There is no need to transmit or store the transform coefficients because they are estimated from previous blocks. The compression efficiency of the transform is demonstrated using the JPEG image coding scheme. In general, the suggested transformation results in better PSNR values for a given compression level.

In this paper, a feature extraction method based on two-dimensional (2D) mel-cepstrum is introduced. Feature matrices resulting from the 2D mel-cepstrum, Fourier LDA approach and original image matrices are individually applied to the Common Matrix Approach (CMA) based face recognition system. For each of these feature extraction methods, recognition rates are obtained in the AR face database, ORL database and Yale database. Experimental results indicate that recognition rates obtained by the 2D mel-cepstrum method is superior to the recognition rates obtained using Fourier LDA approach and raw image matrices. This indicates that 2D mel-cepstral analysis can be used in image feature extraction problems.

In this article, we propose a motion vector based feature set for Content Based Copy Detection (CBCD) of video clips. Motion vectors of image frames are one of the signatures of a given video. However, they are not descriptive enough when consecutive image frames are used because most vectors are too small. To overcome this problem we calculate motion vectors in a lower frame rate than the actual frame rate of the video. As a result we obtain longer vectors which form a robust parameter set representing a given video. Experimental results are presented.

Most video processing applications require object tracking as it is the base operation for real-time implementations such as surveillance, monitoring and video compression. Therefore, accurate tracking of an object under varying scene conditions is crucial for robustness. It is well known that illumination variations on the observed scene and target are an obstacle against robust object tracking causing the tracker lose the target. In this paper, a 2D-cepstrum based approach is proposed to overcome this problem. Cepstral domain features extracted from the target region are introduced into the covari-ance tracking algorithm and it is experimentally observed that 2D-cepstrum analysis of the target object provides robustness to varying illumination conditions. Another contribution of the paper is the development of the co-difference matrix based object tracking instead of the recently introduced covariance matrix based method.

A computer vision based algorithm for wildfire detection is developed. The main detection algorithm is composed of four sub-algorithms detecting (i) slow moving objects, (ii) gray regions, (iii) rising regions, and (iv) shadows. Each algorithm yields its own decision as a real number in the range [-1,1] at every image frame of a video sequence. Decisions from subalgorithms are fused using an adaptive algorithm. In contrast to standard Weighted Majority Algorithm (WMA), weights are updated using the Least Mean Square (LMS) method in the training (learning) stage. The error function is defined as the difference between the overall decision of the main algorithm and the decision of an oracle, who is the security guard of the forest look-out tower.

In this paper a new Digital Elevation Map (DEM) image compression algorithm is proposed. DEM image can be threated as a grayscale image, whose pixel values are the elevation values of the map points. The grayscale DEM image is compressed using an adaptive wavelet based image compression algorithm. The method, which is an extension of the progressive mesh compression takes advantage of the multiresolution property of the wavelets while coding the map images. This makes it possible to decode different resolutions of the map from the encoded bit stream providing a multiresolution display of a given map. Experimental results are presented.

A fast algorithm for image classification based on a computationally efficient operator forming a semigroup on real numbers is developed. The new operator does not require any multiplications. The co-difference matrix based on the new operator is defined and an image descriptor using the co-difference matrix is developed. In the proposed method, the multiplication operation of the well-known covariance method is replaced by the new operator. The proposed method is experimentally compared with the regular covariance matrix method. The proposed descriptor performs as well as the the regular covariance method without performing any multiplications. Texture recognition and licence plate identification examples are presented.

A novel evolutionary global optimization approach based on adaptive covariance estimation is proposed. The proposed method samples from a multivariate Levy Skew Alpha-Stable distribution with the estimated covariance matrix to realize a random walk and so to generate new solution candidates in the mutation step. The proposed method is compared to the popular Differential Evolution method, which is one of the best general evolutionary global optimizers available. Experimental results indicate that the proposed approach yields a general improvement in the required number of function evaluations to solve global optimization problems. Especially, as shown in experiments, the underlying heavy tailed alpha-stable distribution enables a considerably more effective global search in more complex problems.

In this paper, we consider online separation and detection of superimposed events by applying particle filtering. We concentrate on a model where a background process, represented by a ID-signal, is superimposed by an auto-regressive (AR) 'event signal', but the proposed approach is applicable in a more general setting. The activation and deactivation times of the event-signal are assumed to be unknown. We solve the online detection problem of this superpositional event by extending the state space dimension by one. The additional parameter of the state represents the AR-signal, which is zero when deactivated. Numerical experiments demonstrate the effectiveness of our approach.

A video based method to detect volatile organic compounds (VOC) leaking out of process equipments used in petrochemical refineries is developed. Leaking VOC plume from a damaged component causes edges present in image frames loose their sharpness. This leads to a decrease in the high frequency content of the image. The background of the scene is estimated and decrease of high frequency energy of the scene is monitored using the spatial wavelet transforms of the current and the background images. Plume regions in image frames are analyzed in low-band sub-images, as well. Image frames are compared with their corresponding low-band images. A maximum likelihood estimator (MLE) for adaptive threshold estimation is also developed in this paper.

It is generally accepted that video surveillance system operators lose their concentration after a short period of time and may miss important events taking place. In addition, many surveillance systems are frequently left unattended. Because of these reasons, automated analysis of the live video feed and automatic detection of suspicious activity have recently gained importance. To prevent capture of their images, criminals resort to several techniques such as deliberately obscuring the camera view, covering the lens with a foreign object, spraying or defocusing the camera lens. In this paper, we propose some computationally efficient wavelet domain methods for rapid camera tamper detection and identify some real-life problems and propose solutions to these.

This paper describes an online learning based method to detect flames in video by processing the data generated by an ordinary camera monitoring a scene. Our fire detection method consists of weak classifiers based on temporal and spatial modeling of flames. Markov models representing the flame and flame colored ordinary moving objects are used to distinguish temporal flame flicker process from motion of flame colored moving objects. Boundary of flames are represented in wavelet domain and high frequency nature of the boundaries of fire regions is also used as a clue to model the flame flicker spatially. Results from temporal and spatial weak classifiers based on flame flicker and irregularity of the flame region boundaries are updated online to reach a final decision. False alarms due to ordinary and periodic motion of flame colored moving objects are greatly reduced when compared to the existing video based fire detection systems.

3D video codecs have attracted recently a lot of attention, due to their compression performance comparable with that of state-of-art hybrid codecs and due to their scalability features. In this work, we propose a least mean square (LMS) based adaptive prediction for the temporal prediction step in lifting implementation. This approach improves the overall quality of the coded video, by reducing both the blocking and ghosting artefacts. Experimental results show that the video quality as well as PSNR values are greatly improved with the proposed adaptive method, especially for video sequences with large contrast between the moving objects and the background and for sequences with illumination variations

In this paper, images in Color Filter Array (CFA) format are compressed without converting them to full-RGB color images. Green pixels are extracted from the CFA image data and placed in a rectangular array, and compressed using a transform based method without estimating the corresponding luminance values. In addition, two sets of color difference (or chrominance) coefficients are obtained corresponding to the red and blue pixels of the CFA data and they are also compressed using a transform based method. The proposed method produces better PSNR values compared to the standard approach of bilinear interpolation followed by compression.

A method for detection of tree branches and leaves in video is proposed. It is observed that the motion vectors of tree branches and leaves exhibit random motion. On the other hand regular motion of green colored objects has well-defined directions. In this paper, the wavelet transform of motion vectors are computed and objects are classified according to the wavelet coefficients of motion vectors. Color information is also used to reduce the search space in a given image frame of the video. Motion trajectories of moving objects are modeled as Markovian processes and hidden Markov models (HMMs) are used to classify the green colored objects in the final step of the algorithm.

Lifting-style implementations of wavelets are widely used in image coders. A two-dimensional (2-D) edge adaptive lifting structure, which is similar to Daubechies 5/3 wavelet, is presented. The 2-D prediction filter predicts the value of the next polyphase component according to an edge orientation estimator of the image. Consequently, the prediction domain is allowed to rotate +/-45 degrees in regions with diagonal gradient. The gradient estimator is computationally inexpensive with additional costs of only six subtractions per lifting instruction, and no multiplications are required.

The paper proposes a novel method to detect fire and/or flame by processing the video data generated by an ordinary camera monitoring a scene. In addition to ordinary motion and color clues, flame and fire flicker are detected by analyzing the video in the wavelet domain. Periodic behavior in flame boundaries is detected by performing a temporal wavelet transform. Color variations in fire are detected by computing the spatial wavelet transform of moving fire-colored regions. Other clues used in the fire detection algorithm include irregularity of the boundary of the fire-colored region and the growth of such regions in time. All of the above clues are combined to reach a final decision.

Insect damaged wheat kernels (IDK) are characterized by a small hole bored into the kernel by insect larvae. This damage decreases flour quality as insect proteins interfere with the bread-making biochemistry and insect fragments are very unsightly. A prototype system was set up to detect IDK by dropping them onto a steel plate and processing the acoustic signal generated when kernels impact the plate. The acoustic signal was processed by three different methods: (1) modeling of the signal in the time domain; (2) computing time domain signal variances in short time windows; and (3), analysis of the frequency spectra magnitudes. Linear discriminant analysis was used to select a subset of features and perform classification. 98% of un-damaged kernels and 84.4% of IDK were correctly classified.

We present a novel and computationally simple prediction stage in a Daubechies 5/3 - like lifting structure for lossless image compression. In the 5/3 wavelet, the prediction filter predicts the value of an odd-indexed polyphase component as the mean of its immediate neighbors belonging to the even-indexed polyphase components. The new edge adaptive predictor, however, predicts according to a local gradient direction estimator of the image. As a result, the prediction domain is allowed to flip + or -45 degrees with respect to the horizontal or vertical axes in regions with diagonal gradient. We have obtained good compression results with conventional lossless wavelet coders.

This paper proposes a novel method to detect flames in video by processing the data generated by an ordinary camera monitoring a scene. In addition to ordinary motion and color clues, flame flicker process is also detected by using a hidden Markov model. Markov models representing the flame and flame colored ordinary moving objects are used to distinguish flame flicker process from motion of flame colored moving objects. Spatial color variations in flame are also evaluated by the same Markov models, as well. These clues are combined to reach a final decision. False alarms due to ordinary motion of flame colored moving objects are greatly reduced when compared to the existing video based fire detection systems.

An algorithm was developed to separate pistachio nuts with closed shells from those with open shells. It was observed that upon impact on a steel plate, nuts with closed shells emit different sounds than nuts with open shells. Two feature vectors extracted from the sound signals were Mel cepstrum coefficients and eigenvalues obtained from the principle component analysis of the autocorrelation matrix of the signals. Classification of a sound signal was done by linearly combining feature vectors from both Mel cepstrum and PCA feature vectors. An important property of the algorithm is that it is easily trainable. During the training phase, sounds of the nuts with closed shells and open shells were used to obtain a representative vector of each class. The accuracy of closed shell nuts was more than 99% on the test set.

We used transmittance images and different learning algorithms to classify insect damaged and un-damaged wheat kernels. Using the histogram of the pixels of the wheat images as the feature, and the linear model as the learning algorithm, we achieved a false positive rate (1-specificity) of 0.2 at the true positive rate (sensitivity) of 0.8 and an area under the ROC curve (AUC) of 0.86. Combining the linear model and a radial basis function network in a committee resulted in a FP rate of 0.1 at the TP rate of 0.8 and an AUC of 0.92.

We present a computer vision based text and equation editor for LATEX. The user writes text and equations on paper and a camera attached to a computer records the actions of the user. In particular, positions of the pen-tip in consecutive image frames are detected. Next, directional and positional information about characters are calculated using these positions. Then, this information is used for on-line character classification. After characters and symbols are found, the corresponding LATEX code is generated.

We used transmittance images and different learning algorithms to classify insect damaged and un-damaged wheat kernels. Using the histogram of the pixels of the wheat images as the feature, and the linear model as the learning algorithm, we achieved a false positive rate (1-specificity) of 0.12 at the true positive rate (sensitivity) of 0.8 and an area under the ROC curve (AUC) of 0.90/spl plusmn/0.02. Combining the linear model and a radial basis function network in a committee resulted in a FP rate of 0.09 at the TP Rate of 0.8 and an AUC of 0.93/spl plusmn/0.03.

An affine invariant function for object recognition is constructed from wavelet coefficients of the object boundary. In previous works, undecimated dyadic wavelet transform was used to construct affine invariant functions. In this paper, an algorithm based on decimated wavelet transform is developed to compute an affine invariant function. As a result computational complexity is reduced without decreasing recognition performance. Experimental results are presented.

There is an accelerating demand to access the visual content of documents stored in historical and cultural archives. Availability of electronic imaging tools and effective image processing techniques makes it feasible to process the multimedia data in large databases. In this paper, a framework for content-based retrieval of historical documents in the Ottoman Empire archives is presented. The documents are stored as textual images, which are compressed by constructing a library of symbols occurring in a document, and the symbols in the original image are then replaced with pointers into the codebook to obtain a compressed representation of the image. The features in wavelet and spatial domain based on angular and distance span of shapes are used to extract the symbols. In order to make content-based retrieval in historical archives, a query is specified as a rectangular region in an input image and the same symbol-extraction process is applied to the query region. The queries are processed on the codebook of documents and the query images are identified in the resulting documents using the pointers in textual images. The querying process does not require decompression of images. The new content-based retrieval framework is also applicable to many other document archives using different scripts.

In this paper, an affine invariant function is presented for object recognition from wavelet coefficients of the object boundary. In previous works, undecimated wavelet transform was used for affine invariant functions. In this paper, an algorithm based on decimated wavelet transform is developed to compute the affine invariant function. As a result, computational complexity is significantly reduced without decreasing recognition performance. Experimental results are presented.

A multi-stage moving object detection algorithm in video is described in this paper. First, the camera motion is eliminated by motion compensation. An adaptive subband decomposition structure is then used to analyze the difference image. In the high-band subimages, moving objects which produce outliers are detected using a statistical test determining non-Gaussian regions. It turns out that the distribution of the subimage pixels is almost Gaussian in general. But, at the object boundaries the distribution of the pixels in the subimages deviates from Gaussianity due to the existence of outliers. Regions containing moving objects in the original image frame are detected by detecting regions containing outliers in subimages. Finally, active contours are initiated in these regions in the wavelet domain and object boundaries are accurately estimated.

In this paper, a unistroke keyboard based on computer vision is described for the handicapped. The keyboard can be made of paper or fabric containing an image of a keyboard, which has an upside down U-shape. It can even be displayed on a computer screen. Each character is represented by a non-overlapping rectangular region on the keyboard image and the user enters a character by illuminating a character region with a laser pointer. The keyboard image is monitored by a camera and illuminated key locations are recognized. During the text entry process the user neither have to turn the laser light off nor raise the laser light from the keyboard. A disabled person who has difficulty using his/her hands may attach the laser pointer to an eyeglass and easily enter text by moving his/her head to point the laser beam on a character location. In addition, a mouse-like device can be developed based on the same principle. The user can move the cursor by moving the laser light on the computer screen which is monitored by a camera.

In this paper, computer-aided diagnosis of microcalcification clusters in mammogram images is considered. Microcalcification clusters which are an early sign of breast cancer appear as isolated bright spots in mammogram images. Therefore they correspond to local maxima of the mammogram image. In our method we first detect the local maxima of the image, and rank the maxima according to a higher order statistical test performed over the subband domain data obtained by the adaptive wavelet transform.

We describe a computer vision based mouse, which can control and command the cursor of a computer or a computerized system using a camera. In order to move the cursor on the computer screen the user simply moves the mouse shaped passive device placed on a surface within the viewing area of the camera. The video generated by the camera is analyzed using computer vision techniques and the computer moves the cursor according to mouse movements. The computer vision based mouse has regions corresponding to buttons for clicking. To click a button the user simply covers one of these regions with his/her finger.

A subband decomposition based lossless image compression algorithm based on adaptive methods is described. The decomposition is achieved by a two-channel adaptive filter bank. The resulting coefficients are lossy coded first, and then the residual error between the lossy and error free coefficients are compressed. The locations and the magnitudes of the nonzero coefficients are encoded separately by a hierarchical enumerative coding method. The locations of the nonzero coefficients in child bands are predicted from those in the parent band. The proposed compression algorithm, on the average, provides higher compression ratios than the state-of-the-art methods.

We present a new image denoising method based on adaptive subband decomposition (or adaptive wavelet transform) in which the filter coefficients are updated according to a least mean square (LMS) type algorithm. Adaptive subband decomposition filter banks have the perfect reconstruction property. Since the adaptive filter bank adjusts itself to the changing input environment, denoising is more effective compared to fixed filter banks. Simulation examples are presented.

A novel inverse halftoning method is proposed to restore a continuous tone image from a given half-tone image. A set theoretic formulation is used where three sets are defined using the prior information about the problem. A new space-domain projection is introduced assuming the halftoning is performed using error diffusion, and the error diffusion filter kernel is known. The space-domain, frequency-domain, and space-scale domain projections are used alternately to obtain a feasible solution for the inverse halftoning problem which does not have a unique solution.

In this paper, we propose a method for detection of small moving objects in video. We first eliminate the camera motion using motion compensation. We then use an adaptive predictor to estimate the current pixel using neighboring pixels in the motion compensated image and, in this way, obtain a residual error image. Small moving objects appear as outliers in the residual image and are detected using a statistical Gaussianity detection test based on higher order statistics. It turns out that in general, the distribution of the residual error image pixels is almost Gaussian. On the other hand, the distribution of the pixels in the residual image deviates from Gaussianity in the existence of outliers. Simulation examples are presented.

Subband decomposition techniques have been extensively used for data coding and analysis. In most filter banks, the goal is to obtain subsampled signals corresponding to different spectral regions of the original data. However, this approach leads to various artifacts in images having spatially varying characteristics, such as images containing text, subtitles, or sharp edges. In this paper, adaptive filter banks with perfect reconstruction property are presented for such images. The filters of the decomposition structure which can be either linear or nonlinear vary according to the nature of the signal. This leads to improved image compression ratios. Simulation examples are presented.

Halftoning is a process that deliberately injects noise into the original image in order to obtain visually pleasing output images with a smaller number of bits per pixel for displaying or printing purposes. In this paper, a novel inverse halftoning method is proposed to restore a continuous tone image from the given halftone image. A set theoretic formulation is used where three sets are defined using the prior information about the problem. A new space domain projection is introduced assuming the halftoning is performed with error diffusion, and the error diffusion filter kernel is known. The space domain, frequency domain, and space-scale domain projections are used alternately to obtain a feasible solution for the inverse halftoning problem which does not have a unique solution.

In this paper, a fixed point variable bit-rate (VBR) mixed excitation linear predictive coding (MELP/sup TM/) vocoder is presented. The VBR-MELP vocoder is also implemented on a TMS320C54x and it achieves virtually indistinguishable federal standard MELP quality at bit-rates between 1.0 to 1.6 kb/s. The backbone of VBR-MELP vocoder is similar to that of federal standard MELP. It utilizes a novel sub-band based voice activity detector in the back-end of encoder to discriminate background noise from speech activity. Since proposed detector uses only parameters extracted in the encoder, its computational complexity is very low.

In this paper, a new set of speech feature parameters based on multirate signal processing and the Teager energy operator is developed. The speech signal is first divided into nonuniform subbands in mel-scale using a multirate filter-bank, then the Teager energies of the subsignals are estimated. Finally, the feature vector is constructed by log-compression and inverse DCT computation. The new feature parameters have a robust speech recognition performance in car engine noise which is low pass in nature.

In this paper, computer-aided diagnosis of microcalcifications in mammogram images is considered. Microcalcification clusters are an early sign of breast cancer. Microcalcifications appear as single bright spots in mammogram images. We propose an effective method for the detection of these abnormalities. The first step of this method is two-dimensional adaptive filtering. The filtering produces an error image which is divided into overlapping square regions. In each square region, a Gaussianity test is applied. Since microcalcifications have an impulsive appearance, they are treated as outliers. In regions with no microcalcifications, the distribution of the error image is almost Gaussian, on the other hand, in regions containing microcalcification clusters, the distribution deviates from Gaussianity. Using the theory of the influence function and sensitivity curves, we develop a Gaussianity test. Microcalcification clusters are detected using the Gaussianity test. Computer simulation studies are presented.

In this letter, a new set of speech feature parameters based on multirate signal processing and the Teager energy operator is introduced. The speech signal is first divided into nonuniform subbands in mel-scale using a multirate filterbank, then the Teager energies of the subsignals are estimated. Finally, the feature vector is constructed by log-compression and inverse discrete cosine transform (DCT) computation. The new feature parameters have robust speech recognition performance in the presence of car engine noise.

In this work, a subband domain textual image compression method is developed. The document image is first decomposed into subimages using binary subband decompositions. Next, the character locations in the subbands and the symbol library consisting of the character images are encoded. The method is suitable for keyword search in the compressed data. It is observed that very high compression ratios are obtained with this method. Simulation studies are presented.

Subband decomposition techniques have been extensively used for data coding and analysis. In most filter banks, the goal is to obtain subsampled signals corresponding to different spectral bands of the original data. However, this approach leads to various artifacts in images containing text, subtitles, or sharp edges. In this paper, adaptive filter banks with perfect reconstruction property are presented for such images. The filters of the decomposition structure vary according to the nature of the signal. This leads to higher compression ratios for images containing subtitles compared to fixed filter banks. Simulation examples are presented.

Printing color images on color printers, and displaying them on computer monitors requires a significant reduction of physically distinct colors, which causes degradation in image quality. An efficient method to improve the display quality of a quantized image is error diffusion which works by distributing the previous quantization errors to neighboring pixels exploiting the eye's averaging of colors in the neighborhood of the point of interest. This creates the illusion of more colors. A new error diffusion method is presented in which the adaptive recursive least squares (RLS) algorithm is used rather than the deterministic approaches in literature. To improve the performance, a diagonal scan is used in processing the image.

Computer-aided detection and enhancement of microcalcifications in mammogram images are considered. The mammogram image is first decomposed into subimages using a 'subband' decomposition filter bank which uses nonlinear filters. A suitably identified subimage is divided into overlapping square regions in which skewness and kurtosis as measures of the asymmetry and impulsiveness of the distribution are estimated. All regions with high positive skewness and kurtosis are marked as a regions of interest. Next, an outlier labeling method is used to find the locations of microcalcifications in these regions. An enhanced mammogram image is also obtained by emphasizing the microcalcification locations. Linear and nonlinear subband decomposition structures are compared in terms of their effectiveness in finding microcalcificated regions and their computational complexity. Simulation studies based on real mammogram images are presented.

We present a new video shot classification and clustering technique to support content-based indexing, browsing and retrieval in video databases. The proposed method is based on the analysis of both the audio and visual data tracks. The visual stream is analyzed using a 3-D wavelet transform and segmented into shot units which are matched and clustered by visual content. Simultaneously, speaker changes are detected by tracking voiced phonemes in the audio signal. The clues obtained from the video and speech data are combined to classify and group the isolated video shots. This integrated approach also allows effective indexing of the audio-visual objects in multimedia databases.

A new method for detecting microcalcifications in mammograms is described. In this method, the mammogram image is first processed by a subband decomposition filterbank. The bandpass subimage is divided into overlapping square regions in which skewness and kurtosis as measures of the asymmetry and impulsiveness of the distribution are estimated. The detection method utilizes these two parameters. A region with high positive skewness and kurtosis is marked as a region of interest. Simulation results show that this method is successful in detecting regions with microcalcifications.

Most color image printing and display devices do not have the capability of reproducing true color images. A common remedy is the use of dithering techniques that take advantage of the lower sensitivity of the eye to spatial resolution and exchange higher color resolution with lower spatial resolution. An adaptive error diffusion method for color images is presented. The error diffusion filter coefficients are updated by a normalized least mean square-type (LMS-type) algorithm to prevent textural contours, color impulses, and color shifts, which are among the most common side effects of the standard dithering algorithms. Another novelty of the new method is its vector character: previous applications of error diffusion have treated the individual color components of an image separately. We develop a general vector approach and demonstrate through simulation studies that superior results are achieved.

Efficient compression of binary textual images is very important for applications such as document archiving and retrieval, digital libraries and facsimile. The basic property of a textual image is the repetitions of small character images and curves inside the document. Exploiting the redundancy of these repetitions is the key step in most of the coding algorithms. We use a similar compression method in the subband domain. Four different subband decomposition schemes are described and their performance on a textual image compression algorithm is examined. Experimentally, it is found that the described methods accomplish high compression ratios and they are suitable for fast database access and keyword search.

Linear filter banks with critical subsampling and perfect reconstruction (PR) property have received much interest and found numerous applications in signal and image processing. Nonlinear filter bank structures with PR and critical subsampling have been proposed and used in image coding. It is shown that PR nonlinear subband decomposition can be performed using the Galois field (GF) arithmetic. The result of the decomposition of an n-ary (e.g. 256-ary) input signal is still n-ary at different resolutions. This decomposition structure can be utilized for binary and 2/sup k/ (k is an integer) level signal decompositions. Simulation studies are presented.

A large class of physical phenomenon observed in practice exhibit non-Gaussian behavior. In this paper, /spl alpha/-stable distributions, which have heavier tails than Gaussian distribution, are considered to model non-Gaussian signals. Adaptive signal processing in the presence of such kind of noise is a requirement of many practical problems. Since, direct application of commonly used adaptation techniques fail in these applications, new approaches for adaptive filtering for /spl alpha/-stable random processes are introduced.

A new set of speech feature representations for robust speech recognition in the presence of car noise is proposed. These parameters are based on subband analysis of the speech signal. Line spectral frequency (LSF) representation of the linear prediction (LP) analysis in subbands and cepstral coefficients derived from subband analysis (SUBCEP) are introduced, and the performances of the new feature representations are compared to mel scale cepstral coefficients (MELCEP) in the presence of car noise. Subband analysis based parameters are observed to be more robust than the commonly employed MELCEP representations.

Most color image printing and display devices do not have the capability of reproducing true color images. A common remedy is the use of dithering techniques that exploit the lower sensitivity of the eye to spatial resolution and exchange higher color resolution with lower spatial resolution. In this paper an adaptive error diffusion method is presented. The error diffusion filter coefficients are updated by a normalized LMS type algorithm to prevent textural contours, color impulses and color shifts which are among the the most common side effects of the standard dithering algorithms.

The wavelet transform provides a multiresolution representation of images. Edges, which are visually important, produce large coefficients across several scales in the wavelet transform domain. By tracking and predicting these edge coefficients across scales in the wavelet transform domain, we can greatly improve the compressed image quality with little degradation in compression ratio. This paper proposes a novel model-based edge tracking and prediction in the wavelet domain. It separates textures from edges and codes them differently. Edges are coded via an edge tracking and prediction, while textures are coded with either ordinary wavelet based image coding techniques or a "wavelet-like" filter bank which is similar to the tuning channels in the human vision system. The coding noise is then coded with a noise modelling. Visual masking models are also used to ensure the compressed image has little or almost no perceptual distortion.

A nonlinear least-squares (LS) method is developed for modeling empirically obtained data in array signal processing. The new method is robust with respect to modeling errors in the noise distribution. Robustness is achieved by introducing a nonlinear function which weights the squared error term in the LS criterion. Weighting functions for various observation noise scenarios are determined by using maximum likelihood estimation theory. The computational complexity of the new method is comparable to the standard least-squares estimation procedures. Simulation examples of direction-of-arrival (DOA) estimation are presented.>

A large class of physical phenomena observed in practice exhibit non-Gaussian behavior. In the letter /spl alpha/-stable distributions, which have heavier tails than Gaussian distributions, are considered to model non-Gaussian signals. Adaptive signal processing in the presence of such a noise is a requirement of many practical problems. Since direct application of commonly used adaptation techniques fail in these applications, new algorithms for adaptive filtering for /spl alpha/-stable random processes are introduced.>

Discrete cosine transform (DCT) coding is widely used for compression of rectangularly sampled images. We address efficient DCT coding of nonrectangularly sampled images. To this effect, we discuss an efficient method for the computation of the DCT on nonrectangular sampling grids using the Smith-normal decomposition. Simulation results are provided.>

Line spectrum frequencies (LSF's) uniquely represent the linear predictive coding (LPC) filter of a speech frame. In many vocoders LSF's are used to encode the LPC parameters. In this paper, an interframe differential coding scheme is presented for the LSF's. The LSF's of the current speech frame are predicted by using both the LSF's of the previous frame and some of the LSF's of the current frame. Then, the difference resulting from prediction is quantized.>

A new class of block transforms is presented. These transforms are constructed from subband decomposition filter banks corresponding to regular wavelets. New transforms are compared to the discrete cosine transform (DCT). Image coding schemes that use the block wavelet transform (BWT) are developed. BWT's can be implemented by fast (O(N log N)) algorithms.>

A class of biorthogonal systems leading to linear-phase wavelets is presented. A notable feature of this structure is that the wavelets are derived from a filter bank where the lowpass analysis filter is constrained to be a halfband filter. The authors derive finite impulse response (FIR) biorthogonal solutions from a pair of Lagrange halfband filters. They also consider infinite impulse response (IIR) biorthogonal solutions based on a pair of zero-phase halfband filters derived from Butterworth halfband filters.>

The authors have described the framework underlying continuous and discrete families of nonseparable two-dimensional wavelets and an M-channel nonrectangular multiresolution wavelet representation for L/sup 2/(R/sup 2/) functions and I/sup 2/(Z/sup 2/) sequences. Focusing on the case of quincunx sampled signals, solutions of digital filter banks for implementing the decomposition with different characteristics of linear phase and regularity for smoothing and wavelet functions are provided.>

The author presents a procedure for reconstructing the impulse response of a minimum- or nonminimum-phase linear time-invariant (LTI) system from its bispectrum. The algorithm is iterative and uses the method of projections onto convex sets (POCS). Prior information such as an energy bound on the impulse response sequence of the LTI system can be incorporated into the algorithm. Corresponding to a given bispectrum and other prior information such as energy, closed and convex sets in an inner product space are constructed. The iterative algorithm consists of successively projecting an initial guess onto these closed and convex sets. Convergence of the algorithm regardless of the initial guess is assured. An algorithm that employs energy information was found to produce better results in simulation that one that ignores energy constraints.>

In this paper, a new iterative method for any shaped pattern synthesis for one- and two-dimensional antenna arrays is described. The object is to meet prescribed constraints on the power pattern, with an assumption on the number of antenna array elements at specified locations. The synthesis problem is solved using the method of projection on convex sets, by modeling the constraints in terms of convex sets whose elements are vectors representing the excitation coefficients. Two underlying frameworks used in solving the problem are the l2Hilbert Space and an inner product space based on convolution. The synthesis procedure is implemented using a Fast Fourier Transform algorithm.