Jiahui Geng, Fengyu Cai, Shaobo Cui, Qing Li, Liangwei Chen, Chenyang Lyu, Haonan Li, Derui Zhu, Alexander Pretschner, Heinz Koeppl, Fakhri Karray. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2026.

Warning: This paper contains offensive content that may disturb some readers. Vision-language models (VLMs) demonstrate strong multimodal capabilities but have been found to be more susceptible to generating harmful content compared to their backbone large language models (LLMs). Our investigation reveals that the integration of images significantly shifts the model's internal activations during the forward pass, diverging from those triggered by textual input. Moreover, the safety alignments of LLMs embedded within VLMs are not sufficiently robust to handle the activations discrepancies, making the models vulnerable to even the simplest jailbreaking attacks. To address this issue, we propose an internal activation revision approach that efficiently revises activations during generation, steering the model toward safer outputs. Our framework incorporates revisions at both the layer and head levels, offering control over the model's generation at varying levels of granularity. In addition, we explore three strategies for constructing positive and negative samples and two approaches for extracting revision vectors, resulting in different variants of our method. Comprehensive experiments demonstrate that the internal activation revision method significantly improves the safety of widely used VLMs, reducing attack success rates by an average of 48.94%, 34.34%, 43.92%, and 52.98% on SafeBench, Safe-Unsafe, Unsafe, and MM-SafetyBench, respectively, while minimally impacting model helpfulness.

6173

Advancements in face recognition (FR) technologies have amplified privacy concerns, necessitating methods that protect identity while maintaining recognition utility. Existing face anonymization methods typically focus on obscuring identity but fail to meet the requirements of biometric template protection, including revocability, unlinkability, and irreversibility. We propose FaceAnonyMixer, a cancelable face generation framework that leverages the latent space of a pre-trained generative model to synthesize privacy-preserving face images. The core idea of FaceAnonyMixer is to irreversibly mix the latent code of a real face image with a synthetic code derived from a revocable key. The mixed latent code is further refined through a carefully designed multi-objective loss to satisfy all cancelable biometric requirements. FaceAnonyMixer is capable of generating high-quality cancelable faces that can be directly matched using existing FR systems without requiring any modifications. Extensive experiments on benchmark datasets demonstrate that FaceAnonyMixer delivers superior recognition accuracy while providing significantly stronger privacy protection, achieving over an 11% absolute gain on commercial API compared to recent cancelable biometric methods. Code is available at: https://github.com/talha-alam/faceanonymixer

Federated Learning (FL) combined with Parameter-Efficient Fine-Tuning (PEFT) methods, such as Low-Rank Adaptation (LoRA) has emerged as a promising approach to address data scarcity challenges in fine-tuning Large Language Models (LLMs) while ensuring privacy and computational efficiency. However, when applying LoRA in traditional FL, separately averaging the adapters during aggregation results in non-exact aggregation. While recent research has investigated this issue, its application to heterogeneous data settings remains largely unexplored. Data heterogeneity across clients can significantly affect the effectiveness of parameter-efficient adaptations and complicate the aggregation process.In this work, we explore the concept of exact aggregation in heterogeneous federated fine-tune settings, specifically focusing on LoRA-based approaches. We propose HEXA (Heterogeneity-aware EXact Aggregation), a novel method that mitigates the effects of data heterogeneity while preserving the benefits of exact aggregation in LoRA-enabled FL. We present a comprehensive theoretical framework for extending exact aggregation to heterogeneous settings and validate our approach through extensive empirical evaluation on the GLUE benchmark. Our results show that HEXA improves model performance in heterogeneous contexts while maintaining the computational efficiency of PEFT methods.

Microscopic assessment of histopathology images is vital for accurate cancer diagnosis and treatment. Whole Slide Image (WSI) classification and captioning have become crucial tasks in computer-aided pathology. However, microscopic WSIs face challenges such as redundant patches and unknown patch positions due to subjective pathologist captures. Moreover, generating automatic pathology captions remains a significant challenge. To address these challenges, a novel GNN-ViTCap framework is introduced for classification and caption generation from histopathological microscopic images. A visual feature extractor is used to extract feature embeddings. The redundant patches are then removed by dynamically clustering images using deep embedded clustering and extracting representative images through a scalar dot attention mechanism. The graph is formed by constructing edges from the similarity matrix, connecting each node to its nearest neighbors. Therefore, a graph neural network is utilized to extract and represent contextual information from both local and global areas. The aggregated image embeddings are then projected into the language model’s input space using a linear layer and combined with input caption tokens to fine-tune the large language models for caption generation. Our proposed method is validated using the BreakHis and PatchGastric microscopic datasets. The GNN-ViTCap method achieves an F1-Score of 0.934 and AUC of 0.963 for classification, along with BLEU@4 = 0.811 and METEOR = 0.569 for captioning. Experimental analysis demonstrates that the GNN-ViTCap architecture outper-forms state-of-the-art (SOTA) approaches, providing a reliable and efficient approach for patient diagnosis using microscopy images.

Adverse weather conditions, particularly fog, pose a significant challenge to autonomous vehicles, surveillance systems, and other safety-critical applications by severely degrading visual information. We introduce ADAM-Dehaze, an adaptive, density-aware dehazing framework that jointly optimizes image restoration and object detection under varying fog intensities. First, a lightweight Haze Density Estimation Network (HDEN) classifies each input as light, medium, or heavy fog. Based on this score, the system dynamically routes the image through one of three CORUN branches—Light, Medium, or Complex—each tailored to its haze regime. A novel adaptive loss then balances physical-model coherence and perceptual fidelity, ensuring both accurate defogging and preservation of fine details. On Cityscapes and the real-world RTTS benchmark, ADAM-Dehaze boosts PSNR by up to 2.1 dB, reduces FADE by 30%, and improves object detection mAP by up to 13 points, all while cutting inference time by 20%. These results demonstrate the necessity of intensity-specific processing and seamless integration with downstream vision tasks for robust performance in foggy weather conditions.

Depression is a serious mental health illness that significantly affects an individual’s well-being and quality of life, making early detection crucial for adequate care and treatment. Detecting depression is often difficult, as it is based primarily on subjective evaluations during clinical interviews. Hence, the early diagnosis of depression, thanks to the content of social networks, has become a prominent research area. The extensive and diverse nature of user-generated information poses a significant challenge, limiting the accurate extraction of relevant temporal information and the effective fusion of data across multiple modalities. This paper introduces MMF-former, a multimodal depression detection network designed to retrieve depressive spatio-temporal high-level patterns from multimodal social media information. The transformer network with residual connections captures spatial features from videos, and a transformer encoder is exploited to design important temporal dynamics in audio. Moreover, the fusion architecture fused the extracted features through late and intermediate fusion strategies to find out the most relevant intermodal correlations among them. Finally, the proposed network is assessed on two large-scale depression detection datasets, and the results clearly reveal that it surpasses existing state-of-the-art approaches, improving the F1-Score by 13.92% for D-Vlog dataset and 7.74% for LMVD dataset. The code is made available publicly at https://github.com/rezwanh001/Large-Scale-Multimodal-Depression-Detection.

Depression is a major mental health condition that severely impacts the emotional and physical well-being of individuals. The simple nature of data collection from social media platforms has attracted significant interest in properly utilizing this information for mental health research. A Multimodal Depression Detection Network (MDD-Net), utilizing acoustic and visual data obtained from social media networks, is proposed in this work where mutual transformers are exploited to efficiently extract and fuse multimodal features for efficient depression detection. The MDD-Net consists of four core modules: an acoustic feature extraction module for retrieving relevant acoustic attributes, a visual feature extraction module for extracting significant high-level patterns, a mutual transformer for computing the correlations among the generated features and fusing these features from multiple modalities, and a detection layer for detecting depression using the fused feature representations. The extensive experiments are performed using the multimodal D-Vlog dataset, and the findings reveal that the developed multimodal depression detection network surpasses the state-of-the-art by up to 17.37% for F1-Score, demonstrating the greater performance of the proposed system. The source code is accessible at https://github.com/rezwanh001/Multimodal-Depression-Detection.

Recommending cold items remains a significant challenge in billion-scale online recommendation systems. While warm items benefit from historical user behaviors, cold items rely solely on content features, limiting their recommendation performance and impacting user experience and revenue. Current models generate synthetic behavioral embeddings from content features but fail to address the core issue: the absence of historical behavior data. To tackle this, we introduce the LLM Simulator framework, which leverages large language models to simulate user interactions for cold items, fundamentally addressing the cold-start problem. However, simply using LLM to traverse all users can introduce significant complexity in billion-scale systems. To manage the computational complexity, we propose a coupled funnel ColdLLM framework for online recommendation. ColdLLM efficiently reduces the number of candidate users from billions to hundreds using a trained coupled filter, allowing the LLM to operate efficiently and effectively on the filtered set. Extensive experiments show that ColdLLM significantly surpasses baselines in cold-start recommendations, including Recall and NDCG metrics. A two-week A/B test also validates that ColdLLM can effectively increase the cold-start period GMV.

In this paper, we explain the mechanism of bilinear pooling as a module of hard sample generation, and find that bilinear pooling significantly expands variances of the first-order vectors when it produces discriminative bilinear features. In conjunction with the extremely high dimensionality of the obtained bilinear features, those variances lead to overfitting in subsequent learning models. To solve this issue, we construct a bi-level optimization problem, where the high-level problem is the supervised classification loss, and the low-level problem is the principal component analysis (PCA). Then, we find that PCA on bilinear features is equivalent to spectral clustering, which allows us to mathematically prove that the first $\log _{2}(C)$log2(C) principal components can support the discriminant information of $C$C classes. By removing the rest principal components, the dimensionality and variances are simultaneously reduced. To the best of our knowledge, this is the first work providing a lower bound for dimension reduction for bilinear pooling. However, the PCA projection matrix $\mathbf{L}$L is prone to overfitting due to having many parameters. To address this issue, we propose a rank-$k$k general bilinear projection (RK-GBP) that decomposes $\mathbf{L}$L into two small matrices $\mathbf{U}$U and $\mathbf{V}$V, whose learnable parameters are smaller. Different from traditional bilinear projections used in factorized bilinear pooling (FBiP), our RK-GBP can preserve the orthogonality of columns in $\mathbf{L}$L by constraining the orthogonality of columns in $\mathbf{U}$U and $\mathbf{V}$V. For computational efficiency, we relax the PCA in the low-level task into a dictionary learning problem, obtaining the rank-$k$k orthogonal factorization bilinear pooling (RK-OFBP). The RK-OFBP can be considered as a general form of current factorization bilinear pooling methods (e.g., Hadamard product-based ones). Finally, we evaluate our approach on fine-grained images and large-scale datasets, demonstrating that our proposed method not only produces extremely low-dimensional features but also outperforms other methods in classification tasks. For example, our RK-OFBP can employ 32-dimensional vectors to achieve comparable results to B-CNN (Lin, 2015) (dimension: 512*512) for the 200-class classification task.

In this paper, we find that the training of Normalizing Flows (NFs) are easily affected by the outliers and a small number (or high dimensionality) of training samples. To solve this problem, we propose a Kullback–Leibler (KL) divergence regularization on the Jacobian matrix of NFs. We prove that such regularization is equivalent to adding a set of samples whose covariance matrix is the identity matrix to the training set. Thus, it reduces the negative influence of the outliers and the small sample number on the estimation of the covariance matrix, simultaneously. Therefore, our regularization makes the training of NFs robust. Ultimately, we evaluate the performance of NFs on out-of-distribution (OoD) detection tasks. The excellent results obtained demonstrate the effectiveness of the proposed regularization term. For example, with the help of the proposed regularization, the OoD detection score increases at most 30% compared with the one without the regularization.

Domain knowledge exists in various forms, including text, ontologies, graphs, images, audio, and videos. In plant disease detection, most works solely utilize images with disease labels, neglecting textual descriptions of visual disease symptoms used by human experts for diagnosis. These text descriptions and sample images aid expert identification of visual symptoms. We propose a novel method that leverages text descriptions and image data by modeling domain-specific knowledge about visual symptoms in leaf images as separate feature channels. Each channel corresponds to specific features whose absence or presence in the image influences model predictions. We introduce a channel attention-guided fusion module for weighting each channel based on the input and corresponding output. The combined feature channels are transformed into a standardized 3-channel input format, which can then be processed by any pre-trained convolutional neural network (CNN) as input for feature extraction and subsequent classification. Furthermore, intermediate activations of the channel attention layer combined with the weights from the fusion layer make model predictions explainable. Experimental results on three publicly available datasets of apple and cucumber leaf diseases demonstrate improvements of up to 5% utilizing various state-of-the-art CNN architectures, indicating the efficacy of incorporating textual disease descriptions using the proposed approach.

In this paper, we present a peer-to-peer (P2P) energy trading system designed for equitable energy exchanges within a microgrid. Our objective is to enhance the fairness of energy distribution among all microgrid members. To achieve this, we introduce a novel objective function that enables us to simultaneously maximize overall welfare and minimize disparities among microgrid participants, thus promoting equitable transactions. The core of our system is an agent proficiently trained via deep reinforcement learning, ensuring efficient and equitable energy distribution across the network. We simulated the energy consumption of 8 houses, and their energy production to assess the feasibility and efficiency of the proposed system. The utilization of the proposed approach has yielded promising results. The intelligent agent was able to execute transactions and improve the fairness of energy trading within the microgrid. The overall average welfare of the microgrid over a day increased by 99.8%, while the disparities among the members were reduced.

Captioning Whole Slide Images (WSIs) for pathological analysis is an essential but not extensively explored aspect of computer-aided pathological diagnosis. Challenges arise from insufficient datasets and the effectiveness of model training. Generating automatic caption reports for various gastric adenocarcinoma images is another challenge. In this paper, we introduce a hybrid method referred to as TransUAAE-CapGen to generate histopathological captions from WSI patches. The TransUAAE-CapGen architecture consists of a hybrid UNet-based Advereasrial Autoencoder (AAE) for feature extraction and a transformer for caption generation. The hybrid UNet-based AAE extracted complex tissue properties from histopathological patches, transforming them into low-dimensional embeddings. The embeddings are then fed into the transformer to generate concise captions. Our proposed method is validated using the PatchGastricADC22 dataset. The TransUAAE-CapGen model provides the best estimated accuracy of BLEU-4 = 86.8%, METEOR = 59.6%, a ROUGE = 89.3%, and CIDEr = 7.72%. Experimental analysis indicates that the TransUAAE-CapGen architecture outperforms the traditional LSTM-based model for the caption generation task. Our findings reveal that the proposed architecture can effectively generate accurate and precise reports for medical image analysis.

Camouflage Object Detection (COD) involves the challenge of isolating a target object from a visually similar background, presenting a formidable challenge for learning algorithms. Drawing inspiration from state-of-the-art (SOTA) Focal Modulation Networks, our objective is to proficiently modulate the foreground and background components, thereby capturing the distinct features of each. We introduce a Feature Split and Modulation (FSM) module to attain this goal. This module efficiently separates the object from the background by utilizing foreground and background modulators guided by a supervisory mask. For enhanced feature refinement, we propose a Context Refinement Module (CRM), which considers features acquired from FSM across various spatial scales, leading to comprehensive enrichment and highly accurate prediction maps. Through extensive experimentation, we showcase the superiority of CamoFocus over recent SOTA COD methods. Our evaluations encompass diverse benchmark datasets, including CAMO, COD10K, CHAMELEON, and NC4K. The findings underscore the potential and significance of the proposed CamoFocus model and establish its efficacy in addressing the critical challenges of camouflage object detection.

The emergence of the Internet of Things (IoT) has revolutionized people’s daily lives, providing superior quality services in cognitive cities, healthcare, and smart buildings. However, smart buildings use heterogeneous networks. The massive number of interconnected IoT devices increases the possibility of IoT attacks, emphasizing the necessity of secure and privacy-preserving solutions. Federated learning (FL) has recently emerged as a promising machine learning (ML) paradigm for IoT networks to address these concerns. In FL, multiple devices collaborate to learn a global model without sharing their raw data. However, FL still faces privacy and security concerns due to the transmission of sensitive data (i.e., model parameters) over insecure communication channels. These concerns can be addressed using fully homomorphic encryption (FHE), a powerful cryptographic technique that enables computations on encrypted data without requiring them to be decrypted first. In this study, we propose a secure FL approach in IoT-enabled smart cities that combines FHE and FL to provide secure data and maintain privacy in distributed environments. We present four different FL-based FHE approaches in which data are encrypted and transmitted over a secure medium. The proposed approaches achieved high accuracy, recall, precision, and F-scores, in addition to providing strong privacy and security safeguards. Furthermore, the proposed approaches effectively reduced communication overhead and latency compared to the baseline approach. These approaches yielded improvements ranging from 80.15% to 89.98% in minimizing communication overhead. Additionally, one of the approaches achieved a remarkable latency reduction of 70.38%. The implementation of these security models is nontrivial, and the code is publicly available athttps://github.com/Artifitialleap-MBZUAI/Secure-Federated-Learning-with-Fully-Homomorphic-Encryption-for-IoT-Communications.

Prejudice is a pre-conceived depiction of an entity within a person's mind. It tends to devalue people as a consequence of their perceived membership in a social group. The origin of prejudice can be traced back to the categorization process people use to form a plausible perception of their surroundings. The process of constructing these perceptions generally results in prejudices, which authorizes inequalities to develop across a variety of social groups. In all their forms, biases can be relayed in language by generalizing a negative adjective onto an social group as a function of prejudgement. Using this reduced linguistic formulation, we set out to (1) create a benchmark of 23,736 prejudiced sentences that encompass a plethora of bias types including racism, sexism, classism, ethnic discrimination, and religious discrimination; (2) propose a prejudice score that incorporates both the masked prediction probability and the top-k index (rank) of the matched word; (3) conduct a case study, using our benchmark, to evaluate bias in three pre-trained language models: BERT, DistilBERT, and Context-Debias DistilBERT.

The General Chairs and Technical Program Chairs (TPC) are delighted to welcome you to the International Conference on Blockchain and Cryptocurrency (ICBC 2023) being held as an inperson event between May 1 and May 5, 2023, at the Sofitel Downtown hotel Dubai, UAE.

The development of the Internet of Things (IoT) has dramatically expanded our daily lives, playing a pivotal role in the enablement of smart cities, healthcare, and buildings. Emerging technologies, such as IoT, seek to improve the quality of service in cognitive cities. Although IoT applications are helpful in smart building applications, they present a real risk as the large number of interconnected devices in those buildings, using heterogeneous networks, increases the number of potential IoT attacks. IoT applications can collect and transfer sensitive data. Therefore, it is necessary to develop new methods to detect hacked IoT devices. This paper proposes a Feature Selection (FS) model based on Harris Hawks Optimization (HHO) and Random Weight Network (RWN) to detect IoT botnet attacks launched from compromised IoT devices. Distributed Machine Learning (DML) aims to train models locally on edge devices without sharing data to a central server. Therefore, we apply the proposed approach using centralized and distributed ML models. Both learning models are evaluated under two benchmark datasets for IoT botnet attacks and compared with other well-known classification techniques using different evaluation indicators. The experimental results show an improvement in terms of accuracy, precision, recall, and F-measure in most cases. The proposed method achieves an average F-measure up to 99.9%. The results show that the DML model achieves competitive performance against centralized ML while maintaining the data locally.

Fresh produce (FP) yield forecasting is crucial for both: farmers to estimate fair prices for their crops and retailers to protect against highly priced FPs. To precisely forecast future yield, a deep learning forecasting model is proposed in this work which is trained and tested using relevant input parameters retrieved from satellite images and mapped to tabular yield data recorded for strawberry as an output parameter. To enhance the model performance, the preprocessing approaches and the set of input parameters are improved. The best satellite image preprocessing technique has to be found to represent the images with less data for efficiency. Therefore, a preprocessing approach based on averaging is proposed and implemented then compared with the literature approach which is based on histograms, where the proposed approach improved performance by 20%. The proposed Deep Feed Forward Neural Network with Embedded Gated Recurrent Units (DFNNGRU) ensembled with Attention Deep GRUs (ADGRU) is then tested against well-performing models of Stacked-AutoEncoder (SAE) ensembled with Convolution Neural Networks with Long-short term memory (CNNLSTM), where the proposed model is found to outperform the literature model by 12.5%. To have a better set of parameters, a Normalized Vegetation Difference Index (NDVI) is added to the input parameters which further enhances the performance by 2%. Finally, a quality assurance technique using metamorphic testing is applied and it is found that the model fulfills all expected metamorphic relations which proves the soundness and quality of the model as compared with other solutions.

The increasing number of applications and devices in the Sixth-generation (6G) networks and the diversity of mobile data, architectures, and technologies make security and privacy a critical concern. Advanced metaheuristics algorithms (MHAs) have recently become a viable solution for optimizing security and privacy in wireless networks, combining game theory and convex optimization, and several other advanced models. As a subfield of Artificial Intelligence (AI), MHAs are inspired by concepts from Evolutionary Algorithms (EAs), Trajectory-based Algorithms (TAs), and Swarm Intelligence (SI). Recent implementations of MHAs in the 6G networks have effectively solved complex security and privacy problems. This study examines MHAs’ utilization in addressing security and privacy challenges in 6G networks. The paper provides a comprehensive overview of MHAs and their use in solving security and privacy problems in 6G. The current limitations of the literature are also identified, and avenues for further research are suggested. The reader will have a clear image of the needed technologies and tools for securing 6G networks using MHAs.

Nowadays, the Internet has spread to practically every country around the world and is having unprecedented effects on people’s lives. The Internet of Things (IoT) is getting more popular and has a high level of interest in both practitioners and academicians in the age of wireless communication due to its diverse applications. The IoT is a technology that enables everyday things to become savvier, everyday computation toward becoming intellectual, and everyday communication to become a little more insightful. In this article, the most common and popular IoT device capabilities, architectures, and protocols are demonstrated in brief to provide a clear overview of the IoT technology to the researchers in this area. The common IoT device capabilities, including hardware (Raspberry Pi, Arduino, and ESP8266) and software (operating systems (OSs), and built-in tools) platforms are described in detail. The widely used architectures that have recently evolved and used are the three-layer architecture, service-oriented architecture, and middleware-based architecture. The popular protocols for IoT are demonstrated which include constrained application protocol, message queue telemetry transport, extensible messaging and presence protocol, advanced message queuing protocol, data distribution service, low power wireless personal area network, Bluetooth low energy, and ZigBee that are frequently utilized to develop smart IoT applications. Additionally, this research provides an in-depth overview of the potential healthcare applications based on IoT technologies in the context of addressing various healthcare concerns. Finally, this article summarizes state-of-the-art knowledge, highlights open issues and shortcomings, and provides recommendations for further studies which would be quite beneficial to anyone with a desire to work in this field and make breakthroughs to get expertise in this area.

Recently, IoT devices have become affordable to support various types of applications which have encouraged their usability in data collection, behavior tracking, and pattern analysis to gain knowledge to achieve certain goals. The Internet of Behavior (IoB) allows organizations and individuals to achieve all of this simultaneously. Various technologies and approaches can be used to support IoB systems to operate efficiently, such as 6G networks and decentralized systems structure that support IoT-based systems to distribute operations across devices and influence each device individually. Therefore, this paper proposes a decentralized IoB framework for achieving energy sustainability by tracking, analyzing, and influencing IoT devices’ behavior. The collected results from an extensive decentralized IoB electrical power consumption experiment show that the decentralized system achieved higher accuracy compared to the centralized system, thus sending 3.5% fewer alerts and saving 3.4% more power for 3 sub-meters over a period of 500 hours.

Accurate estimates of fresh produce (FP) yields and prices are crucial for having fair bidding prices by retailers along with informed asking prices by farmers, leading to the best prices for customers. To have accurate estimates, the state-of-the-art deep learning (DL) models for forecasting FP yields and prices are improved in this work while a novel transfer learning (TL) framework is proposed for better generalizability. The proposed models are trained and tested using real world datasets for the Santa Barbara region in California, which contain environmental input parameters mapped to FP yield and price output parameters. Based on an aggregated measure (AGM), the proposed model, an ensemble of Attention Deep Feedforward Neural Network with Gated Recurrent Unit (GRU) units and Deep Feedforward Neural Network with embedded GRU units, is found to significantly outperform the state-of-the-art models. Beside finding the best DL, the TL framework is utilizing FP similarity, clustering, and TL techniques customized to fit the problem in hand and enhance the model generalization to other FPs. The literature similarity algorithms are improved by considering the time series features rather than the absolute values of their points. In addition, the FPs are clustered using a hierarchical clustering technique utilizing the complete linkage of a dendrogram to automate the process of finding the similarity thresholds and avoid setting them arbitrarily. Finally, the transfer learning is applied by freezing some layers of the proposed ensemble model and fine-tuning the rest leading to significant improvement in AGM compared to the best literature model.

The problem of Emergency Department (ED) over-crowding is a worldwide public health issue that has several side effects, such as overworked medical staff, increased infections, and high mortality rates among patients. The process of conducting initial medical assessment and sorting for ED patients without the need for direct contact between medical staff and patients is called “remote triage”. In this work, we tackle the automation of this process. Three fully automated computer vision-based architectures for IoMT are proposed, namely home-based, portable and smart triage road units. The proposed methods utilize state-of-the-art deep learning architectures to automate the remote triage process. The utilized deep architectures are lightweight, thus, mobile-friendly, and capable of assigning triage scores to a broad spectrum of medical conditions. We furthermore formulate patients' ED wait time mathematically. We setup all architectures to consider EDs at a regional level in order to facilitate making a convenient ED choice for patients. Moreover, a novel ED selection criteria that considers the ED distance and the expected wait time is proposed in order to minimize the wait time and commuting distance for patients. Our experiments show an improvement in the quality and duration of patients' wait time throughout the triage process. Additionally, our experiments show that the proposed methods achieve accurate triage results with an average macro F-score of 97.8% with the capability of providing triage to 98 patients/second compared to the non-automated current approach followed in EDs which takes 15 minutes/patient in the best case.

The COVID-19 rapid antigen self-test kits are widely administered in several countries to increase the testing frequency and reduce the load on clinics for in-person tests. Yet, the telehealth worker supervision is mandatory to ensure proper sampling procedure is followed and high-quality swab samples are taken. To reduce the load on the health workers in telehealth, we propose a system that eliminates the need for any human supervision by guiding the testers throughout the self-test to ensure the collection of high-quality swab samples. The proposed system takes a live video stream of the frontal face of a user as input and provides real-time instructions to do the self-test correctly with corrective actions when detecting wrong steps. This is mainly done using a collection of deep learning (DL) models. The system uses a novel swab position classification model, Small-MobileNetV2 with Depth-Wise Attention (S-MBNV2-DWAtt), to detect whether a swab is in one of the nostrils or not, which is an optimized version of MobileNetV2 in terms of parameter count and inference speed. The depth-wise attention block allows it to focus on specific parts of the images where the swabs would possibly lie. Lastly, a large-scale synthetic dataset is created to increase the generalization to a variety of swabs and users and a small real dataset is collected to finetune the model on scenes that are similar to the deployment scenarios. The proposed swab position classification model is found to have outstanding performance in terms of both accuracy and speed; it outperforms the ResNet and VGG architectures by 22.83% and 35.11% respectively on a real-world test set while operating at 25 FPS on CPU.

Reinforcement Learning (RL) algorithms suffer from the dependency on accurately engineered reward functions to properly guide the learning agents to do the required tasks. Preference-based reinforcement learning (PbRL) addresses that by utilizing human preferences as feedback from the experts instead of numeric rewards. Due to its promising advantage over traditional RL, PbRL has gained more focus in recent years with many significant advances. In this survey, we present a unified PbRL framework to include the newly emerging approaches that improve the scalability and efficiency of PbRL. In addition, we give a detailed overview of the theoretical guarantees and benchmarking work done in the field, while presenting its recent applications in complex real-world tasks. Lastly, we go over the limitations of the current approaches and the proposed future research directions.

Human Activity Recognition (HAR) has emerged as a potential research topic for smart healthcare owing to the fast growth of wearable and smart devices in recent years. The significant applications of HAR in ambient assisted living environments include monitoring the daily activities of elderly and cognitively impaired individuals to assist them by observing their health status. In this research, we present a deep learning-based fusion approach for multimodal HAR that fuses the different modalities of data to obtain robust outcomes. Here, Convolutional Neural Networks (CNNs) retrieve the high-level attributes from the image data, and the Convolutional Long Short Term Memory (ConvLSTM) is utilized to capture significant patterns from the multi-sensory data. Finally, the extracted features from the modalities are fused through self-attention mechanisms that enhance the relevant activity data and inhibit the superfluous and possibly confusing information by measuring their compatibility. Lastly, extensive tests have been performed to measure the efficiency and robustness of the developed fusion approach using the UP-Fall detection dataset. It is evident from the experimental findings that the proposed fusion technique outperforms the existing state-of-the-art and achieves relatively better performance.

Developing fresh produce yield forecasting service is essential for estimating fair prices to protect against overpriced agricultural commodities and minimize the bid ask spread which not only benefits the retailers and customers but also protects farmers. Forecasting the fresh produce yield is achieved using state of the art deep learning (DL) models. Those models are trained and built using data retrieved from Santa Barbara region in California using an ensemble of Attention Deep Feedforward Neural Network with Gated Recurrent Units (GRU) and Deep Feedforward Neural Network with embedded GRU units. The ensemble takes as input the soil moisture and temperature parameters as well as vegetation indices (VIs) calculated from images retrieved from multiple satellites. The effect of adding the VIs as input parameters on the forecasting performance of the deep learning model is assessed and the most effective VIs are selected. In addition, interpolation techniques are used to estimate the missing VIs due to the low frequency of capturing the images by the satellites. A comparative analysis is conducted to choose the most effective technique, which is found to be Cubic Spline interpolation. One VI, which is the Normalized Difference Vegetation Index (NDVI), proves to be the most effective index in forecasting the yield. Based on the aggregated error measure (AGM) score, the yield forecasting performance of the DL ensemble is enhanced by 12.51% after adding the complete interpolated NDVI to the input parameters used in training the model.

With the rising popularity of Generative Adversarial Networks (GANs) in generating synthetic data, time series are no exception to this trend. In this work, we propose two novel loss functions sDTW-p and sDTW-m based on SoftDynamic Time Warping that can be used to improve the generated time series without modifications to the existing architecture. We also present the first evaluation of the generated samples across different sequence length. Lastly, we show empirically that the result of leveraging our loss function can lead to a 9% improvement according to our metric.

The shift towards Industry 4.0 has seen significant steps forward with the advancements in processing, communication, and storage capabilities of Internet of Things (IoT) devices. Cyber-physical systems (CPS) have become more intelligent and withhold advanced processing, storage, and communication capabilities. Rejuvenated network and service management architectures must incorporate the capabilities of intelligent CPS. With that said, this article introduces a cooperative blockchain (BC)-assisted resource and capability sharing approach to fulfill CPS tasks. The solution uses Federated Learning (FL)-enabled Intelligent IoT (IIoT) devices to support Next-Generation Networks (NGNs). A clustering multi-stage blockchain and FL algorithm is used to create local and global models for CPS tasks. Local models are created for each cluster during the first stage. At the second stage, Federated Averaging is used by fog devices to create fog models. A global deep model is then created on the cloud using Federated Aggregation. Blockchain is used to record and validate the added models and ensure that records are not altered under cyber-attacks. Simulation results have shown that the proposed solution outperforms conventional FL and blockchain approaches in terms of accuracy and delay tolerance.

This work concentrates on optimization on Riemannian manifolds. The Limited-memory Broyden-Fletcher-Goldfarb-Shanno (LBFGS) algorithm is a commonly used quasi-Newton method for numerical optimization in Euclidean spaces. Riemannian LBFGS (RLBFGS) is an extension of this method to Riemannian manifolds. RLBFGS involves computationally expensive vector transports as well as unfolding recursions using adjoint vector transports. In this article, we propose two mappings in the tangent space using the inverse second root and Cholesky decomposition. These mappings make both vector transport and adjoint vector transport identity and therefore isometric. Identity vector transport makes RLBFGS less computationally expensive and its isometry is also very useful in convergence analysis of RLBFGS. Moreover, under the proposed mappings, the Riemannian metric reduces to Euclidean inner product, which is much less computationally expensive. We focus on the Symmetric Positive Definite (SPD) manifolds which are beneficial in various fields such as data science and statistics. This work opens a research opportunity for extension of the proposed mappings to other well-known manifolds.

To overcome the computational complexity of retraining Deep Learning (DL) yield forecasting models for each type of Fresh Produce (FP), it is necessary to have a generalization of the models' application to similar FP. This can be done by transferring the learning among similar FP with minimal retraining. Hence, Transfer Learning (TL) is used in this work amongst berries which are similar in nature. First, the proposed DL model is trained using station-based data and satellite images as inputs mapped to the strawberry yield as output. The weights obtained from this learning are transferred to the raspberry yield forecasting model since raspberry and strawberry yields are similar and are both planted in California. The proposed model is an ensemble of two models: the station-based ensemble model (ATT-CNN-LSTM-SeriesNet_Ens) with its compound DL components, SeriesNet with Gated Recurrent Unit (GRU) and Convolutional Neural Network LSTM with Attention layer (Att-CNN-LSTM); trained and tested using station-based data as input and the corresponding strawberry yields as output. Second, the remote sensing ensemble model (SIM_CNN-LSTM_Ens), which is an ensemble of Convolutional Neural Network LSTM (CNN-LSTM) models; trained and tested using satellite images as input mapped to the same yields as output. The weights obtained are transferred to the raspberry yield forecasting ensemble model with minimal retraining. It is found that the voting ensemble improves performance by 27% compared to the best performing component model. Based on an aggregated measure, the performance obtained from TL is comparable to that obtained by training the models on the raspberry data without TL, while having around 55 % reduction in processing time.

The ability to predict crop yield is vital for food security worldwide and forecasting crop prices can help farmers avoid price crash. In this work, an investigation of using satellite images and deep learning models to predict crop yields as well as forecasting farmers' prices is conducted. For tractability, dimensionality reduction is achieved by converting the images to histograms representing the pixel frequency. The models tested are LSTM, CNN, CNN-LSTM, CNN-LSTM ensemble as well as a Gaussian Process added to each for enhanced performance. It is found that the proposed ensemble of CNN-LSTMs is the best in predicting the yearly soybean yields in addition to forecasting the daily strawberry yields and prices. It outperforms models suggested in the literature with an improvement of 31% in terms of average Root Mean Square Error (RMSE).

It is common for a time series dataset to have missing values, and it is necessary to fill these missing elements before using the dataset for training forecasting models. Usually this problem is tackled using non-machine learning methods that introduce bias into the system which results in unreliable forecasting results. Moreover, most of the work found in the literature tackles imputation of missing values when they are randomly scattered in the dataset while very little work is found tackling the case of consecutive occurrence of missing data; i.e. missing data chunks in the dataset. Therefore, in this work, comprehensive imputation models are developed to impute both random as well as chunks of missing values. Alongside, a framework is found enabling the user to impute any time series data with the optimal models. In order to carry out the task, one non-deep machine learning model (Bidirectional Imputation model) and three deep learning (DL) imputation models (Ensemble model, Transfer Learning model and Hybrid model), are tested using complete time series. The results show that the hybrid model yields a maximum of 38% improvement in the Aggregate Error (AGE) when compared with other models.

Procuring apple yield prior to harvest is essential since it helps in estimating the apple production and prices. A compound Deep Learning (DL) model, SeriesNet with Gated Recurrent Unit (GRU) and Attention (Att-SeriesNet-GRU), is used in this work to predict the apple yield for 15 counties across 6 different Crop Reporting Districts (CRD) in California. The DL model is trained using static soil parameters, which remain constant over years per county and dynamic parameters, which change daily or monthly for a specific county as input, and the corresponding annual apple yield for that county as output. If the training is done based on a single county data then the static parameters won’t add information to the DL model since they remain constant over years per county. Therefore, considering different counties across California is decided to study the effect of considering the static soil parameters along with the dynamic ones. The county level annual apple yield forecast using both static and dynamic parameters together gives promising results. Experimenting with the test set as input shows that adding the static parameters together with the dynamic ones gives an improvement of around 34% in the value of Aggregated Measure (AGM) over the case of using the dynamic parameters alone for yield forecasting. It is also found that training the DL model with augmented training set improves the AGM value by around 12%.

Market price and yield forecasting models for Fresh Produce (FP) are crucial to protect retailers and consumers from overpriced FP. However, utilizing the data for forecasting is obstructed by the occurrence of missing values. Therefore, it is imperative to impute the encountered missing instances to enable effective forecasting. Most of the work found in literature tackles imputation of missing values when they are randomly scattered in the dataset while very little work is found tackling both: consecutive occurrence of missing data, i.e. missing data chunks, as well as those randomly missing. In this work, the data used for forecasting has missing values in chunks as well as at random points. Therefore, various comprehensive imputation models are used to impute both random as well as chunks of missing values. Since the imputed time series are incomplete, the only way to evaluate those imputation models is to analyze their effect on forecasting performance. The ensemble of two compound deep learning (DL) models, namely Attention Convolutional Neural Networks Long Short Term Memory (Att-CNN-LSTM) and SeriesNet with Gated Recurrent Unit (GRU), is used for forecasting. For imputation, three DL models are tested: The Ensemble imputation model which is a Voting Regressor of two DL submodels, Residual GRU and LSTM-Deep-GRU. Another deep learning imputation model is used which is a Transfer Learning (TL) model. Finally, a Hybrid model of both DL models is designed to take the pros of each of its integrated models by using the Ensemble model in case of random missing data and the Transfer Learning model in case of missing data chunks. It is observed that, in general, imputing the missing values improves the forecasting result as compared to eliminating the instances with missing values. The Hybrid model improves the overall forecasting performance by up to 60% compared to the case of using the second-best Transfer Learning model and around 64% as compared to the case of imputation using the Ensemble model.

Forecasting crop yields and prices is crucial for both global food security and providing farmers with valuable information to avoid a price crash. This work proposes a hybrid deep learning model that uses satellite images to forecast strawberry yield along with farmers’ prices, applied in three counties in California. For tractability, a dimensionality reduction technique is applied by converting the images to histograms representing the pixel frequency. The models tested are Convolutional Neural Network (CNN), Variational AutoEncoder (VAE), CNN-Long Short-Term Memory (CNN-LSTM), Stacked AutoEncoder (SAE), and a voting ensemble of CNN-LSTM and SAE. It is found that the proposed voting ensemble of CNN-LSTM and SAE is the best at forecasting the daily strawberry yields and prices in all three counties. Based on an aggregated performance measure (AGM), the voting ensemble model outperforms the models suggested in literature with up to 70% forecasting improvement compared to the CNN model and up to 22% improvement over the CNN-LSTM model.

Searching similarity in time series (TS) datasets has gained widespread attention lately in databases classification and forecast domain. In this study, a TS similarity detection framework is proposed to explore alike-behavior fresh produce (FP) in the yield forecast domain through several factors. The sequential daily yield datasets of three types of FP, including strawberry, raspberry, and blueberry, as well as environmental information related to the Santa Maria region, California, between the years 2011 to 2019, are used to develop and evaluate the models. The framework's output is decided to be the similarity percentage (SP) by considering some thresholds that have been tuned using several synthetic yield datasets. According to the results, the SP is 82% and 52% for strawberry versus raspberry and strawberry versus blueberry, respectively. This indicates the fact that strawberry and raspberry have a relatively similar yield pattern compared to blueberry, which is a considerable matter in generalizing forecast models.

Catastrophic forgetting has long been a major obstacle to continual learning. In this paper, we explore the effect of certain task characteristics, in particular inter-task similarity, on the extent of forgetting. Moreover, we experimentally study the effect of these characteristics on the effectiveness of recent state-of-the-art continual learning approaches. We show that the performance of some methods, for example, the recently proposed Learning without Forgetting (LwF) and elastic weight consolidation (EWC) models, is significantly dependent on these characteristics. We propose a rehearsal-based extension to continual learning models to address this vulnerability. We develop this extension first in the context of LwF and later demonstrate its effectiveness with other models such as EWC. We show that a memory budget of 1% of training data is sufficient to significantly improve on performance in cases of low inter-task similarity.

In computer vision, learning discriminative features to detect anomalies in images is a challenge. The majority of deep learning-based image anomaly detection approaches are compression-reconstruction or generation-based models that were not initially intended for the anomaly detection task. Only a few methods involve dedicated objective function to help detect anomalies and they are not visually explainable as well as reconstruction based approaches. Though the popular reconstruction-based approach for anomaly detection using Convolutional AutoEncoder has achieved the state of the art results, there is no provision to induce the fabrication of discriminatively learnt embeddings from the inputs to well reflect anomalies in the output. We propose an approach using Support Vector Data Description as a regularizer to enforce discriminative ability to easily segregate anomalies from normality with little effort in modelling and tuning the AutoEncoders in our work. We evaluate our approach on several visual anomaly detection datasets to show the capability of our approach. We also perform extensive ablation studies for efficient tuning of parameters.

Variants of Triplet networks are robust entities for learning a discriminative embedding subspace. There exist different triplet mining approaches for selecting the most suitable training triplets. Some of these mining methods rely on the extreme distances between instances, and some others make use of sampling. However, sampling from stochastic distributions of data rather than sampling merely from the existing embedding instances can provide more discriminative information. In this work, we sample triplets from distributions of data rather than from existing instances. We consider a multivariate normal distribution for the embedding of each class. Using Bayesian updating and conjugate priors, we update the distributions of classes dynamically by receiving the new mini-batches of training data. The proposed triplet mining with Bayesian updating can be used with any triplet-based loss function, e.g., triplet-loss or Neighborhood Component Analysis (NCA) loss. Accordingly, Our triplet mining approaches are called Bayesian Updating Triplet (BUT) and Bayesian Updating NCA (BUNCA), depending on which loss function is being used. Experimental results on two public datasets, namely MNIST and histopathology colorectal cancer (CRC), substantiate the effectiveness of the proposed triplet mining method.

Siamese neural network is a very powerful architecture for both feature extraction and metric learning. It usually consists of several networks that share weights. The Siamese concept is topology-agnostic and can use any neural network as its backbone. The two most popular loss functions for training these networks are the triplet and contrastive loss functions. In this paper, we propose two novel loss functions, named Fisher Discriminant Triplet (FDT) and Fisher Discriminant Contrastive (FDC). The former uses anchor-neighbor-distant triplets while the latter utilizes pairs of anchor-neighbor and anchor-distant samples. The FDT and FDC loss functions are designed based on the statistical formulation of the Fisher Discriminant Analysis (FDA), which is a linear subspace learning method. Our experiments on the MNIST and two challenging and publicly available histopathology datasets show the effectiveness of the proposed loss functions.

Building highly precise prediction models for Fresh Produce (FP) market price is crucial to protect retailers from overpriced FP. In this paper we are comparing the price prediction models performance of deep learning (DL) models with statistical as well as standard machine learning (ML) models. Five types of FP are considered in performance testing. It is found that the conventional ML models outperform the statistical models such as ARIMA. On the other hand, the winning model among the conventional ML models (the Gradient Boosting model) proves to be less performant as compared with the simple or compound DL models. Moreover, the simple DL models, such as the Long Short-Term Memory (LSTM), are outperformed by the compound one, the Convolutional Long Short-Term Memory Recurrent Neural Network (CNN-LSTM), whose performance improves by adding attention. The model is capable of precisely predicting FP prices for up to three weeks ahead.

The currently deployed prediction models for strawberry fresh produce (FP) are based on either conventional machine learning (ML) or on simple deep learning (DL) models that are mostly applied for yield prediction. In this paper, we propose more comprehensive DL models that are applied for the first time to predict strawberry yield. The strawberry price is predicted as well directly from weather input parameters and yield. The strawberry price prediction is achieved using compound DL models such as Convolutional Long Short-Term Memory Recurrent Neural Network (CNN-LSTM). It is found that by adding attention, the performance of the compound models usually improves. After utilizing an aggregated performance measure to find the best model, the Attention-CNN-LSTM model proved to be the best compared to the rest of the deployed conventional ML models as well as the compound and simple DL models. The aggregated measure shows that this model is capable of precisely predicting strawberry prices five weeks ahead while maintaining the lowest prediction error and the highest model correlation.

Agent-based modeling has been proposed to simulate real world situations where autonomous agents take their own decisions based on simple rules and data from the environment. The strawberry market in California is a challenging example as prices can vary suddenly due to change in supplies and the fruits cannot be stored for long durations. The microeconomic theory that is expected to model this market is implemented within the simulation model to predict the strawberry price based on the difference between total supply and total demand. In this study, the observed yield of strawberry for the two main suppliers in California is considered as total supply; and for predicting the demand, different demand functions are presented. To estimate the ABM model parameters, two optimization methods are applied with Python-Netlogo. Finally, the computational results are presented to show the performance of the prediction model with directions for future research for improving the results.

Adequately priced orders and time for fresh produce (FP) are two factors that bring commercial benefits to vendors and minimizes waste. However, many factors, such as income, labor, and other trade issues, affect the price that include uncertainties due to climate change, making decisions on FP procurement prices and quantities extremely challenging. Two artificial intelligence-based forecasting tools, i.e., a single variate and a multivariate model, are trained, tested, and compared in this study to predict future daily offer prices up to 7 days ahead for strawberries using mutual transactions for the distribution centers of Loblaws Companies Limited (LCL) in Canada. Results reveal that the developed multivariate model, utilizing both prices of the LCL dataset and California's strawberries yield dataset as predictors, outperforms the best single variate model.

In this study, a comparison of time-series modeling with linear and nonlinear ML tools is conducted for fresh produce (FP) yield forecast. The consecutive monthly weather and yield dataset of Oxnard, California, corresponding to the years 2007 to 2014, are applied for models' development and training by examining the diverse combinations of predictors. The forecast performance is then assessed on the next two years ahead. The sensitivity analysis is performed as the preprocessing approach to ascertain the effective lag-time of the predictors. Results reveal the efficiency of time-series analysis and modeling in FP yield forecast as the implementation of autoregressive predictors along with the exogenous variables, significantly improves the forecast accuracy.

There has been a tremendous increase in the costs of caring for older adults owing to the fact that societies are aging around around the world. This has led to a decrease in the number of caregivers who are able to assist. Investigative studies indicate that older adults require social as well as physical support for their well-being which prompted researchers to use social and cognitive robots and advanced human machine interaction devices. However, most of these studies have shortcomings when it comes to providing means of a natural interaction with the machine. With speech being the most natural way for human communication and the huge developments in the Internet of Things and smart homes, equipping a robotic system with powerful natural speech interaction capabilities to maintain a conversation with an elderly while being linked to other smart home devices shows a promising direction. This paper describes a scalable and expandable system with main goal of designing a natural speech-enabled system for older adults that is capable of linking to multiple active agents with minimal integration efforts. The system makes use of the power of commercially available digital assistant systems, integrated with an intelligent conversational agent, robotics, and smart wearables. The main advantage of the system is that it could provide a portion of the population, namely older adults and the disabled, the flexibility of interacting naturally with powerful social robots in smart home environments, hence providing them with much needed independence.

The importance of imputation for having highly performing prediction models is highlighted in this work. Three imputation techniques are tested against a non-imputation approach that discards records with any missing values; the complete-case analysis (CCA). The deep learning linear memory vector recurrent neural network-RNN (LIME) imputation model is tested along with two other nondeep learning models such as the linear function and Last Observation Carried Forward (LOCF). The simple LSTM deep learning (DL) prediction model is deployed to decide the best performing imputation model, the one resulting in the lowest price and yield prediction errors. Five performance evaluation measures are utilized; the mean absolute error (MAE), the root mean square error (RMSE), R2correlation measure along with two aggregated measures summarizing these three measures to decide the overall prediction performance; the average aggregated measure (AGM) for each considered step ahead and the average of the AGM across all considered steps ahead (AAGM). Based on AGM, it is found that the LIME imputation model leads to the best prediction performance of the simple LSTM DL model across both applications of 5 weeks ahead strawberry price and yield predictions using weather; W2P and W2Y. Therefore, the LIME imputed file is reused to train two compound DL models, Convolutional Long Short-Term Memory RNN with attention (ATT-ConvLSTM) and ATT-CNN-LSTM along with their Voting Regressor ensemble (VR). The same models are retrained with files preprocessed with the non-imputation approach, CCA. It is found that the overall AAGM of the compound DL and ensemble prediction models across all the 1, 2, 3, and 4 weeks ahead price predictions confirm that using LIME highly improves the prediction performance of the ensemble and its compound DL components. The VR ensemble price prediction performance is improved by 72% and the ATTConvLSTM component is improved by 89% compared to their performances without imputation; using CCA preprocessed files.

Time series prediction has been challenging topic in several application domains. In this paper, an ensemble of two top performing deep learning architectures across different applications such as fresh produce (FP) yield prediction, FP price prediction and crude oil price prediction is proposed. First, the input data is trained on an array of different machine learning architectures, the top two performers are then combined using a stacking ensemble. The top two performers across the three tested applications are found to be Attention CNN-LSTM (AC-LSTM) and Attention ConvLSTM (ACV-LSTM). Different ensemble techniques, mean prediction, Linear Regression (LR) and Support vector Regression (SVR), are then utilized to come up with the best prediction. An aggregated measure that combines the results of mean absolute error (MAE), mean squared error (MSE) and R2coefficient of determination (R2) is used to evaluate model performance. The experiment results show that across the various examined applications, the proposed model which is a stacking ensemble of the AC-LSTM and ACV-LSTM using a linear SVR is the best performing based on the aggregated measure.

It is common for a time series dataset to have missing values, and it is necessary to fill these missing elements before fitting any model for forecasting or prediction. Time series imputation remains a challenging task due to the existence of non-linear dependencies between current and past values. Conventional methods, such as deletion of rows containing missing values or filling them with the last observed value, add bias to the data and are therefore inefficient. There are situations where data is missing at consecutive points or random points in the dataset, and one particular method may not work well for all cases. In this paper, nine commonly used models in the field of imputation, based on tools of statistics, machine learning, and deep learning, are compared. Results show that Linear Memory Vector Gated Recurrent Unit (LIME-GRU) outperforms the other tested models by having the least Mean Square Error (MSE) and Root Mean Squared Error (RMSE). A predictive model to gauge the impact of imputation on prediction is also used to validate the findings. The results of the prediction model illustrate that with LIME-GRU, there was a 39% improvement in Average Aggregated Measure (AAGM) when compared with mode imputation on a particular test case.

Missing data are commonly found in time series datasets. These missing elements are usually a hurdle in utilizing the datasets in prediction or forecasting, making imputation of those missing values imperative. Due to the non-linear dependencies between the current and previous values, imputation remains a challenging task. Conventional methods such as averaging, deletion or filling with the last observed value add bias to the data and are therefore inefficient. Since different time series showcase varying characteristics, figuring out which imputation method works best for the respective time series is essential. In this work, seven different deep learning (DL) imputation methods are examined along with three machine learning (ML) ensembles. To enable a recommendation of the best imputation method for each time series type, the imputation models are tested using the four main types of time series: trend (T), seasonal (S), combined trend and seasonal time series (T&S) and random (R) time series. Results indicate that the Gated Recurrent Unit (GRU) neural networks are, in general, the best for missing values imputation with varying complexity based on the time series type. For example, it is found that the residual GRU is recommended for the trend and seasonal time series while the GRU is recommended for the combined type. Conversely, all tested DL imputation models can be used with the random time series type. In addition, the considered ML ensembles do not perform as high as the DL models with all tested types of times series.

It is recognized that system trustworthiness is a hyperstructure embodied by the structural, behavioral, and system dimensions with a set of coherent attributes. We explore a theoretical framework of tripartite trustworthiness that can be applied to real-world autonomous systems. We present a formal study of the essences and mathematical models of system trustworthiness and their quantitative measurements in the contexts of autonomous and mission-critical intelligent systems where humans and machines interact in a hybrid environment.

Driver inattention is among major contributing factors to traffic accidents. There have been and continue to be efforts by governing bodies, car manufacturers, and researchers to prevent driver inattention or, failing that, to mitigate its effects. Many vehicles nowadays come equipped with driver monitoring systems that can alert the driver to, or compensate for, inattention. Moreover, the research community continues to explore and investigate more robust approaches to deal with inattention. Meanwhile, vehicle automation, to various degrees, is becoming more prevalent, with the human's role in the driving task changing depending on the level of autonomy. This necessitates that inattention detection, moving forward, be studied and designed in view of automation and in the context of a specific level of vehicle autonomy. Driver inattention and vehicle automation interact in a complex way, and that needs to be taken into account in the design of future vehicles. We explore this interaction in this paper in light of research findings, and survey inattention detection systems and attempt to contextualize them within popular frameworks for next-generation autonomous vehicles.

Neural networks trained on tasks sequentially tend to degrade in performance, on the average, the more tasks they see, as the representations learned for one task get progressively modified while learning subsequent tasks. This phenomenon- known as catastrophic forgetting-is a major obstacle on the road toward designing agents that can continually learn new concepts and tasks the way, say, humans do. A common approach to containing catastrophic forgetting is to use regularization to slow down learning on weights deemed important to previously learned tasks. We argue in this paper that, on their own, such post hoc measures to safeguard what has been learned can, even in their more sophisticated variants, paralyze the network and degrade its capacity to learn and counter forgetting as the number of tasks learned increases. We propose instead- or possibly in conjunction-that, in anticipation of future tasks, regularization be applied to drive the optimization of network weights toward reusable solutions. We show that one way to achieve this is through an auxiliary unsupervised reconstruction loss that encourages the learned representations not only to be useful for solving, say, the current classification task, but also to reflect the content of the data being processed-content that is generally richer than it is discriminative for any one task. We compare our approach to the recent elastic weight consolidation regularization approach, and show that, although we do not explicitly try to preserve important weights or pass on any information about the data distribution of learned tasks, our model is comparable in performance, and in some cases better.

This paper focuses on the finite-time synchronization for a class of fuzzy inertial cellular neural networks (FICNNs) with time-varying delays. First, by constructing a proper variable substitution, the original FICNNs with time-varying delays can be rewritten as first-order differential system. Second, based on Lyapunov functionals, we derive some new and sufficient conditions of finite-time synchronization for the addressed system. We illustrate the effectiveness of the approach through a few examples.

In this paper, a class of neutral type fuzzy neural networks with non-operator based neutral functional differential equations is analysed. Using the Banach's fixed point principle, some sufficient conditions for the existence, the uniqueness and the global exponential stability of the almost automorphic solutions are obtained. Two examples are given to illustrate the theoretical results.

Distracted driving is the main cause for car accidents. Driver inattention monitoring systems are promising solutions to mitigate this problem. In this paper, we propose a novel driver inattention monitoring system utilizing deep learning and fuzzy logic theory. A driver head pose estimation module is able to determine whether the driver is focusing on his frontal view, while a deep-learning-based distraction recognition module would detect whether the driver is performing a distraction activity. A danger level inference module based on fuzzy logic combines information from the head pose estimation and the distraction recognition modules to infer the danger level in a real-time manner. In the experimental work, a Convolutional Neural Network model is trained on data of high diversity allowing a more robust driver distraction detection compared to the model trained with only data collected by simulation experiments. In addition, we show that the proposed danger level inference strategy is an effective solution to detect dangerous driving situations by providing timely alerts depending on the vehicle speed.

We introduce a new autoencoder-based algorithm for non-negative matrix factorization. Instead of stochastic gradient descent-based update rules, our approach employs exponentiated gradient decent which, unlike the former, inherently guarantees the non-negativity of basis vectors when they are initialized to non-negative values. Moreover, we explore the potential of our autoencoder-based non-negative matrix factorization model for clustering applications, and show that it can learn hierarchical factorizations, each of which corresponding to a different and meaningful clustering. Further, we show that adding a supervised loss at intermediate layers results in more diverse representations at the different layers of the NMF hierarchy. We provide extensive empirical evaluations on text and image datasets and compare our proposed model to two alternative approaches, including another autoencoder-based algorithm.

Multi-step time series prediction is known to suffer from increasing performance degradation the farther in the future the predictions are made. In this paper, we introduce two approaches to address this weakness in recursive and multioutput prediction models. In particular, we present a model that allows recursive prediction approaches to take into account the time-step index when making predictions. In addition, we propose a conditional generative adversarial network-based data augmentation model to improve prediction performance in multioutput models. We show on real-world time series datasets that the two methods improve on multi-step time series prediction in recursive and multi-output models, respectively.

This work addresses non-stationary traffic flow prediction by implementing an intelligent update scheme to deep neural networks. The intelligent update scheme works by monitoring the frequency domain features extracted from the traffic flow time series. The features at present are compared with the previous ones through a distance function. The resulting similarity is then fed to the exponentially weighted moving average to detect whether a notable change in the traffic flow is present or not. It has been shown in the experiments that the proposed method is able to handle the non-stationarity and produce acceptable traffic flow prediction. Moreover, the proposed method performance is comparable to the fully stochastic gradient descent update scheme while saving computational and time resources up to around 13%.

We propose a visual-based end to end lane following system which fuses temporal and spatial visual information to predict current and future control variables. Previous works only predict control variables for the next time point with the current visual information. In contrast, based on a long-term recurrent convolutional neural network, we investigate the effect of fusing history information of different lengths to predict the imminent control variable in different future horizons. Experimental results show that with long history visual information, the neural network can approximate human driving behaviours with high precision. Consistent with intuition is that the influence of history information declines as time moves forward. Meanwhile, history information of the past 0.6 seconds is of most information for the prediction, and the Mean Square Error (MSE) for the steering command prediction with 0.6s history information is 8.378 × 10-3m-1. By training the model with control signals that lag behind visual information as targets, the testing result shows that it is possible to predict future control variables with great accuracy, while the best prediction accuracy happens to the steering command 0.4 seconds later.

Although the future impact of plug-in electric vehicles (PEVs) on distribution grids is disputed, all parties agree that mass operation of PEVs will greatly affect load profiles and grid assets. The large-scale penetration of domestic energy storage, such as with photovoltaics (PVs), into the edges of low-voltage grids is increasing the amount of customer-generated electricity. Distribution grids, which are inherently unbalanced, tend to become even more so with the uneven spread of PVs and PEVs. In combination, PEVs and local generation could provide voltage support for distribution networks, and support increased penetration. This paper develops an interactive energy management system for incorporating PEVs in demand response (DR). Using this system, owners can immediately choose whether they want to discharge their PEV battery back into the grid. The system not only provides owners with a flexible scheme for contributing to DR but also ensures that, through real-time collaboration of PEVs and PVs, the three-phase grid operates within acceptable voltage unbalance. An extensive performance evaluation using MATLAB/GAMS simulation of the 123-bus test system verifies the effectiveness of the proposed approach.

This issue marks the first anniversary issue since I was honored to serve as the Editor-in-Chief (EiC) of the IEEE Transactions on Neural Networks and Learning Systems (TNNLS). I am happy to report that we had a very successful year and here are a few highlights that I would like to share with the community.•The latest impact factor of TNNLS is 4.854 according to the Journal Citation Reports. This marks a record high impact factor for our journal and places TNNLS as the number one scholarly publication in Computer Science (Hardware & Architecture), number three in Computer Science (Theory & Methods), and number ten in Electrical and Electronic Engineering journals.

Situation prediction is a crucial part of active Advanced Driver Assistance Systems (ADAS) to prevent rear, lateral and other collisions. Majority of road crashes can be prevented if the ADAS issue a warning about a potential threat at least one-half second prior to the prominent accident. To take the suitable maneuver, active safety systems should assess succinctly the danger caused by other errant drivers and analyse surrounding drivers intent. This study presents a game theory impact assessment and decision making model that allows drivers to assess threat level caused during different road scenarios. Our model is based on Fuzzy Multi-Entity Bayesian Network (Fuzzy-MEBN) enriched by a game theory component. Illustrative scenarios are provided to show the merit of our model.

For VANET safety services a context aware system for the Automatic Crash Notification (ACN) is developed while the context aware Congested Road Notification system (CRN) is developed for the convenience services. A simple fuzzy logic model is proposed and compared to different severity estimation models deployed for both systems. The performance of the ACN models is compared using a test collection that is based on nineteen years of real life crash records associated with their severity levels while the performance of the CRN models is tested using nearly 500,000 different urban and rural freeways flow situations associated with their congestion severity levels. The non-binary Spearman correlation coefficient and the Average Distance Measure (ADM) are used to evaluate the performance of the tested models. Results show that the simple fuzzy severity estimation model has a comparable performance to more complicated systems such as the CoTEC (CoOperative Traffic congestion detECtion) fuzzy system and the URGENCY algorithm, and outperforms the binary severity estimation models for the ACN and CRN systems.

Clustering has raised as an important problem in many different domains like biology, computer vision, text analysis and robotics. Thus, many different clustering techniques were developed to address this essential problem and propose astonishing solutions to conquer it. However, traditional clustering techniques suffer either from their limitations to detect specific shapes like K-means and PAM or from their limitations to detect clusters with specific densities as in DBSCAN and SNN. Moreover, exploiting the data relations and similarities has been proven to provide better insights to enhance the clustering quality as shown in spectral clustering and affinity propagation. Our observations have shown that using variance of similarities between each data point and its neighbors can well distinguish between within-cluster points, points connecting two clusters and outlier points. Therefore, we have utilized this variance measure to calculate each data point density and developed a Local Variance-based Clustering (LVC) technique that employs this measure to cluster the data. Experimental results show that LVC outperforms spectral clustering and affinity propagation in clustering quality using control charts, ecoli and images datasets, while maintaining a good running time. In addition, results show that LVC can detect topics from Twitter with higher topic recall by 15% and higher term precision by 3% over DBSCAN.

This work addresses short-term traffic flow prediction by proposing a big-data-based framework. The proposed framework uses data fusion to deal with heterogeneous data generated from various sources. The data are categorized into two types: streams of data and event-based data. In this work, Deep Belief Networks (DBNs) are used to independently predict traffic flow using streams of data, i.e., historical traffic flow and weather data, and event-based data, i.e., tweets. Furthermore, Dempster's conditional rule for updating belief is used to fuse evidence coming from streams of data and event-based data modules to achieve enhanced prediction. The experimental results using real-world data show the merit of the proposed framework compared to the state-of-the-art ones.

Driver inattentiveness constitutes the main cause of road accidents, which makes it a major factor in road safety. In this paper, we propose a comprehensive framework to address the road safety problem by tackling it from a high-level information fusion standpoint, considering vehicular ad hoc networks (VANETs) as the deployment platform. The proposed framework relies on the multientity Bayesian networks (MEBNs), which exploit the expressiveness of first-order logic for semantic relations, and the strength of the Bayesian networks in handling uncertainty. First, the entities that influence the inattention phenomenon, as well as both their causal and semantic relationships, are identified. Next, an MEBN-based high-level information fusion framework is proposed through which entities, situations, and their relationships in specific contexts are modeled using MEBN fragments. Furthermore, MEBN inference is used to assess the situations of interest by estimating their states. To demonstrate the capabilities of the proposed framework, a collision warning system simulator has been developed, which evaluates the likelihood of a vehicle being in a near-collision situation using a wide variety of local and global information sources available in various VANET environments. If the threat of being in a near-collision situation is determined to be high, then the driver is warned accordingly. Our experimental results for two distinct single-vehicle and multivehicle categories of driving scenarios, as well as a novel hybrid MEBN inference, demonstrate the capability of the proposed framework to efficiently achieve situation and threat assessment on the road.

In this paper, a soft+hard data fusion model is proposed that is capable of combining the data generated from human-based sources with those generated by physical sensors. The basis of this model is our previously introduced Fuzzy extension to the Mutli-Entity Bayesian Network (MEBN) language, which is a High-Level Information Fusion (HLIF) framework capable of expressing the semantic and causal relationships between the entities constituting a world model, as well as managing their ambiguity and uncertainty. In our proposed model, the unstructured soft data is presented by undergoing a novel soft-data-association process, through which the data is semantically analyzed, and accurately structured in a fuzzy random variable. Moreover, the clique tree inference algorithm for Bayesian Networks is modified to handle fuzzy evidence in Fuzzy-MEBN. The simulation results, in transportation domain, show that our improved HLIF model is capable of handling both soft and hard data, and consequently, provide the user with more precise situation assessment.

A distributed nonlinear estimation method based on soft-data-constrained multimodel particle filtering and applicable to a number of distributed state estimation problems is proposed. This method needs only local data exchange among neighboring sensor nodes and thus provides enhanced reliability, scalability, and ease of deployment. To make the multimodel particle filtering work in a distributed manner, a Gaussian approximation of the particle cloud obtained at each sensor node and a consensus propagation-based distributed data aggregation scheme are used to dynamically reweight the particles' weights. The proposed method can recover from failure situations and is robust to noise, since it keeps the same population of particles and uses the aggregated global Gaussian to infer constraints. The constraints are enforced by adjusting particles' weights and assigning a higher mass to those closer to the global estimate represented by the nodes in the entire sensor network after each communication step. Each sensor node experiences gradual change; i.e., if a noise occurs in the system, the node, its neighbors, and consequently the overall network are less affected than with other approaches, and thus recover faster. The efficiency of the proposed method is verified through extensive simulations for a target tracking system which can process both soft and hard data in sensor networks.

This paper presents a novel comprehensive Fuzzy extension to Multi-Entity Bayesian Networks (MEBN) that is deemed a well-studied and theoretically rich language that expressively handles semantics analysis, and effectively model uncertainty management. However, MEBN lack the capability of modeling the inherent conceptual and structural ambiguity that is delivered with the knowledge gained through human language. In this paper, Fuzzy MEBN that is a new version of MEBN which is based on First-order Fuzzy Logic, and Fuzzy Bayesian Networks is introduced. Furthermore, its applicability is evaluated by implementing an application related to Vehicular Ad-hoc Networks area. The results demonstrate that Fuzzy MEBN is capable of dealing with ambiguous semantical and uncertain causal relationships between the knowledge entities very efficiently.

Tracking multiple targets with non-linear dynamics is a challenging problem. One of the popular solutions, Sequential Monte Carlo-Probability Hypothesis Density (SMC-PHD) filter, deploys a Random Set (RS) theoretic formulation along with the Sequential Monte Carlo approximation, which is a variant of Bayes filtering. The performance of Bayesian filtering-based methods can be enhanced by using extra information incorporated as specific constraints into the filtering process. Following the same principle, this paper proposes a constrained variant of the SMC-PHD filter, in which the inherently vague human-generated data are transformed into a set of constraints using a fuzzy logic approach. These constraints are enforced to the filtering process by applying coefficients to the particles' weights. The Soft Data (SD) reports on target agility level; wherein, the agility refers to the case in which the observed dynamics of the targets deviates from its given probabilistic characterization. Consequently, the proposed constrained filtering approach enables dealing with multitarget tracking scenarios in presence of target agility, as demonstrated by the experimental results presented in this paper.

This work proposes an approach for solving the linear regression problem by maximizing the dependence between prediction values and the response variable. The proposed algorithm uses the Hilbert-Schmidt independence criterion as a generic measure of dependence and can be used to maximize both nonlinear and linear dependencies. The algorithm is important in applications such as continuous analysis of affective speech, where linear dependence, or correlation, is commonly set as the measure of goodness of fit. The applicability of the proposed algorithm is verified using two synthetic, one affective speech, and one affective bodily posture datasets. Experimental results show that the proposed algorithm outperforms support vector regression (SVR) in 84% (264/314) of studied cases, and is noticeably faster than SVR, as an order of 25, on average.

The kernel k-means is an effective method for data clustering which extends the commonly-used k-means algorithm to work on a similarity matrix over complex data structures. It is, however, computationally very complex as it requires the complete kernel matrix to be calculated and stored. Further, its kernelized nature hinders the parallelization of its computations on modern scalable infrastructures for distributed computing. In this paper, we are defining a family of kernelbased low-dimensional embeddings that allows for scaling kernel k-means on MapReduce via an efficient and unified parallelization strategy. Afterwards, we propose two practical methods for low-dimensional embedding that adhere to our definition of the embeddings family. Exploiting the proposed parallelization strategy, we present two scalable MapReduce algorithms for kernel k-means. We demonstrate the effectiveness and efficiency of the proposed algorithms through an empirical evaluation on benchmark datasets.

A source-reliability-adaptive distributed non-linear estimation method based on distributed Soft-Data-Constrained Multi-Model Particle Filtering (SDCMMPF) and applicable to a number of distributed state estimation problems is proposed. The proposed method requires only local data exchange among neighbouring sensor nodes, it therefore provides enhanced reliability, scalability, and ease of deployment. In particular, by taking into account the estimate reliability of each sensor node at any point in time, it yields a more robust distributed estimation. To perform the Multi-Model Particle Filtering (MMPF) in an adaptive distributed manner, a Gaussian approximation of the particle cloud obtained at each sensor node along with a weighted Consensus Propagation (CP) based distributed data aggregation scheme are deployed to dynamically re-weight the particles' weights. The filtering approach in this paper is a soft-data constrained variant of the multi-model particle filter presented in our earlier work, and is capable of processing both soft human-generated data and conventional hard sensory data. In case of permanent noise in the estimation provided by a sensor node, due to either a faulty sensing device or misleading soft data, the contribution of that node in the weighted consensus process is immediately reduced in order to alleviate its effect on the estimation provided by the neighbouring nodes and the entire network. The robustness of the proposed method is demonstrated through simulation results for an agile target tracking task.

Recently, a supervised dictionary learning (SDL) approach based on the Hilbert-Schmidt independence criterion (HSIC) has been proposed that learns the dictionary and the corresponding sparse coefficients in a space where the dependency between the data and the corresponding labels is maximized. In this paper, two multiview dictionary learning techniques are proposed based on this HSIC-based SDL. While one of these two techniques learns one dictionary and the corresponding coefficients in the space of fused features in all views, the other learns one dictionary in each view and subsequently fuses the sparse coefficients in the spaces of learned dictionaries. The effectiveness of the proposed multiview learning techniques in using the complementary information of single views is demonstrated in the application of speech emotion recognition (SER). The fully-continuous sub-challenge (FCSC) of the AVEC 2012 dataset is used in two different views: baseline and spectral energy distribution (SED) feature sets. Four dimensional affects, i.e., arousal, expectation, power, and valence are predicted using the proposed multiview methods as the continuous response variables. The results are compared with the single views, AVEC 2012 baseline system, and also other supervised and unsupervised multiview learning approaches in the literature. Using correlation coefficient as the performance measure in predicting the continuous dimensional affects, it is shown that the proposed approach achieves the highest performance among the rivals. The relative performance of the two proposed multiview techniques and their relationship are also discussed. Particularly, it is shown that by providing an additional constraint on the dictionary of one of these approaches, it becomes the same as the other.

Input-output monotonicity is an important constraint found in many application domains. A monotonic fuzzy system (MFS) is defined as a Takagi-Sugeno-Kang (TSK) system whose output is monotonically increasing or decreasing with respect to one or more inputs. This paper reviews the authors' previous work, which derived the parametric conditions for the MFS, and discusses the rationale lying behind the conditions. An MFS is developed by creating a monotonic rule base while preserving the relative monotonicity among the membership functions corresponding to the fuzzy rules. This paper also proves that the parametric conditions are necessary and sufficient to build a single-input zeroth-order MFS with two rules. Only the sufficiency of the conditions holds for a multi-input first-order or higher order TSK fuzzy system with three or more rules.

Various regression models are used to predict the continuous emotional contents of social signals. The common trend to train those models is by minimizing a sense of prediction error or maximizing the likelihood of the training data. According to those optimization criteria, among two models, the one which results in a lower prediction error, or higher likelihood, should be favored. However, that might not be the case, since to compare the prediction quality of different models, the correlation coefficient of their prediction with the actual values is prevalently used. Hence, given the fact that a lower prediction error does not imply a higher correlation coefficient, we might need to reconsider the optimization criteria that we undertake in order to learn the regression coefficients, in order to synchronize it with the hypothesis testing criteria. Motivated by this reasoning, in this work we suggest to maximize a sense of correlation for learning regression coefficients. Two senses of correlation, namely Pearson's correlation coefficient and Hilbert-Schmidt independence criterion, are seen for this purpose. We have chosen the continuous audio/visual emotion challenge 2012 as the framework of our experiments. The numerical results of this study show that compared to support vector regression, the suggested learning algorithms offer higher correlation coefficient and lower prediction error.

Many systems have been implemented towards achieving effective human-machine interaction, but run the risk of being ignored if appropriate performance metrics are not in place. As a result, our goal becomes that of providing a foundation upon which we can assess how well the human and the robot perform as a team. Toward the efficient modelling of such metrics, we attempt to determine the true amount of time that an operator has to dedicate to the robot. Therefore, we define the robot attention demand (RAD) as a function of both direct interaction time (DIT) and indirect interaction time (IIT), where the IIT is a direct consequence of the human trust in automation. We propose a two-level fuzzy temporal model to evaluate the human trust in automation while collaborating with robots to complete some tasks. The model combines the advantages of fuzzy logic and finite state machines to best model this phenomenon, and reduces the system complexity and the size of the knowledge base by grouping perceptions into first- and second-order perceptions. The fuzzy knowledge base is further updated by implementing an application robotic platform where robots and users interact via natural language to complete tasks with varying levels of complexity. User feedback is noted and used to tune the knowledge base where needed.

This work intends to identify common performance metrics for task-oriented human-robot interaction. We present a methodology to assess the system performance of a human-robot team in achievement of collective tasks. We propose a systematic approach that addresses the performance of both the human user and the robotic agent as a team. Toward this end, we attempt to determine the true time that an operator has to dedicate to a robot in action. We define the robot attention demand (RAD) as a function of both direct interaction time (DIT) and indirect interaction time (IIT), where the IIT is a direct consequence of the human trust in automation. We propose a two-level fuzzy temporal model to evaluate the human trust in automation while interacting with robots. Another fuzzy temporal model is presented to evaluate the human reliability during interaction time. The model is then generalized to accommodate multi-robot scenarios. Sequential and parallel robot cooperation schemes with varying levels of task dependency are considered. The fuzzy knowledge bases are further updated by implementing an application robotic platform where robots and users interact naturally to complete tasks with varying levels of complexity. User feedback is noted and used to tune the knowledge base rules where needed, to better represent a human expert's knowledge.

Given the fact that the length of the feature vector that is being used for the paralinguistic recognition of speech has exceeded some thousands, the importance of a sparse representation of a model becomes notable. The importance of a sparse representation is mainly due to the more interpretability, higher generalization capability, and numerically more efficiency of such a model. In this work, as an endeavor to search for a sparse representation of speech features used for paralinguistic speech modeling, we make use of the elastic net. As for the benchmark, we use the frameworks of the second audio/visual emotion challenge and the Interspeech 2012 speaker trait challenge. Also proposed in this work is the use of part-of-speech tags as syntactic features of speech for emotional speech recognition. Results of this work show that despite the relatively small number of features that is used for the modeling tasks, generalization capability of the suggested models is comparable to those of other models that use thousands of features and more elaborate learning algorithms.

Input-output monotonicity is an important property found in many physical systems. This paper reviews parametric conditions of monotonic Takagi-Sugeno-Kang (TSK) fuzzy systems, and discusses its sufficiency and necessity. In author's previous work, the sufficiency of the conditions was shown, but the necessity has remained an open question. This study analytically proves that the necessity holds when a single-input TSK fuzzy system employs two rules. For the cases of three rules, counter examples are provided to verify that the necessity of the conditions violates.

Recognizer Output Voting Error Reduction (ROVER), is a well-known procedure for decoders' combination aiming at reducing the Word Error Rate (WER) in transcription applications. However, it appears that this technique has reached a plateau in terms of performance. This paper presents a novel approach, cROVER, in order to boost the current ROVER performance, by relying on a contextual analysis to trim erroneous words. Experiments have proven that it is possible to outperform ROVER, despite the high false positive rate of the error detection technique.

A common characteristics of the computational models of visual attention is they execute the two modes of visual attention (visual exploration and visual search) separately. This makes a visual attention model unsuitable for real-world robotic applications. This paper focuses on integrating visual exploration and visual search in a common framework of visual attention and the challenges resulting from such integration. It proposes a visual attention-oriented speech-based human robot interaction framework which helps a robot to switch back and-forth between the two modes of visual attention. A set of experiments are presented to demonstrate the performance of the proposed framework.

A challenging problem in smoothing spline regression is determining a value for the smoothing parameter. The parameter establishes the tradeoff between the closeness of the data, versus the smoothness of the regression function. This paper proposes a new method of finding the optimum smoothness value based on Stein's Unbiased Risk Estimator (SURE). This approach employs Newton's method to solve for the optimal value directly, while minimizing the true error of the regression. Experimental results demonstrate the effectiveness of this method, particularly for small datasets.

IAM ABOUT to start my second year of service as the Editor-in-Chief (EiC) of the IEEE TRANSACTIONS ON NEURAL NETWORKS (TNN). Needless to say, my first year as the EiC has been full of excitement and challenges. Transitioning this position from my predecessor to me went very smoothly during the months of September 2009 to January 2010. During the past year, we have accumulated 50+ Associate Editors (AEs) handling roughly 600 new submissions (not counting resubmissions and revised submissions). With the help of these AEs and my predecessor, I was quickly able to learn to do my job, and as such, the transition had very few glitches. The easy part of my job is checking whether a submission is in compliance with our guidelines and where it is within the scope of the TRANSACTIONS, before it is assigned to an AE for handling. The difficult part of my job has been dealing with some papers with three or more reviewers, all of whom agreed to review them but for some reason failed to respond to repeated automatic-review reminders. AEs handling these papers have to take several extra steps to remind reviewers through phone calls or e-mails, look for replacement reviewers, or review the papers themselves. Most authors have been appreciative of the work of the AEs and reviewers, and they accept our decisions without a problem. The backlog of papers has been kept short over the last year. We have maintained an organized printing and paperacceptance schedule, with papers typically printed in the journal within 2‐3 months of acceptance. Our page budget has been kept constant in the past few years (roughly 2060 pages per year), and we expect to hold the same page count for next year.

Although calculating the all-terminal reliability (ATR) of a stochastic network is a computationally expensive task, deciding whether the ATR is greater than a preset value could be done with less effort. This study proposes a new method that generates the sequential lower and upper bounds of the ATR, based on greedy network factoring. The proposed method begins by finding the most reliable spanning tree and most unreliable cut set in the given network. Their operative and failing probabilities are used to update the lower and upper bounds of the ATR. Subnetworks are then produced, corresponding to each state of the spanning tree or cut set. This procedure is applied to the subnetworks in a recursive manner to update the ATR bounds further, until either the lower or upper bound reaches the preset ATR requirement. Due to the rapid convergence of the ATR bounds, the feasibility of a given network is likely to be decided at an early stage of the network factoring process. This study proposes several different implementations of the greedy algorithm and introduces the results of the computer experiments comparing them. Based on the experimental results, this study suggests a relationship between the performance of each implementation and the characteristics of the given network, such as layout and edge operating probabilities.

Cognitive robotics have been recently touted as effective tools that could be used in a number of applications. In order for cognitive robots to act adequately and safely in real world, they must be able to achieve effective human-machine interaction or collaboration. Toward this end, performance evaluation metrics are used as important measures to achieve these goals. Toward the efficient modelling of such metrics, we attempt to determine the true time that an operator has to dedicate to the robot. Therefore, we define the robot attention demand (RAD) as a function of both Direct Interaction Time (DIT) and Indirect Interaction Time (IIT), where the IIT is a direct consequence of human trust in automation. We then propose a two-level fuzzy temporal model to evaluate and estimate the human trust in automation while collaborating and interacting with robots and machines to complete some tasks. The model combines the advantages of fuzzy logic and finite state machines to best model this phenomenon, and reduces the system complexity and the size of the knowledge base by grouping perception into first- and second-order perceptions.

Global optimization is the problem of finding the global optimum of any given function in a certain search space. Simulated Annealing (SA) and Genetic Algorithms (GA) are among the well-known techniques used for global optimization. Adjusting the parameters of SA such as the temperature schedule and the neighborhood range plays an important role in the performance of the algorithm. Furthermore, many studies in literature showed that the best values for SA parameters depend on the optimization problem. We introduce a novel hybrid approach that uses SA to solve an optimization problem and uses GA simultaneously to adapt the parameters of SA. This new approach is referred to as Geno-Simulated Annealing (GSA). It does not require any predefined values for the parameters of SA. To evaluate the performance of the proposed approach, we used seven well-known benchmark optimization functions. The obtained results indicate the superiority of the proposed approach as compared to a similar approach and to conventional SA.

Diabetic retinopathy (DR) is a common complication of diabetes that damages the retina and leads to sight loss if treated late. In its earliest stage, DR can be diagnosed by micro aneurysm (MA). Although some algorithms have been developed, the accurate detection of MA in color retinal images is still a challenging problem. In this paper we propose a new method to detect MA based on Sparse Representation Classifier (SRC). We first roughly locate MA candidates by using multi-scale Gaussian correlation filtering, and then classify these candidates with SRC. Particularly, two dictionaries, one for MA and one for non-MA, are learned from example MA and non-MA structures, and are used in the SRC process. Experimental results on the ROC database show that the proposed method can well distinguish MA from non-MA objects.

This paper intends to present a straightforward, extensive, and noise resistant method for efficiently tagging a web query, submitted to a search engine, with proper category labels. These labels are intended to represent the closest categories related to the query which can ultimately be used to enhance the results of any typical search engine by either restricting the results to matching categories or enriching the query itself. The presented method effectively rules out noise words within a query, forms the optimal keyword packs using a density function, and returns a set of category labels which represent the common topics of the given query using Wikipedia category hierarchy.

Most of text categorization techniques are based on word and/or phrase analysis of the text. Statistical analysis of a term frequency captures the importance of the term within a document only. However, two terms can have the same frequency in there documents, but one term contributes more to the meaning of its sentences than the other term. Thus, the underlying model should identify terms that capture the semantics of text. In this case, the model can capture terms that present the concepts of the sentence, which leads to discovering the topic of the document. A new concept‐based model that analyzes terms on the sentence, document, and corpus levels rather than the traditional analysis of document only is introduced. The concept‐based model can effectively discriminate between nonimportant terms with respect to sentence semantics and terms which hold the concepts that represent the sentence meaning. A set of experiments using the proposed concept‐based model on different datasets in text categorization is conducted in comparison with the traditional models. The results demonstrate the substantial enhancement of the categorization quality using the sentence‐based, document‐based and corpus‐based concept analysis.

Most of the common techniques in text mining are based on the statistical analysis of a term, either word or phrase. Statistical analysis of a term frequency captures the importance of the term within a document only. However, two terms can have the same frequency in their documents, but one term contributes more to the meaning of its sentences than the other term. Thus, the underlying text mining model should indicate terms that capture the semantics of text. In this case, the mining model can capture terms that present the concepts of the sentence, which leads to discovery of the topic of the document. A new concept-based mining model that analyzes terms on the sentence, document, and corpus levels is introduced. The concept-based mining model can effectively discriminate between nonimportant terms with respect to sentence semantics and terms which hold the concepts that represent the sentence meaning. The proposed mining model consists of sentence-based concept analysis, document-based concept analysis, corpus-based concept-analysis, and concept-based similarity measure. The term which contributes to the sentence semantics is analyzed on the sentence, document, and corpus levels rather than the traditional analysis of the document only. The proposed model can efficiently find significant matching concepts between documents, according to the semantics of their sentences. The similarity between documents is calculated based on a new concept-based similarity measure. The proposed similarity measure takes full advantage of using the concept analysis measures on the sentence, document, and corpus levels in calculating the similarity between documents. Large sets of experiments using the proposed concept-based mining model on different data sets in text clustering are conducted. The experiments demonstrate extensive comparison between the concept-based analysis and the traditional analysis. Experimental results demonstrate the substantial enhancement of the clustering quality using the sentence-based, document-based, corpus-based, and combined approach concept analysis.

The detection of microaneurysms in digital color fundus photographs is a critical first step in automated screening for diabetic retinopathy (DR), a common complication of diabetes. To accomplish this detection numerous methods have been published in the past but none of these was compared with each other on the same data. In this work we present the results of the first international microaneurysm detection competition, organized in the context of the Retinopathy Online Challenge (ROC), a multiyear online competition for various aspects of DR detection. For this competition, we compare the results of five different methods, produced by five different teams of researchers on the same set of data. The evaluation was performed in a uniform manner using an algorithm presented in this work. The set of data used for the competition consisted of 50 training images with available reference standard and 50 test images where the reference standard was withheld by the organizers (M. Niemeijer, B. van Ginneken, and M. D. Abràmoff). The results obtained on the test data was submitted through a website after which standardized evaluation software was used to determine the performance of each of the methods. A human expert detected microaneurysms in the test set to allow comparison with the performance of the automatic methods. The overall results show that microaneurysm detection is a challenging task for both the automatic methods as well as the human expert. There is room for improvement as the best performing system does not reach the performance of the human expert. The data associated with the ROC microaneurysm detection competition will remain publicly available and the website will continue accepting submissions.

Designing a reliable network becomes a time-consuming task if it involves All-Terminal Reliability (ATR) calculation, which belongs to the class of NP-hard problems. To make this task easier to address, we propose a new algorithm to decide the ATR feasibility of a given networkGwithout performing exhaustive calculation. The proposed algorithm cumulatively updates the lower and upper bounds of the ATR using the set of subnetworks decomposed or branched fromG. Once the lower or upper bound reaches the predetermined ATR requirement, the feasibility ofGis determined. The proposed algorithm is characterized by four existing ATR calculation methods, which decompose or branchGinto multiple subnetworks. The four implementations of the proposed algorithm will be tested via computer experiments. The results show that the proposed algorithm can make feasibility decision dramatically faster. The arrangement of subnetworks that can improve the performance of the proposed algorithm is also discussed.

Visual attention is one of the major requirements for a robot to serve as a cognitive companion for human. The robotic visual attention is mostly concerned with overt attention which accompanies head and eye movements of a robot. In this case, each movement of the camera head triggers a number of events, namely transformation of the camera and the image coordinate systems, change of content of the visual field, and partial appearance of the stimuli. All of these events contribute to the reduction in probability of meaningful identification of the next focus of attention. These events are specific to overt attention with head movement and, therefore, their effects are not addressed in the classical models of covert visual attention. This paper proposes a Bayesian model as a robot-centric solution for the overt visual attention problem. The proposed model, while taking inspiration from the primates visual attention mechanism, guides a robot to direct its camera toward behaviorally relevant and/or visually demanding stimuli. A particle filter implementation of this model addresses the challenges involved in overt attention with head movement. Experimental results demonstrate the performance of the proposed model.

Intrusion Detection Systems (IDSs) deal with large amount of data containing irrelevant and redundant features, which leads to slow training and testing processes, heavy computational resources and low detection accuracy. Therefore, the features selection is an important issue in intrusion detection. Reducing the features set improves the system accuracy and speeds up the training and testing phases considerably. In this paper, we improve the Enhancing Support Vector Decision Function (ESVDF) approach by integrate it with a fuzzy inferencing model. The fuzzy inferencing model is used to accommodate the learning approximation and the small differences in the decision making steps of the ESVDF approach. It simplifies the design complexity and reduces the execution time of the ESVDF, which speeds up the features selection processing and facilitates any modification or changes in the features selection process that may happen later. In addition, it improves the overall performance of the ESVDF. We have examined the feasibility of our approach by conducting several experiments using the DARPA dataset. The experimental results indicate that the proposed algorithm can deliver a satisfactory performance in terms of classification accuracy, training and testing time.

Intrusion detection systems (EDSs) deal with large amounts of data containing irrelevant and/or redundant features. These features result in a slow training and testing process, heavy computational resources, and low detection accuracy. Features selection, therefore, is an important issue in EDSs. A reduced features set improves system accuracy and speeds up the training and testing process considerably. In this paper, we propose a novel and simple method - enhanced support vector decision function (ESVDF)-for features selection. This method selects features based on two important factors: the feature's rank (weight), which is calculated using support vector decision function (SVDF), and the correlation between the features, which is determined by either the forward selection ranking (FSR) or backward elimination ranking (BER) algorithm. Our method significantly decreases training and testing times without loss in detection accuracy. Moreover, it selects the features set independently of the classifier used. We have examined the feasibility of our approach by conducting several experiments using the DARPA dataset. The experimental results indicate that the proposed algorithms can deliver satisfactory results in terms of classification accuracy, training time, and testing time.

Due to the rapid growth of network technologies and substantial improvement in attack tools and techniques, a distributed intrusion detection system (dIDS) is required to allocate multiple IDSs across a network to monitor security events and to collect data. However, dIDS architectures suffer from many limitations such as the lack of a central analyzer and a heavy network load. In this paper, we propose a new architecture for dIDS, called a collaborative architecture for dIDS (C-dIDS), to overcome these limitations. The C-dIDS contains one-level hierarchy dIDS with a non-central analyzer. To make the detection decision for a specific IDS module in the system, this IDS module needs to collaborate with the IDS in the lower level of the hierarchy. Cooperating with lower level IDS module improves the system accuracy with less network load (just one bit of information). Moreover, by using one hierarchy level, there is no central management and processing of data so there is no chance for a single point of failure. We have examined the feasibility of our dIDS architecture by conducting several experiments using the DARPA dataset. The experimental results indicate that the proposed architecture can deliver satisfactory system performance with less network load.

The interaction between a robot and a human becomes meaningful when the robotic agent possesses some level of human-like cognition. This paper proposes an attention-based approach for multi-modal HRI. The core of the proposed approach is a bio-inspired artificial model of visual attention which enables a robot to focus on the visually salient and/or behaviorally relevant stimuli in the surrounding. The attention model provides the human partner with the opportunity to manipulate the attention behavior of the robot through natural speech command. Similarly the robot is able to manipulate the attention of the human partner using its actuators. Thus the bio-inspired visual attention mechanism along with the sensors and actuators enables the robot to establish joint attention with the human partner. Formation of this joint attention is the basis for further human-robot interaction. Experimental results validate different aspects of the proposed attention-based HRI framework.

Extracting different clusters of the given data is an appealing topic in swarm intelligence applications. This paper introduces multiple cooperative swarms and single swarm clustering approaches and provides mathematical descriptions explaining why the former approach outperform the other one. Moreover, the stability analysis is proposed to obtain the model order of the data using multiple cooperative swarms clustering approach. The proposed clustering approach is evaluated using three data sets and its performance is compared with that of other clustering techniques.

This paper proposes and describes an effective utilization of the heuristic optimization. The focus of this research is on a hybrid method combining two heuristic optimization techniques; Differential evolution algorithms (DE) and particle swarm optimization (PSO), to train the beta basis function neural network (BBFNN). Denoted as PSO- DE, this hybrid technique incorporates concepts from DE and PSO and creates individuals in a new generation not only by crossover and mutation operations as found in DE but also by mechanisms of PSO. The results of various experimental studies using the Mackey time prediction have demonstrated the superiority of the hybrid PSO-DE approach over the other four search techniques in terms of solution quality and convergence rates.

Extracting natural groups of the unlabeled data is known as clustering. To improve the stability and robustness of the clustering outputs, clustering ensembles have emerged recently. In this paper, an ensemble of particle swarm clustering algorithms is proposed. That is, the members of the ensemble are based on the cooperative swarms clustering approaches. The performance of the proposed particle swarm clustering ensemble is evaluated using di®erent data sets and is compared to that of other clustering techniques.

For intelligent sensory systems, it is highly desirable to develop assessment methods that can continuously evaluate the reliability of potential sensory strategies taking into consideration changes in observation conditions. This relies on measuring a set of complementary features from multiple sensors and combining these features in an "intelligent" way that maximizes information gather and minimizes the impact of noise coming from the individual sensors. In this work, we formulate a statistical assessment method for estimating the reliability of observation conditions and propose an optimal mapping into weighting measures using genetic algorithms. Our approach is particularly beneficial for multimodal systems such as audio-visual speech recognition (AVSR).

In this paper, a new method is proposed based on the side information and non-dominated sorting evolution strategy (NSES)-based K-means clustering algorithm. In a distance metric learning approach, data points are transformed to a new space where the Euclidean distances between similar and dissimilar points are at their minimum and maximum, respectively. However, the NSES-based K-means clustering yields globally optimized Gaussian components for an accent classification system. This hybrid clustering and classification approach enhances the performance of natural language call-routing systems. Accent classification performs the task of acoustic model switching based on the confidence measure for the callerpsilas query.

Many methods for solving optimization problems, whether direct or indirect, rely upon gradient information and therefore may converge to a local optimum. Global optimization methods like evolutionary algorithms, overcome this problem. In this work it is investigated how to construct a quality BBF network for a specific application can be a time-consuming process as the system must select both a suitable set of inputs and a suitable BBF network structure. Evolutionary methodologies offer the potential to automate all or part of these steps. This study illustrates how a hybrid BBFN-PSO system can be constructed, and applies the system to a number of datasets. The utility of the resulting BBFNs on these optimization problems is assessed and the results from the BBFN-PSO hybrids are shown to be competitive against the best performance on these datasets using alternative optimization methodologies. The results show that within these classes of evolutionary methods, particle swarm optimization algorithms are very robust, effective and highly efficient in solving the studied class of optimization problems.

This paper aims at presenting different strategies for the construction of beta basis function (BBF) fuzzy neural network. These strategies lead to the determination of the network architecture by determining the structure of the hidden layer and parameters of its centers based on data structure. For that, we use self organizing maps (SOM) clustering to construct a mapped structure of the real training data. By analyzing this structure, we proceed to neuron selection. Data sets were also analyzed with the fuzzy c-means (FCM) clustering technique to generate fuzzy membership values presenting fuzzy outputs for our fuzzy neural model. We propose to estimate the parameters of beta basis function in order to obtain better data coverage. Experimental results show that the use of the proposed technique produces better results.

This paper argues that the object- and space-based modes of visual attention can be naturally integrated in a common mathematical framework. In an earlier work we have proposed a mathematical model of visual attention for robotic system exploiting the knowledge of visual attention mechanism of the primates. This paper investigates on the validity of the proposed model for robotic systems through experimentation on a real robot. The paper sheds light on a number of real world issues involved with the design of visual attention system for physically embodied robots and explains how the proposed Bayesian model of visual attention addresses these issues. The object- and space-based modes of visual attention are naturally integrated in the model and is reflected in the sequential Monte Carlo implementation of the model on a real robot.

This paper describes a speaker-independent accent-based natural language call-routing system. Based on a speaker's accent group, this system directs customer calls to the automatic speech recognition system that is most suitable to recognize the input query. The speech recognition system understands the caller's query and converts it into routing keywords. Accent identification is the most important factor for improving the performance of natural language call-routing systems because accents vary widely, even within the same country or community. This variation occurs when non-native speakers start to learn a second language; the substitution of native language phoneme pronunciation is a common occurrence. In this paper, a new method is proposed based on class inequivalent side information and an evolutionary-based K-means clustering algorithm. In a distance metric learning approach, data points are transferred to a new space where the Euclidean distances between similar and dissimilar points are at their minimum and maximum, respectively. However, the evolutionary-based K-means clustering approach yields globally optimized Gaussian components for an accent classification system.

This study explores a new domain representation method for natural language processing based on an application of possibility theory. In our method, domain-specific information is extracted from natural language documents using a mathematical process based on Rieger's notion of semantic distances, and represented in the form of possibility distributions. We implement the distributions in the context of a possibilistic domain classifier, which is trained using the SchoolNet corpus.

In this paper, a framework is proposed for the distributed control and coordination of multiagent systems (MASs). In the proposed framework, the control of MASs is regarded as achieving decentralized control and coordination of agents. Each agent is modeled as a coordinated hybrid agent, which is composed of an intelligent coordination layer and a hybrid control layer. The intelligent coordination layer takes the coordination input, plant input, and workspace input. In the proposed framework, we describe the coordination mechanism in a domain-independent way, i.e., as simple abstract primitives in a coordination rule base for certain dependence relationships between the activities of different agents. The intelligent coordination layer deals with the planning, coordination, decision making, and computation of the agent. The hybrid control layer of the proposed framework takes the output of the intelligent coordination layer and generates discrete and continuous control signals to control the overall process. To verify the feasibility of the proposed framework, experiments for both heterogeneous and homogeneous MASs are implemented. The proposed framework is applied to a multicrane system, a multiple robot system, and a MAS consisting of an overhead crane, a mobile robot, and a robot manipulator. It is demonstrated that the proposed framework can model the three MASs. The agents in these systems are able to cooperate and coordinate to achieve a global goal. In addition, the stability of systems modeled using the proposed framework is also analyzed.

In this paper, we propose a steady-state genetic algorithm (GA) with cycle-based representation and a contraction digraph model to deal with the guideway network design problem of personal rapid transit (PRT). PRT is a novel transportation paradigm, where many computer-controlled vehicles running on an elevated guideway network (GN). A GN may contain hundreds of guideway links and how to design the minimum-cost feasible GN is a challenging problem. Given a set of stations, the proposed GA models a candidate GN as a union of one or more simple directed cycles visiting two or more stations. This cycle representation not only provides high solution locality but allows us to establish a contraction digraph model, where its feasibility can be efficiently evaluated. We also develop special genetic operators well suited for the cycle representation. Numerical experiments conducted for various problem instances show the proposed GA outperforms the conventional ones once the solution is represented by a moderate number of cycles.

It is believed that modeling temporal structure of the speech data may be useful for the problem of speech emotion recognition (T. Nwe et al., 2003). In this paper, Gaussian mixture vector autoregressive model is proposed as a statistical classifier for this task. The main motivation behind using such a model is its ability to model the dependency among extracted speech feature vectors as well as the multi-modality in their distribution. When applied to the Berlin emotional speech database, the proposed technique provides a classification accuracy of 76% versus 71% for the hidden Markov model, 67% for the k-nearest neighbors, 55% for feed-forward neural networks. The model gives also better discrimination between high-arousal, low arousal, and neutral emotions than the HMM.

Most of text categorization techniques are based on word and/or phrase analysis of the text. Statistical analysis of a term frequency captures the importance of the term within a document only. However, two terms can have the same frequency in their documents, but one term contributes moreto the meaning of its sentences than the other term. Thus, the underlying model should indicate terms that capture these mantics of text. In this case, the model can capture terms that present the concepts of the sentence, which leads todiscover the topic of the document. A new concept-based model that analyzes terms on the sentence and document levels rather than the traditional analysis of document only is introduced. The concept-based model can effectively discriminate between non-important terms with respect to sentence semantics and terms which hold the concepts that represent the sentence meaning. The proposed model consists of concept-based statistical analyzer, conceptual ontological graph representation,and concept extractor. The term which contributes to the sentence semantics is assigned two different weights by the concept-based statistical analyzer and the conceptual ontological graph representation. These two weights are combined into a new weight. The concepts that have maximum combined weights are selected by the concept extractor. A set of experiments using the proposed concept-basedmodel on different datasets in text categorization is conducted. The experiments demonstrate the comparison between traditional weighting and the concept-based weighting obtained by the combined approach of the concept-based statistical analyzer and the conceptual ontological graph. The evaluation of results is relied on two quality measures, the Macro-averaged F1 and the Error rate. These quality measures are improved when the newly developedconcept-based model is used to enhance the quality of thetext categorization.

A new clustering technique by the use of multiple swarms is proposed. The proposed technique mimics the behavior of biological swarms which explore food situated in several places. We model the clustering problem using particle swarm optimization (PSO) approach. The proposed method considers multiple cooperating swarms to find centers of clusters. By assigning a portion of the solution space to each swarm, the exploration ability to find the solution is enhanced. Moreover, the cooperation among swarms increases the between-class distance. The proposed method outperforms k-means clustering as well as conventional PSO-based clustering techniques

Most of the common techniques in text retrieval are based on the statistical analysis of a term either as a word or a phrase. Statistical analysis of a term frequency captures the importance of the term within a document only. Thus, to achieve a more accurate analysis, the underlying representation should indicate terms that capture the semantics of text. In this case, the representation can capture terms that present the concepts of the sentence, which leads to discover the topic of the document. A new concept-based representation, called Conceptual Ontological Graph (COG), where a concept can be either a word or a phrase and totally dependent on the sentence semantics, is introduced. The aim of the proposed representation is to extract the most important terms in a sentence and a document with respect to the meaning of the text. The COG representation analyzes each term at both the sentence and the document levels. This is different from the classical approach of analyzing terms at the document level. First, the proposed representation denotes the terms which contribute to the sentence semantics. Then, each term is chosen based on its position within the COG representation. Lastly, the selected terms are associated to their documents as features for the purpose of indexing before text retrieval. The COG representation can effectively discriminate between non-important terms with respect to sentence semantics and terms which hold the key concepts that represent the sentence meaning. Large sets of experiments using the proposed COG representation on different datasets in text retrieval are conducted. Experimental results demonstrate the substantial enhancement of the text retrieval quality using the COG representation over the traditional techniques. The evaluation of results relies on two quality measures, the bpref and P(10). Both the quality measures improved when the newly developed COG representation is used to enhance the quality of the text retrieval results.

Modern statistical techniques used in the field of natural language processing are limited in their applications by the fact they suffer from the loss of most of the semantic information contained in text documents. Fuzzy techniques have been proposed as a way to correct this problem through the modelling of the relationships between words while accommodating the ambiguities of natural languages. However, these techniques are currently either restricted to modelling the effects of simple words or are specialized in a single domain. In this paper, we propose a novel statistical-fuzzy methodology to represent the actions described in a variety of text documents by modelling the relationships between subject-verb-object triplets. The research will focus in the first place on the technique used to accurately extract the triplets from the text, on the necessary equations to compute the statistics of the subject-verb and verb-object pairs, and on the formulas needed to interpolate the fuzzy membership functions from these statistics and on those needed to de fuzzify the membership value of unseen triplets. Taken together, these sets of equations constitute a comprehensive system that allows the quantification and evaluation of the meaning of text documents, while being general enough to be applied to any domain. In the second phase, this paper will proceed to experimentally demonstrate the validity of our new methodology by applying it to the implementation of a fuzzy classifier conceived especially for this research. This classifier is trained using a section of the Brown Corpus, and its efficiency is tested with a corpus of 20 unseen documents drawn from three different domains. The positive results obtained from these experimental tests confirm the soundness of our new approach and show that it is a promising avenue of research.

We propose a new feature selection method based on distributed genetic algorithms and bi-coded genes. This solution uses homogeneous and heterogeneous population strategies to minimize the complexity and to accelerate the algorithm convergence. The importance rate is computed for each feature measure to estimate the contribution of each feature in the finale selected vector. A new fitness function was proposed to take into consideration the recognition rate relatively to the size of the selected features subset. Two genetic codes are used to represent each member; a binary code to represent when the corresponding feature was selected or not; the second real code was used to estimate the importance rate of the selected feature or the selection probability for the non selected feature.

This paper ments a multiobjective genetic algorithm (MOGA) to find the optimal guideway networks (GNs) of personal rapid transit (PRT). The objective of the GN design problem (GNDP) is to find the GNs that minimize the construction cost and peak-hour traffic while satisfying the connectivity constraint. To solve the GNDP, we develop an MOGA by modifying an impmved nondominated sorting genetic algorithm (NSGA-11). The developed MOGA inherits the advantages of NSGA-I1 and adopts a GNDP-specific mutation operator, which provides better solution quality. To verify the effectiveness and efficiency of the developed MOGA, we conducted numerical experiments on the GNDPs with up to 15 stations and 210 links.

In this research, we introduce FPGA based fuzzy logic controller (FLC). The benefit of using FPGA based FLC compare to software FLC is that the computation time reduction. Using this FLC, we design automated car back parallel parking system also with complete FPGA based controller. We build a small-scaled robot car and test on a real environment with VHDL code for wall following and parking. This paper describes the background of fuzzy logic system, the design of fuzzy logic system with FPGA and the experimental results.

We propose here to develop a methodology to extract semantic knowledge from plain written English documents and represent it using a formal mathematical expression, in order to facilitate its use in practical applications. Our fundamental conjecture is that most of the semantic information of a sentence lies in the action described by that sentence. Consequently, we focus on extracting from the text the words whose relationships represent actions, and on modelling those relationships. We then demonstrate the practicality of our methodology by applying it to a domain classifier.

Most of text mining techniques are based on word and/or phrase analysis of the text. The statistical analysis of a term (word or phrase) frequency captures the importance of the term within a document. However, to achieve a more accurate analysis, the underlying mining technique should indicate terms that capture the semantics of the text from which the importance of a term in a sentence and in the document can be derived. A new concept-based mining model that relies on the analysis of both the sentence and the document, rather than, the traditional analysis of the document dataset only is introduced. The proposed mining model consists of a concept-based analysis of terms and a concept-based similarity measure. The term which contributes to the sentence semantics is analyzed with respect to its importance at the sentence and document levels. The model can efficiently find significant matching terms, either words or phrases, of the documents according to the semantics of the text. The similarity between documents relies on a new concept-based similarity measure which is applied to the matching terms between documents. Experiments using the proposed concept-based term analysis and similarity measure in text clustering are conducted. Experimental results demonstrate that the newly developed concept-based mining model enhances the clustering quality of sets of documents substantially.

One of the many issues that confront traditional statistical approaches of natural language understanding (NLU) is on how to overcome the insufficient co-occurrence information caused by the limited boundary of statistical approaches. Researches have long used the imparting of human knowledge into statistical approaches, including definition of rules and collections of hierarchy of concepts. However, these are difficult to define even for a domain expert. They are also very much people and domain dependent. This study proposes a fuzzy approach to tackle these issues in a way as to provide a methodology for logical reorganizing context in order to tackle the issue of boundary limitation, to create the more reasonable and understandable word association which will be referenced as membership degree in latter stage, and to make the processes of imparting of human knowledge easier and less domain dependent. The accomplishment of these tasks could be achieved through the concept of precisiated natural language (PNL)

The coordinated hybrid agent (CHA) framework for the control of multi-agent systems (MASs) has been used to model both the homogeneous and the heterogeneous multi-agent systems. In this framework, the control of the MASs is regarded as a decentralized control and coordination of agents. The CHA framework is able to implement coordination tasks for multi-agent systems. In this study, the optimization of MASs modelled by the CHA framework is studied. The time-driven dynamics and the event-driven dynamics for the optimization of a CHA system are given. The optimization problem of the MASs is analyzed. An example is also given to illustrate how to define the optimization problem for a CHA. The forward algorithm is also introduced for solving the optimization problem for a CHA system.

Dempster-Shafer evidence theory (DSET) is a popular paradigm for dealing with uncertainty and imprecision. Its corresponding evidential reasoning framework is theoretically attractive. However, there are outstanding issues that hinder its use in real-life applications. Two prominent issues in this regard are 1) the issue of basic probability assignments (masses) and 2) the issue of dependence among information sources. This paper attempts to deal with these issues by utilizing neural networks in the context of pattern classification application. First, a multilayer perceptron neural network with the mean squared error as a cost function is implemented to calculate, for each information source, posteriori probabilities for all classes. Second, an evidence structure construction scheme is developed for transferring the estimated posteriori probabilities to a set of masses along with the corresponding focal elements, from a Bayesian decision point of view. Third, a network realization of the Dempster-Shafer evidential reasoning is designed and analyzed, and it is further extended to a DSET-based neural network, referred to as DSETNN, to manipulate the evidence structures. In order to tackle the issue of dependence between sources, DSETNN is tuned for optimal performance through a supervised learning process. To demonstrate the effectiveness of the proposed approach, we apply it to three benchmark pattern classification problems. Experiments reveal that the DSETNN out-performs DSET and provide encouraging results in terms of classification accuracy and the speed of learning convergence.

In this work, we provide a mathematical framework for estimating the expected value of the carrier to interference ratio and evaluating the probability of error in a wireless ad hoc network. Our approach differs from the earlier studies in that previous valuable work has been mostly based on simulations, on simplified regular patterns for node distribution, has been limited to single service networks, or has not integrated the medium access model in the network. In our analysis, we consider a multiservice interference limited wireless ad hoc network using a multiple access scheme with carrier sensing at the data link layer. The population of nodes in the network is considered as a number governed by a two-dimensional Poisson process. We derive explicit formulas for the carrier to interference ratio and probability of error experienced by nodes in the network. Numerical results are provided for the case of two service classes, where carrier to interference ratios and probabilities of packet loss experienced by nodes in the network are presented as functions of the node densities.

We propose an evolutionary method for the design of Beta fuzzy systems (BFS). Classical training algorithms start with a predetermined number of fuzzy rules for fuzzy systems. Generally speaking, the fuzzy system created is either insufficient or over-complicated. This paper describes a hierarchical genetic learning model of the BFS. In order to examine the performance of the proposed algorithm, it is used for the identification of an induction machine fuzzy plant model. The results obtained have been encouraging.

There exist several industrial applications for large crane systems. Most of them experience serious problems with load swing. This paper presents a fuzzy based control scheme to minimize load swing for crane systems while maintaining continuous payload transportation. The control system of the crane is built using behavior-based approaches. In the control system developed, each module generates behaviors, and improvement in the performance of the system proceeds by adding new modules to the system. In order to develop the anti-swing module, fuzzy logic controller is applied using information extracted from potentiometers. The fuzzy controller provides a mechanism for dealing with imprecise sensor data. The anti-swing behaviors are successfully implemented by formulating a set of fuzzy rules. The performance of the developed system is illustrated by both simulations and experiments. The simulation and experimental results of the system show that the system remains stable under several operating situations.

This paper tackles the issue of bandwidth allocation in asynchronous transfer mode (ATM) networks using recently developed tools of computational intelligence. The efficient bandwidth allocation technique implies effective resources utilization of the network. The fluid flow model has been used effectively among other conventional techniques to estimate the bandwidth for a set of connections. However, such methods have been proven to be inefficient at times in coping with varying and conflicting bandwidth requirements of the different services in ATM networks. This inefficiency is due to the computational complexity of the model. To overcome this difficulty, many approximation-based solutions, such as the fluid flow approximation technique, were introduced. Although such solutions are simple, in terms of computational complexity, they nevertheless suffer from potential inaccuracies in estimating the required bandwidth. Soft computing-based bandwidth controllers, such as neural networks- and neurofuzzy-based controllers, have been shown to effectively solve an indeterminate nonlinear input-output (I-O) relations by learning from examples. Applying these techniques to the bandwidth allocation problem in ATM network yields a flexible control mechanism that offers a fundamental tradeoff for the accuracy-simplicity dilemma.

The problem of simultaneous approximations of a given function and its derivatives, has been addressed frequently in pure and applied mathematics. In pure mathematics, Bernstein polynomials get their importance from the fact that they provide simultaneous approximation of a function and its derivatives. In neural network theory, feedforward networks were shown to be universal approximators of an unknown function and its derivatives. In this paper, we consider fuzzy logic systems with the membership functions of each input variables are chosen as the translations and dilations of one appropriately fixed function. We prove, by a constructive proof based on discretization of the convolution operator, that under certain conditions made on the input variables membership functions, fuzzy logic systems of Sugeno type are universal approximators of a given function and its derivatives.

In this paper, a neuro-fuzzy based classification technique is adopted to efficiently deal with the classification problem of the heterogeneous medical data sets. The Proposed classification technique is based on the neuro-fuzzy classification (NE-FCLASS) system. However, several improvements are introduced to the origin NEFCLASS. The motivation of this work is triggered by the fact that most conventional classification techniques are capable of handling the numeric data sets but not the heterogenous ones. This can be seen in the classification of the medical data sets. This paper tackles the data classification problem of two medical diseases. The first data set, which is a numeric data set, is related to the Wisconsin breast cancer diagnosis. The second is a heterogeneous data set and is the Wisconsin heart disease diagnosis. Experimental results demonstrate that the proposed technique can effectively improve the classification performance of heterogenous data sets.

We report on a fuzzy logic-based language understanding system applied to speech recognition. This system acquires conceptual knowledge from corpus data and organizes such knowledge into fuzzy logic inference rules. The system parses speech recognition results into conceptual structures in a robust manner, and thus is able to tolerate noise caused by speech recognition errors. We discuss the fuzzy inference rule learning method and explain its organization. Experimental results that demonstrate the ability of the system to deal with complex speech input instances are reported.

We examine in this article the complex problem of simultaneous position and internal force control in multiple cooperative manipulator systems. This is done in the presence of unwanted parametric and modeling uncertainties as well as external disturbances. A decentralized adaptive hybrid intelligent control scheme is proposed here. The controller makes use of a multi-input multi-output fuzzy logic engine and a systematic online adaptation mechanism. Unlike conventional adaptive controllers, the proposed one does not require a precise model of the system's dynamics. The performance of the proposed controller is compared to that of a well known conventional adaptive controller.

The robustness of a given vision system in the field of robotics is a very challenging problem and represents a major bottleneck in any industrial setting. Nevertheless, there is a hypothesis that the fusion of multiple natural features facilitates a robust detection and object tracking in scenes of real world complexity. Several fusion methods have been tested for cue integration with good results, but the computational effort grows as the number of features increases. This research work represents a variant of the fusion method based both on distributed systems and on an agent concept. In this work, multiple agents interact with each other to perform different roles. The structure has a cooperative approach so that the agents work as a team. The communication among the agents is based on the Event Service of CORBA technology. By using this architecture, we are exploiting the parallelism and concurrency of distributed systems, and by using the concept of agents we are exploiting the encapsulation concept to built modular systems.

This paper presents a novel technique to fuse multi information sources for the purpose of pattern classification. The proposed data fusion technique is based on the Dempster-Shafer evidence theory. Mass functions are derived from probabilistic and fuzzy measures that are associated with discriminant functions for pattern classification. Simulated synthetic images as well as real human brain magnetic resonance images (MRI) are tested to demonstrate the performance and effectiveness of the proposed approach. It is concluded from the experimental results that the proposed algorithm is quite effective and superior to other approaches such as the Bayesian approach. Furthermore, the paper explains how this approach exhibits a capability to handle uncertainty, imprecision and conflicts which often hinders multi information fusion.

Pipeline surface defects such as holes and cracks cause major problems for utility managers, particularly when the pipeline is buried under the ground. Manual inspection for surface defects in the pipeline has a number of drawbacks, including subjectivity, varying standards, and high costs. Automatic inspection system using image processing and artificial intelligence techniques can overcome many of these disadvantages and offer utility managers an opportunity to significantly improve quality and reduce costs. A recognition and classification of pipe cracks using images analysis and neuro-fuzzy algorithm is proposed. In the preprocessing step the scanned images of pipe are analyzed and crack features are extracted. In the classification step the neuro-fuzzy algorithm is developed that employs a fuzzy membership function and error backpropagation algorithm. The idea behind the proposed approach is that the fuzzy membership function will absorb variation of feature values and the backpropagation network, with its learning ability, will show good classification efficiency.

We present soft computing-based results pertaining to the hierarchical tuning process of PID controllers located within the control loop of a class of nonlinear systems. The results are compared with PID controllers implemented either in a stand alone scheme or as a part of conventional gain scheduling structure. This work is motivated by the increasing need in the industry to design highly reliable and efficient controllers for dealing with regulation and tracking capabilities of complex processes characterized by nonlinearities and possibly time varying parameters. The soft computing-based controllers proposed are hybrid in nature in that they integrate within a well-defined hierarchical structure the benefits of hard algorithmic controllers with those having supervisory capabilities. The controllers proposed also have the distinct features of learning and auto-tuning without the need for tedious and computationally extensive online systems identification schemes.

This paper presents a survey about different types of fuzzy information measures. A number of schemes have been proposed to combine the fuzzy set theory and its application to the entropy concept as a fuzzy information measurements. The entropy concept, as a relative degree of randomness, has been utilized to measure the fuzziness in a fuzzy set or system. However, a major difference exists between the classical Shannon entropy and the fuzzy entropy. In fact while the later deals with vagueness and ambiguous uncertainties, the former tackles probabilistic uncertainties (randomness).

A fuzzy integral based region merging algorithm is presented to deal with the issue of oversegmentation due to the watershed transform. The algorithms integrates region and edge features together using a fuzzy logic based fusion method. Firstly, preprocessing and watershed segmentation are performed on the image. Depending on the complexity of the image content the segmentation process may produce many more regions than what are really expected to exist in the image. To reduce the number of regions that results due to oversegmentation, a fusion process is applied to these regions recursively according to the principle of the maximum fuzzy integral. After transferring these features to memberships that reflect the degree that a given region to belong to its neighboring regions, an integration (fusion) scheme is used to compute a fuzzy integral based on which a decision is made with respect to the region merging. To evaluate the performance of the proposed approach it has been applied to real images such as MRI and natural images.

This paper presents the application of soft computing-based techniques to the bandwidth allocation (BA) problem in ATM networks. Efficient bandwidth allocation technique implies effective resources utilization. The fluid flow model has been known to be among the most accurate conventional methods to estimate the bandwidth of a set of connections. However, and due to the computational complexity, such methods have been proven to be inefficient in coping with varying and conflicting bandwidth requirements in ATM networks. To overcome this difficulty, many approximation-based solutions were introduced. Although such solutions are not simple, they nevertheless suffer from possible inaccuracy in estimating the required bandwidth. Soft computing-based bandwidth controllers, such as neural networks and neurofuzzy based controllers, have the capability to solve indeterminate non-linear input-output relations by learning from examples. Applying these techniques to the bandwidth allocation problem in ATM network yields a flexible control mechanism that offers a fundamental trade-off for the accuracy-simplicity dilemma.

Game theory deals with decision-making processes involving two or more parties, also known as players, with partly or completely conflicting interests. Decision-makers in a conflict must often make their decisions under risk and under unclear or fuzzy information. In this paper, two distinct fuzzy approaches are employed to investigate an extensively studied 2/spl times/2 game model-the game of Chicken. The first approach uses a fuzzy multicriteria decision analysis method to obtain optimal strategies for the players. It incorporates subjective factors into the decision-makers' objectives and aggregates objectives using a weight vector. The second approach applies the theory of fuzzy moves (TFM) to the game of Chicken. The theory of moves (TOM) is designed to bring a dynamic dimension to the classical theory of games by allowing decision-makers to look ahead for one or several steps so that they can make a better decision. TOM is the crisp counterpart of TFM, the approach we implement here to deal with games that include fuzzy and uncertain information. The application of fuzzy approaches to the game of Chicken demonstrates their effectiveness in manipulating subjective, uncertain, and fuzzy information and provides valuable insights into the strategic aspects of Chicken.

A variable-structure based fuzzy-logic identifier (VSFI) is introduced to model a class of black-box nonlinear systems. The proposed identifier adopts a serial-parallel structure and, unlike most fuzzy identifiers, does not require measurements of all the system's states. Based on output measurements, the system states are estimated using a high gain observer. It is shown that the proposed VSFI is stable provided that the system identified is stable. Furthermore, we show that the estimator state-errors converge exponentially to an arbitrarily small ball. Simulation results illustrates that the identification scheme proposed might serve as a potential candidate for nonlinear system identification.

We discuss here the implementation aspects of recently developed tools of computational intelligence for tackling the issue of joint trajectory generation of a class of multi-joint cooperative robotic systems. This is closely related to the inverse kinematics problem which usually represents a heavy computational burden on the processing power of any complex robotic structure. High nonlinearities, heavy coupling between the degrees of freedom, and time variant configuration of the robot structure heavily contribute to these difficulties. Soft computing techniques have surged in recent years as effective computational tools for emulating the human capabilities when dealing with complex systems. Some of them are used here to synthesize approaches capable of substantially improving solving the inverse kinematics problem for a class of robotic systems and help generating the joint trajectories in a faster way.

The layout of temporary facilities in a construction site deals with the selection of their most efficient layout in order to operate efficiently and cost effectively. The layout design seeks the best arrangement of facilities within the available area. In the design process of the layout, many objectives must be considered to effectively utilize people resources, equipment, space, and energy. This study proposes a soft-computing-based approach to improve the layout process of facilities. The main objective is on obtaining the closeness relationship values between each pair of facilities in a construction site. To achieve this, an integrated approach, using fuzzy set theory and genetic algorithms, is used to investigate the layout of temporary facilities in relation with the planned building(s) in a construction site. An example application is presented to illustrate the proposed approach and the results are then discussed along with recommendations for further work. Depending on the importance of relationships among the various facilities in the construction site, this study is expected to provide engineers with an appropriate tool to compare and evaluate different layouts and select the most appropriate and efficient one.

A robust control design for high performance joint trajectory tracking of a flexible lightweight manipulator system is proposed. The design is based on a combined controller-observer scheme involving the sliding manifold approach and the optimal interpolation technique. This controller provides the designer with an enhanced joint tracking performance when the system is subject to parametric variations due to structural disturbances caused by link flexibility and load uncertainties. The parametric variations are handled by sliding control and the estimation of the nonlinearly excited elastic dynamics by an optimal interpolator of the structure's dynamic responses. The design procedure is progressive, i.e., we start with a basic controller and then modify it in order to improve the performance. Closed loop simulations with the various designed controllers are used to validate the analytical results and to help choosing the most suitable one.

Presents a procedure for modeling and eliminating an end effector configuration error as a result of a faulty joint or a damaged link. This procedure provides an inexpensive software alternative to hardware replacement. A neural network model was developed and tested an a 6 DOF PUMA robot. The network approximates the error based on data obtained through observing the robot while executing a set of MOVE commands. The results show that, regardless of the error source, the robot's accuracy could be highly improved even when a small number of data points are used.

An investigation pertaining to identification and expert control design of a diesel engine system is carried out. Diesel engines are known for their high versatility, making them useful in a wide range of industrial applications. Despite several of their attractive features, diesel engines suffer from highly nonlinear and time varying dynamics making them hard to control. The current study provides a means of comparing several types of controllers, some of which are adaptive based and others which are knowledge based. The main goal is to obtain the best possible performance in terms of speed regulation, robustness to load disturbances, and fuel efficiency. This is very helpful in gaining insights for designing a hierarchical based controller under which the system can switch smoothly from one controller to another depending on the operating conditions of the system.