The immense success of the Transformer architecture in Natural Language Processing has led to its adoption in Time Series Forecasting (TSF), where superior performance has been shown. However, a recent important paper questioned their effectiveness by demonstrating that a simple single layer linear model outperforms Transformer-based models. This was soon shown to be not as valid, by a better transformer-based model termed PatchTST. More recently, TimeLLM demonstrated even better results by repurposing a Large Language Model (LLM) for the TSF domain. Again, a follow up paper challenged this by demonstrating that removing the LLM component or replacing it with a basic attention layer in fact yields better performance. One of the challenges in forecasting is the fact that TSF data favors the more recent past, and is sometimes subject to unpredictable events. Based upon these recent insights in TSF, we propose a strong Mixture of Experts (MoE) framework. Our method combines the state-of-the-art (SOTA) models including xLSTM, enhanced Linear, PatchTST, and minGRU, among others. This set of complimentary and diverse models for TSF are integrated in a Transformer based MoE gating network. Our proposed model outperforms all existing TSF models on standard benchmarks, surpassing even the latest approaches based on MoE frameworks. Our code is available at: https://github.com/muslehal/EMTSF

While considerable research has been recently done in improving the accuracy of face liveness detection, the best current approaches use a two-step process of first applying non-linear anisotropic diffusion to the incoming image and then using a deep network for final liveness decision. We develop a novel deep architecture for face liveness detection on video frames that uses the diffusion of images followed by a deep Convolutional Neural Network (CNN) and a Long Short-Term Memory (LSTM) to classify the video sequence as real or fake. Even though the use of CNN followed by LSTM is not new, combining it with diffusion (that has proven to be the best approach for single image liveness detection) is novel. Performance evaluation of our architecture on the REPLAY-ATTACK dataset yields competitive results with 98.64% accuracy and an HTER of 3.63%.

Only weights and biases are learned by gradient descent-based training of Deep Neural Networks (DNN). The other parameters, i.e., hyperparameters have a huge influence on the quality of the model but finding optimal values for them is not a trivial solution. The hyperparameter space grows exponentially and they also display non-linearity and interactions. A network with twelve hyperparameters, each with five potential values can have about 250 million unique combinations of hyperparameter sequences. If training on each set takes 6 minutes, exhaustive training on all potential combinations of the hyperparameters to find the optimal values will take almost 3000 years. Expert knowledge or random selection are some alternate options, but they are not scalable or consistently reliable. Metaheuristics such as evolutionary algorithms are a great choice for solving combinatorial optimization problems like hyperparameter optimization. While other researchers have used evolutionary algorithms such as standard implementation of genetic algorithm (GA), we introduce additional nature-inspired enhancements to GA for better exploration of the hyperparameter solution space to optimize the DNN architecture. The training is complemented with Monte Carlo based variance reduction method called importance sampling. We demonstrate that these fine-tunings result in improvements in the network accuracy on MNIST and CIFAR-10 datasets that outperforms standard use of genetic algorithm.

The human face facial complexity and the face changes make the face recognition system a challenging task to design and difficult to implement. The correlation between the training images which has a high impact on the accuracy of the face recognition system never considered by researchers. In this paper, we presented an enhanced framework to improve the face recognition using the classical conventional Principal Component Analysis (PCA) and the Back-Propagation Neural Network (BPNN). A key contribution of this work is based on obtaining a robust training dataset called the T-Set using the correlation between all the images in the training dataset not based on the image density which adds a distinct layer between the dataset. We used the PCA descriptor for features extraction and dimension reduction to show that there is a promising enhancement even with using traditional algorithms. We combined five distance methods (Correlation, Euclidean, Canberra, Manhattan, and Mahalanobis) to obtain the T-Set using the square-root of the sum of the squares to achieve higher accuracy. We added a strength factor to each of the distance methods, and we achieved higher face recognition accuracy than the current approach. Our experimental results on YALE and ORL datasets demonstrate that the approach we proposed improved the accuracy of face recognition system with respect to the existing methods.

Face recognition has become a fascinating field for researchers. The motivation behind the enormous interest in the topic is the need to improve the accuracy of many real-time applications. The complexity of the human face and the changes due to different effects make it more challenging to design as well as implement a powerful computational system for human face recognition. In this work, we presented an enhanced approach to improve human face recognition using a Back-Propagation Neural Network (BPNN) and features extraction based on the correlation between the training images. A key contribution of this work is the generation of a new set called the T-Dataset from the original training dataset, which is used to train the BPNN. We generated the T-Dataset utilizing the correlation between the training images without using a conventional technique of image density. The correlated T-Dataset provides a high distinction layer between the training images, which helps the BPNN to converge faster and achieve better accuracy. Data and features reduction is essential in the face recognition process, and researchers have recently focused on the modern Neural Network (NN). Therefore, we used a Principal Component Analysis (PCA) descriptor to prove that there is a potential improvement even using traditional methods. We applied five distance measurement algorithms and then combined them to obtain the T-Dataset, which we fed into the BPNN. We achieved higher face recognition accuracy with less computational cost compared to the current approach by using reduced image features. We test the proposed framework on two small datasets, the YALE and AT&T datasets, as the ground truth. We achieved tremendous accuracy. Furthermore, we evaluate our method on one of the state-of-the-art benchmark datasets, Labeled Faces in the Wild (LFW), where we produce a competitive face recognition performance. Also, we proposed an enhanced framework to improve the face registration using deep transfer learning. We used pre-trained models such as ResNet50 and SeNet50 which trained on the VGGFace2 dataset. We used the LFW dataset to evaluate our method, and we achieve high accuracy.

Natural Language Processing (NLP) systems conventionally treat words as distinct atomic symbols. The model can leverage small amounts of information regarding the relationship between the individual symbols. Today when it comes to texts; one common technique to extract fixed-length features is bag-of-words. Despite its popularity the bag-of-words feature has two major weaknesses: it ignores semantics of the words and the order of words. In this paper, we propose a neural language model that relies on Convolutional Neural Network (CNN) and Bidirectional Recurrent Neural Network (BRNN) over pre-trained word vectors. We utilize bidirectional layers as a substitute of pooling layers in CNN in order to reduce the loss of detailed local information, and to capture long-term dependencies across input sequences. We validate the proposed model on two benchmark sentiment analysis datasets, Stanford Large Movie Review (IMDB), and Stanford Sentiment Treebank (SSTb). Our model achieves a competitive advantage compared with neural language models on the sentiment analysis datasets.

Recent advancements in deep Convolutional Neural Networks (CNNs) have led to impressive progress in computer vision, especially in image classification. CNNs involve numerous hyperparameters that identify the network's structure such as depth of the network, kernel size, number of feature maps, stride, pooling size and pooling regions etc. These hyperparameters have a significant impact on the classification accuracy of a CNN. Selecting proper CNN architecture is different from one dataset to another. An empirical approach is often used in determining the near optimal value of these hyperparameters. Some recent works have tried optimization techniques for hyperparameter selection as well. In this paper, we develop a framework for hyperparameter optimization that is based on a new objective function that combines the information from the visualization of learned feature maps via deconvolutional networks, and the accuracy of the trained CNN model. Nelder-Mead Algorithm (NMA) is used in guiding the CNN architecture towards near optimal hyperparameters. Our proposed approach is evaluated on CIFAR-10 and Caltech-101 benchmarks. The experimental results indicate that the final architecture of a CNN obtained by our objective function outperforms other approaches in terms of accuracy. It is shown that our optimization framework contributes to increase in the depth of network, shrinks the size of stride and pooling sizes to obtain the best CNN architecture.

Human gait recognition is still a very challenging problem because natural human gait is affected by many covariate conditions such as changes in the clothing, variations in viewing angle, and changes in carrying condition. Although existing gait recognition methods perform well under a controlled environment where the gait is in normal condition with no covariate factors, the performance drastically decreases in practical conditions where it is susceptible to many covariate factors. This paper analyzes the most important features of gait under the carrying and clothing conditions. We find that the intra-class variations of the features that remain static during the gait cycle affect the recognition accuracy adversely. Thus, we introduce an effective and robust feature selection method based on the Gait Energy Image. We also propose an augmentation technique to overcome some of the problems associated with the intra-class gait fluctuations, as well as when the amount of the training data is relatively small. Finally, we use dictionary learning with sparse coding and Linear Discriminant Analysis (LDA) to seek the best discriminative data representation before feeding it to the Nearest Centroid classifier. When our method is applied on the large CASIAB and OU-ISIR-B gait data sets, it outperforms existing gait methods.

With increasing number of cores in modern processors, an efficient network on a chip (NOC) is the key to next generation processors. Of the many NOC architectures being proposed, the Spider on network has gained popularity due to its low cost and high performance, and has been implemented in commercial processors. Recently, an improved version of this network termed as the Spider on-Donut (SD) network that extends the Spider on network in a second dimension was developed. The SD network results in lower diameter than a 2-D mesh and thus it is a good candidate for future NOC implementations. This paper presents a new routing algorithm for the SD network that is deadlock free and maximally adaptive. By employing our deadlock free algorithm on one of the virtual channels, maximal flexibility can be achieved in routing. We compute the flexibility provided by our deadlock free routing and show that it results in balanced traffic in the network without causing hotspots.

In circuit-switched routing, the path between a source and its destination is established by incrementally reserving all required links before the data transmission can begin. If the routing algorithm is not carefully designed, deadlocks can occur in reserving these links. Deadlock-free algorithms based on dimension-ordered routing, such as the E-cube , exist. However, E-cube does not provide any flexibility in choosing a path from a source to its destination and can thus result in long latencies under heavy or uneven traffic. Adaptive, minimum-distance routing algorithms, such as the Turn Model and the UP Preference algorithms, have previously been reported. In this article, we present a new class of adaptive, provably deadlock-free, minimum-distance routing algorithms. We prove that the algorithms developed here are optimally adaptive in the sense that any further flexibility in communication will result in deadlock. We show that the Turn Model is actually a member of our new class of algorithms that does not perform as well as other algorithms within the new class. It creates artificial hotspots in routing the traffic and allows fewer total paths. We present an analytical comparison of the flexibility and balance in routing provided by various algorithms and a comparison based on uniform and nonuniform traffic simulations. The Extended UP Preference algorithm developed in this article is shown to have improved performance with respect to existing algorithms. The methodology and the algorithms developed here can be used to develop routing for other schemes such as wormhole routing, and for other recursively defined networks such as k -ary n -cubes.

This paper explores parallel logic simulation on a network of workstations using a parallel virtual machine (PVM). A novel parallel implementation of the centralized-time event-driven logic simulation algorithm is carried out such that no global controlling workstation is needed to synchronize the advance of simulation time. Further advantages of our new approach include a random partitioning of the circuit onto available workstations and a pipelined execution of the different phases of the simulation algorithm. To achieve a better load balance, we employ a semioptimistic scheme for gate evaluations (in conjunction with a centralized-time algorithm) such that no rollback is required. The performance of this implementation has been evaluated using the ISCAS benchmark circuits. Speedups improve with the size of the circuit and the activity level in the circuit. Analyses of the communication overhead show that the techniques developed here will yield even higher gains as newer networking technologies like ATM are employed to connect workstations.

Sparse matrix solution (SOLVE) is a dominant part of the total execution time in a circuit simulation program such as SPICE. For simulation of modern VLSI circuit designs, it is important that this time be reduced. This paper presents a block partitionable sparse matrix solution algorithm in which a matrix is divided into equal size blocks, and blocks are assigned to different processors for parallel execution. The algorithm developed in this work exploits sparsity at the block level as well as within a non-zero block. An efficient mapping scheme to assign matrix blocks to processors is developed which maximizes concurrency and minimizes communication between processors. Associated reordering and efficient sparse storage schemes are also developed. An implementation of this parallel algorithm is carried out on a Transputer processor array which plugs into a PC bus. The sparse matrix solver is tested on matrices generated from a transistor-level expansion of ISCAS-85 benchmark logic circuits. Good speedup is obtained for all benchmark matrices up to the number of Transputers available.

An optimum application-specific data flow architecture for accelerating the standard event driven logic simulation is developed. A conservative distributed simulation algorithm which minimizes the use of NULL messages is also developed, along with a pseudodynamic data flow architecture to implement this distributed algorithm efficiently. A comparison of the standard event driven algorithm-based data flow accelerator to the distributed simulation algorithm-based accelerator is made on several benchmark circuits. The distributed simulation algorithm on the specialized data flow accelerator outperforms the standard event driven algorithm based data flow accelerator by a factor of three in most cases.>