Depression is a serious mental health illness that significantly affects an individual’s well-being and quality of life, making early detection crucial for adequate care and treatment. Detecting depression is often difficult, as it is based primarily on subjective evaluations during clinical interviews. Hence, the early diagnosis of depression, thanks to the content of social networks, has become a prominent research area. The extensive and diverse nature of user-generated information poses a significant challenge, limiting the accurate extraction of relevant temporal information and the effective fusion of data across multiple modalities. This paper introduces MMF-former, a multimodal depression detection network designed to retrieve depressive spatio-temporal high-level patterns from multimodal social media information. The transformer network with residual connections captures spatial features from videos, and a transformer encoder is exploited to design important temporal dynamics in audio. Moreover, the fusion architecture fused the extracted features through late and intermediate fusion strategies to find out the most relevant intermodal correlations among them. Finally, the proposed network is assessed on two large-scale depression detection datasets, and the results clearly reveal that it surpasses existing state-of-the-art approaches, improving the F1-Score by 13.92% for D-Vlog dataset and 7.74% for LMVD dataset. The code is made available publicly at https://github.com/rezwanh001/Large-Scale-Multimodal-Depression-Detection.

Depression is a major mental health condition that severely impacts the emotional and physical well-being of individuals. The simple nature of data collection from social media platforms has attracted significant interest in properly utilizing this information for mental health research. A Multimodal Depression Detection Network (MDD-Net), utilizing acoustic and visual data obtained from social media networks, is proposed in this work where mutual transformers are exploited to efficiently extract and fuse multimodal features for efficient depression detection. The MDD-Net consists of four core modules: an acoustic feature extraction module for retrieving relevant acoustic attributes, a visual feature extraction module for extracting significant high-level patterns, a mutual transformer for computing the correlations among the generated features and fusing these features from multiple modalities, and a detection layer for detecting depression using the fused feature representations. The extensive experiments are performed using the multimodal D-Vlog dataset, and the findings reveal that the developed multimodal depression detection network surpasses the state-of-the-art by up to 17.37% for F1-Score, demonstrating the greater performance of the proposed system. The source code is accessible at https://github.com/rezwanh001/Multimodal-Depression-Detection.

Fresh produce (FP) yield forecasting is crucial for both: farmers to estimate fair prices for their crops and retailers to protect against highly priced FPs. To precisely forecast future yield, a deep learning forecasting model is proposed in this work which is trained and tested using relevant input parameters retrieved from satellite images and mapped to tabular yield data recorded for strawberry as an output parameter. To enhance the model performance, the preprocessing approaches and the set of input parameters are improved. The best satellite image preprocessing technique has to be found to represent the images with less data for efficiency. Therefore, a preprocessing approach based on averaging is proposed and implemented then compared with the literature approach which is based on histograms, where the proposed approach improved performance by 20%. The proposed Deep Feed Forward Neural Network with Embedded Gated Recurrent Units (DFNNGRU) ensembled with Attention Deep GRUs (ADGRU) is then tested against well-performing models of Stacked-AutoEncoder (SAE) ensembled with Convolution Neural Networks with Long-short term memory (CNNLSTM), where the proposed model is found to outperform the literature model by 12.5%. To have a better set of parameters, a Normalized Vegetation Difference Index (NDVI) is added to the input parameters which further enhances the performance by 2%. Finally, a quality assurance technique using metamorphic testing is applied and it is found that the model fulfills all expected metamorphic relations which proves the soundness and quality of the model as compared with other solutions.

Accurate estimates of fresh produce (FP) yields and prices are crucial for having fair bidding prices by retailers along with informed asking prices by farmers, leading to the best prices for customers. To have accurate estimates, the state-of-the-art deep learning (DL) models for forecasting FP yields and prices are improved in this work while a novel transfer learning (TL) framework is proposed for better generalizability. The proposed models are trained and tested using real world datasets for the Santa Barbara region in California, which contain environmental input parameters mapped to FP yield and price output parameters. Based on an aggregated measure (AGM), the proposed model, an ensemble of Attention Deep Feedforward Neural Network with Gated Recurrent Unit (GRU) units and Deep Feedforward Neural Network with embedded GRU units, is found to significantly outperform the state-of-the-art models. Beside finding the best DL, the TL framework is utilizing FP similarity, clustering, and TL techniques customized to fit the problem in hand and enhance the model generalization to other FPs. The literature similarity algorithms are improved by considering the time series features rather than the absolute values of their points. In addition, the FPs are clustered using a hierarchical clustering technique utilizing the complete linkage of a dendrogram to automate the process of finding the similarity thresholds and avoid setting them arbitrarily. Finally, the transfer learning is applied by freezing some layers of the proposed ensemble model and fine-tuning the rest leading to significant improvement in AGM compared to the best literature model.

The COVID-19 rapid antigen self-test kits are widely administered in several countries to increase the testing frequency and reduce the load on clinics for in-person tests. Yet, the telehealth worker supervision is mandatory to ensure proper sampling procedure is followed and high-quality swab samples are taken. To reduce the load on the health workers in telehealth, we propose a system that eliminates the need for any human supervision by guiding the testers throughout the self-test to ensure the collection of high-quality swab samples. The proposed system takes a live video stream of the frontal face of a user as input and provides real-time instructions to do the self-test correctly with corrective actions when detecting wrong steps. This is mainly done using a collection of deep learning (DL) models. The system uses a novel swab position classification model, Small-MobileNetV2 with Depth-Wise Attention (S-MBNV2-DWAtt), to detect whether a swab is in one of the nostrils or not, which is an optimized version of MobileNetV2 in terms of parameter count and inference speed. The depth-wise attention block allows it to focus on specific parts of the images where the swabs would possibly lie. Lastly, a large-scale synthetic dataset is created to increase the generalization to a variety of swabs and users and a small real dataset is collected to finetune the model on scenes that are similar to the deployment scenarios. The proposed swab position classification model is found to have outstanding performance in terms of both accuracy and speed; it outperforms the ResNet and VGG architectures by 22.83% and 35.11% respectively on a real-world test set while operating at 25 FPS on CPU.

Developing fresh produce yield forecasting service is essential for estimating fair prices to protect against overpriced agricultural commodities and minimize the bid ask spread which not only benefits the retailers and customers but also protects farmers. Forecasting the fresh produce yield is achieved using state of the art deep learning (DL) models. Those models are trained and built using data retrieved from Santa Barbara region in California using an ensemble of Attention Deep Feedforward Neural Network with Gated Recurrent Units (GRU) and Deep Feedforward Neural Network with embedded GRU units. The ensemble takes as input the soil moisture and temperature parameters as well as vegetation indices (VIs) calculated from images retrieved from multiple satellites. The effect of adding the VIs as input parameters on the forecasting performance of the deep learning model is assessed and the most effective VIs are selected. In addition, interpolation techniques are used to estimate the missing VIs due to the low frequency of capturing the images by the satellites. A comparative analysis is conducted to choose the most effective technique, which is found to be Cubic Spline interpolation. One VI, which is the Normalized Difference Vegetation Index (NDVI), proves to be the most effective index in forecasting the yield. Based on the aggregated error measure (AGM) score, the yield forecasting performance of the DL ensemble is enhanced by 12.51% after adding the complete interpolated NDVI to the input parameters used in training the model.

To overcome the computational complexity of retraining Deep Learning (DL) yield forecasting models for each type of Fresh Produce (FP), it is necessary to have a generalization of the models' application to similar FP. This can be done by transferring the learning among similar FP with minimal retraining. Hence, Transfer Learning (TL) is used in this work amongst berries which are similar in nature. First, the proposed DL model is trained using station-based data and satellite images as inputs mapped to the strawberry yield as output. The weights obtained from this learning are transferred to the raspberry yield forecasting model since raspberry and strawberry yields are similar and are both planted in California. The proposed model is an ensemble of two models: the station-based ensemble model (ATT-CNN-LSTM-SeriesNet_Ens) with its compound DL components, SeriesNet with Gated Recurrent Unit (GRU) and Convolutional Neural Network LSTM with Attention layer (Att-CNN-LSTM); trained and tested using station-based data as input and the corresponding strawberry yields as output. Second, the remote sensing ensemble model (SIM_CNN-LSTM_Ens), which is an ensemble of Convolutional Neural Network LSTM (CNN-LSTM) models; trained and tested using satellite images as input mapped to the same yields as output. The weights obtained are transferred to the raspberry yield forecasting ensemble model with minimal retraining. It is found that the voting ensemble improves performance by 27% compared to the best performing component model. Based on an aggregated measure, the performance obtained from TL is comparable to that obtained by training the models on the raspberry data without TL, while having around 55 % reduction in processing time.

The ability to predict crop yield is vital for food security worldwide and forecasting crop prices can help farmers avoid price crash. In this work, an investigation of using satellite images and deep learning models to predict crop yields as well as forecasting farmers' prices is conducted. For tractability, dimensionality reduction is achieved by converting the images to histograms representing the pixel frequency. The models tested are LSTM, CNN, CNN-LSTM, CNN-LSTM ensemble as well as a Gaussian Process added to each for enhanced performance. It is found that the proposed ensemble of CNN-LSTMs is the best in predicting the yearly soybean yields in addition to forecasting the daily strawberry yields and prices. It outperforms models suggested in the literature with an improvement of 31% in terms of average Root Mean Square Error (RMSE).

It is common for a time series dataset to have missing values, and it is necessary to fill these missing elements before using the dataset for training forecasting models. Usually this problem is tackled using non-machine learning methods that introduce bias into the system which results in unreliable forecasting results. Moreover, most of the work found in the literature tackles imputation of missing values when they are randomly scattered in the dataset while very little work is found tackling the case of consecutive occurrence of missing data; i.e. missing data chunks in the dataset. Therefore, in this work, comprehensive imputation models are developed to impute both random as well as chunks of missing values. Alongside, a framework is found enabling the user to impute any time series data with the optimal models. In order to carry out the task, one non-deep machine learning model (Bidirectional Imputation model) and three deep learning (DL) imputation models (Ensemble model, Transfer Learning model and Hybrid model), are tested using complete time series. The results show that the hybrid model yields a maximum of 38% improvement in the Aggregate Error (AGE) when compared with other models.

Procuring apple yield prior to harvest is essential since it helps in estimating the apple production and prices. A compound Deep Learning (DL) model, SeriesNet with Gated Recurrent Unit (GRU) and Attention (Att-SeriesNet-GRU), is used in this work to predict the apple yield for 15 counties across 6 different Crop Reporting Districts (CRD) in California. The DL model is trained using static soil parameters, which remain constant over years per county and dynamic parameters, which change daily or monthly for a specific county as input, and the corresponding annual apple yield for that county as output. If the training is done based on a single county data then the static parameters won’t add information to the DL model since they remain constant over years per county. Therefore, considering different counties across California is decided to study the effect of considering the static soil parameters along with the dynamic ones. The county level annual apple yield forecast using both static and dynamic parameters together gives promising results. Experimenting with the test set as input shows that adding the static parameters together with the dynamic ones gives an improvement of around 34% in the value of Aggregated Measure (AGM) over the case of using the dynamic parameters alone for yield forecasting. It is also found that training the DL model with augmented training set improves the AGM value by around 12%.

Market price and yield forecasting models for Fresh Produce (FP) are crucial to protect retailers and consumers from overpriced FP. However, utilizing the data for forecasting is obstructed by the occurrence of missing values. Therefore, it is imperative to impute the encountered missing instances to enable effective forecasting. Most of the work found in literature tackles imputation of missing values when they are randomly scattered in the dataset while very little work is found tackling both: consecutive occurrence of missing data, i.e. missing data chunks, as well as those randomly missing. In this work, the data used for forecasting has missing values in chunks as well as at random points. Therefore, various comprehensive imputation models are used to impute both random as well as chunks of missing values. Since the imputed time series are incomplete, the only way to evaluate those imputation models is to analyze their effect on forecasting performance. The ensemble of two compound deep learning (DL) models, namely Attention Convolutional Neural Networks Long Short Term Memory (Att-CNN-LSTM) and SeriesNet with Gated Recurrent Unit (GRU), is used for forecasting. For imputation, three DL models are tested: The Ensemble imputation model which is a Voting Regressor of two DL submodels, Residual GRU and LSTM-Deep-GRU. Another deep learning imputation model is used which is a Transfer Learning (TL) model. Finally, a Hybrid model of both DL models is designed to take the pros of each of its integrated models by using the Ensemble model in case of random missing data and the Transfer Learning model in case of missing data chunks. It is observed that, in general, imputing the missing values improves the forecasting result as compared to eliminating the instances with missing values. The Hybrid model improves the overall forecasting performance by up to 60% compared to the case of using the second-best Transfer Learning model and around 64% as compared to the case of imputation using the Ensemble model.

Forecasting crop yields and prices is crucial for both global food security and providing farmers with valuable information to avoid a price crash. This work proposes a hybrid deep learning model that uses satellite images to forecast strawberry yield along with farmers’ prices, applied in three counties in California. For tractability, a dimensionality reduction technique is applied by converting the images to histograms representing the pixel frequency. The models tested are Convolutional Neural Network (CNN), Variational AutoEncoder (VAE), CNN-Long Short-Term Memory (CNN-LSTM), Stacked AutoEncoder (SAE), and a voting ensemble of CNN-LSTM and SAE. It is found that the proposed voting ensemble of CNN-LSTM and SAE is the best at forecasting the daily strawberry yields and prices in all three counties. Based on an aggregated performance measure (AGM), the voting ensemble model outperforms the models suggested in literature with up to 70% forecasting improvement compared to the CNN model and up to 22% improvement over the CNN-LSTM model.

Searching similarity in time series (TS) datasets has gained widespread attention lately in databases classification and forecast domain. In this study, a TS similarity detection framework is proposed to explore alike-behavior fresh produce (FP) in the yield forecast domain through several factors. The sequential daily yield datasets of three types of FP, including strawberry, raspberry, and blueberry, as well as environmental information related to the Santa Maria region, California, between the years 2011 to 2019, are used to develop and evaluate the models. The framework's output is decided to be the similarity percentage (SP) by considering some thresholds that have been tuned using several synthetic yield datasets. According to the results, the SP is 82% and 52% for strawberry versus raspberry and strawberry versus blueberry, respectively. This indicates the fact that strawberry and raspberry have a relatively similar yield pattern compared to blueberry, which is a considerable matter in generalizing forecast models.

Building highly precise prediction models for Fresh Produce (FP) market price is crucial to protect retailers from overpriced FP. In this paper we are comparing the price prediction models performance of deep learning (DL) models with statistical as well as standard machine learning (ML) models. Five types of FP are considered in performance testing. It is found that the conventional ML models outperform the statistical models such as ARIMA. On the other hand, the winning model among the conventional ML models (the Gradient Boosting model) proves to be less performant as compared with the simple or compound DL models. Moreover, the simple DL models, such as the Long Short-Term Memory (LSTM), are outperformed by the compound one, the Convolutional Long Short-Term Memory Recurrent Neural Network (CNN-LSTM), whose performance improves by adding attention. The model is capable of precisely predicting FP prices for up to three weeks ahead.

The currently deployed prediction models for strawberry fresh produce (FP) are based on either conventional machine learning (ML) or on simple deep learning (DL) models that are mostly applied for yield prediction. In this paper, we propose more comprehensive DL models that are applied for the first time to predict strawberry yield. The strawberry price is predicted as well directly from weather input parameters and yield. The strawberry price prediction is achieved using compound DL models such as Convolutional Long Short-Term Memory Recurrent Neural Network (CNN-LSTM). It is found that by adding attention, the performance of the compound models usually improves. After utilizing an aggregated performance measure to find the best model, the Attention-CNN-LSTM model proved to be the best compared to the rest of the deployed conventional ML models as well as the compound and simple DL models. The aggregated measure shows that this model is capable of precisely predicting strawberry prices five weeks ahead while maintaining the lowest prediction error and the highest model correlation.

Adequately priced orders and time for fresh produce (FP) are two factors that bring commercial benefits to vendors and minimizes waste. However, many factors, such as income, labor, and other trade issues, affect the price that include uncertainties due to climate change, making decisions on FP procurement prices and quantities extremely challenging. Two artificial intelligence-based forecasting tools, i.e., a single variate and a multivariate model, are trained, tested, and compared in this study to predict future daily offer prices up to 7 days ahead for strawberries using mutual transactions for the distribution centers of Loblaws Companies Limited (LCL) in Canada. Results reveal that the developed multivariate model, utilizing both prices of the LCL dataset and California's strawberries yield dataset as predictors, outperforms the best single variate model.

The importance of imputation for having highly performing prediction models is highlighted in this work. Three imputation techniques are tested against a non-imputation approach that discards records with any missing values; the complete-case analysis (CCA). The deep learning linear memory vector recurrent neural network-RNN (LIME) imputation model is tested along with two other nondeep learning models such as the linear function and Last Observation Carried Forward (LOCF). The simple LSTM deep learning (DL) prediction model is deployed to decide the best performing imputation model, the one resulting in the lowest price and yield prediction errors. Five performance evaluation measures are utilized; the mean absolute error (MAE), the root mean square error (RMSE), R2correlation measure along with two aggregated measures summarizing these three measures to decide the overall prediction performance; the average aggregated measure (AGM) for each considered step ahead and the average of the AGM across all considered steps ahead (AAGM). Based on AGM, it is found that the LIME imputation model leads to the best prediction performance of the simple LSTM DL model across both applications of 5 weeks ahead strawberry price and yield predictions using weather; W2P and W2Y. Therefore, the LIME imputed file is reused to train two compound DL models, Convolutional Long Short-Term Memory RNN with attention (ATT-ConvLSTM) and ATT-CNN-LSTM along with their Voting Regressor ensemble (VR). The same models are retrained with files preprocessed with the non-imputation approach, CCA. It is found that the overall AAGM of the compound DL and ensemble prediction models across all the 1, 2, 3, and 4 weeks ahead price predictions confirm that using LIME highly improves the prediction performance of the ensemble and its compound DL components. The VR ensemble price prediction performance is improved by 72% and the ATTConvLSTM component is improved by 89% compared to their performances without imputation; using CCA preprocessed files.

Time series prediction has been challenging topic in several application domains. In this paper, an ensemble of two top performing deep learning architectures across different applications such as fresh produce (FP) yield prediction, FP price prediction and crude oil price prediction is proposed. First, the input data is trained on an array of different machine learning architectures, the top two performers are then combined using a stacking ensemble. The top two performers across the three tested applications are found to be Attention CNN-LSTM (AC-LSTM) and Attention ConvLSTM (ACV-LSTM). Different ensemble techniques, mean prediction, Linear Regression (LR) and Support vector Regression (SVR), are then utilized to come up with the best prediction. An aggregated measure that combines the results of mean absolute error (MAE), mean squared error (MSE) and R2coefficient of determination (R2) is used to evaluate model performance. The experiment results show that across the various examined applications, the proposed model which is a stacking ensemble of the AC-LSTM and ACV-LSTM using a linear SVR is the best performing based on the aggregated measure.

Missing data are commonly found in time series datasets. These missing elements are usually a hurdle in utilizing the datasets in prediction or forecasting, making imputation of those missing values imperative. Due to the non-linear dependencies between the current and previous values, imputation remains a challenging task. Conventional methods such as averaging, deletion or filling with the last observed value add bias to the data and are therefore inefficient. Since different time series showcase varying characteristics, figuring out which imputation method works best for the respective time series is essential. In this work, seven different deep learning (DL) imputation methods are examined along with three machine learning (ML) ensembles. To enable a recommendation of the best imputation method for each time series type, the imputation models are tested using the four main types of time series: trend (T), seasonal (S), combined trend and seasonal time series (T&S) and random (R) time series. Results indicate that the Gated Recurrent Unit (GRU) neural networks are, in general, the best for missing values imputation with varying complexity based on the time series type. For example, it is found that the residual GRU is recommended for the trend and seasonal time series while the GRU is recommended for the combined type. Conversely, all tested DL imputation models can be used with the random time series type. In addition, the considered ML ensembles do not perform as high as the DL models with all tested types of times series.

For VANET safety services a context aware system for the Automatic Crash Notification (ACN) is developed while the context aware Congested Road Notification system (CRN) is developed for the convenience services. A simple fuzzy logic model is proposed and compared to different severity estimation models deployed for both systems. The performance of the ACN models is compared using a test collection that is based on nineteen years of real life crash records associated with their severity levels while the performance of the CRN models is tested using nearly 500,000 different urban and rural freeways flow situations associated with their congestion severity levels. The non-binary Spearman correlation coefficient and the Average Distance Measure (ADM) are used to evaluate the performance of the tested models. Results show that the simple fuzzy severity estimation model has a comparable performance to more complicated systems such as the CoTEC (CoOperative Traffic congestion detECtion) fuzzy system and the URGENCY algorithm, and outperforms the binary severity estimation models for the ACN and CRN systems.