Osteoarthritis (OA) is a highly frequent musculoskeletal condition defined by the progressive degradation of joint cartilage and the underlying bone, resulting in the manifestation of pain and functional limitations. Identification of symptoms as early as possible for timely intervention is critical for effective pain management and treatment. Knee and hip OA are common in older patients which can greatly affect their mobility, lifestyle, and lead to other health complications. Electronic Medical Records (EMR) in primary care settings contain patients’ structured historical data including unstructured text data in the encounter chart notes. The unstructured notes are often very long, compiled from multiple patient-physician encounters, and contain medical jargon including personal data. The data offers a variety of computational challenges but it contains valuable information for disease diagnosis especially for detecting hip or knee OA. We demonstrate multiple keyword-based strategies to detect the OA-affected bone joints, including a simple rule-based approach and a machine learning based approach. We also provide an ablation study to show the effects of the different natural language text processing methods and validate our results against gold standard data labelled by a human expert. Our Random Forest (RF) model achieved the best result of 74.89% F1-score with OA related paragraph extraction.

In various applications, the multivariate time series often suffers from missing data. This issue can significantly disrupt systems that rely on the data. Spatial and temporal dependencies can be leveraged to impute the missing samples. Existing imputation methods often ignore dynamic changes in spatial dependencies. We propose a Spatial Dynamic Aware Graph Recurrent Imputation Network (SDA-GRIN) which is capable of capturing dynamic changes in spatial dependencies. SDA-GRIN leverages a multi-head attention mechanism to adapt graph structures with time. SDA-GRIN models multivariate time series as a sequence of temporal graphs and uses a recurrent message-passing architecture for imputation. We evaluate SDA-GRIN on four real-world datasets: SDA-GRIN reduces MSE by 9.51% for the AQI and 9.40% for AQI-36. On the PEMS-BAY dataset, it achieves a 1.94% reduction in MSE. Detailed ablation study demonstrates the effect of window sizes and missing data on the performance of the method. Project page: https://ameskandari.github.io/sda-grin/.

Voice-enabled conversation bots powered by Large Language Models (LLMs) offer promising opportunities to enhance communication accessibility and companionship for older adults. However, current systems often overlook age-related speech characteristics and interaction preferences. This study presents an adaptive conversation bot system that adjusts voice synthesis based on user-specific speech features such as pitch and speech rate. To test the hypothesis that speech-based voice adaptation improves usability and user experience, we develop predictive models using the Mozilla Common Voice Delta Segment 19.0 dataset. We also benchmark three compact LLMs, Mistral 7B, Llama 2 7B, and Llama 3.1 8B, based on latency, throughput, and memory usage to identify a model suitable for real-time interaction. Given the sensitivity of the target population, we conducted a pilot user study with younger adult participants, comparing a baseline and an adapted version of the conversation bot across multiple usability criteria. Results of user evaluation showed that the adaptive version improved clarity, comfort, and naturalness, supporting the effectiveness of the proposed approach for voice-enabled conversation systems.

Efficient big data management is a dire necessity to manage the exponential growth in data generated by digital information systems to produce usable knowledge. Structured databases, data lakes, and warehouses have each provided a solution with varying degrees of success. However, a new and superior solution, the data Lakehouse, has emerged to extract actionable insights from unstructured data ingested from distributed sources. By combining the strengths of data warehouses and data lakes, the data Lakehouse can process and merge data quickly while ingesting and storing high-speed unstructured data with post-storage transformation and analytics capabilities. The Lakehouse architecture offers the necessary features for optimal functionality and has gained significant attention in the big data management research community. In this paper, we compare data lake, warehouse, and lakehouse systems, highlight their strengths and shortcomings, identify the desired features to handle the evolving challenges in big data management and analysis and propose an advanced data Lakehouse architecture. We also demonstrate the performance of three state-of-the-art data management systems namely HDFS data lake, Hive data warehouse, and Delta lakehouse in managing data for analytical query responses through an experimental study.

With the increasing elderly population, healthcare systems are facing a crisis in providing 24/7 assistance to seniors. Voice-enabled robots can serve as assistants and companions for seniors. Many voice and chat systems, such as Alexa, Siri, and Google, have emerged and become the top consumer brands. However, a significant gap still exists in making these systems mimic human-like conversation. Most of these systems lack empathy and cannot understand users' expressions and intent in delivering information as expected by users. Studies have shown that seniors struggle to effectively communicate with the existing voice conversation systems, leading to a stigma around technology acceptance. In this pilot project, we explore existing literature to identify the key challenges and the types of care that seniors need, and present a personalized voice communication system that can remind about tasks such as taking medication and doing health checks; answer predefined questions about personalized routines and medications; launch specific applications such as the contactless wellness monitoring application “Veyetals”, “Uber” or a music plugin, and answer basic questions from a pre-defined Frequently Asked Questions (FAQ) database. We extend existing voice bots to develop the Assist-Bot to serve as a voice-enabled assistant for nontechnical seniors. Caregivers can configure medication and check-up schedules, and contact numbers for different needs.

During the COVID-19 pandemic, the traditional emergency healthcare systems faced unprecedented strain due to the sharp rise in demands for urgent care, scarcity of resources, and increased risks of people getting infected while waiting at the emergency care facility. We present Triage-Bot, an online medical triage provisioning service, that can revolutionize emergency care by decreasing the load on emergency departments (ED), reducing healthcare expenses, and improving the quality of care. Empowered by artificial intelligence and natural language processing, the Triage-Bot service assesses and prioritizes patients' needs based on symptoms, medical history, and perceived conditions from multimodal video, audio, and text data captured during patients' interactions. The captured summarized information with a severity ranking is sent to a human expert to suggest the next action on the user's part. The diverse data types used by the Triage-Bot in communication, authentication, data collection, storage, and analytics requires a robust and scalable system architecture for online service provisioning. In this paper, we specifically focus on the system design and architecture of the Triage-Bot for emergency healthcare settings. With integrated electronic medical records (EMR) and online platforms, the bot fosters collaboration among healthcare professionals and enables swift and informed decision-making even in the face of crises. By partially automating and offering a hybrid triage process, the Triage-Bot improves resource allocation, reduces healthcare management costs for emergency care, minimizes patient waiting times, and improves wellbeing. To address the complexities and demands of healthcare data management, our proposed system incorporates MongoDB database for flexibility, scalability, and versatility in supporting different types of data. Additionally, we implement a data linking and analytics pipeline utilizing a data Lakehouse system to effectively ingest, manage, process, and generate knowledge from heterogeneous data sources.

With the advancements in digital media, a large number of news items get posted on various social media platforms every minute, which has a significant impact on society. Fake news and hate speech are pervading all social media platforms. With the emergence of neural networks and their promising performance in automation, multiple methods have been introduced to discriminate between fake and real news. One of these methods focuses on the propagation pattern of news in social media since fake news and real news spread differently. Graph Neural Networks (GNNs) can efficiently model linked entities and information propagation among the entities. In this study, we examine multiple graph neural networks (GNNs) to assess their effectiveness in predicting fake news items based on the news propagation pattern. We implement and explore the complexity of five different GNNs namely the Graph Convolution Neural Network (BiGCN-A, BiGCN-B), Graph Attention Neural Network (BiGAT), GraphSage (BiSAGE), Graph Convolution with ARMA filters (BiARMA), and Simplified Graph Convolution Neural Network (BiSGCN). We identify networks with reduced complexity and high efficiency for early detection of fake news at a lower computational cost in our study. The experiments show that BiSAGE performs the best with 94% accuracy in 980 seconds for Twitter16 dataset which is comparable to the BiGCN-B which reported 93% accuracy. Additionally, BiGCN-A performs the best with 94 % accuracy in 896 seconds for Twitter 15 dataset.

Modern adoption of network-connected devices have led to an abundance of high speed data streams and a call for the development of analytical models capable of processing them. Much of the real world data is unlabeled which requires development and application of unsupervised learning algorithms for advanced machine perception. However, the datasets that are used to train and validate these algorithms are often synthetic or simulated for the study and too simple, which leads to an inflation of reported performance. Based on the recent literature, there is a lack in research on developing deep learning models for clustering highly complex Human Activity Recognition (HAR) data streams. We, therefore, explore the area and apply three modern high-performing stream clustering algorithms on a complex HAR data stream. We evaluate the ability of these models to adapt and extract the volatile knowledge that is contained within the HAR data streams. Our goal is to set a baseline of performance for clustering timestamped HAR data streams. The results show that despite the reported performance of these algorithms in the literature, they lack the analytical depth to effectively extract sufficient knowledge from the complex HAR streams to construct viable clusters. So, new thoughts are needed to develop models that can efficiently handle HAR data streams.

Cooperative perception allows a Connected Autonomous Vehicle (CAV) to interact with the other CAVs in the vicinity to enhance perception of surrounding objects to increase safety and reliability. It can compensate for the limitations of the conventional vehicular perception such as blind spots, low resolution, and weather effects. An effective feature fusion model for the intermediate fusion methods of cooperative perception can improve feature selection and information aggregation to further enhance the perception accuracy. We propose adaptive feature fusion models with trainable feature selection modules. One of our proposed models Spatial-wise Adaptive feature Fusion (S-AdaFusion) outperforms all other State-of-the-Arts (SO-TAs) on two subsets of the OPV2V dataset: Default CARLA Towns for vehicle detection and the Culver City for domain adaptation. In addition, previous studies have only tested cooperative perception for vehicle detection. A pedestrian, however, is much more likely to be seriously injured in a traffic accident. We evaluate the performance of cooperative perception for both vehicle and pedestrian detection using the CODD dataset. Our architecture achieves higher Average Precision (AP) than other existing models for both vehicle and pedestrian detection on the CODD dataset. The experiments demonstrate that cooperative perception also improves the pedestrian detection accuracy compared to the conventional single vehicle perception process.

Machine learning and artificial intelligence can harness the immense stream of operational data from clouds, to services, to social and communication networks. In the era of big data and connected devices of all varieties, machine learning and artificial intelligence have found ways to improve operations and management of information technology and communications.

Digital information systems currently generate a vast amount of data every minute which emphasizes the continuing need to advance big data management systems with efficient data ingestion and knowledge extraction capabilities. To address the ‘big data’ problems due to high volume, velocity, variety, and veracity, data management systems evolved from structured databases to big data storage systems, graph databases, data warehouses, and data lakes but each solution has its strengths and shortcomings. The need to produce actionable knowledge fast from unstructured data ingested from distributed sources requires a marriage of data warehouses and data lakes to create a data Lakehouse (LH). The objective is to use the strengths of the data warehouse in producing insights fast from processed merged data, and of the data lake in ingesting and storing high-speed unstructured data with post-storage transformation and analytics capabilities. In this paper, we present a comparative review of the existing data warehouse and data lake technology to highlight their strengths and weaknesses and propose the desired and necessary features of the LH architecture, which has recently gained a lot of attention in the big data management research community.

Machine learning and artificial intelligence can harness the immense stream of operational data from clouds, to services, to social and communication networks. In the era of big data and connected devices of all varieties, machine learning and artificial intelligence have found ways to improve operations and management of information technology and communications.

Methods of estimating heart rate without the use of sensor devices provides essential benefits in both the medical field as well as the other computing applications. Smartphones are the handiest devices available to everyone today. By using videos of fingertip captured with smartphone camera, heart rate (HR) can be estimated using the photoplethysmography (PPG) technique. It is based on tracking subtle color changes on the skin owing to cardiovascular activities. These color changes are invisible to the human eye but can be detected by digital cameras. The method is divided into three main steps: first, reading the video frames and processing them to obtain the PPG data, next, extracting the Blood Volume Pulse (BVP) signal, and finally, estimating the HR from the signal. In this project, the color intensity of the skin pixels is used, and filters are applied to eliminate the noise and retain only the pulses of interest. The extracted signal is fed into a convolutional regression neural network which outputs the estimated HR. The results obtained are compared with the ground truth HR obtained by using a contact PPG sensor. We obtained a Mean Absolute Error (MAE) of 7.01 beats per minute (bpm) and an error percentage of 8.3% on test data.

Due to the intensive treatment process of coronavirus pneumonia cases, it is important to predict the Length of Stay (LOS) of patients at the hospital to allow better management of resources and increase the efficiency of hospital services to provide improved healthcare. To predict LOS, we used four artificial neural network models namely the Multilayer Perceptron (MLP), Convolutional Neural Network (CNN), Multilayer Perceptron with PCA (PCA+MLP), and the Bidirectional Long Short Term Memory (BiLSTM) model to analyze the advantages and disadvantages of the different models using the Microsoft Hospital Length of Stay data. The proposed method is compared with the state-of-the-art models and a simple MLP model. Our models achieved an accuracy between 73% and 88% with the CNN model providing the highest accuracy.

During recent years, Question Answering (QA) systems have been widely used in many industries to provide round the clock online services to consumers from all over the world. The importance of such services became more evident during the pandemic in online medical services, education, training, marketing, system support and administration. Most of the existing systems apply simple rule-based QA strategy. Human-defined rules are used to apply pattern matching for extracting information from a given data or knowledge base to generate responses to user queries. However, rule-based pattern matching techniques are not intelligent enough to understand the context of the question to always generate appropriate responses and are static. In this work, we explored different data preprocessing strategies and BERT-style pre-trained models to build an information retrieval (IR)-based Domain specific QA framework named BIRD-QA, and created a domain specific knowledge base using website data of a university department. We implemented multiple variations of extended BERT and ALBERT-base models and validated our framework on reading comprehension task using the Stanford Question Answering Dataset (SQuAD) 1.1 and 2.0 datasets. Our extended ALBERT-based model achieved 75.4% Exact Match (EM) score and 78.8% F1 score. We also present a small feasibility test of our framework for departmental QA using data from a university website.

Drivable area or free space detection is an important task in Advanced Driver-Assistance Systems (ADAS) and autonomous driving system. It can help intelligent vehicles understand road conditions and determine safe driving area. Semantic segmentation is a pixel-wise prediction which can classify each pixel into its category. In this paper, we propose a deep learning-based semantic segmentation architecture to predict the drivable area in front of the vehicle. Our model is built based on ResNet backbone with the Feature Pyramid Network (FPN) and Atrous Spatial Pyramid Pooling (ASPP) modules. The backbone in the bottom-up architecture extracts features and an ASPP is attached to the last decoder layer. Additionally, a top-down architecture with lateral connections is added in the decoder and the FPN utilizes the multi-scale features for final prediction. Our model is evaluated on the Cityscapes street scene dataset and achieves 95.90% mIoU on road segmentation. Next, the model is evaluated on the BDD100K large-scale diverse driving dataset with direct drivable region and alternative drivable region annotations. For this dataset our model achieves 84.58% mIoU which is comparable to some State-of-the-Art models.

Vital signs are important parameters that can reflect people's physiological status and help physicians provide medical advice. Remote Photoplethysmography (rPPG) is a fast, low-cost and convenient method to remotely collect biometric data, and requires only a facial video recorded using a smartphone or other camera. Remote medical service provisioning proved to be a dire need during the COVID-19 pandemic. To leverage the cloud-based medical advice provisioning platform of Your Doctors Online, we propose a rPPG methodology to measure people's Heart Rate (HR) and Heart Rate Variability (HRV) based on a facial video recorded by the users using a smartphone. We validate our model on the TokyoTech remote PPG dataset.

During all trimesters of the pregnancy, measuring the fetal Head Circumference (HC) from ultrasound images can estimate the gestational age, monitor the growth status of the fetus and infer newborn's state. Precise segmentation of fetal ultrasound images can help physicians measure HC efficiently and accurately and make further predictions. In this paper, we leverage deep learning encode-decode architecture to segment the fetal skull boundary and fetal skull for fetal HC measurement. We modify our network based on U-Net due to its outstanding performance in biomedical image analysis. We add dilated convolution layers after the last encoder and Squeeze-and-Excitation (SE) blocks on the skip connections of U-Net to segment fetal skull boundary and fetal skull in 2D ultrasound images. The model is trained and evaluated on the HC18 grand challenge dataset, which has 2D ultrasound images at different trimesters of pregnancy. We achieved 2.27 ± 3.61 mm mean absolute difference in HC measurement. The model also achieved 97.31 ± 1.84% mean Dice score in fetal skull segmentation.

The primary objective of this study was to develop a predictive model that used structured data from the Electronic Medical Records (EMR) to identify Chronic Obstructive Pulmonary Disease (COPD). The symptoms of COPD overlap with many other diseases, therefore it is important to identify a group of COPD symptoms that are frequently documented in the EMR. We explored seven tables from the Manitoba Primary Care Research Network (MaPCReN): patient demographics, health condition, billing, disease case, examinations, medication and risk factors. We applied three supervised machine learning models, a Multilayer Neural Networks (MLNN) model, a Support Vector Machine (SVM), and an Extreme Gradient Boosting (XGB) to identify COPD patients using EMR data. When used to predict COPD, the XGB model achieved an accuracy of 83%, compared to 81% accuracy of the SVM and 80% of the MLNN. Utilizing feature importance, we identified a set of key symptoms for diagnosing COPD within the EMR data namely medications, conditions, risk factors and patient age.

Human Activity Recognition (HAR) is a field that uses collected data to classify different human actions. One simple and general approach to HAR is to use the sensor data from a mobile device to recognize different patterns behind complex motions. Early studies show promising results on simple activities using manually selected features from accelerometer readings. As newer publicly available datasets include more complex data and activities, manual feature selection have become cumbersome, impractical and face limitations in finding the optimal feature sets for HAR. In this paper, we present an empirical approach to defining models of 3D tensor data structures from 2D time series data obtained from multiple sensors on a smart phone, and a new Convolutional Neural Network (CNN) model, which uses the tensor data and performs automatic feature extraction and classification for HAR. We use the public benchmark dataset, MobiAct v2.0, to train and validate our model, which achieved an overall better performance in classifying 11 Activities of Daily Living (ADL) than the state-of-the-art approaches. Compared to the approach presented by Chatzaki et al. which has a very high rate of misclassifications for car-step out (CSO), car-step in (CSI), sit to stand (CHU), and stand to sit (SCH) classes, our proposed approach has 15% higher sensitivity for each of these activities with the optimal number of training epochs being only 25.

The exponential growth of information on the Internet is a big challenge for information retrieval systems towards generating relevant results. Novel approaches are required to reformat or expand user queries to generate a satisfactory response and increase recall and precision. Query expansion (QE) is a technique to broaden users' queries by introducing additional tokens or phrases based on some semantic similarity metrics. The tradeoff is the added computational complexity to find semantically similar words and a possible increase in noise in information retrieval. Despite several research efforts on this topic, QE has not yet been explored enough and more work is needed on similarity matching and composition of query terms with an objective to retrieve a small set of most appropriate responses. QE should be scalable, fast, and robust in handling complex queries with a good response time and noise ceiling. In this paper, we propose Xu, an automated QE technique, using high dimensional clustering of word vectors and Datamuse API, an open source query engine to find semantically similar words. We implemented Xu as a command line tool and evaluated its performances using datasets containing news articles and human-generated QEs. The evaluation results show that Xu was better than Datamuse by achieving about 88% accuracy with reference to the human-generated QE.

The rapid growth of data in velocity, volume, value, variety, and veracity has enabled exciting new opportunities and presented big challenges for businesses of all types. Recently, there has been considerable interest in developing systems for processing continuous data streams with the increasing need for real-time analytics for decision support in the business, healthcare, manufacturing, and security. The analytics of streaming data usually relies on the output of offline analytics on static or archived data. However, businesses and organizations like our industry partner Gnowit, strive to provide their customers with real time market information and continuously look for a unified analytics framework that can integrate both streaming and offline analytics in a seamless fashion to extract knowledge from large volumes of hybrid streaming data. We present our study on designing a multilevel streaming text data analytics framework by comparing leading edge scalable open-source, distributed, and in-memory technologies. We demonstrate the functionality of the framework for a use case of multilevel text analytics using deep learning for language understanding and sentiment analysis including data indexing and query processing. Our framework combines Spark streaming for real time text processing, the Long Short Term Memory (LSTM) deep learning model for higher level sentiment analysis, and other tools for SQL-based analytical processing to provide a scalable solution for multilevel streaming text analytics.

Systems powered by artificial intelligence are being developed to be more user-friendly by communicating with users in a progressively “human-like” conversational way. Chatbots, also known as dialogue systems, interactive conversational agents, or virtual agents are an example of such systems used in a wide variety of applications ranging from customer support in the business domain to companionship in the healthcare sector. It is becoming increasingly important to develop chatbots that can best respond to the personalized needs of their users, so that they can be as helpful to the user as possible in a real human way. This paper investigates and compares three popular existing chatbots API offerings and then propose and develop a voice interactive and multilingual chatbot that can effectively respond to users’ mood, tone, and language using IBM Watson Assistant, Tone Analyzer, and Language Translator. The chatbot was evaluated using a use case that was targeted at responding to users’ needs regarding exam stress based on university students survey data generated using Google Forms. The results of measuring the chatbot effectiveness at analyzing responses regarding exam stress indicate that the chatbot responding appropriately to the user queries regarding how they are feeling about exams 76.5%. The chatbot could also be adapted for use in other application areas such as student info-centers, government kiosks, and mental health support systems.

Fall detection is a major problem in the healthcare department. Elderly people are more prone to fall than others. There are more than 50% of injury-related hospitalizations in people aged over 65. Commercial fall detection devices are expensive and charge a monthly fee for their services. A more affordable and adaptable system is necessary for retirement homes and clinics to build a smart city powered by IoT and artificial intelligence. An effective fall detection system would detect a fall and send an alarm to the appropriate authorities. We propose a framework that uses edge computing where instead of sending data to the cloud, wearable devices send data to a nearby edge device like a laptop or mobile device for real-time analysis. We use cheap wearable sensor devices from MbientLab, an open source streaming engine called Apache Flink for streaming data analytics, and a long short-term memory (LSTM) network model for fall classification. The model is trained using a published dataset called “MobiAct.” Using the trained model, we analyse optimal sampling rates, sensor placement, and multistream data correction. Our edge computing framework can perform real-time streaming data analytics to detect falls with an accuracy of 95.8%.

An essential part of building a data-driven organization is the ability to handle and process continuous streams of data to discover actionable insights. The explosive growth of interconnected devices and the social Web has led to a large volume of data being generated on a continuous basis. Streaming data sources such as stock quotes, credit card transactions, trending news, traffic conditions, time-sensitive patient's data are not only very common but can rapidly depreciate if not processed quickly. The ever-increasing volume and highly irregular nature of data rates pose new challenges to data stream processing systems. One such challenging but important task is how to accurately ingest and integrate data streams from various sources and locations into an analytics platform. These challenges demand new strategies and systems that can offer the desired degree of scalability and robustness in handling failures. This paper investigates the fundamental requirements and the state of the art of existing data stream ingestion systems, propose a scalable and fault-tolerant data stream ingestion and integration framework that can serve as a reusable component across many feeds of structured and unstructured input data in a given platform, and demonstrate the utility of the framework in a real-world data stream processing case study that integrates Apache NiFi and Kafka for processing high velocity news articles from across the globe. The study also identifies best practices and gaps for future research in developing large-scale data stream processing infrastructure.

Prediction of stock markets is a challenging problem because of the number of potential variables as well as unpredictable noise that may contribute to the resultant prices. However, the ability to analyze stock market trends could be invaluable to investors and researchers, and thus has been of continued interest. Numerous statistical and machine learning techniques have been explored for stock analysis and prediction. We present a comparative study of two very promising artificial neural network models namely a Long Short-Term Memory (LSTM) recurrent neural network (RNN) and a deep neural network (DNN) in forecasting the daily and weekly movements of the Indian BSE Sensex index. With both networks, measures were taken to reduce overfitting. Daily predictions of the Tech Mahindra (NSE: TECHM) stock price were made to test the generalizability of the models. Both networks performed well at making daily predictions, and both generalized well to make daily predictions of the Tech Mahindra data. The LSTM RNN outperformed the DNN in terms of weekly predictions and thus, holds more promise for making longer term predictions.

Stock market forecasting is very important in the planning of business activities. Stock price prediction has attracted many researchers in multiple disciplines including computer science, statistics, economics, finance, and operations research. Recent studies have shown that the vast amount of online information in the public domain such as Wikipedia usage pattern, news stories from the mainstream media, and social media discussions can have an observable effect on investors' opinions towards financial markets. The reliability of the computational models on stock market prediction is important as it is very sensitive to the economy and can directly lead to financial loss. In this paper, we retrieved, extracted, and analyzed the effects of news sentiments on the stock market. Our main contributions include the development of a sentiment analysis dictionary for the financial sector, the development of a dictionary-based sentiment analysis model, and the evaluation of the model for gauging the effects of news sentiments on stocks for the pharmaceutical market. Using only news sentiments, we achieved a directional accuracy of 70.59% in predicting the trends in short-term stock price movement.

Graph processing is one of the most important topics in big data processing. The graph architecture is suitable for distributed processing as the processing works in an iterative manner allowing parallelism. Also, the structure has proved to be suitable in representing social networks, web page indexes, and many other problems. However, graph processing introduce many problems as well. Partitioning the graph to distribute the data on multiple machines and minimizing data movement is a serious challenge. Also many of the graph algorithms have high complexity. GraphX is one of the frameworks that introduce an abstraction on top of Spark, an iterative data processing engine. However, GraphX and other novel graph abstractions still do not support processing data streams with online graphs. In this work we try to use IndexedRDD, a library to enable fine grained updates as a key-value store on top of Spark to represent a graph structure and test if it can be used as an efficient online graph storage for spark streaming. We did experiments to compare our data streaming implementation using IndexedRDD with the obvious elementary solution of using RDD transformations to join the old RDD with the new one to make a new composite RDD on each micro-batch. We also want to compare the above two with a distributed in-memory key-value store (such as Redis). The results show big advantage of using Redis over RDD transformations and IndexedRDD. However, it has some limitations such as lacking the support for property graphs. IndexedRDD, on the other hand, has shown good performance for insertions and a shortcoming in its need to rebuild the index after each data update, which add extra time on each lookup that cannot be tolerated when lookup speed is essential.

Cloud-based Analytics-as-a-Service (CLAaaS) was developed by Zulkernine et al. with a goal to simplifying big data analytics users. It provides software-as-a-service access to a variety of back end analytics tools and data stores. One of the tools is the Workflow Instance Generation and Selection (WINGS). WINGS allows users to reuse predefined workflows and their components containing semantic meta-data to define new workflows; late binding of the workflows to data at the time of execution to enable the use of most recent data, and definition of domain specific software code as custom analytic components in workflows. How ever, the data used in WINGS for the workflows are mostly flat files that are stored on the WINGS server or shared directories. The goal of this project is to add support for big data storage systems to WINGS and validate the extensions using multiple data analytic workflows of different complexities with data residing in a variety of back end data sources. The extension allows the CLAaaS users to create, validate and execute analytic workflows in a distributed environment and use data from multiple big data storage systems. We validate our work using four big data storage systems in WINGS workflows namely, Apache HBase, MongoDB, MySQL with a front-end interface.

Predicting stock prices using computer generated models has been a popular research topic and has also been widely explored. However, the connectivity of the global financial market, availability of big data in multiple domains that influence the financial market, accessibility of information in real time and the demand for fast analytics continue to offer new research challenges. One of the complexities stems from the numerous ways in which we seek to set prediction parameters, whether it is the difference in an individual stocks' growth pattern or the time frame in which the predictions occur. The level of complexity has created a trend towards more advanced techniques in this field namely, the research into developing hybrid models that are composed of multiple prediction models with a view to yield more accurate results. The Proposed Hybrid Model (PHM) used in this paper is a combination of an Exponential Smoothing Model (ESM), an Auto Regressive Integrated Moving Average (ARIMA) model, and a Back-propagation Neural Network (BPNN) model. PHM combines the predictions of each of the component model based on weights assigned by a genetic algorithm, which is designed to provide an optimum output. In this paper, we seek to use the S&P 400 and 500 indexes to train and test the PHM to find daily closing values. For comparison of the results, Directional Accuracy (DA) is used as a metric. It was found that the results for the ARIMA and ESM on daily stock index data were far less accurate than that of the BPNN, which received comparable results to the baseline. However, due to the poor results of the ARIMA and ESM the hybrid model showed no significant results for the data and was inferior to the baseline.

Enormous amounts of data are generated everyday of both structured and unstructured nature. Regardless of their differences, data sources must be used in tandem in any effective big data operation. This paper proposes a Software as a Service (SaaS) framework called BINARY which provides a back-end infrastructure for ad-hoc querying, accessing, visualizing and joining data from different data sources such as Relational Database Management Systems like MySQL and big data storage systems like Apache Hive. BINARY is extendable and allows adding other storage engines (e.g. HBase) and analytics engines (e.g. R) as needed. A REST software architecture is used in the framework to enable loose connections between the engines and user interface programs to facilitate their independent updates without affecting the data infrastructure. Our approach is validated with a proof-of-concept prototype implemented on the OpenStack cloud system.

In this research, we have developed a model for predicting the profitability class of a movie namely "Profit" and "Loss" based on the data about movies released between the years 2010 and 2015. Our methodology considers both historical data as well as data extracted from the social media. This data is normalized and then given a weight using standard normalization techniques. The cleaned and normalized dataset is then used to train a back-propagation cross entropy validated neural network. Results show that our strategy of identifying the class of success is highly effective and accurate when compared to the results from using a support machine vector on the data.

The cloud computing paradigm provides the "illusion" of infinite resources and, therefore, becomes a promising candidate for large-scale data-intensive computing. In this paper, we explore experiment-driven performance models for data-intensive workloads executing in an infrastructure-as-a-service (IaaS) public cloud. The performance models help in predicting the workload behaviour, and serve as a key component of a larger framework for resource provisioning in the cloud. We determine a suitable prediction technique after comparing popular regression methods. We also enumerate the variables that impact variance in the workload performance in a public cloud. Finally, we build a performance model for a multi-tenant data service in the Amazon cloud. We find that a linear classifier is sufficient in most cases. On a few occasions, a linear classifier is unsuitable and non-linear modeling is required, which is time consuming. Consequently, we recommend that a linear classifier be used in training the performance model in the first instance. If the resulting model is unsatisfactory, then non-linear modeling can be carried out in the next step.

The effective use of services to compose business processes in services computing demands that the Quality of Services (QoS) meet consumers' expectations. Automated web-based negotiation of Service Level Agreements (SLA) can help define the QoS requirements of critical service-based processes. We propose a novel trusted Negotiation Broker (NB) framework that performs adaptive and intelligent bilateral bargaining of SLAs between a service provider and a service consumer based on each party's high-level business requirements. We define mathematical models to map business-level requirements to low-level parameters of the decision function, which obscures the complexity of the system from the parties. We also define an algorithm for adapting the decision functions during an ongoing negotiation to comply with an opponent's offers or with updated consumer preferences. The NB uses intelligent agents to conduct the negotiation locally by selecting the most appropriate time-based decision functions. The negotiation outcomes are validated by extensive experimental study for Exponential, Polynomial, and Sigmoid time-based decision functions using simulations on our prototype framework. Results are compared in terms of a total utility value of the negotiating parties to demonstrate the efficiency of our proposed approach.

Negotiation of service level agreements (SLAs) is very important for maintaining quality of service (QoS) of composite Web services-based business processes. The process of negotiation involves specification of negotiation parameters, exchanging offers to conduct the actual negotiation process, and then finally generating the formal SLA if the negotiating parties come to a consensus. We propose a negotiation broker (NB) middleware framework to facilitate automated negotiations of SLAs for Web services in a service oriented architecture (SOA). High level business goals, contexts, preferences, constraints, and values of the negotiation issues are expressed as a policy specification by each of the negotiating parties. The NB maps the policy specifications to low level negotiation strategy models and parameters in order to conduct the negotiation locally as a trusted broker. We present a model and an example of the high level negotiation policy specification. We also present our NB framework including a prototype implementation to illustrate the mapping of the policy to a time-dependent negotiation strategy model.

With an increasing number of Web services providing similar functionalities, quality of service (QoS) is becoming an important criterion for selection of the best available service. Currently the problem is twofold. The Universal Description, Discovery and Integration (UDDI) registries do not have the ability to publish the QoS information, and the authenticity of the advertised QoS information available elsewhere may be questionable. We propose a model of reputation-enhanced QoS- based Web services discovery that combines an augmented UDDI registry to publish the QoS information and a reputation manager to assign reputation scores to the services based on customer feedback of their performance. A discovery agent facilitates QoS-based service discovery using the reputation scores in a service matching, ranking and selection algorithm. The novelty of our model lies in its simplicity and in its coordination of the above mentioned components. We present experiments to evaluate the effectiveness of our approach using a prototype implementation of the model.