Association rule mining techniques can generate a large volume of sequential data when implemented on transactional databases. Extracting insights from a large set of association rules has been found to be a challenging process. When examining a ruleset, the fundamental question is how to summarise and represent meaningful mined knowledge efficiently. Many algorithms and strategies have been developed to address issue of knowledge extraction; however, the effectiveness of this process can be limited by the data structures. A better data structure can sufficiently affect the speed of the knowledge extraction process. This paper proposes a novel data structure, called the Trie of rules, for storing a ruleset that is generated by association rule mining. The resulting data structure is a prefix-tree graph structure made of pre-mined rules . This graph stores the rules as paths within the prefix-tree in a way that similar rules overlay each other. Each node in the tree represents a rule where a consequent is this node, and an antecedent is a path from this node to the root of the tree. The evaluation showed that the proposed representation technique shows significant value. It compresses a ruleset with no data loss and benefits in terms of time for basic operations such as searching for a specific rule, which is the base for many knowledge discovery methods. Moreover, our method demonstrated a significant improvement in graph traversal time compared to traditional data structures.

When the global pandemic struck in 2020, most countries established task forces to meet a challenge that impacted governmental resources. It became apparent that data, intelligence gathering, and both modelling and predictive capabilities were required. While artificial intelligence (AI) based solutions had already begun to emerge within the public sector, the Covid-19 pandemic accelerated this process. In particular, modelling of case numbers with the development of predictive algorithms. The development of AI solutions for public sector organizations is inherently multidisciplinary. This is crucial to understanding how solutions can be developed, outputs understood, and the benefits and risks measured. Furthermore, the development of AI solutions often requires data which may not be accessible from a single location. In the case of Covid-19 modelling, data must be extracted from multiple locations to construct data assets. In this research, a collaborative approach to developing machine learning expertise for the public sector is presented. Using Covid-19 as a case study, the role of different government sectors when building data assets is examined along with the use of standard data models, and how this type of cooperation led to the development of a pipeline for data assets to underpin AI solutions for the public sector.

While using online datasets for machine learning is commonplace today, the quality of these datasets impacts on the performance of prediction algorithms. One method for improving the semantics of new data sources is to map these sources to a common data model or ontology. While semantic and structural heterogeneities must still be resolved, this provides a well established approach to providing clean datasets, suitable for machine learning and analysis. However, when there is a requirement for a close to real time usage of online data, a method for dynamic Extract-Transform-Load of new sources data must be developed. In this work, we present a framework for integrating online and enterprise data sources, in close to real time, to provide datasets for machine learning and predictive algorithms. An exhaustive evaluation compares a human built data transformation process with our system's machine generated ETL process, with very favourable results, illustrating the value and impact of an automated approach.

The global food and agricultural industry has a total market value of USD 8 trillion in 2016, and decision makers in the Agri sector require appropriate tools and up-to-date information to make predictions across a range of products and areas. Traditionally, these requirements are met with information processed into a data warehouse and data marts constructed for analyses. Increasingly however, data are coming from outside the enterprise and often in unprocessed forms. As these sources are outside the control of companies, they are prone to change and new sources may appear. In these cases, the process of accommodating these sources can be costly and very time consuming. To automate this process, what is required is a sufficiently robust extract–transform–load process; external sources are mapped to some form of ontology, and an integration process to merge the specific data sources. In this paper, we present an approach to automating the integration of data sources in an Agri environment, where new sources are examined before an attempt to merge them with existing data marts. Our validation uses a case study of real world Agri data to demonstrate the robustness of our approach and the efficiency of materializing data marts.

Summary Stream processing systems are designed to analyze data arriving in real time and using continuous queries and respond when a specific event or sequence of events are detected. An important aspect of these systems is Streaming Analytics, which facilitates statistical calculations on continuous data within the stream. These systems must be designed to handle high volumes of data, be scalable, and accommodate a multitude of long‐lived concurrently running analytics. The challenges involved in the development of stream processing include on‐the‐fly transformation of data streams to match the query needs of users and the ability to model stream transformations to detect overlaps and possibilities for optimizations and to specify a methodology to deliver optimizations. In particular, this work focuses on exposing data stream application internals in order to detect reusable parts and then consolidate applications to optimize computational resource usage. The Streaming Data Analytics Model presented in this paper adopts a declarative approach that enables processing and manipulation of data streams in a simple manner while facilitating powerful optimizations necessary for managing high volumes of streaming data in real time. An evaluation is provided to demonstrate in both theoretical and quantitative aspects the high performance offered by our approach.

Many organizations require the ability to manage high-volume high-speed streaming data to perform analysis and other tasks in real-time. In this work, we present the Information Streaming Engine, a high-performance data stream processing system capable of scaling to high data volumes while maintaining very low-latency. The Information Streaming Engine adopts a declarative approach which enables processing and manipulation of data streams in a simple manner. Our evaluation demonstrates the high levels of performance achieved when compared to existing systems.

Current estimates predict 1 in 3 people born today will develop dementia, suggesting a major impact on future population health. As such, research needs to connect specialist clinicians, data scientists and the general public. The In-MINDD project seeks to address this through the provision of a Profiler, a socio-technical information system connecting all three groups. The public interact, providing raw data; data scientists develop and refine prediction algorithms; and clinicians use in-built services to inform decisions. Common across these groups are Risk Factors, used for dementia-free survival prediction. Risk interactions could greatly inform prediction but determining these interactions is a problem underpinned by massive numbers of possible combinations. Our research employs a machine learning approach to automatically select best performing hyperparameters for prediction and learns variable interactions in a non-linear survival-analysis paradigm. Demonstrating effectiveness, we evaluate this approach using longitudinal data with a relatively small sample size.

Clinical studies provide interesting case studies for data mining researchers, given the often high degree of dimensionality and long term nature of these studies. In areas such as dementia, accurate predictions from data scientists provide vital input into the understanding of how certain features (representing lifestyle) can predict outcomes such as dementia. Most research involved has used traditional or shallow data mining approaches which have been shown to offer varying degrees of accuracy in datasets with high dimensionality. In this research, we explore the use of deep learning architectures, as they have been shown to have high predictive capabilities in image and audio datasets. The purpose of our research is to build a framework which allows easy reconfiguration for the performance of experiments across a number of deep learning approaches. In this paper, we present our framework for a configurable deep learning machine and our evaluation and analysis of two shallow approaches: regression and multi-layer perceptron, as a platform to a deep belief network, and using a dataset created over the course of 12 years by researchers in the area of dementia.

XML has become a widely used and well structured data format for digital document handling and message transmission. To find useful knowledge in XML data, data warehouse and OLAP applications aimed at providing supports for decision making should be developed. Apache Hadoop is an open source cloud computing framework that provides a distributed file system for large scale data processing. In this paper, we discuss an XML data cube model which offers us the complete views to observe XML data, and present a basic algorithm to implement its building process on Hadoop. To improve the efficiency, an optimized algorithm more suitable for this kind of XML data is also proposed. The experimental results given in the paper prove the effectiveness of our optimization strategies.

XML data warehouses are becoming more popular as data is harvested from the web or as output from web services. As these warehouses tend to grow significantly over time, various techniques for expediting queries have been developed. One such technique is to materialize some or all of the queries in advance of query processing. These views are then subject to change either when underlying data changes or view definitions themselves are modified by users. The work in this paper focuses on changes to view definitions or view adaptation as it is known. Our approach is to segment the materialized view into fragments to minimize the effect of view changes. One crucial aspect to this approach is how to select the best fragments for materialization. In this paper, we introduce a new approach to selecting fragments based on heuristics derived from costs associated with the view graph. Copyright © 2013 John Wiley & Sons, Ltd.

In many ubiquitous computing environments, where applications and systems are required to determine the location of individuals, context information is necessary to support decision making. In practical terms, this requires a hybrid query interface where the user can query live streaming data (to confirm their location) while also using more traditional database expressions to query or mine context repositories. The research presented in this paper describes our approach to developing a framework to support this form of hybrid query application. Our industry collaborator provides a real-world application in which spaces that are equipped with ubiquitous sensing environments are prone to frequent change, requiring a flexible approach to the management of context information.

Many of the emergent digital ecosystems will employ sensor networks to generate data. Using XML to introduce structure and semantics assists the ecosystem as standard XML query languages such as XPath and XQuery are used to extract information and results of analyses. However, the creation of XML digital archives is hindered by the performance of these query languages. Furthermore, the multi-disciplinary nature of digital ecosystems means that knowledge workers and end users will rarely be IT professionals and therefore, unable to express XPath or XQuery easily. In this work, we present a method for precomputing and storing query results, leading to far higher levels of performance and removing the requirement for non-IT users to learn complex query languages.

As the deployment of wireless sensor networks increase and their application domain widens, the opportunity for effective use of XML filtering and streaming query engines is ever more present. XML filtering engines aim to provide efficient real-time querying of streaming XML encoded data. This paper provides a detailed analysis of several such engines, focusing on the technology involved, their capabilities, their support for XPath and their performance. Our experimental evaluation identifies which filtering engine is best suited to process a given query based on its properties. Such metrics are important in establishing the best approach to filtering XML streams on-the-fly.

Traditional sensor network deployments consisted of fixed infrastructures and were relatively small in size. More and more, we see the deployment of ad-hoc sensor networks with heterogeneous devices on a larger scale, posing new challenges for device management and query processing. In this paper, we present our design and prototype implementation of XSense, an architecture supporting metadata and query services for an underlying large scale dynamic P2P sensor network. We cluster sensor devices into manageable groupings to optimise the query process and automatically locate appropriate clusters based on keyword abstraction from queries. We present experimental analysis to show the benefits of our approach and demonstrate improved query performance and scalability.

Personal Health (pHealth) sensor networks are generally used to monitor the well being of both athletes and the general public to inform health specialists of future and often serious ailments. The problem facing these domain experts is the scale and quality of data they must search in order to extract meaningful results. By using peer-to-peer sensor architectures and a mechanism for reducing the search space, we can, to some extent, address the scalability issue. However, synchronisation and normalisation of distributed sensor streams remains a problem in many networks. In the case of pHealth sensor networks, it is crucial for experts to align multiple sensor readings before query or data mining activities can take place. This paper presents a system for clustering and synchronising sensor streams in preparation for user queries.

As the popularity of areas including document storage and distributed systems continues to grow, the demand for high performance XML databases is increasingly evident. This has led to a number of research efforts aimed at exploiting the maturity of relational database systems in order to increase XML query performance. In our approach, we use an index structure based on a metamodel for XML databases combined with relational database technology to facilitate fast access to XML document elements. The query process involves transforming XPath expressions to SQL which can be executed over our optimised query engine. As there are many different types of XPath queries, varying processing logic may be applied to boost performance not only to individual XPath axes, but across multiple axes simultaneously. This paper describes a pattern based approach to XPath query processing, which permits the execution of a group of XPath location steps in parallel.

In distributed computing systems, it is unwise to move data to the point of program code, but instead process data at the point of storage. This concept is even more appropriate to multimedia repositories where large files must be processed as a result of each retrieval operation. Two problems with multimedia databases are that they require a large volume of disk storage and processing power and that it is generally difficult to query video content or optimise the retrieval process. In the EGTV project, both of these issues are addressed. In the first case, the data server is distributed across multiple autonomous sites where users are permitted to modify schemas and in some cases the data model. In the second case, a mechanism for storing behaviour has been devised which allows functions such as retrieve_frame, retrieve_segment and retrieve_context to be implemented and stored at individual servers. This provides selective retrieval and thus, large data items can be processed locally to filter unwanted data before the transfer process must begin. In this way, storage of operations forms the basis for optimisation of retrieval content and volumes.

Global transactions are still an issue for federated database systems. In the IOMPAR project, one of the goals is to develop a transaction protocol for ODMG databases which act as wrappers to information systems in a federation. In this short paper we describe an architecture for transporting secure data in a federated database system. Our on-going work involves providing the transaction service which can guarantee global transactions within our architecture. These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Parallel and distributed programming is conceptually harder to undertake and to understand than sequential programming, because a programmer often has to manage the coexistence and coordination of multiple concurrent activities. The model of 'Generative Communication' in Linda-a paradigm that has been developed for parallel computing-emphasizes the decoupling of cooperating parallel processes; thus, relieving the programmer from the burden of having to consider all process inter-relations explicitly. In many application areas, data is distributed over a multitude of heterogeneous, autonomous information systems. These systems are often isolated and an exchange of data among them is not easy. On the other hand, support for dynamic exchange of data is required to improve the business processes. Cooperative information systems enable such autonomous systems to interoperate. They are complex systems of systems which require a well designed and flexible software architecture. The Linda model had a great influence on research in parallel programming languages. Stimulated by this success, a Generative Communication Service, which offers a very flexible associative addressing mechanism based on metadata matching, has been developed for supporting interoperability of cooperative information systems. Some design patterns guided the construction of the resulting communication service that has been implemented on top of CORBA for an ODMG canonical data model.

Coursework is a necessary part of most subjects in most disciplines. Some subjects require a series of continuous assessments to be taken during the course of the semester, resulting in a large amount of paperwork and administrative overhead. In this paper we describe a mechanism for collecting coursework, providing a means of verifying receipt of coursework, and publishing the results automatically. The aim of the work is to provide a user-friendly interface for students to submit all forms of electronic coursework (textual and binary), and an equally user-friendly interface for tutors to download coursework, and subsequently publish the coursework grade. In this way, neither the student nor the tutor need be expert in computing, and the overhead of processing coursework is greatly diminished.