In Smart grid ecosystems, it is important to carefully choose the placement of the datasets across different kind of big data systems in order to achieve high performance of the workloads and conformity with the business and data ecosystem. Our approach for datasets placement is based on metadata about datasets, workloads, and systems. This paper gives a general overview of the data placement module, proposes a high-level design and data model for our solution and presents the placement criteria.

Nowadays, IoT data come from multiple sources from a large number of devices. To manage them, IoT frameworks rely on Big Data ecosystems hosted in the cloud to provide scalable storage systems and to achieve scalable processing. Although these ecosystems scale well to process large sizes of data, in many cases this is done naively. Many datasets, such as IoT Energy measurement data, consist, even partially, of attributes that can be indexed to avoid unnecessary and costly data scan at these scales. In this work, we propose the iBig architecture thatprovides secondary indexing to Big Data processing for energy IoT datasets. Indexes are considered as metadata stored in a separate component integrated in the ecosystem. Subsequently, MapReduce-based and MPP (massively parallel processing)-based processing leverage indexing to handle only relevant data in a given dataset. Our experimental evaluation on the Grid5000 cloud testbed demonstrates that performance gains can exceed 98% for the MapReduce-based Spark and 81% for the MPP-based Drill for Energy IoT Data. Furthermore, we provide comparison insights about the Spark and Drill frameworks when processing the whole dataset or only with relevant data.

Nowadays, more and more data are being generated and collected in electrical smart grids. Most of these data are coming from smart meters and sensors deployed massively throughout the power grid. As the generation of data is becoming ever more frequent and with the constantly increasing volumes, it is becoming harder and harder to manage and process these data at the scale of a smart grid within legacy systems. In this work, we focus on investigating the scalability and performance of different data management approaches for meter data processing. To this end, we conduct a thorough experimental study of various systems including a parallel relational database system, MapReduce based systems including Hadoop and Spark, and a NoSQL datastore system. Our experiment sets were conducted on up to 140 nodes on Grid5000 and up to 1.4 TB of meter data. Our results demonstrate that parallel relational systems are more suited for most processing types on smart meter data in the smart grid but at the cost of very slow data loading. In contrast, we show that with the appropriate distribution model, data partitioning and modeling choices we achieve very fast and scalable bill computations, the main complex processing for utilities providers.

International audience

The use of scalable and heterogeneous data stores within a single system is gradually becoming a common practice in application development. Modern applications tend to rely on a polyglot approach to persistence, where traditional databases, non-relational data stores, and scalable systems associated to the emerging New SQL movement, are used simultaneously. However, considering that a large number of these systems rely on schema-less data models, developers in charge of maintaining applications that depend on these data stores need to manually analyze the application source code, in order to discover their data schemas. To help overcome this situation, in this paper we demonstrate ExSchema, a tool for the automatic discovery of external data schemas, directly from the source code of polyglot persistence applications. ExSchema is available as an Eclipse plug in, and can be easily included within existing development environments. Our tool can also be integrated with Git repositories, in order to analyze the evolution of the schemas used by an application over a period of time. In particular, this paper presents a demonstration scenario where the schemas from a system that relies on multiple document, graph, relational and column-family data stores, are discovered from the application source code.

The emergence of mobile and ambient computing technologies brings democratization in the access to information and data; services play a crucial role, thereby opening new research challenges for data querying. A promising method to access data within these novel dynamic environments in a convenient and efficient way is declarative querying. Such queries, which we characterize as hybrid queries, involve streaming and on-demand data originated from services, possibly with temporal and mobile properties. To evaluate these queries we propose an approach implemented in the HYPATIA system, which addresses two main aspects (i) using service coordination for query evaluation and (ii) an efficient and flexible mechanism that facilitates incorporating new capabilities.

An UBIQUEST system provides a high level programming abstraction for rapid prototyping of heterogeneous and distributed applications in a dynamic environment. Such a system is perceived as a distributed database and the applications interact through declarative queries including declarative networking programs (e.g. routing) and/or specific data-oriented distributed algorithms (e.g. distributed join). Case-Based Reasoning is used for optimization of distributed queries when as there is no prior knowledge on data (sources) in networking applications, and certainly no related metadata such as data statistics.

This paper concerns the integration of the Case Based Reasoning (CBR) paradigm in query processing, providing a way to optimize queries when there is no prior knowledge on queried data sources and certainly no related metadata such as data statistics. Our Query Optimization by Learning (QOL) approach optimizes queries using cases generated from the evaluation of similar past queries. A query case comprises: (i) the query, (ii) the query plan and (iii) the measures (computational resources consumed) of the query plan. The work also concerns the way the CBR process interacts with the query plan generation process. This process uses classical heuristics and makes decisions randomly (e.g. when there is no statistics for join ordering and selection of algorithms, routing protocols); It also (re)uses cases (existing query plans) for similar queries parts, improving the query optimization and evaluation efficiency.

This paper presents an approach for building reliable applications based on services oriented paradigm and the use of a contract based coordination model. Thanks to contracts describing the logic of a services based application running in a dynamic environment it is possible to associate a personalized behavior in an orthogonal way. Contracts ensure, for example, transactional properties at execution time in the presence of exceptions and make applications aware of their execution context.

This paper focuses on the software architecture and components for supporting core adaptive and interactive query processing. The architecture permits the reuse of most of the existing adaptation techniques and facilitates the development of new ones. Developers will use it to program adaptable and interactive query evaluators, called Query Brokers. Three adaptability levels are considered: (i) static adaptability, achieved at query broker building time, (ii) personalization, achieved for each (user) query having specific needs, and (iii) dynamic adaptability, achieved during query evaluation.

This paper proposes a flexible transactional behavior model for services coordination. We assume that given a flow describing the application logic of a service based application, it is possible to associate to it a personalised transactional behaviour in an orthogonal way. This behaviour is defined by specifying contracts and associating a well defined behaviour to the activities participating in the coordination. Such contracts ensure transactional properties at execution time in the presence of exceptions

Service oriented computing has gained a considerable momentum as a new paradigm for building enterprise information systems. Notable efforts have been made recently from both researchers and industrials to support the construction of service-based applications, nevertheless several issues still need to be tackled including service definition and adaptation, and services orchestration. This work proposes an approach for building and finely orchestrating open and adaptable services. An open service is represented by a workflow that coordinates calls to service provider methods. Thereby component activities and the way they are synchronized are rendered visible. Service adaptability refers to the possibility to modify an open service. Through adaptation operations a service can be customized according to given user (application) requirements. In order to finely orchestrate services, they are associated with entry points. An entry point acts as a gateway for inserting and getting information about the progress of service execution. Defined services and orchestration are verified to ensure a correct behaviour of the resulting application. The paper details our approach for building and orchestrating services, and presents associated architectural choices.

This paper introduces definition and enactment models to characterize workflows. The proposed knowledge model provides concepts for modeling activities, data used by such activities, agents, and ordering operators and synchronization modes used for specifying activities dependencies. The behavior model provides dimensions that characterize workflow enactment. Both models are currently implemented in our workflow management system called AFLOWS. The originality of our approach is related to its flexibility as different enactment models can be associated to a workflow simply by giving values to dimensions according to application needs. Such an approach and system provide a suitable support for specifying and enacting workflow processes that may be useful today to describe and monitor process-oriented applications (i.e., large Web-based applications that run across wide area networks and multiple enterprises).

Applications are now highly distributed, heterogeneous and scalable. They comprise autonomous, loosely coupled components, some of them being database management systems, Web servers, etc. Considering such a framework, database technology has to evolve towards cooperation and integration. We propose open and distributed active services that can be used for specifying and generating event and rule managers that support cooperation and interaction between distributed database software (i.e. applications and systems). Event managers are brokers supporting anonymous event-passing communications. Rule managers are able to support global (business) rules and execute them using different policies that are adapted to specific application needs. Both managers can reconfigure and adapt themselves with respect to application requirements and to their environment.

Summary form only given. The Native Active Object System (NAOS) incorporates an active behavior within the object-oriented database management system O/sub 2/. NAOS rules are event-condition-action (EGA) rules belonging to an O/sub 2/ database schema. The authors focus on user and temporal event detection as well as composite event detection.

No abstract available.