Modern ISR as well as tactical missions have to "operate off the same map" and "get the right data at the right time, in the right format". This poses high challenges as data of substantial variety have to be gathered, homogenized, and distributed in near realtime, including both static, large scale data centers and moving edge devices as heterogeneous sources and sinks. Further, different players, ranging from human analysts to automated effectors, have different needs.Space/time datacubes, AI, and federation are technical concepts generally accepted as advantageous for such tasks aiming at easier, flexible, scalable, and interoperable ad-hoc data insights. The Cube4EnvSec project, supported by the NATO Science for Peace and Security (SPS) program, has set out to demonstrate how a suitable combination of these concepts can substantially enhance the information availability for ISR and the NATO Federated Mission Network (FMN). The Cube4EnvSec services are part of the Earth-Server datacube federation.We present the project, its technology, and use cases.

Array databases specialize in storage, management, and query processing on massive multidimensional array data such as satellite image time-series, weather forecast models, IoT sensor measurements, medical imaging data, etc. Some of the values in this data may be ”null” for a variety of reasons, such as unknown, known false, etc. The DBMS must handle null values correctly and efficiently. As of today, the effects of different data structures for representing null values on query processing in array databases have not been systematically studied. As a consequence, it is not clear what the optimal way for handling null values is. In this paper, we measure how four different methods for encoding null values perform across most common categories of array operations. The result is a comprehensive overview, publicly available in open source, providing relevant insights into the performance characteristics of these data structures allowing to discern which among them offers the most optimal approach for representing null values in an array processing context.

Abstract This paper provides an in-depth survey on the integration of machine learning and array databases. First,machine learning support in modern database management systems is introduced. From straightforward implementations of linear algebra operations in SQL to machine learning capabilities of specialized database managers designed to process specific types of data, a number of different approaches are overviewed. Then, the paper covers the database features already implemented in current machine learning systems. Features such as rewriting, compression, and caching allow users to implement more efficient machine learning applications. The underlying linear algebra computations in some of the most used machine learning algorithms are studied in order to determine which linear algebra operations should be efficiently implemented by array databases. An exhaustive overview of array data and relevant array database managers is also provided. Those database features that have been proven of special importance for efficient execution of machine learning algorithms are analyzed in detail for each relevant array database management system. Finally, current state of array databases capabilities for machine learning implementation is shown through two example implementations in Rasdaman and SciDB.

Arrays are an accepted data structure in database world today, next to sets, graphs, etc. Only relatively late, however, arrays have found acceptance in the database community as a first-class abstraction. Meantime, array databases with their declarative query support on "datacubes" have been built and are successfully applied in science, engineering, fintech, and other domains.In this case study we report about using array databases for the flexible management and analysis of spatio-temporal data for radio networks planning and optimization. We present the use case and how datacubes respond to the challenges of this field. To the best of our knowledge, this has not been attempted before, so it heralds a new field of application for geo array databases, in addition to well-known satellite imagery and climate datacubes.

Rapid environmental changes due to climate change call for innovative technological approaches, and likewise is this required for security missions. Essentially, organizations like NATO need advanced capabilities allowing geographically distributed joint mission forces with all their information sources and consumers to (i) operate off the same map and (ii) get the right data at the right moment. Datacubes, aka spatio-temporally homogenized raster data, are an accepted cornerstone for providing Big Geo Data easier for users and applications, ready for analysis, visualization, fusion, etc. Recently, NATO has started a project exploiting Big Earth Datacube Analytics for transnational security and Environment protection. A series of relevant use cases will be established proving the capabilities of realtime datacube management, analytics, and federation of fixed and moving data sources and sinks. We present background, mission, and thrust of the project.

Arrays as a fundamental data category have found their way into the orchestration of data models supported by databases. While OLAP "datacubes" can be emulated relationally to some extent it was in particular applications in science and engineering that have prompted support for arrays regardless of sparsity and with dedicated powerful array operators supporting n-D Tensor Algebra.

Datacubes form an accepted cornerstone for analysis (and visualization) ready spatio-temporal data offerings. Geo datacubes have been standardized since long under the umbrella concept of coverages, and such data structures are well understood in concept and practice. This, however, is not paired by a similar understanding of coverage analytics.We present a formal model for datacube analytics which is based on Linear Algebra, incorporates space and time semantics, and allows a wide range of common datacube operations, up to, say, the Discrete Fourier Transform. For convenience, the formalism is based on a language allowing expressions of any complexity.The specification is currently in the avanced adoption process of ISO for becoming the future 19123-3 standard.

Coverages represent a unifying concept for space/time Big Data which is mature, conceptually elaborate, and practically proven. Recently, coverage services have gained high interest, driven by the fact that getting insights through analytics is only feasible through advanced server-side analytics. The need for data fusion adds further challenges. Not surprisingly, considerable efforts have been invested by various standardization bodies to establish concise, interoperable specifications. In this contribution we inspect and relate coverage standards in OGC, ISO, and INSPIRE, thereby providing “in a nutshell” the core concepts, standards, and trends in the field of coverages.

Server-side extensibility through dynamically linked external code is a common method in relational databases. In the field of Array Databases such User-Defined Functions (UDFs) sometimes even represent the architectural cornerstone for array functionality. On the downside, UDF implementation often suffers from high coding complexity.The rasdaman Array DBMS is a full-stack C++ implementation, so does not rely on some generic UDF mechanism. This allowed designing such an API from scratch, with particular emphasis on UDF coder convenience. The rasdaman UDFs rely on the general C++ client API classes. Based on a straightforward UDF interface definition adapter code is generated automatically. Experimental evaluation shows encouraging results, and the mechanism is going to be used in research and under operational conditions. We present the approach and motivate it through practical use cases.

Machine Learning is increasingly being applied to many different application domains. From cancer detection to weather forecast, a large number of different applications leverage machine learning algorithms to get faster and more accurate results over huge datasets. Although many of these datasets are mainly composed of array data, a vast majority of machine learning applications do not use array databases. This tutorial focuses on the integration of machine learning algorithms and array databases. By implementing machine learning algorithms in array databases users can boost the native efficient array data processing with machine learning methods to perform accurate and fast array data analytics.

Flexible, scalable services on massive Earth data receive much attention today, in particular spatio-temporal data homogenized into datacubes. OGC has taken a lead in defining interoperable spatio-temporal datacubes through its coverage paradigm, together with tailored services, the Web Coverage Service (WCS) standards suite. This OGC Big Geo Data standard has attracted a who's who of major implementers of both open-source and proprietary servers and clients. We present status, uptake, and recent progress of the coverage datacube standards in OGC, ISO, and INSPIRE, in particular the ongoing work on the OpenAPI based OAPI-Coverages specification.

Array Databases, offering actionable datacubes, are increasingly used for flexible's callable Earth data services. Technically, Array Databases converge several concepts that are also taught - separately - in science ang engineering: Linear Algebra as in Analytical Geometry, query languages as in databases, and image processing as in remote sensing. This begs the question: can such a tool be used for such diverse teaching in a meaningful way, thereby also provoking possibly surprising new correlations with students? In following this idea, in this contribution we explore how Array Databases can serve for educational purposes in undergraduate education.

We present the EarthServer datacube federation which, based on Array DBMS technology, has established an intercontinental network of Earth data centers. Users get provided with a single, uniform information space acting like a local data holding, thereby establishing full location transparency. We present the technology and show a broad range of real-life distributed data fusion examples.

Array Databases close a gap in the database ecosystem by adding modelling, storage, and processing support for multi-dimensional arrays. Built into “datacubes” such structures are known since long in OLAP and statistics, but they also appear as spatio-temporal sensor, image, simulation, and statistics data in all science and engineering domains.

The European Copernicus programme generates massive amounts of Earth Observation (EO) data, with the goal of improving our environmental understanding and management on local, regional, and global level. An initiative of this magnitude comes with a set of challenges, especially pertaining to effective service management once the huge volumes of raster data files are distributed across open data centers and commercial companies.The BigDataCube project responds to this challenge through the concept of federated, analysis-ready datacubes, exposed via the open OGC geo standard interfaces for interoperable access and processing, WMS, WCS, and WCPS.Such federated datacube services have been established in the project between the public German Copernicus hub, CODE-DE, and a commercial cloud provider, cloudeo AG, altogether offering access to more than 500 TB. Both data pools are federated in a location-transparent manner, establishing a common information space where users can query and combine datacubes without knowing their location of storage. In this nucleus of a growing federation recently the Alfred Wegener maritime research institute has joined, and further data centers are in the line. The spectrum of functionality available is showcased through several realistic use cases, including a value-adding Sentinel-1 SAR product on sea state and wind speed in the North Sea. The platform used is the pioneer datacube engine and OGC reference implementation, rasdaman. We report on the outcomes of the project, lessons learned, and further work foreseen in this area.

Despite the current wave of providing data analysis-ready we claim that some essential properties for easy, non-EO-expert and non-programmer exploitation of EO data are not usually considered in service design. These properties relate to the quality of service (human or machine) users experience, and conversely to the burden that is imposed when accessing archives. We spot some critical features and propose solutions.

With high-volume, function-rich Web services on spatio-temporal raster data on the rise there is a growing need for adaptive access control. However, while the data-cube paradigm has proven suitable for large-scale, efficient raster services it is not yet clear how appropriate access control mechanisms can be provided to service administrators. In this contribution we provide a model which extends the SQL datacube query language, SQL/MDA, with flexible definition and enforcement of array-aware access control.

Datacub es provide a suitable paradigm for storing, accessing and processing la rge-scale, multidimensional spatio-temporal raster data. As hardware and distributed infrastructure, such as cloud and federations, become common, enabling datacubes to fully exploit the available capabilities is a crucial step in building scalable systems for complex query answering in real time. In this contribution, we describe an approach to distributed datacube processing that enables datacube engin es - in our case rasdaman - to analyze, for every incoming query, a large number of equivalent distributed execution variants, and to pick an efficient one.

Database systems are essential components of modern services and applications. The remote sensing domain has predominantly been an exception to this trend so far, however, as common data types did not fit as well into the standard database model. Instead, various data management challenges have been addressed ad-hoc with custom proprietary / non-portable solutions. This is finally changing with the recent standardization of ISO SQL/MDA which extends SQL, the standard database query language, with support for data cube storage, management, and processing. This paper gives a high-level overview of SQL/MDA and how it can be used for building flexible, high-performance, and scalable remote sensing applications.

Within the Open Geospatial Consortium (OGC), different data models have evolved for the provision of sensor data (Observation Model) and datacubes (Coverage Model). While these models are formally distinct, there are great similarities; in addition, they are often used conjunctively, as each model provides relevant information missing from the other model. In this paper, we show how both models can be utilized concurrently, thus overcoming the deficits of the other model. We detail where constraints should be applied to attributes of the individual models to assure consistency between the data being provided, and go on to show how OGC Web Services can be utilized to provide this data in a consistent and user friendly manner. Finally, we provide insights as to how these models may be harmonized within OGC.

Scientists are often interested in sampling buffered regions of data across multiple time-slices in array datacubes. For instance, in studying sea-ice distributions, a string of geographic coordinates with timestamps are requested, representing a sample or ship track line of a measurement campaign. A defined region is sampled around each of those data points using a nearestneighbour approach in time and a buffer or polygon clipping in the spatial domain. Objectively, such queries can be handled discretely across the time domain, as there is no temporal interpolation, and as a result, the tiling of extracted rasters is well-defined by the tiling of the source data. What happens when the resulting object should also be represented by a 3-D raster, such as in the case where the trackline consists of continuous buffered sampling across the timeseries? Spatio-temporal data is typically stored in chunked 3-D arrays, where multiple time-slices appear in the same "tile" or subarray. Unlike the discrete version, tracing out a polygonally-shaped buffer along a ship's path in a 3-D spatio-temporal datacube leads to shearing across the spatial tiles in the result raster, and this shearing prevents an a priori tiling of the result. Here, we present several approaches to tiling the result raster, and we provide a mathematical investigation of the impact these approaches can have on performance. To substantiate the theoretical investigation, an implementation and performance benchmarks on the different tiling approaches are provided, and the implementation is demonstrated on sea-ice data as a casestudy. In future work, we discuss different approaches towards parallelization utilizing these techniques as a basis for thread-safety, establishing the results on arbitrary R+ trees and extending these results to R* trees.

We demonstrate the rasdaman ("raster data manager") scalable datacube engine in a series of multi-dimensional live scenarios of spatio-temporal datacube analytics, distributed processing in federations, as well as simple, rapid construction of datacubes.

Datacubes are emerging as a paradigm which allows offering massive spatio-temporal data more in a more homogenized and better to analyze fashion, in particular when it comes to mixed spatial and temporal evaluation. While this is agreed by many, there is no broad consensus yet on access APIs and paradigms, such as query languages versus procedural interfaces. Further, questions of efficient storage organization and processing of the underlying multi-dimensional arrays arise. In our contribution we provide a technology overview on datacube approaches, involving Array Databases, systems with procedural (typically python) interfaces, array libraries, and more.

Flexible, scalable services on massive geo data receive much attention today. OGC has taken a lead in defining interoperable spatio-temporal datacubes (the coverage paradigm) together with flexible scalable Web services (the Web Coverage Service / WCS standards suite). This OGC Big Geo Data standard has attracted all major implementers of both open-source and proprietary servers and clients. We present status, uptake, and recent progress of the coverage datacube standards in OGC, ISO, and INSPIRE.

With the unprecedented availability of continuously observed and generated data there is a likewise unprecedented potential for new and timely insights; yet, benefits are not fully leveraged as of today. The plethora of formats in combination with heterogeneous services remains is an obstacle - e.g., image services prefer binary formats, SPARQL endpoints like to think in RDF triples, and browsers integrate JSON data smoothly. We propose a model-based multi-encoding approach for overcoming the limitations of individual formats while still supporting their use. Concretely, this approach is being followed by the OGC Coverage Implementation Schema (CIS) standard which establishes a concrete, interoperable data model unifying n-D spatiotemporal regular and irregular grids, point clouds, and meshes. We describe how independence from data formats is achieved, in particular for three practically relevant formats - XML, JSON, and RDF -, thereby fostering integration of hitherto rather separate application domains.

Geo data like satellite imagery and climate simulation output constitute major contributors to today's Big Data deluge. The datacube paradigm is currently being considered not only for getting data analysis ready, but also to offer advanced functionality in an easy-to-use way while allowing for highly effective server-side optimizations. Datacube standards can help substantially in unifying access protocols, thereby easing access for users and also paving the way for cross-data-center datacube fusion. We present OGC datacube model which is based on the coverage data and service model standardized by OGC, ISO, and INSPIRE. This paper aims at contributing to a cross-disciplinary exchange on modeling challenges, successes, and possible overlaps amenable to generalization and harmonization.

The Internet of Things is about to increase the existing data deluge even further. Standards help to unleash data and services through a well-defined, interoperable manner. In this paper we characterize the role and interplay of two OGC standards suites addressing sensors and “Big Data”, thereby illustrating how both concepts can be merged in services.

Array databases are used to manage and query large N-dimensional arrays, such as sensor data, simulation models and imagery, as well as various time-series. Modern database systems and database applications make extensive use of caching techniques to improve performance. Research on array databases on the other hand has not explored the potential benefits of caching in query processing on big arrays. In this work we propose a design for a content-aware cache for array databases which allows to reuse results of previously evaluated queries. Besides identical query matching, our method also takes into account spatially overlapping queries and queries with common subexpressions. We evaluate performance of the query cache implementation by varying data and query parameters and show that it decreases query execution time by up to 93%, with a potential for even higher savings with increasing query complexity.

The data deluge we face does not only overwhelm us with sheer data volume, but also with an increasing variety of multi-dimensional datasets. As is more and more accepted, spatio-temporal data sets often can be represented through a common unifying paradigm, namely datacubes. In this contribution, we present datacubes as a common modeling paradigm upon which flexible, scalable services can be offered based on the concept of a datacube query language. With the use case of an intercontinental datacube federation, EarthServer, we point out how the technicalities of the query language can be hidden, thereby allowing user-friendly visual data interaction.

“Big Data” in the geo domain typically are represented by spatio-temporal sensor, image, simulation, and statistics data. The OGC coverage concept represents a unifying model for an informationally complete description of such data, based on which the OGC Web Coverage Service (WCS), as OGC's “Big Data” standard, provides versatile, functionality. Large and growing tool support as well as the massive data offerings existing highlight relevance of these standards. This has prompted further standardization bodies such as ISO and INSPIRE at adopting these standards, too. In the contribution, we present coverage data and service definitions and indicate their impact on the design of large-scale data-intensive services.

With the deluge of scientific big data affecting a large variety of research institutions, support for large multidimensional arrays has gained traction in the database community in the past decade. Array databases aim to cover the gap left by traditional relational database systems in the domains of large scientific data by enabling researchers to efficiently store and process their data through rich declarative query languages. Such large amounts of data need effective systems that are able to distribute the processing at both local level, through exploitation of heterogeneous hardware as well as at network level, enabling both intra-cloud and intra-federation distribution of data and processing. In this demonstration we aim to showcase the capabilities of rasdaman by allowing users to execute queries that combine petabyte datasets stored at two institutions on different continents.

Array Databases close a gap in the database ecosystem by adding modeling, storage, and processing support on multi-dimensional arrays. Declarative queries provide processing of arrays of regularly massive size, such as Tera-to Petabyte datacubes, while allowing internal degrees of freedom in partitioning the large arrays into tractable sub-arrays. Among the important new operations is the array Theta-Join, such as overlaying two images. Evaluation of such joins is complicated by the fact that the participating arrays likely do not align in their partitioning schemes. This can lead to inefficient multiple reads of sub-arrays. We introduce array joins and present an efficient way of pairing corresponding sub-arrays. As a byproduct, this technique delivers information on optimal data placement for parallel join evaluation. The method is implemented in the Array DBMS rasdaman which is in operational use at data centers and mapping agencies.

This contribution introduces the forthcoming extension of the ISO SQL standard for multi-dimensional arrays, SQL/MDA. We present concepts, the language, and highlight how it can be implemented in a scalable manner. Examples used stem from Earth Observation and related domains.

Several scientific domains operate on mesh data structures. Computations on meshes share a set of basic abstractions which relates to topological and geometric structure of meshes. Due to the complexity of modeling and querying meshes, the database community has neglected the need for a dedicated mesh database and in particular a high-level mesh query language. Currently, such computations is done by using general-purpose programming languages. Such implementations are tightly coupled with dataset characteristics and thus are less reusable, their maintenance cost is high, and their performance is limited to the memory size. In this paper, we describe syntax and semantic of a high-level query language for scientific meshes (called MQuery) with a level of abstraction that can express queries based on topological and geometric navigations. The language is build on top of a graph-based conceptual mesh data model. We report our results comparing two implementations of the language on top of relational and graph databases with a mesh library.

Support for large arrays has been increasingly gaining attention by the database community. Array databases are a quickly expanding category of database management systems that treat large, multidimensional array data as first-class database citizens, allowing convenient and efficient storage and retrieval. Large array data on its own, however, is rarely meaningful or machine understandable. Therefore, arrays are almost always linked to additional, small metadata, typically managed in relational databases. This leads to an information integration gap, an "impedance mismatch" that we address with the work presented in this paper.

Flexible, scalable services on massive geo data receive much attention today. In particular, the OGC Web Coverage Service (WCS) standards suite has established a best practice for versatile access and retrieval on spatio-temporal "Big Data". Fewer efforts have been devoted, though, to an easy-to-use, standardized way of maintaining a service's offering. Our experience from supporting a series of heterogeneous, large-scale services reveals that this can become tedious indeed, due to heterogeneity and incompleteness of incoming data, operator-less transformation and ingest of large amounts of files, as well as the need for narrowly focused manual corrections and updates sometimes.

Earth-Science data are composite, multi-dimensional and of significant size, and as such, continue to pose a number of ongoing problems regarding their management. With new and diverse information sources emerging as well as rates of generated data continuously increasing, a persistent challenge becomes more pressing: To make the information existing in multiple heterogeneous resources readily available. The widespread use of the XML data-exchange format has enabled the rapid accumulation of semi-structured metadata for Earth-Science data. In this paper, we exploit this popular use of XML and present the means for querying metadata emanating from multiple sources in a succinct and effective way. Thereby, we release the user from the very tedious and time consuming task of examining individual XML descriptions one by one. Our approach, termed Meta-Array Data Search (MAD Search), brings together diverse data sources while enhancing the user-friendliness of the underlying information sources. We gather metadata using different standards and construct an amalgamated service with the help of tools that discover and harvest such metadata; this service facilitates the end-user by offering easy and timely access to all metadata. The main contribution of our work is a novel query language termed xWCPS, that builds on top of two widely-adopted standards: XQuery and the Web Coverage Processing Service (WCPS). xWCPS furnishes a rich set of features regarding the way scientific data can be queried with. Our proposed unified language allows for requesting metadata while also giving processing directives. Consequently, the xWCPS-enabled MAD Search helps in both retrieval and processing of large data sets hosted in an heterogeneous infrastructure. We demonstrate the effectiveness of our approach through diverse use-cases that provide insights into the syntactic power and overall expressiveness of xWCPS. We evaluate MAD Search in a distributed environment that comprises five high-volume array-databases whose sizes range between 20 and 100 GB and so, we ascertain the applicability and potential of our proposal.

Arrays are among those data types which contribute the most to Big Data -- examples include satellite images and weather simulation output in the Earth sciences, confocal microscopy and CAT scans in the Life sciences, as well as telescope and cosmological observations in Space science, to name but a few. Traditionally, the database community has neglected this, with the effect that ad-hoc implementations prevail. With the advent of NewSQL in recent years, however, the database scope has broadened, and array modelling and query support is seriously considered. Different models have been suggested, some of which are implemented or under implementation, and a consolidation of concepts can be observed. Consequently, integration of array queries into SQL is being addressed.

Although, many applications use unstructured meshes, there is no specialized mesh database which supports storing and querying mesh data. Existing mesh libraries do not support declarative querying and are expensive to maintain. A mesh database can benefit the domains in several ways such as: declarative query language, ease of maintenance, etc. In this paper, we propose the Incidence multi-Graph Complex (ImG-Complex) data model for storing topological aspects of meshes in a database. ImG-Complex extends incidence graph (IG) model with multi-incidence information to represent a new object class which we call ImG-Complexes. We introduce optional and application-specific constraints to limit the ImG model to smaller object classes and validate mesh structures based on the modeled object class properties. We show how Neo4j graph database can be used to query mesh topology based on the (possibly constrained) ImG model. Finally, we experiment Neo4j and PostgreSQL performance on executing topological mesh queries.

The use of large multidimensional, complex datasets to study the roots and the consequences of real-world phenomena is systematically gaining importance in the recent years. These datasets and their requisite metadata can be managed by query able databases, and the increasing processing capabilities of distributed computing and Array DBMS are opening new scenarios for Web-based access and analysis. The spatio-temporal nature of these observations requires the establishment of an automated machine-readable system that seamlessly accounts for time as not just an attribute, but rather as one more axis in the n-dimensional structure. This paper proposes a conceptual framework for the integration of temporal with spatial dimensions. Based on the widely used Geography Markup Language (GML) it exploits temporal Coordinate Reference Systems (CRSs) in analogy to, and interoperable with, well-known spatial CRSs. The proposed approach is then concretely applied to the OGC Web protocols Web Coverage Service (WCS) and Web Coverage Processing Service (WCPS), by means of which a user can visualize, aggregate, process remotely stored archives of scientific data. It is based on an open-source CRS resolver, SECORE, which translates URL identifiers of CRSs into their GML definitions. SECORE has been adopted by OGC as its standard CRS resolver, and the whole concept is supported by an ad-hoc OGC working group.

In the era of Big Data, being able to work with multidimensional arrays in a robust and consistent manner as the number and variety of dimensions increase, is just as important as being able to handle the large volumes inherent to this type of data. Usually, array analytics is carried out to extract meaningful information for further applications, e.g. slicing and subsetting. While domain-specific dimensions, which are beyond spatio-temporal, underlie rich domain anchor semantics, assigning consistent dimension schema for Points Of Interest (POI) across multidisciplinary data sets is challenging. New compositions of CRSs need to be constructed on the fly by a heterogeneous community with different backgrounds and applications in mind, consequently, linking dimensions via different resolvers to drive away dimension fragments from high-dimensional spaces. We propose to identify dimensions via a linked resolver approach. Such an approach allows CRSs to be referred to and looked up across multidisciplinary applications. Finally, we present a planetary use case, and specification- and scenario-based testing results to validate our approach.

Never before in history mankind has collected data at the rates we face today. Alone in 2002, an estimated 403 Petabyte of data has been acquired, equivalent to all printed information ever created before. Earth orbiting satellites, as well as ground, airborne, and underwater sensors, space observatories scan their environment at unprecedented resolutions, giving rise to "Big Science". The same holds for the life sciences where genomic data, high-resolution scans, and other modalities are collected in steadily increasing streams. Social network analysis, OLAP, and stock exchange trading represent further examples, the latter involving real-time correlation of thousands of ticker time series resulting in Terabytes of data to be analysed per single run. Summarized under Large-Scale Analytics we are witnessing an exploding demand for flexible access to massive volumes of scientific and business data sets. Arguably a large class of these massive data is represented by multi-dimensional arrays. Consequently, large arrays pose new challenges to data modelling, querying, optimization, and maintenance -- in short: we need Large-Scale Array Analytics. This tutorial introduces to the topic from a database perspective. Aspects addressed include modelling, query languages, query optimization and parallelization, and storage management. High emphasis will be devoted to applications in "Big Science", particularly geo, space, and life sciences; real-life use cases will be presented and discussed which stem from our 15 years of experience with the open-source rasdaman array DBMS and our work on geo raster service standardization. We will highlight requirements, achievements, open research issues, and avenues for future research. Discussion will make use of real-life examples, many of which Internet connected participants can replay hands-on.

In the classical triad of vector, raster, and meta data, it is the raster part which is not yet sufficiently supported in SDIs nowadays. Consequently, integration of earth observation imagery, LIDAR, legacy map scans, etc. into Spatial Data Infrastructures (SDIs) remains incomplete. In terms of standards, the OGC Web Coverage Service (WCS) Standard defines open interfaces for accessing and processing of raster data, more generally: coverages. In August 2010, the completely overhauled WCS 2.0 has been adopted by OGC. To make coverages interchangeable across all OGC-based services, WCS 2.0 has been based on Geography Markup Language (GML) 3.2.1, with a small, backwards compatible addition to achieve informational completeness. In parallel to specification writing, its reference implementation and an online demo are being pursued.

SOA is often used today for architectures distributing geo-processing computation tasks. These tasks usually are both data and computationally expensive. A fundamental issue is finding an automated, efficient task distribution and coordination among different services. We propose a language-based approach to this issue. First, the distributed geo-processing problem is written up as a declarative query using the Open Geospatial Consortium (OGC) Web Coverage Processing Service (WCPS) language standard. Second, translate the query is translated into a graph on which a rule-based graph decomposition algorithms generates a task distribution among a subset of the service hosts available. A heuristic cost function, preliminarily based on the data output sizes of each query operator, is used to partition the query and determine the execution hosts of sub-queries. The result is reassembled into a recursively nested query which also performs coordination of its distributed evaluation.

Originally Binary Large Objects (BLOBs) in databases were conceived as a means to capture any large data (whatever large meant at the time of writing) which, for whatever reason, cannot or should not be modeled relationally. Today we find images, movies, XML, formatted documents, and many more data types stored in database BLOBs. A particular challenge obviously is moving such large units of data as fast as possible, hence performance benchmarks are of interest.

Storage management of multidimensional arrays aims at supporting the array model needed by applications and insuring fast execution of access operations. Current approaches to store multidimensional arrays rely on partitioning data into chunks (equally sized subarrays). Regular partitioning however, does not adapt to access patterns, leading to suboptimal access performance. We propose a storage approach for multidimensional discrete data (MDD) based on multidimensional arbitrary tiling. Tiling is arbitrary in that any partitioning into disjoint multidimensional intervals as well as incomplete coverage of n-D space and gradual growth of MDDs are supported. The proposed approach allows the storage structure to be configured according to user access patterns through tunable tiling strategies. We describe four strategies and respective tiling algorithms and present performance measurements which show their effectiveness in reducing disk access and post-processing times for range queries.

Storing multidimensional data in databases is an important topic both in the industrial and scientific database communities. Arrays are offered as a multidimensional data structure by most programming languages. Conventional database systems, however, do not support arrays of arbitrary dimensionality and base type. RasDaMan is a DBMS integrating arrays as a first class data type offering both a declarative query language and a specialised storage structure for arrays. The work presented evaluates the performance of queries on multidimensional array data stored in RasDaMan versus storage in a conventional RDBMS. In the relational system, the data is both mapped to relations and stored directly as binary data in BLOBs. The queries executed were modelled after queries common in scientific applications and decision support.

RasDaMan is a universal — i.e., domain-independent — array DBMS for multidimensional arrays of arbitrary size and structure. A declarative, SQL-based array query language offers flexible retrieval and manipulation. Efficient server-based query evaluation is enabled by an intelligent optimizer and a streamlined storage architecture based on flexible array tiling and compression.

In the RasDaMan project a database system for management of multidimensional arrays is being built. It offers a declarative query language extending SQL-92 with operations on arrays of arbitrary base types and a C++ programming interface. Integrating arrays in the query language enables the system to process complex queries on high-volume multidimensional data in the database server close to physical data storage. Storage of arrays is done in tiles of arbitrary size. Operations on arrays are transformed into operations on tiles during query optimization and execution. These operations are then executed on tiles loaded from mass storage. This paper describes the underlying formal model for tile based operations on multidimensional arrays and its efficient implementation in C++ as part of the RasDaMan system.

Geometric modellers are used in a wide spectrum of applications in computer graphics. As many other applications use the generated internal representation of solids, semantic correctness is of special importance. On the other hand, the methodology of formal specification offers, in particular, the advantage of ensuring correctness by mathematical means. It is therefore a good idea to formally specify a geometric modeller. This paper describes an abstract data type defining a boundary representation. Special attention is paid to the operations for manipulating solids, i.e. the so-called Euler operations. The operation mfe (make face & edge) is used to show in detail how pre- and post-condition can be derived. Finally, mappings of this abstract data type onto implementations with different data models are discussed.