Array databases specialize in storage, management, and query processing on massive multidimensional array data such as satellite image time-series, weather forecast models, IoT sensor measurements, medical imaging data, etc. Some of the values in this data may be ”null” for a variety of reasons, such as unknown, known false, etc. The DBMS must handle null values correctly and efficiently. As of today, the effects of different data structures for representing null values on query processing in array databases have not been systematically studied. As a consequence, it is not clear what the optimal way for handling null values is. In this paper, we measure how four different methods for encoding null values perform across most common categories of array operations. The result is a comprehensive overview, publicly available in open source, providing relevant insights into the performance characteristics of these data structures allowing to discern which among them offers the most optimal approach for representing null values in an array processing context.

Server-side extensibility through dynamically linked external code is a common method in relational databases. In the field of Array Databases such User-Defined Functions (UDFs) sometimes even represent the architectural cornerstone for array functionality. On the downside, UDF implementation often suffers from high coding complexity.The rasdaman Array DBMS is a full-stack C++ implementation, so does not rely on some generic UDF mechanism. This allowed designing such an API from scratch, with particular emphasis on UDF coder convenience. The rasdaman UDFs rely on the general C++ client API classes. Based on a straightforward UDF interface definition adapter code is generated automatically. Experimental evaluation shows encouraging results, and the mechanism is going to be used in research and under operational conditions. We present the approach and motivate it through practical use cases.

The European Copernicus programme generates massive amounts of Earth Observation (EO) data, with the goal of improving our environmental understanding and management on local, regional, and global level. An initiative of this magnitude comes with a set of challenges, especially pertaining to effective service management once the huge volumes of raster data files are distributed across open data centers and commercial companies.The BigDataCube project responds to this challenge through the concept of federated, analysis-ready datacubes, exposed via the open OGC geo standard interfaces for interoperable access and processing, WMS, WCS, and WCPS.Such federated datacube services have been established in the project between the public German Copernicus hub, CODE-DE, and a commercial cloud provider, cloudeo AG, altogether offering access to more than 500 TB. Both data pools are federated in a location-transparent manner, establishing a common information space where users can query and combine datacubes without knowing their location of storage. In this nucleus of a growing federation recently the Alfred Wegener maritime research institute has joined, and further data centers are in the line. The spectrum of functionality available is showcased through several realistic use cases, including a value-adding Sentinel-1 SAR product on sea state and wind speed in the North Sea. The platform used is the pioneer datacube engine and OGC reference implementation, rasdaman. We report on the outcomes of the project, lessons learned, and further work foreseen in this area.

We demonstrate the rasdaman ("raster data manager") scalable datacube engine in a series of multi-dimensional live scenarios of spatio-temporal datacube analytics, distributed processing in federations, as well as simple, rapid construction of datacubes.

With the unprecedented availability of continuously observed and generated data there is a likewise unprecedented potential for new and timely insights; yet, benefits are not fully leveraged as of today. The plethora of formats in combination with heterogeneous services remains is an obstacle - e.g., image services prefer binary formats, SPARQL endpoints like to think in RDF triples, and browsers integrate JSON data smoothly. We propose a model-based multi-encoding approach for overcoming the limitations of individual formats while still supporting their use. Concretely, this approach is being followed by the OGC Coverage Implementation Schema (CIS) standard which establishes a concrete, interoperable data model unifying n-D spatiotemporal regular and irregular grids, point clouds, and meshes. We describe how independence from data formats is achieved, in particular for three practically relevant formats - XML, JSON, and RDF -, thereby fostering integration of hitherto rather separate application domains.

Array databases are used to manage and query large N-dimensional arrays, such as sensor data, simulation models and imagery, as well as various time-series. Modern database systems and database applications make extensive use of caching techniques to improve performance. Research on array databases on the other hand has not explored the potential benefits of caching in query processing on big arrays. In this work we propose a design for a content-aware cache for array databases which allows to reuse results of previously evaluated queries. Besides identical query matching, our method also takes into account spatially overlapping queries and queries with common subexpressions. We evaluate performance of the query cache implementation by varying data and query parameters and show that it decreases query execution time by up to 93%, with a potential for even higher savings with increasing query complexity.

This contribution introduces the forthcoming extension of the ISO SQL standard for multi-dimensional arrays, SQL/MDA. We present concepts, the language, and highlight how it can be implemented in a scalable manner. Examples used stem from Earth Observation and related domains.

Support for large arrays has been increasingly gaining attention by the database community. Array databases are a quickly expanding category of database management systems that treat large, multidimensional array data as first-class database citizens, allowing convenient and efficient storage and retrieval. Large array data on its own, however, is rarely meaningful or machine understandable. Therefore, arrays are almost always linked to additional, small metadata, typically managed in relational databases. This leads to an information integration gap, an "impedance mismatch" that we address with the work presented in this paper.

Arrays are among those data types which contribute the most to Big Data -- examples include satellite images and weather simulation output in the Earth sciences, confocal microscopy and CAT scans in the Life sciences, as well as telescope and cosmological observations in Space science, to name but a few. Traditionally, the database community has neglected this, with the effect that ad-hoc implementations prevail. With the advent of NewSQL in recent years, however, the database scope has broadened, and array modelling and query support is seriously considered. Different models have been suggested, some of which are implemented or under implementation, and a consolidation of concepts can be observed. Consequently, integration of array queries into SQL is being addressed.

In the era of Big Data, being able to work with multidimensional arrays in a robust and consistent manner as the number and variety of dimensions increase, is just as important as being able to handle the large volumes inherent to this type of data. Usually, array analytics is carried out to extract meaningful information for further applications, e.g. slicing and subsetting. While domain-specific dimensions, which are beyond spatio-temporal, underlie rich domain anchor semantics, assigning consistent dimension schema for Points Of Interest (POI) across multidisciplinary data sets is challenging. New compositions of CRSs need to be constructed on the fly by a heterogeneous community with different backgrounds and applications in mind, consequently, linking dimensions via different resolvers to drive away dimension fragments from high-dimensional spaces. We propose to identify dimensions via a linked resolver approach. Such an approach allows CRSs to be referred to and looked up across multidisciplinary applications. Finally, we present a planetary use case, and specification- and scenario-based testing results to validate our approach.