Publications proposing novel machine learning methods are often primarily rated by exhibited predictive performance on selected problems. In this position paper we argue that predictive performance alone is not a good indicator for the worth of a publication. Using it as such even fosters problems like inefficiencies of the machine learning research community as a whole and setting wrong incentives for researchers. We therefore put out a call for the publication of “negative” results, which can help alleviate some of these problems and improve the scientific output of the machine learning research community. To substantiate our position, we present the advantages of publishing negative results and provide concrete measures for the community to move towards a paradigm where their publication is normalized.

This paper presents the MLAB project, a research and development activity funded by ESA General Support Technology Programme under the lead of Airbus Defence and Space GmbH, with the goal of developing a machine learning application benchmark for space applications. First, the need for a benchmark dedicated to machine learning applications in spacecraft is explained, and examples of applications are described including their design challenges. Then the benchmark design is presented, including the rules of the metrics, guidelines and scenarios for references. These scenarios include a description of the reference workloads that have been selected during the activity as representative for spacecraft applications. Lastly, the submission concept is introduced.

Compute in space, e.g., in miniaturized satellites, requires dealing with special physical and boundary constraints, including the limited energy budget. These constraints impose strict operational conditions on the on-board data processing system and its capability in dealing with sophisticated workloads suchlike Machine Learning (ML). In the meantime, the breakthroughs in ML based on Deep Neural Networks (DNNs) in the last decade promise innovative solutions to expand the functional capabilities of on-board data processing and to drive the space industry forward. Therefore, due to the aforementioned special requirements, performance- and power-efficient, and novel solutions and architectures for deploying ML via, e.g., FPGA-enabled SoC, particularly Commercial-Off-The-Shelf (COTS) solutions, are gaining significant interest in the space industry. Therefore it is essential to conduct extensive benchmarking and feasibility and efficiency analyses in different aspects: such analyses would require the investigation of options for programming and deployment as well as the investigation of various real-world models and datasets. To this end, a research and development activity is funded by the European Space Agency (ESA) General Support Technology Programme and is led by Airbus Defence and Space GmbH with the goal of developing an ML Application Benchmark (MLAB) that covers benchmarking aspects mentioned above.

Computer vision has seen some breakthroughs in the last decade based on some methodological advances, but as well based on the availability of huge datasets like ImageNet for training. However, training data is generally scarce in remote sensing and even more in high-resolution or high-quality remote sensing of sensitive areas. Some efforts have been made to provide labeled public domain data, but aside lowresolution data, these activities are not sufficient yet. In this paper, we propose an alternative approach: we transform satellite images into a representation in which features learnt from Internet photography are more meaningful. We show how learnt colorspace transformations can enable significantly more stable transfer learning from ImageNet. As a consequence, small training datasets suffice allowing for significantly more diverse Earth observation applications. We present experiments on high-resolution remote sensing images of airplanes as featured in the 2020 Gaofen Challenge on Automated High-Resolution Earth Observation Image Interpretation.

In this paper, we present a novel approach for trajectory similarity based on Kolmogorov complexity approximated by a lossy compression of the original trajectory data using selected features compressed into a concise memory representation by means of a Bloom filter. Given the importance of trajectory data, a linear-time distance measure with all theoretical guarantees implied by a proper metric is very powerful if it is capturing enough detail for important trajectory mining tasks. This stack of feature extraction combined with feature embedding in a Bloom filter constitutes a lossy compression for trajectory data which can easily be extended with other discrete data like travel mode. In addition, this compression has the needed properties for efficient calculation of a normalized compression distance (NCD) which approximates Kolmogorov complexity. We evaluate this novel trajectory distance measurement using very simple features and k-nearest-neighbor classification on selected realworld datasets with remarkable classification accuracies. Furthermore, we argue that the distance measure is very suited to geospatial big data applications as each trajectory is first transformed into few bits using the lossy compression stack. At time of comparison, the original trajectory geometry is not needed, instead, the sketches suffice. Despite very compressible parameters (equal or less than 1024 bit per trajectory) and very simple features, we already reach classification accuracies for real-world trajectory classification tasks of more than 80% across various datasets.