In many domains, large amounts of time series data are being collected and analyzed in a semi-automatic manner. A prominent approach is the seasonal and trend decomposition using locally estimated scatterplot smoothing (STL) technique, which has been applied extensively in the past. However, STL quickly becomes computationally very expensive when applied to large data sets. In this work, we propose the first parallel implementation for the STL decomposition approach, which is tailored to the specific needs of graphics processing units (GPU). Our experimental evaluation on two global-scale case studies in temperature and vegetation trend analysis exhibits at least three-to-four orders of magnitude speed-up, demonstrating the effectiveness of the overall approach and the immense potential of the implementation in spatio-temporal data analyses. The source code is publicly available at https://github.com/diku-dk/hastl. An artifact that allows the experimental results to be reproduced is available at https://sid.erda.dk/sharelink/hOUrqJJ FfA.

Deep learning currently rules edge detection. However, the impressive progress heavily relies on high-quality manually annotated labels which require a significant amount of labor and time. In this study, we propose a novel unsupervised learning framework for deep edge detection. It adopts a gradient-based method to generate scale-dependent pseudo edge maps, which match with the hierarchical structure of deep networks. It leverages both the representation learning capability of deep learning, and the simplicity of traditional methods. Experiments on three popular data sets show that the proposed method can suppress non-object edges and reduce the gap with its supervised counterpart due to the introduction of information of various scales and smoothing strategy.

Knowledge of forest biomass stocks and their development is important for implementing effective climate change mitigation measures. Remote sensing using airborne LiDAR can be used to measure vegetation structure at large scale. We present deep learning systems for predicting wood volume, above-ground biomass (AGB), and subsequently above-ground carbon stocks directly from airborne LiDAR point clouds. Specifically, we devise different neural network architectures for point cloud regression and evaluate them on remote sensing data of areas for which AGB estimates have been obtained from field measurements in a national forest inventory. Our adaptation of Minkowski convolutional neural networks for regression gave the best results. The deep neural networks produced significantly more accurate wood volume, AGB, and carbon estimates compared to state-of-the-art approaches operating on basic statistics of the point clouds. In contrast to other methods, no digital terrain model is required. We expect this finding to have a strong impact on LiDAR-based analyses of terrestrial ecosystem dynamics.

Data are often accommodated on centralized storage servers. This is the case, for instance, in remote sensing and astronomy, where projects produce several petabytes of data every year. While machine learning models are often trained on relatively small subsets of the data, the inference phase typically requires transferring significant amounts of data between the servers and the clients. In many cases, the bandwidth available per user is limited, which then renders the data transfer to be one of the major bottlenecks. In this work, we propose a framework that automatically selects the relevant parts of the input data for a given neural network. The model as well as the associated selection masks are trained simultaneously such that a good model performance is achieved while only a minimal amount of data is selected. During the inference phase, only those parts of the data have to be transferred between the server and the client. We propose both instance-independent and instance-dependent selection masks. The former ones are the same for all instances to be transferred, whereas the latter ones allow for variable transfer sizes per instance. Our experiments show that it is often possible to significantly reduce the amount of data needed to be transferred without affecting the model quality much.

Canopy height is a vital indicator to asses carbon uptake and productivity of forests. However, precise measurements, such as from airborne or spaceborne 3D laser scanning (LiDAR), are expensive and usually cover only small areas. In this work, we propose a novel deep learning model that can generate detailed maps of tree canopy heights. In contrast to previous approaches that use a single image as input, we process multi-temporal data via a an adaptation of the popular U-Net architecture that is based on the EfficientNet and 3D convolution operators. To that end, our model receives multi-spectral Landsat satellite imagery as input and can predict continuous height maps. As labeled data, we resort to spatially sparse LiDAR data from ICESat-2. Thus, with such a model, one can produce dense canopy height maps given only multi-spectral Landsat data. Our experimental evaluation shows that our our model outperforms existing and improved single-temporal models. To test generalizability, we created a non-overlapping dataset to evaluate our approach and further tested the model performance on out-of-distribution data. The results show that our model can successfully learn drastic changes in distribution.

Feature fusion, the combination of features from different layers or branches, is an omnipresent part of modern network architectures. It is often implemented via simple operations, such as summation or concatenation, but this might not be the best choice. In this work, we propose a uniform and general scheme, namely attentional feature fusion, which is applicable for most common scenarios, including feature fusion induced by short and long skip connections as well as within Inception layers. To better fuse features of inconsistent semantics and scales, we propose a multiscale channel attention module, which addresses issues that arise when fusing features given at different scales. We also demonstrate that the initial integration of feature maps can become a bottleneck and that this issue can be alleviated by adding another level of attention, which we refer to as iterative attentional feature fusion. With fewer layers or parameters, our models outperform state-of-the-art networks on both CIFAR-100 and ImageNet datasets, which suggests that more sophisticated attention mechanisms for feature fusion hold great potential to consistently yield better results compared to their direct counterparts. Our codes and trained models are available online1.

Activation functions and attention mechanisms are typically treated as having different purposes and have evolved differently. However, both concepts can be formulated as a nonlinear gating function. Inspired by their similarity, we propose a novel type of activation units called attentional activation (ATAC) units as a unification of activation functions and attention mechanisms. In particular, we propose a local channel attention module for the simultaneous non-linear activation and element-wise feature refinement, which locally aggregates point-wise cross-channel feature contexts. By replacing the well-known rectified linear units by such ATAC units in convolutional networks, we can construct fully attentional networks that perform significantly better with a modest number of additional parameters. We conducted detailed ablation studies on the ATAC units using several host networks with varying network depths to empirically verify the effectiveness and efficiency of the units. Furthermore, we compared the performance of the ATAC units against existing activation functions as well as other attention mechanisms on the CIFAR-10, CIFAR-100, and ImageNet datasets. Our experimental results show that networks constructed with the proposed ATAC units generally yield performance gains over their competitors given a comparable number of parameters.

Neural networks have achieved dramatic improvements in recent years and depict the state-of-the-art methods for many real-world tasks nowadays. One drawback is, however, that many of these models are overparameterized, which makes them both computationally and memory intensive. Furthermore, overparameterization can also lead to undesired overfitting side-effects. Inspired by recently proposed magnitude-based pruning schemes and the Wald test from the field of statistics, we introduce a novel magnitude and uncertainty (M&U) pruning criterion that helps to lessen such shortcomings. One important advantage of our M&U pruning criterion is that it is scale-invariant, a phenomenon that the magnitude-based pruning criterion suffers from. In addition, we present a “pseudo bootstrap” scheme, which can efficiently estimate the uncertainty of the weights by using their update information during training. Our experimental evaluation, which is based on various neural network architectures and datasets, shows that our new criterion leads to more compressed models compared to models that are solely based on magnitude-based pruning criteria, with, at the same time, less loss in predictive power.

Massive amounts of satellite data have been gathered over time, holding the potential to unveil a spatiotemporal chronicle of the surface of Earth. These data allow scientists to investigate various important issues, such as land use changes, on a global scale. However, not all land-use phenomena are equally visible on satellite imagery. In particular, the creation of an inventory of the planet's road infrastructure remains a challenge, despite being crucial to analyze urbanization patterns and their impact. Towards this end, this work advances data-driven approaches for the automatic identification of roads based on open satellite data. Given the typical resolutions of these historical satellite data, we observe that there is inherent variation in the visibility of different road types. Based on this observation, we propose two deep learning frameworks that extend state-of-the-art deep learning methods by formalizing road detection as an ordinal classification task. In contrast to related schemes, one of the two models also resorts to satellite time series data that are potentially affected by missing data and cloud occlusion. Taking these time series data into account eliminates the need to manually curate datasets of high-quality image tiles, substantially simplifying the application of such models on a global scale. We evaluate our approaches on a dataset that is based on Sentinel 2 satellite imagery and OpenStreetMap vector data. Our results indicate that the proposed models can successfully identify large and medium-sized roads. We also discuss opportunities and challenges related to the detection of roads and other infrastructure on a global scale.

Choosing the best hyperparameters for neural networks is a big challenge. This paper proposes a method that automatically initializes and adjusts hyperparameters during the training process of stacked autoencoders. A population of autoencoders is trained with gradient-descent-based weight updates, while hyperparameters are mutated and weights are inherited in a Lamarckian kind of way. The training is conducted layer-wise, while each new layer initiates a new neuroevolutionary optimization process. In the fitness function of the evolutionary approach a dimensionality reduction quality measure is employed. Experiments show the contribution of the most significant hyperparameters, while analyzing their lineage during the training process. The results confirm that the proposed method outperforms a baseline approach on MNIST, FashionMNIST, and the Year Prediction Million Song Database.

In this work, we propose a new dimensionality reduction approach for generating low-dimensional embeddings of high-dimensional data based on an iterative procedure. The data set's dimensions are sorted depending on their variance. Starting with the highest variance, the dimensions are iteratively projected onto the embedding. The projection can be seen as taking a photo from a two-dimensional motive employing a depth effect. The approach is flexible and offers numerous extensions for future work. We introduce a basic variant and illustrate it working mechanisms with numerous visualizations. The approach is experimentally analyzed on a small set of benchmark problems. Exemplary embeddings and evaluations based on the Shepard-Kruskal measure and the co-ranking matrix complement the analysis. The new approach shows competitive results in comparison to well-established dimensionality reduction methods.

Virtual sensors are getting more and more important as replacement and quality control tool for expensive and fragile hardware sensors. We introduce a virtual sensor application with marine sensor data from two data sources. The virtual sensor models are built upon recurrent neural networks (RNNs). To take full advantage of past data, we employ the time dimensionality reduction method piecewise approximate aggregation (PAA). We present an extension of this method, called exponential PAA (ExPAA) that pulls finer details from recent values, but preserves less exact information about the past. Experimental results demonstrate that RNNs benefit from this extension and confirm the stability and usability of our virtual sensor models over a five-month period of multivariate marine time series data.

The imputation of partially missing multivariate time series data is critical for its correct analysis. The biggest problems in time series data are consecutively missing values that would result in serious information loss if simply dropped from the dataset. To address this problem, we adapt the k-Nearest Neighbors algorithm in a novel way for multivariate time series imputation. The algorithm employs Dynamic Time Warping as distance metric instead of point-wise distance measurements. We preprocess the data with linear interpolation to create complete windows for Dynamic Time Warping. The algorithm derives global distance weights from the correlation between features and consecutively missing values are penalized by individual distance weights to reduce error transfer from linear interpolation. Finally, efficient ensemble methods improve the accuracy. Experimental results show accurate imputations on datasets with a high correlation between features. Further, our algorithm shows better results with consecutively missing values than state-of-the-art algorithms.

Diary studies are often applied in HCI research to collect qualitative user impressions. Unfortunately, the period between creation of a diary entry and the later reflection can be too long, which leads to a limited currentness and contextuality. This eventually results in incomplete or misinterpreted data. In this paper we present Storyteller, a mobile application that allows a quick creation of diary entries and encourages users to reflect on these in-situ through a storytelling approach. We argue that this can lead to more accurate and substantial qualitative insights.