Loss of plasticity is a phenomenon where neural networks can become more difficult to train over the course of learning. Continual learning algorithms seek to mitigate this effect by sustaining good performance while maintaining network trainability. We develop a new technique for improving continual learning inspired by the observation that the singular values of the neural network parameters at initialization are an important factor for trainability during early phases of learning. From this perspective, we derive a new spectral regularizer for continual learning that better sustains these beneficial initialization properties throughout training. In particular, the regularizer keeps the maximum singular value of each layer close to one. Spectral regularization directly ensures that gradient diversity is maintained throughout training, which promotes continual trainability, while minimally interfering with performance in a single task. We present an experimental analysis that shows how the proposed spectral regularizer can sustain trainability and performance across a range of model architectures in continual supervised and reinforcement learning settings. Spectral regularization is less sensitive to hyperparameters while demonstrating better training in individual tasks, sustaining trainability as new tasks arrive, and achieving better generalization performance..

Deep neural networks can struggle to learn continually in the face of non-stationarity, a phenomenon known as loss of plasticity. In this paper, we identify underlying principles that lead to plastic algorithms. We provide theoretical results showing that linear function approximation, as well as a special case of deep linear networks, do not suffer from loss of plasticity. We then propose deep Fourier features, which are the concatenation of a sine and cosine in every layer, and we show that this combination provides a dynamic balance between the trainability obtained through linearity and the effectiveness obtained through the nonlinearity of neural networks. Deep networks composed entirely of deep Fourier features are highly trainable and sustain their trainability over the course of learning. Our empirical results show that continual learning performance can be improved by replacing ReLU activations with deep Fourier features combined with regularization. These results hold for different continual learning scenarios (e.g., label noise, class incremental learning, pixel permutations) on all major supervised learning datasets used for continual learning research, such as CIFAR10, CIFAR100, and tiny-ImageNet.

Continual learning is often motivated by the idea, known as the big world hypothesis, that ``the world is bigger'' than the agent. Recent problem formulations capture this idea by explicitly constraining an agent relative to the environment. These constraints lead to solutions in which the agent continually adapts to best use its limited capacity, rather than converging to a fixed solution. However, explicit constraints can be ad hoc, difficult to incorporate, and may limit the effectiveness of scaling up the agent's capacity. In this paper, we characterize a problem setting in which an agent, regardless of its capacity, is constrained by being embedded in the environment. In particular, we introduce a computationally-embedded perspective that represents an embedded agent as an automaton simulated within a universal (formal) computer. Such an automaton is always constrained; we prove that it is equivalent to an agent that interacts with a partially observable Markov decision process over a countably infinite state-space. We propose an objective for this setting, which we call interactivity, that measures an agent's ability to continually adapt its behaviour by learning new predictions. We then develop a model-based reinforcement learning algorithm for interactivity-seeking, and use it to construct a synthetic problem to evaluate continual learning capability. Our results show that deep nonlinear networks struggle to sustain interactivity, whereas deep linear networks sustain higher interactivity as capacity increases.

We present the validation methodology and results of Dynamic Surface Water eXtent from Harmonized Landsat Sentinel-2 (DSWx-HLS). The DSWx-HLS product is the first of the DSWx suite, comprised of products each which map water from Earth Observation optical and SAR satellites. We detail the generation of high-resolution (3 m) validation datasets from a globally-stratified sample of dry, moderate, and wet sites. We provide the precise accounting of the classification metrics used to verify the Observational Products for End-users from Remote Sensing Analysis (OPERA) project requirements. We also report broader classification metrics across the validation datasets considered. OPERA performs validation in the public domain to ensure that the validation activities are transparent and reproducible. The resulting validation datasets and provisional OPERA products are publicly available; the software used for validation is also open-source.

This paper describes the concepts and recent activities of the HydroSAR project, a collaborative effort between the University of Alaska Fairbanks, the NASA Goddard and Marshall Space Flight Centers, and the Jet Propulsion Laboratory, focused on the automatic production of SAR-derived information for weather-related hazards. This paper will introduce the main processing algorithms as well as the broad product portfolio of the HydroSAR project, which includes information on surface water extent and surface water depth, in addition to capturing agriculture activity and other event-related changes. We will also highlight the cloud-based implementation of HydroSAR, which enables the system to generate data quickly and over large spatial scales. To demonstrate the performance of the service, we present results from a recent event response, where HydroSAR was used to support mapping activities related to the 2020 South Asia monsoon floods, inundating at least 25% of Bangladesh and large areas of the Bihar Province, India. In-region partner ICIMOD supported the performance assessment of the delivered products and algorithms.