We introduce a new training paradigm that enforces interval constraints on neural network parameter space to control forgetting. Contemporary Continual Learning (CL) methods focus on training neural networks efficiently from a stream of data, while reducing the negative impact of catastrophic forgetting, yet they do not provide any firm guarantees that network performance will not deteriorate uncontrollably over time. In this work, we show how to put bounds on forgetting by reformulating continual learning of a model as a continual contraction of its parameter space. To that end, we propose Hyperrectangle Training, a new training methodology where each task is represented by a hyperrectangle in the parameter space, fully contained in the hyperrectangles of the previous tasks. This formulation reduces the NP-hard CL problem back to polynomial time while providing full resilience against forgetting. We validate our claim by developing InterContiNet (Interval Continual Learning) algorithm which leverages interval arithmetic to effectively model parameter regions as hyperrectangles. Through experimental results, we show that our approach performs well in a continual learning setup without storing data from previous tasks.

Most continual learning evaluations to date have focused on fully supervised image classification problems. This work for the first time extends such an analysis to the domain of 3D point cloud generation, showing that 3D object generators are prone to catastrophic forgetting along the same vein as image classifiers. Classic mitigation techniques, such as regularization and replay, are only partially effective in alleviating this issue. We show that due to the specifics of generative tasks, it is possible to maintain most of the generative diversity with a simple technique of uniformly sampling from different columns of a progressive neural network. While such an approach performs well on a typical synthetic class-incremental setup, more realistic scenarios might hinder strong concept separation by shifting task boundaries and introducing class overlap between tasks. Therefore, we propose an autonomous branch construction (ABC) method. This learning adaptation relevant to parameter-isolation methods employs the reconstruction loss to map new training examples to proper branches of the model. Internal routing of training data allows for a more effective and robust continual learning and generation of separate concepts in overlapping task setups.

One of the obstacles in research activities concentrating on environmental sound classification is the scarcity of suitable and publicly available datasets. This paper tries to address that issue by presenting a new annotated collection of 2000 short clips comprising 50 classes of various common sound events, and an abundant unified compilation of 250000 unlabeled auditory excerpts extracted from recordings available through the Freesound project. The paper also provides an evaluation of human accuracy in classifying environmental sounds and compares it to the performance of selected baseline classifiers using features derived from mel-frequency cepstral coefficients and zero-crossing rate.