In Continual Learning (CL), a model is required to learn a stream of tasks sequentially without significant performance degradation on previously learned tasks. Current approaches fail for a long sequence of tasks from diverse domains and difficulties. Many of the existing CL approaches are difficult to apply in practice due to excessive memory cost or training time, or are tightly coupled to a single device. With the intuition derived from the widely applied mini-batch training, we propose Batch Model Consolidation (BMC) to support more realistic CL under conditions where multiple agents are exposed to a range of tasks. During a regularization phase, BMC trains multiple expert models in parallel on a set of disjoint tasks. Each expert maintains weight similarity to a base model through a stability loss, and constructs a buffer from a fraction of the task's data. During the consolidation phase, we combine the learned knowledge on 'batches' of expert models using a batched consolidation loss in memory data that aggregates all buffers. We thoroughly evaluate each component of our method in an ablation study and demonstrate the effectiveness on standardized benchmark datasets Split-CIFAR-100, Tiny-ImageNet, and the Stream dataset composed of 71 image classification tasks from diverse domains and difficulties. Our method outperforms the next best CL approach by 70% and is the only approach that can maintain performance at the end of 71 tasks.

This paper describes an on-going project to develop diagnostic profiling tools for healthcare professionals to identify potential speech and language developmental problems of Chinese-speaking children. The tools aim to provide a technical advance in helping children who may have speech impairment or language delay. The project is currently being carried out as a collaboration with Chinese healthcare professionals, and a multidisciplinary team including applied linguists, speech and language pathologists, and computer scientists (staff and students) at the first Sino-foreign higher education institution in China, University of Nottingham Ningbo China (UNNC). The paper presents the background, development, and current state of the project. Challenges to project completion, including difficulties encountered due to the Covid-19 outbreak and subsequent world-wide emergency and lock-down are also discussed.