Monolithic large language models (LLMs) like GPT-4 have paved the way for modern generative AI applications. Training, serving, and maintaining monolithic LLMs at scale, however, remains prohibitively expensive and challenging. The disproportionate increase in compute-to-memory ratio of modern AI accelerators have created a memory wall, necessitating new methods to deploy AI. Recent research has shown that a composition of many smaller expert models, each with several orders of magnitude fewer parameters, can match or exceed the capabilities of monolithic LLMs. Composition of Experts (CoE) is a modular approach that lowers the cost and complexity of training and serving. However, this approach presents two key challenges when using conventional hardware: (1) without fused operations, smaller models have lower operational intensity, which makes high utilization more challenging to achieve; and (2) hosting a large number of models can be either prohibitively expensive or slow when dynamically switching between them. In this paper, we describe how combining CoE, streaming dataflow, and a three-tier memory system scales the AI memory wall. We describe Samba-CoE, a CoE system with 150 experts and a trillion total parameters. We deploy Samba-CoE on the SambaNova SN40L Reconfigurable Dataflow Unit (RDU) -a commercial dataflow accelerator architecture that has been codesigned for enterprise inference and training applications. The chip introduces a new three-tier memory system with on-chip distributed SRAM, on-package HBM, and off-package DDR DRAM. A dedicated inter-RDU network enables scaling up and out over multiple sockets. We demonstrate speedups ranging from 2× to 13× on various benchmarks running on eight RDU sockets compared with an unfused baseline. We show that for CoE inference deployments, the 8-socket RDU Node reduces machine footprint by up to 19 ×, speeds up model switching time by 15× to 31×, and achieves an overall speedup of 3.7× over a DGX H100 and 6.6× over a DGX A100.

In this paper we propose a two-stage lattice Monte Carlo approach for optimization of bimetallic nanoalloys: simulated annealing on a larger lattice, followed by simulated diffusion.Both algorithms are fairly similar in structure, but their combination was found to give significantly better solutions than simulated annealing alone.We also discuss how to tune the parameters of the algorithms so that they work together optimally.

We propose a novel architecture for face and emotion recognition and discuss modifications for different types of mobile applications. Emotion recognition task is challenging due to the absence of large-scale datasets and non-uniform labelling. We propose easy-to-implement five class classification approach and suggest modifications for large-scale emotion recognition on three different platforms: desktop, mobile and VPU, and compare the resulting speed and performance. We demonstrate that our results can be compared with the performance of state-of-the-art neural networks and can be implemented on mobile and stand-alone devices.

We propose a cross-platform application for emotion recognition. The application is based on convolutional neural network (CNN) model and adopted for usage on smartphones and tablets. The system is capable of recognition of human emotions on arousal-valence scale.