As the foundational component of versatile AI applications, training an multimodal large language model (MLLM) relies on multimodal datasets with dynamic modality mixture proportions and sample length distributions. However, existing MLLM systems remain inefficient under dynamic workloads, due to statically coupled decisions of resource allocation and model parallelization between encoders and the LLM backbone. This paper presents MegaScale-Omni, an industrial-grade MLLM training system tailored for dynamic workload adaption and hyper-scale deployment. MegaScale-Omni is built upon the training scheme of encoder-LLM multiplexing with three key innovations: (1) Decoupled parallelism strategies with long-short sequence parallelism for encoders to process variable-length samples, and full-fledged 5D parallelism for the LLM backbone, both organized under a communication-efficient parallelization layout. (2) Unified encoder-LLM representations for flexible, extensible colocation, and a new paradigm of encoder-LLM joint pipeline with workload resilience. (3) Workload balancing techniques via decentralized grouped reordering in data loaders and adaptive resharding from encoder to LLM ranks. MegaScale-Omni is deployed as the foundation of our in-house large-scale MLLM training tasks with thousands of GPUs. Our experimental results demonstrate 1.27×–7.57× throughput improvement under production-grade dynamic workloads, as compared to four state-of-the-art systems.

Extending the context length (i.e., the maximum supported sequence length) of LLMs is of paramount significance. To facilitate long context training of LLMs, sequence parallelism has emerged as an essential technique, which scatters each input sequence across multiple devices and necessitates communication to process the sequence. In essence, existing sequence parallelism methods assume homogeneous sequence lengths (i.e., all input sequences are equal in length) and therefore leverages a single, static scattering strategy for all input sequences. However, in reality, the sequence lengths in LLM training corpora exhibit substantial variability, often following a long-tail distribution, which leads to workload heterogeneity.

Few-shot object detection is a promising approach to solving the problem of detecting novel objects with only limited annotated data for training. Most existing methods are developed based on the progress in few-shot classification, which pay little attention to improving the localization module and modelling class interrelation. To address these issues, this paper proposes two novel modules, namely Region-interactive Proposal Network (Ri-PN) and Class-interactive Feature Learning (Ci-FL), for better localization and classification performance, respectively. In the Ri-PN, regions of novel classes are interacted with base classes via graph convolution instead of background due to the stronger relevance between base and novel classes together with the guidance of supervised regions loss. On the other hand, the Ci-FL refines class-specific features in prototypical learning by attentive graph convolutional network. Experimental results on PASCAL VOC and MS COCO datasets verify the superiority of our method for few-shot object detection.

Few-shot object detection is an important but challenging task where only a few instances of novel categories are available. The widely used approach is to pretrain a detector on base classes with abundant samples and then fine-tune it for novel classes. Due to the extreme data imbalance between base and novel classes, the detection performance of novel classes degrades with the distribution bias. To overcome this limitation, we propose a distribution calibration strategy and a class discrimination regularization method for better few-shot detection. Based on theoretical analysis on decision margins of base and novel classes, the decision area of novel classes is enlarged to balance the prediction probability. On the other hand, to increase the separability of inter-class distributions, the similarity between class-specific representations is minimized. Extensive experiments on PASCAL VOC and MS COCO datasets verify the effectiveness and generalization ability of our method to improve few-shot object detection.

Current state-of-the-art methods usually utilize feature pyramid to provide various receptive fields for detecting objects at different scales. However, the feature maps from low- to high-level layers have large semantic gaps and are with different spatial resolutions, so that their representational capacity differs and noise is introduced when fusing them. To overcome this limitation and carry out better object detection, we design a novel network named Receptive Field Pyramid Network (RFPN). The proposed method is derived based on a receptive field pyramid through dilated convolutions, such that all of the extracted feature maps are with strong semantics and the same resolution. Moreover, we propose a pyramid attention mechanism by iteratively leveraging information from previous receptive fields to give higher responses for objects of interest. Experimental results on publicly available datasets show that the proposed method achieves better results than existing methods for comparison.