Speech large language models (LLMs) have emerged as a prominent research focus in speech processing.In this work, we introduce VocalNet, a series of high-performance speech LLMs featuring a scalable and model-agnostic training framework as well as a novel multitoken prediction (MTP) paradigm for speech generation.We first propose an efficient twostage training framework that enables LLMs to acquire real-time speech interaction capabilities.Through extensive experiments on various training configurations, we ensure both simplicity and effectiveness in the training strategy.Furthermore, inspired by advances in language modeling, we introduce MTP into the domain of speech LLMs-an alternative to traditional next-token prediction (NTP)-which enables the model to predict multiple future tokens at each step.Through systematic analysis and improved implementation, we show that MTP not only accelerates inference speed but also significantly enhances speech quality.Experimental results demonstrate that VocalNet achieves performance comparable to state-ofthe-art Omni LLMs while outperforming existing open-source speech LLMs, despite using limited training data.

Dynamic scene reconstruction poses a persistent challenge in 3D vision. Deformable 3D Gaussian Splatting has emerged as an effective method for this task, offering real-time rendering and high visual fidelity. This approach decomposes a dynamic scene into a static representation in a canonical space and time-varying scene motion. Scene motion is defined as the collective movement of all Gaussian points, and for compactness, existing approaches commonly adopt implicit neural fields or sparse control points. However, these methods predominantly rely on gradient-based optimization for all motion information. Due to the high degree of freedom, they struggle to converge on real-world datasets exhibiting complex motion. To preserve the compactness of motion representation and address convergence challenges, this paper proposes heterogeneous 3D control points, termed \textbf{H3D control points}, whose attributes are obtained using a hybrid strategy combining optical flow back-projection and gradient-based methods. This design decouples directly observable motion components from those that are geometrically occluded. Specifically, components of 3D motion that project onto the image plane are directly acquired via optical flow back projection, while unobservable portions are refined through gradient-based optimization. Experiments on the Neu3DV and CMU-Panoptic datasets demonstrate that our method achieves superior performance over state-of-the-art deformable 3D Gaussian splatting techniques. Remarkably, our method converges within just 100 iterations and achieves a per-frame processing speed of 2 seconds on a single NVIDIA RTX 4070 GPU.

Efficiently compressing HD/UHD content has long been challenging due to high bitrate costs. Instance-adaptive enhancement methods try to tackle this issue by compressing a video at reduced resolution and enhancing it using a neural model specifically overfitted for this video. However, existing methods focus solely on spatial super-resolution (SR) and under-utilize the videos' temporal redundancy. Their limited management of the model's updated parameters also causes excessive overfitting overheads. Therefore, this paper introduces IASTE, the first instance-adaptive enhancement method based on spatial-temporal enhancement (STE), and incorporates low-rank adaptation (LoRA) for efficient model overfitting. Specifically, we downscale videos spatially and temporally to reduce the data volume and achieve efficient video compression. Then, we overfit a specific STE model for each video and use it to enhance the decoded video's spatiotemporal resolution. Leveraging the video swin transformer's strong capability in capturing spatiotemporal correlations, we design a lightweight and efficient model to implement video STE. The model is overfitted for each video using LoRA. By freezing the pre-trained model and selectively updating a few low-rank matrices, the bitrate overhead for model storage can be mitigated. Experiments prove that compared to directly compressing high-frame-rate (HFR), high-resolution (HR) videos, our method achieves around 30% BD-Rate gains on the CTC and UVG datasets, about 15% gains on the YoutubeUGC dataset, and about 10% gains on the ultra-long videos in the Xiph dataset.

Lossless video coding is used when perfect preservation of video data is required. In HEVC, lossless coding is accomplished by bypassing the transform and quantization stages. Prediction residuals are coded with the entropy coder in the spatial domain. In this paper, two-stage coding with sub-frame adaptive quantization is proposed. The DCT is firstly applied to the prediction residuals and the DCT coefficients are quantized. The quantized DCT coefficients and the quantization error are coded. Adaptive quantization parameters are used for each Coding Unit. Simulation results show that the proposed method significantly outperforms the HEVC lossless coding.