Recommender systems traditionally represent items using unique identifiers (ItemIDs), but this approach struggles with large, dynamic item corpora and sparse long-tail data, limiting scalability and generalization. Semantic IDs, derived from multimodal content such as text and images, offer a promising alternative by mapping items into a shared semantic space, enabling knowledge transfer and improving recommendations for new or rare items. However, existing methods face two key challenges: (1) balancing cross-modal synergy with modality-specific uniqueness, and (2) bridging the semantic-behavioral gap, where semantic representations may misalign with actual user preferences. To address these challenges, we propose Multimodal Mixture-of-Quantization (MMQ), a two-stage framework that trains a novel multimodal tokenizer. First, a shared-specific tokenizer leverages a multi-expert architecture with modality-specific and modality-shared experts, using orthogonal regularization to capture comprehensive multimodal information. Second, behavior-aware fine-tuning dynamically adapts semantic IDs to downstream recommendation objectives while preserving modality information through a multimodal reconstruction loss. Extensive offline experiments and online A/B tests demonstrate that MMQ effectively unifies multimodal synergy, specificity, and behavioral adaptation, providing a scalable and versatile solution for both generative retrieval and discriminative ranking tasks.

In recent years, significant advancements in deep learning have expanded its application in a variety of computer vision tasks. However, the performance of these models heavily depends on the quality of the training data. While existing crowd counting methods yield satisfactory results on labeled datasets, they often face serious domain adaptation issues when applied to unlabeled data, the latter being more common in real-world scenarios. To mitigate this issue, we present a novel Scene-adaptive Unsupervised Crowd Counting (SUCC) framework aimed at enhancing the domain adaptability of counting models. This framework integrates a bi-branch attention network (BBA-Net) that leverages human prior knowledge to generate highly accurate density and anchor maps, which are essential for producing intermediate domain data as pseudo labels. Our SUCC framework eliminates the need for laborious manual annotation within the new data domain. Instead, it continually performs adaptive intermediate domain generation and model fine-tuning, establishing a beneficial feedback loop. Comprehensive experiments on multiple video crowd counting datasets show that our SUCC framework significantly improves domain generalizability. Furthermore, it exhibits satisfactory model stability and algorithm interpretability, attributes that are vital for the practical deployment of counting applications. The open-source code and model weights can be found on Github.

The burgeoning field of Multimodal Large Language Models (MLLMs) has exhibited remarkable performance in diverse tasks such as captioning, commonsense reasoning, and visual scene understanding. However, the deployment of these large-scale MLLMs on client devices is hindered by their extensive model parameters, leading to a notable de-cline in generalization capabilities when these models are compressed for device deployment. Addressing this chal-lenge, we introduce a Cloud-Device Collaborative Contin-ual Adaptation framework, designed to enhance the performance of compressed, device-deployed MLLMs by lever-aging the robust capabilities of cloud-based, larger-scale MLLMs. Our framework is structured into three key components: a device-to-cloud uplink for efficient data transmission, cloud-based knowledge adaptation, and an optimized cloud-to-device downlink for model deployment. In the up-link phase, we employ an Uncertainty-guided Token Sam-pling (UTS) strategy to effectively filter out-of-distribution tokens, thereby reducing transmission costs and improving training efficiency. On the cloud side, we propose Adapter-based Knowledge Distillation (AKD) method to transfer refined knowledge from large-scale to compressed, pocket-size MLLMs. Furthermore, we propose a Dynamic Weight update Compression (DWC) strategy for the down-link, which adaptively selects and quantizes updated weight parameters, enhancing transmission efficiency and reducing the representational disparity between cloud and de-vice models. Extensive experiments on several multimodal benchmarks demonstrate the superiority of our proposed framework over prior Knowledge Distillation and device-cloud collaboration methods. Notably, we also validate the feasibility of our approach to real-world experiments.

Blind watermarking is a technology for image copyright protection and digital fingerprinting. However, the introduction of non-differentiable noise makes it challenging to be trained end-to-end for black-box scenes. The phased training technique is used for coping with black-box noise, but it limits the performance since the encoder and decoder cannot be end-to-end optimized. This work proposes a blind watermarking framework CIN+ based on the CIN to address black-box noise. Combining the structural characteristics of an Invertible Neural Network (INN) with the two-stage strategy allows joint updates of the encoder and decoder when encountering non-differentiable noise. We utilize Noise and Gradient Propagation Gate (NGPG) modules to perform a batch-level optimization akin to the two-stage approach, allowing encoder parameters to remain unlocked, thereby enhancing the model’s ability to resist blackbox attacks. Additionally, a Pre-Extraction Module (PEM) is introduced to simplify the complexity and usability of CIN. Our experimental results reveal that CIN+ achieves a new state-of-the-art performance.