With the rapid expansion of large language model inference service users, cloud computing resource costs have become a critical challenge for service providers. Although utilizing end-device resources for auxiliary inference provides new possibilities to reduce cloud computing costs, existing solutions struggle to achieve an ideal balance across multi-task accuracy, end-to-end latency, and cloud computing costs.

Large-scale vision-language models enable powerful cross-modal understanding and generation, driving rapidly growing demand for online inference services. However, cloud-centric serving often suffers from high latency, rising costs, and network dependency, while purely on-device deployment is constrained by limited memory and reduced accuracy on complex tasks. To address this accuracy–latency–cost trilemma, we propose ShiftVL, a task-aware end–cloud serving framework that shifts suitable execution to the end device with a cloud fallback. ShiftVL serves high-frequency requests on an end-side small VLM enhanced with ViTexLoRA, a modality-disentangled parameter-efficient tuning method that preserves cross-modal alignment, while routing low-frequency or complex requests to a cloud-hosted large VLM for higher accuracy. Under tight device budgets, ShiftVL employs a predictive adapter scheduler that combines LRU-style caching with imitation learning to pre-load task-specific adapters. Experiments with InternVL models show that ShiftVL reduces cloud cost by up to 76.3% and latency by up to 42.9% while maintaining high multi-task accuracy, demonstrating its practicality for real-world vision-language model serving.

The rapid proliferation of IoT-enabled cameras has driven increasing demand for low-latency, intelligent video understanding in real-world applications such as smart cities and industrial automation. While Multimodal Large Language Models (MLLMs) offer unprecedented capabilities in semantic reasoning and natural language-based video comprehension, their deployment in latency-sensitive IoT environments remains challenging due to high computational costs and sequential decoding bottlenecks. Moreover, conventional edge-cloud video analysis frameworks often rely on semantic-agnostic frame sampling, leading to information loss or redundant data transmission. In this paper, we proposeSEC, a semantic-aware edge-cloud collaborative framework for efficient and accurate video analysis.SECintroduces a task-aware key frame selection mechanism at the edge to maximize semantic relevance while minimizing bandwidth usage, and a novel adaptive speculative decoding framework with tree-based parallel generation on the cloud to accelerate MLLM inference. Extensive experiments under realistic edge-cloud deployment settings on four video understanding benchmarks demonstrate that the proposedSECframework achieves superior performance, significantly reducing the end-to-end inference latency while improving the accuracy, a rare win-win in latency-critical IoT systems.

Recently, prompt-based methods have emerged as a new alternative ‘parameter-efficient fine-tuning’ paradigm, which only fine-tunes a small number of additional parameters while keeping the original model frozen. However, despite achieving notable results, existing prompt methods mainly focus on ‘what to add’, while overlooking the equally important aspect of ‘where to add’, typically relying on the manually crafted placement. To this end, we propose a region-based Adaptive Visual Prompt, named AdaViPro, which integrates the ‘where to add’ optimization of the prompt into the learning process. Specifically, we reconceptualize the ‘where to add’ optimization as a problem of regional decision-making. During inference, AdaViPro generates a regionalized mask map for the whole image, which is composed of 0 and 1, to designate whether to apply or discard the prompt in each specific area. Therefore, we employ Gumbel-Softmax sampling to enable AdaViPro’s end-to-end learning through standard back-propagation. Extensive experiments demonstrate that our AdaViPro yields new efficiency and accuracy trade-offs for adapting pre-trained models.

Recently, transformer models have demonstrated superior performance in video tasks. However, a prevalent limitation in most current video Transformers lies in their tendency to overlook inherent temporal regions of interest, such as motion trajectories, leading to susceptibility to redundant information during temporal modeling. Existing methods that pay attention to motion trajectories have high computational demands, lacking in lightweight efficiency. To strike a balance between effective modeling of temporal regions of interest and computational efficiency, we propose a video transformer backbone with deformable temporal attention (DTA). Inspired by the work on deformable receptive fields, DTA employs a lightweight decision network to enhance the flexibility of temporal attention. The decision network computes the offsets of tokens in the input feature map, enabling them to move to temporally relevant regions of interest and efficiently model temporal information. We conducted extensive experiments on three popular datasets and surpassed the baseline. Additionally, we performed ablation experiments specifically targeting the model structure and parameters. These results confirm the effectiveness of the proposed deformable temporal attention mechanism.

Video scene detection, an initial step of video analysis, temporally divides heterogeneous video into semantic segments, which is widely used in video summarization, search, browsing and retrieval. Video scene detection always cuts video into shots first and then groups these shots into segments. In this process, how to solve complex dependency relationship among shots is a barrier. The existing methods consider using Recurrent Neural Networks and Hidden Markov Model to simulate the dependency relationship between shots. However, linear approaches work a little on hierarchical video structure. In this paper, a GNN-based network is proposed to model complex structures of videos instead. Besides, the semantic gap between low-level features and high-level semantics is also a big obstacle of video scene detection. Here, three visual semantic elements in shot, i.e., environment, object, action and audio feature are extracted as shot representation. Later, we utilize a multi-modal fusion strategy, which combines early fusion and late fusion, to bridge the semantic gap between low-level features and high-level semantics. The proposed method was evaluated on BBC Planet Earth dataset and Open Video Scene Detection (OSVD) dataset, the experimental results demonstrate that the proposed method outperforms the state-of-the-art in video scene detection task.

Unsupervised anomaly detection in cloud computing is crucial for system security and efficiency. However, the challenges posed by large data volumes, low anomaly rates, and diverse anomaly patterns in time series within cloud computing scenarios make it difficult for previous methods to obtain consistent and reliable representations for distinguishing anomalies. To avoid the degradation of model representation ability caused by abnormal sparsity, we propose the Density Transformer, a novel reconstruction-based explicit association modeling model that can amplify the non-trivial correlation of abnormal points with adjacent time points. Specifically, we express the density association by calculating the kernel density estimate at each time point, and the series association by calculating the self-attention at each time point. Then, the model uses an adversarial training strategy to produce a more significant difference in "association discrepancy" between normal points and abnormal points, thereby ensuring robust results in anomaly detection. Our model has been rigorously evaluated on a comprehensive collection of 6 publicly available real-world datasets, and the Density Transformer can achieve up to 46% improvement in F1-score compared to existing methods.

Information cascade popularity prediction is critical for many applications, including but not limited to identifying fake news and accurate recommendations. Traditional feature-based methods heavily rely on handcrafted features, which are domain-specific and lack generalizability to new domains. To address this problem, researchers have turned to neural network-based approaches. However, most existing methods follow a sampling-based modeling approach, potentially losing continuous dynamic information that emerges during the information diffusion process. In this paper, we propose Hierarchical Temporal Graph Attention Networks for cascade popularity prediction (HierCas), which operates on the entire cascade graph by a dynamic graph modeling approach. By leveraging time-aware node embedding, graph attention mechanisms, and hierarchical pooling structures, HierCas effectively captures the popularity trend implicit in the complex cascade. Extensive experiments conducted on two real-world datasets in different scenarios demonstrate that our HierCas significantly outperforms the state-of-the-art approaches. We have released our code at https://github.com/Daisy-zzz/HierCas.

Vision-Language Models (VLMs) built on contrastive learning, such as CLIP, demonstrate great transferability and excel in downstream tasks like zero-shot classification and retrieval. To further enhance the performance of VLMs, existing methods have introduced additional parameter modules or fine-tuned VLMs on downstream datasets. However, these methods often fall short in scenarios where labeled data for downstream tasks is either unavailable or insufficient for fine-tuning, and the training of additional parameter modules may considerably impair the existing transferability of VLMs on open-set tasks. To alleviate this issue, we introduce WaveDN, a wavelet-based distribution normalization method that can boost the VLMs' performance on downstream tasks without parametric modules or labeled data. Initially, wavelet distributions are extracted from the embeddings of the sampled, unlabeled test samples. Subsequently, WaveDN conducts a hierarchical normalization across the wavelet coefficients of all embeddings, thereby incorporating the distributional characteristics of the test data. Finally, the normalized embeddings are reconstructed via inverse wavelet transformation, facilitating the computation of similarity metrics between the samples. Through extensive experiments on two downstream tasks, using a total of 14 datasets covering text-image and text-audio modal data, WaveDN has demonstrated superiority compared to state-of-the-art methods.

Masked image modeling (MIM), as a self-supervised learning paradigm in computer vision, has gained widespread attention among researchers. MIM operates by training the model to predict masked patches of the image. Given the sparse nature of image semantics, it is imperative to devise a masking strategy that steers the model towards reconstructing high-semantic regions. However, conventional mask strategies often miss these high-semantic regions or lack alignment with the masks and semantics. To solve this, we propose the Global Patch-wise Attention (GPA) framework, a transferable and efficient framework for MIM pre-training. We observe that the attention between patches can be the metric of identifying high-semantic regions, which can guide the model to learn more effective representations. Therefore, we firstly define the global patch-wise attention via vision transformer blocks. Then we design the soft-to-hard mask generation to guide the model gradually focusing on high semantic regions identified by GPA (GPA as a teacher). Finally, we design an extra task to predict GPA (GPA as a feature). Experiments conducted under various settings demonstrate that our proposed GPA framework enables MIM to learn better representations, which benefit the model across a wide range of downstream tasks. Furthermore, our GPA framework can be easily and effectively transferred to various MIM architectures.

Video Corpus Moment Retrieval aims to retrieve the relevant video from a large corpus and localize the corresponding moment within the target video based on a specific query. Existing methods have achieved promising accuracy and efficiency through dedicated model designs. However, we suggest these methods overly exploit dataset biases instead of semantics. This will lead to exaggerated performances on biased datasets and implicit a significant deficiency in generalizability, which is an important metric yet not considered in existing studies. In this paper, we observe the degradation caused by a spurious dependency and design a model to mitigate this harm. Specifically, we generate an Out-Of-Distributed (OOD) test set from a widely used TV Retrieval dataset, revealing the existing models' erroneous dependency on the temporal locations of target moments. Therefore, we utilize a theoretical Structural Causal Model (SCM) to dig into the roots of this dependency by constructing causal paths for the models. Furthermore, we propose a concrete Clip Location Deconfounding Model (CLDM) to disentangle the confounded video features into the content part and the location confounder part, then produce results with causal intervention. Experiments show that CLDM significantly alleviates the impact brought by dataset biases thus providing advanced generalizability among existing works.

Video popularity prediction, also known as predicting the future popularity of a video, is a task that generally uses various modalities of the video, such as visual, audio, text, and metadata, to make predictions. However, the traditional approach puts all the modalities directly into a model for processing, which introduces noise and redundancy into the model. To address this issue, we propose Cost-effective Modality Selection(CeMS) for video popularity prediction, which can adaptively select cost-effective modalities as input based on the features of the metadata. Specifically, a policy network first obtains the prior information from metadata to analyze the differences in representation ability and computation cost between various modalities. Then, with the policy estimation, an optimal set of cost-effective modalities is determined, and the backbone network corresponding to the selected modalities is activated for feature extraction. Finally, the semantic information of all selected modalities is fed into the decoder to yield the predicted popularity. Experiments demonstrate that our proposed approach yields 63%-69% reduction in computation when compared to the traditional baseline that simply uses all the modalities. Additionally, we achieve consistent improvements in accuracy over the state-of-the-art methods.

Recent adaptive methods for efficient video recognition mostly follow the two-stage paradigm of "preview-then-recognition" and have achieved great success on multiple video benchmarks. However, this two-stage paradigm involves two visits of raw frames from coarse-grained to fine-grained during inference (cannot be parallelized), and the captured spatiotemporal features cannot be reused in the second stage (due to varying granularity), being not friendly to efficiency and computation optimization.To this end, inspired by human cognition, we propose a novel recognition paradigm of "View while Moving" for efficient long-untrimmed video recognition.In contrast to the two-stage paradigm, our paradigm only needs to access the raw frame once.The two phases of coarse-grained sampling and fine-grained recognition are combined into unified spatiotemporal modeling, showing great performance.Moreover, we investigate the properties of semantic units in video and propose a hierarchical mechanism to efficiently capture and reason about the unit-level and video-level temporal semantics in long-untrimmed videos respectively.Extensive experiments on both long-untrimmed and short-trimmed videos demonstrate that our approach outperforms state-of-the-art methods in terms of accuracy as well as efficiency, yielding new efficiency and accuracy trade-offs for video spatiotemporal modeling.

Dynamic adaptive streaming over HTTP (DASH) has emerged as a popular Internet video service, constituting a growing fraction of LTE network traffic today. We identify the root causes of DASH performance problems in bit-ate stability, energy consumption of User Equipment (UE) and efficiency of bandwidth utilization, from both the users' and network operators' perspectives. Unlike the existing researches that separately studied two important performance metrics in DASH, i.e., Quality of Experience (QoE) and UEs' energy consumption. We propose an energy-aware DASH delivery framework over LTE networks (EDASH), jointly optimizing the network throughput, users' QoE and UEs' energy efficiency. We formulate the bandwidth allocation problem as a nonlinear integer program, and design the EDASH Online Allocation algorithm (EOA). EOA assigns bandwidth based on channel conditions and buffer occupancy of UEs to achieve efficient video delivery among multiple users. Furthermore, we present the detailed design and implementation of EDASH using Apache HTTP server and Android smartphones. Both simulation and experiment results reveal that our scheme can improve the network throughput while striking a better balance between users' QoE and UEs' energy consumption.

Web users are immersed in their roles as information producers and propagation pushers. They are unaware of being potential threats to privacy-protection towards themselves and their friends. It is necessary to know who they should beware of most in their friend-networks once their privacy information is divulged inadvertently. In this paper, we aim to identify the vulnerable friend who maximizes the dissemination of privacy information. First we develop a Privacy Receiving-Disseminating (PRD) model to simulate the iterative course of privacy information dissemination within social graph. The subgraph constituted of those users who are involved in the dissemination, called Ultimate Circle of Disseminating (UCD), is then detected by an iterative algorithm. The contribution of each direct friend could be evaluated by comparing the disseminating intensities of detected UCDs before and after unfriending himself. The performance of our work has been validated empirically with the comparison of different unfriending strategies.