The generalization capability of deepfake detectors is crucial for real-world applications. Data augmentation to generate synthetic fake faces has served as an effective strategy to enhance generalization. Interestingly, current state-of-the-art (SoTA) methods rely on fixed augmentation strategies, raising a fundamental question: Can a single static augmentation approach suffice, or does the diversity of forgery features necessitate dynamic strategies? We argue that existing methods overlook the evolving complexity of real-world forgery patterns, such as facial warping, expression manipulation, and compression artifacts, which cannot be fully simulated by fixed policies. To bridge this gap, we propose CRDA (Curriculum Reinforcement-Learning Data Augmentation), a novel framework that guides the detector to progressively master multi-domain forgery features from simple to complex. CRDA synthesizes augmented samples using a configurable pool of forgery operations and dynamically generates adversarial samples tailored to the detector’s current learning state. Key to our approach is the integration of reinforcement learning (RL) and causal inference. To efficiently explore the vast augmentation space, an RL agent dynamically selects augmentation actions based on the detector’s performance, ensuring continuous adaptation to increasingly challenging forgeries. Simultaneously, the agent’s output is designed to introduce variations in action spaces, generating heterogeneous forgery patterns. These variations are guided by causal inference theory, which mitigates spurious correlations by suppressing task-irrelevant biases and enforcing the model to focus on causally invariant features. This integration ensures robust generalization by decoupling synthetic augmentation patterns from the model’s learned representations. Extensive experiments demonstrate that the proposed method significantly improves the generalizability of the detector, achieving superior performance compared to state-of-the-art methods on multiple cross-domain datasets.

Composed Image Retrieval (CIR) enables image search by combining a reference image with modification text. Intrinsic noise in CIR triplets incurs intrinsic uncertainty and threatens model's robustness. Probabilistic learning approaches have shown promise in addressing such issues; however, they fall short for CIR due to their instance-level holistic modeling and homogeneous treatments for queries and targets. This paper introduces a Heterogeneous Uncertainty-Guided (HUG) paradigm to overcome these limitations. HUG utilizes a fine-grained probabilistic learning framework, where queries and targets are represented by Gaussian embeddings capturing detailed concepts and uncertainties. We customize heterogeneous uncertainty estimations for multi-modal queries and uni-modal targets. Given a query, we capture uncertainties not only regarding uni-modal content quality but also multi-modal coordination, followed by a provable dynamic weighting mechanism to derive the comprehensive query uncertainty. We further design uncertainty-guided objectives, including query-target holistic contrast and fine-grained contrasts with comprehensive negative sampling strategies, which effectively enhance discriminative learning. Experiments on benchmarks demonstrate HUG's effectiveness beyond state-of-the-art baselines, with faithful analysis justifying the technical contributions.

Deep learning-based 3D anomaly detection methods have demonstrated significant potential in industrial manufacturing. However, many approaches are specifically designed for anomaly detection tasks, which limits their generalizability to other 3D tasks. In contrast, self-supervised point cloud models aim for general representation learning, yet our investigation reveals that these classical models are suboptimal at anomaly detection under the unified fine-tuning paradigm. This motivates us to develop a more generalizable 3D model that can effectively detect anomalies without relying on task-specific designs. Interestingly, we find that using only the curvature of each point as its anomaly score already outperforms several classical self-supervised and dedicated anomaly detection models, highlighting the critical role of curvature in 3D anomaly detection. In this paper, we propose a Curvature-Augmented Self-supervised Learning (CASL) framework based on a reconstruction paradigm. Built upon the classical U-Net architecture, our approach introduces multi-scale curvature prompts to guide the decoder in predicting the coordinates of each point. Without relying on any dedicated anomaly detection mechanisms, it achieves leading detection performance through straightforward anomaly classification fine-tuning. Moreover, the learned representations generalize well to standard 3D understanding tasks such as point cloud classification.

Hao Fang, Xiaohang Sui, Hongyao Yu, Kuofeng Gao, Jiawei Kong, Sijin Yu, Bin Chen, Shu-Tao Xia. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2026.

Niu Lian, Yuting Wang, Hanshu Yao, Jinpeng Wang, Bin Chen, Yaowei Wang, Min Zhang, Shu-Tao Xia. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2026.

Xiaoxiao Ma, Kuofeng Gao, Zeyi Lu, Wenxi Jiang, Hao Fang, Hao Wu, Bin Chen, Shu-Tao Xia. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2026.

Maximum distance separable (MDS) codes are widely used in distributed storage, but naively repairing a single failure in an $(n,k)$ MDS code requires downloading the full contents of $k$ surviving nodes. Minimum storage regenerating (MSR) codes, introduced by Dimakis et al., minimize repair bandwidth while preserving the MDS property by contacting $d>k$ helper nodes and downloading only a fraction of each helper. For scalar MDS codes, Guruswami and Wootters established a linear repair framework, and Tamo, Ye, and Barg subsequently gave the first explicit Reed-Solomon (RS) codes achieving the MSR point. Their construction yields RS-MSR codes with subpacketization $\ell=s\prod_{i=1}^n p_i$, where $s=d+1-k$ and the distinct primes $p_i$ satisfy $p_i\equiv 1\pmod{s}$. In this paper, we show that this congruence condition is not intrinsic to the RS repair problem. We develop a basis-transformation approach to the construction of repair-enabling subspaces. The approach consists of three deterministic operations -- Euclidean Square Partition, Transposition, and Column Aggregation -- which construct the required repair-enabling subspaces directly from the standard monomial basis of the repair field. Consequently, we obtain RS-MSR codes with subpacketization $\ell=s\prod_{i=1}^n p_i$ for arbitrary distinct primes $p_i>s$. For fixed $s$, this improves the subpacketization of the Tamo--Ye--Barg construction by a factor asymptotic to $φ(s)^{n+\mathrm{o}(n)}$, where $φ(\cdot)$ denotes Euler's totient function.

Transformer channel decoders, such as the Error Correction Code Transformer (ECCT), have shown strong empirical performance in channel decoding, yet their generalization behavior remains theoretically unclear. This paper studies the generalization performance of ECCT from a learning-theoretic perspective. By establishing a connection between multiplicative noise estimation errors and bit-error-rate (BER), we derive an upper bound on the generalization gap via bit-wise Rademacher complexity. The resulting bound characterizes the dependence on code length, model parameters, and training set size, and applies to both single-layer and multi-layer ECCTs. We further show that parity-check-based masked attention induces sparsity that reduces the covering number, leading to a tighter generalization bound. To the best of our knowledge, this work provides the first theoretical generalization guarantees for this class of decoders.

Model Inversion Attacks (MIAs) pose a significant threat to data privacy by reconstructing sensitive training samples from the knowledge embedded in trained machine learning models. Despite recent progress in enhancing the effectiveness of MIAs across diverse settings, defense strategies have lagged behind—struggling to balance model utility with robustness against increasingly sophisticated attacks. In this work, we propose the ideal inversion error to measure the privacy leakage, and our theoretical and empirical investigations reveals that higher-rank features are inherently more prone to privacy leakage. Motivated by this insight, we propose a lightweight and effective defense strategy based on low-rank feature filtering, which explicitly reduces the attack surface by constraining the dimension of intermediate representations. Extensive experiments across various model architectures and datasets demonstrate that our method consistently outperforms existing defenses, achieving state-of-the-art performance against a wide range of MIAs. Notably, our approach remains effective even in challenging regimes involving high-resolution data and high-capacity models, where prior defenses fail to provide adequate protection. The code is available at https://github.com/Chrisqcwx/LoFt.

Sequential recommendation models often struggle to capture latent periodic patterns in user interests, primarily due to the noise inherent in time-domain behavioral data. While frequency-domain analysis offers a global perspective to address this, existing approaches typically treat user sequences in isolation, overlooking the crucial context of the target item. In this work, we present a novel empirical observation: user attention scores exhibit distinct spectral entropy distributions when conditioned on positive versus negative target items. Specifically, true user interests manifest as highly concentrated spectral patterns with lower entropy in the frequency domain, whereas irrelevant behaviors appear as high-entropy noise. Leveraging this insight, we propose the Frequency-Enhanced Deep Interest Network (FEDIN). FEDIN introduces a frequency-domain branch that utilizes a target-aware spectrum filtering mechanism to isolate these periodic interest signals. Extensive experiments on three public datasets demonstrate that FEDIN consistently outperforms state-of-the-art sequential recommendation baselines, demonstrating superior robustness against noise. We have released our code at: https://github.com/otokoneko/FEDIN.

Traditional vertical federated learning schema suffers from two main issues: 1) restricted applicable scope to overlapped samples and 2) high system challenge of real-time federated serving, which limits its application to advertising systems. To this end, we advocate a new practical learning setting, Semi-VFL (Vertical Semi-Federated Learning), for real-world industrial applications, where the learned model retains sufficient advantages of federated learning while supporting independent local serving. To achieve this goal, we propose the carefully designed Joint Privileged Learning framework (JPL) to i) alleviate the absence of the passive party's feature with federated equivalence imitation and ii) adapt to the heterogeneous full sample space with cross-branch rank alignment. Extensive experiments conducted on real-world advertising datasets validate the effectiveness of our method over baseline methods.

The web-scale surge of multimodal content, including short-video feeds and autonomous sensing streams, has made web-native lossless compression a prerequisite for delivery and storage across browsers and edge–cloud pipelines. However, existing methods often fail to adapt to shifting distributions across different batches and struggle to balance computational resources in the face of large conditional entropy disparities among diverse modalities. To address these limitations, we propose MoE-LC, a new mixture-of-experts framework for multi-modal lossless compression that dynamically accommodates heterogeneous data distributions and varying complexity levels. First, the Batch-Adaptive Experts (BAE) module introduces batch-specific parameters with a residual gating mechanism, ensuring stable modeling under non-stationary distributions. Second, the Entropy-Aware Multi-Expert Selection (MES) strategy adaptively allocates the number of experts according to the data's estimated compression difficulty (entropy), thereby improving resource utilization and computational efficiency. Finally, the Precision-Aware Expert Routing (PER) component applies high-precision computation solely to the most critical experts, significantly reducing overhead without sacrificing compression accuracy. Experimental results across multiple real-world datasets demonstrate that MoE-LC achieves 5.33%--70.89% improvements in compression ratio and 37.25%--1532.41% gains in throughput compared to advanced baselines, offering a scalable solution for real-time, large-scale multi-modal data compression. Our code is available at https://github.com/Magie0/MoE_LC.

Backdoor attacks intend to inject hidden backdoor into the deep neural networks (DNNs), such that the predictions of infected models will be maliciously changed if the hidden backdoor is activated by the attacker-specified trigger pattern. Since the infected models behave normally on predicting benign samples, the backdoor attack is stealthy and therefore a serious threat to practical applications of DNNs. Currently, most existing backdoor attacks adopted the setting of static trigger, i.e ., triggers across the training and testing images follow the same appearance and are located in the same area. In this paper, we revisit this attack paradigm by analyzing trigger characteristics. We demonstrate that this attack paradigm is vulnerable when the trigger in testing images is not consistent with the one used for training. As such, those attacks are far less effective in the physical world, where the location and appearance of the trigger contained in digitized test samples may be different from that of the one used for training. Besides, we introduce a plug-in attack enhancement module during training, inspired by the expectation over transformation (EOT), to alleviate such inconsistency vulnerability. Based on this plug-in module, we also reveal that the widely adopted data augmentation may exacerbate the security risks of backdoor attacks, although it can enhance model performance. Moreover, we evaluate our methods on multiple benchmark datasets to verify their effectiveness. We hope that our work could inspire more explorations on the properties of backdoor attacks, to facilitate the design of more robust and secure DNNs.

Recently, deep learning has demonstrated improvements over classical decoding algorithms in various families of error-correcting codes with short to moderate block lengths. Neural decoders like neural belief propagation (NBP), constructed based on the Tanner graphs of linear codes, have been extensively studied, but their application to longer codes is limited due to increased network complexity with longer block lengths. This complexity leads to higher computational costs and deployment challenges, such as GPU memory usage. To address this, we propose a novel revolving framework for NBP-like decoders tailored to cyclic and QC-LDPC codes, a commonly used class of error correction codes. Our approach leverages the cyclic structure and section-wise cyclic structure inherent in cyclic and QC-LDPC codes respectively, significantly simplifying the complexity of the network. Experimental results demonstrate the effectiveness of our method across various cyclic and QC-LDPC codes. Especially, compared to the traditional decoding scheme based on the section-wise cyclic structure, our proposed decoder shows substantial improvements on 5G LDPC codes, exhibits better performance than the traditional min-sum decoder, and approaches the sum-product decoder without a noticeable error floor within the investigated noise levels.

Text-to-image (T2I) diffusion models enable high-quality image generation conditioned on textual prompts. However, fine-tuning these pre-trained models for personalization raises concerns about unauthorized dataset usage. To address this issue, dataset ownership verification (DOV) has recently been proposed, which embeds watermarks into fine-tuning datasets via backdoor techniques. These watermarks remain dormant on benign samples but produce owner-specified outputs when triggered. Despite its promise, the robustness of DOV against copyright evasion attacks (CEA) remains unexplored. In this paper, we investigate how adversaries can circumvent these mechanisms, enabling models trained on watermarked datasets to bypass ownership verification. We begin by analyzing the limitations of potential attacks achieved by backdoor removal, including TPD and T2IShield. In practice, TPD suffers from inconsistent effectiveness due to randomness, while T2IShield fails when watermarks are embedded as local image patches. To this end, we introduce CEAT2I, the first CEA specifically targeting DOV in T2I diffusion models. CEAT2I consists of three stages: (1) motivated by the observation that T2I models converge faster on watermarked samples with respect to intermediate features rather than training loss, we reliably detect watermarked samples; (2) we iteratively ablate tokens from the prompts of detected samples and monitor feature shifts to identify trigger tokens; and (3) we apply a closed-form concept erasure method to remove the injected watermarks. Extensive experiments demonstrate that CEAT2I effectively evades state-of-the-art DOV mechanisms while preserving model performance. The code is available at https://github.com/csyufei/CEAT2I.

Sampling strategy (e.g., fixed farthest point sampling) of point cloud has been an essential step for developing practical solutions in 3D computer vision tasks. Previous fixed sampling is simple, but suffer from suboptimal performance for downstream tasks. To adapt to target networks properly, adaptive sampling methods with trainable parameters have been recently developed to enhance the performance. However, existing adaptive sampling methods still suffer from the over-coupling problem of target network, and thus become model-specific, which limits their practical applications. To address this issue, we propose a novel general cross-scale decoupled sampling method (GCD-sampling) for point cloud, which consists of original feature cache, cross-scale feature fusion and convex combination learning for better feature extraction. To reduce the coupling relationship with the target task network, our method only utilizes the point cloud coordinates as the input and output of itself. Besides, we introduce an arbitrary scale structure to enable parameter sharing across multi-scale sampling in point cloud networks. Extensive experiments on different architectures demonstrate the effectiveness of our method over other existing adaptive sampling methods.

Self-supervised video hashing (SSVH) is a practical task in video indexing and retrieval. Although Transformers are predominant in SSVH for their impressive temporal modeling capabilities, they often suffer from computational and memory inefficiencies. Drawing inspiration from Mamba, an advanced state-space model, we explore its potential in SSVH to achieve a better balance between efficacy and efficiency. We introduce S5VH, a Mamba-based video hashing model with an improved self-supervised learning paradigm. Specifically, we design bidirectional Mamba layers for both the encoder and decoder, which are effective and efficient in capturing temporal relationships thanks to the data-dependent selective scanning mechanism with linear complexity. In our learning strategy, we transform global semantics in the feature space into semantically consistent and discriminative hash centers, followed by a center alignment loss as a global learning signal. Our self-local-global (SLG) paradigm significantly improves learning efficiency, leading to faster and better convergence. Extensive experiments demonstrate S5VH's improvements over state-of-the-art methods, superior transferability, and scalable advantages in inference efficiency.

Deep learning (e.g., Transformer) has been widely and successfully used in multivariate time series forecasting (MTSF). Unlike existing methods that focus on training models from a single modal of time series input, large language models (LLMs) based MTSF methods with cross-modal text and time series input have recently shown great superiority, especially with limited temporal data. However, current LLM-based MTSF methods usually focus on adapting and fine-tuning LLMs, while neglecting the distribution discrepancy between textual and temporal input tokens, thus leading to sub-optimal performance. To address this issue, we propose a novel Cross-Modal LLM Fine-Tuning (CALF) framework for MTSF by reducing the distribution discrepancy between textual and temporal data, which mainly consists of the temporal target branch with temporal input and the textual source branch with aligned textual input. To reduce the distribution discrepancy, we develop the cross-modal match module to first align cross-modal input distributions. Additionally, to minimize the modality distribution gap in both feature and output spaces, feature regularization loss is developed to align the intermediate features between the two branches for better weight updates, while output consistency loss is introduced to allow the output representations of both branches to correspond effectively. Thanks to the modality alignment, CALF establishes state-of-the-art performance for both long-term and short-term forecasting tasks with low computational complexity, and exhibits favorable few-shot and zero-shot abilities similar to that in LLMs.

In noisy partial label learning, each training sample is associated with a set of candidate labels, and the ground-truth label may be contained within this set. With the emergence of powerful pre-trained vision-language models, e.g. CLIP, it is natural to consider using these models to automatically label training samples instead of relying on laborious manual annotation. In this paper, we investigate the pipeline of learning with CLIP annotated noisy partial labels and propose a novel collaborative consistency regularization method, in which we simultaneously train two neural networks, which collaboratively purify training labels for each other, called Co-Pseudo-Labeling, and perform consistency regularization between label and representation levels. For instance-dependent noise that embodies the underlying patterns of the pre-trained model, our method employs multiple mechanisms to avoid overfitting to noisy annotations, effectively mines information from potentially noisy sample set while iteratively optimizing both representations and pseudo-labels during the training process. Comparison experiments with various kinds of annotations and weakly supervised methods, as well as other pre-trained model application methods demonstrates the effectiveness of method and the feasibility of incorporating weakly supervised learning into the distillation of pre-trained models.

Diffusion models represent the state-of-the-art in generative modeling. Due to their high training costs, many works leverage pre-trained diffusion models' powerful representations for downstream tasks, such as face super-resolution (FSR), through fine-tuning or prior-based methods. However, relying solely on priors without supervised training makes it challenging to meet the pixel-level accuracy requirements of discrimination task. Although prior-based methods can achieve high fidelity and high-quality results, ensuring consistency remains a significant challenge. In this paper, we propose a masking strategy with strong and weak constraints and iterative refinement for real-world FSR, termed Diffusion Prior Interpolation (DPI). We introduce conditions and constraints on consistency by masking different sampling stages based on the structural characteristics of the face. Furthermore, we propose a condition Corrector (CRT) to establish a reciprocal posterior sampling process. DPI can balance consistency and diversity and can be seamlessly integrated into pre-trained models. In extensive experiments conducted on synthetic and real datasets, along with consistency validation in face recognition, DPI demonstrates superiority over SOTA FSR methods.

Large Audio-Language Models (LALMs), such as GPT-4o, have recently unlocked audio dialogue capabilities, enabling direct spoken exchanges with humans.The potential of LALMs broadens their applicability across a wide range of practical scenarios supported by audio dialogues.However, given these advancements, a comprehensive benchmark to evaluate the performance of LALMs in the open-ended audio dialogue understanding remains absent currently.To address this gap, we propose an Audio Dialogue Understanding Benchmark (ADU-Bench), which consists of 4 benchmark datasets.They assess the open-ended audio dialogue ability for LALMs in 3 general scenarios, 12 skills, 9 multilingual languages, and 4 categories of ambiguity handling.Notably, we firstly propose the evaluation of ambiguity handling in audio dialogues that expresses different intentions beyond the same literal meaning of sentences, e.g., "Really!?" with different intonations.In summary, ADU-Bench includes over 20,000 open-ended audio dialogues for the assessment of LALMs.Through extensive experiments on 16 LALMs, our analysis reveals that existing LALMs struggle with mathematical symbols and formulas, understanding human behavior such as roleplay, comprehending multiple languages, and handling audio dialogue ambiguities from different phonetic elements, such as intonations, pause positions, and homophones.The benchmark is available at https://adu-bench.github.io/.

Haomiao Tang, Jinpeng Wang, Yuang Peng, GuangHao Meng, Ruisheng Luo, Bin Chen, Long Chen, Yaowei Wang, Shu-Tao Xia. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2025.

Visual In-Context Learning (VICL) enables adaptively solving vision tasks by leveraging pixel demonstrations, mimicking human-like task completion through analogy. Prompt selection is critical in VICL, but current methods assume the existence of a single "ideal" prompt in a pool of candidates, which in practice may not hold true. Multiple suitable prompts may exist, but individually they often fall short, leading to difficulties in selection and the exclusion of useful context. To address this, we propose a new perspective: prompt condensation. Rather than relying on a single prompt, candidate prompts collaborate to efficiently integrate informative contexts without sacrificing resolution. We devise Condenser, a lightweight external plugin that compresses relevant fine-grained context across multiple prompts. Optimized end-to-end with the backbone, Condenser ensures accurate integration of contextual cues. Experiments demonstrate Condenser outperforms state-of-the-arts across benchmark tasks, showing superior context compression, scalability with more prompts, and enhanced computational efficiency compared to ensemble methods, positioning it as a highly competitive solution for VICL. Code is open-sourced at https://github.com/gimpong/CVPR25-Condenser.

The Mamba-based image restoration backbones have recently demonstrated significant potential in balancing global reception and computational efficiency. However, the inherent causal modeling limitation of Mamba, where each token depends solely on its predecessors in the scanned sequence, restricts the full utilization of pixels across the image and thus presents new challenges in image restoration. In this work, we propose MambaIRv2, which equips Mamba with the non-causal modeling ability similar to ViTs to reach the attentive state space restoration model. Specifically, the proposed attentive state-space equation allows to attend beyond the scanned sequence and facilitate image unfolding with just one single scan. Moreover, we further introduce a semantic-guided neighboring mechanism to encourage interaction between distant but similar pixels. Extensive experiments show our MambaIRv2 outperforms SRFormer by even 0.35dB PSNR for lightweight SR even with 9.3% less parameters and suppresses HAT on classic SR by up to 0.29dB. Code is available at https://github.com/csguoh/MambaIR.

Self-Supervised Video Hashing (SSVH) compresses videos into hash codes for efficient indexing and retrieval using unlabeled training videos. Existing approaches rely on random frame sampling to learn video features and treat all frames equally. This results in suboptimal hash codes, as it ignores frame-specific information density and reconstruction difficulty. To address this limitation, we propose a new framework, termed AutoSSVH, that employs adversarial frame sampling with hash-based contrastive learning. Our adversarial sampling strategy automatically identifies and selects challenging frames with richer information for reconstruction, enhancing encoding capability. Additionally, we introduce a hash component voting strategy and a point-to-set (P2Set) hash-based contrastive objective, which help capture complex inter-video semantic relationships in the Hamming space and improve the discriminability of learned hash codes. Extensive experiments demonstrate that Au-toSSVH achieves superior retrieval efficacy and efficiency compared to state-of-the-art approaches. Code is available at https://github.com/EliSpectre/CVPR25-AutoSSVH.

Recently, video-based large language models (video-based LLMs) have achieved impressive performance across various video comprehension tasks. However, this rapid advancement raises significant privacy and security concerns, particularly regarding the unauthorized use of personal video data in automated annotation by video-based LLMs. These unauthorized annotated video-text pairs can then be used to improve the performance of downstream tasks, such as text-to-video generation. To safeguard personal videos from unauthorized use, we propose two series of protective video watermarks with imperceptible adversarial perturbations, named Ramblings and Mutes. Concretely, Ramblings aim to mislead video-based LLMs into generating inaccurate captions for the videos, thereby degrading the quality of video annotations through inconsistencies between video content and captions. Mutes, on the other hand, are designed to prompt video-based LLMs to produce exceptionally brief captions, lacking descriptive detail. Extensive experiments demonstrate that our video watermarking methods effectively protect video data by significantly reducing video annotation performance across various video-based LLMs, showcasing both stealthiness and robustness in protecting personal video content. Our code is available at https://github.com/ttthhl/Protecting_Your_Video_Content.

Point cloud video understanding is becoming increasingly important in fields such as robotics, autonomous driving, and augmented reality, as they can accurately represent object motion and environmental changes. Despite the progress made in self-supervised learning methods for point cloud video understanding, the limited availability of 4D data and the high computational cost of training 4D-specific models remain significant obstacles. In this paper, we investigate the potential of transferring pre-trained static 3D point cloud models to the 4D domain, pointing out the limitations of static models that capture only spatial information while neglecting temporal dynamics. To address this, we propose a novel Cross-frame Spatio-temporal Adaptation (CSA) strategy by introducing the Point Tube Adapter as the embedding layer and the Geometric Constraint Temporal Adapter (GCTA) to enforce temporal consistency across frames. This strategy extracts both short-term and long-term temporal dynamics, effectively integrating them with spatial features and enriching the model’s understanding of temporal changes in point cloud videos. Extensive experiments on 3D action and gesture recognition tasks demonstrate that our method achieves state-of-the-art performance, establishing its effectiveness for point cloud video understanding. Code is available at: https://github.com/LvBaixuan/Point-CSA.

Applying pre-trained models to assist point cloud understanding has recently become a mainstream paradigm in 3D perception. However, existing application strategies are straightforward, utilizing only the final output of the pre-trained model for various task heads. It neglects the rich complementary information in the intermediate layer, thereby failing to fully unlock the potential of pre-trained models. To overcome this limitation, we propose an orthogonal solution: Point Mamba Adapter (PMA), which constructs an ordered feature sequence from all layers of the pre-trained model and leverages Mamba to fuse all complementary semantics, thereby promoting comprehensive point cloud understanding. Constructing this ordered sequence is non-trivial due to the inherent isotropy of 3D space. Therefore, we further propose a geometry-constrained gate prompt generator (G2PG) shared across different layers, which applies shared geometric constraints to the output gates of the Mamba and dynamically optimizes the spatial order, thus enabling more effective integration of multi-layer information. Extensive experiments conducted on challenging point cloud datasets across various tasks demonstrate that our PMA elevates the capability for point cloud understanding to a new level by fusing diverse complementary intermediate features. Code is available at https://github.com/zyh16143998882/PMA.

Dataset distillation is an emerging dataset reduction method, which condenses large-scale datasets while maintaining task accuracy. Current parameterization methods achieve enhanced performance under extremely high compression ratio by optimizing determined synthetic dataset in informative feature domain. However, they limit themselves to a fixed optimization space for distillation, neglecting the diverse guidance across different informative latent spaces. To overcome this limitation, we propose a novel parameterization method dubbed Hierarchical Parameterization Distillation (H-PD), to systematically explore hierarchical feature within provided feature space (e.g., layers within pre-trained generative adversarial networks). We verify the correctness of our insights by applying the hierarchical optimization strategy on GAN-based parameterization method. In addition, we introduce a novel class-relevant feature distance metric to alleviate the computational burden associated with synthetic dataset evaluation, bridging the gap between synthetic and original datasets. Experimental results demonstrate that the proposed H-PD achieves a significant performance improvement under various settings with equivalent time consumption, and even surpasses current generative distillation using diffusion models under extreme compression ratios IPC=1 and IPC=10. Our code is available at https://github.com/ndhg1213/H-PD

Hao Fang, Jiawei Kong, Tianqu Zhuang, Yixiang Qiu, Kuofeng Gao, Bin Chen, Shu-Tao Xia, Yaowei Wang, Min Zhang. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. 2025.

The rapid advancement of large language models has intensified public concerns about the potential misuse.Therefore, it is important to build trustworthy AI-generated text detection systems.Existing methods neglect stylistic modeling and mostly rely on static thresholds, which greatly limits the detection performance.In this paper, we propose the Mixture of Stylistic Experts (MoSEs) framework that enables stylistics-aware uncertainty quantification through conditional threshold estimation.MoSEs contain three core components, namely, the Stylistics Reference Repository (SRR), the Stylistics-Aware Router (SAR), and the Conditional Threshold Estimator (CTE).For input text, SRR can activate the appropriate reference data in SRR and provide them to CTE.Subsequently, CTE jointly models the linguistic statistical properties and semantic features to dynamically determine the optimal threshold.With a discrimination score, MoSEs yields prediction labels with the corresponding confidence level.Our framework achieves an average improvement 11.34% in detection performance compared to baselines.More inspiringly, MoSEs shows a more evident improvement 39.15% in the low-resource case.Our code is available at https://github.com/ creator-xi/MoSEs.

Existing Implicit Neural Representation (INR) video compression techniques have opened up new avenues in the field of video compression. NeRV maps the temporal coordinates to high-resolution images using neural networks, providing a more flexible and efficient encoding method for video data. However, NeRV implicitly stores all video information in the network, requiring post network compression techniques such as pruning. To integrate explicit compression and implicit representation into an end-to-end framework, this study proposes a novel neural representation-based video compression paradigm called Latent code based Neural representation video compression (LNeRV). Specifically, LNeRV consists of hierarchy feature grids, synthesis network and entropy coding network. With a single-stage training process, LNeRV achieves video compression and dynamically allocates bits according to video complexity, better fitting dynamic videos. We provide a comprehensive compression-to-decompression workflow for our approach. Extensive experimental results verify the effectiveness of our LNeRV.

As point clouds gain widespread application in fields such as autonomous driving and scene modeling, an increasing number of point cloud learning networks have emerged. As a result, research on 3D adversarial attacks and defenses has rapidly advanced. To the best of our knowledge, existing 3D defense methods primarily focus on enhancing the robustness of networks through point cloud processing or adversarial training, without attention given to network architecture. In this paper, we propose RobNAS, enhancing the adversarial robustness of point cloud classification networks by integrating adversarial training with Neural Architecture Search (NAS) from an architectural robustness perspective. Specifically, we first incorporate PGD-based adversarial training during the architecture search phase of RobNAS to obtain the most robust architecture. Subsequently, during the adversarial training phase, we introduce various adversarial examples to enhance the robustness of the model weights. Our experimental results demonstrate that our method achieves State-Of-The-Art (SOTA) performance. Furthermore, we aim to shed light on the promising potential of architectural robustness for learning robust point cloud representation.

Vision-Language Pre-training (VLP) models have exhibited unprecedented capability in many applications by taking full advantage of the multimodal alignment. However, previous studies have shown they are vulnerable to maliciously crafted adversarial samples. Despite recent success, these methods are generally instance-specific and require generating perturbations for each input sample. In this paper, we reveal that VLP models are also vulnerable to the instance-agnostic universal adversarial perturbation (UAP). Specifically, we design a novel Contrastive-training Perturbation Generator with Cross-modal conditions (C-PGC) to achieve the attack. In light that the pivotal multimodal alignment is achieved through the advanced contrastive learning technique, we devise to turn this powerful weapon against themselves, i.e., employ a malicious version of contrastive learning to train the C-PGC based on our carefully crafted positive and negative image-text pairs for essentially destroying the alignment relationship learned by VLP models. Besides, C-PGC fully utilizes the characteristics of Vision-and-Language (V+L) scenarios by incorporating both unimodal and cross-modal information as effective guidance. Extensive experiments show that C-PGC successfully forces adversarial samples to move away from their original area in the VLP model's feature space, thus essentially enhancing attacks across various victim models and V+L tasks. The GitHub repository is available at https://github.com/ffhibnese/CPGC_VLP_Universal_Attacks.

Visual Autoregressive (VAR) modeling has gained popularity for its shift towards next-scale prediction. However, existing VAR paradigms process the entire token map at each scale step, leading to the complexity and runtime scaling dramatically with image resolution. To address this challenge, we propose FastVAR, a post-training acceleration method for efficient resolution scaling with VARs. Our key finding is that the majority of latency arises from the large-scale step where most tokens have already converged. Leveraging this observation, we develop the cached token pruning strategy that only forwards pivotal tokens for scale-specific modeling while using cached tokens from previous scale steps to restore the pruned slots. This significantly reduces the number of forwarded tokens and improves the efficiency at larger resolutions. Experiments show the proposed FastVAR can further speedup FlashAttention-accelerated VAR by 2.7$\times$ with negligible performance drop of <1%. We further extend FastVAR to zero-shot generation of higher resolution images. In particular, FastVAR can generate one 2K image with 15GB memory footprints in 1.5s on a single NVIDIA 3090 GPU. Code is available at https://github.com/csguoh/FastVAR.

Federated learning is usually regarded as a privacy-preserving training paradigm for it enables multiple clients to participate in a training task without sharing their private data. However, recent studies revealed that a malicious server can still recover private data from the victim clients based on the shared gradients via deep leakage from gradients (DLG). Currently, almost all DLG attacks are designed based on the average loss, leading to a significant decrease in attack efficiency when the batch size is greater than 1. In this paper, we revisit DLG attacks from the perspective of the loss function. We reveal that not all samples in the target batch are equally susceptible to DLG attacks: the sample with the highest loss value tends to be easily recovered by DLG attacks. Based on these observations, we propose a simple yet effective DLG method under practical FL settings. Specifically, the adversaries can enhance the effectiveness of DLG by perturbing the global model through finetuning it with a few mislabeled samples (dubbed ‘Anti-FT’). Extensive experiments are conducted on benchmark datasets, which verify the effectiveness of our method and its resistance to potential defenses. The codes are available at https://github.com/zlh-thu/anti-finetune.

3D Gaussian Splatting (3DGS) has witnessed its rapid development in novel view synthesis, which attains high quality reconstruction and real-time rendering. At the same time, there is still a gap before implicit neural representation (INR) can become a practical compressor due to the lack of stream decoding and real-time frame reconstruction on consumer-grade hardware. It remains a question whether the fast rendering and partial parameter decoding characteristics of 3DGS are applicable to video compression. To address these challenges, we propose a Toast-like Sliding Window (TSW) orthographic projection for converting any 3D Gaussian model into a video representation model. This method efficiently represents video by leveraging temporal redundancy through a sliding window approach. Additionally, the converted model is inherently stream-decodable and offers a higher rendering frame rate compared to INR methods. Building on TSW, we introduce an end-to-end trainable video compression method, GSVC, which employs deformable Gaussian representation and optical flow guidance to capture dynamic content in videos. Experimental results demonstrate that our method effectively transforms a 3D Gaussian model into a practical video compressor. GSVC further achieves better rate-distortion performance than NeRV on the UVG dataset, while achieving higher frame reconstruction speed (+30%~40% fps) and stream decoding. Code is available at [Github](https://github.com/actcwlf/GSVC)

Personalized image generation holds great promise in assisting humans in everyday work and life due to its impressive function in creatively generating personalized content. However, current evaluations either are automated but misalign with humans or require human evaluations that are time-consuming and expensive. In this work, we present DreamBench++, a human-aligned benchmark that advanced multimodal GPT models automate. Specifically, we systematically design the prompts to let GPT be both human-aligned and self-aligned, empowered with task reinforcement. Further, we construct a comprehensive dataset comprising diverse images and prompts. By benchmarking 7 modern generative models, we demonstrate that \dreambench results in significantly more human-aligned evaluation, helping boost the community with innovative findings.

Uncertainty quantification in time series prediction is challenging due to the temporal dependence and distribution shift on sequential data. Conformal prediction provides a pivotal and flexible instrument for assessing the uncertainty of machine learning models through prediction sets. Recently, a series of online conformal inference methods updated thresholds of prediction sets by performing online gradient descent on a sequence of quantile loss functions. A drawback of such methods is that they only use the information of revealed non-conformity scores via miscoverage indicators but ignore error quantification, namely the distance between the non-conformity score and the current threshold. To accurately leverage the dynamic of miscoverage error, we propose Error-quantified Conformal Inference (ECI) by smoothing the quantile loss function. ECI introduces a continuous and adaptive feedback scale with the miscoverage error, rather than simple binary feedback in existing methods. We establish a long-term coverage guarantee for ECI under arbitrary dependence and distribution shift. The extensive experimental results show that ECI can achieve valid miscoverage control and output tighter prediction sets than other baselines.

Dataset distillation (DD) aims to minimize the time and memory consumption needed for training deep neural networks on large datasets, by creating a smaller synthetic dataset that has similar performance to that of the full real dataset. However, current dataset distillation methods often result in synthetic datasets that are excessively difficult for networks to learn from, due to the compression of a substantial amount of information from the original data through metrics measuring feature similarity, e,g., distribution matching (DM). In this work, we introduce conditional mutual information (CMI) to assess the class-aware complexity of a dataset and propose a novel method by minimizing CMI. Specifically, we minimize the distillation loss while constraining the class-aware complexity of the synthetic dataset by minimizing its empirical CMI from the feature space of pre-trained networks, simultaneously. Conducting on a thorough set of experiments, we show that our method can serve as a general regularization method to existing DD methods and improve the performance and training efficiency.

Model inversion attacks (MIAs) aim to reconstruct the private training data by accessing the public model, raising concerns about privacy leakage. Black-box MIAs, where attackers can only query the model and obtain outputs, are closer to real-world scenarios. The latest black-box attacks have outperformed state-of-the-art white-box attacks, and existing defenses cannot resist them effectively. To fill this gap, we propose Stealthy Shield Defense (SSD), a post-processing algorithm against black-box MIAs. Our idea is to modify the model's outputs to minimize the conditional mutual information (CMI). We mathematically prove that CMI is a special case of Information Bottleneck (IB), and thus inherits the benefits of IB---making predictions less dependent on inputs and more dependent on ground truths. This theoretically guarantees our effectiveness, both in resisting MIAs and preserving utility. To minimize CMI, we formulate a convex optimization problem and solve it via the water-filling method. Without the need to retrain the model, our defense is plug-and-play and easy to deploy. Experimental results indicate that SSD outperforms existing defenses, in terms of MIA resistance and model's utility, across various attack algorithms, private datasets, and model architectures. Our code is available at https://github.com/ZhuangQu/Stealthy-Shield-Defense.

Fine-tuning pre-trained diffusion models under limited budgets has gained great success. In particular, the recent advances that directly fine-tune the quantized weights using Low-rank Adaptation (LoRA) further reduces training costs. Despite these progress, we point out that existing adaptation recipes are not inference-efficient. Specifically, additional post-training quantization (PTQ) on tuned weights is needed during deployment, which results in noticeable performance drop when the bit-width is low. Based on this observation, we introduce IntLoRA, which adapts quantized diffusion models with integer-type low-rank parameters, to include inference efficiency during tuning. Specifically, IntLoRA enables pre-trained weights to remain quantized during training, facilitating fine-tuning on consumer-level GPUs. During inference, IntLoRA weights can be seamlessly merged into pre-trained weights to directly obtain quantized downstream weights without PTQ. Extensive experiments show our IntLoRA achieves significant speedup on both training and inference without losing performance.

Time series forecasting methods generally fall into two main categories: Channel Independent (CI) and Channel Dependent (CD) strategies. While CI overlooks important covariate relationships, CD captures all dependencies without distinction, introducing noise and reducing generalization. Recent advances in Channel Clustering (CC) aim to refine dependency modeling by grouping channels with similar characteristics and applying tailored modeling techniques. However, coarse-grained clustering struggles to capture complex, time-varying interactions effectively. To address these challenges, we propose TimeFilter, a GNN-based framework for adaptive and fine-grained dependency modeling. After constructing the graph from the input sequence, TimeFilter refines the learned spatial-temporal dependencies by filtering out irrelevant correlations while preserving the most critical ones in a patch-specific manner. Extensive experiments on 13 real-world datasets from diverse application domains demonstrate the state-of-the-art performance of TimeFilter. The code is available at https://github.com/TROUBADOUR000/TimeFilter.

Existing multi-view image compression methods often rely on 2D projection-based similarities between views to estimate disparities. While effective for small disparities, such as those in stereo images, these methods struggle with the more complex disparities encountered in wide-baseline multi-camera systems, commonly found in virtual reality and autonomous driving applications. To address this limitation, we propose 3D-LMVIC, a novel learning-based multi-view image compression framework that leverages 3D Gaussian Splatting to derive geometric priors for accurate disparity estimation. Furthermore, we introduce a depth map compression model to minimize geometric redundancy across views, along with a multi-view sequence ordering strategy based on a defined distance measure between views to enhance correlations between adjacent views. Experimental results demonstrate that 3D-LMVIC achieves superior performance compared to both traditional and learning-based methods. Additionally, it significantly improves disparity estimation accuracy over existing two-view approaches.

Non-stationarity poses significant challenges for multivariate time series forecasting due to the inherent short-term fluctuations and long-term trends that can lead to spurious regressions or obscure essential long-term relationships. Most existing methods either eliminate or retain non-stationarity without adequately addressing its distinct impacts on short-term and long-term modeling. Eliminating non-stationarity is essential for avoiding spurious regressions and capturing local dependencies in short-term modeling, while preserving it is crucial for revealing long-term cointegration across variates. In this paper, we propose TimeBridge, a novel framework designed to bridge the gap between non-stationarity and dependency modeling in long-term time series forecasting. By segmenting input series into smaller patches, TimeBridge applies Integrated Attention to mitigate short-term non-stationarity and capture stable dependencies within each variate, while Cointegrated Attention preserves non-stationarity to model long-term cointegration across variates. Extensive experiments show that TimeBridge consistently achieves state-of-the-art performance in both short-term and long-term forecasting. Additionally, TimeBridge demonstrates exceptional performance in financial forecasting on the CSI 500 and S&P 500 indices, further validating its robustness and effectiveness. Code is available at https://github.com/Hank0626/TimeBridge.

Recently, learning-based point cloud analysis has played a crucial role in robotic perception. Masked Point Modeling (MPM), owing to its powerful representational capabilities, has become the mainstream point cloud self-supervised learning method. However, existing MPM-based methods often suffer from the problem of negative transfer, due to the disparity in semantic distribution between upstream data and downstream data. To address this issue, we propose an expert enhancement strategy for existing MPM-based methods. Specifically, we insert a Sparse Mixture of Experts (SMoE) layer after each block of the backbone network, which utilizes a multi-branch expert architecture with routers that allocate data of different semantics to the appropriate experts for analysis. During the pre-training phase, our expert-enhanced model not only learns universal 3D representations for the backbone network but also acquires powerful semantic routing capabilities for all expert layers. In the fine-tuning phase, we freeze all backbones and conduct end-to-end fine-tuning solely on our expert layers to adaptively select multiple experts most relevant to the semantics of each downstream data for analysis. Extensive downstream experiments demonstrate the superiority of our method, especially outperforming baseline (Point-MAE) by 5.16%, 5.86%, and 4.62% in three variants of ScanObjectNN while utilizing only 12% of its trainable parameters. Our code is released at https://github.com/chenchen1104/point_e2mae.

Low-rank regularization (LRR) has been widely applied in various machine learning tasks, but the associated optimization is challenging. Directly optimizing the rank function under constraints is NP-hard in general. To overcome this difficulty, various relaxations of the rank function were studied. However, optimization of these relaxed LRRs typically depends on singular value decomposition, which is a time-consuming and nondifferentiable operator that cannot be optimized with gradient-based techniques. To address these challenges, in this paper we propose an efficient differentiable approximation of the generalized LRR. The considered LRR form subsumes many popular choices like the nuclear norm, the Schatten-p norm, and various nonconvex relaxations. Our method enables LRR terms to be appended to loss functions in a plug-and-play fashion, and the GPU-friendly operations enable efficient and convenient implementation. Furthermore, convergence analysis is presented, which rigorously shows that both the bias and the variance of our rank estimator rapidly reduce with increased sample size and iteration steps. In the experimental study, the proposed method is applied to various tasks, which demonstrates its versatility and efficiency. Code is available at https://github.com/naiqili/EDLRR.

Point clouds, as a primary representation of 3D data, can be categorized into scene domain point clouds and object domain point clouds. Point cloud self-supervised learning (SSL) has become a mainstream paradigm for learning 3D representations. However, existing point cloud SSL primarily focuses on learning domain-specific 3D representations within a single domain, neglecting the complementary nature of cross-domain knowledge, which limits the learning of 3D representations. In this paper, we propose to learn a comprehensive Point cloud Mixture-of-Domain-Experts model (Point-MoDE) via a block-to-scene pre-training strategy. Specifically, We first propose a mixture-of-domain-expert model consisting of scene domain experts and multiple shared object domain experts. Furthermore, we propose a block-to-scene pretraining strategy, which leverages the features of point blocks in the object domain to regress their initial positions in the scene domain through object-level block mask reconstruction and scene-level block position regression. By integrating the complementary knowledge between object and scene, this strategy simultaneously facilitates the learning of both object-domain and scene-domain representations, leading to a more comprehensive 3D representation. Extensive experiments in downstream tasks demonstrate the superiority of our model.

Sequential recommendation (SR) aims to capture dynamic user preferences from users' historical behaviors. Recently, benefiting from astonishing understanding ability of pre-trained language models (PLMs), text-enhanced sequential recommender becomes a promising direction, which employs PLMs to extract semantic information for user/item representation. Although promising in improving performance and transferability, few existing text-enhanced SR studies have analyzed the differences between PLMs and recommenders, restricting the ability of PLMs for recommendation. In this paper, we make an in-depth comparison and conclude their discrepancies in representation and knowledge level, respectively, caused by different multi-modal content and task-oriented capabilities. Based on this, we propose a Recommendation-Oriented Language Model Adaptation framework (named ROMA) using multi-modal multi-domain item sequences. To empower PLMs with a rational understanding of user/item modeling and the recommendation task, ROMA partitions a PLM into bottom and top layers, respectively, allowing representation-level and task-level adaptation with elaborately designed architectures, transferring strategy and learning framework. Our experimental results on public benchmarks demonstrate the effectiveness and transferability of our framework. Additionally, we showcase the application value of ROMA on the recommender system of Huawei's AppGallery through online A/B testing, which shows significant improvements in online metrics.

The explosive growth of multi-source multimedia data has significantly increased the demands for transmission and storage, placing substantial pressure on bandwidth and storage infrastructures. While Autoregressive Compression Models (ACMs) have markedly improved compression efficiency through probabilistic prediction, current approaches remain constrained by two critical limitations: suboptimal compression ratios due to insufficient fine-grained feature extraction during probability modeling, and real-time processing bottlenecks caused by high resource consumption and low compression speeds. To address these challenges, we propose Efficient Dual-path Parallel Compression (EDPC), a hierarchically optimized compression framework that synergistically enhances modeling capability and execution efficiency via coordinated dual-path operations. At the modeling level, we introduce the Information Flow Refinement (IFR) metric grounded in mutual information theory, and design a Multi-path Byte Refinement Block (MBRB) to strengthen cross-byte dependency modeling via heterogeneous feature propagation. At the system level, we develop a Latent Transformation Engine (LTE) for compact high-dimensional feature representation and a Decoupled Pipeline Compression Architecture (DPCA) to eliminate encoding-decoding latency through pipelined parallelization. Experimental results demonstrate that EDPC achieves comprehensive improvements over state-of-the-art methods, including a 2.7× faster compression speed, and a 3.2% higher compression ratio. These advancements establish EDPC as an efficient solution for real-time processing of large-scale multimedia data in bandwidth-constrained scenarios. Our code is available at https://github.com/Magie0/EDPC.

Large Vision-Language Models (LVLMs) are susceptible to hallucinations, where generated responses seem semantically plausible yet exhibit little or no relevance to the input image. Previous studies reveal that this issue primarily stems from LVLMs' over-reliance on language priors while disregarding the visual information during decoding. To alleviate this issue, we introduce a novel Conditional Pointwise Mutual Information (C-PMI) calibrated decoding strategy, which adaptively strengthens the mutual dependency between generated texts and input images to mitigate hallucinations. Unlike existing methods solely focusing on text token sampling, we propose to jointly model the contributions of visual and textual tokens to C-PMI, formulating hallucination mitigation as a bi-level optimization problem aimed at maximizing mutual information. To solve it, we design a token purification mechanism that dynamically regulates the decoding process by sampling text tokens remaining maximally relevant to the given image, while simultaneously refining image tokens most pertinent to the generated response. Extensive experiments across various benchmarks reveal that the proposed method significantly reduces hallucinations in LVLMs while preserving decoding efficiency.

Conventional compressed sensing (CS) algorithms typically apply a uniform sampling rate to different image blocks. A more strategic approach could be to allocate the number of measurements adaptively, based on each image block's complexity. In this paper, we propose a Measurement-Bounds-based Rate-Adaptive Image Compressed Sensing Network (MB-RACS) framework, which aims to adaptively determine the sampling rate for each image block in accordance with traditional measurement bounds theory. Moreover, since in real-world scenarios statistical information about the original image cannot be directly obtained, we suggest a multi-stage rate-adaptive sampling strategy. This strategy sequentially adjusts the sampling ratio allocation based on the information gathered from previous samplings. We formulate the multi-stage rate-adaptive sampling as a convex optimization problem and address it using a combination of Newton's method and binary search techniques. Our experiments demonstrate that the proposed MB-RACS method surpasses current leading methods, with experimental evidence also underscoring the effectiveness of each module within our proposed framework.

Currently, deep neural networks (DNNs) are widely adopted in different applications. Despite its commercial values, training a well-performing DNN is resource-consuming. Accordingly, the well-trained model is valuable intellectual property for its owner. However, recent studies revealed the threats of model stealing, where the adversaries can obtain a function-similar copy of the victim model, even when they can only query the model. In this paper, we propose an effective and harmless model ownership verification (MOVE) to defend against different types of model stealing simultaneously, without introducing new security risks. In general, we conduct the ownership verification by verifying whether a suspicious model contains the knowledge of defender-specified external features. Specifically, we embed the external features by modifying a few training samples with style transfer. We then train a meta-classifier to determine whether a model is stolen from the victim. This approach is inspired by the understanding that the stolen models should contain the knowledge of features learned by the victim model. In particular, we develop our MOVE method under both glass-boxand closed-box settings and analyze its theoretical foundation to provide comprehensive model protection. Extensive experiments on benchmark datasets verify the effectiveness of our method and its resistance to potential adaptive attacks.

Network traffic classification plays a crucial role in network management and cyberspace security. As the Internet evolves with new applications and protocols, traditional machine learning-based methods relying on feature mining have become obsolete. Instead, deep learning-based methods are becoming more popular in the field of traffic classification due to their end-to-end processing approach. However, the vulnerability of neural networks to adversarial examples significantly compromises their performance. In this paper, we propose Robust Byte-Label Joint Attention Network (RBLJAN), an efficient and robust deep learning-based framework for encrypted network traffic classification at both the packet-level and the flow-level. RBLJAN comprises a classifier and an adversarial traffic generator. The classifier utilizes mechanisms such as header-payload parallel processing and byte-label joint attention learning to capture implicit correlations between bytes and labels, enabling the construction of powerful packet representations. The generator produces adversarial examples that are fed to the classifier to enhance its robustness. Experimental results demonstrate that RBLJAN achieves over 99% average F1-score on real-world legitimate traffic datasets and achieves 97.86% average F1-score on malware identification. Moreover, RBLJAN exhibits superior performance in terms of detection speed and robustness compared to state-of-the-art methods in real-world scenarios.

Currently, sample-specific backdoor attacks (SSBAs) are the most advanced and malicious methods since they can easily circumvent most of the current backdoor defenses. In this paper, we reveal that SSBAs are not sufficiently stealthy due to their poisoned-label nature, where users can discover anomalies if they check the image-label relationship. In particular, we demonstrate that it is ineffective to directly generalize existing SSBAs to their clean-label variants by poisoning samples solely from the target class. We reveal that it is primarily due to two reasons, including(1)the ‘antagonistic effects’ of ground-truth features and(2)the learning difficulty of sample-specific features. Accordingly, trigger-related features of existing SSBAs cannot be effectively learned under the clean-label setting due to their mild trigger intensity required for ensuring stealthiness. We argue that the intensity constraint of existing SSBAs is mostly because their trigger patterns are ‘content-irrelevant’ and therefore act as ‘noises’ for both humans and DNNs. Motivated by this understanding, we propose to exploit content-relevant features,$a.k.a.$(human-relied) attributes, as the trigger patterns to design clean-label SSBAs. This new attack paradigm is dubbed backdoor attack with attribute trigger (BAAT). Extensive experiments are conducted on benchmark datasets, which verify the effectiveness of our BAAT and its resistance to existing defenses.

Gradient Inversion Attacks invert the transmitted gradients in Federated Learning (FL) systems to reconstruct the sensitive data of local clients and have raised considerable privacy concerns. A majority of gradient inversion methods rely heavily on explicit prior knowledge (e.g., a well pre-trained generative model), which is often unavailable in realistic scenarios. This is because real-world client data distributions are often highly heterogeneous, domain-specific, and unavailable to attackers, making it impractical for attackers to obtain perfectly matched pre-trained models, which inevitably suffer from fundamental distribution shifts relative to target private data. To alleviate this issue, researchers have proposed to leverage the implicit prior knowledge of an over-parameterized network. However, they only utilize a fixed neural architecture for all the attack settings. This would hinder the adaptive use of implicit architectural priors and consequently limit the generalizability. In this paper, we further exploit such implicit prior knowledge by proposing Gradient Inversion via Neural Architecture Search (GI-NAS), which adaptively searches the network and captures the implicit priors behind neural architectures. Extensive experiments verify that our proposed GI-NAS can achieve superior attack performance compared to state-of-the-art gradient inversion methods, even under more practical settings with high-resolution images, large-sized batches, and advanced defense strategies. To the best of our knowledge, we are the first to successfully introduce NAS to the gradient inversion community. We believe that this work exposes critical vulnerabilities in real-world federated learning by demonstrating high-fidelity reconstruction of sensitive data without requiring domain-specific priors, forcing urgent reassessment of FL privacy safeguards.

Theq-ary maximum distance separable (MDS) array codes, especiallyq= 2v, have important applications in the storage systems to prevent data loss. At now, almost all algebraic constructions of such codes are carried out over the quotient ring Rm= Fq[x]/⟨xm+1⟩ or its subring Fq[x]/⟨xm−1+xm−2+· · ·+1⟩. In this paper, we consider the construction of MDS array codes derived form Rm,λ= Fq[x]/⟨xm+ λ⟩, where λ ∈ F∗q= Fq\{0}, gcd(m, q) = 1. By giving λ-GCD Constraint of polynomials and the punctured λ-twisted circulant matrices, a generic construction and its extensions of such codes are presented. Under this framework, four new explicit constructions are also provided. Specifically, some low-density MDS array codes are obtained in Explicit Constructions I and I ′ . Explicit Constructions II and III provide long MDS array codes that can be applied in the large-scale storage systems. Explicit Construction IV gives MDS array codes both with the optimal repair bandwidth and the lowest update complexity, whose sub-packetization level is smaller than that of codes in [31]. In addition, by the LU factorization, a decoding method for erasures is obtained. For long codes, the syndromes can be computed fast. When the erased number ρ ≤ 3 (the most common errors in the storage systems), its computational complexity is asymptotically optimal. By an example, the new encoder possesses fewer exclusive ORs per data bit than that of MDS array codes constructed over Rmin [29].

At present, existing multi-receiver authentication (MRA) schemes can only handle situations where the capacities of all receivers in the schemes are the same. However, in reality, different receivers may need to have different storage capacities. In this paper, inspired by the secret sharing scheme based on the Chinese Remainder Theorem (CRT) for polynomial rings where each participant holds the share with different sizes, we propose three new constructions of unconditionally secure MRA schemes for multiple messages using the CRT for polynomial rings, including a$(k,n)$-threshold MRA scheme, a$(k,n,\omega)$-weighted threshold MRA scheme, and a$(\mathcal {Q},\mathcal {F})$-general MRA scheme. As far as we know, our proposed MRA schemes are the first MRA schemes with different storage capacities for different receivers and the first ones based on the CRT for polynomial rings. Moreover, the proposed schemes can be seen as extensions of the MRA scheme in Safavi-Naini and Wang. In particular, as for our$(\mathcal {Q},\mathcal {F})$-general MRA scheme, it has generally more communication complexity and much less computation complexity than the existing$(\mathcal {Q},\mathcal {F})$-general MRA scheme.

The symbol-pair coding theory was put forward by Cassuto and Blaum [IEEE TIT, 2011] to be applicable in high-density storage situations. Yaakobiet al. [IEEE TIT, 2016] extended the concept of symbol-pair metric tob-symbol metric whenb≥ 2. The extensive research on theb-symbol Hamming weight spectra of cyclic codes has been centered on the case where the alphabet is a finite field. The case of cyclic codes over Z4, an extremely important class of codes, has been overlooked for a long time in the exploration of theb-symbol Hamming weight spectra. In this paper, we study theb-symbol Hamming weight spectra of the shortened Kerdock codesK−mand the Kerdock codesKmover Z4. The formulas for calculating the symbol-pair Hamming weight of the codewords inK−mandKmare given, and their values hinge on the trace-like function values of specific elements in the Teichmüller set. In particular, we present a class of Z4-cyclic codes with three non-zerob-symbol Hamming weights. As by-products, theb-symbol Hamming weight hierarchies of the Preparata codes and the Goethals codes are provided.

The maximum distance separable (MDS) codes are the optimal codes to achieve Singleton bound, providing maximum error tolerance under a given number of parity nodes. Ye and Barg leveraged permutation matrices and Reed-Solomon type codes to devise 7 explicit constructions for constructing MDS array codes with optimal repair property (as known as MSR codes) or even optimal access property. Drawing inspiration from these explicit constructions, we provide two generic constructions for constructing MSR codes from high-rate MDS codes or MDS array codes. In this paper, we introduce the concepts of s-pairwise MDS codes sets and s-pairwise MDS array codes sets. Two generic constructions (Generic Constructions IandII) for constructing the MSR code using thes-pairwise MDS codes sets or thes-pairwise MDS array codes sets are given. Constructions 1 to 3 proposed by Ye and Barg can be regarded as some special cases ofGeneric Construction I, and Constructions 1 to 3 proposed by Li et al., can be regarded as some special cases ofGeneric Construction II. It is worth mentioning thatGeneric Construction IIcan be applied to any finite field, including the binary field. We also demonstrate how to obtain thes-pairwise MDS code sets and thes-pairwise MDS array code sets from a high-rate MDS code or MDS array code over$\mathbb {F}_{q}$. We obtain a novel class of MSR codes by utilizing the MDS array codes provided by Lv et al., as component codes according toGeneric Construction II. As a byproduct of Constructions 4 to 7, we obtain a new class of MSR codes with optimal access property. Using two types of sets$\Gamma _{1}$and$\Gamma _{2}$with the property that matrices commute, we present several new constructions for MSR codes with the optimal access property over any finite field. In this paper, compared with the constructions over binary field proposed by Li et al., the sub-packetization of our constructions applicable to the binary field is significantly reduced.

Displaying high-quality images on edge devices, such as augmented reality devices, is essential for enhancing the user experience. However, these devices often face power consumption and computing resource limitations, making it challenging to apply many deep learning-based image compression algorithms in this field. Implicit Neural Representation (INR) for image compression is an emerging technology that offers two key benefits compared to cutting-edge autoencoder models: low computational complexity and parameter-free decoding. It also outperforms many traditional and early neural compression methods in terms of quality. In this study, we introduce a new Mixed AutoRegressive Model (MARM) to significantly reduce the decoding time for the current INR codec, along with a new synthesis network to enhance reconstruction quality. MARM includes our proposed AutoRegressive Upsampler (ARU) blocks, which are highly computationally efficient, and ARM from previous work to balance decoding time and reconstruction quality. We also propose enhancing ARU's performance using a checkerboard two-stage decoding strategy. Moreover, the ratio of different modules can be adjusted to maintain a balance between quality and speed. Comprehensive experiments demonstrate that our method significantly improves computational efficiency while preserving image quality. With different parameter settings, our method can achieve over a magnitude acceleration in decoding time without industrial level optimization or achieve state-of-the-art reconstruction quality compared with other INR codecs. To the best of our knowledge, our method is the first INR-based codec comparable with Ballé et al. [1] in both decoding speed and quality while maintaining low complexity.

Data clustering is a fundamental machine learning task that seeks to categorize a dataset into homogeneous groups. However, real data usually contain noise, which poses significant challenges to clustering algorithms. In this article, motivated by how the k-means algorithm is derived from a Gaussian mixture model (GMM), we propose a robust k-means-type algorithm, named k-means-type clustering based on t-distribution (KMTD), by assuming that the data points are drawn from a special multivariate t-mixture model (TMM). Compared to the Gaussian distribution, the t-distribution has a fatter tail. The proposed algorithm is more robust to noise. Like the k-means algorithm, the proposed algorithm is simpler than those based on a full TMM. Both synthetic and actual data are used to illustrate the proposed algorithm's performance and efficiency. The experimental results demonstrated that the proposed algorithm operates more quickly than other sophisticated algorithms and, in most cases, achieves higher accuracy than the other algorithms.

In today’s big data era, a key requirement is to implement intelligent semantic analysis (such as image recognition) on data gathered from an extensive array of smart devices in Artificial Intelligence IoT (AIoT) scenarios, all of which is processed at central cloud service providers. Recent advancements in deep-learning-based image compression have fostered semantic compression between machines. However, the deployment of an overparameterized encoder on Internet of Things (IoT) devices remains a challenge due to their restricted computing and storage capabilities. To tackle this issue, we propose a novel approach named compressed sensing (CS)-based asymmetric semantic image compression (CS-ASIC), explicitly designed for resource-constrained AIoT systems. This asymmetric semantic compression scheme intends to surpass the limitations of IoT devices, thereby facilitating efficient semantic compression for machine vision tasks. CS-ASIC notably includes a lightweight front encoder founded on deep image CS techniques, which utilizes rich image priors to learn measurement matrices for sampling. In tandem, a deep iterative decoder is designed cooperatively with the linear encoder offloaded at the server to enhance image reconstruction and semantic analysis across various semantic analysis tasks. Furthermore, we introduce a groundbreaking lossy CS semantic rate-distortion theoretical framework that justifies a compromise in rate for extended semantic distortion. Extensive experimental results underscore the superiority of the proposed CS-ASIC concerning the signal-semantic rate-distortion tradeoff, and its lower encoding complexity over existing codecs in an AIoT simulation environment.

Given a text query, partially relevant video retrieval (PRVR) seeks to find untrimmed videos containing pertinent moments in a database. For PRVR, clip modeling is essential to capture the partial relationship between texts and videos. Current PRVR methods adopt scanning-based clip construction to achieve explicit clip modeling, which is information-redundant and requires a large storage overhead. To solve the efficiency problem of PRVR methods, this paper proposes GMMFormer, a Gaussian-Mixture-Model based Transformer which models clip representations implicitly. During frame interactions, we incorporate Gaussian-Mixture-Model constraints to focus each frame on its adjacent frames instead of the whole video. Then generated representations will contain multi-scale clip information, achieving implicit clip modeling. In addition, PRVR methods ignore semantic differences between text queries relevant to the same video, leading to a sparse embedding space. We propose a query diverse loss to distinguish these text queries, making the embedding space more intensive and contain more semantic information. Extensive experiments on three large-scale video datasets (i.e., TVR, ActivityNet Captions, and Charades-STA) demonstrate the superiority and efficiency of GMMFormer.

Partial label learning (PLL) learns from training examples each associated with multiple candidate labels, among which only one is valid. In recent years, benefiting from the strong capability of dealing with ambiguous supervision and the impetus of modern data augmentation methods, consistency regularization-based PLL methods have achieved a series of successes and become mainstream. However, as the partial annotation becomes insufficient, their performances drop significantly. In this paper, we leverage easily accessible unlabeled examples to facilitate the partial label consistency regularization. In addition to a partial supervised loss, our method performs a controller-guided consistency regularization at both the label-level and representation-level with the help of unlabeled data. To minimize the disadvantages of insufficient capabilities of the initial supervised model, we use the controller to estimate the confidence of each current prediction to guide the subsequent consistency regularization. Furthermore, we dynamically adjust the confidence thresholds so that the number of samples of each class participating in consistency regularization remains roughly equal to alleviate the problem of class-imbalance. Experiments show that our method achieves satisfactory performances in more practical situations, and its modules can be applied to existing PLL methods to enhance their capabilities.

Learning 3D representation plays a critical role in masked autoencoder (MAE) based pre-training methods for point cloud, including single-modal and cross-modal based MAE. Specifically, although cross-modal MAE methods learn strong 3D representations via the auxiliary of other modal knowledge, they often suffer from heavy computational burdens and heavily rely on massive cross-modal data pairs that are often unavailable, which hinders their applications in practice. Instead, single-modal methods with solely point clouds as input are preferred in real applications due to their simplicity and efficiency. However, such methods easily suffer from limited 3D representations with global random mask input. To learn compact 3D representations, we propose a simple yet effective Point Feature Enhancement Masked Autoencoders (Point-FEMAE), which mainly consists of a global branch and a local branch to capture latent semantic features. Specifically, to learn more compact features, a share-parameter Transformer encoder is introduced to extract point features from the global and local unmasked patches obtained by global random and local block mask strategies, followed by a specific decoder to reconstruct. Meanwhile, to further enhance features in the local branch, we propose a Local Enhancement Module with local patch convolution to perceive fine-grained local context at larger scales. Our method significantly improves the pre-training efficiency compared to cross-modal alternatives, and extensive downstream experiments underscore the state-of-the-art effectiveness, particularly outperforming our baseline (Point-MAE) by 5.16%, 5.00%, and 5.04% in three variants of ScanObjectNN, respectively. Code is available at https://github.com/zyh16143998882/AAAI24-PointFEMAE.

In recent years, vision language pre-training frameworks have made significant progress in natural language processing and computer vision, achieving remarkable performance improvement on various downstream tasks. However, when extended to point cloud data, existing works mainly focus on building task-specific models, and fail to extract universal 3D vision-language embedding that generalize well. We carefully investigate three common tasks in semantic 3D scene understanding, and derive key insights into the development of a pre-training model. Motivated by these observations, we propose a vision-language pre-training framework 3DVLP (3D vision-language pre-training with object contrastive learning), which transfers flexibly on 3D vision-language downstream tasks. 3DVLP takes visual grounding as the proxy task and introduces Object-level IoU-guided Detection (OID) loss to obtain high-quality proposals in the scene. Moreover, we design Object-level Cross-Contrastive alignment (OCC) task and Object-level Self-Contrastive learning (OSC) task to align the objects with descriptions and distinguish different objects in the scene, respectively. Extensive experiments verify the excellent performance of 3DVLP on three 3D vision-language tasks, reflecting its superiority in semantic 3D scene understanding. Code is available at https://github.com/iridescentttt/3DVLP.

Contrastive Vision-Language Pre-training, known as CLIP, has shown promising effectiveness in addressing downstream image recognition tasks. However, recent works revealed that the CLIP model can be implanted with a downstream-oriented backdoor. On downstream tasks, one victim model performs well on clean samples but predicts a specific target class whenever a specific trigger is present. For injecting a backdoor, existing attacks depend on a large amount of additional data to maliciously fine-tune the entire pre-trained CLIP model, which makes them inapplicable to data-limited scenarios. In this work, motivated by the recent success of learnable prompts, we address this problem by injecting a backdoor into the CLIP model in the prompt learning stage. Our method named BadCLIP is built on a novel and effective mechanism in backdoor attacks on CLIP, i.e., influencing both the image and text encoders with the trigger. It consists of a learnable trigger applied to images and a trigger-aware context generator, such that the trigger can change text features via trigger-aware prompts, resulting in a powerful and generalizable attack. Extensive experiments conducted on 11 datasets verify that the clean accuracy of BadCLIP is similar to those of advanced prompt learning methods and the attack success rate is higher than 99% in most cases. BadCLIP is also generalizable to unseen classes, and shows a strong generalization capability under cross-dataset and cross-domain settings. The code is available at https://github.com/jiawangbai/BadCLIP.

Given the power of vision transformers, a new learning paradigm, pre-training and then prompting, makes it more efficient and effective to address downstream visual recog-nition tasks. In this paper, we identify a novel security threat towards such a paradigm from the perspective of back-door attacks. Specifically, an extra prompt token, called the switch token in this work, can turn the backdoor mode on, i.e., converting a benign model into a backdoored one. Once under the backdoor mode, a specific trigger can force the model to predict a target class. It poses a severe risk to the users of cloud API, since the malicious behavior can not be activated and detected under the benign mode, thus making the attack very stealthy. To attack a pre-trained model, our proposed attack, named SWARM, learns a trigger and prompt tokens including a switch token. They are optimized with the clean loss which encourages the model always be-haves normally even the trigger presents, and the backdoor loss that ensures the backdoor can be activated by the trig-ger when the switch is on. Besides, we utilize the cross-mode feature distillation to reduce the effect of the switch token on clean samples. The experiments on diverse vi-sual recognition tasks confirm the success of our switchable backdoor attack, i.e., achieving 95%+ attack success rate, and also being hard to be detected and removed. Our code is available at https://github.com/20000yshust/SWARM.

Data augmentation has been widely applied in anomaly detection, which generates synthetic anomalous data for training. However, most existing anomaly augmentation methods focus on image-level cut-and-paste techniques, resulting in less realistic synthetic results, and are restricted to a few predefined patterns. In this paper, we propose our Controllable Anomaly Generator (CAGen) for anomaly data augmentation, which can generate high-quality images, and be flexibly controlled with text prompts. Specifically, our method fine-tunes a ControlNet model by using binary masks and textual prompts to control the spatial localization and style of generated anomalies. To further augment the resemblance between the generated features and normal samples, we propose a fusion method that integrates the generated anomalous features with the features of normal samples. Experiments on standard anomaly detection benchmarks show that the proposed data augmentation method significantly leads to a 0.4/3.1 improvement in the AUROC/AP metric.

Recent CNN and Transformer-based models tried to utilize frequency and periodicity information for long-term time series forecasting. However, most existing work is based on Fourier transform, which cannot capture fine-grained and local frequency structure. In this paper, we propose a Wavelet-Fourier Transform Network (WFTNet) for long-term time series forecasting. WFTNet utilizes both Fourier and wavelet transforms to extract comprehensive temporal-frequency information from the signal, where Fourier transform captures the global periodic patterns and wavelet transform captures the local ones. Furthermore, we introduce a Periodicity-Weighted Coefficient (PWC) to adaptively balance the importance of global and local frequency patterns. Extensive experiments on various time series datasets show that WFTNet consistently outperforms other state-of-the-art baseline. Code is available at https://github.com/Hank0626/WFTNet.

Recent studies have demonstrated that diffusion probabilistic models (DPMs) have numerous advantages in image generation through learning a decodable latent representation. This characteristic makes DPMs an appropriate reversible model for encoding and decoding of image watermarking. We present WaterDiff, which leverages pretrained DPMs for perceptual image watermarking problem. Specifically, WaterDiff embeds the watermark into the decomposed stochastic feature, then the stochastic features is combined with the corresponding semantic latent vector to produce a watermarked image via DPMs. This process balances the perceptual quality (stealthiness) and watermarking capacity by fully exploiting the latent diffusion prior. Extensive experiments indicate that WaterDiff guarantee both perceptual imperceptibility and robustness against state-of-the-art watermarking attacks.

In this paper, we propose a progressive learning paradigm for transformer-based variable-rate image compression. Our approach covers a wide range of compression rates with the assistance of the Layer-adaptive Prompt Module (LPM). Inspired by visual prompt tuning, we use LPM to extract prompts for input images and hidden features at the encoder side and decoder side, respectively, which are fed as additional information into the swin transformer layer of a pre-trained transformer-based image compression model to affect the allocation of attention region and the bits, which in turn changes the target compression ratio of the model. To ensure the network is more lightweight, we involves the integration of prompt networks with less convolutional layers. Exhaustive experiments show that compared to methods based on multiple models, which are optimized separately for different target rates, the proposed method arrives at the same performance with 80% savings in parameter storage and 90% savings in datasets. Meanwhile, our model outperforms all current variable bitrate image methods in terms of rate-distortion performance and approaches the state-of-the-art fixed bitrate image compression methods trained from scratch.

Convolutional neural network (CNN)-based and Transformer-based methods have recently made significant strides in time series forecasting, which excel at modeling local temporal variations or capturing long-term dependencies. However, real-world time series usually contain intricate temporal patterns, thus making it challenging for existing methods that mainly focus on temporal variations modeling from the 1D time series directly. Based on the intrinsic periodicity of time series, we propose a novel Periodicity Decoupling Framework (PDF) to capture 2D temporal variations of decoupled series for long-term series forecasting. Our PDF mainly consists of three components: multi-periodic decoupling block (MDB), dual variations modeling block (DVMB), and variations aggregation block (VAB). Unlike the previous methods that model 1D temporal variations, our PDF mainly models 2D temporal variations, decoupled from 1D time series by MDB. After that, DVMB attempts to further capture short-term and long-term variations, followed by VAB to make final predictions. Extensive experimental results across seven real-world long-term time series datasets demonstrate the superiority of our method over other state-of-the-art methods, in terms of both forecasting performance and computational efficiency. Code is available at https://github.com/Hank0626/PDF.

Large vision-language models (VLMs) such as GPT-4 have achieved exceptional performance across various multi-modal tasks. However, the deployment of VLMs necessitates substantial energy consumption and computational resources. Once attackers maliciously induce high energy consumption and latency time (energy-latency cost) during inference of VLMs, it will exhaust computational resources. In this paper, we explore this attack surface about availability of VLMs and aim to induce high energy-latency cost during inference of VLMs. We find that high energy-latency cost during inference of VLMs can be manipulated by maximizing the length of generated sequences. To this end, we propose verbose images, with the goal of crafting an imperceptible perturbation to induce VLMs to generate long sentences during inference. Concretely, we design three loss objectives. First, a loss is proposed to delay the occurrence of end-of-sequence (EOS) token, where EOS token is a signal for VLMs to stop generating further tokens. Moreover, an uncertainty loss and a token diversity loss are proposed to increase the uncertainty over each generated token and the diversity among all tokens of the whole generated sequence, respectively, which can break output dependency at token-level and sequence-level. Furthermore, a temporal weight adjustment algorithm is proposed, which can effectively balance these losses. Extensive experiments demonstrate that our verbose images can increase the length of generated sequences by 7.87× and 8.56× compared to original images on MS-COCO and ImageNet datasets, which presents potential challenges for various applications.

Saliency-based representation visualization (SRV) ($e.g.$, Grad-CAM) is one of the most classical and widely adopted explainable artificial intelligence (XAI) methods for its simplicity and efficiency. It can be used to interpret deep neural networks by locating saliency areas contributing the most to their predictions. However, it is difficult to automatically measure and evaluate the performance of SRV methods due to the lack of ground-truth salience areas of samples. In this paper, we revisit the backdoor-based SRV evaluation, which is currently the only feasible method to alleviate the previous problem. We first reveal its \emph{implementation limitations} and \emph{unreliable nature} due to the trigger generalization of existing backdoor watermarks. Given these findings, we propose a generalization-limited backdoor watermark (GLBW), based on which we design a more faithful XAI evaluation. Specifically, we formulate the training of watermarked DNNs as a min-max problem, where we find the `worst' potential trigger (with the highest attack effectiveness and differences from the ground-truth trigger) via inner maximization and minimize its effects and the loss over benign and poisoned samples via outer minimization in each iteration. In particular, we design an adaptive optimization method to find desired potential triggers in each inner maximization. Extensive experiments on benchmark datasets are conducted, verifying the effectiveness of our generalization-limited watermark. Our codes are available at \url{https://github.com/yamengxi/GLBW}.

In this short work, a new explicit construction of$q-\mathbf{ary}$MDS array codes with multiple parities will be provided, whose code lengths can be up to$q^{m-1}$, where$m-1$is the size of subpackage. As far as we know, this may be the first explicit construction of practical MDS array codes with such long code lengths for general$q$. In addition, to demonstrate the applicability of our MDS array codes, by the LU factorization of Vandermonde matrices, we present an efficient decoding method aimed at the erased errors, whose computational complexity is$O(m^{2})$in total. Furthermore, if one stores a small number of polynomials in advance or computes the syndrome in a scheduled algorithm, the decoding efficiency of these MDS array codes can be further improved.

At present, existing multi-receiver authentication (MRA) schemes can only handle situations where the capabilities of all receivers are the same. However, in reality, different receivers may need to have different storage capabilities. In this paper, inspired by the secret sharing scheme based on the Chinese Remainder Theorem (CRT) for polynomial rings where distinct participants save shares with distinct sizes, we propose a new unconditionally secure MRA scheme for multiple messages by the same technique. As far as we know, our MRA scheme is the first MRA scheme with different storage capacities for different receivers and the first one based on the CRT for polynomial rings supporting general access structures. In contrast to the existing general MRA scheme, although our MRA scheme has more communication complexity, it has less computation complexity.

In this paper, for the first time, we propose a new explicit hierarchical threshold secret sharing (HTSS) scheme based on the Chinese Remainder Theorem (CRT) for polynomial rings, where the participant set is divided into disjoint subsets and the threshold of a superior subset is less than the threshold of an inferior subset. In addition, we present a rigorous security analysis to show that our HTSS scheme is both perfect and ideal. Moreover, a toy example of our HTSS scheme is given to enable readers to better understand our construction. By comparison, it appears that our scheme is the first CRT-based HTSS for polynomial rings and also the first ideal and perfect CRT-based HTSS scheme, which is easier to construct than its counterpart for integer rings, where different participants hold shares of different sizes. Besides, our HTSS can also distribute shares of the same size, similar to other HTSS.

Recently, deep learning has demonstrated improvements over classical decoding algorithms in various families of error-correcting codes with short to moderate block lengths. Neural decoders like neural belief propagation (NBP), constructed based on the Tanner graphs of linear codes, have been extensively studied, but their application to longer codes is limited due to increased network complexity with longer block lengths. This complexity leads to higher computational costs and deployment challenges, such as GPU memory usage. To address this, we propose a novel revolving framework for NBP-like decoders tailored to QC-LDPC codes, a commonly used class of error correction codes. Our approach leverages the section-wise cyclic structure inherent in QC-LDPC codes, significantly simplifying the complexity of the network. Experimental results demonstrate the effectiveness of our method across various QC-LDPC codes. Especially, compared to the traditional decoding scheme based on the section-wise cyclic structure, our proposed decoder shows substantial improvements on 5G LDPC codes, exhibits better performance than the traditional min-sum decoder, and approaches the sum-product decoder without a noticeable error floor within the investigated noise levels.

Pose-agnostic anomaly detection refers to the situation where the pose of test samples is inconsistent with the training dataset, allowing anomalies to appear at any position in any pose. We propose a novel method IGSPAD to address this challenge. Specifically, we employ 3D Gaussian splatting to represent the normal information from the training dataset. To accurately determine the pose of the test sample, we introduce an approach termed Inverting 3D Gaussian Splatting (IGS) to address the challenge of 6D pose estimation for anomalous images. The pose derived from IGS is utilized to render a normal image well-aligned with the test sample. Subsequently, the image encoder of the Segment Anything Model is employed to identify discrepancies between the rendered image and the test sample, predicting the location of anomalies. Experimental results on the MAD dataset demonstrate that the proposed method significantly surpasses the existing state-of-the-art method in terms of precision (from 97.8% to 99.7% at pixel level and from 90.9% to 98.0% at image level) and efficiency.

Deep neural networks (DNNs) have recently achieved remarkable advancements in time series forecasting (TSF) due to their powerful ability of sequence dependence modeling. To date, existing DNN-based TSF methods still suffer from unreliable predictions for real-world data due to its non-stationarity characteristics, i.e., data distribution varies quickly over time. To mitigate this issue, several normalization methods (e.g., SAN) have recently been specifically designed by normalization in a fixed period/window in the time domain. However, these methods still struggle to capture distribution variations, due to the complex time patterns of time series in the time domain. Based on the fact that wavelet transform can decompose time series into a linear combination of different frequencies, which exhibits distribution variations with time-varying periods, we propose a novel Dual-domain Dynamic Normalization (DDN) to dynamically capture distribution variations in both time and frequency domains. Specifically, our DDN tries to eliminate the non-stationarity of time series via both frequency and time domain normalization in a sliding window way. Besides, our DDN can serve as a plug-in-play module, and thus can be easily incorporated into other forecasting models. Extensive experiments on public benchmark datasets under different forecasting models demonstrate the superiority of our DDN over other normalization methods. Code will be made available following the review process.

Adaptation of pretrained vision-language models such as CLIP to various downstream tasks have raised great interest in recent researches. Previous works have proposed a variety of test-time adaptation (TTA) methods to achieve strong generalization without any knowledge of the target domain. However, existing training-required TTA approaches like TPT necessitate entropy minimization that involves large computational overhead, while training-free methods like TDA overlook the potential for information mining from the test samples themselves. In this paper, we break down the design of existing popular training-required and training-free TTA methods and bridge the gap between them within our framework. Specifically, we maintain a light-weight key-value memory for feature retrieval from instance-agnostic historical samples and instance-aware boosting samples. The historical samples are filtered from the testing data stream and serve to extract useful information from the target distribution, while the boosting samples are drawn from regional bootstrapping and capture the knowledge of the test sample itself. We theoretically justify the rationality behind our method and empirically verify its effectiveness on both the out-of-distribution and the cross-domain datasets, showcasing its applicability in real-world situations.

Designing single-task image restoration models for specific degradation has seen great success in recent years. To achieve generalized image restoration, all-in-one methods have recently been proposed and shown potential for multiple restoration tasks using one single model. Despite the promising results, the existing all-in-one paradigm still suffers from high computational costs as well as limited generalization on unseen degradations. In this work, we introduce an alternative solution to improve the generalization of image restoration models. Drawing inspiration from recent advancements in Parameter Efficient Transfer Learning (PETL), we aim to tune only a small number of parameters to adapt pre-trained restoration models to various tasks. However, current PETL methods fail to generalize across varied restoration tasks due to their homogeneous representation nature. To this end, we propose AdaptIR, a Mixture-of-Experts (MoE) with orthogonal multi-branch design to capture local spatial, global spatial, and channel representation bases, followed by adaptive base combination to obtain heterogeneous representation for different degradations. Extensive experiments demonstrate that our AdaptIR achieves stable performance on single-degradation tasks, and excels in hybrid-degradation tasks, with training only 0.6% parameters for 8 hours.

Recent advances in diffusion-based Large Restoration Models (LRMs) have significantly improved photo-realistic image restoration by leveraging the internal knowledge embedded within model weights. However, existing LRMs often suffer from the hallucination dilemma, i.e., producing incorrect contents or textures when dealing with severe degradations, due to their heavy reliance on limited internal knowledge. In this paper, we propose an orthogonal solution called the Retrieval-augmented Framework for Image Restoration (ReFIR), which incorporates retrieved images as external knowledge to extend the knowledge boundary of existing LRMs in generating details faithful to the original scene. Specifically, we first introduce the nearest neighbor lookup to retrieve content-relevant high-quality images as reference, after which we propose the cross-image injection to modify existing LRMs to utilize high-quality textures from retrieved images. Thanks to the additional external knowledge, our ReFIR can well handle the hallucination challenge and facilitate faithfully results. Extensive experiments demonstrate that ReFIR can achieve not only high-fidelity but also realistic restoration results. Importantly, our ReFIR requires no training and is adaptable to various LRMs.

Vision-and-Language Navigation (VLN) requires an agent to dynamically explore environments following natural language. The VLN agent, closely integrated into daily lives, poses a substantial threat to the security of privacy and property upon the occurrence of malicious behavior. However, this serious issue has long been overlooked. In this paper, we pioneer the exploration of an object-aware backdoored VLN, achieved by implanting object-aware backdoors during the training phase. Tailored to the unique VLN nature of cross-modality and continuous decision-making, we propose a novel backdoored VLN paradigm: IPR Backdoor. This enables the agent to act in abnormal behavior once encountering the object triggers during language-guided navigation in unseen environments, thereby executing an attack on the target scene. Experiments demonstrate the effectiveness of our method in both physical and digital spaces across different VLN agents, as well as its robustness to various visual and textual variations. Additionally, our method also well ensures navigation performance in normal scenarios with remarkable stealthiness.

The pre-trained point cloud model based on Masked Point Modeling (MPM) has exhibited substantial improvements across various tasks. However, these models heavily rely on the Transformer, leading to quadratic complexity and limited decoder, hindering their practice application. To address this limitation, we first conduct a comprehensive analysis of existing Transformer-based MPM, emphasizing the idea that redundancy reduction is crucial for point cloud analysis. To this end, we propose a Locally constrained Compact point cloud Model (LCM) consisting of a locally constrained compact encoder and a locally constrained Mamba-based decoder. Our encoder replaces self-attention with our local aggregation layers to achieve an elegant balance between performance and efficiency. Considering the varying information density between masked and unmasked patches in the decoder inputs of MPM, we introduce a locally constrained Mamba-based decoder. This decoder ensures linear complexity while maximizing the perception of point cloud geometry information from unmasked patches with higher information density. Extensive experimental results show that our compact model significantly surpasses existing Transformer-based models in both performance and efficiency, especially our LCM-based Point-MAE model, compared to the Transformer-based model, achieved an improvement of 1.84%, 0.67%, and 0.60% in performance on the three variants of ScanObjectNN while reducing parameters by 88% and computation by 73%. The code is available at https://github.com/zyh16143998882/LCM.

Different from data-oriented communication systems that primarily focus on how to accurately transmit every bit of data, task-oriented semantic communication systems only transmit the specific semantic information required by downstream tasks, strive to minimize the communication overhead and maintain competitive tasks execution performance in the presence of channel noise. However, it is worth noting that in many scenarios, the transmitted semantic information needs to be dynamically modified according to the users' preferences in a conversational and interactive way, which few existing works take into consideration. In this paper, we propose a novel cross-modal editable semantic communication system, named Editable-DeepSC, to tackle this challenge. By utilizing inversion methods based on StyleGAN priors, Editable-DeepSC takes cross-modal text-image pairs as the inputs and transmits the edited information of images based on textual instructions. Extensive numerical results demonstrate that our proposed Editable-DeepSC can achieve remarkable editing effects and transmission efficiency under the perturbations of channel noise, outperforming existing data-oriented communication methods.

With the thriving of deep learning in processing point cloud data, recent works show that backdoor attacks pose a severe security threat to 3D vision applications. The attacker injects the backdoor into the 3D model by poisoning a few training samples with trigger, such that the backdoored model performs well on clean samples but behaves maliciously when the trigger pattern appears. Existing attacks often insert some additional points into the point cloud as the trigger, or utilize a linear transformation (e.g., rotation) to construct the poisoned point cloud. However, the effects of these poisoned samples are likely to be weakened or even eliminated by some commonly used pre-processing techniques for 3D point cloud, e.g., outlier removal or rotation augmentation. In this paper, we propose a novel imperceptible and robust backdoor attack (IRBA) to tackle this challenge. We utilize a nonlinear and local transformation, called weighted local transformation (WLT), to construct poisoned samples with unique transformations. As there are several hyper-parameters and randomness in WLT, it is difficult to produce two similar transformations. Consequently, poisoned samples with unique transformations are likely to be resistant to aforementioned pre-processing techniques. Besides, the distortion caused by a fixed WLT is both controllable and smooth, resulting in the generated poisoned samples that are imperceptible to human inspection. Extensive experiments on three benchmark datasets and four models show that IRBA achieves$80\%+$attack success rate (ASR) in most cases even with pre-processing techniques, which is significantly higher than previous state-of-the-art attacks. Our code is available athttps://github.com/KuofengGao/IRBA.

Deep neural networks (DNNs) are vulnerable to backdoor attacks, where the adversary manipulates a small portion of training data such that the victim model predicts normally on the benign samples but classifies the triggered samples as the target class. The backdoor attack is an emerging yet threatening training-phase threat, leading to serious risks in DNN-based applications. In this paper, we revisit the trigger patterns of existing backdoor attacks. We reveal that they are either visible or not sparse and therefore are not stealthy enough. More importantly, it is not feasible to simply combine existing methods to design an effective sparse and invisible backdoor attack. To address this problem, we formulate the trigger generation as a bi-level optimization problem with sparsity and invisibility constraints and propose an effective method to solve it. The proposed method is dubbed sparse and invisible backdoor attack (SIBA). We conduct extensive experiments on benchmark datasets under different settings, which verify the effectiveness of our attack and its resistance to existing backdoor defenses. The codes for reproducing main experiments are available athttps://github.com/YinghuaGao/SIBA.

MDS array codes have been extensively studied due to their applications in storage systems. In this paper, we first propose a novel method of constructing MDS array codes by deleting one row and one column from the circulant matrices associated to some polynomials. Several new classes of MDS array codes with flexible parameters are constructed. In particular, we give a new algebraic presentation of the Blaum-Roth codes with sparser parity-check matrices. We also obtain a family of MDS array codes over finite fields with even characteristics whose parity-check matrices have the lowest density. Furthermore, based on these new MDS array codes, we give a general construction of optimal locally repairable array codes (LRACs) achieving the Singleton-type bound. Additionally, we obtain some new optimal LRACs of long lengths. Finally, we present a scheduled algorithm for syndrome computations of binary optimal LRACs with redundancy 4, which can tolerate three failures. The number of XORs per data bit required in our algorithm approaches 2 as the length approaches infinity, which is the same as the MDS codes tolerating three failures. However, the number of nodes required during the repair of a failed node in our optimal LRACs is only about half of that in MDS array codes.

An$(n, k, d; r)_{q}$-locally repairable code (LRC) is called a Singleton-optimal LRC if it achieves the Singleton-type bound. Analogous to the classical MDS conjecture, the maximal length problem of Singleton-optimal LRCs has attracted a lot of attention in recent years. In this paper, we give an improved upper bound for the length of q-ary Singleton-optimal LRCs with disjoint repair groups such that$(r+1)\mid n$based on the parity-check matrix approach. In particular, for any Singleton-optimal$(n, k, d; r)_{q}$-LRCs, we show that: 1)$n\le q+d-4$, when$r=2$and$d=3e+8$with$e\ge 0$; 2)$n\leq (r+1)\left \lfloor {{\frac {2(q^{2}+q+1)}{r(r+1)} +e+1}}\right \rfloor $, when$d\ge 8$and$\max \left \{{{3,\frac {d-e-6}{e+1}}}\right \}\le r\le \frac {d-e-3}{e+1}$for any$0\le e\le \left \lfloor {{\frac {d-6}{4} }}\right \rfloor $. Furthermore, we establish equivalent connections between the existence of Singleton-optimal$(n,k,d;r)_{q}$-LRCs for$d=6, r=3$and$d=7, r=2$with disjoint repair groups and some subsets of lines in finite projective space with certain properties. Consequently, we prove that the length of q-ary Singleton-optimal LRCs with minimum distance$d=6$and locality$r=3$is upper bounded by$O(q^{1.5})$. We construct Singleton-optimal$(8\le n\le q+1,k,d=6,r=3)_{q}$-LRC with disjoint repair groups such that$4\mid n$and determine the exact value of the maximum code length for some specific q. We also prove the existence of$(n, k, d=7; r=2)_{q}$-Singleton-optimal LRCs for$n \approx \sqrt {2}q$.

Although adversarial training (AT) is regarded as a potential defense against backdoor attacks, AT and its variants have only yielded unsatisfactory results or have even inversely strengthened backdoor attacks. The large discrepancy between expectations and reality motivates us to thoroughly evaluate the effectiveness of AT against backdoor attacks across various settings for AT and backdoor attacks. We find that the type and budget of perturbations used in AT are important, and AT with common perturbations is only effective for certain backdoor trigger patterns. Based on these empirical findings, we present some practical suggestions for backdoor defense, including relaxed adversarial perturbation and composite AT. This work not only boosts our confidence in AT's ability to defend against backdoor attacks but also provides some important insights for future research.

Backdoor attack intends to embed hidden backdoors into deep neural networks (DNNs), so that the attacked models perform well on benign samples, whereas their predictions will be maliciously changed if the hidden backdoor is activated by attacker-specified triggers. This threat could happen when the training process is not fully controlled, such as training on third-party datasets or adopting third-party models, which poses a new and realistic threat. Although backdoor learning is an emerging and rapidly growing research area, there is still no comprehensive and timely review of it. In this article, we present the first comprehensive survey of this realm. We summarize and categorize existing backdoor attacks and defenses based on their characteristics, and provide a unified framework for analyzing poisoning-based backdoor attacks. Besides, we also analyze the relation between backdoor attacks and relevant fields (i.e., adversarial attacks and data poisoning), and summarize widely adopted benchmark datasets. Finally, we briefly outline certain future research directions relying upon reviewed works. A curated list of backdoor-related resources is also available at https://github.com/THUYimingLi/backdoor-learning-resources.

Real-world recognition system often encounters the challenge of unseen labels. To identify such unseen labels, multi-label zero-shot learning (ML-ZSL) focuses on transferring knowledge by a pre-trained textual label embedding (e.g., GloVe). However, such methods only exploit single-modal knowledge from a language model, while ignoring the rich semantic information inherent in image-text pairs. Instead, recently developed open-vocabulary (OV) based methods succeed in exploiting such information of image-text pairs in object detection, and achieve impressive performance. Inspired by the success of OV-based methods, we propose a novel open-vocabulary framework, named multi-modal knowledge transfer (MKT), for multi-label classification. Specifically, our method exploits multi-modal knowledge of image-text pairs based on a vision and language pre-training (VLP) model. To facilitate transferring the image-text matching ability of VLP model, knowledge distillation is employed to guarantee the consistency of image and label embeddings, along with prompt tuning to further update the label embeddings. To further enable the recognition of multiple objects, a simple but effective two-stream module is developed to capture both local and global features. Extensive experimental results show that our method significantly outperforms state-of-the-art methods on public benchmark datasets.

Beyond achieving higher compression efficiency over classical image compression codecs, deep image compression is expected to be improved with additional side information, e.g., another image from a different perspective of the same scene. To better utilize the side information under the distributed compression scenario, the existing method only implements patch matching at the image domain to solve the parallax problem caused by the difference in viewing points. However, the patch matching at the image domain is not robust to the variance of scale, shape, and illumination caused by the different viewing angles, and can not make full use of the rich texture information of the side information image. To resolve this issue, we propose Multi-Scale Feature Domain Patch Matching (MSFDPM) to fully utilizes side information at the decoder of the distributed image compression model. Specifically, MSFDPM consists of a side information feature extractor, a multi-scale feature domain patch matching module, and a multi-scale feature fusion network. Furthermore, we reuse inter-patch correlation from the shallow layer to accelerate the patch matching of the deep layer. Finally, we find that our patch matching in a multi-scale feature domain further improves compression rate by about 20% compared with the patch matching method at image domain.

Deep neural networks (DNNs) have witnessed remarkable achievement in image super-resolution (SR), and plenty of DNN-based SR models with elaborated network designs have recently been proposed. However, existing methods usually require substantial computations by operating in spatial domain. To address this issue, we propose a general frequency-oriented framework (FSR) to accelerate SR networks by considering data characteristics in frequency domain. Our FSR mainly contains dual feature aggregation module (DFAM) to extract informative features in both spatial and transform domains, followed by a four-path SR-Module with different capacities to super-resolve in the frequency domain. Specifically, DFAM further consists of a transform attention block (TABlock) and a spatial context block (SCBlock) to extract global spectral information and local spatial information, respectively, while SR-Module is a parallel network container that contains four to-be-accelerated branches. Furthermore, we propose an adaptive weight strategy for a trade-off between image details recovery and visual quality. Extensive experiments show that our FSR can save FLOPs by almost 40% while reducing inference time by 50% for other SR methods (e.g., FSRCNN, CARN, SRResNet and RCAN). Code is available at https://github.com/THU-Kingmin/FSR.

Self-Supervised Video Hashing (SSVH) models learn to generate short binary representations for videos without ground-truth supervision, facilitating large-scale video retrieval efficiency and attracting increasing research attention. The success of SSVH lies in the understanding of video content and the ability to capture the semantic relation among unlabeled videos. Typically, state-of-the-art SSVH methods consider these two points in a two-stage training pipeline, where they firstly train an auxiliary network by instance-wise mask-and-predict tasks and secondly train a hashing model to preserve the pseudo-neighborhood structure transferred from the auxiliary network. This consecutive training strategy is inflexible and also unnecessary. In this paper, we propose a simple yet effective one-stage SSVH method called ConMH, which incorporates video semantic information and video similarity relationship understanding in a single stage. To capture video semantic information for better hashing learning, we adopt an encoder-decoder structure to reconstruct the video from its temporal-masked frames. Particularly, we find that a higher masking ratio helps video understanding. Besides, we fully exploit the similarity relationship between videos by maximizing agreement between two augmented views of a video, which contributes to more discriminative and robust hash codes. Extensive experiments on three large-scale video datasets (i.e., FCVID, ActivityNet and YFCC) indicate that ConMH achieves state-of-the-art results. Code is available at https://github.com/huangmozhi9527/ConMH.

Models notoriously suffer from dataset biases which are detrimental to robustness and generalization. The identify-emphasize paradigm shows a promising effect in dealing with unknown biases. However, we find that it is still plagued by two challenges: A, the quality of the identified bias-conflicting samples is far from satisfactory; B, the emphasizing strategies just yield suboptimal performance. In this work, for challenge A, we propose an effective bias-conflicting scoring method to boost the identification accuracy with two practical strategies --- peer-picking and epoch-ensemble. For challenge B, we point out that the gradient contribution statistics can be a reliable indicator to inspect whether the optimization is dominated by bias-aligned samples. Then, we propose gradient alignment, which employs gradient statistics to balance the contributions of the mined bias-aligned and bias-conflicting samples dynamically throughout the learning process, forcing models to leverage intrinsic features to make fair decisions. Experiments are conducted on multiple datasets in various settings, demonstrating that the proposed solution can alleviate the impact of unknown biases and achieve state-of-the-art performance.

The dynamic nature of stock market styles, referred to as concept drift, poses a formidable challenge when applying deep learning to stock prediction. Models trained on historical data often struggle to adapt to the latest market styles, as the patterns they have learned may no longer hold true over time. To alleviate this issue, the recently popularized concept of In-Context learning has provided us with valuable insights. In this approach, large language models (LLMs) are exposed to multiple examples of input-label pairs, also known as demonstrations, as part of the prompt before performing a task on an unseen example. By thoroughly analyzing these demonstrations, LLMs can uncover potential patterns and effectively adapt to new tasks. Building upon this concept, we propose a Context-Informed drift-aware method for Stock Prediction (CISP), which continually adjusts to the latest market styles and offers more accurate predictions. Our proposed method consists of two key parts. Firstly, we introduce a straightforward and efficient technique for designing demonstrations that aggregate current market information, thereby indicating the prevailing stock market style. Secondly, we incorporate a prediction module with dynamic parameters, allowing it to appropriately adjust its model parameters based on the market patterns embedded in the aforementioned demonstrations. Through extensive experiments conducted on real-world stock market datasets, our approach consistently outperforms the most advanced existing methods for stock prediction.

Backdoor defenses have been studied to alleviate the threat of deep neural networks (DNNs) being backdoor attacked and thus maliciously altered. Since DNNs usually adopt some external training data from an untrusted third party, a robust backdoor defense strategy during the training stage is of importance. We argue that the core of training-time defense is to select poisoned samples and to handle them properly. In this work, we summarize the training-time defenses from a unified framework as splitting the poisoned dataset into two data pools. Under our framework, we propose an adaptively splitting dataset-based defense (ASD). Concretely, we apply loss-guided split and meta-learning-inspired split to dynamically update two data pools. With the split clean data pool and polluted data pool, ASD successfully defends against backdoor attacks during training. Extensive experiments on multiple benchmark datasets and DNN models against six state-of-the-art backdoor attacks demonstrate the superiority of our ASD. Our code is available at https://github.com/KuofengGao/ASD.

Pre-training on large-scale video data has become a common recipe for learning transferable spatiotemporal representations in recent years. Despite some progress, existing methods are mostly limited to highly curated datasets (e.g., K400) and exhibit unsatisfactory out-of-the-box representations. We argue that it is due to the fact that they only capture pixel-level knowledge rather than spatiotemporal semantics, which hinders further progress in video understanding. Inspired by the great success of image-text pre-training (e.g., CLIP), we take the first step to exploit language semantics to boost transferable spatiotemporal representation learning. We introduce a new pre-text task, Turning to Video for Transcript Sorting (TVTS), which sorts shuffled ASR scripts by attending to learned video representations. We do not rely on descriptive captions and learn purely from video, i.e., leveraging the natural transcribed speech knowledge to provide noisy but useful semantics over time. Our method enforces the vision model to contextualize what is happening over time so that it can re-organize the narrative transcripts, and can seamlessly apply to large-scale uncurated video data in the real world. Our method demonstrates strong out-of-the-box spatiotemporal representations on diverse benchmarks, e.g., +13.6% gains over VideoMAE on SSV2 via linear probing. The code is available at https://github.com/TencentARC/TVTS.

Recent studies revealed that deep neural networks (DNNs) are exposed to backdoor threats when training with third-party resources (such as training samples or backbones). The back-doored model has promising performance in predicting benign samples, whereas its predictions can be maliciously manipulated by adversaries based on activating its backdoors with pre-defined trigger patterns. Currently, most of the existing backdoor attacks were conducted on the image classification under the targeted manner. In this paper, we reveal that these threats could also happen in object detection, posing threatening risks to many mission-critical applications (e.g., pedestrian detection and intelligent surveillance systems). Specifically, we design a simple yet effective poison-only backdoor attack in an untargeted manner, based on task characteristics. We show that, once the backdoor is embedded into the target model by our attack, it can trick the model to lose detection of any object stamped with our trigger patterns. We conduct extensive experiments on the benchmark dataset, showing its effectiveness in both digital and physical-world settings and its resistance to potential defenses.

Deep neural network (DNN) based scene text recognition (STR) methods usually require a large amount of annotated data for training, which is time-consuming and cost-expensive in practice. To address this issue, many data augmentation methods have been developed to train recognizers by improving the diversity of training samples. However, most existing methods neglect the difficulty inherent in samples, and easily suffer from the problem of over-diversity, i.e., the distribution of the augmented data significantly deviates from that of clean data. In this paper, we propose a novel difficulty-aware data augmentation framework for scene text recognition, which jointly considers the difficulty of samples and the strength of augmentations. Specifically, our framework first predicts the sample difficulty, followed by an adaptive data augmentation strategy. Furthermore, we build a more diverse set of augmentation methods for STR and integrate it into our augmentation framework. Extensive experiments on scene text recognition benchmarks show that our augmentation framework significantly improves the performance of recognizers.

Recent years have witnessed a tremendous growth in the scale and resolution of point clouds. To facilitate the applications of point cloud in downsampling tasks (e.g., point cloud classification), several task-oriented downsampling works have been developed by training with the task-specific loss with one-hot encoded label. However, these methods still suffer from performance degradation at high downsampling scales. In this paper, we propose a general semantic-preserved downsampling framework (SPDF) for point clouds by exploiting the rich knowledge inherent in the task network. Specifically, we firstly refine the previous pipeline to generate richer semantic supervised information. Then, the semantic feature learning is subdivided into label-level and feature-level to guide the training of downsampling network, which can better limit the semantic loss during downsampling. Extensive experiments on the benchmark dataset show that SPDF outperforms state-of-the-art downsampling methods.

Deep neural networks (DNNs) are vulnerable to backdoor attacks. The backdoor adversaries intend to maliciously control the predictions of attacked DNNs by injecting hidden backdoors that can be activated by adversary-specified trigger patterns during the training process. One recent research revealed that most of the existing attacks failed in the real physical world since the trigger contained in the digitized test samples may be different from that of the one used for training. Accordingly, users can adopt spatial transformations as the image pre-processing to deactivate hidden backdoors. In this paper, we explore the previous findings from another side. We exploit classical spatial transformations (i.e., rotation and translation) with the specific parameter as trigger patterns to design a simple yet effective poisoning-based backdoor attack. For example, only images rotated to a particular angle can activate the embedded backdoor of attacked DNNs. Extensive experiments are conducted, verifying the effectiveness of our attack under both digital and physical settings and its resistance to existing backdoor defenses.

Recent studies have demonstrated that deep neural networks (DNNs) are vulnerable to backdoor attacks during the training process. Specifically, the adversaries intend to embed hidden backdoors in DNNs so that malicious model predictions can be activated through pre-defined trigger patterns. In this paper, we explore the backdoor mechanism from the angle of the model structure. We select the skip connection for discussions, inspired by the understanding that it helps the learning of model ‘shortcuts’ where backdoor triggers are usually easier to be learned. Specifically, we demonstrate that the attack success rate (ASR) decreases significantly when reducing the outputs of some key skip connections. Based on this observation, we design a simple yet effective backdoor removal method by suppressing the skip connections in critical layers selected by our method. We also implement fine-tuning on these layers to recover high benign accuracy and to further reduce ASR. Extensive experiments on benchmark datasets verify the effectiveness of our method.

Multi-view projection methods have demonstrated their ability to reach state-of-the-art performance in point cloud downstream tasks(e.g., classification and retrieval). These methods first require rendering the point cloud into 2D multi-view images. However, conventional methods only project the geometry of the point cloud, and such projections inevitably suffer from a loss of point cloud semantic information due to dimensionality reduction. We propose a semantic-aware and task-oriented differentiable feature rendering (SFR), which reduces the information loss during projection by generating rendered images with more point cloud semantic information for downstream tasks. Our SFR method can be applied as a plug-and-play module added to any multi-view-based backbone network for end-to-end training. Extensive experiments on benchmark datasets show that our SFR method reaches state-of-the-art performance and brings general improvements to point cloud classification and retrieval tasks.

As a pioneering and deep-learning driven neural channel code, Product Autoencoder (ProductAE) shows significant superiority over Turbo Autoencoder (TurboAE) and other classical codes. However, received noisy codewords with different noise levels have different decoding difficulties for the neural decoder of ProductAE. The existing neural decoder processes all noisy codewords equally without discrimination, and neglects the prior knowledge of signal-to-noise ratio (SNR), thus leading to high decoding complexity. In this paper, we propose a new approach to speed up the decoding process of ProductAE. Intuitively, noisy codewords with different SNR can be recovered by decoders of different complexity, and our proposed novel pipeline, Adaptive ProductAE, an SNR-aware adaptive decoding strategy, adopts deep decoders to different SNR. Specifically, adaptive ProductAE combines an encoding module, an additional classification module, and two independent decoding branches in a unified framework. After receiving noisy codewords, it first judges the decoding difficulty of each signal and assigns them to different branches to get the decoded messages. Furthermore, we introduce a novel training strategy with a gap loss to maintain the classification and decoding performance. It can thus switch to a simpler branch of decoding networks automatically when it comes to signals with a higher SNR, and the overall computational cost can be reduced. Experiments show that our adaptive ProductAE saves up to 30.4% FLOPs for a moderate-length code of parameters (225, 100) and 25.5% FLOPs for a code of parameters (441, 196) in a higher SNR range.

Deep neural networks (DNNs) are widely deployed on real-world devices. Concerns regarding their security have gained great attention from researchers. Recently, a new weight modification attack called bit flip attack (BFA) was proposed, which exploits memory fault inject techniques such as row hammer to attack quantized models in the deployment stage. With only a few bit flips, the target model can be rendered useless as a random guesser or even be implanted with malicious functionalities. In this work, we seek to further reduce the number of bit flips. We propose a training-assisted bit flip attack, in which the adversary is involved in the training stage to build a high-risk model to release. This high-risk model, obtained coupled with a corresponding malicious model, behaves normally and can escape various detection methods. The results on benchmark datasets show that an adversary can easily convert this high-risk but normal model to a malicious one on victim’s side by flipping only one critical bit on average in the deployment stage. Moreover, our attack still poses a significant threat even when defenses are employed. The codes for reproducing main experiments are available at https://github.com/jianshuod/TBA.

Federated Learning (FL) has recently emerged as a promising distributed machine learning framework to preserve clients' privacy, by allowing multiple clients to upload the gradients calculated from their local data to a central server. Recent studies find that the exchanged gradients also take the risk of privacy leakage, e.g., an attacker can invert the shared gradients and recover sensitive data against an FL system by leveraging pre-trained generative adversarial networks (GAN) as prior knowledge. However, performing gradient inversion attacks in the latent space of the GAN model limits their expression ability and generalizability. To tackle these challenges, we propose Gradient Inversion over Feature Domains (GIFD), which disassembles the GAN model and searches the feature domains of the intermediate layers. Instead of optimizing only over the initial latent code, we progressively change the optimized layer, from the initial latent space to intermediate layers closer to the output images. In addition, we design a regularizer to avoid unreal image generation by adding a small l1ball constraint to the searching range. We also extend GIFD to the out-of-distribution (OOD) setting, which weakens the assumption that the training sets of GANs and FL tasks obey the same data distribution. Extensive experiments demonstrate that our method can achieve pixel-level reconstruction and is superior to the existing methods. Notably, GIFD also shows great generalizability under different defense strategy settings and batch sizes.

Deep neural networks are valuable assets considering their commercial benefits and huge demands for costly annotation and computation resources. To protect the copyright of DNNs, backdoor-based ownership verification becomes popular recently, in which the model owner can watermark the model by embedding a specific backdoor behavior before releasing it. The defenders (usually the model owners) can identify whether a suspicious third-party model is "stolen" from them based on the presence of the behavior. Unfortunately, these watermarks are proven to be vulnerable to removal attacks even like fine-tuning. To further explore this vulnerability, we investigate the parameter space and find there exist many watermark-removed models in the vicinity of the watermarked one, which may be easily used by removal attacks. Inspired by this finding, we propose a mini-max formulation to find these watermark-removed models and recover their watermark behavior. Extensive experiments demonstrate that our method improves the robustness of the model watermarking against parametric changes and numerous watermark-removal attacks. The codes for reproducing our main experiments are available at https://github.com/GuanhaoGan/robust-model-watermarking.

Pre-trained point cloud models have found extensive applications in 3D understanding tasks like object classification and part segmentation. However, the prevailing strategy of full fine-tuning in downstream tasks leads to large per-task storage overhead for model parameters, which limits the efficiency when applying large-scale pre-trained models. Inspired by the recent success of visual prompt tuning (VPT), this paper attempts to explore prompt tuning on pre-trained point cloud models, to pursue an elegant balance between performance and parameter efficiency. We find while instance-agnostic static prompting, e.g. VPT, shows some efficacy in downstream transfer, it is vulnerable to the distribution diversity caused by various types of noises in real-world point cloud data. To conquer this limitation, we propose a novel Instance-aware Dynamic Prompt Tuning (IDPT) strategy for pre-trained point cloud models. The essence of IDPT is to develop a dynamic prompt generation module to perceive semantic prior features of each point cloud instance and generate adaptive prompt tokens to enhance the model's robustness. Notably, extensive experiments demonstrate that IDPT outperforms full finetuning in most tasks with a mere 7% of the trainable parameters, providing a promising solution to parameter-efficient learning for pre-trained point cloud models. Code is available at https://github.com/zyh16143998882/ICCV23-IDPT.

Unsupervised surface anomaly detection aims at discovering and localizing anomalous patterns using only anomaly-free training samples. Reconstruction-based models are among the most popular and successful methods, which rely on the assumption that anomaly regions are more difficult to reconstruct. However, there are three major challenges to the practical application of this approach: 1) the reconstruction quality needs to be further improved since it has a great impact on the final result, especially for images with structural changes; 2) it is observed that for many neural networks, the anomalies can also be well reconstructed, which severely violates the underlying assumption; 3) since reconstruction is an ill-conditioned problem, a test instance may correspond to multiple normal patterns, but most current reconstruction-based methods have ignored this critical fact. In this paper, we propose DiffAD, a method for unsupervised anomaly detection based on the latent diffusion model, inspired by its ability to generate high-quality and diverse images. We further propose noisy condition embedding and interpolated channels to address the aforementioned challenges in the general reconstruction-based pipeline. Extensive experiments show that our method achieves state-of-the-art performance on the challenging MVTec dataset, especially in localization accuracy.

Recent advances have shown the potential for post-training quantization (PTQ) to reduce excessive hardware resources and quantize deep models to low bits in a short time, compared with Quantization-Aware Training (QAT). However, existing PTQ approaches lose a lot of accuracies when quantizing the model to extremely low bits, e.g., 1 bit. In this work, we propose layer-by-layer self-knowledge distillation binary post-training quantization (LKBQ), the first method capable of quantizing the weights of neural networks to 1 bit in PTQ domain. We show that careful use of layer-by-layer self-distillation within the LKBQ can provide a significant performance boost. Furthermore, our evaluation results show that the initialization of quantized network weights can have a huge impact on the results. Then we propose three methods for weight initialization. Finally, in light of the characteristics of the binarized network, we propose a method named gradient scaling to further improve efficiency. Our experiments show that LKBQ pushes the limit of PTQ to extreme 1-bit for the first time.

Unsupervised anomaly detection (UAD) has been widely implemented in industrial and medical applications, which reduces the cost of manual annotation and improves efficiency in disease diagnosis. Recently, deep auto-encoder with its variants has demonstrated its advantages in many UAD scenarios. Training on the normal data, these models are expected to locate anomalies by producing higher reconstruction error for the abnormal areas than the normal ones. However, this assumption does not always hold because of the uncontrollable generalization capability. To solve this problem, we present LSGS, a method that builds on Vector Quantised-Variational Autoencoder (VQVAE) with a novel aggregated codebook and transformers with global attention. In this work, the VQVAE focus on feature extraction and reconstruction of images, and the transformers fit the manifold and locate anomalies in the latent space. Then, leveraging the generated encoding sequences that conform to a normal distribution, we can reconstruct a more accurate image for locating the anomalies. Experiments on various datasets demonstrate the effectiveness of the proposed method.

Scene text image super-resolution (STISR), aiming to improve image quality while boosting downstream scene text recognition accuracy, has recently achieved great success. However, most existing methods treat the foreground (character regions) and background (non-character regions) equally in the forward process, and neglect the disturbance from the complex background, thus limiting the performance. To address these issues, in this paper, we propose a novel method LEMMA that explicitly models character regions to produce high-level text-specific guidance for super-resolution. To model the location of characters effectively, we propose the location enhancement module to extract character region features based on the attention map sequence. Besides, we propose the multi-modal alignment module to perform bidirectional visual-semantic alignment to generate high-quality prior guidance, which is then incorporated into the super-resolution branch in an adaptive manner using the proposed adaptive fusion module. Experiments on TextZoom and four scene text recognition benchmarks demonstrate the superiority of our method over other state-of-the-art methods. Code is available at https://github.com/csguoh/LEMMA.

Deep neural networks (DNNs) have been recently widely used in image super-resolution (SR) and have achieved remarkable performance. However, most existing methods focus on elaborate network design, while rarely considering the training strategy, which affects the model performance and training efficiency. In practice, most SR methods still train the networks with the commonly-used data augmentation (e.g., random crop and sampling), which is shown to converge slowly for deep SR networks. To address this issue, in this paper, we propose a dynamic difficulty-aware data augmenter, named DDA, by considering the restoration difficulty and distribution of input patches. Our DDA mainly consists of difficulty-aware divider, dynamic sampler, and adaptive re-weighter. Specifically, our DDA first uses the difficulty-aware divider to divide the input image into small over-lapping patches, followed by classification into$N$different classes based on the restoration difficulty. Next, dynamic sampler samples the training patches from each class with probability based on training loss. Furthermore, to remedy the imbalance of training patches between different classes, adaptive re-weighter updates the weight of each training patch according to the accumulated training loss. Extensive experiments demonstrate the effectiveness of our DDA on different SR methods by improving training efficiency and model performance across a wide range of scenarios.

When conducting anomaly detection on server monitoring data, it is important to consider the heterogeneity of the data, which is characterized by the diverse and irregular nature of events. The event values can vary widely, encompassing both continuous and discrete values, and there may be a multitude of randomly occurring events. However, many commonly used anomaly detection methods tend to overlook or discard this heterogeneous data, resulting in a significant loss of valuable information. As such, we propose a novel method, called Heterogeneous Time Series Anomaly Detection (HTSAD), to overcome this difficulty. The approach introduces event gates in the Long Short-Term Memory (LSTM) model while using unsupervised learning to overcome the challenges mentioned above. The results of our experiments on real-world datasets show that HTSAD could achieve an f-score of 0.958, which demonstrates the effectiveness of our approach in detecting anomalies in heterogeneous time series data.

In this short paper, we will provide a new explicit construction of binary MDS array codes with triple parities from their parity-check matrices, which contains array codes with array number 8 (8 bits=1 byte). In addition, to demonstrate the applicability of our MDS array codes, we present an effective decoding method aimed at the erased errors. Furthermore, a fast encoding algorithm of our extended MDS array codes is also explored, whose computational complexity is 2 XORs per bit when their code lengths approach infinity.

Recognizing characters from low-resolution (LR) text images poses a significant challenge due to the information deficiency as well as the noise and blur in low-quality images. Current solutions for low-resolution text recognition (LTR) typically rely on a two-stage pipeline that involves super-resolution as the first stage followed by the second-stage recognition. Although this pipeline is straightforward and intuitive, it has to use an additional super-resolution network, which causes inefficiencies during training and testing. Moreover, the recognition accuracy of the second stage heavily depends on the reconstruction quality of the first stage, causing ineffectiveness.In this work, we attempt to address these challenges from a novel perspective: adapting the recognizer to low-resolution inputs by transferring the knowledge from the high-resolution. Guided by this idea, we propose an efficient and effective knowledge distillation framework to achieve multi-level knowledge transfer.Specifically, the visual focus loss is proposed to extract the character position knowledge with resolution gap reduction and character region focus, the semantic contrastive loss is employed to exploit the contextual semantic knowledge with contrastive learning, and the soft logits loss facilitates both local word-level and global sequence-level learning from the soft teacher label.Extensive experiments show that the proposed one-stage pipeline significantly outperforms super-resolution based two-stage frameworks in terms of effectiveness and efficiency, accompanied by favorable robustness.Code is available at https://github.com/csguoh/KD-LTR.

The goal of sequential recommendation (SR) is to predict a user's potential interested items based on her/his historical interaction sequences. Most existing sequential recommenders are developed based on ID features, which, despite their widespread use, often underperform with sparse IDs and struggle with the cold-start problem. Besides, inconsistent ID mappings hinder the model's transferability, isolating similar recommendation domains that could have been co-optimized. This paper aims to address these issues by exploring the potential of multi-modal information in learning robust and generalizable sequence representations. We propose MISSRec, a multi-modal pre-training and transfer learning framework for SR. On the user side, we design a Transformer-based encoder-decoder model, where the contextual encoder learns to capture the sequence-level multi-modal synergy while a novel interest-aware decoder is developed to grasp item-modality-interest relations for better sequence representation. On the candidate item side, we adopt a dynamic fusion module to produce user-adaptive item representation, providing more precise matching between users and items. We pre-train the model with contrastive learning objectives and fine-tune it in an efficient manner. Extensive experiments demonstrate the effectiveness and flexibility of MISSRec, promising an practical solution for real-world recommendation scenarios.

The prosperity of deep neural networks (DNNs) is largely benefited from open-source datasets, based on which users can evaluate and improve their methods. In this paper, we revisit backdoor-based dataset ownership verification (DOV), which is currently the only feasible approach to protect the copyright of open-source datasets. We reveal that these methods are fundamentally harmful given that they could introduce malicious misclassification behaviors to watermarked DNNs by the adversaries. In this paper, we design DOV from another perspective by making watermarked models (trained on the protected dataset) correctly classify some `hard' samples that will be misclassified by the benign model. Our method is inspired by the generalization property of DNNs, where we find a \emph{hardly-generalized domain} for the original dataset (as its \emph{domain watermark}). It can be easily learned with the protected dataset containing modified samples. Specifically, we formulate the domain generation as a bi-level optimization and propose to optimize a set of visually-indistinguishable clean-label modified data with similar effects to domain-watermarked samples from the hardly-generalized domain to ensure watermark stealthiness. We also design a hypothesis-test-guided ownership verification via our domain watermark and provide the theoretical analyses of our method. Extensive experiments on three benchmark datasets are conducted, which verify the effectiveness of our method and its resistance to potential adaptive methods.

Many attack paradigms against deep neural networks have been well studied, such as the backdoor attack in the training stage and the adversarial attack in the inference stage. In this article, we study a novel attack paradigm, the bit-flip based weight attack, which directly modifies weight bits of the attacked model in the deployment stage. To meet various attack scenarios, we propose a general formulation including terms to achieve effectiveness and stealthiness goals and a constraint on the number of bit-flips. Furthermore, benefitting from this extensible and flexible formulation, we present two cases with different malicious purposes, i.e., single sample attack (SSA) and triggered samples attack (TSA). SSA which aims at misclassifying a specific sample into a target class is a binary optimization with determining the state of the binary bits (0 or 1); TSA which is to misclassify the samples embedded with a specific trigger is a mixed integer programming (MIP) with flipped bits and a learnable trigger. Utilizing the latest technique in integer programming, we equivalently reformulate them as continuous optimization problems, whose approximate solutions can be effectively and efficiently obtained by the alternating direction method of multipliers (ADMM) method. Extensive experiments demonstrate the superiority of our methods.

Deep product quantization networks (DPQNs) have been successfully used in image retrieval tasks, due to their powerful feature extraction ability and high efficiency of encoding high-dimensional visual features. Recent studies show that deep neural networks (DNNs) are vulnerable to input with small and maliciously designed perturbations (a.k.a., adversarial examples) for classification. However, little effort has been devoted to investigating how adversarial examples affect DPQNs, which raises the potential safety hazard when deploying DPQNs in a commercial search engine. To this end, we propose an adversarial example generation framework by generating adversarial query images for DPQN-based retrieval systems. Unlike the adversarial generation for the classic image classification task that heavily relies on ground-truth labels, we alternatively perturb the probability distribution of centroids assignments for a clean query, then we can induce effective non-targeted attacks on DPQNs in white-box and black-box settings. Moreover, we further extend the non-targeted attack to a targeted attack by a novel sample space averaging scheme ([Formula: see text]AS), whose theoretical guarantee is also obtained. Extensive experiments show that our methods can create adversarial examples to successfully mislead the target DPQNs. Besides, we found that our methods both significantly degrade the retrieval performance under a wide variety of experimental settings. The source code is available at https://github.com/Kira0096/PQAG.

Deep models have been widely and successfully used in image manipulation detection, which aims to classify tampered images and localize tampered regions. Most existing methods mainly focus on extracting global features from tampered images, while neglecting the relationships of local features between tampered and authentic regions within a single tampered image. To exploit such spatial relationships, we propose Proposal Contrastive Learning (PCL) for effective image manipulation detection. Our PCL consists of a two-stream architecture by extracting two types of global features from RGB and noise views respectively. To further improve the discriminative power, we exploit the relationships of local features through a proxy proposal contrastive learning task by attracting/repelling proposal-based positive/negative sample pairs. Moreover, we show that our PCL can be easily adapted to unlabeled data in practice, which can reduce manual labeling costs and promote more generalizable features. Extensive experiments among several standard datasets demonstrate that our PCL can be a general module to obtain consistent improvement. The code is available athttps://github.com/Sandy-Zeng/PCL.

Deep learning, especially deep neural networks (DNNs), has been widely and successfully adopted in many critical applications for its high effectiveness and efficiency. The rapid development of DNNs has benefited from the existence of some high-quality datasets (e.g., ImageNet), which allow researchers and developers to easily verify the performance of their methods. Currently, almost all existing released datasets require that they can only be adopted for academic or educational purposes rather than commercial purposes without permission. However, there is still no good way to ensure that. In this paper, we formulate the protection of released datasets as verifying whether they are adopted for training a (suspicious) third-party model, where defenders can only query the model while having no information about its parameters and training details. Based on this formulation, we propose to embed external patterns via backdoor watermarking for the ownership verification to protect them. Our method contains two main parts, including dataset watermarking and dataset verification. Specifically, we exploit poison-only backdoor attacks (e.g., BadNets) for dataset watermarking and design a hypothesis-test-guided method for dataset verification. We also provide some theoretical analyses of our methods. Experiments on multiple benchmark datasets of different tasks are conducted, which verify the effectiveness of our method. The code for reproducing main experiments is available at https://github.com/THUYimingLi/DVBW.

From the perspective of parity-check matrices, we present new constructions of$q$-ary maximum distance separable (MDS) array codes with multiple parities. Applying these constructions, some new types of MDS array codes with array numbers$m-\tau $can be derived, where${\mathrm{ gcd}}(m,q)=1$. Moreover, an explicit construction of binary MDS array codes is also presented. Compared to the existing MDS array codes, one important characteristic of these codes is that their available code lengths are much longer, which is suitable for large-scale storage systems. In some particular cases, the maximum code lengths of these codes and their extension can be up to$2^{m-\tau }$and$2^{m-\tau }+1$(or$2^{m-\tau }+2$), respectively. Moreover, to demonstrate the applicability of our constructed MDS array codes, we present an effective generic decoding method for the erased errors. In particular, when there are no more than three erasures occurring, a scheduled algorithm for the syndrome computation of our explicit construction is further proposed, whose computational complexity is asymptotically optimal. Furthermore, this algorithm can be directly applied to the encoding procedure of their extended form. The simulation shows that our new MDS array codes have better encoding and decoding performances than the corresponding extended RS codes coupled with different algorithms.

Obtaining a well-trained model involves expensive data collection and training procedures, therefore the model is a valuable intellectual property. Recent studies revealed that adversaries can `steal' deployed models even when they have no training samples and can not get access to the model parameters or structures. Currently, there were some defense methods to alleviate this threat, mostly by increasing the cost of model stealing. In this paper, we explore the defense from another angle by verifying whether a suspicious model contains the knowledge of defender-specified external features. Specifically, we embed the external features by tempering a few training samples with style transfer. We then train a meta-classifier to determine whether a model is stolen from the victim. This approach is inspired by the understanding that the stolen models should contain the knowledge of features learned by the victim model. We examine our method on both CIFAR-10 and ImageNet datasets. Experimental results demonstrate that our method is effective in detecting different types of model stealing simultaneously, even if the stolen model is obtained via a multi-stage stealing process. The codes for reproducing main results are available at Github (https://github.com/zlh-thu/StealingVerification).

The high efficiency in computation and storage makes hashing (including binary hashing and quantization) a common strategy in large-scale retrieval systems. To alleviate the reliance on expensive annotations, unsupervised deep hashing becomes an important research problem. This paper provides a novel solution to unsupervised deep quantization, namely Contrastive Quantization with Code Memory (MeCoQ). Different from existing reconstruction-based strategies, we learn unsupervised binary descriptors by contrastive learning, which can better capture discriminative visual semantics. Besides, we uncover that codeword diversity regularization is critical to prevent contrastive learning-based quantization from model degeneration. Moreover, we introduce a novel quantization code memory module that boosts contrastive learning with lower feature drift than conventional feature memories. Extensive experiments on benchmark datasets show that MeCoQ outperforms state-of-the-art methods. Code and configurations are publicly released.

Music recommendation is usually modeled as a Click-Through Rate (CTR) prediction problem, which estimates the probability of a user listening a recommended song. CTR prediction can be formulated as a binary classification problem where the played songs are labeled as positive samples and the skipped songs are labeled as negative samples. However, such naively defined labels are noisy and biased in practice, causing inaccurate model predictions. In this work, we first identify serious label noise issues in an industrial music App, and then propose an adaptive Label Correction method for Denoising (LCD) music recommendation by ensembling the noisy labels and the model outputs to encourage a consensus prediction. Extensive offline experiments are conducted to evaluate the effectiveness of LCD on both industrial and public datasets. Furthermore, in a one-week online AB test, LCD also significantly increases both the music play count and time per user by 1% to 5%.

Generative model based image lossless compression algorithms have seen a great success in improving compression ratio. However, the throughput for most of them is less than 1 MB/s even with the most advanced AI accelerated chips, preventing them from most real-world applications, which often require 100 MB/s. In this paper, we propose PILC, an end-to-end image lossless compression framework that achieves 200 MB/s for both compression and decom-pression with a single NVIDIA Tesla V100 GPU, 10× faster than the most efficient one before. To obtain this result, we first develop an AI codec that combines auto-regressive model and VQ-VAE which performs well in lightweight setting, then we design a low complexity entropy coder that works well with our codec. Experiments show that our framework compresses better than PNG by a margin of 30% in multiple datasets. We believe this is an important step to bring AI compression forward to commercial use.

This work studies black-box adversarial attacks against deep neural networks (DNNs), where the attacker can only access the query feedback returned by the attacked DNN model, while other information such as model parameters or the training datasets are unknown. One promising approach to improve attack performance is utilizing the adversarial transferability between some white-box surrogate models and the target model (i.e., the attacked model). However, due to the possible differences on model architectures and training datasets between surrogate and target models, dubbed “surrogate biases”, the contribution of adversarial transferability to improving the attack performance may be weakened. To tackle this issue, we innovatively propose a black-box attack method by developing a novel mechanism of adversarial transferability, which is robust to the surrogate biases. The general idea is transferring partial parameters of the conditional adversarial distribution (CAD) of surrogate models, while learning the untransferred parameters based on queries to the target model, to keep the flexibility to adjust the CAD of the target model on any new benign sample. Extensive experiments on benchmark datasets and attacking against real-world API demonstrate the superior attack performance of the proposed method. The code will be available at https://github.com/Kira0096/CGATTACK.

The widespread application of Internet-of-Things (IoT) and deep learning have made machine-to-machine semantic communication possible. However, it remains challenging to deploy DNN model on IoT devices, due to their limited computing and storage capacity. In this paper, we propose Compressed Sensing based Asymmetric Semantic Image Compression (CS-ASIC) for resource-constrained IoT systems, which consists of a lightweight front encoder and a deep iterative decoder offloaded at the server. We further consider a task-oriented scenario and optimize CS-ASIC for the semantic recognition tasks. The experiment results demonstrate that CS-ASIC achieves considerable data-semantic rate-distortion trade-off, and low encoding complexity over prevailing codecs.

Unsupervised domain adaptation (UDA) aims to transfer knowledge from a label-rich source domain to a different yet related fully-unlabeled target domain. Existing approaches utilize self-training scheme to learn discriminative target features and thus enforce class-level distribution alignment implicitly across the source and target domains. However, inherent noise of the pseudo labels due to domain shift could compromise the training process to negatively affect the adapted model performance. In this paper, we propose Self-Ensemble Variance Regularization for Domain Adaptaton (VRDA) method to rectify the learning with pseudo labels. To be specific, we regard the prediction distinction between the student and its self-ensemble teacher model as prediction variance, to regularize target domain prediction bias from pseudo labels. The experimental results reveal that the proposed VRDA achieves the state-of-the-art performance on several standard UDA datasets.

HyperGraph Convolutional Neural Networks (HGCNNs) have demonstrated their potential in modeling high-order relations preserved in graph-structured data. However, most existing convolution filters are localized and determined by the pre-defined initial hypergraph topology, neglecting to explore implicit and long-range relations in real-world data. In this paper, we propose the first learning-based method tailored for constructing adaptive hypergraph structure, termed HypERgrAph Laplacian aDaptor (HERALD), which serves as a generic plug-and-play module for improving the representational power of HGCNNs. Specifically, HERALD adaptively optimizes the adjacency relationship between vertices and hyperedges in an end-to-end manner and thus the task-aware hypergraph is learned. Furthermore, HERALD employs the self-attention mechanism to capture the non-local paired-nodes relation. Extensive experiments on various popular hypergraph datasets for node classification and graph classification tasks demonstrate that our approach obtains consistent and considerable performance enhancement, proving its effectiveness and generalization ability.

In class incremental learning (CIL), models are expected to be able to learn new categories continuously. However, the standard DNNs suffer from catastrophic forgetting. Recent studies show class imbalance is an essential factor that causes catastrophic forgetting in CIL. In this paper, from the perspective of energy-based model, we demonstrate that the free energies of categories are aligned with the label distribution theoretically, thus the energies of different classes are expected to be close to each other when aiming for "balanced" performance. However, we discover a severe energy-bias phenomenon in the models trained in CIL. To eliminate the bias, we propose a simple and effective method named Energy Alignment by merely adding the calculated shift scalars onto the output logits, which does not require to (i) modify the network architectures, (ii) intervene the standard learning paradigm. Experimental results show that energy alignment can achieve good performance on several CIL benchmarks.

Image representation is critical for many visual tasks. Instead of representing images discretely with 2D arrays of pixels, a recent study, namely local implicit image function (LIIF), denotes images as a continuous function where pixel values are expansion by using the corresponding coordinates as inputs. Due to its continuous nature, LIIF can be adopted for arbitrary-scale image super-resolution tasks, resulting in a single effective and efficient model for various up-scaling factors. However, LIIF often suffers from structural distortions and ringing artifacts around edges, mostly because all pixels share the same model, thus ignoring the local properties of the image. In this paper, we propose a novel adaptive local image function (A-LIIF) to alleviate this problem. Specifically, our A-LIIF consists of two main components: an encoder and a expansion network. The former captures cross-scale image features, while the latter models the continuous up-scaling function by a weighted combination of multiple local implicit image functions. Accordingly, our A-LIIF can reconstruct the high-frequency textures and structures more accurately. Experiments on multiple benchmark datasets verify the effectiveness of our method. Our codes are available at https://github.com/LeeHW-THU/A-LIIF.

Mass image transmission has undergone an explosion of growth with the development of the internet, DCT-based lossy image compression like JPEG is pervasively conducted to save the transmission bandwidth. Recently, DCT-domain coefficient estimation approaches have been proposed to further improve the compression ratio by discarding DC coefficients at the sender’s end while recovering them at the receiver’s end via DC estimation. However, known DC estimation needs to enumerate all possible DC coefficients. Consequently, they are limited and resource-consuming due to the low delay requirements in real-time transmission. In this paper, we propose an improved DC estimation method via convex relaxation, which achieves state-of-the-art performance in terms of both recovery image quality and time complexity. Extensive experiments across various data sets demonstrate the advantages of our method.

In recent years, Graph Neural Networks (GNNs) have emerged as powerful techniques for graph-structure data, which are essential for a wide range of graph-based tasks like link prediction and node classification. However, over-smoothing and over-fitting are two main challenges that impact negatively on model performance. Data augmentation is a good solution to these two problems, and it is also proven very effective in computer vision and nature language processing. But there is a relatively small body of literature when it comes to graph data augmentation. In this paper, we propose a Graph Data Augmentation (GDA) strategy to optimize the graph topology for node classification tasks. Our GDA approach consists of two operations: edge manipulation based on similarities of node pairs (GDA-E) and new nodes addition to under-informed old nodes (GDA-N). GDA-E is designed to add missing edges and remove noisy edges, while GDA-N is established to help nodes with low degree. Both operations can improve the information-to-noise ratio of the whole graph and lead to better performance of GNNs. The comparative results of experiments on three different datasets show that our GDA approach achieves considerable improvement (11.0% average) over origin graphs, and the ablation study verifies the effectiveness of both GDA-E and GDA-N.

Most existing weakly supervised aspect detection algorithms utilize pre-trained language models as their backbone networks by constructing discriminative tasks with seed words. Once the number of seed words decreases, the performance of current models declines significantly. Recently, prompt tuning has been proposed to bridge the gap of objective forms in pre-training and fine-tuning, which is hopeful of alleviating the above challenge. However, directly applying the existing prompt-based methods to this task not only fails to effectively use large amounts of unlabeled data, but also may cause serious over-fitting problems. In this paper, we propose a lightweight teacher-student network (PTS) based on prompts to solve the above two problems. Concretely, the student network is a hybrid prompt-based classification model to detect aspects, which innovatively compounds hand-crafted prompts and auto-generated prompts. The teacher network comprehensively considers the representation of the sentence and the masked aspect token in the template to guide classification. To utilize unlabeled data and seed words intelligently, we train the teacher and student network alternately. Furthermore, in order to solve the problem that the uneven quality of training data obviously affects the iterative efficiency of PTS, we design a general dynamic data selection strategy to feed the most pertinent data into the current model. Experimental results show that even given the minimum seed words, PTS significantly outperforms previous state-of-the-art methods on three widely used benchmarks.

Optimizing portfolios is an important concern for all investors. Nowadays, deep learning has been applied to the study of portfolio investment. Still, the widely used deep learning methods based on asset return prediction do not guarantee to maximize the performance of a portfolio. In this paper, we design a neural network with the overall return-risk ratio of the portfolio as the optimization objective to determine the optimal allocation weights of the portfolio. We design the network architecture based on the Conv-Transformer with graph attention mechanisms(CTG) to better model the temporal dependence of assets and the correlation relationship between assets. The empirical results on the Chinese stock market show that our approach achieves the best results compared to the current SOTA model.

Unsupervised aspect detection (UAD) aims to identify the aspect categories mentioned in product reviews automatically. Existing unsupervised methods mainly focus on single-label prediction which cannot work well for the realistic application scene since a review usually contain multiple aspect categories. Recent attempts alleviate this issue by setting a threshold. However, the imbalance of the number of category-related words in review segments makes these methods difficult to find optimal thresholds to recall all categories, which is a common but neglected phenomenon. In this paper, we propose a novel unsupervised method termed Aspect-Category Experts (ACEs) to address this problem. Our goal is to train a set of aspect-category experts to encode the sentence in parallel, where experts and aspect categories correspond one-to-one. Experts in different aspects weight the embedding of representative words with aspect-specific attention to avoid the negative impact of accumulation. Besides, to enhance the complementarity between different experts to reduce inter-class feature entanglement, we construct a novel mutual exclusion loss (ME loss) to improve the aspect detection performance. Extensive experimental results on four datasets demonstrate that our proposed ACEs model outperforms the previous state-of-the-art methods.

In this paper, we firstly present a new construction of binary maximum distance separable (MDS) array codes, from which some types of new MDS array codes of minimum distance 4 with array dimension (p−1)×(ℓ+2) can be deduced. Based on the construction, binary locally repairable array codes (LRACs) of minimum distance 4 are also explored, whose array dimension is (p−1)×2ℓ and column locality is ℓ − 1. Particularly, when 2 is a primitive root module p, a scheduled algorithm for syndrome computation of the LRACs is proposed, which converges to 2 XORs per data bit when ℓ approaches infinity.

Characterizing the end-to-end network available bandwidth (ABW) is an important but challenging task. Although a number of ABW estimation tools have been introduced over the past two decades, applying them to the real-world networks is still difficult because of the biased results, heavy load, and long measurement time. In this paper, we propose a novel Burst Queue Recovery (BQR) model to infer the ABW. BQR first induces an instant network congestion and then observes the one-way delay (OWD) variation until the tight link recovers from the congestion. By correlating the OWDs with the queue length variation, BQR can calculate the ABW accurately. Compared to the traditional probe gap model (PGM) and probe rate model (PRM), our theoretical analysis and simulations show that BQR is more tolerant to the transient traffic burst and supports the scenarios with multiple congestible links. Based on the model, we build FABMon, a fast and accurate ABW estimation tool. Our experiments show that FABMon can measure ABW within 50 milliseconds, and achieve much more accurate measurement results than the existing tools with a very small volume of probe packets.

Deep neural networks (DNNs) have demonstrated their superiority in practice. Arguably, the rapid development of DNNs is largely benefited from high-quality (open-sourced) datasets, based on which researchers and developers can easily evaluate and improve their learning methods. Since the data collection is usually time-consuming or even expensive, how to protect their copyrights is of great significance and worth further exploration. In this paper, we revisit dataset ownership verification. We find that existing verification methods introduced new security risks in DNNs trained on the protected dataset, due to the targeted nature of poison-only backdoor watermarks. To alleviate this problem, in this work, we explore the untargeted backdoor watermarking scheme, where the abnormal model behaviors are not deterministic. Specifically, we introduce two dispersibilities and prove their correlation, based on which we design the untargeted backdoor watermark under both poisoned-label and clean-label settings. We also discuss how to use the proposed untargeted backdoor watermark for dataset ownership verification. Experiments on benchmark datasets verify the effectiveness of our methods and their resistance to existing backdoor defenses.

In this paper we propose the deep Dirichlet process mixture (DDPM) model, which is an unsupervised method that simultaneously performs clustering and feature learning. The traditional Dirichlet process mixture model can infer the number of mixture components, but its flexibility is restricted since the clustering is performed in the raw feature space. Our method alleviates this limitation by using the flow-based deep neural network to learn more expressive features. DDPM unifies Dirichlet processes and the flow-based model with Monte Carlo expectation-maximization, and uses Gibbs sampling to sample from the posterior. This combination allows our method to exploit the mutually beneficial relation between clustering and feature learning. The effectiveness of DDPM is demonstrated by thorough experiments in various synthetic and real-world datasets.

Although federated learning improves privacy of training data by exchanging local gradients or parameters rather than raw data, the adversary still can leverage local gradients and parameters to obtain local training data by launching reconstruction and membership inference attacks. To defend against such privacy attacks, many noises perturbed methods (like differential privacy or CountSketch matrix) have been widely designed. However, the strong defence ability and high learning accuracy of these schemes cannot be ensured at the same time, which will impede the wide application of FL in practice (especially for medical or financial institutions that require both high accuracy and strong privacy guarantee). To overcome this issue, we propose an efficient model perturbation method for federated learning to defend against reconstruction and membership inference attacks launched by curious clients. On the one hand, similar to the differential privacy, our method also selects random numbers as perturbed noises added to the global model parameters, and thus it is very efficient and easy to be integrated in practice. Meanwhile, the random selected noises are positive real numbers and the corresponding value can be arbitrarily large, and thus the strong defence ability can be ensured. On the other hand, unlike differential privacy or other perturbation methods that cannot eliminate added noises, our method allows the server to recover the true aggregated gradients by eliminating the added noises. Therefore, our method does not hinder learning accuracy at all. Extensive experiments demonstrate that for both regression and classification tasks, our method achieves the same accuracy as non-private approaches and outperforms the state-of-the-art defence schemes. Besides, the defence ability of our method against reconstruction and membership inference attack is significantly better than the state-of-the-art related defence schemes.

With the recent boom of video-based social platforms (e.g., YouTube and TikTok), video retrieval using sentence queries has become an important demand and attracts increasing research attention. Despite the decent performance, existing text-video retrieval models in vision and language communities are impractical for large-scale Web search because they adopt brute-force search based on high-dimensional embeddings. To improve efficiency, Web search engines widely apply vector compression libraries (e.g., FAISS [26]) to post-process the learned embeddings. Unfortunately, separate compression from feature encoding degrades the robustness of representations and incurs performance decay. To pursue a better balance between performance and efficiency, we propose the first quantized representation learning method for cross-view video retrieval, namely Hybrid Contrastive Quantization (HCQ). Specifically, HCQ learns both coarse-grained and fine-grained quantizations with transformers, which provide complementary understandings for texts and videos and preserve comprehensive semantic information. By performing Asymmetric-Quantized Contrastive Learning (AQ-CL) across views, HCQ aligns texts and videos at coarse-grained and multiple fine-grained levels. This hybrid-grained learning strategy serves as strong supervision on the cross-view video quantization model, where contrastive learning at different levels can be mutually promoted. Extensive experiments on three Web video benchmark datasets demonstrate that HCQ achieves competitive performance with state-of-the-art non-compressed retrieval methods while showing high efficiency in storage and computation. Code and configurations are available at https://github.com/gimpong/WWW22-HCQ.

Federated learning (FL) enables data owners to train a global model with shared gradients while keeping private training data locally. However, recent research demonstrated that the adversary may infer private training data of clients from the exchanged local gradients, e.g., having deep leakage from gradients (DLGs). Many existing privacy-preserving approaches take usage of differential privacy (DP) to guarantee privacy. Nevertheless, the widely used privacy budget of DP (e.g., evenly distribution) leads to a sharp decline of model accuracy. To improve the model accuracy, some schemes only consider allocating the privacy budget to the fully connected layers. However, we reveal that the adversary may still reconstruct the private training data by adopting the DLG attack with the gradients of convolutional layers. In this article, we propose a fine-grained DP federated learning (DPFL) scheme, which guarantees privacy and remains high model performance simultaneously. Specifically, inspired by the methods that measure the importance of layers in deep learning, we propose a fine-grained method to allocate noise according to the importance value of layers in order to remain high model performance. Besides, we combine an active client selection strategy with DPFL and perform fine-tuning with a public data set on the server to further ensure the model performance. We evaluate DPFL under both independent and identically distributed (i.i.d) and non-i.i.d data settings to show that our method can achieve similar accuracy as the plain FL (e.g., FedAvg). We also demonstrate that our DPFL can resist the DLG attack to verify its privacy guarantee.

Locally repairable codes (LRCs) are important for distributed storage systems due to their efficient repairing ability of the failed storage nodes. A$q$-ary optimal$(n,k,r)$-LRC is an$[n,k,d]$linear code over$\mathbb {F}_{q}$such that every code symbol has locality$r$, and the minimum distance attains the well-known Singleton-like bound. In this paper, we study the maximal code length, code constructions and weight distributions of$q$-ary optimal LRCs with locality 2 and distance 5, which are of both practical and theoretical interest. Firstly, it is proved that when the code dimension is even or odd, corresponding maximal code lengths of such$q$-ary optimal LRCs are$3 \cdot \lfloor \frac {q+1}{3} \rfloor $and$3 \cdot \left \lfloor{ \frac {q-1}{3} }\right \rfloor +5$, respectively. Up to the equivalence of linear codes, we propose constructions of all the possible$q$-ary optimal LRCs with locality 2, distance 5 and maximal code length. Then, by characterizing the weight type hierarchy of codewords, we show that the weight distribution of any$q$-ary optimal LRC with locality 2, distance 5 and even code dimension can be uniquely determined and explicit expression of the weight distribution is given. Moreover, it is shown that all$q$-ary optimal LRCs with locality 2, distance 5 and even code dimension are maximally recoverable.

Network traffic classification is important to intrusion detection and network management. Most of existing methods are based on machine learning techniques and rely on the features extracted manually from flows or packets. However, with the rapid growth of network applications, it is difficult for these approaches to handle new complex applications. In this article, we design a novel neural network, the Extended Byte Segment Neural Network (EBSNN), to classify netwrk traffic. EBSNN first divides a packet into header segments and payload segments, which are then fed into encoders composed of the recurrent neural networks with the attention mechanism. Based on the outputs, another encoder learns the high-level representation of the whole packet. In particular, side-channel features are learned from header segments to improve the performance. Finally, the label of the packet is obtained by the softmax function. Furthermore, EBSNN can classify network flows by examining the first few packets. Thorough experiments on the real-world datasets show that EBSNN achieves better performance than the state-of-the-art methods in both the application identification task and the website identification task.

Recently, a variety of regularization techniques have been widely applied in deep neural networks, which mainly focus on the regularization of weight parameters to encourage generalization effectively. Label regularization techniques are also proposed with the motivation of softening the labels while neglecting the relation of classes. Among them, the technique of knowledge distillation proposes to distill the soft label, which contains the knowledge of class relations. However, this technique needs to pre-train an extra cumbersome teacher model. In this paper, we propose a method called Knowledge Refinery (KR), which enables the neural network to learn the relation of classes on-the-fly without the teacher-student training strategy. We propose the definition of decoupled labels, which consist of the original hard label and the residual label. To exhibit the generalization of KR, we evaluate our method in both fields of computer vision and natural language processing. Our empirical results show consistent performance gains under all experimental settings.

Deep quantization methods have shown high efficiency on large-scale image retrieval. However, current models heavily rely on ground-truth information, hindering the application of quantization in label-hungry scenarios. A more realistic demand is to learn from inexhaustible uploaded images that are associated with informal tags provided by amateur users. Though such sketchy tags do not obviously reveal the labels, they actually contain useful semantic information for supervising deep quantization. To this end, we propose Weakly-Supervised Deep Hyperspherical Quantization (WSDHQ), which is the first work to learn deep quantization from weakly tagged images. Specifically, 1) we use word embeddings to represent the tags and enhance their semantic information based on a tag correlation graph. 2) To better preserve semantic information in quantization codes and reduce quantization error, we jointly learn semantics-preserving embeddings and supervised quantizer on hypersphere by employing a well-designed fusion layer and tailor-made loss functions. Extensive experiments show that WSDHQ can achieve state-of-art performance in weakly-supervised compact coding.

Federated Learning (FL) refers to the machine learning scheme that enables decentralized model training over massive separate data sources without privacy concerns. However, existing works rarely consider difficulty of obtaining sufficient data labels due to uncontrollable user behavior, especially in cross-device FL scenarios. In this paper, we consider semi-supervised federated learning (SSFL) setups and mainly focus on the disjoint scenario where local clients only have access to unlabeled data. By integrating self-training scheme for unlabeled data, we propose self-training loss as part of local training objective within federated learning framework. To further stablize and improve the learning process, we propose global distillation loss that utilize output logits of global model for per client-sample as supervision and also soften such distillation by temperature to obtain more discriminative information. Based on self-training and global distillation loss, combined with server-side training, we propose Global Distillation Self-Training (GDST) Federated Learning algorithm, which enables to distributedly learn a global model in the disjoint scenario of SSFL. Finally, we do sufficient ablation study to explore the role of each component of our GDST method, experimentally guarantee the interpretability.

Deep neural networks (DNNs) have been widely used in facial manipulation. Existing methods focus on training deeper networks in indirect supervision ways (e.g., feature constraint), or in unsupervised ways (e.g., cycle-consistency loss) due to the lack of ground-truth face images for manipulated outputs. However, such methods can not synthesize realistic face images well and suffer from very high training overhead. To address this issue, we propose a novel Feature Disentanglement and Reintegraion network (FDRNet), which employs ground-truth images as informative supervision and dynamically adapts the fusion of informative features of the ground-truth images effectively and efficiently. FDRNet consists of a Feature Disentanglement (FD) Network and a Feature Reintegration (FR) Network, which encodes informative disentangled representations from the ground-truth images and fuses the disentangled representations to reconstruct the face images. By learning disentangled representations, our method can generate plausible faces conditioned on both landmarks and identities, which can be used for a variety of face manipulation tasks. Experiments on the CelebA-HQ and FFHQ datasets are conducted to demonstrate the superiority of our method over state-of-the-art methods in terms of effectiveness and efficiency.

Learning Directed Acyclic Graph (DAG) from purely observational data is a critical problem for causal inference. Most existing works tackle this problem by exploring gradient-based learning methods with a smooth characterization of acyclicity. A major shortcoming of current gradient based works is that they independently optimize SEMs with a single sample and neglect the interactions between different samples. In this paper, we consider DAG structure learning from the perspective of distributional optimization and design an adversarial framework named DAG-GAN to detect the DAG structure from data. We theoretically analyze the Nash equilibrium property of DAG-GAN and propose a novel score function to exploit the interactions between different samples. In addition, extensive experiments are conducted to validate the efficiency of DAG-GAN against several state-of-the-art DAG learning methods.

With the massive volume of data emerging from both scientific and industrial domains, it has become a desideratum to improve the scalability of Gaussian process regression (GPR). There are two major approaches to assuage its $\mathcal{O}\left( {{n^3}} \right)$ training complexity: the aggregation based methods and the sparse approximation methods. This paper proposes a hybrid strategy called H-GPR to combine these two well-established approaches. We show that it is possible to improve the performance of aggregation based methods by removing some data points that severely violate its underlying assumption, and then this information loss can be recovered by a set of inducing points generated by the sparse approximation methods. A novel metric called conditional independent score is proposed, which measures to what extent the assumption made by the aggregation based methods is satisfied. A heuristic rule is developed to adjust the relative size of the local experts and the inducing subset, so that their distinction can be better reflected. Thorough experiments on synthetic and realistic datasets were performed, demonstrating that the proposed method can improve both the predictive means and variances.

Privacy protection is an important research area, which is especially critical in this big data era. To a large extent, the privacy of visual classification data is mainly in the mapping between the image and its corresponding label, since this relation provides a great amount of information and can be used in other scenarios. In this paper, we propose the mapping distortion based protection (MDP) and its augmentation-based extension (AugMDP) to protect the data privacy by modifying the original dataset. In the modified dataset generated by MDP, the image and its label are not consistent (e.g., a cat-like image is labeled as the dog), whereas the DNNs trained on it can still achieve good performance on benign testing set. As such, this method can protect privacy when the dataset is leaked. Extensive experiments are conducted, which verify the effectiveness and feasibility of our method. The code for reproducing main results is available at https://github.com/PerdonLiu/Visual-Privacy-Protection-via-Mapping-Distortion.

k-means algorithm is one of the most classical clustering methods, which has been widely and successfully used in signal processing. However, due to the thin-tailed property of the Gaussian distribution, k-means algorithm suffers from relatively poor performance on the dataset containing heavy-tailed data or outliers. Besides, standard k-means algorithm also has relatively weak stability, i.e. its results have a large variance, which reduces its credibility. In this paper, we propose a robust and stable k-means variant, dubbed the t-k-means, as well as its fast version to alleviate those problems. Theoretically, we derive the t-k-means and analyze its robustness and stability from the aspect of the loss function and the expression of the clustering center, respectively. Extensive experiments are also conducted, which verify the effectiveness and efficiency of the proposed method. The code for reproducing main results is available at https://github.com/THUYimingLi/t-k-means.

Dynamic time warping (DTW) is one of the most successful methods that addresses the challenge of measuring the discrepancy between two series, which is robust to shift and distortion along the time axis of the sequence. Based on DTW, we propose a novel loss function for time series data called Gumbel-Softmin based fast DTW (GDTW). To the best of our knowledge, this is the first differentiable DTW loss for series data that scales linearly with the sequence length. The proposed Gumbel-Softmin replaces the simple minimization operator in DTW so as to better integrate the acceleration technology. We also design a deep learning model combining GDTW as a feature extractor. Thorough experiments over a broad range of time series analysis tasks were performed, showing the efficiency and effectiveness of our method.

Convolutional neural networks (CNNs) have obtained great success in single image super-resolution (SR). More recent works (e.g., RCAN and SAN) have obtained remarkable performance with channel attention based on first- or second-order statistics of features. However, these methods neglect the rich feature statistics higher than second-order, thus hindering the representation ability of CNNs. To address this issue, we propose a higher-order channel attention (HOCA) module to enhance the representation ability of CNNs. In our HOCA module, to capture different types of semantic information, we first compute k-order of feature statistics, followed by channel attention to learn the feature interdependencies. Considering the diversity of input contents, we design a gate mechanism to adaptively select a specific k-order channel attention. Besides, our HOCA module serves as a plug-and-play module and can be easily plugged into existing state-of-art CNN-based SR methods. Extensive experiments on public benchmarks show that our HOCA module effectively improves the performance of various CNN-based SR methods.

Learning to hash has been widely applied in large-scale image retrieval. Although current deep hashing methods yield state-of-the-art performance, their heavy dependence on groundtruth information actually makes it difficult to deploy in practical applications such as social media. To solve this problem, we propose a novel method termed Webly Supervised Deep Attentive Quantization (WSDAQ), where deep quantization is trained on web images associated with some userprovided weak tags, without consulting any ground-truth labels. Specifically, we design a tag processing module to leverage semantic information of tags so as to better supervised quantization learning. Besides, we propose an end-to-end trainable Attentive Product Quantization Module (APQM) to quantize deep features of images. Furthermore, we use a noise-contrastive estimation loss to train the model from the perspective of contrastive learning. Experiments validate that WSDAQ is superior to state-of-the-art baselines in compact coding trained on weakly-tagged web images.

Adversarial training (AT) has been one of the most effective ways to defend adversarial attack. However, existing AT variants exhibit a large imbalanced robust accuracies among different classes, which might harm the robustness of some important class(es) in some real-world applications. For instance, diseased cells are much more important than healthy ones in medical image recognition. Given a certain task, the important class is often a priori. To improve robust accuracy of the important class(es), we are the first to propose a novel adversarial training method with class imbalance taken into account. We term it Class-Aware Robust Training (CART). CART can significantly increase the robustness of the important class(es) by an optional weighted combination of original adversarial example generation and that of the important class. Extensive experiments on three benchmark datasets verify the efficacy of CART for enhancing the robust accuracy of important classes while keeping competitive average robust accuracy.

Speaker verification has been widely and successfully adopted in many mission-critical areas for user identification. The training of speaker verification requires a large amount of data, therefore users usually need to adopt third-party data (e.g., data from the Internet or third-party data company). This raises the question of whether adopting untrusted third-party data can pose a security threat. In this paper, we demonstrate that it is possible to inject the hidden backdoor for infecting speaker verification models by poisoning the training data. Specifically, we design a clustering-based attack scheme where poisoned samples from different clusters will contain different triggers (i.e., pre-defined utterances), based on our understanding of verification tasks. The infected models behave normally on benign samples, while attacker-specified unenrolled triggers will successfully pass the verification even if the attacker has no information about the enrolled speaker. We also demonstrate that existing back-door attacks cannot be directly adopted in attacking speaker verification. Our approach not only provides a new perspective for designing novel attacks, but also serves as a strong baseline for improving the robustness of verification methods. The code for reproducing main results is available at https://github.com/zhaitongqing233/Backdoor-attack-against-speaker-verification.

Image inpainting remains a challenging task in computer vision which aims to fill in the missing area of a corrupted image with proper contents and generate photorealistic images by using the information from the existing area. Most existing methods always generate contents with blurry texture caused by propogating the convolutional feature through a fully connected layer. To address this problem, Shift-Net[1] is proposed to shift the encoder feature from existing area to serve as an estimation of the missing parts. However, the decoder feature with new encoding information is ingnored. Inspired by this, we propose a new inpainting model, which is called BiShift-Net. BiShift-Net adopts the structure of U-Net, and we introduce a BiShift layer to it. We use the BiShift layer to capture the information from both encoder and decoder features, rearranging the features to generate sharp texture. Experiments show that Bishift-Net outperforms the other state-of-the-art CNN-based methods, while produce more faithful results at the same time.

Financial signal processing is a matter of great concern in FinTech. Traditionally, recurrent networks are often used to model time series, while the latest research shows that convolutional networks, especially temporal convolutional networks (TCNs), are also powerful and effective for a large number of sequence modeling tasks. The temporal convolutional network uses dilation convolution to expand the receptive field, resulting in very sparse connections in high network layers and no connection to neighbor points. Considered that short-term performance often has a more significant influence on the assets price movement, we suggest that TCNs are too sparse for financial signal processing. For a better solution, we propose a novel Attention on Attention Sparse Dense Convolutional Network (AoA-SDCN), which strengthens time decay characteristics by adding dense connections at close points. Moreover, we use the Attention on Attention mechanism to improve the performance further. Experimental results show that these techniques are effective for financial signal processing. The AoA-SDCN significantly outperforms state-of-the-art methods on Chinese commodity futures and stock datasets.

Human pose estimation deeply relies on visual clues and anatomical constraints between parts to locate keypoints. Most existing CNN-based methods do well in visual representation, however, lacking in the ability to explicitly learn the constraint relationships between keypoints. In this paper, we propose a novel approach based on Token representation for human Pose estimation (TokenPose). In detail, each keypoint is explicitly embedded as a token to simultaneously learn constraint relationships and appearance cues from images. Extensive experiments show that the small and large TokenPose models are on par with state-of-the-art CNN-based counterparts while being more lightweight. Specifically, our TokenPose-S and TokenPose-L achieve 72.5 AP and 75.8 AP on COCO validation dataset respectively, with significant reduction in parameters (↓80.6% ; ↓ 56.8%) and GFLOPs (↓ 75.3%; ↓24.7%). Code is publicly available1.

The Border Gateway Protocol (BGP) is arguably the most important and irreplaceable protocol in the network. However, the lack of routing authentication and validation makes it vulnerable to attacks, including routing leaks, route hijacking, prefix hijacking, etc. Therefore, in this paper we propose a generalized framework for ISP self-operated BGP anomaly detection based on weakly supervised learning. To tackle the problem of insufficient data in BGP anomaly detection, we propose an approach to learn from the other anomaly detection systems through knowledge distillation. To reduce the impact of inaccurate supervision, we design a self-attention-based Long Short-Term Memory (LSTM) model to self-adaptively mine the differences between BGP anomaly categories, including both feature and time dimensions. Finally, we implement a system and demonstrate the performance through a set of comprehensive experiments. Compared with the state-of-the-art schemes, our scheme has better generalization on various anomaly types.

360-degree video streaming, which is becoming more and more popular as the fast development of VR/AR applications nowadays due to the immersive viewing experience it can offer, poses enormous challenges to the current network infrastructure in terms of high bandwidth and low latency requirements. To address this problem and to ensure the QoE (quality of experience) of end-users, this paper presents CUBIST, a method and system for high-quality 360-degree video streaming in networks with cache nodes at the edge. To the best of our knowledge, it is the first tile-based edge caching solution that incorporates proactive tile prefetching and hierarchical cache organization into reactive caching to maximize the caching benefit while reducing the cost of 360-degree video streaming. Experimental results show that CUBIST can achieve a cache hit ratio of 87 % and improve the effective video bitrate by 12.9 % with most rate transitions being small when compared with the latest FoV-aware edge caching scheme.

Knowledge distillation aims at transferring sufficient knowledge from one cumbersome teacher network to another compressed student network. Most previous knowledge distillation methods mainly focus on mimicking the teacher's output of each instance or inter-instance relations from the teacher to the student, while neglecting the attribute structured relations at an instance level. In this paper, we propose a novel Attribute Structured Knowledge Distillation (ASKD) to transfer attribute-level structured relations. It models two types of structured relations, including inter-region structure and inter-class structure, which captures informative knowledge from the teacher model. Inter-region structure captures local feature relations, while interclass structure focuses on cross-class relations. Transferring such attribute-level structure from the teacher to the student would make the student focus on more informative knowledge of the teacher. Extensive experiments on public datasets, including CIFAR-100 and TinyImageNet, demonstrate the superiority of our method over the state-of-the-art methods under similar architecture and different architecture teacher-student pairs.

Rumor detection is a task of identifying information that spread among people whose truth value is false or unverified, and it has been a great challenge due to the rapid development of social media. The traditional machine learning based detection methods can make full use of informative features but cannot extract high-level representations. Other methods involved deep learning neural networks exploit propagation structural information to achieve high accuracy, for example, Bi-Directional Graph Convolution Networks(BiGCN) achieved the best performance on rumor detection by operating on bottom-up and top-down structures. However, those deep learning methods ignore other useful features like content-based features. In this paper, we not only make full use of three aspects of features based on a new concept: kernel subtree, which focus more on informative features of influential nodes of an event, but also propose a new model, which consists of Separation Convolution blocks, Long Short Term Memory(LSTM) and Squeeze and Excitation Networks(SENet), to make comprehensive use of features extracted on the basis of kernel subtree. First, we utilize Separation Convolutions to learn more local information with different kernel size, then LSTM can learn high-level interactions among features and find more global information. After that, SENet applies attention mechanism to put more weights on informative channels of feature maps. Meanwhile, on test set, Gradient Boosting Decision Tree(GBDT) is used to assist our model with few events. The experiments on the PHEME dataset show that our approach can identify rumors with accuracy 95% which outperforms BiGCN by 10% at least.

Although deep neural networks (DNNs) have achieved great success in many areas, they are extremely vulnerable to imperceptible adversarial attacks. DNNs can improve robustness through adversarial training at the expense of massive computing and large network capacity, which makes the model difficult to be lightweight and deployed on edge or resource-constrained devices. To solve this problem, we take both the advantages of structured and unstructured pruning to get a lightweight and sparse model. With the fact that adversarial attacks mainly exist in the high-frequency component of images, we analyze the Batch Normalization (BN) layer to find that the magnitude of its scaling factor can identify those high-frequency components. Therefore, we propose to consider the magnitude of the scaling factor for the pruning criterion. Furthermore, unlike existing methods that prune networks only base on weights, we define “effective weight” which considers both the scaling factor and weight magnitude in unstructured pruning. A novel pruning method is proposed by considering the effective weight. We also verify the Lottery Ticket Hypothesis in adversarial pruning methods yet unknown to the adversarial community.

Convolutional neural networks (CNNs) have recently been widely applied in computer vision tasks, yet they are seriously vulnerable to imperceptible adversarial perturbations. Such phenomena have caused great attention on the adversary topic. Existing adversarial defense methods mainly focus on improving the robustness of models (e.g., adversarial training) or removing adversarial perturbations (e.g., input-transformation based methods) directly, while rarely considering the accurate recovery of image structures of the input, which also play a vital role in making predictions for CNNs. To this end, we propose a Dual-Domain based Defense (D2Defend) method by recovering low-frequency and high-frequency image structures in both spatial and transform domains, while removing adversarial perturbations simultaneously. Unlike the existing input-transformation based methods, our method can decompose the input image into edge feature and texture feature layers, accompanied with bilateral filtering and short-time fourier transform (STFT) filtering. Experimental results demonstrate the effectiveness of our method against various adversarial attacks, and show the superiority of our method over other adversarial defense methods especially at strong adversarial strength.

Deep neural networks (DNNs) have recently achieved wonderful results in single image super-resolution (SISR). These DNN-based SR methods are usually trained on supervised training datasets downscaled with a fixed kernel (e.g., bicubic), and suffer from the generalization problem on real world images due to the complexity of the degradation process. More recent works (e.g., KernelGAN [1]) use a generator network to realize kernel estimation from a single image. However, such methods neglect the anisotropy of kernel, thus limiting the estimation accuracy of kernel in practice. To address this issue, in this paper, we propose a novel encoder-decoder based kernel estimation (EDKE) framework by considering the anisotropy of kernel. In our EDKE framework, the input image is first fed into an encoder network to learn the degradation process, followed by a decoder network to map the encoder image into the original image itself. In this way, the encoder image contains more compact information for the degradation kernel. Besides, our EDKE can be plugged into other blind SR methods. Extensive experiments demonstrate the effectiveness of our method in kernel estimation, and show the superiority of our method over other state-of-the-art methods.

Locally repairable codes (LRCs) have emerged as an important coding scheme in distributed storage systems (DSSs) with relatively low repair cost by accessing fewer non-failure nodes. Theoretical bounds and optimal constructions of LRCs have been widely investigated. Optimal LRCs via cyclic codes provide significant benefit of elegant algebraic structure and efficient encoding procedure. In this paper, we continue to consider the constructions of optimal LRCs via cyclic codes with longer code length. Specifically, we first obtain two classes of Singleton-optimal cyclic LRCs with length$n=3(q+1)$when$3\vert (q-1)$and$q$is even, and length$n=\frac{3}{2}(q+1)$when$3\vert (q-1)$and$q$is odd, respectively. To the best of our knowledge, this is the first construction of q-ary cyclic Singleton-optimal LRCs with length$n > q+1$and minimum distance$d\geq 5$. By using cyclic codes as well, we construct a new family of perfect LRCs with$d=5$, which generalize the result of Goparaju and Calderbank.

A$q$-ary ($n, k, r$) locally repairable code (LRC) is an [$n, k, d$] linear code where every code symbol can be repaired by accessing at most$r$other code symbols. Its minimum distance satisfies the well-known Singleton-like bound. In this paper, we determine all the possible parameters of quaternary LRCs attaining this Singleton-like bound by employing a parity-check matrix approach. Explicit optimal code constructions are given for all the possible parameters.

Network traffic classification (TC) is to classify network traffic into a specific class which plays a fundamental role in terms of network measurement, network management, and so on. In this work, we focus on packet-grained traffic classification. We find that previous packet-grained methods based on the analogy between traffic packet and image or text are not sufficiently reasonable, leading to a sub-optimal performance on both accuracy and efficiency that still can be largely improved. In this paper, we devise a new method, called BLJAN, to jointly learn from byte sequence and labels for packet-grained traffic classification. BLJAN embeds the packet’s bytes and all labels into a joint embedding space to capture their implicit correlations with a dual attention mechanism. It finally builds a more powerful packet representation with an enhancement from label embeddings to achieve high classification accuracy and interpretability. Extensive experiments on two benchmark traffic classification tasks, including application identification and traffic characterization, with three real-world datasets, demonstrate that BLJAN can achieve high performance (96.2%, 96.7%, and 99.7% Macro F1-scores on three datasets) for packet-grained traffic classification, outperforming six representative state-of-the-art baselines in terms of both accuracy and detection speed.

Convolutional neural networks (CNNs) have obtained great success in image restoration tasks, like single image denoising, demosaicing, and super-resolution. However, most existing CNN-based methods neglect the diversity of image contents and degradations in the corrupted images and treat channel-wise features equally, thus hindering the representation ability of CNNs. To address this issue, we propose deep mix-order attention networks (MAN) to extract features that capture rich feature statistics within networks. Our MAN is mainly built on simple residual blocks and our mix-order channel attention (MOCA) module, which further consists of feature gating and feature pooling blocks to capture different types of semantic information. With our MOCA, our MAN can be flexible to handle various types of image contents and degradations. Besides, our MAN can be generalized to different image restoration tasks, like image denoising, super-resolution, and demosaicing. Extensive experiments demonstrate that our method obtains favorably against state-of-the-art methods in terms of quantitative and qualitative metrics.

Compared with tedious per-pixel mask annotating, it is much easier to annotate data by clicks, which costs only several seconds for an image. However, applying clicks to learn video semantic segmentation model has not been explored before. In this work, we propose an effective weakly-supervised video semantic segmentation pipeline with click annotations, called WeClick, for saving laborious annotating effort by segmenting an instance of the semantic class with only a single click. Since detailed semantic information is not captured by clicks, directly training with click labels leads to poor segmentation predictions. To mitigate this problem, we design a novel memory flow knowledge distillation strategy to exploit temporal information (named memory flow) in abundant unlabeled video frames, by distilling the neighboring predictions to the target frame via estimated motion. Moreover, we adopt vanilla knowledge distillation for model compression. In this case, WeClick learns compact video semantic segmentation models with the low-cost click annotations during the training phase yet achieves real-time and accurate models during the inference period. Experimental results on Cityscapes and Camvid show that WeClick outperforms the state-of-the-art methods, increases performance by 10.24% mIoU than baseline, and achieves real-time execution.

Adversarial robustness has received increasing attention along with the study of adversarial examples. So far, existing works show that robust models not only obtain robustness against various adversarial attacks but also boost the performance in some downstream tasks. However, the underlying mechanism of adversarial robustness is still not clear. In this paper, we interpret adversarial robustness from the perspective of linear components, and find that there exist some statistical properties for comprehensively robust models. Specifically, robust models show obvious hierarchical clustering effect on their linearized sub-networks, when removing or replacing all non-linear components (e.g., batch normalization, maximum pooling, or activation layers). Based on these observations, we propose a novel understanding of adversarial robustness and apply it on more tasks including domain adaption and robustness boosting. Experimental evaluations demonstrate the rationality and superiority of our proposed clustering strategy. Our code is available at https://github.com/bymavis/AdvWeightNeurIPS2021.

Real-time deep learning inference serving systems often require prohibitive resources and diverse user requirements. The existing design of inference serving systems mainly focusing on computation resource efficiency, largely ignoring the trade-off between computation and bandwidth resources in need. Sub-optimal resource utilization usually leads to huge serving cost waste. In this paper, we tackle the dual challenge of computation-bandwidth trade-off and cost-effectiveness by proposing A2, an efficient joint Adaptive model, and Adaptive data deep learning serving solution across the geo-datacenters. Inspired by the insight that a trade-off between computational cost and bandwidth cost in achieving the same accuracy, we design a real-time inference serving framework, which selectively places different "versions" of the deep learning models at different geo-locations, and schedules different data sample versions to be sent to those model versions for inference. The goal is to minimize the total serving cost while meeting latency and accuracy demand for the serving requests. We formulate a joint placement and serving problem and propose an efficient approximation algorithm to solve it with a theoretical performance guarantee. We deploy A2on Amazon EC2 for experiments, which shows that A2achieves 30%-50% serving cost reduction under the same required latency and accuracy as compared to baselines.

Repair locality has been an important metric in a distributed storage system (DSS). Erasure codes with small locality are more popular in a DSS, which means fewer available nodes participating in the repair process of failed nodes. Locally repairable codes (LRCs) as a new coding scheme have given more rise to the system performance and attracted a lot of interest in the theoretical research in coding theory. The particular concern among the research problems is the bounds and optimal constructions of LRCs. The problem of optimal constructions of LRCs includes the most important case of Singleton-optimal LRCs whose minimum distance achieves the Singleton-like bound, which is the core consideration in this paper. In this work, we first of all derive an improved and general upper bound on the code length of Singleton-optimal LRCs with minimum distance d = 5, 6, some known constructions are shown to exactly achieve our new bound, which verifies its tightness. For locality r = 2 and distance d = 6, we construct three newSingleton-optimal LRCs whose code length n = 3(q + 1), n = 3(q + √q + 1) and n = 3(2q - 4), respectively. Moreover, we obtain a complete characterization for Singletonoptimal LRCs with r = 2 and d = 6. Such characterization has established an important connection between the existence of Singleton-optimal LRCs and that of a special subset of lines of finite projective plane P G(2, q), thus provides a methodology for constructing LRCs with longer length based on any advance on finite projective plane P G(2, q). In the end, we employ the well-known line-point incidence matrix and Johnson bounds for constant weight codes to derive tighter upper bounds on the code length. These new bounds further help us to prove that some of the previous Singleton-optimal constructions or their extensions achieve the longest possible code length for q = 3, 4, 5, 7. It's worth noting that all of our Singleton-optimal constructions possess small locality r = 2, which are attractive in a DSS.

The parameters of a q-ary MDS Euclidean self-dual codes are completely determined by its length and the construction of MDS Euclidean self-dual codes with new length has been widely investigated in recent years. In this paper, we give a further study on the construction of MDS Euclidean self-dual codes via generalized Reed-Solomon (GRS) codes and their extended codes. The main idea of our construction is to choose suitable evaluation points such that the corresponding (extended) GRS codes are Euclidean self-dual. Firstly, we consider the evaluation set consists of two disjoint subsets, one of which is based on the trace function, the other one is a union of a subspace and its cosets. Then four new families of MDS Euclidean self-dual codes are constructed. Secondly, we give a simple but useful lemma to ensure that the symmetric difference of two intersecting subsets of finite fields can be taken as the desired evaluation set. Based on this lemma, we generalize our first construction and provide two new families of MDS Euclidean self-dual codes. Finally, by using two multiplicative subgroups and their cosets which have nonempty intersection, we present three generic constructions of MDS Euclidean self-dual codes with flexible parameters. Several new families of MDS Euclidean self-dual codes are explicitly constructed.

With the growth of computer vision-based applications, an explosive amount of images have been uploaded to cloud servers that host such online computer vision algorithms, usually in the form of deep learning models. JPEG has been used as the de facto compression and encapsulation method for images. However, standard JPEG configuration does not always perform well for compressing images that are to be processed by a deep learning model—for example, the standard quality level of JPEG leads to 50% of size overhead (compared with the best quality level selection) on ImageNet under the same inference accuracy in popular computer vision models (e.g., InceptionNet and ResNet). Knowing this, designing a better JPEG configuration for online computer vision-based services is still extremely challenging. First, cloud-based computer vision models are usually a black box to end-users; thus, it is challenging to design JPEG configuration without knowing their model structures. Second, the “optimal” JPEG configuration is not fixed; instead, it is determined by confounding factors, including the characteristics of the input images and the model, the expected accuracy and image size, and so forth. In this article, we propose a reinforcement learning (RL)-based adaptive JPEG configuration framework, AdaCompress. In particular, we design an edge (i.e., user-side) RL agent that learns the optimal compression quality level to achieve an expected inference accuracy and upload image size, only from the online inference results, without knowing details of the model structures. Furthermore, we design an explore-exploit mechanism to let the framework fast switch an agent when it detects a performance degradation, mainly due to the input change (e.g., images captured across daytime and night). Our evaluation experiments using real-world online computer vision-based APIs from Amazon Rekognition, Face++, and Baidu Vision show that our approach outperforms existing baselines by reducing the size of images by one-half to one-third while the overall classification accuracy only decreases slightly. Meanwhile, AdaCompress adaptively re-trains or re-loads the RL agent promptly to maintain the performance.

Deep product quantization network (DPQN) has recently received much attention in fast image retrieval tasks due to its efficiency of encoding high-dimensional visual features especially when dealing with large-scale datasets. Recent studies show that deep neural networks (DNNs) are vulnerable to input with small and maliciously designed perturbations (a.k.a., adversarial examples). This phenomenon raises the concern of security issues for DPQN in the testing/deploying stage as well. However, little effort has been devoted to investigating how adversarial examples affect DPQN. To this end, we propose product quantization adversarial generation (PQ-AG), a simple yet effective method to generate adversarial examples for product quantization based retrieval systems. PQ-AG aims to generate imperceptible adversarial perturbations for query images to form adversarial queries, whose nearest neighbors from a targeted product quantizaiton model are not semantically related to those from the original queries. Extensive experiments show that our PQ-AQ successfully creates adversarial examples to mislead targeted product quantization retrieval models. Besides, we found that our PQ-AG significantly degrades retrieval performance in both white-box and black-box settings.

Almost all adversarial attacks in computer vision are aimed at pre-known object categories, which could be offline trained for generating perturbations. But as for visual object tracking, the tracked target categories are normally unknown in advance. However, the tracking algorithms also have potential risks of being attacked, which could be maliciously used to fool the surveillance systems. Meanwhile, it is still a challenging task that adversarial attacks on tracking since it has the free-model tracked target. Therefore, to help draw more attention to the potential risks, we study adversarial attacks on tracking algorithms. In this paper, we propose a novel one-shot adversarial attack method to generate adversarial examples for free-model single object tracking, where merely adding slight perturbations on the target patch in the initial frame causes state-of-the-art trackers to lose the target in subsequent frames. Specifically, the optimization objective of the proposed attack consists of two components and leverages the dual attention mechanisms. The first component adopts a targeted attack strategy by optimizing the batch confidence loss with confidence attention while the second one applies a general perturbation strategy by optimizing the feature loss with channel attention. Experimental results show that our approach can significantly lower the accuracy of the most advanced Siamese network-based trackers on three benchmarks.

Deep neural networks (DNNs) have been applied in class incremental learning, which aims to solve common real-world problems of learning new classes continually. One drawback of standard DNNs is that they are prone to catastrophic forgetting. Knowledge distillation (KD) is a commonly used technique to alleviate this problem. In this paper, we demonstrate it can indeed help the model to output more discriminative results within old classes. However, it cannot alleviate the problem that the model tends to classify objects into new classes, causing the positive effect of KD to be hidden and limited. We observed that an important factor causing catastrophic forgetting is that the weights in the last fully connected (FC) layer are highly biased in class incremental learning. In this paper, we propose a simple and effective solution motivated by the aforementioned observations to address catastrophic forgetting. Firstly, we utilize KD to maintain the discrimination within old classes. Then, to further maintain the fairness between old classes and new classes, we propose Weight Aligning (WA) that corrects the biased weights in the FC layer after normal training process. Unlike previous work, WA does not require any extra parameters or a validation set in advance, as it utilizes the information provided by the biased weights themselves. The proposed method is evaluated on ImageNet-1000, ImageNet-100, and CIFAR-100 under various settings. Experimental results show that the proposed method can effectively alleviate catastrophic forgetting and significantly outperform state-of-the-art methods.

Recently, deep neural networks (DNNs) are shown to be susceptible to data-agnostic quasi-imperceptible noises called Universal Adversarial Perturbations (UAPs). Moreover, the techniques to craft UAPs can be categorized into data-driven and data-free. However, data-free techniques craft UAPs without utilizing any data samples and therefore result in weaker attack capacity. In this paper, we propose a novel method to craft UAPs in the absence of data, via adaptively perturbing mid-layer outputs of the CNN. Based on our proposed self-adaptive attention mechanism, we explore the effects of feature correlation of the internal representations on generating UAPs for the first time. Experimental evaluation demonstrates that UAPs crafted by our Self-Adaptive Feature Fool (SAFF) approach achieve state-of-the-art performance in data-free scenarios.

The deep learning-based visual tracking algorithms such as MDNet achieve high performance leveraging to the feature extraction ability of a deep neural network. However, the tracking efficiency of these trackers is not very high due to the slow feature extraction for each frame in a video. In this paper, we propose an effective tracking algorithm to alleviate the time-consuming problem. Specifically, we design a deep flow collaborative network, which executes the expensive feature network only on sparse keyframes and transfers the feature maps to other frames via optical flow. Moreover, we raise an effective adaptive keyframe scheduling mechanism to select the most appropriate keyframe. We evaluate the proposed approach on large-scale datasets: OTB2013 and OTB2015. The experiment results show that our algorithm achieves considerable speedup and high precision as well.

Visual object tracking has made important breakthroughs with the assistance of deep learning models. Unfortunately, recent research has clearly proved that deep learning models are vulnerable to malicious adversarial attacks, which mislead the models making wrong decisions by perturbing the input image. The threat to the models alerts us to pay attention to the model security of deep learning- based tracking algorithms. Therefore, we study the adversarial attacks against advanced trackers based on deep learning to better identify the vulnerability of tracking algorithms. In this paper, we propose to add slight adversarial perturbations to the input image by an inconspicuous but powerful attack strategy-hijacking algorithm. Specifically, the hijacking strategy misleads trackers in two aspects: one is shape hijacking that changes the shape of the model output; the other is position hijacking that gradually pushes the output to any position in the image frame. Besides, we further propose an adaptive optimization approach to integrate two hijacking mechanisms efficiently. Eventually, the hijacking algorithm results in fooling the tracker to track the wrong target gradually. The experimental results demonstrate the powerful attack ability of our method-quickly hijacking state-of-the-art trackers and reducing the accuracy of these models by more than 90% on OTB2015.

Convolutional neural networks (CNNs) can be easily over-fitted when they are over-parametered. The popular dropout that drops feature units randomly can't always work well for CNNs, due to the problem of under-dropping. To eliminate this problem, some structural dropout methods such as SpatialDropout, Cutout and DropBlock have been proposed. However, these methods that drop feature units in continuous regions randomly, may have the risk of over-dropping, thus leading to degradation of performance. To address these issues, we propose a novel structural dropout method, Correlation based Dropout (CorrDrop), to regularize CNNs by dropping feature units based on feature correlation, which reflects the discriminative information in feature maps. Specifically, the proposed method first obtains correlation map based on the activation in the feature maps, and then adaptively masks out those regions with small average correlation. Thus, the proposed method can regularize CNNs well by discarding part of contextual regions. Extensive experiments on image classification demonstrate the superiority of our method compared with other counterparts.

Principal Component Analysis (PCA) is a popular tool for dimension reduction and feature extraction in data analysis. Probabilistic PCA (PPCA) extends the standard PCA by using a probabilistic model. However, both standard PCA and PPCA are not robust, as they are sensitive to outliers. To alleviate this problem, we propose a novel method called Self-Paced Probabilistic Principal Component Analysis (SP-PPCA) by introducing the Self-Paced Learning mechanism into PPCA. Furthermore, we design the corresponding optimization algorithm based on an alternative search strategy and an expectation-maximization algorithm, so that SP-PPCA uses an iterative procedure to find the optimal projection vectors and filter out outliers. Experiments on both synthetic data and real data demonstrate that SP-PPCA is more robust than the baselines.

The high accuracy and fast running speed are two goals of tracking algorithms. In recent years, many trackers based on deep learning have improved in both aspects respectively, but it is difficult to balance these two indicators. For example, a real-time tracking algorithm named RT-MDNet has greatly increased speed on the MDNet, but the accuracy is still limited to some extent. On the contrary, a state-of-the-art visual tracker based on adversarial learning named VITAL achieves a significant improvement in performance by alleviating the imbalance between positive and negative samples of tracking data. However, its running speed is seriously limited. In this paper, we attempt to combine the advantages from both methods and propose an improved real-time visual tracking algorithm via adversarial learning to get a more balanced result in accuracy and tracking speed. Specifically, we base on the framework of RT-MDNet and introduce a random feature map masking with adversarial learning to improve the quality of feature maps. Experiments on OTB2015 show our algorithm runs at 17 FPS with a precision of 87.6%.

Deep learning has obtained remarkable success for image restoration. However, most existing deep image restoration models are trained by minimizing the pixel-level reconstruction error between restored images and target images (ground truth), while neglecting the rich information from the intermediate feature layers, thus hindering the representational power of networks. To address this problem, we propose a Supervised Target Feature Transfer (STFT) framework to enhance the power of feature expression of the deep image restoration models. Specifically, we introduce a self-supervised antoencoder-based target feature extractor to extract compact feature representation of target images, which serves as supervision signals to train the deep backbone models at the same time. With such feature-level supervised information, deep backbone model can be enhanced by transfer learning of such target features. Moreover, we theoretically analyze our STFT training strategies and demonstrate that it imposes learnable prior information on the backbone restoration model. Extensive experiments demonstrate the effectiveness of our proposed framework compared with the state-of-the-art image restoration models.

Image downscaling has become a classical problem in image processing and has recently connected to image super-resolution (SR), which restores high-quality images from low-resolution ones generated by predetermined downscaling kernels (e.g., bicubic). However, most existing image downscaling methods are deterministic and lose information during the downscaling process, while rarely designing specific downscaling methods for image SR. In this paper, we propose a novel learning-based image downscaling method, Hamiltonian Rescaling Network (HRNet). The design of HRNet is based on the discretization of Hamiltonian System, a pair of iterative updating equations, which formulate a mechanism of iterative correction of the error caused by information missing during image or feature downscaling. Extensive experiments demonstrate the effectiveness of our proposed method in terms of both quantitative and qualitative results.

Convolutional neural networks (CNNs) have been widely used in image super-resolution (SR). Most existing CNN-based methods focus on achieving better performance by designing deeper/wider networks, while suffering from heavy computational cost problem, thus hindering the deployment of such models in mobile devices with limited resources. To relieve such problem, we propose a novel and efficient SR model, named Feature Affinity-based Knowledge Distillation (FAKD), by transferring the structural knowledge of a heavy teacher model to a lightweight student model. To transfer the structural knowledge effectively, FAKD aims to distill the second-order statistical information from feature maps and trains a lightweight student network with low computational and memory cost. Experimental results demonstrate the efficacy of our method and the effectiveness over other knowledge distillation based methods in terms of both quantitative and visual metrics.

Adversarial defense is a popular and important research area. Due to its intrinsic mechanism, one of the most straightforward and effective ways of defending attacks is to analyze the property of loss surface in the input space. In this paper, we define the local flatness of the loss surface as the maximum value of the chosen norm of the gradient regarding to the input within a neighborhood centered on the benign sample, and discuss the relationship between the local flatness and adversarial vulnerability. Based on the analysis, we propose a novel defense approach via regularizing the local flatness, dubbed local flatness regularization (LFR). We also demonstrate the effectiveness of the proposed method from other perspectives, such as human visual mechanism, and analyze the relationship between LFR and other related methods theoretically. Experiments are conducted to verify our theory and demonstrate the superiority of the proposed method.

Training deep neural networks (DNNs) in the presence of noisy labels is an important and challenging task. Probabilistic modeling, which consists of a classifier and a transition matrix, depicts the transformation from true labels to noisy labels and is a promising approach. However, recent probabilistic methods directly apply transition matrix to DNN, neglect DNN's susceptibility to overfitting, and achieve unsatisfactory performance, especially under the uniform noise. In this paper, inspired by label smoothing, we proposed a novel method, in which a smoothed transition matrix is used for updating DNN, to restrict the overfitting of DNN in probabilistic modeling. Our method is termed Matrix Smoothing. We also empirically demonstrate that our method not only improves the robustness of probabilistic modeling significantly, but also even obtains a better estimation of the transition matrix.

Recently, very deep convolutional neural networks (CNNs) have shown great success in single image super-resolution (SISR). Most of these methods focus on the design of network architecture and adopt a sub-pixel convolution layer at the end of network, but few have paid attention to exploring potential representation ability of upscaling layer. Sub-pixel convolution layer aggregates several low resolution (LR) feature maps and builds super-resolution (SR) images in a single step. However, those LR feature maps share similar patterns as they are extracted from a single trunk network. Inspired by this, we propose a novel progressive splitting and upscaling structure, termed PSUS, which generates decoupled feature maps for upscaling layer to get better SR image. Experiments show that our method can not only speed up the convergence, but also achieve considerable improvement on image quality with fewer parameters and lower computational cost.

Deep neural networks (DNNs) have been widely used in scene text recognition, and achieved remarkable performance. Such DNN-based scene text recognizers usually require plenty of training data for training, but data collection and annotation is usually cost-expensive in practice. To alleviate this issue, data augmentation is often applied to train the scene text recognizers. However, existing data augmentation methods including affine transformation and elastic transformation methods suffer from the problems of under- and over-diversity, due to the complexity of text contents and shapes. In this paper, we propose a sample-aware data augmentor to transform samples adaptively based on the contents of samples. Specifically, our data augmentor consists of three parts: gated module, affine transformation module, and elastic transformation module. In our data augmentor, affine transformation module focuses on keeping the affinity of samples, while elastic transformation module aims to improve the diversity of samples. With the gated module, our data augmentor determines transformation type adaptively based on the properties of training samples and the recognizer capability during the training process. Besides, our framework introduces an adversarial learning strategy to optimize the augmentor and the recognizer jointly. Extensive experiments on scene text recognition benchmarks show that our sample-aware data augmentor significantly improves the performance of state-of-the-art scene text recognizer.

Scene text detection (STD) aims to locate text in images and plays an important role in many computer vision tasks including automatic driving and text recognition systems. Recently, deep neural networks (DNNs) have been widely and successfully used in scene text detection, leading to plenty of DNN-based STD methods including regression-based and segmentation-based STD methods. However, recent studies have also shown that DNN is vulnerable to adversarial attacks, which can significantly degrade the performance of DNN models. In this paper, we investigate the robustness of DNN-based STD methods against adversarial attacks. To this end, we propose a generic and efficient attack method to generate adversarial examples, which are produced by adding small but imperceptible adversarial perturbation to the input images. Experiments on attacking four various models and a real-world STD engine of Google optical character recognition (OCR) show that the state-of-the-art DNN-based STD methods including regression-based and segmentation-based methods are vulnerable to adversarial attacks.

Despite being one of the most popular nonparametric approaches, Gaussian process regression (GPR) suffers from O(n3) computational burden and the computation is infeasible for large-scale scenarios. To reduce the computational complexity, many Shannon-mutual-information-based aggregation methods were proposed, whereas these methods can not effectively identify the importance of experts in some cases. To address this problem, we generalize the traditional mutual information-based methods (GPoE, RBCM, GRBCM) based on Tsallis mutual information. Accordingly, the generated weight distribution is more sparse tending to focus on those experts with good performance. To obtain adaptive and data-dependent entropic-index in Tsallis entropy, we propose three heuristic algorithms to solve our model. Extensive experiments show that, the proposed method can improve the prediction of both the mean and variance, and the improvement of variance prediction is significant in many cases.

Traffic demand forecasting is essential to intelligent transportation systems and is widely used to support urban planning, traffic management and vehicle dispatching. One challenge of this problem is to model the complex spatial-temporal correlation. Although both factors have been studied, many of the existing works have strong limitations. They rely too heavily on the locality assumption (i.e., the local area is more relevant than the remote area) and only use a distance-based correlation measurement. However, the spatial correlation is also global (i.e., areas far away may also be relevant) and sparse. And it's insufficient to measure the spatial correlation using only the distance measurement. In this paper, a sparsity and diversity driven correlation network is proposed to tackle these issues. Firstly, Multiple sparse correlation graphs are carefully generated to encode sparsity and diversity. Then a newly designed hybrid graph filtering module (HGFM) leverages them to learn a more expressive node representation. Finally, the HGFM-based recurrent filtering module (RFM) is introduced to handle the spatial-temporal correlation. Extensive experiments conducted on real-world datasets demonstrate the competitiveness of our model while showing the significance of sparsity and diversity.

Deep neural networks (DNNs) exhibit great success on many tasks with the help of large-scale well annotated datasets. However, labeling large-scale data can be very costly and error-prone so that it is difficult to guarantee the annotation quality (i.e., having noisy labels). Training on these noisy labeled datasets may adversely deteriorate their generalization performance. Existing methods either rely on complex training stage division or bring too much computation for marginal performance improvement. In this paper, we propose a Temporal Calibrated Regularization (TCR), in which we utilize the original labels and the predictions in the previous epoch together to make DNN inherit the simple pattern it has learned with little overhead. We conduct extensive experiments on various neural network architectures and datasets, and find that it consistently enhances the robustness of DNNs to label noise.

From the mutual empowerment of two high-speed development technologies: artificial intelligence and edge computing, we propose a tailored Edge Intelligent Video Surveillance (EIVS) system. It is a scalable edge computing architecture and uses multitask deep learning for relevant computer vision tasks. Due to the potential application of different surveillance devices are widely different, we adopt a smart IoT module to normalize the video data of different cameras, thus the EIVS system can conveniently found proper data for a specific task. In addition, the deep learning models can be deployed at every EIVS nodes, to make computer vision tasks on the normalized data. Meanwhile, due to the training and deploying of deep learning model are usually separated, for the related tasks in the same scenario, we propose to collaboratively train the depth learning models in a multitask paradigm on the cloud server. The simulation results on the publicly available datasets show that the system continuously supports intelligent monitoring tasks, has good scalability, and can improve performance through multitask learning.

Locally repairable codes (LRCs) with locality r were introduced to recover an erased code symbol by accessing at most r other code symbols. An LRC achieving the well-known Singleton-type bound is called an optimal LRC. Constructing optimal LRCs has been a hot topic of coding theory in recent years. Similar to the famous MDS conjecture, the maximum code length of an optimal LRC has been investigated by Guruswami et al. (TIT2019) and some constructions of optimal LRCs with large code length are also presented by Jin (TIT2019) and Xing and Yuan (arXiv2018). In this paper, we consider the maximum code length of optimal LRCs with minimum distance 6 and locality 2. Firstly, we give a complete characterization for optimal LRCs with d = 6 and r = 2, which shows that the existence of such an LRC is equivalent to the existence of a special subset of lines of finite projective plane PG(2, q). Based on this characterization, we generalize the results of Chen et al. (ISIT2018) and obtain two new constructions of optimal (n, k, d = 6; r = 2)-LRCs with n = 3(q + √q + 1) and n = 3(2q -4), respectively. By using the techniques of line-point incidence matrix and Johnson bound, we show that the code length of any q-ary optimal LRCs with d = 6 and r = 2 must be bounded by O(q1.5). To the best of our knowledge, both of the code length of our new constructions and upper bounds are better than previously known ones. Moreover, we also determine the exact value of the maximum code length of q-ary optimal LRCs with d = 6 and r = 2 for q = 4, 5.

Linear codes with locality, called locally repairable codes (LRCs), have been applied in distributed storage systems (DSSs) to minimize the number of storage nodes to be downloaded during repairing a failed node. A linear code has locality r if one can recover an erased code symbol by accessing at most r other code symbols. Bounds and constructions of LRCs have been widely investigated in recent years. In this paper, we first propose the definition of perfect LRCs, whose dimension k achieves the Hamming-type bound proposed by Wang et al. (TIT2019). Then we establish important connections of the existence of LRCs with finite geometry and finite fields, and two systematic constructions of perfect LRCs are obtained. Rewriting the Hamming-type bound by the property of integers, we present a new construction of k-optimal LRCs achieving this bound, which have longer code length than the previously known ones.

The weight distribution of a q-ary [n, k, d] linear code is an important research subject in coding theory. In a linear code, a code symbol is said to have locality r if it can be recovered by accessing at most r other code symbols. A q-ary locally repairable code (LRC) is an [n, k, d] linear code over Fq such that every code symbol has locality r, and is said to be optimal if the minimum distance attains the well-known Singleton-like bound. In this paper, we focus on the weight distributions of q-ary optimal LRCs with locality 2, minimum distance 5 and even dimension k. By analyzing the parity-check matrices involving locality, it is shown that the weight distributions of all q-ary optimal LRCs with locality 2, distance 5, even dimension k and code length n can be uniquely determined and explicit expressions of the weight distributions are given.

The falling accident is one of the largest threats to human health, which leads to broken bones, head injury, or even death. Therefore, automatic human fall recognition is vital for the Activities of Daily Living (ADL). In this paper, we try to define multi-level computer vision tasks for the visually observed fall recognition problem and study the methods and pipeline. We make frame-level labels for the fall action on several ADL datasets to test the methods and support the analysis. While current deep-learning fall recognition methods usually work on the sequence-level input, we propose a novel Dynamic Pose Motion (DPM) representation to go a step further, which can be captured by a flexible motion extraction module. Besides, a sequence-level fall recognition pipeline is proposed, which has an explicit two-branch structure for the appearance and motion feature, and has canonical LSTM to make temporal modeling and fall prediction. Finally, while current research only makes a binary classification on the fall and ADL, we further study how to detect the start time and the end time of a fall action in a video-level task. We conduct analysis experiments and ablation studies on both the simulated and real-life fall datasets. The relabelled datasets and extensive experiments form a new baseline on the recognition of falls and ADL.

Deep neural networks (DNNs) have shown serious vulnerability to adversarial examples with imperceptible perturbation to clean images. Most existing input-transformation based defense methods (e.g., ComDefend) rely heavily on the learned external priors from an external large training dataset, while neglecting the rich image internal priors of the input itself, thus limiting the generalization of the defense models against the adversarial examples with biased image statistics from the external training dataset. Motivated by deep image prior that can capture rich image statistics from a single image, we propose an effective Deep Image Prior Driven Defense (DIPDefend) method against adversarial examples. With a DIP generator to fit the target/adversarial input, we find that our image reconstruction exhibits quite interesting learning preference from a feature learning perspectives, i.e., the early stage primarily learns the robust features resistant to adversarial perturbation, followed by learning non-robust features that are sensitive to adversarial perturbation. Besides, we develop an adaptive stopping strategy that adapts our method to diverse images. In this way, the proposed model obtains a unique defender for each individual adversarial input, thus being robust to various attackers. Experimental results demonstrate the superiority of our method over the state-of-the-art defense methods against white-box and black-box adversarial attacks.

In this paper we propose Stochastic Deep Gaussian Processes over Graphs (DGPG), which are deep structure models that learn the mappings between input and output signals in graph domains. The approximate posterior distributions of the latent variables are derived with variational inference, and the evidence lower bound is evaluated and optimized by the proposed recursive sampling scheme. The Bayesian non-parametric natural of our model allows it to resist overfitting, while the expressive deep structure grants it the potential to learn complex relations. Extensive experiments demonstrate that our method achieves superior performances in both small size (< 50) and large size (> 35,000) datasets. We show that DGPG outperforms another Gaussian-based approach, and is competitive to a state-of-the-art method in the challenging task of traffic flow prediction. Our model is also capable of capturing uncertainties in a mathematical principled way and automatically discovering which vertices and features are relevant to the prediction.

The study on improving the robustness of deep neural networks against adversarial examples grows rapidly in recent years. Among them, adversarial training is the most promising one, which flattens the \textit{input loss landscape} (loss change with respect to input) via training on adversarially perturbed examples. However, how the widely used \textit{weight loss landscape} (loss change with respect to weight) performs in adversarial training is rarely explored. In this paper, we investigate the weight loss landscape from a new perspective, and identify a clear correlation between the flatness of weight loss landscape and robust generalization gap. Several well-recognized adversarial training improvements, such as early stopping, designing new objective functions, or leveraging unlabeled data, all implicitly flatten the weight loss landscape. Based on these observations, we propose a simple yet effective \textit{Adversarial Weight Perturbation (AWP)} to explicitly regularize the flatness of weight loss landscape, forming a \textit{double-perturbation} mechanism in the adversarial training framework that adversarially perturbs both inputs and weights. Extensive experiments demonstrate that AWP indeed brings flatter weight loss landscape and can be easily incorporated into various existing adversarial training methods to further boost their adversarial robustness.

A locally repairable code (LRC) is a linear code such that every code symbol can be recovered by accessing a small number of other code symbols. In this paper, we study bounds and constructions of LRCs from the viewpoint of parity-check matrices. Firstly, a simple and unified framework based on parity-check matrix to analyze the bounds of LRCs is proposed, and several new explicit bounds on the minimum distance of LRCs in terms of the field size are presented. In particular, we give an alternate proof of the Singleton-like bound for LRCs first proved by Gopalan et al. Some structural properties on optimal LRCs that achieve the Singleton-like bound are given. Then, we focus on constructions of optimal LRCs over the binary field. It is proved that there are only five classes of possible parameters with which optimal binary LRCs exist. Moreover, by employing the proposed parity-check matrix approach, we completely enumerate all these five classes of optimal binary LRCs attaining the Singleton-like bound in the sense of equivalence of linear codes.

In large-scale distributed machine learning (DML), the network performance between machines significantly impacts the speed of iterative training. In this paper we propose BML, a scalable, high-performance and fault-tolerant DML network architecture on top of Ethernet and commodity devices. BML builds on BCube topology, and runs a fully-distributed gradient synchronization algorithm. Compared to a Fat-Tree network with the same size, a BML network is expected to take much less time for gradient synchronization, for both low theoretical synchronization time and its benefit to RDMA transport. With server/link failures, the performance of BML degrades in a graceful way. Experiments of MNIST and VGG-19 benchmarks on a testbed with 9 dual-GPU servers show that, BML reduces the job completion time of DML training by up to 56.4% compared with Fat-Tree running state-of-the-art gradient synchronization algorithm.

Recently, deep convolutional neural networks (CNNs) have been widely explored in single image super-resolution (SISR) and obtained remarkable performance. However, most of the existing CNN-based SISR methods mainly focus on wider or deeper architecture design, neglecting to explore the feature correlations of intermediate layers, hence hindering the representational power of CNNs. To address this issue, in this paper, we propose a second-order attention network (SAN) for more powerful feature expression and feature correlation learning. Specifically, a novel train- able second-order channel attention (SOCA) module is developed to adaptively rescale the channel-wise features by using second-order feature statistics for more discriminative representations. Furthermore, we present a non-locally enhanced residual group (NLRG) structure, which not only incorporates non-local operations to capture long-distance spatial contextual information, but also contains repeated local-source residual attention groups (LSRAG) to learn increasingly abstract feature representations. Experimental results demonstrate the superiority of our SAN network over state-of-the-art SISR methods in terms of both quantitative metrics and visual quality.

Reinforcement learning is a framework to make sequential decisions. The combination with deep neural networks further improves the ability of this framework. Convolutional nerual networks make it possible to make sequential decisions based on raw pixels information directly and make reinforcement learning achieve satisfying performances in series of tasks. However, convolutional neural networks still have own limitations in representing geometric patterns and long-term dependencies that occur consistently in state inputs. To tackle with the limitation, we propose the self-attention architecture to augment the original network. It provides a better balance between ability to model long-range dependencies and computational efficiency. Experiments on Atari games illustrate that self-attention structure is significantly effective for function approximation in deep reinforcement learning.

Adversarial perturbations of clean images are usually imperceptible for human eyes, but can confidently fool deep neural networks (DNNs) to make incorrect predictions. Such vulnerability of DNNs raises serious security concerns about their practicability in security-sensitive applications. To defend against such adversarial perturbations, recently developed PixelDefend purifies a perturbed image based on PixelCNN in a raster scan order (row/column by row/column). However, such scan mode insufficiently exploits the correlations between pixels, which further limits its robustness performance. Therefore, we propose a more advanced Hilbert curve scan order to model the pixel dependencies in this paper. Hilbert curve could well preserve local consistency when mapping from 2-D image to 1-D vector, thus the local features in neighboring pixels can be more effectively modeled. Moreover, the defensive power can be further improved via ensembles of Hilbert curve with different orientations. Experimental results demonstrate the superiority of our method over the state-of-the-art defenses against various adversarial attacks.

Dropout has been widely used in fully connected networks but becomes less effective for convolutional neural networks (CNNs), since the spatially correlated features still allow dropped information to flow through the network. To make dropout more practical for CNNs, structured dropout methods have been recently proposed by dropping regions with fixed shapes and random positions, which nonetheless may lead to unexpected discarding of information. To address this problem, in this paper, we propose a novel dropout variant based on attention information named AttentionDrop that drops features adaptively. Specifically, it precisely localizes masks that have irregular shapes according to the values of activation units. In addition, the use of soft values in adaptive masks lowers the risk of a complete loss of indispensable information. Experimental results demonstrate the effectiveness of our AttentionDrop on public datasets for image classification.

Perceptual quality of a video describes the quality consistent with human perception. The growing popularity of short video sharing on mobile platforms such as Tik Tok and WeSee makes the video assessment system based on perceptual quality a necessity. In practice, short videos captured by mobile devices often contain different types of distortions incurred by sensor noise or compression noise, which potentially makes the videos visually unpleasing to users and may degrade the performance of deep neural networks when applied to these noisy videos. Thus, it is necessary to identify noisy videos based on video perceptual quality. However, traditional video/image noise estimation methods are designed to estimate the variance of homogeneously distributed synthetic noise, not real noise. In this paper, we propose a simple yet effective method to recognize the noisy videos using their residual frames. Since the original video frame contains rich content information, which may result in under-or over-estimation of the noise, we construct residual frames to reduce the influence of the content information while maintaining the main noise information in the video. We also create a new data set with more than 30 thousand images captured from videos with real noise. Experimental results demonstrate the effectiveness of our proposed method.

Grassland degradation estimation is essential to prevent global land desertification and sandstorms. Typically, the key to such estimation is to measure the coverage of indicator plants. However, traditional methods of estimation rely heavily on human eyes and manual labor, thus inevitably leading to subjective results and high labor costs. In contrast, deep learning-based image segmentation algorithms are potentially capable of automatic assessment of the coverage of indicator plants. Nevertheless, a suitable image dataset comprising grassland images is not publicly available. To this end, we build an original Automatic Grassland Degradation Estimation Dataset (AGDE-Dataset), with a large number of grassland images captured from the wild. Based on AGDE-Dataset, we are able to propose a brand new scheme to automatically estimate grassland degradation, which mainly consists of two components. 1) Semantic segmentation: we design a deep neural network with an improved encoder-decoder structure to implement semantic segmentation of grassland images. In addition, we propose a novel Focal-Hinge Loss to alleviate the class imbalance of semantics in the training stage. 2) Degradation estimation: we provide the estimation of grassland degradation based on the results of semantic segmentation. Experimental results show that the proposed method achieves satisfactory accuracy in grassland degradation estimation.

Repair locality has been an important metric in distributed storage systems (DSS). Erasure codes with small locality are more popular in DSS, which means fewer available nodes participating in the repair of failed nodes. Locally repairable codes (LRCs) peoposed as a new coding scheme, give more rise to the system performance and attract a lot of interest in the theoretical research in coding theory. The particular concern among the research problems is the bounds and optimal constructions of LRCs. In this direction, we first of all derive an improved upper bound on the code length of optimal LRCs with minimum distance d = 5, 6, some known constructions are shown to exactly achieve our new bound, which verifies its tightness. Then we construct a class of distance-optimal LRCs based on the structure of a sunflower, whose code length n = 3(q + 1), locality r = 2 and distance d = 6. Note that the code length of this class of LRCs outperforms all known optimal constructions with the same parameters. Moreover, by employing the combinatorial structure of the sunflower and the q-Steiner system, we obtain two classes of k-optimal (dimension-optimal) LRCs with respect to our new bound for d = 6. It's worth noting that all of our optimal constructions possess small locality r = 2, which are attractive in DSS.

In a linear code, a code symbol with (r, δ)-locality can be repaired by accessing at most r other code symbols in case of at most δ - 1 erasures. A q-ary (n, k, r, δ) locally repairable codes (LRC) in which every code symbol has (r, δ)-locality is said to be optimal if it achieves the Singleton-like bound derived by Prakash et al.. In this paper, we study the classification of optimal ternary (n, k, r, δ)-LRCs (δ > 2). Firstly, we propose an upper bound on the minimum distance of optimal q-ary LRCs in terms of the field size. Then, we completely determine all the 6 classes of possible parameters with which optimal ternary (n, k, r, δ)-LRCs exist. Moreover, explicit constructions of all these 6 classes of optimal ternary LRCs are proposed in the paper.

With the growth of computer vision based applications and services, an explosive amount of images have been uploaded to cloud servers which host such computer vision algorithms, usually in the form of deep learning models. JPEG has been used as the \em de facto compression and encapsulation method before one uploads the images, due to its wide adaptation. However, standard JPEG configuration does not always perform well for compressing images that are to be processed by a deep learning model, e.g., the standard quality level of JPEG leads to 50% of size overhead (compared with the best quality level selection) on ImageNet under the same inference accuracy in popular computer vision models including InceptionNet, ResNet, etc. Knowing this, designing a better JPEG configuration for online computer vision services is still extremely challenging: 1) Cloud-based computer vision models are usually a black box to end-users; thus it is difficult to design JPEG configuration without knowing their model structures. 2) JPEG configuration has to change when different users use it. In this paper, we propose a reinforcement learning based JPEG configuration framework. In particular, we design an agent that adaptively chooses the compression level according to the input image's features and backend deep learning models. Then we train the agent in a reinforcement learning way to adapt it for different deep learning cloud services that act as the \em interactive training environment and feeding a reward with comprehensive consideration of accuracy and data size. In our real-world evaluation on Amazon Rekognition, Face++ and Baidu Vision, our approach can reduce the size of images by 1/2 -- 1/3 while the overall classification accuracy only decreases slightly.

With the increase of HTTP-based adaptive video streaming over the Internet, multiple clients may compete for a shared bottleneck bandwidth, which brings some damage to the fairness and stability of Quality of Experience (QoE). This paper presents Steward, a system that enforces multi-client joint QoE optimization for bottleneck bandwidth sharing. Joint QoE optimization refers to improving QoE fairness among clients with various video devices and providing differentiated service for clients with different priorities. Steward deploys the adaptive bitrate (ABR) algorithm based on neural networks (NN) and reinforcement learning at the network edge. The ABR agent trains the NN model through experience and makes appropriate bitrate guidance for video chunks to be requested by clients sharing the same bottleneck bandwidth. We compare Steward with state-of-the-art algorithms under different network conditions. Compared with all considered algorithms and conditions, Steward reduces 30%~85% QoE unfairness under the premise of differentiated service.

The ensemble of Convolutional Neural Networks (CNNs) is known to be more accurate and robust than the component CNNs models. Along with the development of a fast training method, current research has managed to make an effective ensemble of several CNNs models and require no additional training cost. However, when the ensemble size of CNNs is further increased, it is hard to observe a corresponding performance enhancement. According to the generalization capability analysis of CNNs, this phenomenon can be explained by the oversaturation of model capacity and the close correlation among the component CNNs, especially when the CNNs are trained within the same dataset. To address this problem, we propose to train CNNs on re-sampled bootstrap datasets. Extensive experiments demonstrate the bootstrap re-sampling is effective for a large ensemble size (up to 80). Besides, benefiting from the usage of the bootstrap re-sampling technique, we can also have an unbiased estimate of the standard deviation of the ensemble output.

Locally repairable codes (LRCs) are introduced in distributed storage systems due to their low repair overhead. An LRC is called optimal if its minimum distance attains the Singleton-like upper bound. Chen et al. (2018) recently studied the constructions of optimal (r, δ)-LRCs with length n | (q+1) and (r + δ - 1) | n, where many classes of optimal cyclic constructions were obtained. In this paper, by employing constacyclic MDS codes, we construct seven classes of optimal (r, δ)-LRCs with new parameters. After adding these new optimal LRCs via constacyclic codes, we have completely obtained all optimal (r, δ)-LRCs with length n | (q + 1) and (r + δ - 1) | n for all possible parameters for the completeness in the coding theory. It is worth noting that the optimal constacyclic LRCs with new parameters provide more alternatives to cyclic LRCs in the practical demands of distributed storage systems, where specific values of n, k, r, and δ are required. Moreover, constacyclic LRCs also possess the encoding and decoding efficiency as cyclic LRCs.

Large-scale datasets possessing clean label annotations are crucial for training Convolutional Neural Networks (CNNs). However, labeling large-scale data can be very costly and error-prone, and even high-quality datasets are likely to contain noisy (incorrect) labels. Existing works usually employ a closed-set assumption, whereby the samples associated with noisy labels possess a true class contained within the set of known classes in the training data. However, such an assumption is too restrictive for many applications, since samples associated with noisy labels might in fact possess a true class that is not present in the training data. We refer to this more complex scenario as the open-set noisy label problem and show that it is nontrivial in order to make accurate predictions. To address this problem, we propose a novel iterative learning framework for training CNNs on datasets with open-set noisy labels. Our approach detects noisy labels and learns deep discriminative features in an iterative fashion. To benefit from the noisy label detection, we design a Siamese network to encourage clean labels and noisy labels to be dissimilar. A reweighting module is also applied to simultaneously emphasize the learning from clean labels and reduce the effect caused by noisy labels. Experiments on CIFAR-10, ImageNet and real-world noisy (web-search) datasets demonstrate that our proposed model can robustly train CNNs in the presence of a high proportion of open-set as well as closed-set noisy labels.

Recently developed Dual Domain Image Denoising (DDID) algorithm is a simple version of block-matching 3D filtering (BM3D) by combining bilateral filter and frequency-based method. DDID and its invariants have achieved competitive results compared with state-of-the-art methods. However, this kind of methods share a common drawback: there are a few parameters of the algorithms that are data- and noise-dependent, and difficult to tune. In this paper, we propose to use Stein's unbiased risk estimate (SURE) to measure the mean square error (MSE) of the DDID algorithm for restoration of an image contaminated with additive white Gaussian noise. We derive an explicit expression for SURE value to optimize parameters without access to the noise-free signal. Experimental results demonstrate the effectiveness of the proposed parameter selection in term of both quantitative and qualitative metrics.

Gaussian mixture model (GMM) is a powerful probabilistic model for representing the probability distribution of observations in the population. However, the fitness of Gaussian mixture model can be significantly degraded when the data contain a certain amount of outliers. Although there are certain variants of GMM (e.g., mixture of Laplace, mixture of t distribution) attempting to handle outliers, none of them can sufficiently mitigate the effect of outliers if the outliers are far from the centroids. Aiming to remove the effect of outliers further, this paper introduces a Self-Paced Learning mechanism into mixture of t distribution, which leads to Self-Paced Mixture of t distribution model (SPTMM). We derive an Expectation-Maximization based algorithm to train SPTMM and show SPTMM is able to screen the outliers. To demonstrate the effectiveness of SPTMM, we apply the model to density estimation and clustering. Finally, the results indicate that SPTMM outperforms other methods, especially on the data with outliers.

A logically centralized controller of Software Defined Networking (SDN) is responsible for the management of routing planning and monitoring of network status. With this central control and global visibility, in this paper, we propose a congestion- aware fast failure recovery scheme (CAFFE), which can not only recover the affected flows fast from a wide range of failure scenarios but also avoid potential congestion in the post-recovery network. CAFFE achieves fast recovery by detouring the traffic from neighbour switches as soon as they detect failures and avoids potential congestion by protected paths constructed with the combined consideration of the knowledge of network topology, failure states and network load distribution. We formulate and solve the protected path planning problems where all potential network failures should be protected and the maximum link load in the post- recovery network should be minimized. The resulting protected paths ensure high traffic reachability and low link load even when network failure happens. All these paths are set up a priori by CAFFE. This allows switches to handle failures automatically without the need on waiting for the response from the controller. Experimental results show that CAFFE achieves fast recovery with relatively low overheads on either switches (in terms of backup rule number to configure) or controller (in terms of computational time). It also shows that CAFFE is able to better load-balance the post-recovery network comparing to shortest path re-calculation and OpenFlow-based Segment Protection (OSP).

Noisy labels modeling makes a convolutional neural network (CNN) more robust for the image classification problem. However, current noisy labels modeling methods usually require an expectation-maximization (EM) based procedure to optimize the parameters, which is computationally expensive. In this paper, we utilize a fast annealing training method to speed up the CNN training in every M-step. Since the training is repeated executed along the entire EM optimization path and obtain many local minimal CNN models from every training cycle, we name it as the Cyclic Annealing Training (CAT) approach. In addition to reducing the training time, CAT can further bagging all the local minimal CNN models at the test time to improve the performance of classification. We evaluate the proposed method on several image classification datasets with different noisy labels patterns, and the results show that our CAT approach outperforms state-of-the-art noisy labels modeling methods.

The goal of artistic style transfer is to transfer the style of artistic works into photos. However, the performances of existing style transfer algorithms on portraits are not very satisfactory, because the synthetic photo is either not sufficiently stylized or distorted severely in the portrait domain (i.e., foreground), which limits the use of style transfer for portraits. In this paper, we propose a novel portrait-aware artistic style transfer algorithm, which treats foreground and background differently. Particularly, we separate the foreground from the background, and apply fine-grained style transfer to the background and coarse-grained style transfer to the entire image at the same time, so that the artistic style of entire image can be transferred with the details of the portrait well preserved. Extensive experiments demonstrate the effectiveness of our proposed method.

Datasets with significant proportions of noisy (incorrect) class labels present challenges for training accurate Deep Neural Networks (DNNs). We propose a new perspective for understanding DNN generalization for such datasets, by investigating the dimensionality of the deep representation subspace of training samples. We show that from a dimensionality perspective, DNNs exhibit quite distinctive learning styles when trained with clean labels versus when trained with a proportion of noisy labels. Based on this finding, we develop a new dimensionality-driven learning strategy, which monitors the dimensionality of subspaces during training and adapts the loss function accordingly. We empirically demonstrate that our approach is highly tolerant to significant proportions of noisy labels, and can effectively learn low-dimensional local subspaces that capture the data distribution.

Pseudo-cyclic codes is a generalization of cyclic codes and provides a way to obtain MDS codes with more parameters in coding theory. Specially, if a is not a quadratic residue in Fq, xn-a only has quadratic factors over Fqfor even n and k. Based on these facts, we consider the constructions of optimal q-ary pseudo-cyclic (r, δ) locally repairable codes (LRCs) with length n | q+1 in this paper. To be specific, we obtain four classes of optimal pseudo-cyclic (r, δ) -LRCs with new parameters.

A code symbol in an$[n,\ k,\ d]$linear code is said to have locality$r$if it can be repaired from at most$r$other code symbols. An$(n,\ k,\ r)$locally repairable code (LRC) in which every code symbol has locality$r$is said to be optimal if its minimum distance achieves the Singleton-like bound derived by Gopalan et al. In this paper, we study the maximal code length of a q-ary optimal$(n,\ k,\ r)$-LRC. Firstly, we give an upper bound on the code length of q-ary optimal LRCs, and then derive some structural properties and the weight hierarchy of optimal LRCs with maximal code length. Finally, we give some constructions of optimal q-ary LRCs with maximal code length.

Locally reparable codes (LRCs) are introduced in distributed storage due to their lower repair degree compared with Reed-Solomon codes. However, Reed-Solomon (RS) codes have recently attracted renewed attention due to the lowbandwidth linear repair scheme proposed by Guruswami and Wootters. Based on the linear repair scheme for RS codes, we consider the bandwidth-efficient repair schemes for distanceoptimal scalar (r, δ) LRCs in this paper. More precisely, we obtain bandwidth-efficient repair schemes for two classes of known distance-optimal scalar (r, δ) LRCs based on polynomial evaluations, whose local sub-codes are proved to be RS codes or GRS codes.

Optimal (r, δ) locally repairable codes ((r, δ)-LRCs for short) with length n | (q+1) have been studied in [5] and [6]. In this paper, along with their ideas, by using cyclic or constacyclic codes, we construct three classes of such LRCs with new parameters which are not obtained in [5] and [6]. Thus, optimal (r, δ)-LRCs with length n | (q+1) and (r + δ - 1) | n are completely determined for all possible parameters.

Network traffic classification, which can map network traffic to protocols in the application layer, is a fundamental technique for network management and security issues such as Quality of Service, network measurement, and network monitoring. Recent researchers focus on extracting features for traditional machine learning methods from flows or datagrams of the specific protocol. However, as the rapid growth of network applications, previous works cannot handle complex novel protocols well. In this paper, we introduce the recurrent neural network to network traffic classification and design a novel neural network, the Byte Segment Neural Network (BSNN). BSNN treats network datagrams as input and gives the classification results directly. In BSNN, a datagram is firstly broken into serval byte segments. Then, these segments are fed to encoders which are based on the recurrent neural network. The information extracted by encoders is combined to a representation vector of the whole datagram. Finally, we apply the softmax function to use this vector for predicting the application protocol of this datagram. There are several key advantages of BSNN: 1) no need for prior knowledge of target applications; 2) can handle both connection-oriented protocols and connection-less protocols; 3) supports multi-classification for protocols; 4) shows outstanding accuracy in both traditional protocols and complex novel protocols. Our thorough experiments on real-world data with different protocols indicate that BSNN gains average F1-measure about 95.82% in multi-classification for five protocols including QQ, PPLive, DNS, 360 and BitTorrent. And it also shows excellent performance for detection of novel protocols. Furthermore, compared with two recent state-of-the-art works, BSNN has superiority over the traditional machine learning-based method and the packet inspection method.

In distributed machine learning (DML), the network performance between machines significantly impacts the speed of iterative training. In this paper we propose BML, a new gradient synchronization algorithm with higher network performance and lower network cost than the current practice. BML runs on BCube network, instead of using the traditional Fat-Tree topology. BML algorithm is designed in such a way that, compared to the parameter server (PS) algorithm on a Fat-Tree network connecting the same number of server machines, BML achieves theoretically 1/k of the gradient synchronization time, with k/5 of switches (the typical number of k is 2∼4). Experiments of LeNet-5 and VGG-19 benchmarks on a testbed with 9 dual-GPU servers show that, BML reduces the job completion time of DML training by up to 56.4%.

A code is said to be an r-local locally repairable code (LRC) if each of its coordinates can be repaired by accessing at most r other coordinates. When some of the r coordinates are also erased, the r-local LRC cannot accomplish the local repair, which leads to the concept of (r, δ)-locality. A q-ary [n, k] linear code C is said to have (r, δ)-locality (δ ≥ 2) if for each coordinate i, there exists a punctured subcode of C with support containing i, whose length is at most r+δ-1, and whose minimum distance is at least δ. The (r, δ)-LRC can tolerate δ-1 erasures in every local code (i.e., punctured subcode), which degenerates to an r-local LRC when δ = 2. A q-ary (r, δ) LRC is called optimal if it meets the singleton-like bound for (r, δ)-LRCs. A class of optimal q-ary cyclic r-local LRCs with lengths n | q - 1 were constructed by Tamo, Barg, Goparaju, and Calderbank based on the q-ary Reed-Solomon codes. In this paper, we construct a class of optimal q-ary cyclic (r, δ)-LRCs (δ ≥ 2) with length n | q - 1, which generalizes the results of Tamo et al. Moreover, we construct a new class of optimal q-ary cyclic r-local LRCs with lengths n | q + 1 and a new class of optimal q-ary cyclic (r, δ)-LRCs (δ ≥ 2) with lengths n | q + 1. The constructed optimal LRCs with length n = q + 1 have the best-known length for a given finite field with size q when the minimum distance is larger than 4.

Random forests (RFs) are recognized as one type of ensemble learning method and are effective for the most classification and regression tasks. Despite their impressive empirical performance, the theory of RFs has yet been fully proved. Several theoretically guaranteed RF variants have been presented, but their poor practical performance has been criticized. In this paper, a novel RF framework is proposed, named Bernoulli RFs (BRFs), with the aim of solving the RF dilemma between theoretical consistency and empirical performance. BRF uses two independent Bernoulli distributions to simplify the tree construction, in contrast to the RFs proposed by Breiman. The two Bernoulli distributions are separately used to control the splitting feature and splitting point selection processes of tree construction. Consequently, theoretical consistency is ensured in BRF, i.e., the convergence of learning performance to optimum will be guaranteed when infinite data are given. Importantly, our proposed BRF is consistent for both classification and regression. The best empirical performance is achieved by BRF when it is compared with state-of-the-art theoretical/consistent RFs. This advance in RF research toward closing the gap between theory and practice is verified by the theoretical and experimental studies in this paper.

Correlation measures are a key element of statistics and machine learning, and essential for a wide range of data analysis tasks. Most existing correlation measures are for pairwise relationships, but real-world data can also exhibit complex multivariate correlations, involving three or more variables. We argue that multivariate correlation measures should be comparable, interpretable, scalable and unbiased. However, no existing measures satisfy all these requirements. In this paper, we propose an unbiased multivariate correlation measure, called UMC, which satisfies all the above criteria. UMC is a cumulative entropy based non-parametric multivariate correlation measure, which can capture both linear and non-linear correlations for groups of three or more variables. It employs a correction for chance using a statistical model of independence to address the issue of bias. UMC has high interpretability and we empirically show it outperforms state-of-the-art multivariate correlation measures in terms of statistical power, as well as for use in both subspace clustering and outlier detection tasks.

This paper studies a class of binary matrices with correlations between distinct columnsequal to zero or one, which has reported comparable performance with random matrices inrecent studies of compressed sensing. For such matrix, we analyze its structure propertyand provide an improved performance estimation.

Owing to its simplicity and flexibility, the decision tree remains an important analysis tool in many real-world learning tasks. A lot of decision tree algorithms have been proposed, such as ID3, C4.5 and CART, which represent three most prevalent criteria of attribute splitting, i.e., Shannon entropy, Gain Ratio and Gini index respectively. These splitting criteria seem to be independent and to work in isolation. However, in this paper, we find that these three attribute splitting criteria can be unified in a Tsallis entropy framework. More importantly, theoretically, we reveal the relations between Tsallis entropy and the above three prevalent attribute splitting criteria. In addition, we generalize the splitting criterion of the decision tree, and provide a new simple but efficient approach, Unified Tsallis Criterion Decision Tree algorithm (UTCDT), to enhance the performance of the decision tree. Experimental evidences demonstrate that UTCDT achieves statistically significant improvement over the classical decision tree algorithms, even yields comparable performance to state-of-the-art classification algorithm.

Recently linear codes with locality properties have attracted a lot of interest due to their desirable applications in distributed storage systems. An [n, k, d] linear code with (r, δ)-locality can enable the local recovery of a failed node in case of more than one node failures. In this paper, we study the theoretical bounds and constructions of linear codes with (r, δ)-locality for all code symbols. A parity-check matrix approach is employed to present an alternate simple proof of the Singleton-like bound for linear codes with all symbol (r, δ)-locality. A refined Singleton-like bound is given for the case that r | k and r + δ - 1 † n. Base on the new proof technique, we enumerate all the possible two classes of optimal binary linear codes meeting the Singleton-like bound. In other words, except the proposed two classes of optimal binary linear codes, there is no other binary linear codes with minimum distance d = n - k - ([k/r] - 1)(δ- 1) + 1.

Recently developed dual domain image denoising (DDID) algorithm and its variants, such as dual domain filter (DDF), achieve remarkable results by combining bilateral filter with frequency-based method. However, this kind of algorithms require large patches to guarantee the denoising performance and most of them produce ringing artifacts due to the Gibbs phenomenon induced by high-contrast details. To address these issues, we propose a Foveated Nonlocal Dual Denoising (FNDD) algorithm by unifying foveated nonlocal means and frequency-based methods. In this way, the ability to preserve the high-contrast details is noticeably improved by exploiting foveated self-similarity (patch similarity) instead of pixel similarity, thus leading to void of artifacts. Moreover, we propose an entropy-based back projection step for compensating the detail loss to further improve the performance. Experimental results validate that FNDD significantly outperforms DDID in terms of both quantitative metrics and subjective visual quality under much smaller patches, and even achieves comparable results against state-of-the-art competitors.

It is known that images available usually undergo some stages of processing (e.g., acquisition, compression, transmission and display), and each stage may introduce certain type of distortion. Hence, images distorted by multiple types of distortions are common in real applications. Research in human visual perception has evidenced that the human visual system (HVS) is sensitive to image structural information. This fact inspires us to design a new blind/no-reference (NR) image quality assessment (IQA) method to evaluate the visual quality of multiply-distorted images based on structural degradation. Specifically, quality-aware features are extracted from both the first- and high-order image structures by local binary pattern (LBP) operators. Experimental results on two well-known multiply-distorted image databases demonstrate the outstanding performance of the proposed method.

Among the schemes proposed for failure recovery in software-defined networks, installing backup paths in advance is considered to be an effective approach to reduce the recovery latency. However, the pre-installation poses undue storage overheads on flow tables. In this paper, we propose a customized and cost-efficient backup scheme, which achieves fast recovery from any single-link failure. We introduce an improved breadth first search algorithm to construct the customized backup paths of flows to accommodate their diverse routing demands. By analyzing the path characteristics carefully, we observe that non-conflicted backup paths can be aggregated with each other to reduce the number of backup rules. We formulate this backup path aggregation as an optimization problem. The challenge is how to aggregate the backup paths without causing routing ambiguity. To this end, we design a two-stage aggregation (2SA) algorithm. At its core, 2SA leverages conflict matrixes to guarantee the correctness of aggregation. We evaluate our scheme comprehensively with both the real-world topologies and generated topologies. Simulation results show that our scheme can pre-install the customized backup paths with much fewer, typically one order of magnitude, backup rules compared to the traditional flow-based protection without aggregation.

Most existing survey aggregation methods assume that the sample data follow Gaussian distribution. However, these methods are sensitive to outliers, due to the thin-tailed property of the Gaussian distribution. To address this issue, we propose a robust survey aggregation method based on Student-t distribution and sparse representation. Specifically, we assume that the samples follow Student-$t$ distribution, instead of the common Gaussian distribution. Due to the Student-t distribution, our method is robust to outliers, which can be explained from both Bayesian point of view and non-Bayesian point of view. In addition, inspired by James-Stain estimator (JS) and Compressive Averaging (CAvg), we propose to sparsely represent the global mean vector by an adaptive basis comprising both data-specific basis and combined generic bases. Theoretically, we prove that JS and CAvg are special cases of our method. Extensive experiments demonstrate that our proposed method achieves significant improvement over the state-of-the-art methods on both synthetic and real datasets.

Gaussian Process Regression (GPR) is a powerful Bayesian method. However, the performance of GPR can be significantly degraded when the training data are contaminated by outliers, including target outliers and input outliers. Although there are some variants of GPR (e.g., GPR with Student-t likelihood (GPRT)) aiming to handle outliers, most of the variants focus on handling the target outliers while little effort has been done to deal with the input outliers. In contrast, in this work, we aim to handle both the target outliers and the input outliers at the same time. Specifically, we replace the Gaussian noise in GPR with independent Student-t noise to cope with the target outliers. Moreover, to enhance the robustness w.r.t. the input outliers, we use a Student-t Process prior instead of the common Gaussian Process prior, leading to Student-t Process Regression with Student-t Likelihood (TPRT). We theoretically show that TPRT is more robust to both input and target outliers than GPR and GPRT, and prove that both GPR and GPRT are special cases of TPRT. Various experiments demonstrate that TPRT outperforms GPR and its variants on both synthetic and real datasets.

In distributed storage systems, locally repairable codes (LRCs) are introduced to realize low disk I/O and repair cost. In order to tolerate multiple node failures, the LRCs with (r, δ)-localitty are further proposed. Since hot data is not uncommon in a distributed storage system, both Zeh et al. and Kadhe et al. focus on the LRCs with multiple localities or unequal localities (ML-LRCs) recently, which said that the localities among the code symbols can be different. ML-LRCs are attractive and useful in reducing repair cost for hot data. In this paper, we generalize the ML-LRCs to the (r, δ)-locality case of multiple node failures, and define an LRC with multiple (ri, δi)i ϵ [s]localities (s > 2), where r12s. Such codes ensure that some hot data could be repaired more quickly and have better failure-tolerance in certain cases because of relatively smaller riand larger δι. Then, we derive a Singleton-like upper bound on the minimum distance for the proposed LRCs by employing the regenerating-set technique. Finally, we obtain a class of explicit and structured constructions of optimal ML-LRCs, and further extend them to the cases of multiple (ri, δi)i ϵ [s]localities.

In an [n, k, d] linear code, a code symbol is said to have locality r if it can be repaired by accessing at most r other code symbols. For an (n, k, r) locally repairable code (LRC), the minimum distance satisfies the well-known Singleton-like bound d ≤ n - k - [k/r] + 2. In this paper, we study optimal ternary LRCs meeting this Singleton-like bound by employing a parity-check matrix approach. It is proved that there are only 8 classes of possible parameters with which optimal ternary LRCs exist. Moreover, we obtain explicit constructions of optimal ternary LRCs for all these 8 classes of parameters, where the minimum distance could only be 2, 3, 4, 5 and 6.

An (n, k, r) locally repairable code (LRC) is an [n, k, d] linear code where every code symbol can be repaired from at most r other code symbols. An LRC is said to be optimal if the minimum distance attains the Singleton-like bound d ≤ n - k - ⌈k/r⌉ + 2. The generalized Hamming weights (GHWs) of linear codes are fundamental parameters which have many useful applications. In this paper, we study the GHWs of LRCs. Firstly, we obtain a generalized Singleton-like bound on the i-th (1 ≤ i ≤ k) GHWs of (n, k, r) LRCs. The proposed bound can give the Singleton-like bound when i = 1 and reduce to the classical generalized Singleton bound when there is no locality constraint. Then, it is shown that for optimal (n, k, r) LRCs with r | k, the weight hierarchy can be completely determined. For optimal (n, k, r) LRCs with r | k, some lower bounds on GHWs of LRCs and their dual codes are given. Finally, two general bounds on linear codes in terms of GHWs are presented.

In this paper we study the well-known greedy coordinate descent (GCD) algorithm to solve $\ell_1$-regularized problems and improve GCD by the two popular strategies: Nesterov's acceleration and stochastic optimization. Firstly, we propose a new rule for greedy selection based on an $\ell_1$-norm square approximation which is nontrivial to solve but convex; then an efficient algorithm called ``SOft ThreshOlding PrOjection (SOTOPO)'' is proposed to exactly solve the $\ell_1$-regularized $\ell_1$-norm square approximation problem, which is induced by the new rule. Based on the new rule and the SOTOPO algorithm, the Nesterov's acceleration and stochastic optimization strategies are then successfully applied to the GCD algorithm. The resulted algorithm called accelerated stochastic greedy coordinate descent (ASGCD) has the optimal convergence rate $O(\sqrt{1/\epsilon})$; meanwhile, it reduces the iteration complexity of greedy selection up to a factor of sample size. Both theoretically and empirically, we show that ASGCD has better performance for high-dimensional and dense problems with sparse solution.

Android has become the most prevalent mobile system, but in the meanwhile malware on this platform is widespread. System call sequences are studied to detect malware. However, malware detection with these approaches relies on common system‐call‐subsequences. It is not so efficient because it is difficult to decide the appropriate length of the common subsequences. To address this issue, the authors propose a new approach, back‐propagation neural network on Markov chains from system call sequences (BMSCS). It treats one system call sequence as a homogeneous stationary Markov chain and applies back‐propagation neural network (BPNN) to detect malware by comparing transition probabilities in the chain. Since transition probabilities from one system call to another in malware are significantly different from those in benign applications, BMSCS can efficiently detect malware by capturing the anomaly in state transitions with the help of BPNN. The authors evaluate the performance of BMSCS by experiments with real application samples. The experiment results show that the F ‐score of BMSCS achieves up to 0.982773, which is higher than the other methods in the literature.

Recently, binary 0-1 measurement matrices, especially those from coding theory, were introduced to compressed sensing. Dimakis et al. found that the linear programming (LP) decoding of LDPC codes is very similar to the LP reconstruction of compressed sensing, and they further showed that the sparse binary parity-check matrices of good LDPC codes can be used as provably good measurement matrices for compressed sensing under basis pursuit (BP). Moreover, Khajehnejad et al. made use of girth to certify the good performances of sparse binary measurement matrices. In this paper, we examine the performance of binary measurement matrices with uniform column weight and arbitrary girth under BP. For a fixed measurement matrix, we first introduce a performance indicator wminBPcalled minimum BP weight, and show that any k-sparse signals could be exactly recovered by BP if and only if k ≤ (wminBP- 1)/2. Then, lower bounds of wminBPare studied. Borrowing ideas from the tree bound for the LDPC codes, we obtain several explicit lower bounds of wBPmin, which improve on the previous results in some cases. These lower bounds also imply explicit ℓ1/ℓ1, ℓ2/ℓ1and ℓ∞/ℓ1sparse approximation guarantees, and further confirm that large girth has positive impacts on the performance of binary measurement matrices under BP.

Gaussian Process Regression (GPR) is a powerful non-parametric method. However, GPR may perform poorly if the data are contaminated by outliers. To address the issue, we replace the Gaussian process with a Student-t process and introduce dependent Student-t noise in this paper, leading to a Student-t Process Regression with Dependent Student-t noise model (TPRD). Closed form expressions for the marginal likelihood and predictive distribution of TPRD are derived. Besides, TPRD gives a probabilistic interpretation to the Student-t Process Regression with the noise incorporated into its Kernel (TPRK), which is a common approach for the Student-t process regression. Moreover, we analyze the influence of different kernels. If the kernel meets a condition, called β-property here, the maximum marginal likelihood estimation of TPRD's hyperparameters is independent of the degrees of freedom ν of the Student-t process, which implies that GPR, TPRD and TPRK have exactly the same predictive mean. Empirically, the degrees of freedom ν could be regarded as a convergence accelerator, indicating that TPRD with a suitable ν performs faster than GPR. If the kernel does not have the β-property, TPRD has better performances than GPR, without additional computational cost. On benchmark datasets, the proposed results are verified.

The construction of efficient and effective decision trees remains a key topic in machine learning because of their simplicity and flexibility. A lot of heuristic algorithms have been proposed to construct near-optimal decision trees. Most of them, however, are greedy algorithms that have the drawback of obtaining only local optimums. Besides, conventional split criteria they used, e.g. Shannon entropy, Gain Ratio and Gini index, cannot select informative attributes efficiently. To address the above issues, we propose a novel Tsallis Entropy Information Metric (TEIM) algorithm with a new split criterion and a new construction method of decision trees. Firstly, the new split criterion is based on two terms of Tsallis conditional entropy, which is better than conventional split criteria. Secondly, the new construction method is based on a two-stage approach that avoids local optimum to a certain extent. The TEIM algorithm takes advantages of the generalization ability of Tsallis entropy and the low greediness property of two-stage approach. Experimental results on UCI datasets indicate that, compared with the state-of-the-art decision trees algorithms, the TEIM algorithm yields statistically significantly better decision trees in classification accuracy as well as tree complexity.

Random forests are a class of ensemble methods for classification and regression with randomizing mechanism in bagging instances and selecting feature subspace. For high dimensional data, the performance of random forests degenerates because of the random sampling feature subspace for each node in the construction of decision trees. To address the issue, in this paper, we propose a new Principal Component Analysis and Stratified Sampling based method, called PCA-SS, for feature subspace selection in random forests with high dimensional data. For each decision tree in the forests, we firstly create the training data by bagging instances and partition the feature set into several feature subsets. Principal Component Analysis (PCA) is applied on each feature subset to obtain transformed features. All the principal components are retained in order to preserve the variability information of the data. Secondly, depending on a certain principal components principle, the transformed features are partitioned into informative and less informative parts. When constructing each node of decision trees, a feature subspace is selected by stratified sampling method from the two parts. The PCA-SS based Random Forests algorithm, named PSRF, ensures enough informative features for each tree node, and it also increases the diversity between the trees to a certain extent. Experimental results demonstrate that the proposed PSRF significantly improves the performance of random forests when dealing with high dimensional data, compared with the state-of-the-art random forests algorithms.

In a linear code, a code symbol is said to have locality r if it can be repaired by accessing at most r other code symbols. For an (n, k, r) locally repairable codes (LRC), the most important bounds on minimum distances might be the well-known Singleton-like bound and the Cadambe-Mazumdar bound which takes the field size into account. In this paper, we study the constructions of optimal LRCs from the view of parity-check matrices. Firstly, all the optimal binary LRCs meeting the Singleton-like bound are found in the sense of equivalence of linear codes, i.e., except the proposed 4 classes of LRCs, there is no other binary (n, k, r) LRC with minimum distance d = n - k - ⌈k/r⌉+2. Then a class of binary LRCs with distance 4 and arbitrary locality is proposed and shown to be optimal with respect to the Cadambe-Mazumdar bound. Moreover, we give a class of high rate optimal q-ary LRCs meeting the Singleton-like bound with minimum distance 4 while the required field size is only q ≥ r - 1. Finally, several methods to obtain short optimal LRCs from long optimal LRCs are proposed at the end of this paper.

Recently, codes with locality have been widely studied to deal with the node repair problem in distributed storage systems. Locally repairable codes are linear codes with locality properties for code symbols. If a code symbol can be repaired respectively by t disjoint groups of other symbols, each of which has size at most r, this code symbol is said to have (r, t)-locality. In this paper, we present recursive bounds for LRCs with (r, t)-locality for all code symbols. The recursive bounds have simple forms and can be used to derive various bounds for LRCs. Moreover, it is shown that many previous well known bounds of LRCs can be derived by using our recursive bounds. Besides the recursive bounds, we also propose a linear programming bound for LRCs with (r, t)-locality for all code symbols.

We introduce a general framework to deterministically construct binary measurement matrices for compressed sensing. The proposed matrices are composed of (circulant) permutation submatrix blocks and zero submatrix blocks, thus making their hardware realization convenient and easy. Firstly, using the famous Johnson bound for binary constant weight codes, we derive a new lower bound for the coherence of binary matrices with uniform column weights. Afterwards, a large class of binary base matrices with coherence asymptotically achieving this new bound are presented. Finally, by choosing proper rows and columns from these base matrices, we construct the desired measurement matrices with various sizes and they show empirically comparable performance to that of the corresponding Gaussian matrices.

Under the current Internet environment, middlebox management has become a significant challenge for network operators. Schemes in prior works tried to simplify it with Software-Defined Networking (SDN) technologies, and they provided reliable and flexible approaches to configure the middlebox-related flow entries. However, these schemes are inefficient in resource utilization with the dynamically changing traffic requirements, as they mainly focus on stationary hardware middleboxes. Further-more, latencies of packets lack controls under these schemes. In this paper, inspired by Network Function Virtualization (NFV), we employ software middleboxes, and build a processing delay model to formulate latency behaviours. And based on this model, we present a latency-aware NFV/SDN scheme called Quokka with portable software-based middleboxes that can be dynamically scheduled (placed) according to the changing traffic. Quokka controls the number of middleboxes by efficient and automatic scheduling of both traffic and middlebox positions, and thus reduces the transmission latencies of the network. Comprehensive experiments show: 1) compared with traditional configuration methods, Quokka reduces the transmission delay by about 20% on average; 2) Quokka requires 30% to 50% less middleboxes than traditional schemes to achieve the same performance.

Patch-based algorithms for image denoising have been widely used in recent years. Most of patch-based methods just exploit patch redundancy in spatial or frequency domain without considering inter-scale dependencies. In this paper, we propose a novel patch-based multiscale products algorithm (PMPA) for image denoising. It is based on patch similarity in spatial domain and multiscale products in wavelet domain. PMPA is divided into two stages to process the smooth areas and non smooth areas (such as edges) individually. The first stage is in the wavelet domain, then a locally adaptive window-based denoising method (LAWML) based on multiscale products is applied to process those wavelet coefficients corresponding to the non smooth areas, then obtain one initial denoised image. The second stage is in the spatial domain, then a non local means algorithm is used to process those pixels in the smooth areas to obtain another initial denoised image. The final denoised image is obtained by a weighted averaging of all common pixels in both initial denoised images. Experiments show that the proposed algorithm can have competitive performance compared with the state-of-the-art patch-based denoising algorithms for most of images.

The current Internet is facing a lot of challenges, including the transmission inefficiency due to the popularity of content-delivery applications. Content-Centric Networking (CCN) solves this problem by name-oriented routing and in-network caching. As a core component of the CCN router, the Pending Interest Table (PIT) is crucial for the forwarding performance. In this paper, we propose the RTT-Aware PIT (RAPIT) to improve the PIT efficiency in CCN by reducing the residence time of non-responded PIT entries. First, we provide an approach to measure the Round-Trip Time (RTT) from a CCN router to the content publisher and set PIT entry residence time dynamically based on the measured RTT. Second, we prioritize the PIT entries based on their aggregated request amounts and the possibility of being responded. When the PIT is exhausted, the lowest-priority entry will be replaced by the latest one. The evaluation results show that: 1) compared with current PIT with scheme in CCNx, RAPIT reduces the PIT size to 57.6% and improve the network throughput by 200%; 2) when replacement is triggered as the PIT is full, RAPIT is more accurate and improves the PIT efficiency by 20–50% compared with FIFO.

Finite fields were successfully used to construct algebraic low-density parity-check (LDPC) codes, especially Quasi-Cyclic LDPC codes. These LDPC codes with large minimum distances have lower error floor, linear complexity of encoding and are more practical for hard-decision algebraic decoding. In this paper, we show that finite fields can also be successfully used to construct algebraic low-density lattice codes (LDLC), denoted by structured LDLC. A general framework to construct algebraic LDLC is presented. LDLC constructed by this general framework have comparable performance to the corresponding random codes over addition white Gaussian noise (AWGN) channel with iterative soft-decision decoding in terms of symbol-error probability. Furthermore, the general framework is extended to complex low-density lattice codes (CLDLC) and results in algebraic CLDLC which perform very well for small dimensions.

We propose a new iterative greedy algorithm to reconstruct sparse signals in compressed sensing. By a simple combination of subspace pursuit and iterative hard thresholding, the proposed algorithm, called subspace thresholding pursuit (STP) shows well improved empirical performance, while still keeps a strong theoretical guarantee in terms of restricted isometry property.

The network coding system COPE [1] shows us the improvements obtained by COPE-type network coding in wireless networks. In this paper, by introducing 802.11 interference model, we analyze the maximal multi-commodity flow problem in wireless network coding and put forward a new scheduling mechanism 802.11st which has higher throughput for multihop wireless networks with network coding. We use linear programming to compute the maximum multi-commodity flow of multiple unicast flows in which the constraints are rebuilt according to the new scheduling mechanism.

In the context of compressed sensing (CS), subspace pursuit (SP) is an important iterative greedy recovery algorithm which could reduce the recovery complexity greatly comparing with${\ell _1}$-minimization. Restricted isometry property (RIP) and restricted isometry constant (RIC) of measurement matrices which ensure the convergence of iterative algorithms play key roles for the guarantee of successful reconstructions. In this letter, we show that for the$s$-sparse recovery, the RIC is enlarged to${\delta _{3s}} < 0.4859$for SP, which improves the known results significantly. The proposed result also applies to almost sparse signals and corrupted measurements.

In this paper, we address a super-resolution (SR) problem that constructs a high-resolution (HR) frame/image from a short sequence of low-resolution (LR) frames/images. It is well known that SR is a difficult problem, especially when the number of LR inputs is small. In particular, our previous work involving multisurface fitting-based SR exhibits relatively poor performance in the above case. To cope with this problem, we take advantage of nonlocal pixels to fit local surfaces. The pixels from nonlocal spatial-temporal positions are selected and weighted based on patch similarity and outlier removal. With this method, the fitted surfaces become more elaborate so that more details can be retrieved in SR results. Experiments demonstrate that the proposed method is very effective in producing HR frames through a small number of LR inputs when compared with some state-of-the-art methods.

Current P2P applications are based on random connected overlays, which lead to generating a significant amount of inter-ISP traffic. Asking for more QoS requirements, such as a short delay and a stable streaming rate, few studies are dedicated to optimizing P2P live streaming applications, despite that recent work have proposed some solutions for P2P file distribution applications. In this paper, current traffic localization strategy is analyzed at first, and its two inherent flaws are revealed. Then we propose a novel strategy for ISP-friendly live streaming: based on a hybrid overlay, which is a two-tier structure, all ISPs are organized into an ISP-tree, and then local peers in each ISP form a mesh overlay. In each tier, a data scheduling is designed for inter-ISP traffic reduction and performance guarantee respectively. Compared with a famous live streaming strategy, R2, simulation results demonstrate that our strategy generates much less inter-ISP traffic and achieves a higher system performance.

In this paper, we address the problem of content replication in segmented peer-to-peer on-demand systems, with the objective of minimizing the content server's workload. We consider the system performance under heterogeneous environment. In this P2P-VoD system, multimedia content is divided into segments and peers can seek and cache any segments. Because different segments may be of different popularity, badly designed segment replication may lead to great server's workload. We deduce the “optimal replication ratio” in segmented P2P-VoD system such that peers will receive upload bandwidth from each other and at the same time, minimize the server's workload. We formulate the segment replication as an optimization problem and propose a model to solve it. We show that the proportional replication strategy is not optimal for segmented P2P-VoD systems and the segmented system can lead to less server's workload than non-segmented. We simulate our model, evaluate the performance of segmented P2P-VoD systems and show that our algorithm can greatly reduce the server's workload.

On a binary erasure channel (BEC) with erasing probability e, the performance of a binary linear code is determined by the incorrigible sets of the code. The incorrigible set distribution (ISD) {Ii}i=0nenumerates the number of incorrigible sets with size i of the code. The probability of unsuccessful decoding under optimal decoding for the code could be formulated by the ISD and ∈. In this paper, we determine the ISDs for the Simplex codes, the Hamming codes, the first order Reed-Muller codes, and the extended Hamming codes, which are some Reed-Muller codes or their shortening or puncturing versions. Then, we show that the probability of unsuccessful decoding under optimal decoding for any binary linear code is monotonously non-decreasing on ∈ in the interval [0,1].

Binary 0-1 measurement matrices, especially those from coding theory, were introduced to compressed sensing (CS) recently. Good measurement matrices with preferred properties, e.g., the restricted isometry property (RIP) and nullspace property (NSP), have no known general ways to be efficiently checked. Khajehnejad et al. made use of girth to certify the good performances of sparse binary measurement matrices. In this paper, we examine the performance of binary measurement matrices with uniform column weight and arbitrary girth under basis pursuit. Explicit sufficient conditions of exact reconstruction are obtained, which improve the previous results derived from RIP for any girth g and results from NSP when g/2 is odd. Moreover, we derive explicit l1/l1, l2/l1and l∞/l1sparse approximation guarantees. These results further show that large girth has positive impacts on the performance of binary measurement matrices under basis pursuit, and the binary parity-check matrices of good LDPC codes are important candidates of measurement matrices.

Recently, Dimakis, Smarandache, and Vontobel indicated that the parity-check matrices of good LDPC codes can be used as provably good measurement matrices for compressed sensing (CS) under basis pursuit (BP). In this paper, we consider the parity-check matrix H(r, q) of the array codes, one of the most important kind of structured LDPC codes. The spark, i.e. the smallest number of linearly dependent columns in a matrix, of H(2, q) and H(3, q) are determined and two lower bounds of the sparks of H(r, q) are given for r ≥ 4. Moreover, we carry out numbers of simulations and show that many parity-check matrices of array codes and their submatrices perform better than the corresponding Gaussian random matrices. The proposed measurement matrices have perfect quasi-cyclic structures and can make the hardware realization convenient and easy, thus having great potentials in practice.

In this paper, we study the maximum multicast set scheduling problem in a plane wireless network under the physical interference model. Specifically, from a given set of multicast transmissions, select a maximum set of multicast transmissions with largest sum of receivers, where each receiver of a multicast transmission is interference-free from other transmissions subject to the physical interference constraint. This problem is known to be NP-hard and in this paper we develop an approximation algorithm with constant approximation bound guarantee. To the best of our knowledge, this is the first multicast scheduling algorithm under the physical interference model. For ease of implementation in distributed settings, we focus on the problem in an oblivious power assignment setting.

In this paper, we propose a keyword proximity search approach that can be applied to large and complex RDF database. We model RDF database as undirected data graph, construct three indexes for each data graph, only one index need be loaded into memory. Keyword graph is defined as search result, keyword tree and minimal keyword tree are proposed as middle structures for Keyword graph extraction, and we present a link join operation based algorithm to retrieve Keyword trees in this paper. We employ a technique of keyword node pruning to accelerate keyword tree retrieval and define a scoring function to rank search results. In experiments, our approach achieves both high efficiency and high accuracy, outperforms the existing approaches.

Stopping sets and stopping set distribution of a linear code are used to determine the performance of this code under iterative decoding over a binary erasure channel (BEC). LetCbe a binary [n,k] linear code with parity-check matrixH, where the rows ofHmay be dependent. A stopping setSofCwith parity-check matrixHis a subset of column indices ofHsuch that the restriction ofHtoSdoes not contain a row of weight one. The stopping set distribution {Ti(H)}i=0nenumerates the number of stopping sets with sizeiofCwith parity-check matrixH. Note that stopping sets and stopping set distribution are related to the parity-check matrixHofC. LetH*be the parity-check matrix ofCwhich is formed by all the nonzero codewords of its dual codeC⊥. A parity-check matrixHis called BEC-optimal ifTi(H)=Ti(H*),i=0,1,...,nandHhas the smallest number of rows. In this paper, we study stopping sets, stopping set distributions and BEC-optimal parity-check matrices of binary linear codes. Using finite geometry in combinatorics, we obtain BEC-optimal parity-check matrices and then determine the stopping set distributions for the Simplex codes, the Hamming codes, the first order Reed-Muller codes, and the extended Hamming codes, which are some Reed-Muller codes or their shortening or puncturing versions.

In this correspondence, we study the minimum pseudoweight and minimum pseudocodewords of low-density parity-check (LDPC) codes under linear programming (LP) decoding. First, we show that the lower bound of Kelley, Sridhara, Xu, and Rosenthal on the pseudoweight of a nonzero pseudocodeword of an LDPC code whose Tanner graph has girth greater than is tight if and only if this pseudocodeword is a real multiple of a codeword. Then, the lower bound of Kashyap and Vardy on the stopping distance of an LDPC code is proved to be also a lower bound on the pseudoweight of a nonzero pseudocodeword of an LDPC code whose Tanner graph has girth , and this lower bound is tight if and only if this pseudocodeword is a real multiple of a codeword. Using these results we further obtain that for some LDPC codes, there are no other minimum pseudocodewords except the real multiples of minimum weight codewords. This means that the LP decoding for these LDPC codes is asymptotically optimal in the sense that the ratio of the probabilities of decoding errors of LP decoding and maximum-likelihood decoding approaches as the signal-to-noise ratio (SNR) tends to infinity. Finally, some LDPC codes are listed to illustrate these results.

In this paper, we study the probability of undetected error for binary constant weight codes (BCWCs). First, we derive a new lower bound on the probability of undetected error. Next, we show that this bound is tight if and only if the BCWCs are generated from certain t-designs. This means that such BCWCs are uniformly optimal for error detection. Thus, we prove a conjecture of Xia, Fu, Jiang and Ling. Furthermore, we determine the distance distributions of such BCWCs. Finally, we derive some bounds on the exponent of the probability of undetected error for BCWCs. These bounds enable us to extend the region in which the exponent of the probability of undetected error is exactly determined

In this correspondence, we study the probability of undetected error for binary constant weight codes. First, we derive a new lower bound on the probability of undetected error for binary constant weight codes. Next, we show that this bound is tight if and only if the binary constant weight codes are generated from certain t-designs in combinatorial design theory. This means that these binary constant weight codes generated from certain t-designs are uniformly optimal for error detection. Along the way, we determine the distance distributions of such binary constant weight codes. In particular, it is shown that binary constant weight codes generated from Steiner systems are uniformly optimal for error detection. Thus, we prove a conjecture of Xia, Fu, Jiang, and Ling. Furthermore, the distance distribution of a binary constant weight code generated from a Steiner system is determined. Finally, we study the exponent of the probability of undetected error for binary constant weight codes. We derive some bounds on the exponent of the probability of undetected error for binary constant weight codes. These bounds enable us to extend the region in which the exponent of the probability of undetected error is exactly determined

In this correspondence, we study the probability of undetected error for binary constant-weight codes. First, we derive a new formula on the probability of undetected error for binary constant-weight codes. Second, using this new formula and linear programming, we give two new lower bounds on the probability of undetected error for binary constant-weight codes. These two new lower bounds improve on previously known lower bounds in certain cases. Furthermore, we show that these two lower bounds are tight if and only if the binary constant-weight codes are generated from certain t-designs in combinatorial design theory. This means that these binary constant-weight codes generated from certain t-designs are uniformly optimal for error detection. Along the way, we determine the distance distributions of such binary constant-weight codes. Finally, several examples are given to illustrate the results obtained in this correspondence.

The well-known m-out-of-n code /spl Omega//sub n//sup m/ consists of all binary vectors of length n and weight m. It is known that it is good for error detection (in the technical sense, that is, the probability of undetected error P/sub ud/(/spl Omega//sub n//sup m/,p)/spl les/P/sub ud/(/spl Omega//sub n//sup m/,1/2) for all p, 0/spl les/p/spl les/1/2) only for a few small values of m and n. It is therefore of interest to determine (bounds for) the threshold in general, that is, find the range of bit-error probabilities p for which P/sub ud/ (/spl Omega//sub n//sup m/,p)/spl les/P/sub ud/ (/spl Omega//sub n//sup m/,1/2). In this article such bounds are given.

The undetected error probability of binary constant weight codes on a binary symmetric channel is studied in this correspondence. First, we present a necessary condition for the binary constant weight codes to be proper for error detection. Then, by using the necessary condition, we study the error detection capability of binary optimum constant weight codes.