Machine unlearning for large language models (LLMs) aims to remove targeted knowledge while preserving general capability.In this paper, we recast LLM unlearning as an asymmetric two-task problem: retention is the primary objective and forgetting is an auxiliary.From this perspective, we propose a retention-prioritized gradient synthesis framework that decouples task-specific gradient extraction from conflict-aware combination.Instantiating the framework, we adapt established PCGrad to resolve gradient conflicts, and introduce SAGO, a novel retention-prioritized gradient synthesis method.Theoretically, both variants ensure non-negative cosine similarity with the retain gradient, while SAGO achieves strictly tighter alignment through constructive sign-constrained synthesis.Empirically, on WMDP Bio/Cyber and RWKU benchmarks, SAGO consistently pushes the Pareto frontier: e.g., on WMDP Bio (SimNPO+GD), recovery of target model MMLU performance progresses from 44.6% (naive) to 94.0% (+PC-Grad) and further to 96.0% (+SAGO), while maintaining comparable forgetting strength.Our results show that re-shaping gradient geometry, rather than re-balancing losses, is the key to mitigating unlearning-retention trade-offs.

Facial recognition is an important user authentication function in many systems. Traditional vision-based approaches are vulnerable to spoofing attacks and are affected by lighting conditions. In this work, we propose a human facial recognition system using a single millimeter-wave (mmWave) radio. Our system actively transmits the mmWave signal toward the user's face and receives the echoes that contain the geometric face information. We design a novel signal processing pipeline to defend against spoofing attacks and extract facial features. The extracted features are fed into a CNN-based classifier to recognize user identities. We implement and evaluate our system on a tiny and low-cost mmWave radar (25$). The experimental results show that the system can detect the spoofing samples with an average accuracy of 99.12% and achieves an average recognition accuracy of 97.5% over 21 volunteers in different environments.

Due to the epidemic COVID-19 spread and Intelligent Transportation System (ITS) development, investigators are now to conduct their research over the generated Transportation Big Data (TBD) in many critical areas, such as medical supplies, food supplies, as well as logistics supplies. At present, Vehicular Edge Computing (VEC) is an emerging paradigm to integrate resources from vehicles, road-site units, base stations, and cloud center to promote the performance of TBD tasks scheduling and running. In this paper, we design a three-layered TBD task processing architecture with a federated learning mechanism for credit priority-based task scheduling and running. In our design, we consider the efficiency of task offloading and misbehavior attack problems simultaneously. We propose a vehicular federated learning framework combined with Multi-Layer Perceptron (MLP) credit measurement, which can preserve the privacy of vehicles and obtain the related features for vehicular credit prediction. We also propose a task offloading algorithm to solve the optimization problem for credit priority task offloading between edge computing servers and vehicles. The proposed solution can prioritize tasks and assign sufficient resources for reliable and active task requesters. Experimental results expose that the proposed mechanism outperforms the state-of-the-art solutions when considering efficiency and attack simultaneously for TBD tasks scheduling and running.

Gait recognition is widely used in diversified practical applications. Currently, the most prevalent approach is to recognize human gait from RGB images, owing to the progress of computer vision technologies. Nevertheless, the perception capability of RGB cameras deteriorates in rough circumstances, and visual surveillance may cause privacy invasion. Due to the robustness and non-invasive feature of millimeter wave (mmWave) radar, radar-based gait recognition has attracted increasing attention in recent years. In this research, we propose a Hierarchical Dynamic Network (HDNet) for gait recognition using mmWave radar. In order to explore more dynamic information, we propose point flow as a novel point clouds descriptor. We also devise a dynamic frame sampling module to promote the efficiency of computation without deteriorating performance noticeably. To prove the superiority of our methods, we perform extensive experiments on two public mmWave radar-based gait recognition datasets, and the results demonstrate that our model is superior to existing state-of-the-art methods.

Based on particle swarm optimization (PSO), a new exponential time delay fraction order grey prediction model is proposed in this paper. Firstly, the original data is preprocessed by fractional-order accumulation; on the basis of fractional-order accumulation, it is proved that the initial value of the original sequence satisfies the fixed point theorem. On the basis of GM(1,1) model, a new model is established by adding exponential time delay term. The model is discretized by integral, the least square estimation of the linear parameters and the approximate time response equation are obtained. Finally, PSO is used to search the optimal parameters of the model and the experimental results are verified by Wilcoxon rank sum test. In order to test the good adaptability and strong prediction ability of the new model, two groups of data are verified for the production and consumption of oil and electricity in China. The results show that the new model has better prediction accuracy and adaptability than the other existing six grey models.

Pre-training language models have achieved thriving success in numerous natural language understanding and autoregressive generation tasks, but non-autoregressive generation in applications such as machine translation has not sufficiently benefited from the pre-training paradigm.In this work, we establish the connection between a pre-trained masked language model (MLM) and non-autoregressive generation on machine translation.From this perspective, we present XLM-D, which seamlessly transforms an off-the-shelf cross-lingual pre-training model into a non-autoregressive translation (NAT) model with a lightweight yet effective decorator.Specifically, the decorator ensures the representation consistency of the pre-trained model and brings only one additional trainable parameter.Extensive experiments on typical translation datasets show that our models obtain state-of-the-art performance while realizing the inference speedup by 19.9×.One striking result is that on WMT14 En⇒De, our XLM-D obtains 29.80 BLEU points with multiple iterations, which outperforms the previous mask-predict model by 2.77 points.

Many applications of complex event processing (CEP) in Cloud can tolerate analytical errors to some extent, and it provides us an opportunity to optimize real-time analytics using methods of approximate query processing over big data streams. In this article, we present a novel rules-based sampling technique, which supports to construct sketch over one-pass and high-speed asynchronous data streams and provides accurate answers for different types of analytical queries. Moreover, we propose two methods of distributed sketching implementation, i.e., D-AQP$_b$and D-AQP$_i$, to make our approach to be compatible with batch processing and interactive processing architectures respectively, and be appropriate for stream processing systems in Cloud. Experimental results with real-world and synthetic datasets indicate that our approach can obtain more accurate estimates and improve two times of system throughput when compared with state-of-the-art Hadoop-based approximate engine BlinkDB. When compared with current batch processing systems Spark and stream processing system Spark-Streaming, our methods of D-AQP$_b$and D-AQP$_i$can achieve 2 and 4 orders of magnitude improvement on query response time respectively.

Thermal sensors play an important role in intelligent transportation system. This paper studies the problem of RGB and thermal (RGBT) tracking in challenging situations by leveraging multimodal data. A RGBT object tracking method is proposed in correlation filter tracking framework based on short term historical information. Given the initial object bounding box, hierarchical convolutional neural network (CNN) is employed to extract features. The target is tracked for RGB and thermal modalities separately. Then the backward tracking is implemented in the two modalities. The difference between each pair is computed, which is an indicator of the tracking quality in each modality. Considering the temporal continuity of sequence frames, we also incorporate the history data into the weights computation to achieve a robust fusion of different source data. Experiments on three RGBT datasets show the proposed method achieves comparable results to state-of-the-art methods.

The key challenge of multi-domain translation lies in simultaneously encoding both the general knowledge shared across domains and the particular knowledge distinctive to each domain in a unified model. Previous work shows that the standard neural machine translation (NMT) model, trained on mixed-domain data, generally captures the general knowledge, but misses the domain-specific knowledge. In response to this problem, we augment NMT model with additional domain transformation networks to transform the general representations to domain-specific representations, which are subsequently fed to the NMT decoder. To guarantee the knowledge transformation, we also propose two complementary supervision signals by leveraging the power of knowledge distillation and adversarial learning. Experimental results on several language pairs, covering both balanced and unbalanced multi-domain translation, demonstrate the effectiveness and universality of the proposed approach. Encouragingly, the proposed unified model achieves comparable results with the fine-tuning approach that requires multiple models to preserve the particular knowledge. Further analyses reveal that the domain transformation networks successfully capture the domain-specific knowledge as expected.1

Modern neural machine translation (NMT) models employ a large number of parameters, which leads to serious over-parameterization and typically causes the underutilization of computational resources.In response to this problem, we empirically investigate whether the redundant parameters can be reused to achieve better performance.Experiments and analyses are systematically conducted on different datasets and NMT architectures.We show that: 1) the pruned parameters can be rejuvenated to improve the baseline model by up to +0.8 BLEU points; 2) the rejuvenated parameters are reallocated to enhance the ability of modeling low-level lexical information.

To prevent serious privacy leakage from website fingerprinting (WF) attacks, many traditional or adversarial WF defenses have been released. However, traditional WF defenses such as Walkie-Talkie (W-T) still generate patterns that might be captured by the deep learning (DL) based WF attacks, which are not effective. Adversarial perturbation based WF defenses better confuse WF attacks, but their requirements for the entire original traffic trace and perturbating any points including historical packets or cells of the network traffic are not practical. To deal with the effectiveness and practicality issues of existing defenses, we proposed a novel WF defense in this paper, called PST. Given a few past bursts of a trace as input, PST Predicts subsequent fuzzy bursts with a neural network, then Searches small but effective adversarial perturbation directions based on observed and predicted bursts, and finally Transfers the perturbation directions to the remaining bursts. Our experimental results over a public closed-world dataset demonstrate that PST can successfully break the network traffic pattern and achieve a high evasion rate of 87.6%, beating W-T by more than 31.59% at the same bandwidth overhead, with only observing 10 transferred bursts. Moreover, our defense adapts to WF attacks dynamically, which could be retrained or updated.

Leveraging lexical constraint is extremely significant in domain-specific machine translation and interactive machine translation. Previous studies mainly focus on extending beam search algorithm or augmenting the training corpus by replacing source phrases with the corresponding target translation. These methods either suffer from the heavy computation cost during inference or depend on the quality of the bilingual dictionary pre-specified by user or constructed with statistical machine translation. In response to these problems, we present a conceptually simple and empirically effective data augmentation approach in lexical constrained neural machine translation. Specifically, we make constraint-aware training data by first randomly sampling the phrases of the reference as constraints, and then packing them together into the source sentence with a separation symbol. Extensive experiments on several language pairs demonstrate that our approach achieves superior translation results over the existing systems, improving translation of constrained sentences without hurting the unconstrained ones.

Keeping the safe distance from the leading vehicle is crucial for transportation companies with a fleet of old cars. While modern Advanced Driver Assistant Systems (ADAS) might be able to estimate the distance from the front-leading vehicle, traditional ADAS do not usually offer this feature. An alternative solution is to monitor the distance using smartphones that are attached to a place such as a sun visor. The basic idea behind this approach is to detect the front-leading vehicle using the smartphone camera and estimate its distance from the car. Although SSD can achieve real-time performance on powerful GPUs, it remains challenging to run this model in real-time on mobile devices. In this paper, we propose a monocular distance estimator for forward-leading vehicles using a smartphone which is faster and more accurate than the state-of-the-art SSD detector. Specifically, we propose a layer-wise method to generate more efficient default boxes for the SSD and develop a lightweight method for estimating the distance accurately. Our experiments show that the proposed method reduces the number of default boxes by an average of 38.4% while it improves the detection rate and the processing speed compared to the original SSD. Moreover, our monocular distance estimator provides a proper safety buffer zone when the distance is greater than 20 meters. A sample video is available at https://youtu.be/-ptvfabBZWA.

Zero-shot translation, translating between language pairs on which a Neural Machine Translation (NMT) system has never been trained, is an emergent property when training the system in multilingual settings.However, naïve training for zero-shot NMT easily fails, and is sensitive to hyper-parameter setting.The performance typically lags far behind the more conventional pivot-based approach which translates twice using a third language as a pivot.In this work, we address the degeneracy problem due to capturing spurious correlations by quantitatively analyzing the mutual information between language IDs of the source and decoded sentences.Inspired by this analysis, we propose to use two simple but effective approaches: (1) decoder pre-training; (2) backtranslation.These methods show significant improvement (4 ∼ 22 BLEU points) over the vanilla zero-shot translation on three challenging multilingual datasets, and achieve similar or better results than the pivot-based approach.

In this paper, we extend an attention-based neural machine translation (NMT) model by allowing it to access an entire training set of parallel sentence pairs even after training. The proposed approach consists of two stages. In the first stage –retrieval stage–, an off-the-shelf, black-box search engine is used to retrieve a small subset of sentence pairs from a training set given a source sentence. These pairs are further filtered based on a fuzzy matching score based on edit distance. In the second stage–translation stage–, a novel translation model, called search engine guided NMT (SEG-NMT), seamlessly uses both the source sentence and a set of retrieved sentence pairs to perform the translation. Empirical evaluation on three language pairs (En-Fr, En-De, and En-Es) shows that the proposed approach significantly outperforms the baseline approach and the improvement is more significant when more relevant sentence pairs were retrieved.

In this paper, we propose to extend the recently introduced model-agnostic meta-learning algorithm (MAML, Finn et al., 2017) for lowresource neural machine translation (NMT).We frame low-resource translation as a metalearning problem, and we learn to adapt to low-resource languages based on multilingual high-resource language tasks.We use the universal lexical representation (Gu et al., 2018b) to overcome the input-output mismatch across different languages.We evaluate the proposed meta-learning strategy using eighteen European languages (Bg, Cs, Da, De, El, Es, Et, Fr, Hu, It, Lt, Nl, Pl, Pt, Sk, Sl, Sv and Ru) as source tasks and five diverse languages (Ro, Lv, Fi, Tr and Ko) as target tasks.We show that the proposed approach significantly outperforms the multilingual, transfer learning based approach (Zoph et al., 2016) and enables us to train a competitive NMT system with only a fraction of training examples.For instance, the proposed approach can achieve as high as 22.04 BLEU on Romanian-English WMT'16 by seeing only 16,000 translated words (⇠ 600 parallel sentences).

Kernelised correlation filter (KCF)‐based trackers have recently attracted considerable attention due to their exciting accuracy and efficiency. Numerous improvements have been made later for coping with scales variation or partial occlusion etc . However, when there is an abrupt motion between the consecutive image frames, these trackers would face failure. To alleviate the problem, the authors present an extended cuckoo search (CS)‐based KCF tracker (called ECSKCF). At first, the extended CS algorithm is constructed by the Simplex method (SM). CS has obvious capability in global search while the SM has exceptional advantage in local search. Based on ECS method, motion prediction is transformed to globally search for optimal position intending to enhance the quality of base image. Then, combined ECS with Gaussian distribution, a hybrid motion model is introduced to KCF framework, which has the capability of capturing abrupt motion. Finally, a unified framework is designed to track smooth or abrupt motion simultaneously. Extensive experimental results in both quantitative and qualitative measures demonstrate the effectiveness of the authors’ proposed method for abrupt motion tracking.

This paper presents a robust object tracking algorithm using a collaborative model. Under the framework of particle filtering, we develop a multi-task learning based generative and discriminative classifier model. In the generative model, we propose a histogram-based subspace learning method that takes advantage of adaptive template update. In the discriminative model, we introduce an effective method to compute the confidence value that assigns more weights to the foreground than the background. A decomposition model is employed to take the outliers of each particle into consideration. The alternating direction method of multipliers (ADMM) algorithm guarantees the optimization problem can be solved robustly and accurately. Qualitative and quantitative comparison with ten state-of-the-art methods demonstrates the effectiveness and efficiency of our method in handling various challenges during tracking.

In this paper, we formulate particle filter based tracking as a multi-task sparse learning problem that exploits context information. The target and context information which modeled as linear combinations of principal component analysis (PCA) basis is formed as dictionary templates. We treat the dictionary templates as the guidance and the incoming candidates are filtered depending on the similarity between the guidance image and each input. The guided filter can help to distinguish the target from numerous candidates via context information. Then multi-task sparse learning is employed to learn the target and context information. The proposed learning problem is efficiently solved using an alternating direction method of multipliers (ADMM) method that yield a sequence of closed form updates. We test our tracker on challenging benchmark sequences that involve drastic illumination changes, large pose variations, and heavy occlusion. Experimental results show that our tracker consistently outperforms state-of-the-art trackers.

Data blocking has been an effective technique of data skipping to reduce data access and shorten query response time in query engines. By generating fine-grained, balanced blocks and corresponding metadata, a query may skip a block if the metadata indicates that the block does not contain relevant data. Obviously, the deciding factor of a promising blocking strategy depends on how to produce effective data layout in reasonable time that is expected to skip most data. In this paper, we propose several algorithms that drastically reduce the time complexity of existent blocking strategies based on workload analysis, at the cost of relatively small loss of estimated tuples could be skipped. Via theoretical analysis, we prove that the time complexity of our algorithms is apparently lower than that of ward algorithm. Afterwards, we demonstrate the whole blocking and skipping workflow, install it into Spark SQL and obtain experimental evaluation results. Experimental results show that our technique gains significant improvement in aspect of blocking efficiency compared to ward algorithm, while keeping almost the same level of skipping ability.

We propose an online tracking algorithm in which the object tracking is achieved by using subspace learning and non-negative matrix factorization (NMF) under the partile filtering framework. The object appearance is modeled by a non-negative combination of non-negative components learned from examples observed in previous frames. In order to robust tracking an object, group sparsity constraints are included to the non-negativity one. In addition, the Alternating Direction Method of Multipliers (ADMM) algorithm is proposed for efficient model updating. Qualitative and quantitative experiments on a variety of challenging sequences show favorable performance of the proposed algorithm against 9 state-of-the-art methods.

In this paper, we cast tracking as a novel multi-task learning problem and exploit various types of visual features. We use an on-line feature selection mechanism based on the two-class variance ratio measure, applied to log likelihood distributions computed with respect to a given feature from samples of object and background pixels. The proposed method is integrated in a particle filtering framework. We jointly consider the underlying relationship across different particles, and tackle it in a unified robust multi-task formulation. We show that the proposed formulation can be efficiently solved using the Alternating Direction Method of Multipliers (ADMM) with a small number of closed-form updates. Both the qualitative and quantitative results demonstrate the superior performance of the proposed approach compared to several state of-the-art trackers.

In multi-view learning applications, like multimedia analysis and information retrieval, we often encounter the corrupted view problem in which the data are corrupted by two different types of noises, i.e., the intra- and inter-view noises. The noises may affect these applications that commonly acquire complementary representations from different views. Therefore, how to denoise corrupted views from multi-view data is of great importance for applications that integrate and analyze representations from different views. However, the heterogeneity among multi-view representations brings a significant challenge on denoising corrupted views. To address this challenge, we propose a general framework to jointly denoise corrupted views in this paper. Specifically, aiming at capturing the semantic complementarity and distributional similarity among different views, a novel Heterogeneous Linear Metric Learning (HLML) model with low-rank regularization, leave-one-out validation, and pseudo-metric constraints is proposed. Our method linearly maps multi-view data to a high-dimensional feature-homogeneous space that embeds the complementary information from different views. Furthermore, to remove the intra- and inter-view noises, we present a new Multi-view Semi-supervised Collaborative Denoising (MSCD) method with elementary transformation constraints and gradient energy competition to establish the complementary relationship among the heterogeneous representations. Experimental results demonstrate that our proposed methods are effective and efficient.

Network traffic classification is of great importance for fine-grained network management and network security. However, with the rapid development of new network applications in recent years, traffic that cannot be identified by classifiers accounts for an increasing ratio, which brings a great challenge for network operators. Most of the unknown traffic is usually generated by only a few or some certain kinds of applications. We call this kind of traffic as the elephant traffic. It is generally recognized that traffic sharing the same server IP and server port is generated by the same application. In this paper, we say that they are belonging to the same service. Therefore, we propose a novel method, in which service-based statistical features are used for cluster analysis, to classify these elephant traffic. Preliminary results on a real network traffic dataset show that our method is able to automatically identify similar unknown applications. We believe that classifying unknown traffic in service perspective is a promising direction.

Classification of network vulnerability is critical to detection and risk analysis of network vulnerability. A broad range of classification methods have been proposed in literature. This paper reviews a total of 25 selected approaches and identifies the differences and relations among them. It also points out some open issues for research in this field.