Search tasks require finding items similar to a given query, making it a crucial aspect of various applications. However, storing and computing similarity for millions or billions of item representations can be computationally expensive. To address this, quantization-based hash methods present memory and inference-efficient solutions by converting continuous representations into non-negative integer codes. Despite their advantages, these methods often encounter difficulties in handling long-tail datasets due to imbalanced class distributions. To address this, we propose LightLT, a lightweight representation quantization framework tailored for long-tail datasets. LightLT produces compact codebooks and discrete IDs, enabling efficient inference by computing distances between query and codewords. Our framework includes innovative designs: 1) Quantization Step: We select the most similar codeword for continuous inputs using the differentiable argmax operation. 2) Double Skip Quantization Connection Module: This module promotes codebook diversity and stability during training. 3) Training Loss: Our comprehensive loss includes class-weighted cross-entropy, center loss, and ranking loss. 4) Model Ensemble: We incorporate a model ensemble step to improve generalization. Theoretical analysis confirms LightLT's low space and inference complexity. Experimental results demonstrate superior performance compared to state-of-the-art baselines in terms of search accuracy, efficiency, and memory usage.

Due to the various applications and high sparsity of deep neural network (DNN), a lot of sparse-aware DNN accelerators have been proposed to exploit the sparsity in DNN. Furthermore, it is essential to optimize for the accumulations and inter-channel aggregations in DNN to reduce memory overhead and improve performance of DNN accelerator. However, the uncertain number and location of non-zero element in DNN pose critical challenges for optimizing such accelerators and this inspires us to explore an efficient spare-aware summation optimization strategy for DNN accelerator. In this paper, we leverage the strategy that trading higher cost memory storage/access for lower cost computation to propose a random index based sparse-aware adder tree (RAT), which achieves a better trade-off among performance, hardware resource overhead and adaptability. Synthesis and simulation results demonstrate that, compared with reference design, the proposed design achieves 1.71× and 1.52× the normalized area efficiency and energy efficiency improvement on ResNet18, respectively.

Due to the bottleneck problems such as scenario-varying application, significant data communication overhead and privacy protection between off-line training and on-line inference, intelligent edge devices capable of adaptively fine-tuning the deep neural network (DNN) models for specific tasks have become the most urgent need. However, the computational cost is intolerable for ordinary on-device training (ODT), which inspires us to explore an efficient ODT processor, named DQ-STP. In this paper, we leverage a series of optimization techniques using software-hardware co-design. On the one hand, the proposed design incorporates SVD-based low-rank decomposition,$2^{n}$quantization and ACBN algorithm on the software side. This unifies the sparse computing mode of convolutional layers and enhancing weight sparsity. On the other hand, the proposed design effectively leverages data sparsity on the hardware side through four techniques: 1) The flag compressed sparse row is proposed to compress input feature maps and gradient maps. 2) A unified processing element (PE) array comprising shifters and adders is proposed to expedite forward and error propagation steps. 3) The PE arrays for error propagation and weight gradients generation are separated to enhance throughput. 4) A sparse alignment strategy is proposed to further enhance PE utilization. Through these software and hardware co-optimization, the proposed DQ-STP achieves an area efficiency and peak energy efficiency of 41.2 GOPS/mm2 and 90.63 TOPS/W. In comparison to state-of-the-art reference designs, the proposed DQ-STP demonstrates a$2.19\times $improvement in normalized area efficiency and a$1.85\times $enhancement in energy efficiency.

Aiming at a better understanding of the search goals in the user search sessions, recent query recommender systems explicitly model the reformulations of queries, which hopes to estimate the intents behind these reformulations and thus benefit the next-query recommendation. However, in real-world e-commercial search scenarios, user intents are much more complicated and may evolve dynamically. Existing methods merely consider trivial reformulation intents from semantic aspects and fail to model dynamic reformulation intent flows in search sessions, leading to sub-optimal capacities to recommend desired queries. To deal with these limitations, we first explicitly define six types of query reformulation intents according to the desired products of two consecutive queries. We then apply two self-attentive encoders on top of two pre-trained large language models to learn the transition dynamics from semantic query and intent reformulation sequences, respectively. We develop an intent-aware query decoder to utilize the predicted intents for suggesting the next queries. We instantiate such a framework with an Intent-aware Variational AutoEncoder (IVAE) under deployment at Amazon. We conduct comprehensive experiments on two real-world e-commercial datasets from Amazon and one public dataset from BestBuy. Specifically, IVAE improves the Recall@15 by 25.44% and 60.47% on two Amazon datasets and 13.91% on BestBuy, respectively.

As training deep learning models on large dataset takes a lot of time and resources, it is desired to construct a small synthetic dataset with which we can train deep learning models sufficiently. There are recent works that have explored solutions on condensing image datasets through complex bi-level optimization. For instance, dataset condensation (DC) matches network gradients w.r.t. large-real data and small-synthetic data, where the network weights are optimized for multiple steps at each outer iteration. However, existing approaches have their inherent limitations: (1) they are not directly applicable to graphs where the data is discrete; and (2) the condensation process is computationally expensive due to the involved nested optimization. To bridge the gap, we investigate efficient dataset condensation tailored for graph datasets where we model the discrete graph structure as a probabilistic model. We further propose a one-step gradient matching scheme, which performs gradient matching for only one single step without training the network weights. Our theoretical analysis shows this strategy can generate synthetic graphs that lead to lower classification loss on real graphs. Extensive experiments on various graph datasets demonstrate the effectiveness and efficiency of the proposed method. In particular, we are able to reduce the dataset size by 90% while approximating up to 98% of the original performance and our method is significantly faster than multi-step gradient matching (e.g. $15$× in CIFAR10 for synthesizing 500 graphs).

With the increasing demands on e-commerce platforms, numerous user action history is emerging. Those enriched action records are vital to understand users’ interests and intents. Recently, prior works for user behavior prediction mainly focus on the interactions with product-side information. However, the interactions with search queries, which usually act as a bridge between users and products, are still under investigated. In this paper, we explore a new problem named temporal event forecasting, a generalized user behavior prediction task in a unified query product evolutionary graph, to embrace both query and product recommendation in a temporal manner. To fulfill this setting, there involves two challenges: (1) the action data for most users is scarce; (2) user preferences are dynamically evolving and shifting over time. To tackle those issues, we propose a novel Retrieval-Enhanced Temporal Event (RETE) forecasting framework. Unlike existing methods that enhance user representations via roughly absorbing information from connected entities in the whole graph, RETE efficiently and dynamically retrieves relevant entities centrally on each user as high-quality subgraphs, preventing the noise propagation from the densely evolutionary graph structures that incorporate abundant search queries. And meanwhile, RETE autoregressively accumulates retrieval-enhanced user representations from each time step, to capture evolutionary patterns for joint query and product prediction. Empirically, extensive experiments on both the public benchmark and four real-world industrial datasets demonstrate the effectiveness of the proposed RETE method.

Haoming Jiang, Danqing Zhang, Tianyu Cao, Bing Yin, Tuo Zhao. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 2021.

We study the problem of query attribute value extraction, which aims to identify named entities from user queries as diverse surface form attribute values and afterward transform them into formally canonical forms. Such a problem consists of two phases: named entity recognition (NER) and attribute value normalization (AVN). However, existing works only focus on the NER phase but neglect equally important AVN. To bridge this gap, this paper proposes a unified query attribute value extraction system in e-commerce search named QUEACO, which involves both two phases. Moreover, by leveraging large-scale weakly-labeled behavior data, we further improve the extraction performance with less supervision cost. Specifically, for the NER phase, QUEACO adopts a novel teacher-student network, where a teacher network that is trained on the strongly-labeled data generates pseudo-labels to refine the weakly-labeled data for training a student network. Meanwhile, the teacher network can be dynamically adapted by the feedback of the student's performance on strongly-labeled data to maximally denoise the noisy supervisions from the weak labels. For the AVN phase, we also leverage the weakly-labeled query-to-attribute behavior data to normalize surface form attribute values from queries into canonical forms from products. Extensive experiments on a real-world large-scale E-commerce dataset demonstrate the effectiveness of QUEACO.

Sequence labeling aims to predict a finegrained sequence of labels for the text.However, such formulation hinders the effectiveness of supervised methods due to the lack of token-level annotated data.This is exacerbated when we meet a diverse range of languages.In this work, we explore multilingual sequence labeling with minimal supervision using a single unified model for multiple languages.Specifically, we propose a Meta Teacher-Student (MetaTS) Network, a novel meta learning method to alleviate data scarcity by leveraging large multilingual unlabeled data.Prior teacher-student frameworks of self-training rely on rigid teaching strategies, which may hardly produce high-quality pseudo-labels for consecutive and interdependent tokens.On the contrary, MetaTS allows the teacher to dynamically adapt its pseudoannotation strategies by the student's feedback on the generated pseudo-labeled data of each language and thus mitigate error propagation from noisy pseudo-labels.Extensive experiments on both public and real-world multilingual sequence labeling datasets empirically demonstrate the effectiveness of MetaTS 1 .

Exploiting label hierarchies has become a promising approach to tackling the zero-shot multi-label text classification (ZS-MTC) problem.Conventional methods aim to learn a matching model between text and labels, using a graph encoder to incorporate label hierarchies to obtain effective label representations (Rios and Kavuluru, 2018).More recently, pretrained models like BERT (Devlin et al., 2018) have been used to convert classification tasks into a textual entailment task (Yin et al., 2019).This approach is naturally suitable for the ZS-MTC task.However, pretrained models are underexplored in the existing work because they do not generate individual vector representations for text or labels, making it unintuitive to combine them with conventional graph encoding methods.In this paper, we explore to improve pretrained models with label hierarchies on the ZS-MTC task.We propose a Reinforced Label Hierarchy Reasoning (RLHR) approach to encourage interdependence among labels in the hierarchies during training.Meanwhile, to overcome the weakness of flat predictions, we design a rollback algorithm that can remove logical errors from predictions during inference.Experimental results on three reallife datasets show that our approach achieves better performance and outperforms previous non-pretrained methods on the ZS-MTC task.

Two-dimensional principal component analysis (2DPCA), as a state-of-the-art method for dimensionality reduction, has been widely used in face recognition. However, it is very sensitive to outliers since it minimizes the sum of squared F-norm, which is least-squares loss in nature. Recently, angle 2DPCA was presented to alleviate this problem by minimizing the sum of Fnorm, which is corresponding to L1 loss. But a vital unresolved problem of angle 2DPCA is that it needs many more coefficients for image representation because it works only in the row direction. In this letter, we first give a new angle 2DPCA called Sin-2DPCA by minimizing the relative error, which has a better explanation than the original one. Furthermore, in order to obtain better performance with fewer reduced coefficients, we project the input image to a lower dimension from right and left simultaneously, and then, the bilateral angle 2DPCA (BA2DPCA) is proposed. The experimental results on two benchmark face recognition datasets with outlier noises illustrate that the Sin-2DPCA has the similar performance with original angle 2DPCA, and BA2DPCA can obtain the highest performance in all compared algorithms with the minimal number of representation coefficients.

Despite the numerous prevention and protection techniques that have been developed, the exploitation of memory corruption vulnerabilities still represents a serious threat to the security of software systems and networks. Because of the adoption of the write or execute only policy (W⊕X) and address space layout randomization (ASLR), modern operate systems have been strengthened against code injection attacks. However, attackers have responded by employing code reuse attacks, in which software vulnerability is exploited to weave control flow through existing code base. Solutions targeting different aspects of the attack itself have had some success, but none of them can be a silver bullet. Under this situation, it is necessary to develop a general prevention to mitigate code reuse attacks. In this paper, we present a novel and general defense technique called control flow lazily check (CFLC), which allows for effective enforcement of control flow integrity. Specifically, instead of immediately determining the violation of control flow before the control flow transfer takes place, CFLC detects the violation after the transfer. Further, CFLC ensures that no deviation can be used to bypass the checking code and craft a malicious system call neither. To reduce the performance overhead, we introduce a coarse-grained CFLC based on the principle that a success intrusion must invoke a system call. We have implemented CFLC with the help of dynamic binary instrumentation tool and the evaluation demonstrates that CFLC can not only prevent code reuse attacks but also code injection attacks. It is shown that CFLC has achieved significant safety than other existing defenses with a modest performance penalty.

Content-based image retrieval (CBIR) has been investigated extensively in the past decade in order to classify and search images according to similarities derived from automatically extracted visual features, such as colours, textures and object shapes. It has now been realised that two fundamental problems in CBIR, namely, feature extraction and similarity measure, are likely to be domain specific. In this paper, we present some early results of applying CBIR to traditional Chinese paintings. Our research is motivated by three main goals: (1) to develop tools for art historians to study evolution and cross-influences of oriental paintings by automatically identifying visual artistic clues from digitised paintings; (2) to verify and further advance the existing CBIR techniques by limiting the images studied to a specific and simpler domain of traditional Chinese paintings; and (3) to verify and further advance the problem of high-dimensional data clustering (especially in relation to the "dimensional curse" problem). We present a framework for modelling traditional Chinese paintings, and examine various existing CBIR proposals and algorithms for their suitability for traditional Chinese paintings. In this paper, we also present a research agenda to study the problems of Chinese paintings classification and retrieval based on the framework.