Post-Training Quantization enables efficient Vision Transformer (ViTs) deployment with a small calibration data, and its prevalent use of uniform quantization harnesses AI accelerator matrix cores for high-speed inference. However, the application of uniform quantization is fundamentally challenged by the extreme non-uniformity of activation distributions.Specifically, the power-law nature of post-Softmax attention scores and the significant inter-channel variance in post-GELU activations create a dilemma for conventional quantization, as it struggles to preserve critical high-magnitude values without sacrificing overall precision. To resolve this core conflict, we introduce UQ-ViT (Uniform Quantization for Vision Transformers), a novel uniform quantization framework designed to reconcile high precision with hardware efficiency. Central to UQ-ViT are two operators: Dynamic Elimination of Maximum (DeMax) and Normalization Quantization (NormQuant). DeMax is a quantization operator for post-Softmax attention scores that utilizes uniform quantization. It dynamically eliminates and preserves dominant values, effectively mitigating quantization loss from the extreme values in the power-law distribution. NormQuant utilizes a per-channel quantization strategy during quantization and reverts to a per-tensor format for dequantization, achieving both high accuracy and computational efficiency. Crucially, it is applicable to any linear layer, enabling effective quantization of post-GELU activations in ViTs. Through extensive experiments on various ViTs and vision tasks, including image classification, object detection, and instance segmentation, we demonstrate that our proposed approach outperforms existing methods, achieving superior accuracy while ensuring hardware friendliness.

Recently, Learned Image Compression (LIC) models have garnered significant attention due to their superior performance in comparison to traditional image codecs. However, the growing complexity of these deep learning-based models results in high memory consumption and computational load, which limits their practical deployment. Quantization has emerged as a promising technique to reduce both the storage requirements and computational overhead. Despite its success in high-level vision tasks like image recognition and object detection, quantization techniques applied to LIC models remain underexplored. In this work, we identify the unique challenges of quantizing LIC models, specifically focusing on the impact of latent distribution ranges in high-bitrate. We observe that the activation layers of high-bitrate models exhibit a wider distribution range, which causes significant performance degradation after quantization. Furthermore, we explore the limitations of existing LIC quantization schemes, such as per-channel quantization for activation layers, which result in poor hardware acceleration performance and increased data storage overhead. To address these challenges, we propose the activation and weight distribution balancing post-training quantization (AWDB-PTQ) method for LIC models, which uses a coarse-to-fine strategy to optimize balancing coefficients. In addition, we employ per-tensor activation quantization and symmetric uniform quantization to better facilitate hardware acceleration. Experimental results demonstrate that our proposed method outperforms existing methods in terms of both compression performance and computational efficiency. Our code and data are available at: https://github.com/jie-yu16/AWDB-PTQ.

With the rapid development of neural networks, learned image compression (LIC) has surpassed traditional methods in the field of image compression. However, current LIC frameworks rely on floating-point operations. On the one hand, the use of conventional floating-point operations results in high computational costs, limiting their deployment in the Internet of Things (IoT) devices. On the other hand, when employing prior models, floating-point errors in prior calculations can cause cross-platform decoding failures. Neural network quantization addresses these issues by using low-bitwidth fixed-point, which reduces computational complexity and memory usage while eliminating inconsistencies from floating-point operations. Although quantizing the entire network to a uniform bitwidth simplifies hardware deployment, it can significantly impair coding efficiency, particularly in LIC networks at high bitrates, where certain layers are more sensitive to quantization noise, and we are first to provide both empirical and theoretical insights into the underlying reasons for these layers’ increased quantization difficulty. To address these challenges, we propose a mixed-precision post-training quantization (MP-PTQ) scheme that requires only a small calibration dataset without retraining, enabling rapid deployment. This method assesses the quantization sensitivity of each layer through rate-distortion loss analysis, thereby allocating the optimal bitwidth and quantization scale for each layer. Our method significantly improves the compression performance of quantized LIC models, achieving a Bjøntegaard-Delta rate (BD-rate) loss of less than 1% compared to their floating-point counterparts. Furthermore, by employing per-tensor activation quantization, we enable hardware-friendly acceleration—achieving approximately 3× speedup over floating-point models.

The push-relabel algorithm is recognized as one of the efficient algorithms in the field of graph cut, finding widespread applications in computer vision. While its pixel-level parallel implementations are prevalent, existing methods predominantly rely on checkerboard scheduling, imposing inherent constraints on neighborhood size, limited to four. This limitation compromises both algorithm precision and efficiency, hindering real-time and high-precision applications. To address these issues, this article introduces a novel approach to accelerate push-relabel algorithm implementation on FPGA in a more universal and efficient manner, supporting variable-sized image block operations. First, by introducing the deferred update strategy, we realize the pulse-enhanced parallel push-relabel (PEPPR) algorithm to address data contention and conflict in parallel processing. Second, the simultaneous weighted push method is proposed, further enhancing parallel operations. Lastly, we introduce the efficient diffusion wave search (DWS) algorithm to expedite algorithm convergence and reduce redundancy. While achieving a modest$1.7\times $acceleration compared to state-of-the-art implementations, the proposed algorithm (PEPPR-DWS) successfully overcomes the inherent limitations of checkerboard scheduling in full pixel-level parallelism. In the test based on Middlebury benchmark V3, the proposed 8-neighborhood implementation exhibits a reduction of error rate by over 1% compared to the typical 4-neighborhood implementation. It provides a versatile and efficient solution for high-precision and real-time applications, holding substantial potential for practical applications.

While language model (LM)-powered chatbots and generative search engines excel at answering concrete queries, discovering information in the terrain of unknown unknowns remains challenging for users.To emulate the common educational scenario where children/students learn by listening to and participating in conversations with their parents/teachers, we create Collaborative STORM (Co-STORM). 1 Unlike QA systems that require users to ask all the questions, Co-STORM lets users observe and occasionally steer the discourse among several LM agents.The agents ask questions on the user's behalf, allowing the user to discover unknown unknowns serendipitously.To facilitate user interaction, Co-STORM assists users in tracking the discourse by organizing the uncovered information into a dynamic mind map, ultimately generating a comprehensive report as takeaways.For automatic evaluation, we construct the WildSeek dataset by collecting real information-seeking records with user goals.Co-STORM outperforms baseline methods on both discourse trace and report quality.In a further human evaluation 2 , 70% of participants prefer Co-STORM over a search engine, and 78% favor it over a RAG (Retrieval Augmented Generation) chatbot.

Yijia Shao, Yucheng Jiang, Theodore Kanell, Peter Xu, Omar Khattab, Monica Lam. Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 2024.

Instead of mining coherent topics from a given text corpus in a completely unsupervised manner, seed-guided topic discovery methods leverage user-provided seed words to extract distinctive and coherent topics so that the mined topics can better cater to the user's interest. To model the semantic correlation between words and seeds for discovering topic-indicative terms, existing seed-guided approaches utilize different types of context signals, such as document-level word co-occurrences, sliding window-based local contexts, and generic linguistic knowledge brought by pre-trained language models. In this work, we analyze and show empirically that each type of context information has its value and limitation in modeling word semantics under seed guidance, but combining three types of contexts (i.e., word embeddings learned from local contexts, pre-trained language model representations obtained from general-domain training, and topic-indicative sentences retrieved based on seed information) allows them to complement each other for discovering quality topics. We propose an iterative framework, SeedTopicMine, which jointly learns from the three types of contexts and gradually fuses their context signals via an ensemble ranking process. Under various sets of seeds and on multiple datasets, SeedTopicMine consistently yields more coherent and accurate topics than existing seed-guided topic discovery approaches.

Given a few seed entities of a certain type (e.g., Software or Programming Language), entity set expansion aims to discover an extensive set of entities that share the same type as the seeds. Entity set expansion in software-related domains such as StackOverflow can benefit many downstream tasks (e.g., software knowledge graph construction) and facilitate better IT operations and service management. Meanwhile, existing approaches are less concerned with two problems: (1) How to deal with multiple types of seed entities simultaneously? (2) How to leverage the power of pre-trained language models (PLMs)? Being aware of these two problems, in this paper, we study the entity set co-expansion task in StackOverflow, which extracts Library, OS, Application, and Language entities from StackOverflow question-answer threads. During the co-expansion process, we use PLMs to derive embeddings of candidate entities for calculating similarities between entities. Experimental results show that our proposed SECoExpan framework outperforms previous approaches significantly.

In this article, we approach to construct reliable deep neural networks (DNNs) for safety-critical artificial intelligent applications. We propose to modify rectified linear unit (ReLU), a commonly used activation function in DNNs, to tolerate the faults incurred by bit-flip perturbation on weights. Through theoretic analysis of the fault propagation in the layers with ReLU activation, we observe that bounding the output of ReLU activation can help to tolerate the weight faults. Then, we propose a novel ReLU design called boundary-aware ReLU (BReLU) to improve the reliability of DNNs, in which an upper bound of ReLU is determined such that the deviation between the boundary and original outputs cannot affect the final result. We propose a gradient-ascent-based algorithm to find the boundaries for BReLU activations of all DNN layers. Without retraining the network, our approach is cost effective and practical when deployed in safety-critical artificial intelligent systems. Detailed experiments and real-life application benchmarking demonstrate that our approach can improve the accuracy of DNN VGG16 from 16.7% to 82.6% on average assuming the practical weight faults, with only 13% memory and 2.78% time overhead, respectively.

In this paper, we approach to improve the fault tolerance of Deep Neural Networks (DNNs) for safety-critical artificial intelligent applications. We propose to remap the range of 32-bit float to weights to reduce the influence of invalid weights caused by bit-flip faults. From preliminary experiments, we observe that weakening bit-flip faults which make positive weights larger can help to improve the reliability of DNNs. Then, we propose a gaussian distribution based mapping method to prevent weights from being influenced by bit-flip faults, in which a novel function is formulated to remap the relation between 32-bit float and the values of weights. Extensive experiments demonstrate that our approach can improve the accuracy of VGG16 from 13.5% to 80.5%, which is better than the other six tolerance approaches of DNNs.