Serving deep learning based recommendation models (DLRM) at scale is challenging. Existing approaches rely on dedicated ANN indexing and filtering services on CPUs, suffering from non-negligible costs and missing co-design opportunities. Such inefficiency makes them difficult to support complex model architectures, such as learned similarities and multi-task retrieval. In this paper, we present SilverTorch, a model-based serving system that brings all components into one unified model. It unifies model serving by replacing standalone indexing and filtering services with model layers. We propose a model-based GPU Bloom index for feature filtering and a fused Int8 ANN kernel for nearest neighbor search. Through co-design of the ANN search and feature filtering, we reduce GPU memory usage and eliminate computation. Benefiting from this design, we scale up retrieval by introducing an OverArch scoring layer and a multi-task retrieval with a Value Model to aggregate scores. These advancements improve the retrieval accuracy and enable future studies for serving more complex models. Our evaluation on industry-scale datasets shows that SilverTorch achieves up to 23.7× higher throughput compared to the state-of-the-art approaches. We also demonstrate that SilverTorch's solution is 13.35× more cost-efficient than CPU-based solution while improving accuracy via serving more complex models.

In large-scale industrial recommendation systems, model checkpoints are instrumental in maintaining training goodput and numerical correctness during system failures and job preemptions. The increasing prevalence of multi-terabyte models has rendered frequent regular model checkpoints impractical, resulting in substantial lost progress when recovering from failures. As model sizes continue to grow, researchers and practitioners are compelled to investigate more efficient and scalable solutions. This paper presents DECK, a novel approach to delta model checkpointing designed for real-world industrial systems. Specifically, DECK focuses on extracting delta states with near-zero overhead, staging and streaming delta checkpoints without interrupting the training process, and merging delta checkpoints in an optimal and decoupled manner. Experimental results demonstrate that DECK achieves a 12-fold increase in checkpoint frequency while maintaining negligible impact on training throughput, thereby attaining state-of-the-art (SOTA) production performance.

This paper considers the problem of jointly processing messages received over a forward link from a single distant transmitter to a cooperative receive cluster connected by a local area network with finite available throughput. For N cooperating receivers, ideal distributed receive beamforming with direct exchange of unquantized observations leads to an N-fold gain in signal-to-noise ratio (SNR) for equal-gain additive white Gaussian noise channels, with significant additional gains over fading channels due to diversity. It is shown in this paper that a significant portion of these gains can be obtained simply by exchanging hard decisions among some or all of the nodes in the receive cluster. Mutual information computations and simulations of LDPC-coded systems show that optimal combining of hard decisions tends to perform within 0.5-2 dB of ideal receive beamforming. For the low per-node SNR regime of interest with large receive clusters, asymptotic analysis of a suboptimal combining technique termed "pseudo-beamforming" shows that distributed reception with hard decision exchanges performs within 1-2 dB of ideal receive beamforming.

In the most general form, clock skew scheduling (CSS) generates a dedicated clock delay for each individual sequential component in the clock distribution network in order to minimize the clock period. Multidomain CSS (MDCSS) relieves this requirement. Instead, sequential components are grouped into several clusters (called clock domains), each of which has a uniform clock delay for all registers within that domain. The skew values of clock domains are provided by a set of deskew buffers with electrically programmable phase shifts and injected after the chip is manufactured. This technique is attractive since, due to process variations, it is becoming overwhelmingly difficult to create precise clock network delays for all sequential elements in a design globally. In this paper, we present a fast algorithm for determining the minimum number of clock domains to be used by MDCSS. The exact solution to this problem cannot be found within a reasonable time if the number of clock domains increases beyond three domains. We show that, even with a small-size circuit, in order to obtain the minimum clock period, more than three clock domains may be required. Therefore, a fast heuristic algorithm is needed to identify these domains. To the best of our knowledge, we present the first efficient heuristic algorithm for this problem. For large benchmark circuits, we solve the problem within 14.7 min on average (as high as 31.7 min for the worst case), while a commercial mixed-integer linear program solver cannot finish in over 5 h. Furthermore, our results show that, for 19 out of 21 small- and medium-size benchmarks, our algorithm yields the optimal solution.

Clock skew scheduling has been traditionally considered as a tool for improving the clock period in a sequential circuit. Timing slack is "stolen" from fast combinational blocks to be used by slower blocks to meet a more stringent clock cycle time. Instead, we can leverage on the borrowed time to achieve leakage power reduction during gate sizing and/or dual Vth assignment. In this paper, we present the first approach to the best of our knowledge for integrating clock skew scheduling, threshold voltage assignment, and gate sizing into one optimization formulation. Over 29 circuits in the ISCAS89 benchmark suite, this integrated approach can reduce leakage power by as much as 55.83% and by 18.79% on average, compared to using combinational circuit based power optimization on each combinational block without considering clock skews. Using a 65nm dual Vth technology library, this corresponds to a 23.87% peak reduction (6.15% on average) in total power at the ambient operating temperature. The average total power reduction further increases to 9.83% if the high temperature library of the same process technology is used.

High-performance microprocessor families employ dynamic-thermal-management techniques to cope with the increasing thermal stress resulting from peaking power densities. These techniques operate on feedback generated from on-die thermal sensors. The allocation and the placement of thermal-sensing elements directly impact the effectiveness of the dynamic management mechanisms. In this paper, we propose systematic techniques for determining the optimal locations for thermal sensors to provide high-fidelity thermal monitoring of a complex microprocessor system. Our strategies can be divided into two main categories: uniform sensor allocation and nonuniform sensor allocation. In the uniform approach, the sensors are placed on a regular grid. The nonuniform allocation identifies an optimal physical location for each sensor such that the sensor's attraction toward steep thermal gradients is maximized, which can result in uneven concentrations of sensors on different locations of the chip. We also present a hybrid algorithm that shows the tradeoffs associated with number of sensors and expected accuracy. Our experimental results show that our uniform approach using interpolation can detect the chip temperature with a maximum error of 5.47degC and an average maximum error of 1.05degC . On the other hand, our nonuniform strategy is able to create a sensor distribution for a given microprocessor architecture, providing thermal measurements with a maximum error of 3.18degC and an average maximum error of 1.63degC across a wide set of applications.

Design decisions made during high-level synthesis usually have great impacts on the later design stages. In this paper, We present a general framework, which plans for the clock skew-scheduling in physical design stages during register binding in high-level synthesis. Our proposed technique pursues the optimality of the native objective functions of the register binding problem. At the same time, it ensures not invalidating the subsequent clock skew scheduling for optimizing the clock period. We use the switching power as the native objective of our register binding problem. The problem is first formulated as a MILP problem. An acceleration scheme based on the concept of weakly compatible edge set (WCES) is proposed to speed up the MILP solver to obtain the optimal solution. Then, we present our heuristic algorithm to reduce the running time further. The experimental results show that on average our acceleration scheme can speed up the solver by 8.6 times, and our heuristic is 70 times faster than the solver with a 5.25% degradation of the native objective. The minimum and maximum degradation among our benchmark set are 0.82% and 12.2%, respectively.

Traditionally, at early design stages, leakage power is associated with the number of transistors in a design. Hence, intuitively an implementation with minimum resource usage would be best for low leakage. Such an allocation would generally be followed by switching optimal resource binding to achieve a low power design. This treatment of leakage power is unaware of operating conditions such as temperature. In this paper, we propose a technique to reduce the total leakage power of a design by identifying the optimal number of resources during allocation and binding. We demonstrate that, contrary to the general tendency to minimize the number of resources, the best solution can actually be achieved if a certain degree of redundancy is allowed. This is due to the fact that leakage is strongly dependent on the on-chip temperature profile. Distributing activity over a higher number of resources can reduce power density, remove potential hotspots and subsequently minimize thermal induced leakage. On the other hand, using an arbitrarily high number of resources will not yield the best solution. In this paper, we show that there is a power density, hence, temperature, at which the total leakage power will reach its optimal value. Such an optimal resource number can be a better starting point for the subsequent switching-driven low power binding. We also present a high-level power density-aware leakage model. Based on the estimates by this model, we optimize the total leakage power by 53.8% on average compared to the minimum resource binding, and 35.7% on average compared to a temperature-aware resource binding technique.