Developing high-performance GPU kernels is critical for AI and scientific computing, but remains challenging due to its reliance on expert crafting and poor portability. While large language models (LLMs) offer promise for automation, both general-purpose and finetuned LLMs suffer from two fundamental and conflicting limitations: correctness and efficiency. The key reason is that existing LLM-based approaches directly generate the entire optimized low-level programs, requiring exploration of an extremely vast space encompassing both optimization policies and implementation codes. To address the challenge of exploring an intractable space, we propose Macro Thinking Micro Coding (MTMC), a hierarchical framework inspired by the staged optimization strategy of human experts. It decouples optimization strategy from implementation details, ensuring efficiency through high-level strategy and correctness through low-level implementation. Specifically, Macro Thinking employs reinforcement learning to guide lightweight LLMs in efficiently exploring and learning semantic optimization strategies that maximize hardware utilization. Micro Coding leverages general-purpose LLMs to incrementally implement the stepwise optimization proposals from Macro Thinking, avoiding full-kernel generation errors. Together, they effectively navigate the vast optimization space and intricate implementation details, enabling LLMs for high-performance GPU kernel generation. Comprehensive results on widely adopted benchmarks demonstrate the superior performance of MTMC on GPU kernel generation in both accuracy and running time. On KernelBench, MTMC achieves near 100% and 70% accuracy at Levels 1-2 and 3, over 50% than SOTA general-purpose and domain-finetuned LLMs, with up to 7.3× speedup over LLMs, and 2.2× over expert-optimized PyTorch Eager kernels. On the more challenging TritonBench, MTMC attains up to 59.64% accuracy and 34× speedup. All models and datasets will be made publicly available.

We introduce a model-free hidden sequence (MHS) estimation problem where the task is to estimate a long sequence of ‘model-free’ process hidden under additive Gaussian noise. To estimate the posterior of the hidden sequence from the noisy observation sequence, we have three main challenges: (a) the process is ‘model-free’, that means there is no underlying state-space-model (SSM), limiting the use of traditional SSM-informed model-based methods like Kalman filter (KF) and particle filter (PF); (b) only an unlabelled dataset comprised of noisy observation sequences is available as training data, and hence no supervised learning possible; and, (c) the use of sequential ancestral sampling based methods, like dynamical variational autoencoders (DVAEs), results in prohibitive complexity. To address the challenges we adopt a recently established recurrent neural network (RNN)-based method called DANSE (data-driven nonlinear state estimation). We develop an iterative DANSE (iDANSE) in unsupervised learning setup where a set of neural networks (NNs) are used iteratively. The set of NNs refines MHS estimation over the iterations. Using simulations, the performance of iDANSE is demonstrated for a benchmark Lorenz process (a chaotic attractor) and compared with SSM-informed extended KF (EKF) and unscented KF (UKF).

Large Language Models (LLMs) and multimodal LLMs (MLLMs) are extensively used in latency-sensitive multimedia applications, with inference services typically offered by generative AI (GenAI) cloud service providers. However, switching between different LLMs or MLLMs on GPU servers for diverse user needs causes GPU computation stalls, leading to high cold-start latency. To address these challenges, we propose an efficient inference scheme. First, we analyze the layer-wise loading and execution characteristics and identify the root causes of GPU computation stalls. Then, we introduce InterLayer, leveraging fine-grained, compute-dependency-aware interleaved scheduling parallel transmission and layer-specific optimization to reduce GPU stalls. Additionally, we theoretically analyze the lower bound of GPU computation stalls using InterLayer. Experimental results show that InterLayer achieves a 2.4–4.9× acceleration over vLLM for multiple LLMs and MLLMs across diverse model architectures and hardware configurations, up to a 3.6× throughput improvement over vLLM, and up to 2.2× speedup over the latest state-of-the-art.

Multimodal Large Language models (MLLMs) are becoming increasingly powerful and are widely deployed. When tackling intricate inference tasks, MLLMs frequently handle multiple images and extensive textual data in tandem, generating lengthy token sequences during decoding, which leads to memory access bottlenecks. In this study, we leverage the temporal locality inherent in grouped queries that share the same KV Cache and introduce Locache, a novel approach that enables parallel processing of such queries. By improving the efficiency of memory access, Locache significantly reduces data movement. Experimental results indicate that our method achieves up to a 1.94× improvement in performance and reduces memory footprint by 3.5× compared to the current state-of-the-art shared KV Cache attention inference framework.

MOTIVATION: Inference of gene regulatory network (GRN) is challenging due to the inherent sparsity of the GRN matrix and noisy expression data, often leading to a high possibility of false positive or negative predictions. To address this, it is essential to leverage the sparsity of the GRN matrix and develop a robust method capable of handling varying levels of noise in the data. Moreover, most existing GRN inference methods produce only fixed point estimates, which lack the flexibility and informativeness for comprehensive network analysis. In contrast, a Bayesian approach that yields closed-form posterior distributions allows probabilistic link selection, offering insights into the statistical confidence of each possible link. Consequently, it is important to engineer a Bayesian GRN inference method and rigorously execute a benchmark evaluation compared to state-of-the-art methods. RESULTS: We propose a method-Bayesian inference of GRN via Sparse Modelling (BiGSM). BiGSM effectively exploits the sparsity of the GRN matrix and infers the posterior distributions of GRN links from noisy expression data by using the maximum likelihood based learning. We thoroughly benchmarked BiGSM using biological and simulated datasets including GeneNetWeaver, GeneSPIDER, and GRNbenchmark. The benchmark test evaluates its accuracy and robustness across varying noise levels and data models. Using point-estimate based performance measures, BiGSM provides an overall best performance in comparison with several state-of-the-art methods including GENIE3, LASSO, LSCON, and Zscore. Additionally, BiGSM is the only method in the set of competing methods that provides posteriors for the GRN weights, helping to decipher confidence across predictions. AVAILABILITY AND IMPLEMENTATION: Code implemented via MATLAB and Python are available at Github: https://github.com/SachLab/BiGSM and archived at zenodo.

This paper attempts to identify the reliability assessment in terms of data aggregation in wireless sensor networks. With the sensor nodes' Bayesian lifetime model, the fault ratio of sensing data among a pair of nodes in the worst case is derived. Leveraging the property of exponential aggregation latency distribution, the optimizing strategy, namely fault-tolerant scheduling for aggregation (FTSA) is proposed. Quantitative analysis of the task scheduling is also presented, which is used for reducing the latency of data aggregation. Moreover, the sensor fault probability and aggregation latency are initially introduced into the data collection scheduling process, and it mathematically shows that the fused local estimate can obtain a global estimation with the reliable data acquisition framework. Performance evaluations of the proposed approach with both sensor node faults and scheduling policy are carried out. These results show that the proposed approach outperforms the earlier algorithms in terms of latency designed without regard to sensor faults.