Yaowenqi Liu, BingXu Meng, Rui Pan, Yuxing Liu, Jerry Huang, Jiaxuan You, Tong Zhang. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2026.

The segmentation of 2D vascular structures via deep learning holds significant clinical value but is hindered by the scarcity of annotated data, severely limiting its widespread application. Developing a universal few-shot vascular segmentation model is highly desirable, yet remains challenging due to the need for extensive training and the inherent complexities of vascular imaging. In this work, we propose UniVG (Generative Data-engine Foundation Model for Universal Few-shot 2D Vascular Image Segmentation), a novel approach that learns the compositionality of vascular images and constructing a generative foundation model for robust vascular segmentation. UniVG enables the synthesis and learning of diverse and realistic vascular images through two key innovations: 1) Compositional learning for flexible and diverse vascular synthesis: It decomposes and recombines vascular structures with varying morphological features and diverse foreground-background configurations to generate richly diverse synthetic image-label pairs. 2) Few-shot generative adaptation for transferable segmentation: It fine-tunes pre-trained models with minimal annotated data to bridge the gap between synthetic and real vascular domains, synthesizing authentic and diverse vessel images for downstream few-shot vascular segmentation learning. To support our approach, we develop UniVG-58K, a large dataset comprising 58,689 vascular images across five imaging modalities, facilitating robust large-scale generative pre-training. Extensive experiments on 11 vessel segmentation tasks cross 5 modalties (only with 5 labeled images on each task) demonstrate that UniVG achieves performance comparable to fully supervised models, significantly reducing data collection and annotation costs. All code and datasets will be made publicly available at https://github.com/XinAloha/UniVG.

Adaptive gradient methods have been widely adopted in training large-scale deep neural networks, especially large foundation models. Despite the huge success in practice, their theoretical advantages over classical gradient methods with uniform step sizes across all coordinates (e.g. SGD) have not been fully understood, especially in the large batch-size setting commonly used in practice. This is because the only theoretical result that can demonstrate this benefit was obtained in the original paper of Adagrad for convex nonsmooth objective functions, which is insufficient for large batch algorithms. In this work, we attempt to resolve this gap between theory and practice by proposing a novel anisotropic generalized smoothness assumption and providing corresponding analysis of Adagrad. It is shown that under anisotropic smoothness and noise conditions, AdaGrad can achieve faster convergence guarantees in terms of better dimensional dependence than algorithms with uniform step sizes across all coordinates. Experiments in logistic regression and instruction following fine-tuning tasks provide strong evidence to support our novel assumption and theoretical analysis.

Anomaly segmentation aims to identify Out-of-Distribution (OoD) anomalous objects within images. Existing pixel-wise methods typi- cally assign anomaly scores individually and employ a global thresh- olding strategy to segment anomalies. Despite their effectiveness, these approaches encounter significant challenges in real-world applications: (1) neglecting spatial correlations among pixels within the same object, resulting in fragmented segmentation; (2) variabil- ity in anomaly score distributions across image regions, causing global thresholds to either generate false positives in background areas or miss segments of anomalous objects. In this work, we intro- duce OoDDINO, a novel multi-level anomaly segmentation frame- work designed to address these limitations through a coarse-to-fine anomaly detection strategy. OoDDINO combines an uncertainty- guided anomaly detection model with a pixel-level segmentation model within a two-stage cascade architecture. Initially, we propose an Orthogonal Uncertainty-Aware Fusion Strategy (OUAFS) that sequentially integrates multiple uncertainty metrics with visual representations, employing orthogonal constraints to strengthen the detection model's capacity for localizing anomalous regions accurately. Subsequently, we develop an Adaptive Dual-Threshold Network (ADT-Net), which dynamically generates region-specific thresholds based on object-level detection outputs and pixel-wise anomaly scores. This approach allows for distinct thresholding strategies within foreground and background areas, achieving fine- grained anomaly segmentation. The proposed framework is compatible with other pixel-wise anomaly detection models, which act as a plug-in to boost the performance. Extensive experiments on two benchmark datasets validate our framework's superiority and compatibility over state-of-the-art methods. Source code is available at: https://github.com/OoDDINO/OoD-DINO.

Training deep neural networks (DNNs) is a structured optimization problem, because the parameters are naturally represented by matrices and tensors rather than simple vectors. Under this structural representation, it has been widely observed that gradients are low-rank and Hessians are approximately block-wise diagonal. These structured properties are crucial for designing efficient optimization algorithms but may not be utilized by current popular optimizers like Adam. In this paper, we present a novel optimization algorithm ASGO that capitalizes on these properties by employing a preconditioner that is adaptively updated using structured gradients. By fine-grained theoretical analysis, ASGO is proven to achieve superior convergence rates compared to existing structured gradient methods. Based on the convergence theory, we further demonstrate that ASGO can benefit from the low-rank and block-wise diagonal properties. We also discuss practical modifications of ASGO and empirically verify the effectiveness of the algorithm on language model tasks.

Programming education is becoming increasingly popular in universities. However, due to a lack of debugging experience, novices often encounter numerous difficulties in the programming process. Automatic fault localization techniques have emerged as a promising solution to address this issue. Among these techniques, Spectrum-Based Fault Localization (SBFL) and Mutation-Based Fault Localization (MBFL) have been widely used in industrial programs. However, there is a significant difference between industrial and novice programs and the performance of these methods on novice programs has not been extensively studied. To fill this gap, we conducted an empirical study to evaluate the fault localization performance and execution overhead of SBFL and MBFL in a typical novice programming environment. Our study specifically examined how different program characteristics, including code coverage and mutation score, affect the accuracy of these localization methods. Additionally, during the study, we identified the tie problem in both methods and further investigated its impact on fault localization techniques in novice programs. To remove the impact of the tie problem, we proposed using PageRank scores as weights for the suspiciousness, sorting, and locating faults based on the weighted suspiciousness. The PageRank algorithm is based on statement coverage information and constructs a directed graph. From the directed graph, a transition matrix generates the weight scores (PageRank scores) for each statement. Our research demonstrates that both SBFL and MBFL are effective for fault localization in novice programs, with MBFL showing significantly better performance in our tests. In TOP-[Formula: see text], MBFL accurately locates 67, 96 and 114 faults, respectively, indicating superior performance. Additionally, calculating weighted suspiciousness significantly alleviates the tie problem.

Programming learning is becoming increasingly prevalent in college curricula, yet novices often encounter substantial difficulties in debugging due to their limited programming experience. In response to these challenges, automatic fault localization methods, such as Spectrum-Based Fault Localization (SBFL) and Mutation-Based Fault Localization (MBFL), have emerged as promising solutions. However, these methods are typically designed for industrial programs, which differ markedly from novice programs in terms of size and complexity. This discrepancy highlights a significant research gap in the application of these methods to novice programs. To address this gap, we conducted an empirical study to evaluate the fault localization performance and execution overhead of SBFL and MBFL in environments typical of novice programmers. Our research specifically examined how various program characteristics, including code coverage and mutation score, affect the accuracy of these localization methods. The study was comprehensive, involving experiments on 190 real novice faulty programs. The findings from our study demonstrate that both SBFL and MBFL are effective for fault localization in novice programs, though MBFL was notably more effective in our tests. MBFL demonstrated superior performance by accurately localizing 67, 96, and 114 faults within the${TOP}-{N} (N=1.\ 3.\ 5)$.

Heavy-ball momentum with decaying learning rates is widely used with SGD for optimizing deep learning models. In contrast to its empirical popularity, the understanding of its theoretical property is still quite limited, especially under the standard anisotropic gradient noise condition for quadratic regression problems. Although it is widely conjectured that heavy-ball momentum method can provide accelerated convergence and should work well in large batch settings, there is no rigorous theoretical analysis. In this paper, we fill this theoretical gap by establishing a non-asymptotic convergence bound for stochastic heavy-ball methods with step decay scheduler on quadratic objectives, under the anisotropic gradient noise condition. As a direct implication, we show that heavy-ball momentum can provide $\tilde{\mathcal{O}}(\sqrt{\kappa})$ accelerated convergence of the bias term of SGD while still achieving near-optimal convergence rate with respect to the stochastic variance term. The combined effect implies an overall convergence rate within log factors from the statistical minimax rate. This means SGD with heavy-ball momentum is useful in the large-batch settings such as distributed machine learning or federated learning, where a smaller number of iterations can significantly reduce the number of communication rounds, leading to acceleration in practice.

This paper considers the optimization problem of the form $\min_{{\bf x}\in{\mathbb R}^d} f({\bf x})\triangleq \frac{1}{n}\sum_{i=1}^n f_i({\bf x})$, where $f(\cdot)$ satisfies the Polyak–Łojasiewicz (PL) condition with parameter $\mu$ and $\{f_i(\cdot)\}_{i=1}^n$ is $L$-mean-squared smooth. We show that any gradient method requires at least $\Omega(n+\kappa\sqrt{n}\log(1/\epsilon))$ incremental first-order oracle (IFO) calls to find an $\epsilon$-suboptimal solution, where $\kappa\triangleq L/\mu$ is the condition number of the problem. This result nearly matches upper bounds of IFO complexity for best-known first-order methods. We also study the problem of minimizing the PL function in the distributed setting such that the individuals $f_1(\cdot),…,f_n(\cdot)$ are located on a connected network of $n$ agents. We provide lower bounds of $\Omega(\kappa/\sqrt{\gamma}\log(1/\epsilon))$, $\Omega((\kappa+\tau\kappa/\sqrt{\gamma})\log(1/\epsilon))$ and $\Omega\big(n+\kappa\sqrt{n}\log(1/\epsilon)\big)$ for communication rounds, time cost and local first-order oracle calls respectively, where $\gamma\in(0,1]$ is the spectral gap of the mixing matrix associated with the network and $\tau>0$ is the time cost of per communication round. Furthermore, we propose a decentralized first-order method that nearly matches above lower bounds in expectation.

This paper studies decentralized optimization problem, where the local objective on each node is an average of a finite set of convex functions and the global function is strongly convex. We propose an efficient stochastic variance reduced first-order method that allows the different nodes to establish their stochastic local gradient estimator with different mini-batch sizes per iteration. We prove the upper bound on the computation time of the proposed method contains the dependence on the global condition number, which is sharper than the previous results that only depend on the local condition numbers. Compared with the state-of-the-art methods, we also show that our method requires less local incremental first-order oracle calls and comparable communication cost. We further perform numerical experiments to validate the advantage of our method.

As a key component in industrial production, industrial spools are critical for ensuring production stability and personnel safety, primarily relying on supervised learning for anomaly detection. Although supervised learning methods achieve high detection accuracy, they depend heavily on numerous manual annotations. Therefore, self-supervised learning methods emerge as potential solutions. However, traditional self-supervised methods often overly rely on the reconstruction capabilities of sub-networks when dealing with anomalous images, leading to unsatisfactory reconstruction accuracy and poor detection results. To address these issues, we propose a self-supervised anomaly detection method for industrial spool surfaces, called MN-Net. This method adapts to complex anomaly detection tasks in various industrial scenarios by using automatically generated pseudo-labels for training, eliminating the need for manual annotation. To handle interference from synthetic anomaly information caused by different feature scales, we introduce a Multi-Scale Feature Fusion (MFF) module. Additionally, to enhance the model's ability to identify anomalies, we incorporate the Neighborhood Attention (NAM) module, which significantly improves anomaly detection by focusing on local anomalies. To evaluate the detection accuracy of MN-Net, we conducted extensive experimental studies on the industrial spool dataset and the BSData dataset. The results demonstrate that MN-Net outperforms existing methods, achieving image-level AUROC (I-AUROC) scores of 96.0% and 93.8%, and pixel-based AUROC (P-AUROC) scores of 95.3% and 90.6% on the industrial spool dataset and BSData dataset, respectively.