This paper addresses the important and challenging task of large-scale unsupervised semantic segmentation (LUSS). We present the first attempt to unleash the power of foundation models (FMs) for the challenging, dense prediction task LUSS, and our main objective is to present simple, effective yet efficient solutions for LUSS, namely Prompting foundation models for LUSS (PLUSS). Firstly, we proposed a cascade framework PLUSS$_\alpha$α by effectively marrying CLIPS, Grounding DINO, and SAM in a zero-shot manner. This cascade architecture automatically generates semantic and spatial prompts for SAM, establishing a strong baseline that significantly outperforms previous state-of-the-art methods. Building upon this foundation, we propose PLUSS$_\beta$β, which addresses the critical bottleneck of prompt quality through two novel tuner modules: a semantic tuner that enhances fine-grained category discrimination via visual prompt tuning, and a box tuner that improves object localization through cross-modal feature fusion. Both tuners are optimized by capitalizing on the knowledge already present within the foundation models themselves, deriving self-supervised signals from internal model consistency. This approach requires no external supervision or updates to the foundation models' parameters. Extensive experiments on ImageNet-S benchmarks demonstrate that PLUSS$_\beta$β achieves remarkable performance improvements, surpassing the previous best method by 39.6%, 27.3%, and 22.6% in mIoU for 50, 300, and 919 categories respectively. Our approach exhibits robust category-shape representation across varying object sizes and dataset scales, while maintaining strong generalization capabilities for open-vocabulary tasks. The proposed framework provides a solid baseline for adapting foundation models to downstream vision tasks.

Current deep learning methods for strip steel defect detection prioritize accuracy and computational efficiency but struggle to generalize to new production environments with unseen defect types. While visual foundation models like segment anything model (SAM) offer broader generalization through pretrained visual knowledge, they face significant domain shifts when applied to industrial defect imagery. To address this, we propose AdaptedSAM, a parameter-efficient few-shot fine-tuning framework for SAM that optimizes adapter placement and feature interaction. We first demonstrate that adapter positioning within the encoder critically impacts few-shot adaptation performance. Building on this, we introduce an efficient global optimizer that dynamically allocates adapter weights via cross-block communication across transformer layers. Furthermore, we develop an adapted mask decoder to enhance detailed mask information. Our adaptedSAM achieves state-of-the-art defect detection with precise boundaries in five-shot learning, outperforming 12 state of the art (SOTA) models on the SD-saliency-900 steel dataset while ensuring computational efficiency. Cross-domain evaluations on six benchmarks (industrial defects, remote sensing, medical imaging, natural scenes) demonstrate robust generalization.

In-process hot-rolled strip steel is suffering from some complicated yet unavoidable surface defects due to its harsh production environment. The automated visual inspection on defects consistently faces challenges of interclass similarity, intraclass difference, low contrast, and overlapping issue, which tend to trigger false or missed detections. This article proposes a prior-modulated and semantic-aligned dynamic transformer, called PMSA-DyTr. In this framework, a long short-term self-attention embedded with local convolution is designed for assisting an encoder to eliminate noise ambiguity between defects and backgrounds. Then, a semantic aligner is cleverly bridged between the encoder and the decoder to align the sematic for speeding up the convergence, and prior-modulated cross attention is proposed to alleviate the deficiency of samples for a data-driven transformer. Furthermore, a gate controller is innovatively constructed to dynamically select the minimal number of encoder blocks while preserving detection accuracy. The proposed PMSA-DyTr outperforms 19 state-of-the-art models on mean average precision with an inference time of 54.67 ms and visually performs best in detecting low-contrast and multiple small defects.