Qi Zhang, Shouqing Yang, Lirong Gao, Hao Chen, Xiaomeng Hu, Jinglei Chen, Jiexiang Wang, Sheng Guo, Bo Zheng, Haobo Wang, Junbo Zhao. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. 2025.

Despite substantial progress, all-in-one image restoration (IR) grapples with persistent challenges in handling intricate real-world degradations. This paper introduces MPerceiver: a novel multimodal prompt learning approach that harnesses Stable Diffusion (SD) priors to enhance adaptiveness, generalizability and fidelity for all-in-one im-age restoration. Specifically, we develop a dual-branch module to master two types of SD prompts: textual for holistic representation and visual for multiscale detail rep-resentation. Both prompts are dynamically adjusted by degradation predictions from the CLIP image encoder, en-abling adaptive responses to diverse unknown degradations. Moreover, a plug-in detail refinement module im-proves restoration fidelity via direct encoder-to-decoder in-formation transformation. To assess our method, MPer-ceiver is trained on 9 tasks for all-in-one IR and outper-forms state-of-the-art task-specific methods across many tasks. Post multitask pre-training, MPerceiver attains a generalized representation in low-level vision, exhibiting remarkable zero-shot and few-shot capabilities in unseen tasks. Extensive experiments on 16 IR tasks underscore the superiority of MPerceiver in terms of adaptiveness, gener-alizability and fidelity.

Adversarial-based adaptation has dominated the area of domain adaptive detection over the past few years. Despite their general efficacy for various tasks, the learned representations may not capture the intrinsic topological structures of the whole images and thus are vulnerable to distributional shifts especially in real-world applications, such as geometric distortions across imaging devices in medical images. In this case, forcefully matching data distributions across domains cannot ensure precise knowledge transfer and are prone to result in the negative transfer. In this paper, we explore the problem of domain adaptive lesion detection from the perspective of relational reasoning, and propose a Graph-Structured Knowledge Transfer (GraphSKT) framework to perform hierarchical reasoning by modeling both the intra- and inter-domain topological structures. To be specific, we utilize cross-domain correspondence to mine meaningful foreground regions for representing graph nodes and explicitly endow each node with contextual information. Then, the intra- and inter-domain graphs are built on the top of instance-level features to achieve a high-level understanding of the lesion and whole medical image, and transfer the structured knowledge from source to target domains. The contextual and semantic information is propagated through graph nodes methodically, enhancing the expressive power of learned features for the lesion detection tasks. Extensive experiments on two types of challenging datasets demonstrate that the proposed GraphSKT significantly outperforms the state-of-the-art approaches for detection of polyps in colonoscopy images and of mass in mammographic images.

We analyze convergence of decentralized cooperative online estimation algorithms by a network of multiple nodes via information exchanging in an uncertain environment. Each node has a linear observation of an unknown parameter with randomly time-varying observation matrices. The underlying communication network is modeled by a sequence of random digraphs and is subjected to nonuniform random time-varying delays in channels. Each node runs an online estimation algorithm consisting of a consensus term taking a weighted sum of its own estimate and neighbours' delayed estimates, and an innovation term processing its own new measurement at each time step. By stochastic time-varying system, martingale convergence theories and the binomial expansion of random matrix products, we transform the convergence analysis of the algorithm into that of the mathematical expectation of random matrix products. Firstly, for the delay-free case, we show that the algorithm gains can be designed properly such that all nodes' estimates converge to the true parameter in mean square and almost surely if the observation matrices and communication graphs satisfy the stochastic spatio-temporal persistence of excitation condition. Secondly, for the case with time delays, we introduce delay matrices to model the random time-varying communication delays between nodes. It is shown that under the stochastic spatio-temporal persistence of excitation condition, for any given bounded delays, proper algorithm gains can be designed to guarantee mean square convergence for the case with conditionally balanced digraphs.

The identification of lesion within medical image data is necessary for diagnosis, treatment and prognosis. Segmentation and classification approaches are mainly based on supervised learning with well-paired image-level or voxel-level labels. However, labeling the lesion in medical images is laborious requiring highly specialized knowledge. We propose a medical image synthesis model named abnormal-to-normal translation generative adversarial network (ANT-GAN) to generate a normal-looking medical image based on its abnormal-looking counterpart without the need for paired training data. Unlike typical GANs, whose aim is to generate realistic samples with variations, our more restrictive model aims at producing a normal-looking image corresponding to one containing lesions, and thus requires a special design. Being able to provide a "normal" counterpart to a medical image can provide useful side information for medical imaging tasks like lesion segmentation or classification validated by our experiments. In the other aspect, the ANT-GAN model is also capable of producing highly realistic lesion-containing image corresponding to the healthy one, which shows the potential in data augmentation verified in our experiments.

We consider a discrete-time distributed averaging algorithm over multi-agent networks with measurement noises and time-varying random graphs. Each agent updates its state by a weighted sum of pairwise state differences between its neighbors and itself with both additive and multiplicative measurement noises. The network structure is modeled by a sequence of time-varying random digraphs, which may be spatially and temporally dependent. By stochastic Lyapunov method and the combination of algebraic graph theory and martingale convergence theory, we obtain sufficient conditions for stochastic approximation type algorithms to achieve mean square and almost sure average consensus. We prove that all states of the agents converge to a common random variable, whose mathematical expectation is the average of initial values, in mean square and almost surely if the sequence of digraphs is conditionally balanced and uniformly conditionally jointly connected. An upper bound of the variance of the limit random variable, that is, the mean square steady-state error for stochastic average consensus is given quantitatively related to the weights, the algorithm gain and the energy level of the noises.