Large-scale Visual-Language Models (LVLMs) have achieved remarkable success in natural visual tasks, yet their application to industrial defect detection remains challenging due to two fundamental limitations: (i) the scarcity of large-scale industrial datasets that cover diverse defect categories across multiple domains, and (ii) the reliance on manual prompts (points, boxes, masks) that introduce subjective noise and lack text-visual interaction for fine-grained understanding. To address these challenges, we introduce a Large-Scale Multi-Modal Industrial Open-Closed benchmark (MMIOC-1 M) containing over one million samples across 14 super-categories, 29 industrial scenes, and 351 defect subcategories. To our knowledge, MMIOC-1 M is the first unified largest benchmark supporting both open-vocabulary and closed-set industrial detection, providing valuable pre-training data for LVLMs in industrial scenarios. Furthermore, we propose a Refined Text-Visual Prompt Network (RTVPNet) that incorporates three key innovations: (1) an expert-assisted domain projection mechanism that enables rapid adaptation of general vision models to industrial domains, (2) an energy-based sparse sampling strategy that automatically generates refined visual prompts without manual intervention, and (3) a bidirectional text-visual interaction module that enhances cross-modal semantic alignment and understanding. Extensive experiments demonstrate that RTVPNet achieves state-of-the-art performance on MMIOC-1 M, LVIS, and COCO benchmarks while maintaining computational efficiency.

In some orthopedic surgeries, the use of three-dimensional (3D) computed tomography (CT) scanning technology is not feasible due to scene limitations, leaving doctors to rely on two-dimensional (2D) X-ray images for real-time diagnosis. However, X-ray images lack 3D information, making accurate diagnosis challenging. Developing an algorithm to convert 2D X-ray images into 3D CT images, while simultaneously combining high-quality 3D reconstruction with precise fracture segmentation, offers a promising solution to the problem. In this study, we propose a novel artificial intelligence (AI)-driven framework named 3D reconstruction and segment anything model (3DRecSAM). The reconstruction image enhancer (RIE) is designed to achieve high-precision 3D reconstruction and provide high-quality feature initialization for fracture segmentation. Meanwhile, the mamba segment anything model (MSAM), based on the segment anything model (SAM) architecture, is developed for accurate fracture segmentation. We introduce a Kolmogorov–Arnold network (KAN)-based attention fusion module (KAF), which facilitates the joint optimization of the RIE reconstruction network and the MSAM segmentation network. Furthermore, the selective scanning mamba with KAN (SKM) is incorporated to enhance feature extraction for both RIE and MSAM. Mamba efficiently captures long-range dependencies and sequential patterns, while KAN’s learnable activation functions facilitate adaptive feature fusion and non-linear representation. To train and evaluate 3DRecSAM, we introduce the real X-ray and CT paired dataset (XCPData), which is publicly available on GitHub: https://github.com/YuanGao1201/XCPData .

Choroidal neovascularization (CNV), a primary characteristic of wet age-related macular degeneration (wet AMD), represents a leading cause of blindness worldwide. In clinical practice, optical coherence tomography angiography (OCTA) is commonly used for studying CNV-related pathological changes, due to its micron-level resolution and non-invasive nature. Thus, accurate segmentation of CNV regions and vessels in OCTA images is crucial for clinical assessment of wet AMD. However, challenges existed due to irregular CNV shapes and imaging limitations like projection artifacts, noises and boundary blurring. Moreover, the lack of publicly available datasets constraints the CNV analysis. To address these challenges, this paper constructs the first publicly accessible CNV dataset (CNVSeg), and proposes a novel multilateral graph convolutional interaction-enhanced CNV segmentation network (MTG-Net). This network integrates both region and vessel morphological information, exploring semantic and geometric duality constraints within the graph domain. Specifically, MTG-Net consists of a multi-task framework and two graph-based cross-task modules: Multilateral Interaction Graph Reasoning (MIGR) and Multilateral Reinforcement Graph Reasoning (MRGR). The multi-task framework encodes rich geometric features of lesion shapes and surfaces, decoupling the image into three task-specific feature maps. MIGR and MRGR iteratively reason about higher-order relationships across tasks through a graph mechanism, enabling complementary optimization for task-specific objectives. Additionally, an uncertainty-weighted loss is proposed to mitigate the impact of artifacts and noise on segmentation accuracy. Experimental results demonstrate that MTG-Net outperforms existing methods, achieving a Dice socre of 87.21% for region segmentation and 88.12% for vessel segmentation.

Optical coherence tomography angiography (OCTA) plays a crucial role in quantifying and analyzing retinal vascular diseases. However, the limited field of view (FOV) inherent in most commercial OCTA imaging systems poses a significant challenge for clinicians, restricting the possibility to analyze larger retinal regions of high resolution. Automatic stitching of OCTA scans in adjacent regions may provide a promising solution to extend the region of interest. However, commonly-used stitching algorithms face difficulties in achieving effective alignment due to noise, artifacts and dense vasculature present in OCTA images. To address these challenges, we propose a novel retinal OCTA image stitching network, named -Net, which integrates multi-scale representation learning and dynamic location guidance. In the first stage, an image registration network with a progressive multi-resolution feature fusion is proposed to derive deep semantic information effectively. Additionally, we introduce a dynamic guidance strategy to locate the foveal avascular zone (FAZ) and constrain registration errors in overlapping vascular regions. In the second stage, an image fusion network based on multiple mask constraints and adjacent image aggregation (AIA) strategies is developed to further eliminate the artifacts in the overlapping areas of stitched images, thereby achieving precise vessel alignment. To validate the effectiveness of our method, we conduct a series of experiments on two delicately constructed datasets, i.e., OPTOVUE-OCTA and SVision-OCTA. Experimental results demonstrate that our method outperforms other image stitching methods and effectively generates high-quality wide-field OCTA images, achieving a structural similarity index (SSIM) score of 0.8264 and 0.8014 on the two datasets, respectively.

Optical Coherence Tomography (OCT) imaging is extensively utilized for non-invasive observation of pathological conditions, such as retinal fluid-associated diseases. Accurate fluid segmentation in OCT images is therefore critical for quantifying disease severity and aiding clinical decision-making. However, achieving precise segmentation remains challenging due to pathological variations in shape and size, uncertain boundaries, and low contrast of fluid. Most importantly, variability in OCT image styles across different vendors and centers significantly affects fluid segmentation, leading to poor generalization to unseen domains. To address this, we propose a novel method, RSAPower, to enhance the generalization ability of fluid perception networks via style augmentation for retinal fluid segmentation. Specifically, RSAPower comprises a plug-and-play random style transform augmentation (RSTAug) module and a novel fluid perception network (FLPNet) for end-to-end training. The RSTAug module generates new random-style data from the source domain, preserving realistic pathological and structural features. The FLPNet benefits from a novel hybrid structure attention (HSA) module to perceive fluid's spatial features and long-range dependence. Furthermore, FLPNet adapts to the diverse augmented data through a saliency-guided multi-scale attention (SGMA) block, boosting its segmentation performance. We validate RSAPower against various state-of-the-art methods using two publicly available datasets, Retouch and Kermany. Experimental results demonstrate the proposed method's superior generalization ability and effectiveness in fluid segmentation.

Cerebrovascular diseases (CVDs) remain a leading cause of global disability and mortality. Digital Subtraction Angiography (DSA) sequences, recognized as the gold standard for diagnosing CVDs, can clearly visualize the dynamic flow and reveal pathological conditions within the cerebrovasculature. Therefore, precise segmentation of cerebral arteries (CAs) and classification between their main trunks and branches are crucial for physicians to accurately quantify diseases. However, achieving accurate CA segmentation in DSA sequences remains a challenging task due to small vessels with low contrast, and ambiguity between vessels and residual skull structures. Moreover, the lack of publicly available datasets limits exploration in the field. In this paper, we introduce a DSA Sequence-based Cerebral Artery segmentation dataset (DSCA), the publicly accessible dataset designed specifically for pixel-level semantic segmentation of CAs. Additionally, we propose DSANet, a spatio-temporal network for CA segmentation in DSA sequences. Unlike existing DSA segmentation methods that focus only on a single frame, the proposed DSANet introduces a separate temporal encoding branch to capture dynamic vessel details across multiple frames. To enhance small vessel segmentation and improve vessel connectivity, we design a novel TemporalFormer module to capture global context and correlations among sequential frames. Furthermore, we develop a Spatio-Temporal Fusion (STF) module to effectively integrate spatial and temporal features from the encoder. Extensive experiments demonstrate that DSANet outperforms other state-of-the-art methods in CA segmentation, achieving a Dice of 0.9033.

Geodesic models are known as an efficient tool for solving various image segmentation problems. Most of existing approaches only exploit local pointwise image features to track geodesic paths for delineating the objective boundaries. However, such a segmentation strategy cannot take into account the connectivity of the image edge features, increasing the risk of shortcut problem, especially in the case of complicated scenario. In this work, we introduce a new image segmentation model based on the minimal geodesic framework in conjunction with an adaptive cut-based circular optimal path computation scheme and a graph-based boundary proposals grouping scheme. Specifically, the adaptive cut can disconnect the image domain such that the target contours are imposed to pass through this cut only once. The boundary proposals are comprised of precomputed image edge segments, providing the connectivity information for our segmentation model. These boundary proposals are then incorporated into the proposed image segmentation model, such that the target segmentation contours are made up of a set of selected boundary proposals and the corresponding geodesic paths linking them. Experimental results show that the proposed model indeed outperforms state-of-the-art minimal paths-based image segmentation approaches.

Abstract We address the computation of paths globally minimizing an energy involving their curvature, with given endpoints and tangents at these endpoints, according to models known as the Reeds‐Shepp car (reversible and forward variants), the Euler‐Mumford elasticae, and the Dubins car. For that purpose, we numerically solve degenerate variants of the eikonal equation, on a three‐dimensional domain, in a massively parallel manner on a graphical processing unit. Due to the high anisotropy and nonlinearity of the addressed Partial Differential Equation, the discretization stencil is rather wide, has numerous elements, and is costly to generate, which leads to subtle compromises between computational cost, memory usage, and cache coherency. Accelerations by a factor 30 to 120 are obtained w.r.t a sequential implementation. The efficiency and the robustness of the method is illustrated in various contexts, ranging from motion planning to vessel segmentation and radar configuration.

The minimal geodesic models established upon the eikonal equation framework are capable of finding suitable solutions in various image segmentation scenarios. Existing geodesic-based segmentation approaches usually exploit image features in conjunction with geometric regularization terms, such as euclidean curve length or curvature-penalized length, for computing geodesic curves. In this paper, we take into account a more complicated problem: finding curvature-penalized geodesic paths with a convexity shape prior. We establish new geodesic models relying on the strategy of orientation-lifting, by which a planar curve can be mapped to an high-dimensional orientation-dependent space. The convexity shape prior serves as a constraint for the construction of local geodesic metrics encoding a particular curvature constraint. Then the geodesic distances and the corresponding closed geodesic paths in the orientation-lifted space can be efficiently computed through state-of-the-art Hamiltonian fast marching method. In addition, we apply the proposed geodesic models to the active contours, leading to efficient interactive image segmentation algorithms that preserve the advantages of convexity shape prior and curvature penalization.

The major task of reflection removal methods is to restore a reflection-free image from a reflection-contaminated image taken through glass. We propose an algorithm to remove reflections from a single image by means of the$l_{0}$-regularized dark channel sparsity prior and an$l_{0}$gradient sparsity prior. In addition, we analyze the difference between the dark channel map in the reflection-contaminated image and the reflection-free image empirically and mathematically. Moreover, a new data fidelity term is introduced to handle strong reflections and preserve high-frequency details in the recovered transmission image. Different from the model used in most state-of-the-art methods, our reflection removal model does not rely on the assumption of out-of-focus objects in the reflection layer. Quantitative evaluation on several publicly available real-world image datasets including ground-truth demonstrates the high accuracy of our algorithm. Qualitative evaluation of extensive experimental results on real-world images shows the competitive performance of the proposed method compared with the state-of-the-art reflection removal methods.

Abstract-domain adaptation action recognition is a hot research topic in machine learning and some effective approaches have been proposed. However, samples in the target domain with label information are often required by these approaches. Moreover, domain-invariant discriminative feature learning, feature fusion, and classifier module learning have not been explored in an end-to-end framework. Thus, in this study, we propose a novel end-to-end multiple-view adversarial learning network (MAN) for unsupervised domain adaptation action recognition in which the fusion of RGB and optical-flow features, domain-invariant discrimination feature learning, and action recognition is conducted in a unified framework. Specifically, a robust spatiotemporal feature extraction network, including a spatial transform network and an adaptive intrachannel weight network, is proposed to improve the scale invariance and robustness of the method. Then, a self-attention mechanism fusion module is designed to adaptively fuse the RGB and optical-flow features. Moreover, a multiview adversarial learning loss is developed to obtain domain-invariant discriminative features. In addition, three benchmark datasets are constructed for unsupervised domain adaptation action recognition, for which all actions and samples are carefully collected from public action datasets, and their action categories are hierarchically augmented, which can guide how to extend existing action datasets. We conduct extensive experiments on four benchmark datasets, and the experimental results demonstrate that our proposed MAN can outperform several state-of-the-art unsupervised domain adaptation action recognition approaches. When the SDAI Action II-6 and SDAI Action II-11 datasets are used, MAN can achieve 3.7% ( H → U ) and 6.1% ( H → U ) improvements over the temporal attentive adversarial adaptation network (published in ICCV 2019) module, respectively. As an added contribution, the SDAI Action II-6, SDAI Action II-11, and SDAI Action II-16 datasets will be released to facilitate future research on domain adaptation action recognition.

Tubular structure tracking is a crucial task in the fields of computer vision and medical image analysis. The minimal paths-based approaches have exhibited their strong ability in tracing tubular structures, by which a tubular structure can be naturally modeled as a minimal geodesic path computed with a suitable geodesic metric. However, existing minimal paths-based tracing approaches still suffer from difficulties such as the shortcuts and short branches combination problems, especially when dealing with the images involving complicated tubular tree structures or background. In this paper, we introduce a new minimal paths-based model for minimally interactive tubular structure centerline extraction in conjunction with a perceptual grouping scheme. Basically, we take into account the prescribed tubular trajectories and curvature-penalized geodesic paths to seek suitable shortest paths. The proposed approach can benefit from the local smoothness prior on tubular structures and the global optimality of the used graph-based path searching scheme. Experimental results on both synthetic and real images prove that the proposed model indeed obtains outperformance comparing with the state-of-the-art minimal paths-based tubular structure tracing algorithms.

In this paper, we propose a new minimal path-based framework for minimally interactive tubular structure tracking in conjunction with a perceptual grouping scheme. The minimal path models have shown great advantages in tubular structures tracing. However, they suffer from shortcuts or short branches combination problems especially in the case of tubular network with complicated structures or background. Thus, we utilize the curvature-penalized minimal paths and the prescribed tubular trajectories to seek the desired shortest path. The proposed approach benefits from the local smoothness prior on tubular structures and the global optimality of the graph-based path searching scheme. Experimental results on synthetic and real images prove that the proposed model indeed obtains outperformance to state-of-the-art minimal path-based algorithms.

The minimal geodesic models based on the Eikonal equations are capable of finding suitable solutions in various image segmentation scenarios. Existing geodesic-based segmentation approaches usually exploit the image features in conjunction with geometric regularization terms (such as curve length or elastica length) for computing geodesic paths. In this paper, we consider a more complicated problem: finding simple and closed geodesic curves which are imposed a convexity shape prior. The proposed approach relies on an orientation-lifting strategy, by which a planar curve can be mapped to an high-dimensional orientation space. The convexity shape prior serves as a constraint for the construction of local metrics. The geodesic curves in the lifted space then can be efficiently computed through the fast marching method. In addition, we introduce a way to incorporate region-based homogeneity features into the proposed geodesic model so as to solve the region-based segmentation issues with shape prior constraints.

The Voronoi diagram-based dual-front scheme is known as a powerful and efficient technique for addressing the image segmentation and domain partitioning problems. In the basic formulation of existing dual-front approaches, the evolving contour can be considered as the interfaces of adjacent Voronoi regions. Among these dual-front models, a crucial ingredient is regarded as the geodesic metrics by which the geodesic distances and the corresponding Voronoi diagram can be estimated. In this paper, we introduce a new dual-front model based on asymmetric quadratic metrics. These metrics considered are built by the integration of the image features and a vector field derived from the evolving contour. The use of the asymmetry enhancement can reduce the risk for the segmentation contours being stuck at false positions, especially when the initial curves are far away from the target boundaries or the images have complicated intensity distributions. Moreover, the proposed dual-front model can be applied for image segmentation in conjunction with various region-based homogeneity terms. The numerical experiments on both synthetic and real images show that the proposed dual-front model indeed achieves encouraging results.

Minimal paths are regarded as a powerful and efficient tool for boundary detection and image segmentation due to its global optimality and the well-established numerical solutions such as fast marching method. In this paper, we introduce a flexible interactive image segmentation model based on the Eikonal partial differential equation (PDE) framework in conjunction with region-based homogeneity enhancement. A key ingredient in the introduced model is the construction of local geodesic metrics, which are capable of integrating anisotropic and asymmetric edge features, implicit region-based homogeneity features and/or curvature regularization. The incorporation of the region-based homogeneity features into the metrics considered relies on an implicit representation of these features, which is one of the contributions of this work. Moreover, we also introduce a way to build simple closed contours as the concatenation of two disjoint open curves. Experimental results prove that the proposed model indeed outperforms state-of-the-art minimal paths-based image segmentation approaches.

The major task of traditional motion deblurring methods is to estimate the blur kernel and restore the latent image. In low-light conditions, the pointolite is likely to produce saturated light streaks in captured blurred images. The light streaks are usually double-edged swords—outliers to the deconvolution, but a cue to kernel estimation. In this paper, we propose a novel blind motion deblurring method for blurred images including light streaks. The main idea is to model the non-linear blur caused by outliers as the Huber's M-estimation in blind deconvolution and take the shape of the light streak as a cue to estimate the blur kernel. Specifically, the optimal light streak patch is selected automatically according to the characteristics of light streaks and the blur kernel. This simple yet effective selection strategy solves the problems of false detection of candidate light streaks and optimal light streak in existing methods. Then, the optimal light streak patch is parameterized as a prior and is combined with other regularizers to estimate the blur kernel. Compared with the state-of-the-art kernel estimation methods, the proposed algorithm reduces the influence of outliers on deconvolution and utilizes more information. Thus, the restored image is more accurate. Experimental results on both synthetic and real images demonstrate the high accuracy of our algorithm.

The minimal path method has proven to be particularly useful and efficient in tubular structure segmentation applications. In this paper, we propose a new minimal path model associated with a dynamic Riemannian metric embedded with an appearance feature coherence penalty and an adaptive anisotropy enhancement term. The features that characterize the appearance and anisotropy properties of a tubular structure are extracted through the associated orientation score. The proposed the dynamic Riemannian metric is updated in the course of the geodesic distance computation carried out by the efficient single-pass fast marching method. Compared to the state-of-the-art minimal path models, the proposed minimal path model is able to extract the desired tubular structures from a complicated vessel tree structure. In addition, we propose an efficient prior path-based method to search for vessel radius value at each centerline position of the target. Finally, we perform the numerical experiments on both synthetic and real images. The quantitive validation is carried out on retinal vessel images. The results indicate that the proposed model indeed achieves a promising performance.

We propose a new dynamic Riemannian metric with adaptive anisotropy enhancement and with appearance feature coherence penalization. The appearance features are characterized by the orientation score maps. Unlike the static geodesic metrics which depend on local pointwise information, the dynamic metric can take into account the nonlocal feature coherence penalty in order to extract a desired structure from complicated background or from a vessel tree. We construct the metric using the information from two external reference points which are identified during the geodesic distance computation. Numerical experiments are performed in retinal vessels, including the independent results from the proposed dynamic metric itself and the comparison against existing minimal path models. The results show that the proposed metric indeed gets better performance than state-of-the-art geodesic metrics.

In this paper, we propose a new curvature penalized minimal path model for image segmentation via closed contour detection based on the weighted Euler elastica curves, firstly introduced to the field of computer vision in [22]. Our image segmentation method extracts a collection of curvature penalized minimal geodesics, concatenated to form a closed contour, by connecting a set of user-specified points. Globally optimal minimal paths can be computed by solving an Eikonal equation. This first order PDE is traditionally regarded as unable to penalize curvature, which is related to the path acceleration in active contour models. We introduce here a new approach that enables finding a global minimum of the geodesic energy including a curvature term. We achieve this through the use of a novel Finsler metric adding to the image domain the orientation as an extra space dimension. This metric is non-Riemannian and asymmetric, defined on an orientation lifted space, incorporating the curvature penalty in the geodesic energy. Experiments show that the proposed Finsler minimal path model indeed outperforms state-of-the-art minimal path models in both synthetic and real images.

International audience

Geodesic methods have been widely applied to image analysis [1]. They are particularly efficient to extract a tubular structure, such as a blood vessel, given its two endpoints in a 2D or 3D medical image [2]. We address here a more difficult problem: the extraction of a full vessel tree structure given a single initial root, by growing a collection of keypoints, connected by geodesic minimal paths as in [3]. Keypoints are iteratively added, using selection criteria which compare geodesic distances with the standard euclidean curve length and a path score. A weakness of existing approaches is that the geodesic length and the euclidean path length are locally proportional, due to the use of an isotropic geodesic potential P(x). In contrast, we use an anisotropic geodesic potential P(x, v), and develop new criteria for selecting keypoints and stopping the tree growth. Experimental results demonstrate that our method can extract vessel structures at a finer scale, with increased accuracy.