The rapid advancement of AI generation technologies has led to the widespread use of AI-generated multimedia content, including images, videos, and 3D contents, across various applications. While significant progress has been made in quality evaluation for 2D content, evaluating the quality of 3D content synthesized from single image remains an underexplored problem. To bridge this gap, we introduce the first comprehensive subjective evaluation database tailored for assessing the quality of 3D content generated from single image. Our database, named AIGC-SI23DCQA, includes three distinct categories of input images, i.e., realistic images, AI-generated images, and computer graphic (CG) images, with 100 images in each category. Using five representative single-image-to-3D algorithms, we produce 1,500 3D contents and collect 94,500 annotations across three quality dimensions, including texture fidelity, shape accuracy, and overall quality. Based on the constructed database, we first benchmark and evaluate the performance of existing quality assessment methods revealing their limitations in addressing this novel task. Thus, we further propose a novel objective quality assessment method, termed I3DQA, for effective single-image-to-3D content quality assessment. Specifically, I3DQA first extracts the reference features from the source image, and the multi-modal features from the generated 3D content, including the projected video, patches, and large-multimodal model (LMM) features. These features are integrated through symmetric transformer blocks, enabling effective quality-related feature fusion and score prediction. Extensive experiments demonstrate the superior performance of our method and validate the effectiveness of its components. This work provides a foundational resource and a robust framework for advancing research in this emerging field, and our database and model are released at https://github.com/ZedFu/SI23DCQA.

Stereoscopic omnidirectional image (SOI) has both omnidirectional and stereoscopic perception features. Many previous models have proved the viewport characteristics and stereoscopic visual features are crucial for quality perception of SOI. However, effective monocular and binocular visual features extraction and fusion are difficult due to the size of SOI and inaccuracy of feature representation. In this paper, we proposed a three-branch feature fusion network (TFFN) by fusing two-stream binocular visual features and the important monocular features based on the viewport perspective. The hierarchical fusion module is first designed to fuse effective binocular visual features from different semantic scales, and the pseudo-difference information extraction module is built to obtain the accuracy monocular visual features to complement the binocular visual features. Finally, the above monocular and binocular visual features are fused together to measure the quality of SOI. The comparison experiments are conducted on three public datasets and the analysis of the results demonstrate the effectiveness of the proposed method.

In recent years, colorectal polyp segmentation has attracted increasing attention in academia and industry. Although most existing methods can achieve commendable outcomes, they often confront difficulty when localizing challenging polyps with complex background, variable shape/size, and ambiguous boundary, because of the limitations in modeling global context and in cross-layer feature interaction. To cope with these challenges, this paper proposes a novel Progressive Feature Enhancement Network (PFENet) for polyp segmentation. Specifically, PFENet follows an encoder-decoder structure and utilizes the pyramid vision transformer as the encoder to capture multi-scale long-term dependencies at different stages. A cross-stage feature enhancement (CFE) module is embedded in each stage. The CFE module enhances the feature representation ability from interaction among adjacent stages, which helps integrate scale information for recognizing polyps with complex background and variable shape/size. In addition, a foreground boundary co-enhancement (FBC) module is used at each decoder to simultaneously enhance the foreground and boundary information by incorporating the output of the adjacent high stage and the coarse segmentation map, which is generated by fusing features of all four stages via a coarse map generation module. Through top-down connections of FBC modules, PFENet can progressively refine the prediction in a coarse-to-fine manner. Extensive experiments show the effectiveness of our PFENet in the polyp segmentation task, with the mIoU and mDic values over 0.886 and 0.931 tested on two in-domain datasets and over 0.735 and 0.809 tested on three out-of-domain datasets.Note to Practitioners—Automated and accurate polyp segmentation in colonoscopy images is a critical prerequisite for subsequent detection, removal, and diagnosis of polyps in clinical practice. This paper proposes a novel deep neural network for polyp segmentation, termed PFENet, with a CFE module to enhance the feature representation ability for better capturing polyps with complex background and variable shape/size, and a FBC module to simultaneously enhance the foreground and boundary information on the feature representation provided by the CFE module. Qualitative and quantitative results on five public datasets show that our PFENet yields accurate predictions and is superior to 9 state-of-the-art polyp segmentation methods. The proposed PFENet will facilitate potential computer-aided diagnosis systems in clinical practice, in which it can better promote medical decision-making than competing methods in polyp detection and removal.

Omnidirectional image quality assessment (OIQA) has become an increasingly vital problem in recent years. Most previous no-reference OIQA methods only extract local features from the distorted viewports, or extract global features from the entire distorted image, lacking the interaction and fusion between local and global features. Moreover, the lack of reference information also limits their performance. Thus, we propose a no-reference OIQA model which consists of three novel modules, including a bidirectional pseudo-reference module, a Mamba-based global feature extraction module, and a multi-scale local-global feature aggregation module. Specifically, by considering the image distortion degradation process, a bidirectional pseudo-reference module capturing the error maps on viewports is first constructed to refine the multi-scale local visual features, which can supply rich quality degradation reference information without the reference image. To well complement the local features, the VMamba module is adopted to extract the representative multi-scale global visual features. Inspired by human hierarchical visual perception characteristics, a novel multi-scale aggregation module is built to strengthen the feature interaction and effective fusion which can extract deep semantic information. Finally, motivated by the multi-task managing mechanism of human brain, a multi-task learning module is introduced to assist the main quality assessment task by digging the hidden information in compression type and distortion degree. Extensive experimental results demonstrate that our proposed method achieves the state-of-the-art performance on the no-reference OIQA task compared to other models.

Compared to ordinary images, omnidirectional image (OI) usually has a broader view and a higher resolution, and image quality assessment (IQA) can help people to understand and improve their visual experience. However, the current IQA works cannot achieve good performance. To address this, we proposed a novel visual perception-based no-reference/blind omnidirectional image quality assessment (NR/B-OIQA) model. The gradient-based global structural features and gray-level co-occurrence matrix-based local structural features are combined together to highlight the rich quality-aware structural information. And a novel steganalysis real model-based color descriptor is extracted to reflect the color information that ignored in most IQA models. With a multi-scale visual perception, we take image entropy and the natural scene statistics features to convey the high-level semantics and quantify the unnaturalness of omnidirectional images. Finally, we apply support vector regression to predict the objective quality value based on the subjective scores and extracted all features. Experiments are conducted on OIQA and CVIQD2018 Databases, and the results illustrate that our model has more reliable performance and stronger competitiveness and receives better conformity with the subjective values.