Capturing sufficient global context and rich spatial structure information is critical for dense prediction tasks. Convolutional Neural Network (CNN) is particularly adept at modeling fine-grained local features, while Transformer excels at modeling global context information. It is evident that CNN and Transformer exhibit complementary characteristics. Exploring the design of a network, that efficiently fuses these two models to leverage their strengths fully and achieve more accurate detection, represents a promising and worthwhile research topic. In this paper, we introduce a novel CNN-Transformer Iterative Fusion Network (CTIF-Net) for salient object detection. It efficiently combines CNN and Transformer to achieve superior performance by using a parallel dual encoder structure and a feature iterative fusion module. Firstly, CTIF-Net extracts features from the image using the CNN and the Transformer, respectively. Secondly, two feature convertors and a feature iterative fusion module are employed to combine and iteratively refine the two sets of features. The experimental results on multiple SOD datasets show that CTIF-Net outperforms 17 state-of-the-art methods, achieving higher performance in various mainstream evaluation metrics such as F-measure, S-measure, and MAE value. The code will be publicly available.

Abstract In clinical medicine, the segmentation of blood vessels in retinal images is essential for subsequent analysis in clinical diagnosis. However, retinal images are often noisy and their vascular structure is relatively tiny, which poses significant challenges for vessel segmentation. To improve the performance of vessel segmentation, an improved model IterNet++ based on the architecture of IterNet is proposed. First, curvelet signal analysis is applied to enhance retinal images. Second, residual convolution (ResConv) blocks and guided filters are introduced to utilise the encoder features of previous iterations in the model to reduce overfitting. Third, offline hard‐sample mining is used to improve segmentation performance by utilising training samples with low segmentation accuracy as many possible on a few‐sample training set. In addition, a test‐time augmentation method is applied to testing samples in test dataset during inference. Extensive experiments show that this model achieves Dice scores of 0.8313, 0.8277, and 0.8372 on DRIVE, CHASE‐DB1, and STARE datasets, respectively, demonstrating the best performance compared with IterNet and other baseline models.

Grammatical Error Correction (GEC) is a challenge in Natural Language Processing research. Although many researchers have been focusing on GEC in universal languages such as English or Chinese, few studies focus on Indonesian, which is a low-resource language. In this article, we proposed a GEC framework that has the potential to be a baseline method for Indonesian GEC tasks. This framework treats GEC as a multi-classification task. It integrates different language embedding models and deep learning models to correct 10 types of Part of Speech (POS) error in Indonesian text. In addition, we constructed an Indonesian corpus that can be utilized as an evaluation dataset for Indonesian GEC research. Our framework was evaluated on this dataset. Results showed that the Long Short-Term Memory model based on word-embedding achieved the best performance. Its overall macro-average F 0.5 in correcting 10 POS error types reached 0.551. Results also showed that the framework can be trained on a low-resource dataset.

Vertical retargeting for stereoscopic images using seam manipulation-based approaches has remained an open challenge over the years. Even though horizontal retargeting had attracted a huge amount of interest, its seam coupling strategies were not capable to construct valid seam pairs for vertical retargeting. In this article, we propose two seam coupling strategies for vertical retargeting, namely, real mapping and virtual mapping. Our proposed mapping strategies were implemented to address the problems of multiple assignments and missing assignments, which are able to occur in the straightforward generalization from horizontal retargeting to vertical retargeting. On the basis of our proposed method, stereo seams were allowed to lay across occluded regions and occluding regions in stereo images. We maintained the geometric consistency by removing occluded pixels and corresponding occluding pixels in both stereo images. As a result, our method guarantees valid and geometrically consistent stereo seam pairs to be found in the horizontal direction. We generate vertically retargeted stereo images by removing or adding horizontal seam pairs iteratively. We conducted experiments on a number of indoor and outdoor scenes. Experimental results demonstrated that our method overcomes the limitations of vertical retargeting and is effective in preserving the geometric consistency.

To implement vertical retargeting for stereoscopic images, this paper proposes an occlusion-guided stereoscopic image retargeting method via the pixel fusion technique. Traditional seam searching-based methods cannot construct valid horizontal seam pairs due to the existence of occluded regions in stereoscopic images and thus fail to implement vertical retargeting. To solve this issue, we propose a novel horizontal seam coupling strategy guided by the occlusion regions that appear on both sides of stereoscopic images. Horizontal seams were able to be laid across the occluded and occluding regions with their geometric consistency maintained. Another important contribution of our method is incorporating occluding masks into energy optimization. The experimental results show that our method can achieve promising performances in both visual experience and depth preservation.

Retrieving 3D shapes from 2D human sketches has received increasing attention in computer vision and computer graphics. Most previous methods projected 3D shapes from numerous viewpoints and then extracted features of 3D shapes from these projections and calculated the similarity with sketches. However, due to the unknown pose of 3D shapes, viewpoints were usually sampled uniformly from a sphere coordinate. Hence, some projections acquired insufficient descriptions of 3D shapes. In this paper, we proposed a view selection algorithm to find the most reasonable viewpoints, which can benefit representation learning for 3D shapes. Additionally, to indicate the apparent discrepancy between sketches and 3D shapes, we leveraged a generalized similarity model to encourage the accuracy of cross-domain feature matching. We first computed line renderings of 3D shapes from an enormous number of viewpoints. Then, we calculated the similarity of shapes between line renderings and sketches. In this vein, we obtained several superior projections. Second, we implemented a sketch network to extract features of the sketch and a shape network to extract features of projections. We combined the features of different projections to secure the compact representation for 3D shapes. Finally, a metric network was constructed using a cross-domain similarity model, and we trained the metric network with triplet loss. Online hard sample mining was leveraged to accelerate the convergence of the network. We evaluated our method on SHREC‘13 and SHREC’14 sketch track benchmark datasets. The experimental results demonstrated that both view selection and cross-domain similarity models were able to encourage retrieval performance.

Stereo image retargeting plays a significant role in the field of image processing, which aims at making major objects as prominent as possible when the resolution of an image is changed, including maintaining disparity and depth information at the same time. Some seam carving methods are proposed to preserve the geometric consistency of the images. However, the regions of occlusion on both sides are not considered properly. In this article, we propose a solution to solve this problem. A new strategy of seams finding is designed by considering occluded and occluding regions on both of the input images, and leaving geometric consistency in both images intact. We also introduced the method of line segment detection and superpixel segmentation to further improve the quality of the images. Imaging effects are optimized in the process and visual comfort, which is also influenced by other factors, can be boosted as well.

Dynamic texture describes images sequence that continuously demonstrates movement of pixels intensity change patterns in time, for example, smoke, fire, waterfall, sea-waves, foliage, traffic on highway and so on. Motion coherence analysis on dynamic textures is usually observed through their motion vector fields. We implement strategic motion coherence analysis to evaluate the coherent motion on dynamic texture. In this article, motion coherence index is proposed as a metric to evaluate the coherent motion. It represents corresponding angles quantity of vectors on the motion vector field. The coherent motion in a video frame will be calculated and accumulated from every four adjacent vectors. The average of the accumulation represents the motion coherence index of dynamic texture in the video frame. The coherence index is utilized as an attribute to distinguish the difference of dynamic textures. We propose a new technique to distinguish dynamic texture by investigating the covariance stationary series of motion coherence index. Our investigation shows that, there are significant differences among the series of motion coherence index of dynamic textures. For instance, dynamic fire texture and dynamic smoke texture can be distinguished by the covariance stationary and covariance nonstationary of motion coherence index series respectively.

Changing dynamic texture appearances can create new looks in both the motion and color appearances of videos. Dynamic textures with sophisticated shape and motion appearance are difficult to represent by physical models and are difficult to predict, especially for transformation to new motion texture. We propose a dynamic texture transformation algorithm for the video sequences based on the motion coherence of patches. We successfully apply the technology in many special effect videos, using the interactive tool we developed. In this paper, we address the issues of 3-D patch creation, motion coherence analysis, and patch matching for the dynamic texture transformation. The main contribution includes two issues. First, we propose a new metric for evaluating motion coherence with solid tests to justify the usefulness (close to human eye perception). Second, the proposed algorithm for an automatic dynamic texture transformation only needs users to segment textures on the first frame using an optional threshold to identify the texture area. The rest of the process is completed automatically. The experimental results show that the motion coherence index is effectively used to find the coherent motion region for patch matching and transformation. The experimental results, test data, source code, and system demonstration videos are posted at http://video.minelab.tw/DTT/index.html.

Content-aware image retargeting has been investigated since the last decade as a paradigm of image modification for proper display on the different screen sizes. Modifications, such as seam carving or seam insertion, have been introduced to achieve aforesaid image retargeting. The changes in an image are not easily recognizable by human eyes. Inspired by the blocking artifact characteristics matrix (BACM), a method to detect tampers caused by seam modification on JPEG retargeted images without knowledge of the original image is proposed in this paper. In a BACM block matrix, we found that the original JPEG image demonstrates a regular symmetrical data, whereas the symmetrical data in a block reconstructed by seam modification is destroyed. Twenty-two features are proposed to train the data using a support vector machine classification method. The experimental results clearly demonstrate that the proposed method provides a very high recognition rate for those JPEG retargeted images. The source codes and the complete experimental data can be accessed at http://video.minelab.tw/DETS/.

In this paper, we demonstrate a new motion in painting technique to allow users to change the dynamic texture used in a video background for special effect production. For instance, the dynamic texture of fire, smoke, water, cloud, and others can be edited through a series of automatic algorithms. Motion estimations of global and local textures are used. Video blending techniques are used in conjunction with a color balancing technique. The editing procedure will search for suitable patches in irregular shape blocks, to reproduce a realistic dynamic background, such as large waterfall, fire scene, or smoky background. The technique is suitable for making science fiction movies. We demonstrate the original and the falsified videos in our website at http://163.13.127.36/www/AINA12. Although video falsifying may create a moral problem, our intension is to create special effects in movie industry.