Object detection and instance segmentation are two fundamental computer vision tasks. They are closely correlated but their relations have not yet been fully explored in most previous work. This paper presents RDSNet, a novel deep architecture for reciprocal object detection and instance segmentation. To reciprocate these two tasks, we design a two-stream structure to learn features on both the object level (i.e., bounding boxes) and the pixel level (i.e., instance masks) jointly. Within this structure, information from the two streams is fused alternately, namely information on the object level introduces the awareness of instance and translation variance to the pixel level, and information on the pixel level refines the localization accuracy of objects on the object level in return. Specifically, a correlation module and a cropping module are proposed to yield instance masks, as well as a mask based boundary refinement module for more accurate bounding boxes. Extensive experimental analyses and comparisons on the COCO dataset demonstrate the effectiveness and efficiency of RDSNet. The source code is available at https://github.com/wangsr126/RDSNet.

Letting a deep network be aware of the quality of its own predictions is an interesting yet important problem. In the task of instance segmentation, the confidence of instance classification is used as mask quality score in most instance segmentation frameworks. However, the mask quality, quantified as the IoU between the instance mask and its ground truth, is usually not well correlated with classification score. In this paper, we study this problem and propose Mask Scoring R-CNN which contains a network block to learn the quality of the predicted instance masks. The proposed network block takes the instance feature and the corresponding predicted mask together to regress the mask IoU. The mask scoring strategy calibrates the misalignment between mask quality and mask score, and improves instance segmentation performance by prioritizing more accurate mask predictions during COCO AP evaluation. By extensive evaluations on the COCO dataset, Mask Scoring R-CNN brings consistent and noticeable gain with different models and outperforms the state-of-the-art Mask R-CNN. We hope our simple and effective approach will provide a new direction for improving instance segmentation. The source code of our method is available at \url{https://github.com/zjhuang22/maskscoring_rcnn}.

Random walks (RW) is a popular technique for object segmentation. Apart from the satisfactory performance in various applications, its most appealing advantage is the computational efficiency. However, RW often fails to produce complete and connected results in fine-structured (FS) object segmentation. To utilize the high efficiency and overcome the drawbacks in tackling FS objects, we develop a novel approach within the RW framework. Specifically, we propose to introduce labeling preference learned from the image data into the RW model to guide the propagation of random walkers. With the help of the guidance, random walkers are more likely to propagate correctly to the FS regions, thus yielding more accurate results. Similar to RW, this approach also bears properties such as computational efficiency, closed-form solution and unique global optimum. Moreover, it has the capacities of handling disconnected objects and transferring segmentation. Comparative experimental results demonstrate that the proposed approach achieves the state-of-the-art performance in FS object segmentation, with a low requirement of runtime.

Fine-structured object segmentation is a challenging problem in object segmentation community. There are mainly two difficulties that can seriously degrade the segmentation quality: 1) insufficient interactions on fine structures due to the high demand of time and manual efforts, and 2) shrinking bias that discourages long object boundaries. To address these two issues, we develop a novel method within the graph cut framework. First, the commonly used operation of scribbling or dragging bounding boxes is replaced by loosely drawing a few rectangles, thus the interaction burden is largely reduced. Second, an edge-guided graph cut model is proposed to mitigate shrinking bias. This model enforces connectivity of fine structures by adjusting the weighting between neighboring pixels. Finally, the segmentation task is formulated as an optimization problem, which can be optimized effectively and efficiently. Comparative experimental results demonstrate the effectiveness of our method.

In this paper, we present a novel method for the challenging task of fine-structured (FS) object segmentation. This task is formulated as a label propagation problem on an affinity graph. To enhance the completeness and connectivity of the FS objects, we introduce a novel neighborhood system combining both local and nonlocal connections, together with a robust scheme for edge weight calculation. Additionally, region cost is incorporated into the energy function to further maintain the connectivity of fine parts where the propagation is hard to reach. An appealing advantage of the proposed method is that the energy minimization has a closed-form solution and global optimum is guaranteed. Comparative experimental results on three datasets demonstrate the effectiveness of the proposed method.

In this paper, we propose a graph cuts (GC) based probability propagation approach to automatically extract road network from complex remote sensing images. First, the support vector machine (SVM) classifier with a sigmoid model is applied to assign each pixel a posterior probability of being labelled as road class, which avoids the weaknesses of hard labels in general SVM. Then a GC based probability propagation algorithm is employed to keep the extracted road results smooth and coherent, which can reduce the connections between roads and road-like objects. Finally, a road-geometrical prior is considered to refine the extraction result, so that the non-road objects in images can be removed. Experimental results on two remote sensing image datasets indicate the validity and effectiveness of our method by comparing with two other approaches.