Recent advances in interactive image segmentation focused on eliminating the user bias during evaluation by simulating their behavior using robot users. However, these robots only work for region-based methods, excluding the important class of approaches that rely on the boundary-tracking paradigm. We propose completely novel robot users that are able to simulate the human user behavior when segmenting an image through the addition of anchor points close to the object's boundary. Our robots constantly evaluate the optimum-boundary segment being computed from a previously selected anchor point to the current virtual mouse position, seeking for the longest possible segment with minimum acceptable error. A new anchor is added when the error is too high and the robots iterate until closing the contour, just like real users. We validate our robots by conducting a user study and extensive experiments, considering two boundary-tracking methods: live-wire-on-the-fly and Riverbed. We further show how our robot users can be used to assess hybrid approaches that combine boundary-tracking with region-based delineation, such as LiveMarkers, while conjecturing that robots might lead to new methods for automatic foreground segmentation.

Interactive image segmentation methods normally rely on cues about the foreground imposed by the user as region constraints (markers/brush strokes) or boundary constraints (anchor points). These paradigms often have complementary strengths and weaknesses, which can be addressed to improve the interactive experience by reducing the user’s effort. We propose a novel hybrid paradigm based on a new form of interaction called live markers, where optimum boundary-tracking segments are turned into internal and external markers for region-based delineation to effectively extract the object. We present four techniques within this paradigm: 1) LiveMarkers; 2) RiverCut; 3) LiveCut; and 4) RiverMarkers. The homonym LiveMarkers couples boundary-tracking via live-wire-on-the-fly (LWOF) with optimum seed competition by the image foresting transform (IFT-SC). The IFT-SC can cope with complex object silhouettes, but presents a leaking problem on weaker parts of the boundary that is solved by the effective live markers produced by LWOF. Conversely, in RiverCut, the long boundary segments computed by Riverbed around complex shapes provide markers for Graph Cuts by the Min-Cut/Max-Flow algorithm (GCMF) to complete segmentation on poorly defined sections of the object’s border. LiveCut and RiverMarkers further demonstrate that live markers can improve segmentation even when the combined approaches are not complementary (e.g., GCMFs shrinking bias is also dramatically prevented when using it with LWOF). Moreover, since delineation is always region based, our methodology subsumes both paradigms, representing a new way of extending boundary tracking to the 3D image domain, while speeding up the addition of markers close to the object’s boundary-a necessary but time consuming task when done manually. We justify our claims through an extensive experimental evaluation on natural and medical images data sets, using recently proposed robot users for boundary-tracking methods.

We have developed interactive tools for graph-based segmentation of natural images, in which the user guides object delineation by drawing strokes (markers) inside and outside the object. A suitable arc-weight estimation is paramount to minimize user time and maximize segmentation accuracy in these tools. However, it depends on discriminative image properties for object and background. These properties can be obtained from some marker pixels, but their identification is a hard problem during delineation. Careless arc-weight re-estimation reduces user control and drops performance, while interactive arc-weight estimation in a step before interactive object extraction is the best option so far, albeit it is not intuitive for nonexpert users. We present an effective solution using the unified framework of the image foresting transform (IFT) with three operators: clustering for interpreting user interaction and determining when and where arc weights need to be re-estimated; fuzzy classification for arc-weight estimation; and marker competition based on optimum connectivity for object extraction. For validation, we compared the proposed approach with another interactive IFT-based method, which computes arc weights before extraction. Evaluation involved multiple users (experts and nonexperts), a dataset with several natural images, and measurements to quantify accuracy, precision, efficiency (user time and computation time), and user control, being some of them novel measurements, proposed in this work.

This paper presents an optimum user-steered boundary tracking approach for image segmentation, which simulates the behavior of water flowing through a riverbed. The riverbed approach was devised using the image foresting transform with a never-exploited connectivity function. We analyze its properties in the derived image graphs and discuss its theoretical relation with other popular methods such as live wire and graph cuts. Several experiments show that riverbed can significantly reduce the number of user interactions (anchor points), as compared to live wire for objects with complex shapes. This paper also includes a discussion about how to combine different methods in order to take advantage of their complementary strengths.

This work presents an optimum user-steered boundary tracking approach for image segmentation, which simulates the behavior of water flowing through a riverbed. The riverbed approach was devised using the Image Foresting Transform with a never exploited connectivity function. We analyze its properties in the derived image graphs and discuss its theoretical relation with other popular methods, such as live wire and graph cuts. Riverbed can significantly reduce the number of user interactions (anchor points) as compared to live wire for objects with complex shapes.