Dormant tree pruning is laborintensive but essential to maintaining modern highly-productive fruit orchards. In this work we present a closed-loop visuomotor controller for robotic pruning. The controller guides the cutter through a cluttered tree environment to reach a specified cut point and ensures the cutters are perpendicular to the branch. We train the controller using a novel orchard simulation that captures the geometric distribution of branches in a target apple orchard configuration. Unlike traditional methods requiring full 3D reconstruction, our controller uses just optical flow images from a wrist-mounted camera. We deploy our learned policy in simulation and the real-world for an example V-Trellis envy tree with zero-shot transfer, achieving a ~30% success rate - approximately half the performance of an oracle planner.

Advancing robotic grasping and manipulation requires the ability to test algorithms and/or train learning models on large numbers of grasps. Towards the goal of more advanced grasping, we present the Grasp Reset Mechanism (GRM), a fully automated apparatus for conducting large-scale grasping trials. The GRM automates the process of resetting a grasping environment, repeatably placing an object in a fixed location and controllable 1-D orientation. It also collects data and swaps between multiple objects enabling robust dataset collection with no human intervention. We also present a standardized state machine interface for control, which allows for integration of most manipulators with minimal effort. In addition to the physical design and corresponding software, we include a dataset of 1,020 grasps. The grasps were created with a Kinova Gen3 robot arm and Robotiq 2F-85 Adaptive Gripper to enable training of learning models and to demonstrate the capabilities of the GRM. The dataset includes ranges of grasps conducted across four objects and a variety of orientations. Manipulator states, object pose, video, and grasp success data are provided for every trial.

We present and evaluate a novel suction-based gripper designed for fruit picking. This work is motivated by common problems observed in field trials of robotic harvesting: Calibration/perception errors, workspace obstacles, fruit swinging/moving when contacted, and varying stem and branch stiffnesses. The gripper consists of three suction-cups located on the palm, along with in-hand perception. To evaluate the gripper, we developed a physical proxy that approximates a realistic apple-stem-branch dynamic system. We performed 756 apple picks on the proxy with varying branch stiffness, stem strength and gripper pose (yaw, roll and offset w.r.t. the apple). Our results show that grasping performance improves when the gripper yaw w.r.t. the apple has two suction cups on the bottom of the apple and one suction cup on top. Even with ±15mm offset, at least two suction cups engaged with the apple 80% of the time, regardless of branch stiffness. Moreover, the gripper withstands ±20mm offset when it approaches the apple near its equator.

With the increasing prevalence of robots in everyday life, there is a growing need for the general public to understand the capabilities and limitations of these technologies. For example, in the case of sidewalk delivery robots, humans ranging from nearby pedestrians to intended end users may miscalculate robot states, plans, and capabilities without this type of intuition. This paper explores the effectiveness of a relatively brief video-based intervention on enhancing public understanding of current delivery robots. The presented study (N = 100) assessed participants’ perceptions of their own knowledge and characteristics of the studied robotic system before and after watching the video. Results indicated significant changes in participants’ confidence levels, robot trust ratings, robot competence ratings, and attachment to the robots. The study can help to inform future techniques for improving the general public’s understanding of day-to-day robotic technologies, including rapid and relatively entertaining tactics like the video compilation considered in this work.

The rapid emergence of food delivery robots in public spaces has raised important questions regarding public perceptions and policy creation. One method for addressing these questions is examining the relationship between delivery robots and the communities they already serve. We assessed our university community’s experiences with, and perceptions of, the Starship Technologies robots (robots that currently operate on campus) using two efforts: analysis of online posting board content related to the robots and interviews of campus community members about the robots. Perspectives captured in the online post analysis tended to be negative, while views tended to be positive in the interview results. At the same time, both results showed differing opinions and complexity; one tension that emerged in both efforts, for example, is the potential of the robots to both benefit and impede disability communities on campus. Further, there were fundamental misunderstandings about what data the robots can and do record. This research can help to inform roboticists and policymakers whose work relates to autonomous robots in public spaces.

This paper introduces a novel interactive approach —Exploratory Experiences— that aims to improve the ability of people to reason about the capabilities and limitations of robotic technology. We focus on two areas: robot navigation and object detection. We evaluate the Exploratory Experiences with a novel approach that measures the participant's ability to predict when the robot will fail, following up with asking the reason and a possible fix. We show that our approach is effective at improving participants' understanding of potential robot navigation failures and that they already have the skills to detect potential object detection failures when presented with the correct stimuli.

Robotic in-hand manipulation increases the capability of robotic hands to interact with the world. The amount of manipulation that a robot is capable of is highly dependent on the design of the robot hand, and previous works have shown success in designing hands to improve performance for different types of grasping and manipulation. In this paper we present a method for designing a fully-actuated planar manipulator that optimizes for specific in-hand motions. We demonstrate that, with the Asterisk Benchmark and a light-weight IK controller, we can translate our results from simulation to the real world with minimal effort and high-fidelity. Using the simulated data (over 4,000 simulated hand-designs) we begin to analyze which features contribute to improved planar manipulation.

This paper explores whether experiential knowledge of computer vision from interacting with daily apps (e.g., Instagram, Zoom, etc.) can be leveraged to improve users’ expectations of robotic capabilities. We evaluate users’ ability to predict when computer vision apps might fail and if they can apply their experience to reason about computer vision in robotic systems. We show that although users can reliably predict computer vision app capabilities and functionality, they tend to ascribe human-level knowledge to those apps and do not reliably correlate app functionality with similar robotic tasks. We propose that experiential knowledge gained through interaction with software apps is a potential way to “calibrate” user expectations of the function and failure states of complex systems.

In this paper we investigate the influence interfaces and feedback have on human-robot trust levels when operating in a shared physical space. The task we use is specifying a “no-go” region for a robot in an indoor environment. We evaluate three styles of interface (physical, AR, and map-based) and four feedback mechanisms (no feedback, robot drives around the space, an AR “fence”, and the region marked on the map). Our evaluation looks at both usability and trust. Specifically, if the participant trusts that the robot “knows” where the no-go region is and their confidence in the robot's ability to avoid that region. We use both self-reported and indirect measures of trust and usability. Our key findings are: 1) interfaces and feedback do influence levels of trust; 2) the participants largely preferred a mixed interface-feedback pair, where the modality for the interface differed from the feedback.

Robotic tree pruning requires highly precise manipulator control in order to accurately align a cutting implement with the desired pruning point at the correct angle. Simultaneously, the robot must avoid applying excessive force to rigid parts of the environment such as trees, support posts, and wires. In this paper, we propose a hybrid control system that uses a learned vision-based controller to initially align the cutter with the desired pruning point, taking in images of the environment and outputting control actions. This controller is trained entirely in simulation, but transfers easily to real trees via a neural network which transforms raw images into a simplified, segmented representation. Once contact is established, the system hands over control to an interaction controller that guides the cutter pivot point to the branch while minimizing interaction forces. With this simple, yet novel, approach we demonstrate an improvement of over 30 percentage points in accuracy over a baseline controller that uses camera depth data.

Apple picking is a challenging manipulation task, but it is difficult to test solutions due to the limited window of time that apples are in season. Previous methods have built simulations of apple trees, but simulations rarely capture soft contact and deformation well, both of which are common in fruit picking. In this paper we present and validate a physical proxy that replicates the mechanics of a real world apple pick. This proxy, in conjunction with a novel hand with multiple sensors, enables large-scale capture of sensor data for data collection and testing. To validate our approach, we train a Long Short-Term Memory network to classify a pick as successful or failed based on sensor feedback from the robot hand. We show that a network trained on the proxy performs as well, or even better, than a network trained solely on real apple trees, with accuracies up to 90%. We determine which sensors are most important for pick classification and also demonstrate that our proxy preserves the most important sensor feature data for pick classification. For our hand, the most informative sensor was the finger's servomotor effort.

Machine vision is a critical subsystem for enabling robots to be able to perform a variety of tasks in orchard environments. However, orchards are highly visually complex environments, and computer vision algorithms operating in them must be able to contend with variable lighting conditions and background noise. Past work on enabling deep learning algorithms to operate in these environments has typically required large amounts of hand-labeled data to train a deep neural network or physically controlling the conditions under which the environment is perceived. In this paper, we train a neural network system in simulation only using simulated RGB data and optical flow. This resulting neural network is able to perform foreground segmentation of branches in a busy orchard environment without additional real-world training or using any special setup or equipment beyond a standard camera. Our results show that our system is highly accurate and, when compared to a network using manually labeled RGBD data, achieves significantly more consistent and robust performance across environments that differ from the training set.

We present a method for classifying the quality of near-contact grasps using spatial metrics that are recoverable from sensor data. Current methods often rely on calculating precise contact points, which are difficult to calculate in real life, or on tactile sensors or image data, which may be unavailable for some applications. Our method, in contrast, uses a mix of spatial metrics that do not depend on the fingers being in contact with the object, such as the object’s approximate size and location. The grasp quality can be calculated before the fingers actually contact the object, enabling near-grasp quality prediction. Using a random forest classifier, the resulting system is able to predict grasp quality with 96% accuracy using spatial metrics based on the locations of the robot palm, fingers and object. Furthermore, it can maintain an accuracy of 90% when exposed to 10% noise across all its inputs.

Studies have shown that picking techniques play an important role in determining fruit quality at harvest (e.g. bruising, stem retention, etc). When picking fruit such as apples and pears, professional pickers use active perception, incorporating both visual and tactile input about fruit orientation, stem location, and the fruit’s immediate surroundings. This combination of tactile, visual, and force feedback is what enables human workers to execute dynamic movements that quickly and efficiently remove fruit from the tree without damage. However, much of the prior work on robotic fruit picking has formulated the harvesting problem as a position-control problem, using visual feedback for closed-loop end-effector placement while disregarding feedback on physical contact. As a first step towards more intelligent fruit picking — combining proprioception, localized sensing, and observed forces — we have developed a custom end-effector with multiple in-hand sensors, including tactile sensors on the fingertips. This paper presents the mechatronic design of the device as well as results from multiple outdoor picking trials with a Honeycrisp apple tree. Preliminary results show that, with multi-modal sensing, fruit slip, fruit separation from the tree, and fruit release from the hand can be detected.

Most people do not have direct access to knowledge about the inner workings of robots. Instead, they must develop mental models of the robot, a process that is not well understood. This article presents findings from a long-term, in-the-wild, qualitative, hypothesis-generating study of the mental model formation process. The focus was on how (qualitatively) users form mental models of the robot—specifically its perceptual capabilities, rules of behavior, and communication with other humans. Participants of diverse ages had multiple interactions with the robot over six weeks in a non-laboratory setting. The robot’s rules of behavior were changed every two weeks. A novel, non-anthropomorphic robot was created for the study with a realistic use case: storing people’s shoes during a yoga class. This article reports findings from a case study analysis of 28 interviews conducted over six weeks with six participants. These findings are organized into six topics: (1) variability in the rate at which mental models are updated to be more predictive, (2) types of reasoning and hypothesizing about the robot, (3) borrowing from existing mental models and use of imagination, (4) attributing sensing capabilities where there are no visible sensors, (5) judgments about whether the robot is autonomous or teleoperated, and (6) experimenting with the robot. Specific suggestions for future research are given throughout, culminating in a set of study design recommendations. This work demonstrates the fruitfulness of long-term, in-the-wild studies of human-robot interaction, of which mental model formation is a foundational aspect.

No abstract available.

In this paper we define two feature representations for grasping. These representations capture hand-object geometric relationships at the near-contact stage - before the fingers close around the object. Their benefits are: 1) They are stable under noise in both joint and pose variation. 2) They are largely hand and object agnostic, enabling direct comparison across different hand morphologies. 3) Their format makes them suitable for direct application of machine learning techniques developed for images. We validate the representations by: 1) Demonstrating that they can accurately predict the distribution of ε-metric values generated by kinematic noise. I.e., they capture much of the information inherent in contact points and force vectors without the corresponding instabilities. 2) Training a binary grasp success classifier on a real-world data set consisting of 588 grasps.

Grasping a simple object from the side is easy - unless the object is almost as big as the hand or space constraints require positioning the robot hand awkwardly with respect to the object. We show that humans - when faced with this challenge - adopt coordinated finger movements which enable them to successfully grasp objects even from these awkward poses. We also show that it is relatively straight forward to implement these strategies autonomously. Our human-studies approach asks participants to perform grasping task by either “puppetteering” a robotic manipulator that is identical (geometrically and kinematically) to a popular underactuated robotic manipulator (the Barrett hand), or using sliders to control the original Barrett hand. Unlike previous studies, this enables us to directly capture and compare human manipulation strategies with robotic ones. Our observation is that, while humans employ underactuation, how they use it is fundamentally different (and more effective) than that found in existing hardware.

In this paper we propose a framework for conceptualizing and demonstrating a good-faith effort when developing autonomous systems. The framework addresses two fundamental problems facing autonomous systems: (1) the disconnect between human-mental models and machine-based sensors and algorithms; and (2) unpredictability in complex systems. We address these problems using a mix of education - explicitly delineating the mapping between human concepts and their machine equivalents in a structured manner - and data sampling with expected ranges as a testing mechanism.

This paper presents an online data collection method that captures human intuition about what grasp types are preferred for different fundamental object shapes and sizes. Survey questions are based on an adopted taxonomy that combines grasp pre-shape, approach, wrist orientation, object shape, orientation and size which covers a large swathe of common grasps. For example, the survey identifies at what object height or width dimension (normalized by robot hand size) the human prefers to use a two finger precision grasp versus a three-finger power grasp. This information is represented as a confidence-interval based polytope in the object shape space. The result is a database that can be used to quickly find potential pre-grasps that are likely to work, given an estimate of the object shape and size.

We present a human-subjects study approach that supports the analysis of the manipulation performance of robotic hands that have the same morphology but different actuation and compliance. Specifically, we use this approach to analyze three different types of hands (one underactuated, one fully actuated, one fully actuated with compliant distal joints) as they are used to perform two manipulation tasks. The first task uses a power grasp (spraying with a spray bottle), the second a precision grasp (tracing a line on a bowl with a pen). We show that compliance in the distal joints significantly improves performance and task completion. We also show that humans choose significantly different poses for the same task when using a fully-actuated versus underactuated hand, which also results in superior task performance. Our results suggest that humans use a combination of under-actuated and fully-actuated techniques, which when used on robotic systems would also improve their performance on manipulation tasks.

The Robotics Program at Oregon State University has beenrunning an NSF-funded summer Research Experiences forUndergraduates (REU) site since 2014. Over twenty studentsper year (on average) have participated in the site, spendingten weeks embedded in the OSU Robotics Program. Our mainfocus with this REU Site is to give the participants a com-plete research experience, from problem definition to the fi-nal presentation of results, "in miniature". Our secondary ed-ucational objectives are: 1) Teach basic non-technical skillsneeded for graduate work, such as time management and lit-erature review, 2) Provide details on how to apply to gradu-ate school and for funding, 3) Clarify what we look for in agraduate student, and 4) Detail what to expect from the grad-uate student experience. In this paper, we describe the over-all structure of the participants’ summer experience, outlinesome of the training materials that we use, describe the moti-vations for our approach, and discuss the lessons that we havelearned after running the program for a number of years.

Understanding 3D shapes through cross-sections is a mental task that appears both in 3D volume segmentation and solid modeling tasks. Similar to other shape understanding tasks --- such as paper folding --- performance on this task varies across the population, and can be improved through training and practice. We are --- long term --- interested in creating training tools for 3D volume segmentation. To this end, we have modified (and evaluated) an existing cross-section performance measure in the context of our intended application. Our primary adaptations were 1) to use 3D stimuli (instead of 2D) to more accurately capture the real-world application and 2) evaluate performance on 3D biological shapes relative to the 3D geometric shapes used in the previous study. Our findings are: 1) Participants had the same pattern of errors as the original study, but overall their performance improved when they could see the objects rotating in 3D. 2) Inferring cross-sections of biological shapes is more challenging than pure geometric shapes.

Privacy-sensitive robotics is an emerging area of HRI research. Judgments about privacy would seem to be context-dependent, but none of the promising work on contextual "frames" has focused on privacy concerns. This work studies the impact of contextual "frames" on local users' privacy judgments in a home telepresence setting. Our methodology consists of using an online questionnaire to collect responses to animated videos of a telepresence robot after framing people with an introductory paragraph. The results of four studies indicate a large effect of manipulating the robot operator's identity between a stranger and a close confidante. It also appears that this framing effect persists throughout several videos. These findings serve to caution HRI researchers that a change in frame could cause their results to fail to replicate or generalize. We also recommend that robots be designed to encourage or discourage certain frames.

The advent of robotics technology raises new privacy concerns, but preliminary research in this area has not included members of the general public in the conversation. This study used three focus groups to see what types of privacy concerns would be mentioned by typical users and to see what other topics were deemed important by the participants. The conversations were based around three concrete scenarios involving telepresence robots: an in-home tele-maid, a boss attending a meeting via telepresence, and a medical robo-ceptionist. Codings of our transcripts yielded privacy-relevant concerns not yet categorized by the literature such as hacking, theft, embarrassment, and marketing. These findings will give privacy-sensitive robotics researchers (1) a more complete list of privacy categories to measure, (2) new research questions to pursue, and (3) privacy-enhancing design suggestions to test.

Perception and gaze are an integral part of determining where and how to grasp an object. In this study we analyze how gaze patterns differ when participants are asked to manipulate a robotic hand to perform a grasping task when compared with using their own. We have three findings. First, while gaze patterns for the object are similar in both conditions, participants spent substantially more time gazing at the robotic hand then their own, particularly the wrist and finger positions. Second, We provide evidence that for complex objects (eg, a toy airplane) participants essentially treated the object as a collection of sub-objects. Third, we performed a follow-up study that shows that choosing camera angles that clearly display the features participants spend time gazing at are more effective for determining the effectiveness of a grasp from images. Our findings are relevant both for automated algorithms (where visual cues are important for analyzing objects for potential grasps) and for designing tele-operation interfaces (how best to present the visual data to the remote operator).

3D image segmentation is a fundamental process in many scientific and medical applications. Automatic algorithms do exist, but there are many use cases where these algorithms fail. The gold standard is still manual segmentation or review. Unfortunately, existing 3D segmentation tools do not currently take into account human mental models, low-level perception actions, and higher-level cognitive tasks. Our goal is to improve the quality and efficiency of manual segmentation by analyzing the process in terms of human mental models and low-level perceptual tasks. Preliminary results from our in-depth field studies suggest that compared to novices, experts have a stronger mental model of the 3D structures they segment. To validate this assumption, we introduce a novel test instrument to explore experts' mental model in the context of 3D image segmentation. We use this test instrument to measure individual differences in various spatial segmentation and visualization tasks. The tasks involve identifying valid 2D contours, slicing planes and 3D shapes.

3D image segmentation is a fundamental process in many scientific and medical applications. Automatic algorithms do exist, but there are many use cases where these algorithms fail. The gold standard is still manual segmentation or review. Unfortunately, even for an expert this is laborious, time consuming, and prone to errors. Existing 3D segmentation tools do not currently take into account human mental models and low-level perception tasks. Our goal is to improve the quality and efficiency of manual segmentation and review by analyzing how experts perform segmentation. As a preliminary step we conducted a field study with 8 segmentation experts, recording video and eye tracking data. We developed a novel coding scheme to analyze this data and verified that it successfully covers and quantifies the low-level actions, tasks and behaviors of experts during 3D image segmentation.

We present a study that examines the efficiency and usability of three different interfaces for specifying which objects should be kept private (i.e., not visible) in an office environment. Our study context is a robot “janitor” system that has the ability to blur out specified objects from its video feed. One interface is a traditional point-and-click GUI on a computer monitor, while the other two operate in the real, physical space: users either place markers on the objects to indicate privacy or use a wand tool to point at them. This late-breaking report presents qualitative feedback from users for improving the interfaces.

We analyze visual cues people used to evaluate a robot grasp. Participants were presented with two (front and side) orthogonal views of a robot hand grasping an object and asked how successful the grasp would be on a scale of 1-5; they were eye-tracked while completing this survey. Ground truth of the success of the grasps is known. Our primary observations were that (1) Most of the failed grasp predictions were false positives, and this was exacerbated for grasps that were ranked as human-like. (2) Two visual cues from human-grasp research (object center-line and top) were used, but not contact points. Instead, participants gazed at robot finger, wrist, and arm locations. (3) There was a difference in the visual patterns between the left and right images, indicating that the second image was primarily used to verify the locations of fingers and wrist while the first was used to establish the object's location and shape. Finally, we generate transition matrices to model the temporal aspect of the gaze patterns.

We present a study that examines the efficiency and usability of three different interfaces for specifying which objects should be kept private (i.e., not visible) in an office environment. Our study context is a robot “janitor” system that has the ability to blur out specified objects from its video feed. One interface is a traditional point-and-click GUI on a computer monitor, while the other two operate in the real, physical space: users either place markers on the objects to indicate privacy or use a wand tool to point at them. We compare the interfaces using both self-report (e.g., surveys) and behavioral measures. Our results showed that (1) the graphical interface performed better both in terms of time and usability, and (2) using persistent markers increased the participants' ability to recall what they tagged. Choosing the right interface appears to depend on the application scenario. We also summarize feedback from the participants for improving interfaces that specify visual privacy preferences.

The output of 3D volume segmentation is crucial to a wide range of endeavors. Producing accurate segmentations often proves to be both inefficient and challenging, in part due to lack of imaging data quality (contrast and resolution), and because of ambiguity in the data that can only be resolved with higher-level knowledge of the structure and the context wherein it resides. Automatic and semi-automatic approaches are improving, but in many cases still fail or require substantial manual clean-up or intervention. Expert manual segmentation and review is therefore still the gold standard for many applications. Unfortunately, existing tools (both custom-made and commercial) are often designed based on the underlying algorithm, not the best method for expressing higher-level intention. Our goal is to analyze manual (or semi-automatic) segmentation to gain a better understanding of both low-level (perceptual tasks and actions) and high-level decision making. This can be used to produce segmentation tools that are more accurate, efficient, and easier to use. Questioning or observation alone is insufficient to capture this information, so we utilize a hybrid capture protocol that blends observation, surveys, and eye tracking. We then developed, and validated, data coding schemes capable of discerning low-level actions and overall task structures.

The ability to direct a viewer's attention has important applications in computer graphics, data visualization, image analysis, and training. Existing computer-based gaze manipulation techniques, which direct a viewer's attention about a display, have been shown to be effective for spatial learning, search task completion, and medical training applications. In this work we extend the concept of gaze manipulation beyond digital imagery to include controlled, real-world environments. We address two main challenges in guiding attention to real-world objects: determining what object the viewer is currently paying attention to, and providing (projecting) a visual cue on a different part of the scene in order to draw the viewer's attention there. Our system consists of a pair of eye-tracking glasses to determine the viewer's gaze location, and a projector to create the visual cue in the physical environment. The results of a user study show that we can effectively direct the viewer's gaze in the real-world scene. Our technique has applicability in a wide range of instructional environments, including pilot training and driving simulators.

Narrative art tells a story, either as a moment in an ongoing story or as a sequence of events unfolding over time. In many works of art separate panels within the same frame are used to depict the sequence of events. Often, there is no clear delineation between these panels, or any indication of the optimal viewing order. To improve visual literacy we propose using Subtle Gaze Direction (SGD) to direct the viewers gaze across an image in a manner which reveals the story. SGD uses small image space modulations in the luminance channel to guide a viewer's gaze about an image without disrupting their normal visual experience. Using a simple ordering task we compared performance using no modulation and using subtle modulation with the correct order of narrative episodes as intended by the artist. Results from experiments show improved performance when SGD is employed. This experiment establishes the potential of the method as an aid to visual navigation in images where the viewing order is unclear.

Contents of Visual Short-Term Memory depend highly on viewer attention. It is possible to influence where attention is allocated using a technique called Subtle Gaze Direction (SGD). SGD combines eye tracking with subtle image-space modulations to guide viewer gaze about a scene. Modulations are terminated before the viewer can scrutinize them with high acuity foveal vision. This approach is preferred to overt techniques that require permanent alterations to images to highlight areas of interest. In our study, participants were asked to recall the location of objects or regions in images. We investigated if using SGD to guide attention to these regions would improve recall. Results showed that the influence of SGD significantly improved accuracy of target count and spatial location recall. This has implications for a wide range of applications including spatial learning in virtual environments as well as image search applications, virtual training and perceptually based rendering.

We use the Subtle Gaze Direction technique (SGD) to guide novices as they try to find abnormalities in mammograms. SGD works by performing image-space modulations on specific regions of the peripheral vision to attract attention. Gaze is monitored and modulations are terminated before they are scrutinized with high-acuity foveal vision. This approach is preferred to overt techniques which permanently alter images to highlight areas of interest. SGD is used to guide novices along the scanpath of an expert radiologist. We hypothesized that this would increase the likelihood of novices correctly identifying irregularities. Results reveal that novices who were guided in this manner performed significantly better than the control group (no gaze manipulation). Furthermore, a short-term post-training lingering effect was observed among subjects guided using SGD. They continued to perform better than the control group once the training was complete and gaze manipulation was disabled.

Significant effort has been devoted to designing clustering algorithms that are responsive to user feedback or that incorporate prior domain knowledge in the form of constraints. However, users desire more expressive forms of interaction to influence clustering outcomes. In our experiences working with diverse application scientists, we have identified an interaction style scatter/gather clustering that helps users iteratively restructure clustering results to meet their expectations. As the names indicate, scatter and gather are dual primitives that describe whether clusters in a current segmentation should be broken up further or, alternatively, brought back together. By combining scatter and gather operations in a single step, we support very expressive dynamic restructurings of data. Scatter/gather clustering is implemented using a nonlinear optimization framework that achieves both locality of clusters and satisfaction of user-supplied constraints. We illustrate the use of our scatter/gather clustering approach in a visual analytic application to study baffle shapes in the bat biosonar (ears and nose) system. We demonstrate how domain experts are adept at supplying scatter/gather constraints, and how our framework incorporates these constraints effectively without requiring numerous instance-level constraints.

Camera keyframing is an integral part of the animation making process. The animator places the camera in a sequence of "key" positions, and the computer produces a set of intermediate camera locations that interpolates between these keyframes. Camera keyframing traditionally treats the camera as just another 3D object in the scene, with intermediate frames produced by interpolating the position and focal point of the camera in space. Unlike a 3D object, however, the camera's role is to project the 3D scene into 2D. The animator indirectly controls the projection --- how objects in the scene are placed in the 2D image --- by adjusting the camera parameters for each keyframe. Automatic 3D camera interpolation adds yet another layer of indirection. The net effect is that the animator must solve a complicated inverse problem in order to move objects across the 2D scene in the desired manner.

Extremal surfaces are a class of implicit surfaces that have been found useful in a variety of geometry reconstruction applications. Compared to iso-surfaces, extremal surfaces are particularly challenging to construct in part due to the presence of boundaries and the lack of a consistent orientation. We present a novel, grid-based algorithm for constructing polygonal approximations of extremal surfaces that may be open or unorientable. The algorithm is simple to implement and applicable to both uniform and adaptive grid structures. More importantly, the resulting discrete surface preserves the structural property of the extremal surface in a grid-independent manner. The algorithm is applied to extract ridge surfaces from intensity volumes and reconstruct surfaces from point sets with unoriented normals.

A new experiment is presented that demonstrates the usefulness of an image space modulation technique called subtle gaze direction (SGD) for guiding the user in a simple searching task. SGD uses image space modulations in the luminance channel to guide a viewer's gaze about a scene without interrupting their visual experience. The goal of SGD is to direct a viewer's gaze to certain regions of a scene without introducing noticeable changes in the image. Using a simple searching task, we compared performance using no modulation, using subtle modulation, and using obvious modulation. Results from the experiments show improved performance when using subtle gaze direction, without affecting the user's perception of the image. We then extend the experiment to evaluate performance with the presence of distractors. The distractors took the form of extra modulations, which do not correspond to a target in the image. Experimentation shows, that, even in the presence of distractors, more accurate results are returned on a simple search task using SGD, as compared to results returned when no modulation at all is used. Results establish the potential of the method for a wide range of applications including gaming, perceptually based rendering, navigation in virtual environments, and medical search tasks.

This article presents a novel technique that combines eye-tracking with subtle image-space modulation to direct a viewer's gaze about a digital image. We call this paradigm subtle gaze direction . Subtle gaze direction exploits the fact that our peripheral vision has very poor acuity compared to our foveal vision. By presenting brief, subtle modulations to the peripheral regions of the field of view, the technique presented here draws the viewer's foveal vision to the modulated region. Additionally, by monitoring saccadic velocity and exploiting the visual phenomenon of saccadic masking, modulation is automatically terminated before the viewer's foveal vision enters the modulated region. Hence, the viewer is never actually allowed to scrutinize the stimuli that attracted her gaze. This new subtle gaze directing technique has potential application in many areas including large scale display systems, perceptually adaptive rendering, and complex visual search tasks.

Abstract We present a framework for interactive sketching that allows users to create three‐dimensional (3D) architectural models quickly and easily from a source drawing. The sketching process has four steps. (1) The user calibrates a viewing camera by specifying the origin and vanishing points of the drawing. (2) The user outlines surface polygons in the drawing. (3) A 3D reconstruction algorithm uses perceptual constraints to determine the closest visual fit for the polygon. (4) The user can then adjust aesthetic controls to produce several stylistic effects in the scene: a smooth transition between day and night rendering, a horizon knockout effect and entourage figures. The major advantage of our approach lies in the combination of perception‐based techniques, which allow us to minimize unnecessary interactions, and a hinging‐angle scheme, which shows significant improvement in numerical stability over previous optimization‐based 3D reconstruction algorithms. We also demonstrate how our reconstruction algorithm can be extended to work with perspective images, a feature unavailable in previous approaches.

Abstract-Orthopedists invest significant amounts of effort and time trying to understand the biomechanics of arthrodial (gliding) joints. Although new image acquisition and processing methods currently generate richer-than-ever geometry and kinematic data sets that are individual specific, the computational and visualization tools needed to enable the comparative analysis and exploration of these data sets lag behind. In this paper, we present a framework that enables the cross-data-set visual exploration and analysis of arthrodial joint biomechanics. Central to our approach is a computer-vision-inspired markerless method for establishing pairwise correspondences between individual-specific geometry. Manifold models are subsequently defined and deformed from one individual-specific geometry to another such that the markerless correspondences are preserved while minimizing model distortion. The resulting mutually consistent parameterization and visualization allow the users to explore the similarities and differences between two data sets and to define meaningful quantitative measures. We present two applications of this framework to human-wrist data: articular cartilage transfer from cadaver data to in vivo data and cross-data-set kinematics analysis. The method allows our users to combine complementary geometries acquired through different modalities and thus overcome current imaging limitations. The results demonstrate that the technique is useful in the study of normal and injured anatomy and kinematics of arthrodial joints. In principle, the pairwise cross-parameterization method applies to all spherical topology data from the same class and should be particularly beneficial in instances where identifying salient object features is a nontrivial task.

We apply simplified image-based lighting methods to reduce the equipment, cost, time, and specialized skills required for high-quality photographic lighting of desktop-sized static objects such as museum artifacts. We place the object and a computer-steered moving-head spotlight inside a simple foam-core enclosure and use a camera to record photos as the light scans the box interior. Optimization, guided by interactive user sketching, selects a small set of these photos whose weighted sum best matches the user-defined target sketch. Unlike previous image-based relighting efforts, our method requires only a single area light source, yet it can achieve high-resolution light positioning to avoid multiple sharp shadows. A reduced version uses only a handheld light and may be suitable for battery-powered field photography equipment that fits into a backpack.

A method is described for texturing surfaces using decals , images placed on the surface using local parameterizations. Decal parameterizations are generated with a novel O ( N log N ) discrete approximation to the exponential map which requires only a single additional step in Dijkstra's graph-distance algorithm. Decals are dynamically composited in an interface that addresses many limitations of previous work. Tools for image processing, deformation/feature-matching, and vector graphics are implemented using direct surface interaction. Exponential map decals can contain holes and can also be combined with conformal parameterization to reduce distortion. The exponential map approximation can be computed on any point set, including meshes and sampled implicit surfaces, and is relatively stable under resampling. The decals stick to the surface as it is interactively deformed, allowing the texture to be preserved even if the surface changes topology. These properties make exponential map decals a suitable approach for texturing animated implicit surfaces.

The ability to identify similarities between shapes is important for applications such as medical diagnosis, object registration and alignment, and shape retrieval. In this paper we present a method, the curvature map that uses surface curvature properties in a region around a point to create a unique signature for that point. These signatures can then be compared to determine the similarity of one point to another. To gather curvature information around a point we explore two techniques, rings (which use the local topology of the mesh) and geodesic fans (which trace geodesies along the mesh from the point). We explore a variety of comparison functions and provide experimental evidence for which ones provide the best discriminatory power. We show that curvature maps are both more robust and provide better discrimination than simply comparing the curvature at individual points.

We present a new screen space widget, the IBar, for effective camera control in 3D graphics environments. The IBar provides a compelling interface for controlling scene perspective based on the artistic concept of vanishing points. Various handles on the widget manipulate multiple camera parameters simultaneously to create a single perceived projection change. For example, changing just the perspective distortion is accomplished by simultaneously decreasing the camera's distance to the scene while increasing focal length. We demonstrate that the IBar is easier to learn for novice users and improves their understanding of camera perspective.

We describe a complete, end-to-end system for taking well-composed photographs using a mobile robot. The general scenario is a reception, or other event, where people are roaming around talking to each other. The robot serves as an "event photographer", roaming around the same space as the participants, periodically taking photographs. These images are then sent to a workstation where participants can print the photographs out, or email them.

This paper presents several methods for shading meshes from scanned paint samples that represent dark to light transitions. Our techniques emphasize artistic control of brush stroke texture and color. We first demonstrate how the texture of the paint sample can be separated from its color gradient. We demonstrate three methods, two real-time and one off-line, for producing rendered, shaded images from the texture samples. All three techniques use texture synthesis to generate additional paint samples. Finally, we develop metrics for evaluating how well each method achieves our goal in terms of texture similarity, shading correctness and temporal coherence.

We present a surface modeling technique using manifolds. Our approach uses a single, simple parameterization for all surfaces of a given genus. This differs from previous approaches which build a parameterization based on the elements of a mesh. The simple parameterization is more appropriate for applications that perform complex operations in parameter space or on the mesh surface. We define a manifold and a corresponding embedding function for three genera (plane, sphere, and torus). The manifold can be used simply as a parameterization tool or as a smooth surface approximating the original mesh. We demonstrate how to build a correspondence between the mesh and the manifold, then how to build an embedding that approximates the mesh.

We have created a system for capturing both the three-dimensional geometry and color and shading information for human facial expressions.We use this data to reconstruct photorealistic, 3D animations of the captured expressions.The system uses a large set of sampling points on the face to accurately track the three dimensional deformations of the face.Simultaneously with the tracking of the geometric data, we capture multiple high resolution, registered video images of the face.These images are used to create a texture map sequence for a three dimensional polygonal face model which can then be rendered on standard 3D graphics hardware.The resulting facial animation is surprisingly life-like and looks very much like the original live performance.Separating the capture of the geometry from the texture images eliminates much of the variance in the image data due to motion, which increases compression ratios.Although the primary emphasis of our work is not compression we have investigated the use of a novel method to compress the geometric data based on principal components analysis.The texture sequence is compressed using an MPEG4 video codec.Animations reconstructed from 512x512 pixel textures look good at data rates as low as 240 Kbits per second.

Editing curves and surfaces is difficult in part because their mathematical representations rarely correspond to most people’s idea of a curve or surface. The implementation (and hence, behavior) of most manipulation tools is intertwined with a particular curve or surface representation; this can make reimplementing the tool with a different representation problematic. A system using a single representation must therefore either limit the types of tools available or convert existing tools to work on the system’s representation. In this paper we present a framework for editing curves or surfaces which supports multiple representations and ensures that they stay synchronized. As a proof of concept, we have created a curve editor which contains several tools each of which manipulate one of three different curve representations: polylines, NURBs, and multi‐resolution B‐splines.

We describe an extension of B-splines to surfaces of arbitrary topology, including arbitrary boundaries. The technique inherits many of the properties of B-splines: local control, a compact representation, and guaranteed continuity of arbitrary degree. The surface is specified using a polyhedral control mesh instead of a rectangular one; the resulting surface approximates the polyhedral mesh much as a B-spline approximates its rectangular control mesh. Like a Bspline, the surface is a single, continuousobject. This is achievedby modeling the domain of the surface with a manifold whose topology matches that of the polyhedral mesh, then embedding this domain into 3-space using a basis-function/control-point formulation. We provide a constructive approach to building a manifold. CR Categories: I.3.5 [Computer Graphics]: Computational Geometry and Object Modeling, Curve, Surface, Solid, and Object Representations, Splines 1 Introduction Surfaces of arbitrary topology are currently attrac...

This paper exploresthe use of visual operators for solids modeling. We focus on designing interfaces for free-form operators such as blends, sweeps, and deformations, because these operators have a large number of interacting parameters whose effects are often determined by an underlying parameterization. In this type of interactive modeling good solutions to the design problem have aesthetic as well as engineering components. Traditionally, interaction with the parameters of these operators has been through text editors, curve editors, or trial-and-error with a slider bar. Parametric values have been estimated from data, but not interactively. These parametersare usually one- or two-dimensional,but the operatorsthemselves are intrinsically three-dimensional in that they are used to model surfaces visualized in 3D. The traditional textual style of interaction is tedious and interposes a level of abstraction between the parameters and the resulting surface. A 3D visual interface has the...