VLDB 2026 Research / reviewers in the wild / expert
Robert DeLine
dblp:d/RobertDeLine · also Rob DeLine, Robert A. DeLine
· DBLP profile ↗
44ranked-venue papers
14as first author
6since 2021 · last 2023
0000-0001-8885-8367ORCID · verified
Domains — the database's venue-derived domains; a paper can count in several
Human-computer interaction and ubiquitous computing · 21 · 5 first-author · 6 since 2021Software engineering, systems software and programming languages · 20 · 9 first-authorDatabases, data management, data science and information retrieval · 3
| Year | Publication | Venue | Position |
|---|---|---|---|
| 2023 | On the Design of AI-powered Code Assistants for NotebooksabstractAI-powered code assistants, such as Copilot, are quickly becoming a ubiquitous component of contemporary coding contexts. Among these environments, computational notebooks, such as Jupyter, are of particular interest as they provide rich interface affordances that interleave code and output in a manner that allows for both exploratory and presentational work. Despite their popularity, little is known about the appropriate design of code assistants in notebooks. We investigate the potential of code assistants in computational notebooks by creating a design space (reified from a survey of extant tools) and through an interview-design study (with 15 practicing data scientists). Through this work, we identify challenges and opportunities for future systems in this space, such as the value of disambiguation for tasks like data visualization, the potential of tightly scoped domain-specific tools (like linters), and the importance of polite assistants. Andrew M. McNutt, Chenglong Wang 0005, Robert DeLine, Steven Mark Drucker |
CHI | 3 |
| 2023 | What Did My AI Learn? How Data Scientists Make Sense of Model BehaviorabstractData scientists require rich mental models of how AI systems behave to effectively train, debug, and work with them. Despite the prevalence of AI analysis tools, there is no general theory describing how people make sense of what their models have learned. We frame this process as a form of sensemaking and derive a framework describing how data scientists develop mental models of AI behavior. To evaluate the framework, we show how existing AI analysis tools fit into this sensemaking process and use it to design AIFinnity , a system for analyzing image-and-text models. Lastly, we explored how data scientists use a tool developed with the framework through a think-aloud study with 10 data scientists tasked with using AIFinnity to pick an image captioning model. We found that AIFinnity ’s sensemaking workflow reflected participants’ mental processes and enabled them to discover and validate diverse AI behaviors. Ángel Alexander Cabrera, Marco Túlio Ribeiro, Bongshin Lee, Robert DeLine, Adam Perer, Steven Mark Drucker |
ACM Trans. Comput. Hum. Interact. | 4 |
| 2022 | Diff in the Loop: Supporting Data Comparison in Exploratory Data AnalysisabstractData science is characterized by evolution: since data science is exploratory, results evolve from moment to moment; since it can be collaborative, results evolve as the work changes hands. While existing tools help data scientists track changes in code, they provide less support for understanding the iterative changes that the code produces in the data. We explore the idea of visualizing differences in datasets as a core feature of exploratory data analysis, a concept we call Diff in the Loop (DITL). We evaluated DITL in a user study with 16 professional data scientists and found it helped them understand the implications of their actions when manipulating data. We summarize these findings and discuss how the approach can be generalized to different data science workflows. April Yi Wang, Will Epperson, Robert DeLine, Steven Mark Drucker |
CHI | 3 |
| 2021 | Glinda: Supporting Data Science with Live Programming, GUIs and a Domain-specific LanguageabstractResearchers have explored several avenues to mitigate data scientists’ frustrations with computational notebooks, including: (1) live programming, to keep notebook results consistent and up to date; (2) supplementing scripting with graphical user interfaces (GUIs), to improve ease of use; and (3) providing domain-specific languages (DSLs), to raise a script’s level of abstraction. This paper introduces Glinda, which combines these three approaches by providing a live programming experience, with interactive results, for a domain-specific language for data science. The language’s compiler uses an open-ended set of “recipes” to execute steps in the user’s data science workflow. Each recipe is intended to combine the expressiveness of a written notation with the ease-of-use of a GUI. Live programming provides immediate feedback to a user’s input, whether in the form of program edits or GUI gestures. In a qualitative evaluation with 12 professional data scientists, participants highly rated the live programming and interactive results. They found the language productive and sufficiently expressive and suggested opportunities to extend it. Robert DeLine |
CHI | 1 |
| 2021 | Fork It: Supporting Stateful Alternatives in Computational NotebooksabstractComputational notebooks, which seamlessly interleave code with results, have become a popular tool for data scientists due to the iterative nature of exploratory tasks. However, notebooks provide a single execution state for users to manipulate through creating and manipulating variables. When exploring alternatives, data scientists must carefully create many-step manipulations in visually distant cells. Nathaniel Weinman, Steven Mark Drucker, Titus Barik, Robert DeLine |
CHI | 4 |
| 2021 | How Teams Communicate about the Quality of ML Models: A Case Study at an International Technology CompanyabstractMachine learning (ML) has become a crucial component in software products, either as part of the user experience or used internally by software teams. Prior studies have explored how ML is affecting development team roles beyond data scientists, including user experience designers, program managers, developers and operations engineers. However, there has been little investigation of how team members in different roles on the team communicate about ML, in particular about the quality of models. We use the general term quality to look beyond technical issues of model evaluation, such as accuracy and overfitting, to any issue affecting whether a model is suitable for use, including ethical, engineering, operations, and legal considerations. What challenges do teams face in discussing the quality of ML models? What work practices mitigate those challenges? To address these questions, we conducted a mixed-methods study at a large software company, first interviewing15 employees in a variety of roles, then surveying 168 employees to broaden our understanding. We found several challenges, including a mismatch between user-focused and model-focused notions of performance, misunderstandings about the capabilities and limitations of evolving ML technology, and difficulties in understanding concerns beyond one's own role. We found several mitigation strategies, including the use of demos during discussions to keep the team customer-focused. Jumana Almahmoud, Robert DeLine, Steven Mark Drucker |
Proc. ACM Hum. Comput. Interact. | 2 |
| 2020 | Wrex: A Unified Programming-by-Example Interaction for Synthesizing Readable Code for Data ScientistsabstractData wrangling is a difficult and time-consuming activity in computational notebooks, and existing wrangling tools do not fit the exploratory workflow for data scientists in these environments. We propose a unified interaction model based on programming-by-example that generates readable code for a variety of useful data transformations, implemented as a Jupyter notebook extension called Wrex. User study results demonstrate that data scientists are significantly more effective and efficient at data wrangling with Wrex over manual programming. Qualitative participant feedback indicates that Wrex was useful and reduced barriers in having to recall or look up the usage of various data transform functions. The synthesized code allowed data scientists to verify the intended data transformation, increased their trust and confidence in Wrex, and fit seamlessly within their cell-based notebook workflows. This work suggests that presenting readable code to professional data scientists is an indispensable component of offering data wrangling tools in notebooks. Ian Drosos, Titus Barik, Philip J. Guo, Robert DeLine, Sumit Gulwani |
CHI | 4 |
| 2019 | Managing Messes in Computational NotebooksabstractData analysts use computational notebooks to write code for analyzing and visualizing data. Notebooks help analysts iteratively write analysis code by letting them interleave code with output, and selectively execute cells. However, as analysis progresses, analysts leave behind old code and outputs, and overwrite important code, producing cluttered and inconsistent notebooks. This paper introduces code gathering tools, extensions to computational notebooks that help analysts find, clean, recover, and compare versions of code in cluttered, inconsistent notebooks. The tools archive all versions of code outputs, allowing analysts to review these versions and recover the subsets of code that produced them. These subsets can serve as succinct summaries of analysis activity or starting points for new analyses. In a qualitative usability study, 12 professional analysts found the tools useful for cleaning notebooks and writing analysis code, and discovered new ways to use them, like generating personal documentation and lightweight versioning. Andrew Head, Fred Hohman, Titus Barik, Steven Mark Drucker, Robert DeLine |
CHI | 5 |
| 2019 | Gamut: A Design Probe to Understand How Data Scientists Understand Machine Learning ModelsabstractWithout good models and the right tools to interpret them, data scientists risk making decisions based on hidden biases, spurious correlations, and false generalizations. This has led to a rallying cry for model interpretability. Yet the concept of interpretability remains nebulous, such that researchers and tool designers lack actionable guidelines for how to incorporate interpretability into models and accompanying tools. Through an iterative design process with expert machine learning researchers and practitioners, we designed a visual analytics system, Gamut, to explore how interactive interfaces could better support model interpretation. Using Gamut as a probe, we investigated why and how professional data scientists interpret models, and how interface affordances can support data scientists in answering questions about model interpretability. Our investigation showed that interpretability is not a monolithic concept: data scientists have different reasons to interpret models and tailor explanations for specific audiences, often balancing competing concerns of simplicity and completeness. Participants also asked to use Gamut in their work, highlighting its potential to help data scientists understand their own data. Fred Hohman, Andrew Head, Rich Caruana, Robert DeLine, Steven Mark Drucker |
CHI | 4 |
| 2019 | What Makes a Great Manager of Software Engineers?abstractHaving great managers is as critical to success as having a good team or organization. In general, a great manager is seen as fuelling the team they manage, enabling it to use its full potential. Though software engineering research studies factors that may affect the performance and productivity of software engineers and teams (like tools and skills), it has overlooked the software engineering manager. The software industry's growth and change in the last decades is creating a need for a domain-specific view of management. On the one hand, experts are questioning how the abundant work in management applies to software engineering. On the other hand, practitioners are looking to researchers for evidence-based guidance on how to manage software teams. We conducted a mixed methods empirical study of software engineering management at Microsoft to investigate what manager attributes developers and engineering managers perceive important and why. We present a conceptual framework of manager attributes, and find that technical skills are not the sign of greatness for an engineering manager. Through statistical analysis we identify how engineers and managers relate in their views, and how software engineering differs from other knowledge work groups in its perceptions about what makes great managers. We present strategies for putting the attributes to use, discuss implications for research and practice, and offer avenues for further work. Eirini Kalliamvakou, Christian Bird, Thomas Zimmermann 0001, Andrew Begel, Robert DeLine, Daniel M. Germán |
IEEE Trans. Software Eng. | 5 |
| 2018 | What makes a great manager of software engineers?abstractHaving great managers is as critical to success as having a good team or organization. A great manager is seen as fuelling the team they manage, enabling it to use its full potential. Though software engineering research studies factors that may affect the performance and productivity of software engineers and teams (like tools and skill), it has overlooked the software engineering manager. On the one hand, experts are questioning how the abundant work in management applies to software engineering. On the other hand, practitioners are looking to researchers for evidence-based guidance on how to manage software teams. We conducted a mixed methods empirical study to investigate what manager attributes developers and engineering managers perceive important and why. We present a conceptual framework of manager attributes, and find that technical skills are not the sign of greatness for an engineering manager. Through statistical analysis we identify how engineers and managers relate in their views, and how software engineering differs from other knowledge work groups. Eirini Kalliamvakou, Christian Bird, Thomas Zimmermann 0001, Andrew Begel, Robert DeLine, Daniel M. Germán |
ICSE | 5 |
| 2018 | Data scientists in software teams: state of the art and challengesabstractThe demand for analyzing large scale telemetry, machine, and quality data is rapidly increasing in software industry. Data scientists are becoming popular within software teams. For example, Face-book, LinkedIn and Microsoft are creating a new career path for data scientists. In this paper, we present a large-scale survey with 793 professional data scientists at Microsoft to understand their educational background, problem topics that they work on, tool usages, and activities. We cluster these data scientists based on the time spent for various activities and identify 9 distinct clusters of data scientists and their corresponding characteristics. We also discuss the challenges that they face and the best practices they share with other data scientists. Our study finds several trends about data scientists in the software engineering context at Microsoft, and should inform managers on how to leverage data science capability effectively within their teams. Miryung Kim, Thomas Zimmermann 0001, Robert DeLine, Andrew Begel |
ICSE | 3 |
| 2018 | Data Scientists in Software Teams: State of the Art and ChallengesabstractThe demand for analyzing large scale telemetry, machine, and quality data is rapidly increasing in software industry. Data scientists are becoming popular within software teams, e.g., Facebook, LinkedIn and Microsoft are creating a new career path for data scientists. In this paper, we present a large-scale survey with 793 professional data scientists at Microsoft to understand their educational background, problem topics that they work on, tool usages, and activities. We cluster these data scientists based on the time spent for various activities and identify 9 distinct clusters of data scientists, and their corresponding characteristics. We also discuss the challenges that they face and the best practices they share with other data scientists. Our study finds several trends about data scientists in the software engineering context at Microsoft, and should inform managers on how to leverage data science capability effectively within their teams. Miryung Kim, Thomas Zimmermann 0001, Robert DeLine, Andrew Begel |
IEEE Trans. Software Eng. | 3 |
| 2016 | Making Sense of Temporal Queries with Interactive VisualizationabstractAs real-time monitoring and analysis become increasingly important, researchers and developers turn to data stream management systems (DSMS's) for fast, efficient ways to pose temporal queries over their datasets. However, these systems are inherently complex, and even database experts find it difficult to understand the behavior of DSMS queries. To help analysts better understand these temporal queries, we developed StreamTrace, an interactive visualization tool that breaks down how a temporal query processes a given dataset, step-by-step. The design of StreamTrace is based on input from expert DSMS users; we evaluated the system with a lab study of programmers who were new to streaming queries. Results from the study demonstrate that StreamTrace can help users to verify that queries behave as expected and to isolate the regions of a query that may be causing unexpected results. Leilani Battle, Danyel Fisher, Robert DeLine, Michael Barnett 0001, Badrish Chandramouli, Jonathan Goldstein |
CHI | 3 |
| 2016 | The emerging role of data scientists on software development teamsabstractCreating and running software produces large amounts of raw data about the development process and the customer usage, which can be turned into actionable insight with the help of skilled data scientists. Unfortunately, data scientists with the analytical and software engineering skills to analyze these large data sets have been hard to come by; only recently have software companies started to develop competencies in software-oriented data analytics. To understand this emerging role, we interviewed data scientists across several product groups at Microsoft. In this paper, we describe their education and training background, their missions in software engineering contexts, and the type of problems on which they work. We identify five distinct working styles of data scientists: (1) Insight Providers, who work with engineers to collect the data needed to inform decisions that managers make; (2) Modeling Specialists, who use their machine learning expertise to build predictive models; (3) Platform Builders, who create data platforms, balancing both engineering and data analysis concerns; (4) Polymaths, who do all data science activities themselves; and (5) Team Leaders, who run teams of data scientists and spread best practices. We further describe a set of strategies that they employ to increase the impact and actionability of their work. Miryung Kim, Thomas Zimmermann 0001, Robert DeLine, Andrew Begel |
ICSE | 3 |
| 2015 | (s|qu)eries: Visual Regular Expressions for Querying and Exploring Event SequencesabstractMany different domains collect event sequence data and rely on finding and analyzing patterns within it to gain meaningful insights. Current systems that support such queries either provide limited expressiveness, hinder exploratory workflows or present interaction and visualization models which do not scale well to large and multi-faceted data sets. In this paper we present (s|qu)eries (pronounced "Squeries"), a visual query interface for creating queries on sequences (series) of data, based on regular expressions. (s|qu)eries is a touch-based system that exposes the full expressive power of regular expressions in an approachable way and interleaves query specification with result visualizations. Being able to visually investigate the results of different query-parts supports debugging and encourages iterative query-building as well as exploratory work-flows. We validate our design and implementation through a set of informal interviews with data scientists that analyze event sequences on a daily basis. Emanuel Zgraggen, Steven Mark Drucker, Danyel Fisher, Robert DeLine |
CHI | 4 |
| 2015 | Supporting exploratory data analysis with live programmingabstractData scientists often conduct exploratory data analysis in scripting environments with a read-eval-print loop (REPL), like R, IPython or MATLAB. This user experience requires diligent management of execution and generates lengthy histories of unwanted command responses. This paper explores the alternative of live programming, a user experience in which the user's edits immediately and automatically update the script results-a “ripple” effect familiar from spreadsheets. Which user experience provides better support for exploratory data analysis, REPL or ripple? We conducted a controlled lab study with 15 data-experienced professionals. Each participant explored four datasets, two in each experience. The REPL sessions left histories with both significantly more data results and significantly more errors than the live sessions. However, both experiences produced comparable numbers of data results that participants self-rated as insightful. Participants largely preferred the live experience for its responsiveness and ability to keep the script content clean, but missed the visible history that a REPL provides. Robert DeLine, Danyel Fisher |
VL/HCC | 1 |
| 2015 | Tempe: Live scripting for live dataabstractData scientists are increasingly working with live streaming data, for example, business telemetry and signals from wearable devices and the Internet of Things. Unfortunately, current tools for exploratory data analysis provide poor support for streaming data. This paper presents Tempe, a data science environment for temporal and streaming data. Tempe's extensible scripting environment allows for live programming, displays interactive, continually updating visualizations, and provides a uniform query language for both stored and live data. We discuss the streaming features of Tempe and evaluate our design choices with a deployment study at Microsoft with a product team who used Tempe continuously for six months. Robert DeLine, Danyel Fisher, Badrish Chandramouli, Jonathan Goldstein, Michael Barnett 0001, James F. Terwilliger, John Robert Wernsing |
VL/HCC | 1 |
| 2014 | Trill: A High-Performance Incremental Query Processor for Diverse AnalyticsabstractThis paper introduces Trill -- a new query processor for analytics. Trill fulfills a combination of three requirements for a query processor to serve the diverse big data analytics space: (1) Query Model : Trill is based on a tempo-relational model that enables it to handle streaming and relational queries with early results, across the latency spectrum from real-time to offline; (2) Fabric and Language Integration : Trill is architected as a high-level language library that supports rich data-types and user libraries, and integrates well with existing distribution fabrics and applications; and (3) Performance : Trill's throughput is high across the latency spectrum. For streaming data, Trill's throughput is 2-4 orders of magnitude higher than comparable streaming engines. For offline relational queries, Trill's throughput is comparable to a major modern commercial columnar DBMS. Trill uses a streaming batched-columnar data representation with a new dynamic compilation-based system architecture that addresses all these requirements. In this paper, we describe Trill's new design and architecture, and report experimental results that demonstrate Trill's high performance across diverse analytics scenarios. We also describe how Trill's ability to support diverse analytics has resulted in its adoption across many usage scenarios at Microsoft. Badrish Chandramouli, Jonathan Goldstein, Michael Barnett 0001, Robert DeLine, John C. Platt, James F. Terwilliger, John Robert Wernsing |
Proc. VLDB Endow. | 4 |
| 2013 | Debugging Revisited: Toward Understanding the Debugging Needs of Contemporary Software DevelopersabstractWe know surprisingly little about how professional developers define debugging and the challenges they face in industrial environments. To begin exploring professional debugging challenges and needs, we conducted and analyzed interviews with 15 professional software engineers at Microsoft. The goals of this study are: 1) to understand how professional developers currently use information and tools to debug, 2) to identify new challenges in debugging in contemporary software development domains (web services, multithreaded/multicore programming), and 3) to identify the improvements in debugging support desired by these professionals that are needed from research. The interviews were coded to identify the most common information resources, techniques, challenges, and needs for debugging as articulated by the developers. The study reveals several debugging challenges faced by professionals, including: 1) the interaction of hypothesis instrumentation and software environment as a source of debugging difficulty, 2) the impact of log file information on accurate debugging of web services, and 3) the mismatch between the sequential human thought process and the non-sequential execution of multithreaded environments as source of difficulty. The interviewees also describe desired improvements to tools to support debugging, many of which have been discussed in research but not transitioned to practice. Lucas Layman, Madeline Diep, Meiyappan Nagappan, Janice Singer, Robert DeLine, Gina Venolia |
ESEM | 5 |
| 2013 | Stat!: an interactive analytics environment for big dataabstractExploratory analysis on big data requires us to rethink data management across the entire stack -- from the underlying data processing techniques to the user experience. We demonstrate Stat! -- a visualization and analytics environment that allows users to rapidly experiment with exploratory queries over big data. Data scientists can use Stat! to quickly refine to the correct query, while getting immediate feedback after processing a fraction of the data. Stat! can work with multiple processing engines in the backend; in this demo, we use Stat! with the Microsoft StreamInsight streaming engine. StreamInsight is used to generate incremental early results to queries and refine these results as more data is processed. Stat! allows data scientists to explore data, dynamically compose multiple queries to generate streams of partial results, and display partial results in both textual and visual form. Michael Barnett 0001, Badrish Chandramouli, Robert DeLine, Steven Mark Drucker, Danyel Fisher, Jonathan Goldstein, Patrick Morrison, John C. Platt |
SIGMOD Conference | 3 |
| 2012 | Debugger Canvas: Industrial experience with the code bubbles paradigmabstractAt ICSE 2010, the Code Bubbles team from Brown University and the Code Canvas team from Microsoft Research presented similar ideas for new user experiences for an integrated development environment. Since then, the two teams formed a collaboration, along with the Microsoft Visual Studio team, to release Debugger Canvas, an industrial version of the Code Bubbles paradigm. With Debugger Canvas, a programmer debugs her code as a collection of code bubbles, annotated with call paths and variable values, on a two-dimensional pan-and-zoom surface. In this experience report, we describe new user interface ideas, describe the rationale behind our design choices, evaluate the performance overhead of the new design, and provide user feedback based on lab participants, post-release usage data, and a user survey and interviews. We conclude that the code bubbles paradigm does scale to existing customer code bases, is best implemented as a mode in the existing user experience rather than a replacement, and is most useful when the user has a long or complex call paths, a large or unfamiliar code base, or complex control patterns, like factories or dynamic linking. Robert DeLine, Andrew Bragdon, Kael Rowan, Jens Jacobsen, Steven P. Reiss |
ICSE | 1 |
| 2012 | Studying developers for fun and profit: (Keynote abstract)abstractSummary form only given. NCsoft Corporation is a leading global developer and publisher of online games. Founded in 1997 by Taek Jin Kim, NCsoft is home to the critically acclaimed Aion, Lineage, Guild Wars, City of Heroes franchises and numerous Casual Games. With offices in North America, United Kingdom, Japan, Taiwan and Thailand, NCsoft aims at providing fun to everyone, everywhere in the world. With over 3,600 employees around the world, NCsoft develops and publishes games of diverse genres for global gamers. Using high quality game scenarios and outstanding technology gained through continuous research and development, NCsoft is creating some of the world's finest games. In this session, the audience will hear about the corporate details of NCsoft and the major titles developed or under development by the company. Robert DeLine |
ICPC | 1 |
| 2011 | A field study of API learning obstacles
Martin P. Robillard, Robert DeLine |
Empir. Softw. Eng. | 2 |
| 2010 | Evaluating cues for resuming interrupted programming tasksabstractDevelopers, like all modern knowledge workers, are frequently interrupted and blocked in their tasks. In this paper we present a contextual inquiry into developers' current strategies for resuming interrupted tasks and investigate the effect of automated cues on improving task resumption. We surveyed 371 programmers on the nature of their tasks, interruptions, task suspension and resumption strategies and found that they rely heavily on note-taking across several types of media. We then ran a controlled lab study to compare the effects of two different automated cues to note taking when resuming interrupted programming tasks. The two cues differed in (1) whether activities were summarized in aggregate or presented chronologically and (2) whether activities were presented as program symbols or as code snippets. Both cues performed well: developers using either cue completed their tasks with twice the success rate as those using note-taking alone. Despite the similar performance of the cues, developers strongly preferred the cue that presents activities chronologically as code snippets. Chris Parnin, Robert DeLine |
CHI | 2 |
| 2010 | Code canvas: zooming towards better development environmentsabstractThe user interfaces of today's development environments have a "bento box" design that partitions information into separate areas. This design makes it difficult to stay oriented in the open documents and to synthesize information shown in different areas. Code Canvas takes a new approach by providing an infinite zoomable surface for software development. A canvas both houses editable forms of all of a project's documents and allows multiple layers of visualization over those documents. By uniting the content of a project and information about it onto a single surface, Code Canvas is designed to leverage spatial memory to keep developers oriented and to make it easy to synthesize information. Robert DeLine, Kael Rowan |
ICSE (2) | 1 |
| 2009 | Designers wanted: participation and the user experience in open source software developmentabstractWe present design concepts and related mockups that support the user experience for projects hosted on CodePlex, an open source project hosting website. Rationale for the design concepts is grounded in the open source literature and a thirteen-week study with the CodePlex team. We propose that fostering ways to build trust, providing opportunities for merit, supporting crossover of work activities, and supporting user experience (UX) best practices in CodePlex will help dismantle the social and technological barriers for UX and encourage UX designer participation. We address UX designer motivation as a challenge for participation and conclude that the mockups presented are a first step in furthering the user experience in open source software development. Paula M. Bach, Robert DeLine, John M. Carroll 0001 |
CHI | 2 |
| 2007 | Let's go to the whiteboard: how and why software developers use drawingsabstractSoftware developers are rooted in the written form of their code, yet they often draw diagrams representing their code. Unfortunately, we still know little about how and why they create these diagrams, and so there is little research to inform the design of visual tools to support developers' work. This paper presents findings from semi-structured interviews that have been validated with a structured survey. Results show that most of the diagrams had a transient nature because of the high cost of changing whiteboard sketches to electronic renderings. Diagrams that documented design decisions were often externalized in these temporary drawings and then subsequently lost. Current visualization tools and the software development practices that we observed do not solve these issues, but these results suggest several directions for future research. Mauro Cherubini, Gina Venolia, Robert DeLine, Amy J. Ko |
CHI | 3 |
| 2007 | Information Needs in Collocated Software Development TeamsabstractPrevious research has documented the fragmented nature of software development work. To explain this in more detail, we analyzed software developers' day-to-day information needs. We observed seventeen developers at a large software company and transcribed their activities in go-minute sessions. We analyzed these logs for the information that developers sought, the sources that they used, and the situations th at prevented inform action from being acquired. We identified twenty-one information types and cataloged the outcome and source when each type of information was sought. The most frequently sought information included awareness about artifacts and coworkers. The most often deferred searches included knowledge about design and program behavior, such as why code was written a particular way, what a program was supposed to do, and the cause of a program state. Developers often had to defer tasks because the only source of knowledge was unavailable coworkers. Amy J. Ko, Robert DeLine, Gina Venolia |
ICSE | 2 |
| 2007 | Building an Ecologically valid, Large-scale Diagram to Help Developers Stay Oriented in Their CodeabstractThis paper presents the creation, deployment, and evaluation of a large-scale, spatially-stable, paper-based visualization of a software system. The visualization was created for a single team, who were involved systematically in its initial design and subsequent design iterations. The evaluation indicates that the visualization supported the "onboarding" scenario but otherwise failed to realize the research team's expectations. We present several lessons learned, and cautions to future research into large-scale, spatially-stable visualizations of software systems. Mauro Cherubini, Gina Venolia, Robert DeLine |
VL/HCC | 3 |
| 2006 | Maintaining mental models: a study of developer work habitsabstractTo understand developers' typical tools, activities, and practices and their satisfaction with each, we conducted two surveys and eleven interviews. We found that many problems arose because developers were forced to invest great effort recovering implicit knowledge by exploring code and interrupting teammates and this knowledge was only saved in their memory. Contrary to expectations that email and IM prevent expensive task switches caused by face-to-face interruptions, we found that face-to-face communication enjoys many advantages. Contrary to expectations that documentation makes understanding design rationale easy, we found that current design documents are inadequate. Contrary to expectations that code duplication involves the copy and paste of code snippets, developers reported several types of duplication. We use data to characterize these and other problems and draw implications for the design of tools for their solution. Thomas D. LaToza, Gina Venolia, Robert DeLine |
ICSE | 3 |
| 2006 | Code Thumbnails: Using Spatial Memory to Navigate Source CodeabstractModern development environments provide many features for navigating source code, yet studies show the developers still spend a tremendous amount of time just navigating. Since existing navigation features rely heavily on memorizing symbol names, we present a new design, called code thumbnails, intended to allow a developer to navigate source code by forming a spatial memory of it. To aid intra-file navigation, we add a thumbnail image of the file to the scrollbar, which makes any part of the file one click away. To aid interfile navigation, we provide a desktop of file thumbnail images, which make any part of any file one click away. We did a formative evaluation of the design with eleven experienced developers and present the results Robert DeLine, Mary Czerwinski, Brian Meyers, Gina Venolia, Steven Mark Drucker, George G. Robertson |
VL/HCC | 1 |
| 2005 | Easing Program Comprehension by Sharing Navigation DataabstractLarge software projects often require a programmer to make changes to unfamiliar source code. This paper describes a set of tools, called Team Tracks, designed to ease program comprehension by showing the source code navigation patterns of fellow development team members. One technique shows a list of related items, given that the user is viewing a given method or class. Another technique shows the favorite classes, by showing a class hierarchy view that hides less frequently visited classes, methods, and members. Two user studies, a laboratory study and a field study, were run to evaluate the effectiveness of these techniques. The results of the two studies demonstrate that sharing navigation data can improve program comprehension and is subjectively preferred by users. Robert DeLine, Mary Czerwinski, George G. Robertson |
VL/HCC | 1 |
| 2004 | Typestates for Objects
Robert DeLine, Manuel Fähndrich |
ECOOP | 1 |
| 2002 | Adoption and Focus: Practical Linear Types for Imperative ProgrammingabstractA type system with linearity is useful for checking software protocols andresource management at compile time. Linearity provides powerful reasoning about state changes, but at the price of restrictions on aliasing. The hard division between linear and nonlinear types forces the programmer to make a trade-off between checking a protocol on an object and aliasing the object. Most onerous is the restriction that any type with a linear component must itself be linear. Because of this, checking a protocol on an object imposes aliasing restrictions on any data structure that directly or indirectly points to the object. We propose a new type system that reduces these restrictions with the adoption and focus constructs. Adoption safely allows a programmer to alias objects on which she is checking protocols, and focus allows the reverse. A programmer can alias data structures that point to linear objects and use focus for safe access to those objects. We discuss how we implemented these ideas in the Vault programming language. Manuel Fähndrich, Robert DeLine |
PLDI | 2 |
| 2001 | Enforcing High-Level Protocols in Low-Level SoftwareabstractThe reliability of infrastructure software, such as operating sys-tems and web servers, is often hampered by the mismanagement of resources, such as memory and network connections. The Vault programming language allows a programmer to describe resource management protocols that the compiler can statically enforce. Such a protocol can specify that operations must be performed in a certain order and that certain operations must be performed before accessing a given data object. Furthermore, Vault enforces stati-cally that resources cannot be leaked. We validate the utility of our approach by enforcing protocols present in the interface between the Windows 2000 kernel and its device drivers. 1. Robert DeLine, Manuel Fähndrich |
PLDI | 1 |
| 2001 | Avoiding Packaging Mismatch with Flexible PackagingabstractTo integrate a software component into a system, it must interact properly with the system's other components. Unfortunately, the decisions about how a component is to interact with other components are typically committed long before the moment of integration and are difficult to change. This paper introduces the flexible packaging method, which allows a component developer to defer some decisions about component interaction until system integration time. The method divides the component's source into two pieces: the ware, which encapsulates the component's functionality; and the packager, which encapsulates the details of interaction. Both the ware and the packager are independently reusable. A ware, as a reusable part, allows a given piece of functionality to be employed in systems in different architectural styles. A packager, as a reusable part, encapsulates conformance to a component standard, like an ActiveX control or an ODBC database accessor. Because the packager's source code is often formulaic, a tool is provided to generate the packager's source from a high-level description of the intended interaction, a description written in the architectural description language UniCon. The method and tools are evaluated with a series of experiments in which three wares and nine types of packaging are combined to form thirteen components. Robert DeLine |
IEEE Trans. Software Eng. | 1 |
| 2001 | Corrections to 'Avoiding Packaging Mismatch with Flexible Packaging'abstractTo integrate a software component into a system, it must interact properly with the system’s other components. Unfortunately, the decisions about how a component is to interact with other components are typically committed long before the moment of integration and are difficult to change. This paper introduces the Flexible Packaging method, which allows a component developer to defer some decisions about component interaction until system integration time. The method divides the component’s source into two pieces: the ware, which encapsulates the component’s functionality; and the packager, which encapsulates the details of interaction. Both the ware and the packager are independently reusable. A ware, as a reusable part, allows a given piece of functionality to be employed in systems in different architectural styles. A packager, as a reusable part, encapsulates conformance to a component standard, like an ActiveX control or an odbc database accessor. Because the packager’s source code is often formulaic, a tool is provided to generate the packager’s source from a highlevel description of the intended interaction, a description written in the architectural description language UniCon. The method and tools are evaluated with two case studies, an image viewer and a database updater. Robert DeLine |
IEEE Trans. Software Eng. | 1 |
| 1999 | Avoiding Packaging Mismatch with Flexible PackagingabstractArticle Free Access Share on Avoiding packaging mismatch with flexible packaging Author: Robert DeLine Carnegie Mellon University, 5000 Forbes Ave., Pittsburgh PA Carnegie Mellon University, 5000 Forbes Ave., Pittsburgh PAView Profile Authors Info & Claims ICSE '99: Proceedings of the 21st international conference on Software engineeringMay 1999 Pages 97–106https://doi.org/10.1145/302405.302456Published:16 May 1999Publication History 28citation364DownloadsMetricsTotal Citations28Total Downloads364Last 12 Months30Last 6 weeks1 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteeReaderPDF Robert DeLine |
ICSE | 1 |
| 1997 | Lessons on Converting Batch Systems to Support Interaction (Experience Report)abstractSoftware often evolves from batch to interactive use.Because these two usage styles are so different, batch systems usually require substantial changes to support interactive use.Specific issues that arise during conversion include assumptions about duration of system execution, incremental and partial processing, scope of processing, unordered and repeated processing, and error handling.Addressing these issues affects the implementation in the areas of memory management, assumptions and invariants, computational organization, and error handling.We use as a working example our conversion of the batch processor .for the UniCon architecture description tool into an interactive architecture development tool.To capture the lessons for practitioners undertaking this type of conversion, we summarize with a checklist of design and implementation considerations. Robert DeLine, Gregory Zelesnik, Mary Shaw |
ICSE | 1 |
| 1995 | Abstractions for Software Architecture and Tools to Support ThemabstractArchitectures for software use rich abstractions and idioms to describe system components, the nature of interactions among the components, and the patterns that guide the composition of components into systems. These abstractions are higher level than the elements usually supported by programming languages and tools. They capture packaging and interaction issues as well as computational functionality. Well-established (if informal) patterns guide the architectural design of systems. We sketch a model for defining architectures and present an implementation of the basic level of that model. Our purpose is to support the abstractions used in practice by software designers. The implementation provides a testbed for experiments with a variety of system construction mechanisms. It distinguishes among different types of components and different ways these components can interact. It supports abstract interactions such as data flow and scheduling on the same footing as simple procedure call. It can express and check appropriate compatibility restrictions and configuration constraints. It accepts existing code as components, incurring no runtime overhead after initialization. It allows easy incorporation of specifications and associated analysis tools developed elsewhere. The implementation provides a base for extending the notation and validating the model.> Mary Shaw, Robert DeLine, Daniel V. Klein, Theodore L. Ross, David M. Young, Gregory Zelesnik |
IEEE Trans. Software Eng. | 2 |
| 1994 | Developing Calendar Visualizers for the Information VisualizerabstractThe increasing mass of information confronting a business or an individual have created a demand for information management applications. Time-based information, in particular, is an important part of many information access tasks. This paper explores how to use 3D graphics and interactive animation to design and implement visualizers that improve access to large masses of time-based information. Two new visualizers have been developed for the Information Visualizer: 1) the Spiral Calendar was designed for rapid access to an individual's daily schedule, and 2) the Time Lattice was designed for analyzing the time relationships among the schedules of groups of people. The Spiral Calendar embodies a new 3D graphics technique for integrating detail and context by placing objects in a 3D spiral. It demonstrates that advanced graphics techniques can enhance routine office information tasks. The Time Lattice is formed by aligning a collection of 2D calendars. 2D translucent shadows provide views and interactive access to the resulting complex 3D object. The paper focuses on how these visualizations were developed. The Spiral Calendar, in particular, has gone through an entire cycle of development, including design, implementation, evaluation, revision and reuse. Our experience should prove useful to others developing user interfaces based on advanced graphics. Jock D. Mackinlay, George G. Robertson, Robert DeLine |
ACM Symposium on User Interface Software and Technology | 3 |
| 1992 | Lessons Learned from SUIT, the Simple User Interface ToolkitabstractIn recent years, the computer science community has realized the advantages of GUIs (Graphical User Interfaces). Because high-quality GUIs are difficult to build, support tools such as UIMSs, UI Toolkits, and Interface Builders have been developed. Although these tools are powerful, they typically make two assumptions: first, that the programmer has some familiarity with the GUI model, and second, that he is willing to invest several weeks becoming proficient with the tool. These tools typically operate only on specific platforms, such as DOS, the Macintosh, or UNIX/X-windows. The existing tools are beyond the reach of most undergraduate computer science majors, or professional programmers who wish to quickly build GUIs without investing the time to become specialists in GUI design. For this class of users, we developed SUIT , the S imple U ser I interface T oolkit. SUIT is an attempt to distill the fundamental components of an interface builder and GUI toolkit, and to explain those concepts with the tool itself, all in a short period of time. We have measured that college juniors with no previous GUI programming experience can use SUIT productively after less than three hours. SUIT is a C subroutine library which provides an external control UIMS, an interactive layout editor, and a set of standard “widgets,” such as sliders, buttons, and check boxes. SUIT-based applications run transparently across the Macintosh, DOS, and UNIX/X platforms. SUIT has been exported to hundreds of external sites on the internet. This paper describes SUIT's architecture, the design decisions we made during its development, and the lessons we learned from extensive observations of over 120 users. Randy F. Pausch, Matthew Conway, Robert DeLine |
ACM Trans. Inf. Syst. | 3 |
| 1991 | SUIT: the Pascal of user interface toolkitsabstractArticle SUIT: the Pascal of user interface toolkits Share on Authors: Randy Pausch Computer Science Department, Thornton Hall, University of Virginia, Charlottesville, VA Computer Science Department, Thornton Hall, University of Virginia, Charlottesville, VAView Profile , Nathaniel R. Young Computer Science Department, Thornton Hall, University of Virginia, Charlottesville, VA Computer Science Department, Thornton Hall, University of Virginia, Charlottesville, VAView Profile , Robert DeLine Computer Science Department, Thornton Hall, University of Virginia, Charlottesville, VA Computer Science Department, Thornton Hall, University of Virginia, Charlottesville, VAView Profile Authors Info & Claims UIST '91: Proceedings of the 4th annual ACM symposium on User interface software and technologyNovember 1991 Pages 117–125https://doi.org/10.1145/120782.120795Published:11 November 1991 12citation276DownloadsMetricsTotal Citations12Total Downloads276Last 12 Months8Last 6 weeks2 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteGet Access Randy F. Pausch, Nathaniel R. Young II, Robert DeLine |
UIST | 3 |