Kevin J. Sullivan

dblp:s/KevinJSullivan · DBLP profile ↗
← Back
55ranked-venue papers
17as first author
1since 2021 · last 2022
0009-0003-0160-0104ORCID · conflict

Domains — the database's venue-derived domains; a paper can count in several

Software engineering, systems software and programming languages · 51 · 17 first-author · 1 since 2021Artificial intelligence and machine learning · 2Applied, interdisciplinary, general and emerging computing · 2Security and privacy · 1Theory of computation · 1

Expertise — from the expertise taxonomy: the topics of the expert's papers under the CCF categories. A weight counts papers with recency: 1 for a paper about the topic, 0.3 when the topic is its context, halved every five years.

Software engineering, system software, and programming languages
37 papers
Requirements engineering and software design · 47% Software maintenance and evolution · 20% Software testing · 15%
Computer architecture, parallel and distributed computing, and storage systems
2 papers
Distributed systems · 62% Embedded and real-time systems · 27% Cloud and datacenter computing · 11%

Topics — the 30 heaviest of 57, each with the papers that count most for it

TopicWeightPapersLastEvidence papers
Requirements engineering and software design
software architecture
0.6162010
Modular aspect-oriented design with XPIs · ACM Trans. Softw. Eng. Methodol. 2010
Architectural style as an independent variable · ASE 2010
Automatic modularity conformance checking · ICSE 2008
Software testing
configuration space exploration
0.612022
ConEx: Efficient Exploration of Big-Data System Configurations for Better Performance · IEEE Trans. Software Eng. 2022
Software maintenance and evolution
software configuration
0.612022
ConEx: Efficient Exploration of Big-Data System Configurations for Better Performance · IEEE Trans. Software Eng. 2022
Requirements engineering and software design
model-driven engineering
0.322013
Bottom-up model-driven development · ICSE 2013
Architectural style as an independent variable · ASE 2010
Programming languages and type systems
aspect-oriented programming
0.242009
Unifying aspect- and object-oriented design · ACM Trans. Softw. Eng. Methodol. 2009
Classpects: unifying aspect- and object-oriented language design · ICSE 2005
Understanding Aspects via Implicit Invocation · ASE 2004
Requirements engineering and software design › modularity
modularity analysis
0.222012
A formal model for automated software modularity and evolvability analysis · ACM Trans. Softw. Eng. Methodol. 2012
Modularity Analysis of Logical Design Models · ASE 2006
Programming languages and type systems
language design
0.232009
Unifying aspect- and object-oriented design · ACM Trans. Softw. Eng. Methodol. 2009
Classpects: unifying aspect- and object-oriented language design · ICSE 2005
Eos: instance-level aspects for integrated system design · ESEC / SIGSOFT FSE 2003
Requirements engineering and software design › design process
design space exploration
0.212014
TradeMaker: automated dynamic analysis of synthesized tradespaces · ICSE 2014
Requirements engineering and software design
modularity
0.232009
Unifying aspect- and object-oriented design · ACM Trans. Softw. Eng. Methodol. 2009
Information hiding interfaces for aspect-oriented design · ESEC/SIGSOFT FSE 2005
The structure and value of modularity in software design · ESEC / SIGSOFT FSE 2001
Requirements engineering and software design › software design methodology
aspect-oriented design
0.222010
Modular aspect-oriented design with XPIs · ACM Trans. Softw. Eng. Methodol. 2010
Information hiding interfaces for aspect-oriented design · ESEC/SIGSOFT FSE 2005
Requirements engineering and software design › software design notation
design structure matrix
0.132006
Modularity Analysis of Logical Design Models · ASE 2006
Simon: modeling and analysis of design space structures · ASE 2005
The structure and value of modularity in software design · ESEC / SIGSOFT FSE 2001
Program analysis
dynamic analysis
0.122017
Automated Synthesis and Dynamic Analysis of Tradeoff Spaces for Object-Relational Mapping · IEEE Trans. Software Eng. 2017
TradeMaker: automated dynamic analysis of synthesized tradespaces · ICSE 2014
Programming languages and type systems › language semantics
compositionality
0.122009
Unifying aspect- and object-oriented design · ACM Trans. Softw. Eng. Methodol. 2009
Classpects: unifying aspect- and object-oriented language design · ICSE 2005
Requirements engineering and software design › software architecture
architectural style
0.112010
Architectural style as an independent variable · ASE 2010
Requirements engineering and software design › software architecture › architectural design
architecture synthesis
0.112010
Architectural style as an independent variable · ASE 2010
Requirements engineering and software design › design cognition
design creativity
0.112010
Better science through art · OOPSLA 2010
Software maintenance and evolution
software modularization
0.112010
Modular aspect-oriented design with XPIs · ACM Trans. Softw. Eng. Methodol. 2010
Software testing › combinatorial testing
bounded exhaustive testing
0.122005
Software Assurance by Bounded Exhaustive Testing · IEEE Trans. Software Eng. 2005
Software assurance by bounded exhaustive testing · ISSTA 2004
Requirements engineering and software design
formal specification
0.122005
Software Assurance by Bounded Exhaustive Testing · IEEE Trans. Software Eng. 2005
Software assurance by bounded exhaustive testing · ISSTA 2004
Software testing
test input generation
0.122005
Software Assurance by Bounded Exhaustive Testing · IEEE Trans. Software Eng. 2005
Software assurance by bounded exhaustive testing · ISSTA 2004
Software maintenance and evolution › software evolution
software evolvability
0.122012
Simon: modeling and analysis of design space structures · ASE 2005
A formal model for automated software modularity and evolvability analysis · ACM Trans. Softw. Eng. Methodol. 2012
Programming languages and type systems
object-oriented programming
0.112009
Unifying aspect- and object-oriented design · ACM Trans. Softw. Eng. Methodol. 2009
Requirements engineering and software design › software design principles
information hiding
0.122005
Information hiding interfaces for aspect-oriented design · ESEC/SIGSOFT FSE 2005
The structure and value of modularity in software design · ESEC / SIGSOFT FSE 2001
Requirements engineering and software design › software architecture › component-based software engineering
component-based development
0.132003
Galileo: a tool built from mass-market applications · ICSE 2000
Multiple mass-market applications as components · ICSE 2000
Sound Methods and Effective Tools for Engineering Modeling and Analysis · ICSE 2003
Software testing
specification-based testing
0.112005
Software Assurance by Bounded Exhaustive Testing · IEEE Trans. Software Eng. 2005
Requirements engineering and software design › software architecture › architectural style
implicit invocation
0.012004
Understanding Aspects via Implicit Invocation · ASE 2004
Empirical software engineering
software economics
0.012002
Software engineering economics: background, current practices, and future directions · ICSE 2002
Services computing and microservices
web services
0.012002
Web services engineering: promises and challenges · ICSE 2002
Requirements engineering and software design › software architecture
architecture-implementation conformance
0.012001
Software Reflexion Models: Bridging the Gap between Design and Implementation · IEEE Trans. Software Eng. 2001
Software maintenance and evolution
software evolution
0.032001
Software Reflexion Models: Bridging the Gap between Design and Implementation · IEEE Trans. Software Eng. 2001
Reconciling Environment Integration and Software Evolution · ACM Trans. Softw. Eng. Methodol. 1992
Exploration Harnesses: Tool-Supported Interactive Discovery of Commercial Component Properties · ASE 1999

Methods — techniques the papers use, named apart from their topics

predictive model learning · 0.6genetic algorithm · 0.6evolutionary markov chain monte carlo · 0.6synthesis of design alternatives · 0.3dynamic analysis · 0.3static analysis · 0.2load synthesis · 0.2design space synthesis · 0.2model transformation · 0.2formal specification · 0.1dominance relation · 0.1divide-and-conquer · 0.1multiple model control · 0.0hierarchical adaptive control · 0.0discrete-state distributed control · 0.0web-based meta-data interfaces · 0.0architectural aspect · 0.0formal modeling · 0.0
YearPublicationVenuePosition
2022 ConEx: Efficient Exploration of Big-Data System Configurations for Better Performance
abstract
Configuration space complexity makes the big-data software systems hard to configure well. Consider Hadoop, with over nine hundred parameters, developers often just use thedefaultconfigurations provided with Hadoop distributions. The opportunity costs in lost performance are significant. Popular learning-based approaches to auto-tune software does not scale well for big-data systems because of the high cost of collecting training data. We present a new method based on a combination ofEvolutionary Markov Chain Monte Carlo (EMCMC)sampling and cost reduction techniques to find better-performing configurations for big data systems. For cost reduction, we developed and experimentally tested and validated two approaches: using scaled-up big data jobs as proxies for the objective function for larger jobs and using a dynamic job similarity measure to infer that results obtained for one kind of big data problem will work well for similar problems. Our experimental results suggest that our approach promises to improve the performance of big data systems significantly and that it outperforms competing approaches based on random sampling, basic genetic algorithms (GA), and predictive model learning. Our experimental results support the conclusion that our approach strongly demonstrates the potential to improve the performance of big data systems significantly and frugally.
Rahul Krishna, Chong Tang 0001, Kevin J. Sullivan, Baishakhi Ray
IEEE Trans. Software Eng.3
2017 Automated Synthesis and Dynamic Analysis of Tradeoff Spaces for Object-Relational Mapping
abstract
Producing software systems that achieve acceptable tradeoffs among multiple non-functional properties remains a significant engineering problem. We propose an approach to solving this problem that combines synthesis of spaces of design alternatives from logical specifications and dynamic analysis of each point in the resulting spaces. We hypothesize that this approach has potential to help engineers understand important tradeoffs among dynamically measurable properties of system components at meaningful scales within reach of existing synthesis tools. To test this hypothesis, we developed tools to enable, and we conducted, a set of experiments in the domain of relational databases for object-oriented data models. For each of several data models, we used our approach to empirically test the accuracy of a published suite of metrics to predict tradeoffs based on the static schema structure alone. The results show that exhaustive synthesis and analysis provides a superior view of the tradeoff spaces for such designs. This work creates a path forward toward systems that achieve significantly better tradeoffs among important system properties.
Hamid Bagheri, Chong Tang 0001, Kevin J. Sullivan
IEEE Trans. Software Eng.3
2016 Model-driven synthesis of formally precise, stylized software architectures
abstract
Abstract Reliably producing software architectures in selected architectural styles requires significant expertise yet remains difficult and error-prone. Our research goals are to better understand the nature of style-specific architectures, and relieve architects of the need to produce such architectures by hand. To achieve our goals, this paper introduces a formally precise approach to separate architectural style design decisions from application-specific decisions, and then uses these separate decisions as inputs to an automated synthesizer. This in effect supports a model-driven development (MDD) approach to architecture synthesis with style as a separate design variable. We claim that it is possible to formalize this separation of concerns, long implicit in software engineering research; to automatically synthesize style-specific architectures; and thereby to improve software design productivity and quality. To test these claims, we employed a combination of experimental systems and case study methods: we developed an MDD tool and used it to carry out case studies using Kitchenham’s methods. Our contributions include: a theoretical framework formalizing our separation of concerns and synthesis approach; an MDD framework, Monarch; and results of case studies that we interpret as supporting our claims. This work advances our understanding of software architectural style as a formal refinement; makes application descriptions an explicit subject of study; and suggests that synthesis of architectures can improve software productivity and quality.
Hamid Bagheri, Kevin J. Sullivan
Formal Aspects Comput.2
2015 Real-World Types and Their Application
John C. Knight, Kevin J. Sullivan
SAFECOMP3
2015 Toward a science of learning systems: a research agenda for the high-functioning Learning Health System
abstract
OBJECTIVE: The capability to share data, and harness its potential to generate knowledge rapidly and inform decisions, can have transformative effects that improve health. The infrastructure to achieve this goal at scale--marrying technology, process, and policy--is commonly referred to as the Learning Health System (LHS). Achieving an LHS raises numerous scientific challenges. MATERIALS AND METHODS: The National Science Foundation convened an invitational workshop to identify the fundamental scientific and engineering research challenges to achieving a national-scale LHS. The workshop was planned by a 12-member committee and ultimately engaged 45 prominent researchers spanning multiple disciplines over 2 days in Washington, DC on 11-12 April 2013. RESULTS: The workshop participants collectively identified 106 research questions organized around four system-level requirements that a high-functioning LHS must satisfy. The workshop participants also identified a new cross-disciplinary integrative science of cyber-social ecosystems that will be required to address these challenges. CONCLUSIONS: The intellectual merit and potential broad impacts of the innovations that will be driven by investments in an LHS are of great potential significance. The specific research questions that emerged from the workshop, alongside the potential for diverse communities to assemble to address them through a 'new science of learning systems', create an important agenda for informatics and related disciplines.
Charles P. Friedman, Joshua C. Rubin, Jeffrey S. Brown, Melinda Buntin, Milton Corn, Lynn Etheredge, Carl A. Gunter, Mark A. Musen, Richard Platt, William W. Stead, Kevin J. Sullivan, Douglas Van Houweling
J. Am. Medical Informatics Assoc.11
2014 TradeMaker: automated dynamic analysis of synthesized tradespaces
abstract
System designers today are focusing less on point solutions for complex systems and more on design spaces, often with a focus on understanding tradeoffs among non-functional properties across such spaces. This shift places a premium on the efficient comparative evaluation of non-functional properties of designs in such spaces. While static analysis of designs will sometimes suffice, often one must run designs dynamically, under comparable loads, to determine properties and tradeoffs. Yet variant designs often present variant interfaces, requiring that common loads be specialized to many interfaces. The main contributions of this paper are a mathematical framework, architecture, and tool for specification-driven synthesis of design spaces and common loads specialized to individual designs for dynamic tradeoff analysis of non-functional properties in large design spaces. To test our approach we used it to run an experiment to test the validity of static metrics for object-relational database mappings, requiring design space and load synthesis for, and dynamic analysis of, hundreds of database designs.
Hamid Bagheri, Chong Tang 0001, Kevin J. Sullivan
ICSE3
2013 Bottom-up model-driven development
abstract
Prominent researchers and leading practitioners are questioning the long-term viability of model-driven development (MDD). Finkelstein recently ranked MDD as a bottom-ten research area, arguing that an approach based entirely on development and refinement of abstract representations is untenable. His view is that working with concrete artifacts is necessary for learning what to build and how to build it. What if this view is correct? Could MDD be rescued from such a critique? We suggest the answer is yes, but that it requires an inversion of traditional views of transformational MDD. Rather than develop complete, abstract system models, in ad-hoc modeling languages, followed by top-down synthesis of hidden concrete artifacts, we envision that engineers will continue to develop concrete artifacts, but over time will recognize patterns and concerns that can profitably be lifted, from the bottom-up, to the level of partial models, in general-purpose specification languages, from which visible concrete artifacts are generated, becoming part of the base of both concrete and abstract artifacts for subsequent rounds of development. This paper reports on recent work that suggests this approach is viable, and explores ramifications of such a rethinking of MDD. Early validation flows from experience applying these ideas to a healthcare-related experimental system in our lab.
Hamid Bagheri, Kevin J. Sullivan
ICSE2
2012 Pol: specification-driven synthesis of architectural code frameworks for platform-based applications
abstract
Developing applications that use complex platforms for functionalities such as authentication and messaging is hard. Model-driven engineering promises to help, but transformation systems are themselves hard to produce. We contribute a new approach using constraint-based synthesis of partial code frameworks that developers complete by hand without the need for hand-coded transformation systems. Rather, synthesis is driven by formal, partial specifications of target platforms and application architectures, and by design (code) fragments encoding application-specific platform us-age patterns. We present results of an early evaluation using the case study method to test hypotheses of feasibility and potential industrial utility, using a laboratory model of a nationwide health information network as a subject system.
Hamid Bagheri, Kevin J. Sullivan
GPCE2
2012 Spacemaker: Practical Formal Synthesis of Tradeoff Spaces for Object-Relational Mapping
Hamid Bagheri, Kevin J. Sullivan, Sang Hyuk Son
SEKE2
2012 A formal model for automated software modularity and evolvability analysis
abstract
Neither the nature of modularity in software design, characterized as a property of the structure of dependencies among design decisions, or its economic value are adequately well understood. One basic problem is that we do not even have a sufficiently clear definition of what it means for one design decision to depend on another. The main contribution of this work is one possible mathematically precise definition of dependency based on an augmented constraint network model. The model provides an end-to-end account of the connection between modularity and its value in terms of options to make adaptive changes in uncertain and changing design spaces. We demonstrate the validity and theoretical utility of the model, showing that it is consistent with, and provides new insights into, several previously published results in design theory.
Yuanfang Cai, Kevin J. Sullivan
ACM Trans. Softw. Eng. Methodol.2
2011 A Formal Approach for Incorporating Architectural Tactics into the Software Architecture
Hamid Bagheri, Kevin J. Sullivan
SEKE2
2010 Architectural style as an independent variable
abstract
A key idea in modern software engineering is that we can and should make architectural style choices separately from choices about many other system properties. There is a fundamental separation of concerns implicit in this idea: given an application model that expresses system properties independently of architectural style, we can choose a compatible architectural style and then map the application model to one or more architectural models (architectures) in this style. The problem is that we do not have a formal account of this separation of concerns, or the associated architectural maps, sufficient to enable automated synthesis of architectures from application models and architecture style specifications. The contribution of this work is such an account and a demonstration that it enables automated formal derivation of style-specific architectures.
Hamid Bagheri, Yuanyuan Song, Kevin J. Sullivan
ASE3
2010 Monarch: Model-Based Development of Software Architectures
Hamid Bagheri, Kevin J. Sullivan
MoDELS (2)2
2010 Better science through art
abstract
How do artists and scientists work? The same.
Richard P. Gabriel, Kevin J. Sullivan
OOPSLA2
2010 Modular aspect-oriented design with XPIs
abstract
The emergence of aspect-oriented programming (AOP) languages has provided software designers with new mechanisms and strategies for decomposing programs into modules and composing modules into systems. What we do not yet fully understand is how best to use such mechanisms consistent with common modularization objectives such as the comprehensibility of programming code, its parallel development, dependability, and ease of change. The main contribution of this work is a new form of information-hiding interface for AOP that we call the crosscut programming interface, or XPI. XPIs abstract crosscutting behaviors and make these abstractions explicit. XPIs can be used, albeit with limited enforcement of interface rules, with existing AOP languages, such as AspectJ. To evaluate our notion of XPIs, we have applied our XPI-based design methodology to a medium-sized network overlay application called Hypercast. A qualitative and quantitative analysis of existing AO design methods and XPI-based design method shows that our approach produces improvements in program comprehensibility, in opportunities for parallel development, and in the ease when code can be developed and changed.
Kevin J. Sullivan, William G. Griswold, Hridesh Rajan, Yuanyuan Song, Yuanfang Cai, Macneil Shonle, Nishit Tewari
ACM Trans. Softw. Eng. Methodol.1
2009 Unifying aspect- and object-oriented design
abstract
The contribution of this work is the design and evaluation of a programming language model that unifies aspects and classes as they appear in AspectJ-like languages. We show that our model preserves the capabilities of AspectJ-like languages, while improving the conceptual integrity of the language model and the compositionality of modules. The improvement in conceptual integrity is manifested by the reduction of specialized constructs in favor of uniform orthogonal constructs. The enhancement in compositionality is demonstrated by better modularization of integration and higher-order crosscutting concerns.
Hridesh Rajan, Kevin J. Sullivan
ACM Trans. Softw. Eng. Methodol.2
2008 Automatic modularity conformance checking
abstract
According to Parnas’s information hiding principle and Baldwin and Clark’s design rule theory, the key step to decomposing a system into modules is to determine the design rules (or in Parnas’s terms, interfaces) that decouple otherwise coupled design decisions and to hide decisions that are likely to change in independent modules. Given a modular design, it is often difficult to determine whether and how its implementation realizes the designed modularity. Manually comparing code with abstract design is tedious and error-prone. We present an automated approach to check the conformance of implemented modularity to designed modularity, using design structure matrices as a uniform representation for both. Our experiments suggest that our approach has the potential to manifest the decoupling effects of design rules in code, and to detect modularity deviation caused by implementation faults. We also show that design and implementation models together provide a comprehensive view of modular structure that makes certain implicit dependencies within code explicit.
Sunny Wong 0001, Yuanfang Cai, Yuanyuan Song, Kevin J. Sullivan
ICSE4
2006 The 8th international workshop on economics-driven software engineering research
abstract
This paper presents the 8th International Workshop on Economics-Driven Software Engineering Research (EDSER-8).
Rick Kazman, Kevin J. Sullivan
ICSE2
2006 Modularity Analysis of Logical Design Models
abstract
Traditional design representations are inadequate for generalized reasoning about modularity in design and its technical and economic implications. We have developed an architectural modeling and analysis approach, and automated tool support, for improved reasoning in these terms. However, the complexity of constraint satisfaction limited the size of models that we could analyze. The contribution of this paper is a more scalable approach. We exploit the dominance relations in our models to guide a divide-and-conquer algorithm, which we have implemented it in our Simon tool. We evaluate its performance in case studies. The approach reduced the time needed to analyze small but representative models from hours to seconds. This work appears to make our modeling and analysis approach practical for research on the evolvability and economic properties of software design architectures
Yuanfang Cai, Kevin J. Sullivan
ASE2
2005 Classpects: unifying aspect- and object-oriented language design
abstract
The contribution of this work is the design, implementation, and early evaluation of a programming language that unifies classes and aspects. We call our new module construct the classpect. We make three basic claims. First, we can realize a unified design without significantly compromising the expressiveness of current aspect languages. Second, such a design improves the conceptual integrity of the programming model. Third, it significantly improves the compositionality of aspect modules, expanding the program design space from the two-layered model of AspectJ-like languages to include hierarchical structures. To support these claims, we present the design and implementation of Eos-U, an AspectJ-like language based on C# that supports classpects as the basic unit of modularity. We show that Eos-U supports layered designs in which classpects separate integration concerns flexibly at multiple levels of composition. The underpinnings of our design include support for aspect instantiation under program control, instance-level advising, advising as a general alternative to object-oriented method invocation and overriding, and the provision of a separate join-point-method binding construct.
Hridesh Rajan, Kevin J. Sullivan
ICSE2
2005 7th international workshop on economics-driven software engineering research
abstract
The 7th International Workshop on Economics-Driven Software Engineering Research (EDSER-7) continues to be the leading forum for the discussion of emerging research ideas in software economics. The focus of the workshop is on the use of economic models for reasoning about technical issues and decisions in the definition, design, development, deployment, and evolution of software and software-intensive systems.
Kevin J. Sullivan
ICSE1
2005 Science of design
abstract
In this plenary panel session, three distinguished scholars of design will provide a range of perspectives on a science of design for software and software-intensive systems. The session will include brief presentations by the panelists as well as dialog among the panelists and with members of the audience.
Kevin J. Sullivan, Jeff Magee
ICSE1
2005 Simon: modeling and analysis of design space structures
abstract
The structure of the coupling relation on design decisions is a key factor influencing the evolvability properties and the economic value of a design. The work of Baldwin and Clark is an important step toward a theory of the relationship between structure and value. A key step to enabling rigorous validation and perhaps the eventual use of their ideas for software engineering is formalization of their model. In this paper, we present a brief overview of such a formal model and a prototype software tool, Simon, implementing it. We present Simon's functions for deriving design structure matrices and computing impacts of changes in design decisions, and we sketch an initial experimental evaluation in the form of a replication study of our earlier analysis of Parnas's 1972 paper on information hiding modularity.
Yuanfang Cai, Kevin J. Sullivan
ASE2
2005 Information hiding interfaces for aspect-oriented design
abstract
The growing popularity of aspect-oriented languages, such as AspectJ, and of corresponding design approaches, makes it important to learn how best to modularize programs in which aspect-oriented composition mechanisms are used. We contribute an approach to information hiding modularity in programs that use quantified advising as a module composition mechanism. Our approach rests on a new kind of interface: one that abstracts a crosscutting behavior, decouples the design of code that advises such a behavior from the design of the code to be advised, and that can stipulate behavioral contracts. Our interfaces establish design rules that govern how specific points in program execution are exposed through a given join point model and how conforming code on either side should behave. In a case study of the HyperCast overlay network middleware system, including a real options analysis, we compare the widely cited oblivious design approach with our own, showing significant weaknesses in the former and benefits in the latter.
Kevin J. Sullivan, William G. Griswold, Yuanyuan Song, Yuanfang Cai, Macneil Shonle, Nishit Tewari, Hridesh Rajan
ESEC/SIGSOFT FSE1
2005 Software Assurance by Bounded Exhaustive Testing
abstract
Bounded exhaustive testing (BET) is a verification technique in which software is automatically tested for all valid inputs up to specified size bounds. A particularly interesting case of BET arises in the context of systems that take structurally complex inputs. Early research suggests that the BET approach can reveal faults in small systems with inputs of low structural complexity, but its potential utility for larger systems with more complex input structures remains unclear. We set out to test its utility on one such system. We used Alloy and TestEra to generate inputs to test the Galileo dynamic fault tree analysis tool, for which we already had both a formal specification of the input space and a test oracle. An initial attempt to generate inputs using a straightforward translation of our specification to Alloy did not work well. The generator failed to generate inputs to meaningful bounds. We developed an approach in which we factored the specification, used TestEra to generate abstract inputs based on one factor, and passed the results through a postprocessor that reincorporated information from the second factor. Using this technique, we were able to generate test inputs to meaningful bounds, and the inputs revealed nontrivial faults in the Galileo implementation, our specification, and our oracle. Our results suggest that BET, combined with specification abstraction and factoring techniques, could become a valuable addition to our verification toolkit and that further investigation is warranted.
David Coppit, Jinlin Yang, Sarfraz Khurshid, Wei Le, Kevin J. Sullivan
IEEE Trans. Software Eng.5
2004 Software assurance by bounded exhaustive testing
abstract
The contribution of this paper is an experiment that shows the potential value of a combination of selective reverse engineering to formal specifications and bounded exhaustive testing to improve the assurance levels of complex software. A key problem is to scale up test input generation so that meaningful results can be obtained. We present an approach, using Alloy and TestEra for test input generation, which we evaluate by experimental application to the Galileo dynamic fault tree analysis tool.
Kevin J. Sullivan, Jinlin Yang, David Coppit, Sarfraz Khurshid, Daniel Jackson 0001
ISSTA1
2004 Understanding Aspects via Implicit Invocation
Hridesh Rajan, Kevin J. Sullivan
ASE3
2003 Sound Methods and Effective Tools for Engineering Modeling and Analysis
abstract
Modeling and analysis is indispensable in engineering. To be safe and effective, a modeling method requires a language with a validated semantics; feature-rich, easy-to-use, dependable tools; and low engineering costs. Today we lack adequate means to develop such methods. We present a partial solution combining two techniques: formal methods for language design, and package-oriented programming for function and usability at low cost. We have evaluated the approach in an end-to-end experiment. We deployed an existing reliability method to NASA in a package-oriented tool and surveyed engineers to assess its usability. We formally specified, improved, and validated the language. To assess cost, we built a package-based tool for the new language. Our data show that the approach can enable cost-effective deployment of sound methods by effective tools.
David Coppit, Kevin J. Sullivan
ICSE2
2003 Shared Semantic Domains for Computational Reliability Engineering
abstract
Modeling languages and the software tools which support them are essential to engineering. However, as these languages become more sophisticated, it becomes difficult to assure both the validity of their semantic specifications and the dependability of their program implementations. To ameliorate this problem we propose to develop shared semantic domains and corresponding implementations for families of related modeling languages. The idea is to amortize investments at the intermediate level across multiple language definitions and implementations. To assess the practicality of this approach for modeling languages, we applied it to two languages for reliability modeling and analysis. In earlier work, we developed the intermediate semantic domain of failure automata (FA), which we used to formalize the semantics of dynamic fault trees (DFTs). in this paper, we show that a variant of the original FA can serve as a common semantic domain for both DFTs and reliability block diagrams (RBDs). Our experiences suggest that the use of a common semantic domain and a shared analyzer for expressions at this level can ease the task of formalizing and implementing modeling languages, reducing development costs and improving their dependability.
David Coppit, Robert R. Painter, Kevin J. Sullivan
ISSRE3
2003 Eos: instance-level aspects for integrated system design
abstract
This paper makes two contributions: a generalization of AspectJ-like languages with first-class aspect instances and instance-level advising, and a mapping of the mediator style for integrated system design into this space. We present Eos as a prototype language design and implementation. It extends C# with AspectJ-like constructs, first-class aspect instances and instance-level advising. These features enable a direct mapping of mediators to aspect instances, with modularity improved, insofar as components need not declare, announce, or register for events.
Hridesh Rajan, Kevin J. Sullivan
ESEC / SIGSOFT FSE2
2002 Web services engineering: promises and challenges
abstract
Web services are emerging technologies to reuse software as services over the Internet by wrapping underlying computing models with XML. Web services are rapidly evolving and are expected to change the paradigms of both software development and use. This panel will discuss the current status and challenges of Web services technologies.
Mikio Aoyama, Sanjiva Weerawarana, Hiroshi Maruyama, Clemens A. Szyperski, Kevin J. Sullivan, Doug Lea
ICSE5
2002 Software engineering economics: background, current practices, and future directions
abstract
The field of software economics seeks to develop technical theories, guidelines, and practices of software development based on sound, established, and emerging models of value and value-creation---adapted to the domain of software development as necessary. The premise of the field is that software development is an ongoing investment activity---in which developers and managers continually make investment decisions requiring the expenditure of valuable resources, such as time, talent, and money. The overriding aim of this activity is to maximize the value added subject to an equitable distribution among the participating stakeholders. The goal of the tutorial is to expose the audience to this line of thinking and introduce the tools pertinent to its pursuit. The tutorial is designed to be self-contained and will cover concepts from introductory to advanced. Both practitioners and researchers with an interest in the impact of value considerations in software decision-making will benefit from attending it.This tutorial is offered in conjunction with the Fourth International Workshop on Economics-Driven Software Engineering Research (EDSER-4). The tutorial is meant in part to enable those who would like to participate in the workshop, but who might not possess the requisite background, to come up to speed.
Hakan Erdogmus, Barry W. Boehm, Warren Harrison, Donald J. Reifer, Kevin J. Sullivan
ICSE5
2001 A Web-Oriented Architectural Aspect for the Emerging Computational Tapestry
abstract
An emerging tapestry of computations will soon integrate systems around the globe. It will evolve without central control. Its complexity will be vast. We need new ideas, tools and methods to help map, understand and manage this tapestry. We contribute a light-weight architectural aspect that designers can use without compromising their own architectural preferences. Widespread use could help. The idea is for objects to provide Web-based interfaces to object-specific meta-data, state, and monitoring and control services. We discuss applications, implementation, scalability, performance, tradeoffs, and related work.
Kevin J. Sullivan, Avneesh Saxena
ICSE1
2001 Third International Workshop on Economics-Driven Software Engineering Research
Kevin J. Sullivan, Mary Shaw, Barry W. Boehm, David Notkin, Warren Harrison
ICSE1
2001 The structure and value of modularity in software design
abstract
The concept of information hiding modularity is a cornerstone of modern software design thought, but its formulation remains casual and its emphasis on changeability is imperfectly related to the goal of creating added value in a given context. We need better explanatory and prescriptive models of the nature and value of information hiding. We evaluate the potential of a new theory---developed to account for the influence of modularity on the evolution of the computer industry---to inform software design. The theory uses design structure matrices to model designs and real options techniques to value them. To test the potential utility of the theory for software we apply it to Parnas's KWIC designs. We contribute an extension to design structure matrices, and we show that the options results are consistent with Parnas's conclusions. Our results suggest that such a theory does have potential to help inform software design.
Kevin J. Sullivan, William G. Griswold, Yuanfang Cai, Ben Hallen
ESEC / SIGSOFT FSE1
2001 Software Reflexion Models: Bridging the Gap between Design and Implementation
abstract
The artifacts constituting a software system often drift apart over time. We have developed the software reflexion model technique to help engineers perform various software engineering tasks by exploiting, rather than removing, the drift between design and implementation. More specifically, the technique helps an engineer compare artifacts by summarizing where one artifact (such as a design) is consistent with and inconsistent with another artifact (such as source). The technique can be applied to help a software engineer evolve a structural mental model of a system to the point that it is "good enough" to be used for reasoning about a task at hand. The software reflexion model technique has been applied to support a variety of tasks, including design conformance, change assessment, and an experimental reengineering of the million-lines-of-code Microsoft Excel product. We provide a formal characterization of the reflexion model technique, discuss practical aspects of the approach, relate experiences of applying the approach and tools, and place the technique into the context of related work.
Gail C. Murphy, David Notkin, Kevin J. Sullivan
IEEE Trans. Software Eng.3
2000 Multiple mass-market applications as components
abstract
Truly successful models for component-based software development continue to prove elusive. One of the few is the use of operating system, database and similar programs in many systems. We address three related problems in this paper. First, we lack needed models. Second, we do not know the conditions under which such models can succeed. In particular, it is unclear whether the notable success with operating systems can be replicated. Third, we do not know whether certain specific models can succeed. We are addressing these problems by evaluating a particular model that shares important characteristics with the successful operating system example: using compatible PC packages as components. Our approach to evaluating such a model is to engage in a case study that aims to build an industrially successful system representative of an important class of systems. We report on our use of the model to develop a computational tool for reliability engineering. We draw two conclusions. First, this kind of model has the potential to succeed. Second, even today, the model can produce significant returns, but it clearly carries considerable risks.
David Coppit, Kevin J. Sullivan
ICSE2
2000 Galileo: a tool built from mass-market applications
abstract
We present Galileo, an innovative engineering modeling and analysis tool built using an approach we call package-oriented programming (POP). Galileo represents an ongoing evaluation of the POP approach, where multiple large, architecturally coherent components are tightly integrated in an overall software system. Galileo utilizes Microsoft Word, Internet Explorer, and Visio to provide a low cost, richly functional fault tree modeling superstructure. Based on the success of previous prototypes of the tool, we are now building a version for industrial use under an agreement with NASA Langley Research Center.
David Coppit, Kevin J. Sullivan
ICSE2
2000 The 2nd International Workshop on Economics-Driven Software Engineering Research
abstract
The need for research in this area is indicated by the serious shortfalls in our understanding of how best to design software for value creation. There are at least two basic dimensions to this shortfall. First, the core competency of software engineers is making technical software product and process design decisions. However, today there is a disconnect between the technical criteria taught to software engineers and the strategic value creation objectives of the organizations for which software is designed.
Kevin J. Sullivan
ICSE1
2000 Formal Semantics for Computational Engineering: A Case Study on Dynamic Fault Trees
abstract
Computational modeling tools are critical to engineering. In the absence of a sufficiently complete, mathematically precise, abstract specification of the semantics of the modeling framework supported by such a tool, rigorous validation of the framework and of models built using it is impossible; there is no sound basis for program implementation, verification or documentation; the scientific foundation of the framework remains weak; and significant conceptual errors in framework definition and implementation are likely. Yet such specifications are rarely defined. We present an approach based on the use of formal specification and denotational semantics techniques from software engineering and programming language design. To illustrate the approach, we present elements of a formal semantics for a dynamic fault tree framework that promises to aid reliability analysis. No such specification of the meaning of dynamic fault trees has been defined previously. The approach revealed important shortcomings in the previous, informal definitions of the framework, and thus led to significant improvements, suggesting that formally specifying framework semantics is critical to effective framework design.
David Coppit, Kevin J. Sullivan, Joanne Bechta Dugan
ISSRE2
2000 COM revisited: tool-assisted modelling of an architectural framework
abstract
Designing architectural frameworks without the aid of formal modeling is error prone. But, unless supported by analysis, formal modeling is prone to its own class of errors, in which formal statements fail to match the designer's intent. A fully automatic analysis tool can rapidly expose such errors, and can make the process of constructing and refining a formal model more effective.
Daniel Jackson 0001, Kevin J. Sullivan
SIGSOFT FSE2
2000 Developing a low-cost high-quality software tool for dynamic fault-tree analysis
abstract
Sophisticated modeling and analysis methods are being developed in academic and industrial research labs for reliability engineering and other domains. The evaluation and evolution of such methods based on use in practice is critical to research progress, but few such methods see widespread use. A critical impediment to disseminating new methods is the inability to produce, at a reasonable cost, supporting software tools that have the: usability and dependability characteristics that industrial users require; and evolvability to accommodate software change as the underlying analysis methods are refined and enhanced. The difficulty of software development thus emerges as a key impediment to advances in engineering modeling and analysis. This paper presents an approach to tool development that attacks these problems. Progress requires synergistic, interdisciplinary collaborations between application-domain and software-engineering researchers. The authors have pursued such an approach in developing Galileo: a fault tree modeling and analysis tool. These innovations are described in two dimensions: (1) the Galileo core reliability modeling and analysis function; and (2) the authors' work on software engineering for high-quality, low-cost modeling and analysis tools.
Joanne Bechta Dugan, Kevin J. Sullivan, David Coppit
IEEE Trans. Reliab.2
1999 Information Survivability Control Systems
abstract
We address the dependence of critical infrastructures-including electric power, telecommunications, finance and transportation-on vulnerable information systems. Our approach is based on the notion of control systems. We envision hierarchical, adaptive, multiple model, discrete-state distributed control systems to monitor infrastructure information systems and respond to disruptions (e.g., security attacks) by changing operating modes and design configurations to minimize loss of utility. To explore and evaluate our approach, we have developed a toolkit for building distributed dynamic models of infrastructure information systems. We used this toolkit to build a model of a simple subset of the United States payment system and a control system for this model information system.
Kevin J. Sullivan, John C. Knight, Steve Geist
ICSE1
1999 First Workshop on Economics-Driven Software Engineering Research
abstract
No abstract available.
Kevin J. Sullivan, David Notkin, Alfonso Fuggetta, John M. Favaro
ICSE1
1999 Developing a high-quality software tool for fault tree analysis
abstract
Sophisticated dependability analysis techniques are being developed in academia and research labs, but few have gained wide acceptance in industry. To be valuable, such techniques must be supported by usable, dependable software tools. We present our approach to addressing these issues in developing a dynamic fault tree analysis tool called Galileo. Galileo is designed to support efficient system-level analysis by automatically decomposing fault trees into modules that are solved separately using appropriate techniques. Usability is addressed by a software architecture based on a component-based design technique that we call package-oriented programming. We integrate multiple, volume-priced mass-market software packages to provide the bulk of the tool superstructure. To address tool dependability, we are developing natural language and partial formal specifications of fault tree elements, and we exploit the inherent redundancy associated with multiple analysis techniques as an aid in testing.
Joanne Bechta Dugan, Kevin J. Sullivan, David Coppit
ISSRE2
1999 Exploration Harnesses: Tool-Supported Interactive Discovery of Commercial Component Properties
abstract
A key problem in component-based software development (CBSD) is that developers have incomplete knowledge of components. In many cases, the only available source of such information is experimentation. In this paper we argue that the provision of tool support for automated and repeatable experiments can provide significant value to designers. Such tools, which we call exploration harnesses, promise to help enterprises to exploit prefabricated evolving third party components. We evaluated the exploration harness concept by building a prototype and using it to support the exploration of large components in the design of a dynamic fault-tree analysis tool called Galileo. Galileo employs package-oriented programming, in which shrink-wrapped packages such as Microsoft Word and Visio Technical are used as large components. Using our exploration harness helped us to discover a range of relevant but undocumented properties of such components, across several versions, which enabled us to make better informed design decisions.
Michael A. Copenhafer, Kevin J. Sullivan
ASE2
1999 Software economics: status and prospects
Barry W. Boehm, Kevin J. Sullivan
Inf. Softw. Technol.2
1999 Analysis of a Conflict between Aggregation and Interface Negotiation in Microsoft's Component Object Model
abstract
Many software projects today are based on the integration of independently designed software components that are acquired on the market, rather than developed within the projects themselves. A component standard, or integration architecture, is a set of design rules meant to ensure that such components can be integrated in defined ways without undue effort. The rules of a component standard define, among other things, component interoperability and composition mechanisms. Understanding the properties of such mechanisms and interactions between them is important for the successful development and integration of software components, as well as for the evolution of component standards. The paper presents a rigorous analysis of two such mechanisms: component aggregation and dynamic interface negotiation, which were first introduced in Microsoft's Component Object Model (COM). We show that interface negotiation does not function properly within COM aggregation boundaries. In particular, interface negotiation generally cannot be used to determine the identity and set of interfaces of aggregated components. This complicates integration within aggregates. We provide a mediator-based example, and show that the problem is in the sharing of interfaces inherent in COM aggregation.
Kevin J. Sullivan, Mark Marchukov, John Socha
IEEE Trans. Software Eng.1
1997 Package-Oriented Programming of Engineering Tools
abstract
No abstract available.
Kevin J. Sullivan, Jake Cockrell, Shengtong Zhang, David Coppit
ICSE1
1997 Using Formal Methods to Reason about Architectural Standards
abstract
We present a study in which we used formal methods to reason precisely about aspects of a widely used software architectural standard, namely Microsoft's Component Object Model (COM).We developed a formal theory of COM to help us reason about a proposed compositional architectural style based on COM, intended for use in a novel commercial multimedia authoring system.The style combined COM objects, integration mediators, and the COM reuse mechanism of aggregation.Our use of formal methods averted an architectural disaster by revealing essential but subtle and counterintuitive properties of COM.We partially validated our theory by subjecting it to review by the designers of COM and by testing it against other available data.The theory has good evidential support.
Kevin J. Sullivan, John Socha, Mark Marchukov
ICSE1
1996 Experience Assessing an Architectural Approach to Large-Scale Systematic Reuse
Kevin J. Sullivan, John C. Knight
ICSE1
1996 Evaluating The Mediator Method: Prism as a Case Study
abstract
A software engineer's confidence in the profitability of a novel design technique depends to a significant degree on previous demonstrations of its profitability in practice. Trials of proposed techniques are thus of considerable value in providing factual bases for evaluation. We present our experience with a previously presented design approach as a basis for evaluating its promise and problems. Specifically, we report on our use of the mediator method to reconcile tight behavioral integration with ease of development and evolution of Prism, a system for planning radiation treatments for cancer patients. Prism is now in routine clinical use in several major research hospitals. Our work supports two claims. In comparison to more common design techniques, the mediator approach eases the development and evolution of integrated systems; and the method can be learned and used profitably by practising software engineers.
Kevin J. Sullivan, Ira J. Kalet, David Notkin
IEEE Trans. Software Eng.1
1995 Software Reflexion Models: Bridging the Gap Between Source and High-Level Models
abstract
article Software reflexion models: bridging the gap between source and high-level models Share on Authors: Gail C. Murphy Dept. of Computer Science & Engineering, University of Washington, Box 352350, Seattle WA Dept. of Computer Science & Engineering, University of Washington, Box 352350, Seattle WAView Profile , David Notkin Dept. of Computer Science & Engineering, University of Washington, Box 352350, Seattle WA Dept. of Computer Science & Engineering, University of Washington, Box 352350, Seattle WAView Profile , Kevin Sullivan Dept. of Computer Science, University of Virginia, Charlottesville VA Dept. of Computer Science, University of Virginia, Charlottesville VAView Profile Authors Info & Claims ACM SIGSOFT Software Engineering NotesVolume 20Issue 4Oct. 1995 pp 18–28https://doi.org/10.1145/222132.222136Published:01 October 1995 305citation2,070DownloadsMetricsTotal Citations305Total Downloads2,070Last 12 Months74Last 6 weeks13 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteGet Access
Gail C. Murphy, David Notkin, Kevin J. Sullivan
SIGSOFT FSE3
1992 Reconciling Environment Integration and Software Evolution
abstract
Common software design approaches complicate both tool integration and software evolution when applied in the development of integrated environments. We illustrate this by tracing the evolution of three different designs for a simple integrated environment as representative changes are made to the requirements. We present an approach that eases integration and evolution by preserving tool independence in the face of integration. We design tool integration relationships as separate components called mediators , and we design tools to implicitly invoke mediators that integrate them. Mediators separate tools from each other, while implicit invocation allows tools to remain independent of mediators. To enable the use of our approach on a range of platforms, we provide a formalized model and requirements for implicit invocation mechanisms. We apply this model both to analyze existing mechanisms and in the design of a mechanism for C++.
Kevin J. Sullivan, David Notkin
ACM Trans. Softw. Eng. Methodol.1
1990 A Transportable Programming Language (TPL) System-II: The Bifunctional Compiler System
abstract
For pt.I see P.A.D. de Maine, S. Leong, and C.G. Dairs, Int. J. Comput. Inform. Sci., vol.14, p.161-82, 1985. The transportable programming language (TPL) method is a high-level-language approach that uses a bifunctional compiler to efficiently convert code among various dialects of a particular high-level language (HLL) via the hypothetical parent of the high-level language (HPHLL). The TPL compiler system that has been implemented has three parts: a rule modifier, a table generator, and a TPL compiler. A metalanguage, called the conversion rule description language (CRDL), is used to describe the conversion of a dialect to HPHLL and of the HPHLL to another dialect. The table generator translates those descriptions to tabular forms that drive the bifunctional compiler. The TPL compiler can then be used to translate programs coded in a local dialect into HPHLL and vice versa. The rule modifier alters the descriptions of a default-a synthetic 'most common'-dialect. It greatly simplifies the task of writing the conversion descriptions for a new environment or dialect. The TPL method is now being extended so that it can be used to retarget a dialect of any HLL to a standard environment such as Ada. Details of the TPL compiler system are given.>
Soklei Leong, Stephen M. Jodis, Kevin J. Sullivan, Oliver Jiang, Paul A. D. de Maine
IEEE Trans. Software Eng.3