Zheng Li 0001

dblp:10/1143-1 · also Zheng Eddie Li 0001 · DBLP profile ↗
← Back
36ranked-venue papers
26as first author
12since 2021 · last 2025
0000-0002-9704-7651ORCID · verified

Domains — the database's venue-derived domains; a paper can count in several

Software engineering, systems software and programming languages · 16 · 10 first-author · 8 since 2021Systems, architecture and hardware · 5 · 2 first-author · 2 since 2021Applied, interdisciplinary, general and emerging computing · 3 · 3 first-authorHuman-computer interaction and ubiquitous computing · 2 · 2 first-author · 1 since 2021Artificial intelligence and machine learning · 1
YearPublicationVenuePosition
2025 Service-Oriented Evolution of Modern AI: A Position Paper
abstract
[Context]: It is well known that understanding the evolution of technologies and its cause is essential for more discoveries and innovations. In the Artificial Intelligence (AI) domain, it has also been identified that scrutinising the development context and path of AI will be able to help both academia and industry better understand the current AI limitations, reveal future AI trends, and facilitate AI/digital transformations. [Objectives]: Given the dramatic boom of modern AI, this research aims to unearth the evolution pattern along the recent three AI waves (namely predictive AI, generative AI and agentic AI), and accordingly to guide AI research and development to focus on the most promising directions. [Method]: We employed analogical reasoning as the research method and referred to the existing software architectural styles to inspire our understanding of the architectural evolution of modern AI technologies. [Results]: We see a service-oriented trend in modern AI's working mechanisms, and the offering of AI power seems to be transiting from a heavyweight and monolithic paradigm to an organisational and collaborative paradigm with more and more specific separation of concerns. Following this service-oriented evolution trend, we borrow software architecture lessons and foresee opportunities to grow the current AI wave to a further height, e.g., standardising AI agent-friendly APIs and developing serverless AI agents. [Conclusions]: What is happening in the AI domain has happened before in the software engineering domain. It is worth reusing software architecture knowledge to evolve the architecture of AI technologies.
Zheng Li 0001, Christopher McKie, Hui Wang 0001, Hamza Shakeel, Rajiv Ranjan 0001
SSE1
2025 Explainable Software Engineering: From State of the Practice to State of the Art
abstract
The evidence-based paradigm has made software engineering (SE) more objective and less imprecise, facilitating extensive decision-making activities in software projects. However, by referring to the four-theory classificatory scheme, it can be observed that evidence-based SE mainly focuses on empirically developing descriptive theories from SE practices, and directly using descriptions to make predictions and suggest prescriptions, without emphasising the establishment and utilisation of explanatory theories for comprehensively understanding SE phenomena. In keeping with the pioneering viewpoint that regards explainability as a first-class citizen of SE in the future, we urge the development of explainable SE as a dedicated research field that pursues the state of the art beyond the state of the practice of SE. To support this forward-looking vision, we have developed an extensible framework, two factorial strategies, and a set of augmented empirical methods as the preliminary responses to two root research questions what to explain and how to explain and one derived research question how to identify proper explanations. As such, although at a vision stage, our current work has paved the way forward towards a more explainable future of SE that will bring various benefits to different stakeholders in the software ecosystem.
Zheng Li 0001, Maria Angela Ferrario, Ben Crossey, Austen Rainer
APSEC1
2025 A Comparative Study Towards Designing a Hybrid Architecture of Microservices and LLM-based Multi-Agent Systems
abstract
LLM-based Multi-Agent Systems (LLM-MAS) present an emerging paradigm for constructing intelligent and adaptive applications that enable autonomous reasoning and collaborative problem-solving. Empirical studies show that current LLM-MAS still suffers from overlapping agent roles, unclear capabilities, and goal misalignment. In contrast, Microservice Systems (MS) are designed with modularity, and each service encapsulates a well-defined context and interface with structured and deterministic execution. From the system architecture principles, these paradigms demonstrate parallel attributes and complementary strengths that lead to a synthesized hybrid architecture. In this paper, we analyze the design factors of hybrid MS and LLM-MAS, as well as the main challenges, through a comparative study across eight architectural dimensions, including function encapsulation, orchestration, API design, auto-correction, data communication, operations, quality attributes, and environment awareness. The analysis reveals critical mismatches, design synergies, and transferable best practices. To motivate future work, we define four research questions to categorize the challenges. The goal is to create a converging design space for exploring architecture design towards intelligent, autonomous, modular, and adaptive systems.
Peyman Yazdanian, Yan Liu 0001, Zheng Li 0001
APSEC3
2025 Fostering Qualified Software Engineers via Software Engineering Conversion Programmes
abstract
Considering that every company is becoming a software company, there will be an ongoing demand for qualified software engineers. However, there seems to be a supply shortage of graduates in Computer Science or Software Engineering from the traditional university education. It has been observed for decades that a considerable number of developers in the software industry do not have computer science degrees and basic software engineering educations. On the other hand, such a phenomenon may have revealed the reality and trend that the software industry will continue the intake of career-shift employees and graduates from non-computer science disciplines. Therefore, we argue software engineering conversion programmes to be a strategic and long-term approach to fostering qualified software engineers. The conversion programmes will substantially supplement the classic university programmes and address the disadvantage of unconventional entry points (e.g., bootcamp training and selflearning). We have conducted critical reflection on our first-hand educational practice together with extensive literature review to justify our argument.
Zheng Li 0001
CSEE&T1
2025 An Empirical Study on Precompilation as an Ops-oriented Micro-optimisation Strategy for Microservice-based Applications
James Dougan, Zheng Li 0001
ICA3PP (6)2
2024 Long Live the Image: On Enabling Resilient Production Database Containers for Microservice Applications
abstract
Microservices architecture advocates decentralized data ownership for building software systems. Particularly, in the Database per Service pattern, each microservice is supposed to maintain its own database and to handle the data related to its functionality. When implementing microservices in practice, however, there seems to be a paradox: The de facto technology (i.e., containerization) for microservice implementation is claimed to be unsuitable for the microservice component (i.e., database) in production environments, mainly due to the data persistence issues (e.g., dangling volumes) and security concerns. As a result, the existing discussions generally suggest replacing database containers with cloud database services, while leaving the on-premises microservice implementation out of consideration. After identifying three statelessness-dominant application scenarios, we proposed container-native data persistence as a conditional solution to enable resilient database containers in production. In essence, this data persistence solution distinguishes stateless data access (i.e., reading) from stateful data processing (i.e., creating, updating, and deleting), and thus it aims at the development of stateless microservices for suitable applications. In addition to developing our proposal, this research is particularly focused on its validation, via prototyping the solution and evaluating its performance, and via applying this solution to two real-world microservice applications. From the industrial perspective, the validation results have proved the feasibility, usability, and efficiency of fully containerized microservices for production in applicable situations. From the academic perspective, this research has shed light on the operation-side micro-optimization of individual microservices, which fundamentally expands the scope of “software micro-optimization” and reveals new research opportunities.
Zheng Li 0001, Nicolás Saldías-Vallejos, Diego Seco Naveiras, M. Andrea Rodríguez, Rajiv Ranjan 0001
IEEE Trans. Software Eng.1
2023 On Code Example-Aided Just-In-Time Learning for Programming Education
abstract
Context: Programming education keeps facing chal-lenges. A significant challenge is the mismatch between the increasing student demand and the shortage of teaching workforce on personal tutoring. Objective: The aim of this research is to address the demand-workforce mismatch and relieve the challenges of programming education. Method: After theoretically discussing a set of methodological principles (e.g., active learning from suitable code examples), a prototyping research method is employed to iteratively verify whether or not the proposed principles can be practically implemented. Results: A methodology, namely code example-aided just-in-time learning, gradually emerged. A code example hunting tool was developed for enabling this methodology. Conclusion: The code example-aided just-in-time learning can effectively and efficiently facilitate both experienced learners and beginners to study programming by practising, without requiring extensive guidance from teachers.
Zheng Li 0001, Sridhar Sai Gorrepati, Des Greer
APSEC1
2023 Exploiting Paired Concepts to Facilitate Software Engineering Education
abstract
[Context]: In the university curriculum, software engineering (SE) is frequently perceived as a difficult course to study due to its concept-intensive nature. [Objectives]: We aim to investigate if and how we can help students conveniently understand and memorise the numerous and various SE concepts. [Method]: We employ critical reflection as the research method to qualitatively examine our educational activities, teaching materials, and students' learning effects. [Results]: By focusing on the paired structural model of knowledge representation, we propose to utilise tangentially-paired concepts together with the conventional bipolar-paired concepts to facilitate SE education. In particular, we have identified three types of tangentially-paired concepts with deriving (stemming), analogical, and cloning relationships respectively. [Conclusions]: Exploiting paired SE concepts can act as an efficient educational approach that supplements the existing strategies for teaching SE knowledge.
Zheng Li 0001, Austen Rainer
APSEC1
2023 The Most Agile Teams Are the Most Disciplined: On Scaling out Agile Development
abstract
As one of the next frontiers of software engineering, agile development at scale has attracted more and more research interests and efforts. When following the existing autonomy-focused and goal-driven lessons and guidelines to scale agile development for a large astronomy project, however, we encountered surprising tech stack sprawl and spreading team coordination issues. By revisiting the unique features of our project (e.g., the data processing-intensive nature and the frequent team member changes), and by identifying a fractal pattern from various data processing logic and processes, we defined disciplined agile teams to clone the best practices of pioneer agile teams, and to work on similar system modules with similar user stories. Such a targeted strategy effectively relieved the tech stack sprawl and facilitated teamwork handover, at least for refactoring and growing the data processing modules in our project. Based on this emerging result and our reflections, we distinguish this targeted strategy as scaling out agile development from the existing agile scaling approaches that are generally in a scaling-up fashion. Considering the popularity of data processing-intensive projects, and also considering the pervasive fractal patterns in modern businesses and organisations, we claim that this targeted strategy still has broad application opportunities. Therefore, developing a well-defined methodology for scaling out agility, and combining both scaling up and scaling out agility, will deserve attentions and new research efforts in the future.
Zheng Li 0001, Austen Rainer
ESEC/SIGSOFT FSE1
2022 On Kubernetes-aided Federated Database Systems
abstract
Cloud computing has made federated database systems (FDBS) significantly more practical to implement than in the past. As part of a recent Web-based Geographic Information System (WebGIS) project, we are employing cloud-native technologies (from the container ecosystem) to develop a federated database (DB) infrastructure, to help manage and utilise the distributed and various geospatial data. Unfortunately, there seem to be inherent challenges and complexity of applying the container and Kubernetes technologies to building and running DB systems. Considering that most of the geospatial and theme data are pre-obtained and fixed in our WebGIS project, we decided to focus on the read-only user queries and still resort to Kubernetes to implement an FDBS instance to use. Unlike the de facto practices (e.g., using the StatefulSets mechanism, extending Kuberentes APIs, or employing KubeFed), our solution for Kubernetes-aided FDBS simplifies the tech stack by investigating the fractal object of federated data management, inclusively containerising DB instances, and using the lightweight Deployment mechanism to handle stateless DB containers. Overall, this research not only reveals an easy-to-implement approach to constructing read-only components in a fully-fledged FDBS, but also proposes and demonstrates a novel methodology for FDBS investigations.
Zheng Li 0001, Nicolás Saldías-Vallejos, M. Andrea Rodríguez, Austen Rainer
CloudCom1
2022 Managing the Root Causes of "Internal API Hell": An Experience Report
Guillermo Cabrera-Vives, Zheng Li 0001, Austen Rainer, Dionysis Athanasopoulos, Diego Rodríguez-Mancini, Francisco Förster
PROFES2
2021 A study on the evaluation of HPC microservices in containerized environment
abstract
Summary Containers are gaining popularity over virtual machines as they provide the advantages of virtualization with the performance of near bare metal. The uniformity of support provided by Docker containers across different cloud providers makes them a popular choice for developers. Evolution of microservice architecture allows complex applications to be structured into independent modular components making them easier to manage. High‐performance computing (HPC) applications are one such application to be deployed as microservices, placing significant resource requirements on the container framework. However, there is a possibility of interference between different microservices hosted within the same container (intracontainer) and different containers (intercontainer) on the same physical host. In this paper, we describe an extensive experimental investigation to determine the performance evaluation of Docker containers executing heterogeneous HPC microservices. We are particularly concerned with how intracontainer and intercontainer interference influences the performance. Moreover, we investigate the performance variations in Docker containers when control groups (cgroups) are used for resource limitation. For ease of presentation and reproducibility, we use Cloud Evaluation Experiment Methodology (CEEM) to conduct our comprehensive set of experiments. We expect that the results of evaluation can be used in understanding the behavior of HPC microservices in the interfering containerized environment.
Devki Nandan Jha, Saurabh Kumar Garg 0001, Prem Prakash Jayaraman, Rajkumar Buyya, Zheng Li 0001, Graham Morgan, Rajiv Ranjan 0001
Concurr. Comput. Pract. Exp.5
2020 Characterising Edge-Cloud Data Transmission for Patient-Centric Healthcare Systems
abstract
Benefiting from the modern information and communication technologies, the healthcare provisioning is actively evolving along a trend toward patient centricity, and the de facto solution seems to be technological collaboration and cooperation within a three-tier architecture. Given the largely distributed tiers and the components, a hot research focus is on minimizing the data transmission latency to improve the quality of healthcare services, especially in the time-critical situations. It has been identified that a typical performance bottleneck is the communication between the last two tiers that are generally represented by the edge and the cloud nowadays. Thus, optimising the edge-cloud data traffic becomes valuable and crucial to addressing the performance bottleneck, while characterising the edge-cloud data transmission plays a prerequisite role in the optimisation efforts. Unlike the existing studies that mainly emphasise the intuitive features (e.g., the overall big data volume), our work argues and reveals the importance of identifying the practical characteristics of edge-cloud data transmission at runtime, w.r.t. different workload regimes and heterogeneous edge nodes.
Zheng Li 0001, Francisco Millar-Bilbao
HealthCom1
2019 In Method We Trust: Towards an Open Method Kit for Characterizing Spot Cloud Service Pricing
abstract
Based on market-driven mechanisms that can improve utilization of idle compute resources at dynamic prices, spot cloud services are becoming increasingly popular to reach a win-win situation of providers' revenue maximization and consumers' budget optimization. Nevertheless, unlike fixed pricing schemes, the spot pricing scheme is both psychologically and practically sophisticated for people to understand and employ. As such, characterizing spot cloud pricing has been identified to be crucial and beneficial for various purposes ranging from facilitating service procurement to addressing service interruptions. In addition, it is also noteworthy that the spot cloud market is inherently volatile, and the providers' (e.g., Amazon's) pricing policies can change from time to time. Consequently, the previous analysis results can quickly be out of date, and the previous analysis methods can barely be reusable if their details are not specified. Therefore, we decided to develop a domain-specific method kit and make it open to improve the repeatability, replicability, and reproducibility of spot pricing characterization studies. This paper reports the typical content of this method kit, including a group of central tendency analysis methods and two types of distribution modeling analysis methods. In a generic sense, we particularly argue that open methods act as a higher-level strategy over open-source tools and open-access data for scientific studies in any research domain.
Zheng Li 0001
CLOUD1
2019 A dataflow-driven approach to identifying microservices from monolithic applications
Shanshan Li 0002, He Zhang 0001, Zijia Jia, Zheng Li 0001, Cheng Zhang 0010, Qiuya Gao, Jidong Ge, Zhihao Shan
J. Syst. Softw.4
2018 A multi-layered performance analysis for cloud-based topic detection and tracking in Big Data applications
Meisong Wang, Prem Prakash Jayaraman, Ellis Solaiman, Lydia Y. Chen, Zheng Li 0001, Jun Song 0003, Dimitrios Georgakopoulos 0001, Rajiv Ranjan 0001
Future Gener. Comput. Syst.5
2017 Performance Overhead Comparison between Hypervisor and Container Based Virtualization
abstract
The current virtualization solution in the Cloud widely relies on hypervisor-based technologies. Along with the recent popularity of Docker, the container-based virtualization starts receiving more attention for being a promising alternative. Since both of the virtualization solutions are not resource-free, their performance overheads would lead to negative impacts on the quality of Cloud services. To help fundamentally understand the performance difference between these two types of virtualization solutions, we use a physical machine with “just-enough” resource as a baseline to investigate the performance overhead of a standalone Docker container against a standalone virtual machine (VM). With findings contrary to the related work, our evaluation results show that the virtualization's performance overhead could vary not only on a feature-by-feature basis but also on a job-to-job basis. Although the container-based solution is undoubtedly lightweight, the hypervisor-based technology does not come with higher performance overhead in every case. For example, Docker containers particularly exhibit lower QoS in terms of storage transaction speed.
Zheng Li 0001, Maria Kihl, Qinghua Lu 0001, Jens A. Andersson
AINA1
2017 From Monolith to Microservices: A Dataflow-Driven Approach
abstract
Emerging from the agile practitioner communities, the microservice-oriented architecture emphasizes implementing and employing multiple small-scale and independently deployable microservices, rather than encapsulating all function capabilities into one monolithic application. Correspondingly, microservice-oriented decomposition, which has been identified to be an extremely challenging and complex task, plays a crucial and prerequisite role in developing microservice-based software systems. To address this challenge and reduce the complexity, we proposed a top-down analysis approach and developed a dataflow-driven decomposition algorithm. In brief, a three-step process is defined: first, engineers together with users conduct business requirement analysis and construct a purified while detailed dataflow diagram of the business logic; then, our algorithm combines the same operations with the same type of output data into a virtual abstract dataflow; finally, the algorithm extracts individual modules of "operation and its output data" from the virtual abstract dataflow to represent the identified microservice candidates. We have employed two use cases to demonstrate our microservice identification mechanism, as well as making comparisons with an existing microservice identification tool. The comparison and evaluation show that, our dataflow-driven identification mechanism is able to deliver more rational, objective, understandable and consistent microservice candidates, through a more rigorous and practical implementation procedure.
Shanshan Li 0002, Zheng Li 0001
APSEC3
2017 A Mapping Study on Mining Software Process
abstract
Background: Mining Software Process (MSP) helps distill important information about software process enactment from software data repositories. An increasing amount of research effort is being dedicated to MSP. These studies differ in various aspects (e.g., topics, data, and techniques) of MSP. Objective: We aim to study the state of the art on MSP from following aspects, i.e., research topics, data sources, data types, mining techniques, and mining tools. Method: We conducted a systematic mapping study on the research relevant to MSP at both microprocess and macroprocess levels. Results: Our mapping study identified 40 relevant studies that can be grouped into microprocess and macroprocess levels. The identified mining techniques have been mapped onto the associated mining tools that fall into four types. Driven by the three research questions which represented in a meta-model, the findings revealed the correlations among the research topics, data sources, data types, mining techniques, and mining tools. Conclusion: It is observed that in order to discover the software process model or map, the main data source is from industrial project. Current mining techniques for microprocess research are mostly business process mining or sequence mining techniques used to recover descriptive software process. In addition, various machine learning algorithms and novel proposed methods are used to improve the accuracy of macroprocess level factors (e.g., software effort estimation).
Liming Dong 0001, Bohan Liu 0003, Zheng Li 0001, Muhammad Ali Babar 0001, Bingbing Xue 0002
APSEC3
2017 Mining Handover Process in Open Source Development: An Exploratory Study
abstract
Background: Handover is a common process in all software development projects. It is one of the most complex and diverse processes in software life cycle which could have a negative impact on software quality and progress. In open source software (OSS) development, handover is a more critical task due to poor planning. Objective: The goal of this work is to investigate whether we can automatically identify the handover process in OSS development. Furthermore, we aim to mine the process of handover and identify the factors and their influences on the duration of handover process. Method: We propose an ADC metric and an HDI algorithm to automatically identify the handover process and conduct a brief survey to evaluate it. We apply the Heuristic mining algorithm to discover the process maps of handover by mining Github repositories. To identify the factors from a large set of variables, we employ the Stepwise regression method. Results: We identified 63 pairs of handover within 44 projects from 314 most popular projects using our proposed method. Our survey received 21 responses. Conclusion: This study confirms that handover can be identified automatically. Although handover processes vary, developers follow a common work-flow during handover. The number of lines of code is positively correlated to the duration of handover process.
Liming Dong 0001, Bohan Liu 0003, Zheng Li 0001, Bingbing Xue 0002, Danni Chen, Tiange Chen
APSEC3
2017 Autonomic deployment decision making for big data analytics applications in the cloud
Qinghua Lu 0001, Zheng Li 0001, Weishan Zhang, Laurence T. Yang
Soft Comput.2
2017 A Survey on Modeling Energy Consumption of Cloud Applications: Deconstruction, State of the Art, and Trade-Off Debates
abstract
Given the complexity and heterogeneity in Cloud computing scenarios, the modeling approach has widely been employed to investigate and analyze the energy consumption of Cloud applications, by abstracting real-world objects and processes that are difficult to observe or understand directly. It is clear that the abstraction sacrifices, and usually does not need, the complete reflection of the reality to be modeled. Consequently, current energy consumption models vary in terms of purposes, assumptions, application characteristics and environmental conditions, with possible overlaps between different research works. Therefore, it would be necessary and valuable to reveal the state-of-the-art of the existing modeling efforts, so as to weave different models together to facilitate comprehending and further investigating application energy consumption in the Cloud domain. By systematically selecting, assessing, and synthesizing 76 relevant studies, we rationalized and organized over 30 energy consumption models with unified notations. To help investigate the existing models and facilitate future modeling work, we deconstructed the runtime execution and deployment environment of Cloud applications, and identified 18 environmental factors and 12 workload factors that would be influential on the energy consumption. In particular, there are complicated trade-offs and even debates when dealing with the combinational impacts of multiple factors.
Zheng Li 0001, Selome Kostentinos Tesfatsion, Saeed Bastani, Ahmed Ali-Eldin, Erik Elmroth, Maria Kihl, Rajiv Ranjan 0001
IEEE Trans. Sustain. Comput.1
2016 Using a Predator-Prey Model to Explain Variations of Cloud Spot Price
abstract
The spot pricing scheme has been considered to be resource-efficient for providers and cost-effective for consumers in the Cloud market. Nevertheless, unlike the static and straightforward strategies of trading on-demand and reserved Cloud services, the market-driven mechanism for trading spot service would be complicated for both implementation and understanding. The largely invisible market activities and their complex interactions could especially make Cloud consumers hesitate to enter the spot market. To reduce the complexity in understanding the Cloud spot market, we decided to reveal the backend information behind spot price variations. Inspired by the methodology of reverse engineering, we developed a Predator-Prey model that can simulate the interactions between demand and resource based on the visible spot price traces. The simulation results have shown some basic regular patterns of market activities with respect to Amazon's spot instance type m3.large. Although the findings of this study need further validation by using practical data, our work essentially suggests a promising approach (i.e.~using a Predator-Prey model) to investigate spot market activities.
Zheng Li 0001, William Tärneberg, Maria Kihl, Anders Robertsson
CLOSER (2)1
2016 Spot pricing in the Cloud ecosystem: A comparative investigation
Zheng Li 0001, He Zhang 0001, Liam O'Brien, Maria Kihl, Rajiv Ranjan 0001
J. Syst. Softw.1
2015 On a Feedback Control-Based Mechanism of Bidding for Cloud Spot Service
abstract
As a cost-effective option for Cloud consumers, spot service has been considered to be a significant supplement for building a full-fledged market economy for the Cloud ecosystem. However, unlike the static and straightforward way of trading on-demand and reserved Cloud services, the market-driven regulations of employing spot service could be too complicated for Cloud consumers to comprehensively understand. In particular, it would be both difficult and tedious for potential consumers to determine suitable bids from time to time. To reduce the complexity in applying spot resources, we propose to use a feedback control to help make bidding decisions. Based on an arccotangent-function-type system model, our novel bidding mechanism imitates fuzzy and intuitive human activities to refine and issue new bids according to previous errors. The validation is conducted by using Amazon's historical spot price trace to perform a set of simulations and comparisons. The result shows that the feedback control-based mechanism obtains a better trade-off between bidding rationality and success rate than the other five comparable strategies. Although this mechanism is only for black-box bidding (price prediction) at this current stage, it can be conveniently and gradually upgraded to take into account external constraints in the future.
Zheng Li 0001, Maria Kihl, Anders Robertsson
CloudCom1
2014 Object-oriented Sokoban solver: A serious game project for OOAD and AI education
abstract
Serious games are beneficial for education in various computer science areas. Numerous works have reported the experiences of using games (not only playing but also development) in teaching and learning. Considering it could be difficult for teachers/students to prepare/develop a game from scratch during one semester, assistant educational materials would be crucial in the corresponding courses. Unfortunately, the literature shows that not many materials from educational game projects are shared. To help different educators identify suitable courseware and help students implement game development, it is worth further investigating and accumulating the educational resources from individual game projects. Following such an idea, this paper proposes a game development project of an object-oriented Sokoban solver, and exposes relevant educational materials. The documented system design can be viewed as a ready-to-use resource for education in object-oriented analysis and design (OOAD), while the Sokoban solver itself may be used as an assignment platform for teaching artificial intelligence (AI). Further documentation, platform, and APIs will be realized and shared in the future to facilitate others' educational activities. Overall, this work is supposed to inspire and encourage other researchers and educators to post available materials of more game projects for the purpose of sharing and reuse.
Zheng Li 0001, Liam O'Brien, Shayne Flint, Ramesh S. Sankaranarayana
FIE1
2014 Towards understanding the runtime configuration management of do-it-yourself content delivery network applications over public clouds
Zheng Li 0001, Karan Mitra, Miranda Zhang, Rajiv Ranjan 0001, Dimitrios Georgakopoulos 0001, Albert Y. Zomaya, Liam O'Brien
Future Gener. Comput. Syst.1
2014 On the Conceptualization of Performance Evaluation of IaaS Services
abstract
Cloud Computing has been increasingly accepted as a promising computing paradigm in industry, with one of the most common delivery models being Infrastructure as a Service (IaaS). An increasing number of providers have started to supply public IaaS services with different terminologies, definitions, and goals. As such, understanding the full scope of performance evaluation of candidate services would be crucial and beneficial for both service customers (e.g., cost-benefit analysis) and providers (e.g., direction of improvement). Given the numerous and diverse IaaS service features to be evaluated, a natural strategy is to implement different types of evaluation experiments separately. Unfortunately, it could be hard to fairly distinguish between different experimental types due to different environments and techniques that may be adopted by different evaluators. To overcome such obstacles, we have first established a novel taxonomy to help profile and clarify the nature of IaaS services performance evaluation and then built a three-layer conceptual model to generalize the existing performance evaluation practices. Using relevant elements/classifiers in the taxonomy and conceptual model, evaluators can construct natural language-style descriptions and experimental design blueprints to outline the evaluation scope and also to guide new evaluation implementations. In essence, the generated descriptions and blueprints abstractly define and characterize the actual evaluation work. This enables relatively fair and rational comparisons between different performance evaluations according to their abstract characteristics.
Zheng Li 0001, Liam O'Brien, He Zhang 0001, Rainbow Cai
IEEE Trans. Serv. Comput.1
2013 Boosting Metrics for Cloud Services Evaluation - The Last Mile of Using Benchmark Suites
abstract
Benchmark suites are significant for evaluating various aspects of Cloud services from a holistic view. However, there is still a gap between using benchmark suites and achieving holistic impression of the evaluated Cloud services. Most Cloud service evaluation work intended to report individual benchmarking results without delivering summary measures. As a result, it could be still hard for customers with such evaluation reports to understand an evaluated Cloud service from a global perspective. Inspired by the boosting approaches to machine learning, we proposed the concept Boosting Metrics to represent all the potential approaches that are able to integrate a suite of benchmarking results. This paper introduces two types of preliminary boosting metrics, and demonstrates how the boosting metrics can be used to supplement primary measures of individual Cloud service features. In particular, boosting metrics can play a summary Response role in applying experimental design to Cloud services evaluation. Although the concept Boosting Metrics was refined based on our work in the Cloud Computing domain, we believe it can be easily adapted to the evaluation work of other computing paradigms.
Zheng Li 0001, Liam O'Brien, He Zhang 0001, Rainbow Cai
AINA1
2013 Early Observations on Performance of Google Compute Engine for Scientific Computing
abstract
Although Cloud computing emerged for business applications in industry, public Cloud services have been widely accepted and encouraged for scientific computing in academia. The recently available Google Compute Engine (GCE) is claimed to support high-performance and computationally intensive tasks, while little evaluation studies can be found to reveal GCE's scientific capabilities. Considering that fundamental performance benchmarking is the strategy of early-stage evaluation of new Cloud services, we followed the Cloud Evaluation Experiment Methodology (CEEM) to benchmark GCE and also compare it with Amazon EC2, to help understand the elementary capability of GCE for dealing with scientific problems. The experimental results and analyses show both potential advantages of, and possible threats to applying GCE to scientific computing. For example, compared to Amazon's EC2 service, GCE may better suit applications that require frequent disk operations, while it may not be ready yet for single VM-based parallel computing. Following the same evaluation methodology, different evaluators can replicate and/or supplement this fundamental evaluation of GCE. Based on the fundamental evaluation results, suitable GCE environments can be further established for case studies of solving real science problems.
Zheng Li 0001, Liam O'Brien, Rajiv Ranjan 0001, Miranda Zhang
CloudCom (1)1
2013 CEEM: A Practical Methodology for Cloud Services Evaluation
abstract
Given an increasing number of Cloud services available in the market, evaluating candidate Cloud services is crucial and beneficial for both service customers (e.g. cost benefit analysis) and providers (e.g. direction of improvement). When it comes to performing any evaluation, a suitable methodology is inevitably required to direct experimental implementations. Nevertheless, there is still a lack of a sound methodology to guide the evaluation of Cloud services. By borrowing the lessons from evaluation of traditional computing systems, referring to the guidelines for Design of Experiments (DOE), and summarizing the existing experiences of real experimental studies, we proposed a generic Cloud Evaluation Experiment Methodology (CEEM) for Cloud services evaluation. Furthermore, we have established a pre-experimental knowledge base and specified corresponding suggestions to make this methodology more practical in the Cloud Computing domain. Through evaluating the Google AppEngine Python runtime as a preliminary validation, we show that Cloud evaluators may achieve more rational and convincing experimental results and conclusions following such an evaluation methodology.
Zheng Li 0001, Liam O'Brien, He Zhang 0001
SERVICES1
2013 On evaluating commercial Cloud services: A systematic review
Zheng Li 0001, He Zhang 0001, Liam O'Brien, Rainbow Cai, Shayne Flint
J. Syst. Softw.1
2012 Towards a Taxonomy of Performance Evaluation of Commercial Cloud Services
abstract
Cloud Computing, as one of the most promising computing paradigms, has become increasingly accepted in industry. Numerous commercial providers have started to supply public Cloud services, and corresponding performance evaluation is then inevitably required for Cloud provider selection or cost-benefit analysis. Unfortunately, inaccurate and confusing evaluation implementations can be often seen in the context of commercial Cloud Computing, which could severely interfere and spoil evaluation-related comprehension and communication. This paper introduces a taxonomy to help profile and standardize the details of performance evaluation of commercial Cloud services. Through a systematic literature review, we constructed the taxonomy along two dimensions by arranging the atomic elements of Cloud-related performance evaluation. As such, this proposed taxonomy can be employed both to analyze existing evaluation practices through decomposition into elements and to design new experiments through composing elements for evaluating performance of commercial Cloud services. Moreover, through smooth expansion, we can continually adapt this taxonomy to the more general area of evaluation of Cloud Computing.
Zheng Li 0001, Liam O'Brien, Rainbow Cai, He Zhang 0001
IEEE CLOUD1
2012 A factor framework for experimental design for performance evaluation of commercial cloud services
abstract
Given the diversity of commercial Cloud services, performance evaluations of candidate services would be crucial and beneficial for both service customers (e.g. cost-benefit analysis) and providers (e.g. direction of service improvement). Before an evaluation implementation, the selection of suitable factors (also called parameters or variables) plays a prerequisite role in designing evaluation experiments. However, there seems a lack of systematic approaches to factor selection for Cloud services performance evaluation. In other words, evaluators randomly and intuitively concerned experimental factors in most of the existing evaluation studies. Based on our previous taxonomy and modeling work, this paper proposes a factor framework for experimental design for performance evaluation of commercial Cloud services. This framework capsules the state-of-the-practice of performance evaluation factors that people currently take into account in the Cloud Computing domain, and in turn can help facilitate designing new experiments for evaluating Cloud services.
Zheng Li 0001, Liam O'Brien, He Zhang 0001, Rainbow Cai
CloudCom1
2011 A Qualitative Approach to Effort Judgment for Web Service Composition Based SOA Implementations
abstract
With more and more availability of services, Web service composition (WSC) based SOA implementations have increasingly become a significant type of SOA projects in practice. However, effort estimation for such a type of SOA project can still be limited because of the numerous and various approaches to WSC. Through viewing WSC based SOA system from a perspective of mechanistic organization, this paper borrows Divide-and-Conquer (D&C) as the generic strategy to narrow down the problem of effort judgment for the entire SOA implementation to that for individual WSCs. Moreover, benefiting from an effort-oriented classification matrix and a set of effort-related hypotheses, we assign scores to effort factors of WSC assisted by a set of rules. These effort scores are used to facilitate qualitatively judging different effort between different types of WSC approaches, and eventually construct an effort checklist for WSC approaches. Finally, this effort checklist can be used together with D&C algorithm to realize the qualitative effort judgment for WSC based SOA implementations.
Zheng Li 0001, Liam O'Brien
AINA1
2011 Towards Technology Independent Strategies for SOA Implementations
Zheng Li 0001, He Zhang 0001, Liam O'Brien
ENASE1