Software service discovery and recommendation help data scientists build scientific workflows - multi-step data analytics procedures - by automating the manual selection of services. Previous research shows that recommending chainable units of work (UoWs), rather than individual services, improves efficiency and reduces data shimming issues. However, UoW recommendation remains an NP-hard problem. To tackle this challenge, this study introduces a novel framework tailored to recommend UoWs in a goal-driven, context-aware manner, thereby facilitating workflow development. The framework is built around layered structure of software service social networks. At its foundation lies a service dependency network, where each edge represents a dependency between a two-service UoW within a specific context. The next layer abstracts each of these edges into a node, with new edges now representing three-service UoWs. This layering process continues iteratively, with subsequent layers capturing increasingly complex UoWs at higher levels of granularity. At high-order layers, UoW nodes are clustered based on their semantic embeddings, with each cluster represented by an intelligent agent. This approach transforms the workflow recommendation problem into a multi-agent collaboration task, where agents work together to identify high-level UoW groupings before refining selections by navigating down the layered structure for finer-grained recommendations. Experimental results over a real-world dataset confirm the effectiveness of the proposed framework in enhancing workflow composition efficiency.

As increasingly more APIs are published on the Internet, effective API recommendation remains a challenge yet highly demanded for mashup developers. This paper formalizes API recommendation as an incremental context-aware recom-mendation problem starting from a set of descriptive words and a set of APIs selected to date, supported by a fine-grained mashup-oriented knowledge graph (MKG). In contrast to traditional knowledge graphs where nodes are coarse-grained entities, entity-and relationship-encapsulated features are extracted as first-class citizens in an MKG, so that implicit feature relationships can be turned into explicit structural relationships. Two models are trained to learn fine-grained API selection strategies through path type patterns in the MKG, starting from intended descriptions and APIs selected, respectively. Extensive experiments over real-world datasets have demonstrated the effectiveness of the method.

As increasingly more software services have been published onto the Internet, it becomes critical yet highly challenging to recommend suitable services to facilitate scientific workflow composition. This paper proposes a novel Natural Language Processing (NLP)-inspired approach to recommending services throughout a workflow development process, based on incrementally learning latent service representation from workflow provenance. A work-flow composition process is formalized as a step-wise, context-aware service selection procedure, which is mapped to next-word prediction in a natural language sentence generation. Historical service dependencies are extracted from workflow provenance to build and enrich a knowledge graph. Each path in the knowledge graph reflects a scenario in a data analytics experiment, which is analogous to a sentence in a conversation. All paths are thus formalized as composable service sequences and are mined, using various patterns, from the established knowledge graph to construct a corpus. Service embeddings are then learned by applying deep learning model from the NLP field. Extensive experiments on the real-world dataset demonstrate the effectiveness and efficiency of the approach.

As service-oriented architecture becoming one prevalent technique to rapidly compose functionalities to customers, increasingly more reusable software components have been published online in the form of web services. To create a mashup, however, it gets not only time-consuming but also error-prone for developers to find suitable services components from such a sea of services. Service discovery and recommendation has thus attracted significant momentum in both academia and industry. This paper proposes a novel incremental recommend-as-you-go approach to recommending next potential service based on the context of a mashup under construction, considering services that have been selected up to the current step as well as the mashup goal. The core technique is an algorithm of learning the embedding of services, which learns their past goal-driven context-aware decision making behaviors in addition to their semantic descriptions and co-occurrence history. A goal exclusionary negative sampling mechanism tailored for mashup development is also developed to improve training performance. Extensive experiments on a real-world dataset demonstrate the effectiveness of this approach.

Service (API) discovery and recommendation is key to the wide spread of service oriented architecture and service oriented software engineering. Service recommendation typically relies on service linkage prediction calculated by the semantic distances (or similarities) among services based on their collection of inherent attributes. Given a specific context (mashup goal), however, different attributes may contribute differently to a service linkage. In this paper, instead of training a model for all attributes as a whole, a novel approach is presented to simultaneously train separate models for individual attributes. Meanwhile, a latent attribute modeling method is developed to reveal context-aware attribute distribution. Experiments over real-world datasets have demonstrated that this fine-grained method yields higher link prediction accuracy.

Workflow is an important way to mashup reusable software services to create value-added data analytics services. Workflow provenance is core to understand how services and workflows behaved in the past, which knowledge can be used to provide a better recommendation. Existing workflow provenance management systems handle various types of provenance separately. A typical data science exploration scenario, however, calls for an integrated view of provenance and seamless transition among different types of provenance. In this paper, a graph-based, uniform provenance model is proposed to link together design-time and run-time provenance, by combining retrospective provenance, prospective provenance, and evolution provenance. Such a unified provenance model will not only facilitate workflow mining and exploration, but also facilitate workflow interoperability. The model is formalized into colored Petri nets for verification and monitoring management. A SQL-like query language is developed, which supports basic queries, recursive queries, and cross-provenance queries. To verify the effectiveness of our model, A web-based, collaborative workflow prototyping system is developed as a proof-of-concept. Experiments have been conducted to evaluate the effectiveness of the proposed SQL-like graph query against SQL query.

The comprehensive and innovative evaluation of climate models with newly available global observations is critically needed for the improvement of climate model current-state representation and future-state predictability. A climate model diagnostic evaluation process requires physics-based multi-variable analyses that typically involve large-volume and heterogeneous datasets, making them both computation- and data-intensive. With an exploratory nature of climate data analyses and an explosive growth of datasets and service tools, scientists are struggling to keep track of their datasets, tools, and execution/study history, let alone sharing them with others. In response, we have developed a cloud-enabled, provenance-supported, web-service system called Climate Model Diagnostic Analyzer (CMDA). CMDA enables the physics-based, multivariable model performance evaluations and diagnoses through the comprehensive and synergistic use of multiple observational data, reanalysis data, and model outputs. At the same time, CMDA provides a crowdsourcing space where scientists can organize their work efficiently and share their work with others. CMDA is empowered by many current state-of-the-art software packages in web service, provenance, and semantic search.

Existing scientific workflow tools, created by computer scientists, require that domain scientists meticulously design their multi-step experiments before analyzing data. However, this is oftentimes contradictory to a domain scientist's routine of conducting research and exploration. This paper presents a novel way to resolve this dispute, in the context of service-oriented science. After scrutinizing how Earth scientists conduct data analytics research in their daily work, a provenance model is developed to record their activities. Reverse-engineering the provenance, a technology is developed to automatically generate workflows for scientists to review and revise, supported by a Petri nets-based workflow verification instrument. In addition, dataset is proposed to be treated as first-class citizen to drive the knowledge sharing and recommendation. A data-centric repository infrastructure is established to catch richer provenance to further facilitate collaboration in the science community. In this way, we aim to revolutionize computer-supported Earth science.