Robotic Process Automation (RPA) enables subject matter experts to use the graphical user interface as a means to automate and integrate systems. This is a fast method to automate repetitive, mundane tasks. To avoid constructing a software robot from scratch, Task Mining approaches can be used to monitor human behavior through a series of timestamped events, such as mouse clicks and keystrokes. From a so-called User Interface log (UI Log), it is possible to automatically discover the process model behind this behavior. However, when the discovered process model shows different process variants, it is hard to determine what drives a human’s decision to execute one variant over the other. Existing approaches do analyze the UI Log in search for the underlying rules, but neglect what can be seen on the screen. As a result, a major part of the human decision-making remains hidden. To address this gap, this paper describes a Task Mining framework that uses the screenshot of each event in the UI Log as an additional source of information. From such an enriched UI Log, by using image-processing techniques and Machine Learning algorithms, a decision tree is created, which offers a more complete explanation of the human decision-making process. The presented framework can express the decision tree graphically, explicitly identifying which elements in the screenshots are relevant to make the decision. The framework has been evaluated through a case study that involves a process with real-life screenshots. The results indicate a satisfactorily high accuracy of the overall approach, even if a small UI Log is used. The evaluation also identifies challenges for applying the framework in a real-life setting when a high density of interface elements is present.

Abstract The robotic process automation (RPA) paradigm is a discipline that is becoming increasingly popular thanks to the great interest shown by the industry. In such context, RPA solutions based on artificial intelligence, i.e., cognitive solutions, are receiving increasing attention. In a cognitive RPA project, the RPA developer is in charge of selecting the most suitable components that solve specific tasks from the sets of components provided by different RPA platforms. This selection is very challenging, especially since there is no homogeneity in component names or component classifications. Such a situation turns an RPA project’s development into a time-consuming, error-prone, and very tedious process. Therefore, supporting the RPA developer in developing a cognitive RPA project is desired. The industry has also pointed out this need. This work presents a proposal for supporting the users in developing a cognitive RPA project. To be more precise, an incremental method to automatically generate taxonomies from cognitive RPA platforms is proposed. Such taxonomies can be dynamically adapted when necessary. In previous work, the initial aspects of this research were presented. However, the current work greatly enhances such previous work by: (1) extending the proposed method to improve the management of real-world use cases from industry, (2) developing a proof-of-concept tool that is based on the proposed approach, (3) validating the proposed method by applying it to real-world use cases from industry, and (4) performing a literature review on related topics. The results obtained are auspicious and demonstrate that the proposed approach substantially improves the support given to users during the development of a cognitive RPA project.

Business process (BP) models are usually defined manually by business analysts through imperative languages considering activity properties, constraints imposed on the relations between the activities as well as different performance objectives. Furthermore, allocating resources is an additional challenge since scheduling may significantly impact BP performance. Therefore, the manual specification of BP models can be very complex and time-consuming, potentially leading to non-optimized models or even errors. To overcome these problems, this work proposes the automatic generation of imperative optimized BP models from declarative specifications. The static part of these declarative specifications (i.e. control-flow and resource constraints) is expected to be useful on a long-term basis. This static part is complemented with information that is less stable and which is potentially unknown until starting the BP execution, i.e. estimates related to (1) number of process instances which are being executed within a particular timeframe, (2) activity durations, and (3) resource availabilities. Unlike conventional proposals, an imperative BP model optimizing a set of instances is created and deployed on a short-term basis. To provide for run-time flexibility the proposed approach additionally allows decisions to be deferred to run-time by using complex late-planning activities, and the imperative BP model to be dynamically adapted during run-time using replanning. To validate the proposed approach, different performance measures for a set of test models of varying complexity are analyzed. The results indicate that, despite the NP-hard complexity of the problems, a satisfactory number of suitable solutions can be produced.