VLDB 2026 Research / reviewers in the wild / expert
Bostjan Sumak
dblp:67/10196
· DBLP profile ↗
14ranked-venue papers
4as first author
5since 2021 · last 2025
0000-0001-5535-3477ORCID · verified
Domains — the database's venue-derived domains; a paper can count in several
Databases, data management, data science and information retrieval · 8 · 3 first-author · 2 since 2021Software engineering, systems software and programming languages · 3 · 2 since 2021Human-computer interaction and ubiquitous computing · 2 · 1 since 2021Applied, interdisciplinary, general and emerging computing · 2 · 1 first-author · 1 since 2021
| Year | Publication | Venue | Position |
|---|---|---|---|
| 2025 | A Comprehensive User-Centric Method for Evaluating and Comparing XAI ExplanationsabstractEvaluation and selection of explanations in explainable artificial intelligence (XAI) is essential as AI systems grow, increasing transparency needs. Existing methods rarely built on theoretical grounding and failed to distinguish perceived and actual user understanding. This paper proposes a comprehensive user evaluation method for XAI explanations in a structured questionnaire format, enabling efficient data collection and comparison. Core explanation dimensions recognized in literature were integrated into our method: trust, satisfaction, mental models, and mental effort. Reliable instruments analyzed with exploratory factor analysis were used to assess trust and satisfaction. Mental models were assessed through reflection, prediction, and glitch detection tasks. Ten XAI techniques were evaluated: image-based (LIME positive and negative masking, feature occlusion, saliency maps, SHAP, LIME overlay mask) and numeric-based (SHAP bar chart, partial dependence plots, decision tree, counterfactual explanations, ANCHOR). Our method effectively differentiates explanations across multiple dimensions, aiding the selection of the most suitable technique for end-users. Sasa Brdnik, Saso Karakatic, Bostjan Sumak |
Int. J. Hum. Comput. Interact. | 3 |
| 2023 | Commit Classification Into Software Maintenance Activities: A Systematic Literature ReviewabstractCommits represent an essential part of software development practices, serving as the means for collaboration and management of software changes made to a software project’s codebase. Changes are necessary for the survival of software, for instance, to fix faults, address security vulnerabilities, meet new functional requirements, and improve software performance. To aid software research and practice, several research endeavors attempted to automate the process of identifying the nature of the maintenance tasks performed in a commit based on the data associated with the commit. This paper presents a systematic literature review of supervised-learning-based models for commit classification into maintenance activities. Through the study selection process, 19 primary studies were identified, published between 2008 and 2023 in various journals and conference proceedings. For the independent variables for classification, features extracted from commit messages are prevalent, followed by features extracted from source code and commit metadata. In the majority of existing studies, multi-class classification approaches are used, whereas multi-label approaches are considered rarely. The most commonly used classification algorithms include Decision Tree, Random Forest, Naive Bayes, and Neural Network. Several datasets exist, consisting mainly of commits from different open-source projects based on the Java programming language. Some research gaps and open challenges were identified that can guide future research efforts. Tjasa Hericko, Bostjan Sumak |
COMPSAC | 2 |
| 2023 | Identification and Analysis of Factors Impacting e-Inclusion in Higher EducationabstractIn this chapter, we investigate the complex process of analyzing and understanding the factors that influence individuals’ access to and participation in digital education within the Higher Education context. While the digital transformation in Higher Education Institutions (HEI) has produced numerous benefits for both students and educators, it has also brought forth challenges, particularly for students with special educational needs and disabilities (SEND). A literature review was conducted, to gain insight into the specific requirements of this student demographic. This review aimed to identify the multifaceted factors that impact e-inclusion within Higher Education. Our research resulted in the identification of 24 different factors that should be considered when evaluating e-inclusion within HEI. These factors serve as essential indicators in the assessment of the accessibility and inclusivity of digital education, allowing for a more multifaceted understanding of the dynamics in the Higher Education landscape. Maja Pusnik, Katja Kous, Tatjana Welzer, Bostjan Sumak |
EJC | 4 |
| 2022 | Enforcing Consistent Code Style in a Repository While Allowing Developer-Specific Preferences in Local Workspaces: An Experience ReportabstractWhen developers collaborate on a software project, the style of the code should be consistent across the codebase. However, as developers do not always concur on code style practices, achieving a consensus can be burdensome. Even when code style guidelines are defined for a project, developers often have difficulties adhering to them. Additionally, when working on several projects with diverse guidelines, following different styles can become a frustrating, error-prone, and time-consuming task for a developer. This work presents experiences from adopting two development workflow patterns that automatically ensure that code’s layout and formatting are consistent in a project’s repository while enabling each developer to utilize their preferences locally. From our experience, using any of these two patterns provides a successful way for collaborative software development that allows developers to relocate their time and concerns from code formatting to more valuable matters. Tjasa Hericko, Bostjan Sumak |
EASE | 2 |
| 2022 | Analysis of the Shortest Path Method Application in Social NetworksabstractThis paper analyzes the shortest path problem (SPP) in social networks, based on the investigation and implementation of different methods on a simulated example. The objectives of the paper include identification of the most commonly used methods for finding the shortest path in a social network as a strategic attempt to speed the search of network nodes, focusing on the application of the two most used SPP methods: the Dijkstra and Bellman-Ford algorithms. A comparative analysis is used as an investigation method for performance evaluation of different algorithms, based on their implementation and behavior, tested on a social network example. The research results indicate that the Dijkstra algorithm is faster, and therefore more suitable for searching the shortest connection in social networks. Bostjan Sumak, Maja Pusnik |
EJC | 1 |
| 2020 | Evaluation of XML Schema Support in Knowledge ManagementabstractWell organized data contributes extensively to the classification possibilities and quality of Knowledge Management. XML schemas play an important role in data organization activities, and provide basic foundations for companies and organizations dealing with large amounts of data. In times where knowledge represents the greatest advantage in a competitive economy and is relatively simple to find through different web providers, the quality of internal data structures and efficient management of a company’s valuable information is of the utmost importance. XML schemas are one of the mechanisms that can provide a data organization system in a qualitative manner, and efficient knowledge management as soon as data have been defined or accumulated. A good XML schema support is a way to increase the competitiveness of an organization by ensuring structured data quality and simplifying the Knowledge Management process. Bostjan Sumak, Marjan Hericko, Maja Pusnik |
EJC | 1 |
| 2020 | Perceived Usability and the Modified Technology Acceptance ModelabstractIn response to recent criticism of the usefulness of the construct of usability, we investigated the relationships between measures of perceived usability and the components of a modified version of the Technology Acceptance Model (mTAM) – Perceived Usefulness (PU) and Perceived Ease-of-Use (PEU). In three surveys, respondents used SUS, UMUX-LITE and mTAM to rate their actual (as opposed to expected) experience with three software products. As expected, the correlations between PEU and other measures of perceived usability tended to be significantly stronger than those with PU. Additional findings support the use of the UMUX-LITE as a compact measure of perceived usability that has a strong relationship to the mTAM and strong correspondence with concurrently collected SUS scores. The main theoretical result of this research were regression results providing evidence that the PEU component of the mTAM appears to be another measure of the construct of perceived usability, connecting the TAM to the construct of perceived usability through the mTAM and providing evidence against the claim that the construct of usability is a theoretical dead end. Urska Lah, James R. Lewis, Bostjan Sumak |
Int. J. Hum. Comput. Interact. | 3 |
| 2019 | A Case-Based Analysis of Process Modeling for Public Administration System Design
Gregor Polancic, Bostjan Sumak, Maja Pusnik |
EJC | 2 |
| 2019 | A Survey-Based Evaluation of Approaches and Tools for Business Process Optimization SupportabstractThe paper addresses the field of Process Optimization in Slovenian companies, and their attitude toward optimization approaches. A survey was conducted, including professionals from companies, as well as students, examining approaches to assure business process quality, its validation and optimization, containing several methods. Survey results, both in companies (24 participants) and among students (24 participants) indicate that, although a combination of methods provides the best results, modeling, KPI identification and simulation are the best indicators of process strengths and weaknesses, as well as optimization potentials. Therefore, the research was complemented with a review of 5 available modeling and simulation tools, providing evaluation and comparison. To get insight into which of the tools best supports the optimization process, a comparative analysis was conducted, identifying key differences between them, and providing suggestions for optimization enhancement in companies. Maja Pusnik, Tatjana Welzer, Marjan Hericko, Bostjan Sumak |
EJC | 4 |
| 2019 | An empirical evaluation of a hands-free computer interaction for users with motor disabilities
Bostjan Sumak, Matic Spindler, Mojca Debeljak, Marjan Hericko, Maja Pusnik |
J. Biomed. Informatics | 1 |
| 2018 | Evaluation of Patterns for Mobile User Interface EnhancementabstractIn the paper, we explore the topic of mobile user experience and we focus on native mobile applications. The goal of this paper is to research what mobile user experience is, identify important mobile patterns, and define processes and guidelines for the design and measurement of the mobile user experience and the identified mobile patterns. Based on the literature review, there are several elements or factors affecting the mobile user experience. We established a big advantage of native applications, which are designed for a specific platform, where there is more time to create a specific customer experience. The main findings of the paper showed that the mobile user experience differs from the desktop user experience, and that there are many mobile patterns, which may significantly improve or degrade the mobile user experience for various target groups. Maja Pusnik, Dimitar Ivanovski, Tatjana Welzer, Bostjan Sumak |
EJC | 4 |
| 2016 | Optimization of Processes in Multimedia Content Publishing: A Proposal for Architecture DesignabstractOrganizations in the publishing field accumulate large amounts of data and store it in different forms of knowledge databases. In the publishing field specifically, various multimedia content presentation is involved, requiring support for contemporary e-book standard formats (e-Pub) as well as traditional standard formats such as HTML, PDF, and others. Publishing organizations must support the same content presentation in various formats in order to meet the needs of their increasingly demanding clients. However, business (and organizational) processes in publishing in the presented case study have little IT support and almost no automation. Successful business flow and a competitive advantage for publishing organizations, especially for SMEs, requires a holistic solution for the optimization of publishing processes, including data/knowledge acquisition, data/knowledge aggregation and the creation of new knowledge based on existing data. Although partial IT solutions are available and can be used to support individual steps in the publishing process, they do not come in an out-of-the box solutions, suitable for adequate publishing processes automation and optimization. Two basic challenges regarding existing solutions available on the market are: (1) they do not provide support for automation or semi automation in the publishing processes, and (2) they are expensive and consequently not acceptable for SMEs. This paper is focused on analyzing publishing work processes and its quality aspects, addressing the automation of all possible process steps: especially the optimization of acquisition, aggregation and building of multimedia content for any device. In addition, a solution built on open source components is proposed as well, supporting the presentation of content in different published formats. Based on a case study, we present the solution's architectural design and analyze standards and technologies, providing an acceptable solution for SME's from an economical point of view, optimizing its work processes to the largest possible extent. A renewed publishing process is proposed and future research directions are also discussed. Bostjan Sumak, Marjan Hericko, Tatjana Welzer, Maja Pusnik |
EJC | 1 |
| 2015 | Acceptance analysis based on behavioural operational research methodsabstractAcceptance analysis is a well-known and widely used approach to evaluate the level of a product's or a solution's acceptance. A lot of research has been conducted in the field of software quality and acceptance. However, there is still a gap between potential users and the percentage of users that positively accept and use certain products. In addition, the process of gathering information regarding users' decisions is difficult and subjective. Users often decide against the use of solutions due to several reasons, not all of them objective, having different attitudes toward solutions due to a variety of factors. The paper focuses on the acceptance analysis based on existing models and users' behaviour characteristics. Research results present a foundation for an integrated acceptance model based on the integration of behavioural operational research and theory acceptance models, proposing a foundation for measuring and increasing the acceptance level of solutions based on human behaviour. Maja Pusnik, Marjan Hericko, Bostjan Sumak |
EJC | 3 |
| 2014 | Investigation of Developer's Perceptions in XML Schema Development Using Textual and Visual Tool TypesabstractThis paper analyses the influence of different tool types (visual or textual) on a developer's perception of efficiency during XML Schema development. We conducted a controlled experiment that focused on discovering which XML Schema development tool type enables better efficiency and also engenders a friendlier environment for developers while developing XML Schemas. The experiment was conducted with 240 participants and divided into two practical parts (visually developing an XML Schema and manually using a mark-up language intelligent textual editor). After the experiment, the participants' opinions were gathered via a web survey. In the survey, a technology acceptance model (TAM) was used as the basis for constructing the measurement items in order to answer the following questions: (1) which tool type is preferred and perceived as better, and (2) which variables influence the user's perceptions and decisions. In this study, we searched for an optimal way of building XML Schemas. The general tendency of most participants was towards a visual tool, suggesting that visual support is perceived as more useful, and can create better results with less effort. Maja Pusnik, Katja Harej Pulko, Marjan Hericko, Matjaz B. Juric, Bostjan Sumak |
Int. J. Softw. Eng. Knowl. Eng. | 5 |