his paper presents a Bayesian model, called the Bayesian Bradley Terry (BBT) model, for comparing multiple algorithms on multiple data sets based on any metric. The model is an extension of the Bradley Terry model, which tracks the number of wins each algorithm has on different data sets. Unlike frequentist methods such as Demsar tests on mean rank or multiple pairwise Wilcoxon tests, the Bayesian approach provides a more nuanced understanding of the algorithms’ performance and allows for the definition of the “region of practical equivalence” (ROPE) for two algorithms. Additionally, the paper introduces the concept of “local ROPE,” which assesses the significance of the difference in mean measure between two algorithms using effect sizes, and can be applied in frequentist approaches as well. Both an R package and a Python program implementing the BBT are available for use.

Often, when dealing with real-world recognition problems, we do not need, and often cannot have, knowledge of the entire set of possible classes that might appear during operational testing. In such cases, we need to think of robust classification methods able to deal with the “unknown” and properly reject samples belonging to classes never seen during training. Notwithstanding, existing classifiers to date were mostly developed for the closed-set scenario, i.e., the classification setup in which it is assumed that all test samples belong to one of the classes with which the classifier was trained. In the open-set scenario, however, a test sample can belong to none of the known classes and the classifier must properly reject it by classifying it as unknown. In this work, we extend upon the well-known support vector machines (SVMs) classifier and introduce the open-set SVMs (OSSVMs), which is suitable for recognition in open-set setups. OSSVM balances the empirical risk and the risk of the unknown and ensures that the region of the feature space in which a test sample would be classified as known (one of the known classes) is always bounded, ensuring a finite risk of the unknown. In this work, we also highlight the properties of the SVM classifier related to the open-set scenario, and provide necessary and sufficient conditions for an RBF SVM to have bounded open-space risk.

We performed a controlled experiment comparing a C and a Python Introductory Programming course. Three faculty members at University of Campinas, Brazil, taught the same CS1 course for the same majors in two different semesters, one version in Python and one in C, with a total of 391 students involved in the experiment. We measured the dropout rate, the failure rate, the grades on the two exams, the proportion of completed lab assignments, and the number of submissions per completed assignment. There was no difference in the dropout rate. The failure rate for Python was 16.9% against 23.1% for C. The effect size (Cohen’s D) on the comparison of Python against C on the midterm exam was 0.27, and 0.38 for the final exam. The effect size for the proportion of completed assignments was 0.39 and the effect size for the number of submissions per assignment was -0.61 (Python had less submissions per completed assignments). Thus, for all measures, with the exception of dropout rate, the version of the course in Python yielded better student outcomes than the version in C and all differences are significant (with 95% confidence) with the exception of the failure rate ( p -value=0.12).

In recent years, software-based face presentation attack detection (PAD) methods have seen a great progress. However, most existing schemes are not able to generalize well in more realistic conditions. The objective of this competition is to evaluate and compare the generalization performances of mobile face PAD techniques under some real-world variations, including unseen input sensors, presentation attack instruments (PAI) and illumination conditions, on a larger scale OULU-NPU dataset using its standard evaluation protocols and metrics. Thirteen teams from academic and industrial institutions across the world participated in this competition. This time typical liveness detection based on physiological signs of life was totally discarded. Instead, every submitted system relies practically on some sort of feature representation extracted from the face and/or background regions using hand-crafted, learned or hybrid descriptors. Interesting results and findings are presented and discussed in this paper.

Tuning the regularisation and kernel hyperparameters is a vital step in optimising the generalisation performance of kernel methods, such as the support vector machine (SVM). This is most often performed by minimising a resampling/cross-validation based model selection criterion, however there seems little practical guidance on the most suitable form of resampling. This paper presents the results of an extensive empirical evaluation of resampling procedures for SVM hyperparameter selection, designed to address this gap in the machine learning literature. We tested 15 different resampling procedures on 121 binary classification data sets in order to select the best SVM hyperparameters. We used three very different statistical procedures to analyse the results: the standard multi- classifier/multi-data set procedure proposed by Dem\v{s}ar, the confidence intervals on the excess loss of each procedure in relation to 5-fold cross validation, and the Bayes factor analysis proposed by Barber. We conclude that a 2-fold procedure is appropriate to select the hyperparameters of an SVM for data sets for 1000 or more datapoints, while a 3-fold procedure is appropriate for smaller data sets.

Diabetic retinopathy (DR) is the leading cause of blindness in adults, but can be managed if detected early. Automated DR screening helps by indicating which patients should be referred to the doctor. However, current techniques of automated screening still depend too much on the detection of individual lesions. In this study, we bypass lesion detection, and directly train a classifier for DR referral. Additional novelties are the use of state-of-the-art mid-level features for the retinal images: BossaNova and Fisher Vector. Those features extend the classical Bags of Visual Words and greatly improve the accuracy of complex classification tasks. The proposed technique for direct referral is promising, achieving an area under the curve of 96.4%, thus, reducing the classification error by almost 40% over the current state of the art, held by lesion-based techniques.

Are computer science papers extended after they are published? We have surveyed 200 computer science publications, 100 journal articles, and 100 conference papers, using self‐citations to identify potential and actual continuations. We are interested in determining the proportion of papers that do indeed continue, how and when the continuation takes place, and whether any distinctions are found between the journal and conference populations. Despite the implicit assumption of a research line behind each paper, manifest in the ubiquitous “future research” notes that close many of them, we find that more than 70% of the papers are never continued.

Screening of Diabetic Retinopathy (DR) with timely treatment prevents blindness. Several researchers have focused their work on the development of computer-aided lesion-specific detectors. Combining detectors is a complex task as frequently the detectors have different properties and constraints and are not designed under a unified framework. We extend our previous work for detecting DR lesions based on points of interest and visual words to include additional detectors for the most common DR lesions and investigate fusion techniques to combine different classifiers for classification of normal or signs of diabetic retinopathy. The combination methods show promising results and shed light on the possible advantages of combining complementary lesion detectors for the DR diagnosis problem.

In health care settings, interactions between providers are uncommon. This study shows that electronic medical records currently available do not favor interactive work and thus a model of design rationale applied to health care is proposed. This model works as an extension to electronic medical records and intends to promote collaborative work among health care providers.

This paper addresses the issues surrounding user-to-user delegation in RBAC. We show how delegations can be incorporated into the RBAC model in a simple and straightforward manner. A special feature of the model is that it allows finegrained control over what rights a user wishes to delegate as opposed to delegation at the role level where all the rights of a role must be delegated. In addition, the model provides a rich set of controls regarding further delegations of a right, generic constraints that further control delegations, and an innovative model for revocations. Properties of both delegation and revocation are discussed, and our work is compared with other related research.

This paper presents a new formalism for workflow process definition, which combines research in programming languages and in database systems. This formalism is based on creating a library of workflow building blocks, which can be progressively combined and nested to construct complex workflows. Workflows are specified declaratively, using a simple high level language, which allows the dynamic definition of exception handling and events, as well as dynamically overriding workflow definition. This ensures a high degree of flexibility in data and control flow specification, as well as in reuse of workflow specifications to construct other workflows. The resulting workflow execution environment is well suited to supporting cooperative work.

Standard client-server workflow management systems are usually designed as client-server systems. The central server is responsible for the coordination of the workflow execution and, in some cases, may manage the activities database. This centralized control architecture may represent a single point of failure, which compromises the availability of the system. We propose a fully distributed and configurable architecture for workflow management systems. It is based on the idea that the activities of a case (an instance of the process) migrate from host to host, executing the workflow tasks, following a process plan. This core architecture is improved with the addition of other distributed components so that other requirements for Workflow Management Systems, besides scalability, are also addressed. The components of the architecture were tested in different distributed and centralized configurations. The ability to configure the location of components and the use of dynamic allocation of tasks were effective for the implementation of load balancing policies.

This paper presents a pair of role-based access control models for workflow systems, collectively known as the W-RBAC models. The first of these models, W0-RBAC is based on a framework that couples a powerful RBAC-based permission service and a workflow component with clear separation of concerns for ease of administration of authorizations. The permission service is the focus of the work, providing an expressive logic-based language for the selection of users authorized to perform workflow tasks, with preference ranking. W1-RBAC extends the basic model by incorporating exception handling capabilities through controlled and systematic overriding of constraints.

This paper introduces the idea of social agents, that is, agents whose goals are to maintain or improve the social context of a group, while working with a groupware system. We discuss Symgroup, a discussion tool augmented with three social agents, one that implements a Symlog agent, that analyses the participant's behavior, and two others that implement ad hoc theories about attitudes toward the discussion.

The focus of traditional workflow management systems is on control flow within one process definition. The process definition describes how a single case (i.e. workflow instance) in isolation is handled. For many applications this paradigm is inadequate. Interaction between cases to support communication and collaboration is at least as important. This paper introduces and advocates the use of interacting proclets, i.e. lightweight workflow processes. By promoting interactions to first-class citizens it is possible to model complex workflows in a more natural manner. In addition, the expressive power and flexibility are improved compared to the more traditional workflow modeling languages.

Presents a group critic system for object-oriented analysis and design. A group critic system is a critiquing system which is aware that the problems it finds in the design are the result of different users acting on different goals, and all are responsible for the problem. The environment also integrates a construction kit and an argumentative hypermedia system. We used annotation to point out criticisms, so that users can view the critiquing system as a true colleague. Annotations are also used as the cooperation medium among the designers.

This paper presents a model-based approach to fault diagnosis in computer networks. The idea behind a model-based diagnosis is to have a complete description of the local area network: the topology, type of equipment, routing tables, and so on. With this information and the knowledge of how IP packages travel in the network, one is able to perform a model-based reasoning for this network.

Standard client-server workflow management systems have an intrinsic scalability limitation: the central server is a bottleneck for large scale applications. It is also a single fault paint that may disable the whole system. We propose a fully distributed architecture for workflow management systems. It is based on the idea that the case (an instance of the process) migrates from host to host, according to the process definition specification, as the corresponding activities are executed. This basic architecture is improved so that other requirements for Workflow Management Systems, besides scalability, are also contemplated. A CORBA-based implementation of such architecture is discussed, with both its limitations and positive points described.

This paper discusses a conceptual model of groupware consisting of three complementary components or models: a description of the objects and operations on these objects available in the system; a description of the activities (and their orderings) that the users of the system can perform; and a description of the interface of users with the system, and with other users. KEYWORDS: Groupware, CSCW, collaboration technology, system modelling, ontological model, coordination model, user interface model. INTRODUCTION This paper is concerned with technology mediated workgroup systems, also sometimes called CSCW systems or groupware. We take the broad definition of these terms, as presented by Schmidt and Bannon [16]. We present a conceptual model which is applicable to a wide range of groupware---from electronic meeting rooms, through video conferencing, to workflow systems. The issues raised by the model are sometimes similar to, but frequently distinct from single user systems issues. T...