The need for advanced automation and artificial intelligence (AI) in various fields, including text classification, has dramatically increased in the last decade, leaving us critically dependent on their performance and reliability. Yet, as we increasingly rely more on AI applications, their algorithms are becoming more nuanced, more complex, and less understandable precisely at a time we need to understand them better and trust them to perform as expected. Text classification in the medical and cybersecurity domains is a good example of a task where we may wish to keep the human in the loop. Human experts lack the capacity to deal with the high volume and velocity of data that needs to be classified, and ML techniques are often unexplainable and lack the ability to capture the required context needed to make the right decision and take action. We propose a new abstract configuration of Human-Machine Learning (HML) that focuses on reciprocal learning, where the human and the AI are collaborating partners. We employ design-science research (DSR) to learn and design an application of the HML configuration, which incorporates software to support combining human and artificial intelligences. We define the HML configuration by its conceptual components and their function. We then describe the development of a system called Fusion that supports human-machine reciprocal learning. Using two case studies of text classification from the cyber domain, we evaluate Fusion and the proposed HML approach, demonstrating benefits and challenges. Our results show a clear ability of domain experts to improve the ML classification performance over time, while both human and machine, collaboratively, develop their conceptualization, i.e., their knowledge of classification. We generalize our insights from the DSR process as actionable principles for researchers and designers of 'human in the learning loop' systems. We conclude the paper by discussing HML configurations and the challenge of capturing and representing knowledge gained jointly by human and machine, an area we feel has great potential.

Prediction methods applied to digital network data offer powerful capabilities that have radically affected a host of industries and services. Specifically, numerous works have shown that the use of network information to predict focal node properties produces significantly more accurate results compared with exclusive reliance on other types of data. In this study, we propose that it may be possible to improve network-information-based predictions by identifying network links that actually carry predictive power for a given prediction task. For this purpose, we suggest a problem referred to as the problem of Increasing Network Predictability (INP) by data-driven link screening. To address this problem we develop a new algorithm with three different implementations. We find that the algorithm is robust and consistently outperforms baseline link selection methods. We thus suggest that our algorithm has the potential to improve the efficacy of network data use for classification purposes.

Text mining have gained great momentum in recent years, with user-generated content becoming widely available. One key use is comment mining, with much attention being given to sentiment analysis and opinion mining. An essential step in the process of comment mining is text pre-processing; a step in which each linguistic term is assigned with a weight that commonly increases with its appearance in the studied text, yet is offset by the frequency of the term in the domain of interest. A common practice is to use the well-known tf-idf formula to compute these weights. This paper reveals the bias introduced by between-participants' discourse to the study of comments in social media, and proposes an adjustment. We find that content extracted from discourse is often highly correlated, resulting in dependency structures between observations in the study, thus introducing a statistical bias. Ignoring this bias can manifest in a non-robust analysis at best and can lead to an entirely wrong conclusion at worst. We propose an adjustment to tf-idf that accounts for this bias. We illustrate the effects of both the bias and correction with with seven Facebook fan pages data, covering different domains, including news, finance, politics, sport, shopping, and entertainment.

Confidential information is all too easily leaked by naive users posting comments. In this paper we introduce DUIL, a system for Detecting Unintentional Information Leakers. The value of DUIL is in its ability to detect those responsible for information leakage that occurs through comments posted on news articles in a public environment, when those articles have withheld material non-public information. DUIL is comprised of several artefacts, each designed to analyse a different aspect of this challenge: the information, the user(s) who posted the information, and the user(s) who may be involved in the dissemination of information. We present a design science analysis of DUIL as an information system artefact comprised of social, information, and technology artefacts. We demonstrate the performance of DUIL on real data crawled from several Facebook news pages spanning two years of news articles.

This study investigates the interplay between online news, reader comments, and social networks to detect and characterize comments leading to the revelation of censored information. Censorship of identity occurs in different contexts–for example, the military censors the identity of personnel and the judiciary censors the identity of minors and victims. We address three objectives: (a) assess the relevance of identity censorship in the presence of user‐generated comments, (b) understand the fashion of censorship circumvention (what people say and how), and (c) determine how comment analysis can aid in identifying decensorship and information leakage through comments. After examining 3,582 comments made on 48 articles containing obfuscated terms, we find that a systematic examination of comments can compromise identity censorship. We identify and categorize information leakage in comments indicative of knowledge of censored information that may result in information decensorship. We show that the majority of censored articles contained at least one comment leading to censorship circumvention.

This paper simulates Companies' ego networks on Twitter, meaning the companies' number and type of followers. Evident from our data, we show that followers' distribution, in our focus, is neither scale free nor random, thus common network simulations cannot be used to mimic observed data. We present novel rate equations model to capture the complex dynamics of these ego networks.

In this work we study information leakage through discussions in online social networks. In particular, we focus on articles published by news pages, in which a person's name is censored, and we examine whether the person is identifiable (decensored) by analyzing comments and social network graphs of commenters. As a case study for our proposed methodology, in this paper we considered 48 articles (Israeli, military related) with censored content, followed by a threaded discussion. We qualitatively study the set of comments and identify comments (in this case referred as "leakers") and the commenter and the censored person. We denote these commenters as "leakers". We found that such comments are present for some 75% of the articles we considered. Finally, leveraging the social network graphs of the leakers, and specifically the overlap among the graphs of the leakers, we are able to identify the censored person. We show the viability of our methodology through some illustrative use cases.

Though biomedical research often draws on knowledge from a wide variety of fields, few visualization methods for biomedical data incorporate meaningful cross-database exploration. A new approach is offered for visualizing and exploring a query-based subset of multiple heterogeneous biomedical databases. Databases are modeled as an entity-relation graph containing nodes (database records) and links (relationships between records). Users specify a keyword search string to retrieve an initial set of nodes, and then explore intra- and interdatabase links. Results are visualized with user-defined semantic substrates to take advantage of the rich set of attributes usually present in biomedical data. Comments from domain experts indicate that this visualization method is potentially advantageous for biomedical knowledge exploration.

We consider a virtual computing environment that provides computational resources on demand to users with multi attribute task descriptions that include a valuation, resource (CPU) needs and a completion deadline. Achieving a high quality of service in this environment depends on finding a balance between processing high priority tasks before their deadlines expire, while maximizing resource utilization. The problem becomes more challenging in an economic setting, where the task valuation is private. We propose a bid-based server that publishes a history of the success rate table (SRT) for processed tasks. Clients use the history to optimize their bid for resources on a (single) multiprocessor server. The server schedules tasks in descending order of their bid- Highest Bid First (HBF) and backfills the schedule with smaller tasks when resources are still available. The scheduler follows a hard deadline model where tasks cannot be processed after their deadline. We propose three variations of the SRT where biding history is publicized at different granularity. Using a simulation based study, we analyze the behavior of clients' bids in respond to the SRT. We compare the best HBF variant with an efficient Earliest Deadline First (EDF) mechanism that charges a fixed price. Our results show that the HBF mechanism is able to exploit price discrimination and therefore complete the execution of more high value jobs under a heavy workload, leading to better weighted throughput. HBF can also maximize server profit and client surplus (the difference between value and the client bid) in different settings. Thus, HBF may yield solutions that benefit both the client and the server.