Information Retrieval systems rely on large test collections to measure their effectiveness in retrieving relevant documents. While the demand is high, the task of creating such test collections is laborious due to the large amounts of data that need to be annotated, and due to the intrinsic subjectivity of the task itself. In this paper we study the topical relevance from a user perspective by addressing the problems of subjectivity and ambiguity. We compare our approach and results with the established TREC annotation guidelines and results. The comparison is based on a series of crowdsourcing pilots experimenting with variables, such as relevance scale, document granularity, annotation template and the number of workers. Our results show correlation between relevance assessment accuracy and smaller document granularity, i.e., aggregation of relevance on paragraph level results in a better relevance accuracy, compared to assessment done at the level of the full document. As expected, our results also show that collecting binary relevance judgments results in a higher accuracy compared to the ternary scale used in the TREC annotation guidelines. Finally, the crowdsourced annotation tasks provided a more accurate document relevance ranking than a single assessor relevance label. This work resulted is a reliable test collection around the TREC Common Core track.

In this paper we propose a novel method of estimating verbal expressions of task and social cohesion by quantifying the dynamic alignment of nonverbal behaviors in speech. As team cohesion has been linked to team effectiveness and productivity, automatically estimating team cohesion can be a useful tool for assessing meeting quality and broader team functioning. In total, more than 20 hours of business meetings (3-8 people) were recorded and annotated for behavioral indicators of group cohesion, distinguishing between social and task cohesion. We hypothesized that behaviors commonly referred to as mimicry can be indicative of verbal expressions of social and task cohesion. Where most prior work targets mimicry of dyads, we investigated the effectiveness of quantifying group-level phenomena. A dynamic approach was adopted in which both the cohesion expressions and the paralinguistic mimicry were quantified on small time windows. By extracting features solely related to the alignment of paralinguistic speech behavior, we found that 2-minute high and low social cohesive regions could be classified with a 0.71 Area under the ROC curve, performing on par with the state-of-the-art where turn-taking features were used. Estimating task cohesion was more challenging, obtaining an accuracy of 0.64 AUC, outperforming the state-of-the-art. Our results suggest that our proposed methodology is successful in quantifying group-level paralinguistic mimicry. As both the state-of-the-art turn-taking features and mimicry features performed worse on estimating task cohesion, we conclude that social cohesion is more openly expressed by nonverbal vocal behavior than task cohesion.

As crowdsourcing gains popularity, organisations seek ways to systematically and reliably involve their workforce with data processing pipelines. Mobile crowdsourcing allows for opportunistic task executions and thus, potentially, for higher throughput. However, how to engage and to retain employees in enterprise crowdsourcing campaigns is still an open research topic. This paper discusses the results of a study performed in IBM Benelux. We surveyed 93 employees to discover the factors that might affect engagement in mobile enterprise crowdsourcing. The survey informed the design of an experiment that aimed at investigating the effectiveness of different task notification strategies. We studied how factors such as time and context of notification can affect the participation and retention of employees. Results show that break times are the most suitable for crowd work, and that ``aggressive'' notification strategies act as deterrent for participation, while moderate yet regular nudges are the most likely to retain contributors.

In 2010 the popular paper by Kwak et al. [11] presented the first comprehensive study of Twitter as it appeared in 2009, using most of the Twitter network at the time. Since then, Twitter's popularity and usage has exploded, experiencing a 10-fold increase. As of 2015, it has more than 500 million users, out of which 316 million are active, i.e. logging into the service at least once a month.1In this study we revisit the network observed by Kwak et al. to examine the changes exhibited in both the graph and the behavior of the users in it. Our results conclude to a denser network, showing an increase in the number of reciprocal edges, despite the fact that around 12.5% of the 2009 users have now left Twitter. However, the network's largest strongly connected component seems to be significantly decreasing, suggesting a movement of edges towards popular users. Furthermore, we observe numerous changes in the lists of influential Twitter users, having several accounts that where not popular in the past securing a position in the top-20 list as new entries.

The aim of the paper is to show that localization of observed moving objects is possible in a multicamera environment, based on the simple foreground mask and the estimation of the size distribution on image plane. The method is flexible, it can handle arbitrary number of cameras and objects. It is based only on change detection masks and does not depend on appearance information. Due to its statistical nature the proposed method efficiently handles such challenging situations as changes in viewpoint, occlusions due to view variations, background clutter.

The paper introduces a video surveillance and event detection framework and application for semi-supervised surveillance use. The systempsilas intended use is in automatic mode on camera feeds that are not actively watched by surveillance personnel, and should raise alarms when unusual events occur. We present the current detector filters, and the extendable modular interface. Filters include local and global unusual motion detectors, left/stolen object detector, motion detector, tampering/failure detector, etc. The system stores the events and associated data, which can be organized, searched, annotated and (re)viewed. It has been tested in real life situation for police street surveillance.

Extraction of foreground is a basic task in surveillance video analysis. In most real cases, its performance is heavily based on the efficiency of shadow detection and on the analysis of lighting conditions and reflections caused by mirrors or other reflective surfaces. This correspondence is focused on the improvement of foreground extraction in the case of planar reflective surfaces. We show that the geometric model of a scene with a planar reflective surface is reduced to the estimation of vanishing-point for the case of an auto-epipolar (skew-symmetric) fundamental matrix. The correspondences for the vanishing-point estimation are extracted from motion statistics. The knowledge of the position of the vanishing point allows us to integrate the geometric model and the motion statistics into image foreground-extraction to separate foreground from reflections, and thus to achieve better performance. The experiments confirm the accuracy of the vanishing point and the improvement of the foreground image mask by removing reflected object parts.

This paper presents a robust walk-detection algorithm, based on our symmetry approach which can be used to extract gait characteristics from video-image sequences. To obtain a useful descriptor of a walking person, we temporally track the symmetries of a person's legs. Our method is suitable for use in indoor or outdoor surveillance scenes. Determining the leading leg of the walking subject is important, and the presented method can identify this from two successive walk steps (one walk cycle). We tested the accuracy of the presented walk-detection method in a possible application: Image registration methods are presented which are applicable to multicamera systems viewing human subjects in motion.

A new motion-based method is presented for automatic registration of images in multicamera systems, to permit synthesis of wide-baseline composite views. Unlike existing static-image and motion-based methods, our approach does not need any a priori information about the scene, the appearance of objects in the scene, or their motion. We introduce an entropy-based preselection of motion histories and an iterative Bayesian assignment of corresponding image areas. Finally, correlated point-histories and data-set optimization lead to the matching of the different views. The method is validated by demonstrating its successful use on several real-life indoor and outdoor stereo video image-sequence pairs.

In the paper a new Bayesian method is presented for the automatic extraction of common areas of images in multi-camera systems through the detection of concurrently changing pixels. Unlike existing still-image and motion-based methods our approach does not need any a priori information about the scene, the appearance of objects in the scene, or their motion. The method is validated by demonstrating its successful use on several real-life outdoor stereo video image-sequence pairs.

We outline our latest methods and algorithmic solutions for detection of concurrent motions and registering cameras in real-life surveillance systems. We show that cameras can be registered in several rather unfavorable conditions, based on: (1) unpredictable motion without structured background or defined object shapes or (2) walking persons of undefined silhouettes and short detectable walking distances or (3) shadows of undefined structures in front of flickering background. These methods are tested in real-life sequences and we show that flexible 3D camera registration is possible even in bad lighting conditions and in the lack of any known structures or motions.

In this paper we present a symmetry-based approach which can be used to detect humans and to extract biometric characteristics from video image-sequences. The method employs a simplified symmetry-feature extracted from the images. To obtain a useful descriptor of a walking person, we track temporally the symmetries which result from the movements of the person's legs. In a further processing stage these patterns are filtered, then re-sampled using Bezier-splines to generate an invariant and noise-cleaned signature or "feature". In our detection method the extracted spatio-temporal feature with a large number of dimensions (800) is transformed to a space with a much smaller number of dimensions (3), which we call the "eigenwalks space"; the method uses principal component analysis (PCA) to reduce the dimensionality, and the support vector machine (SVM) method in the eigenspace for recognition purposes. Finally we present a method by which we can estimate the gait-parameters (the beginning and end of a walk-cycle, identification of the leading leg) from the symmetry-patterns of the walking person, without camera calibration, based on two successive detected walk-steps.

In the paper we introduce an algorithm for matching partially overlapping image-pairs where the object of interest is in motion, even if the motion is discontinuous and in an unstructured environment. In our previous work [Z. Szlavik et al, 2004] we have shown that by using co-motion statistics matching of overlapping views can be done and then the projective geometry can be estimated. Here, we show how to optimize searching for concurrently moving pixels. The robust algorithm we describe here finds point correspondences in two images by using entropy-based thresholding and without searching for any structures and without the need for tracking continuous motion. Our method makes it possible to (re)calibrate multicamera systems without human assistance.

Abstract The concepts of Shannon information and entropy have been applied to a number of information retrieval tasks such as to formalize the probabilistic model, to design practical retrieval systems, to cluster documents, and to model texture in image retrieval. In this report, the concept of entropy is used for a different purpose. It is shown that any positive Retrieval Status Value (RSV)‐based retrieval system may be conceived as a special probability space in which the amount of the associated Shannon information is being reduced; in this view, the retrieval system is referred to as Uncertainty Decreasing Operation (UDO). The concept of UDO is then proposed as a theoretical background for term and query discrimination power, and it is applied to the computation of term and query discrimination values in the vector space retrieval model. Experimental evidence is given as regards such computation; the results obtained compare well to those obtained using vector‐based calculation of term discrimination values. The UDO‐based computation, however, presents advantages over the vector‐based calculation: It is faster, easier to assess and handle in practice, and its application is not restricted to the vector space model. Based on the ADI test collection, it is shown that the UDO‐based Term Discrimination Value (TDV) weighting scheme yields better retrieval effectiveness than using the vector‐based TDV weighting scheme. Also, experimental evidence is given to the intuition that the choice of an appropriate weighting scheme and similarity measure depends on collection properties, and thus the UDO approach may be used as a theoretical basis for this intuition.