Owing to their large field of view, overhead fisheye cameras are becoming a surveillance modality of choice for large indoor spaces. However, traditional people detection and tracking algorithms developed for side-mounted, rectilinear-lens cameras do not work well on images from overhead fisheye cameras due to their viewpoint and unique optics. While several people-detection algorithms have been recently developed for such cameras, they have all been tested on datasets consisting of "staged" recordings with a limited variety of people, scenes and challenges. Clearly, the performance of these algorithms "in the wild", i.e., on recordings with real-world challenges, remains un-known. In this paper, we introduce a new benchmark dataset of in-the-Wild Events for People Detection and Tracking from Overhead Fisheye cameras (WEPDTOF)1. The dataset features 14 YouTube videos captured in a wide range of scenes, 188 distinct person identities consistently labeled across time, and real-world challenges such as extreme occlusions and camouflage. Also, we propose 3 spatiotemporal extensions2of a state-of-the-art people-detection algorithm to enhance the coherence of detections across time. Compared to top-performing algorithms, that are purely spatial, the new algorithms offer a significant performance improvement on the new dataset. Finally, we compare the people tracking performance of these algorithms on WEPDTOF.

Person re-identification using rectilinear cameras has been thoroughly researched to date. However, the topic has received little attention for fisheye cameras and the few developed methods are appearance-based. We propose a geometry-based approach to re-identification for overhead fisheye cameras with overlapping fields of view. The main idea is that a person visible in two camera views is uniquely located in the view of one camera given their height and location in the other camera’s view. We develop a height-dependent mathematical relationship between these locations using the unified spherical model for omnidirectional cameras. We also propose a new fisheye-camera calibration method and a novel automated approach to calibration-data collection. Finally, we propose four re-identification algorithms that leverage geometric constraints and demonstrate their excellent accuracy, which vastly exceeds that of a state-of-the-art appearance-based method, on a fisheye-camera dataset we collected.

Background subtraction is a basic task in computer vision and video processing often applied as a pre-processing step for object tracking, people recognition, etc. Recently, a number of successful background-subtraction algorithms have been proposed, however nearly all of the top-performing ones are supervised. Crucially, their success relies upon the availability of some annotated frames of the test video during training. Consequently, their performance on completely "unseen" videos is undocumented in the literature. In this work, we propose a new, supervised, background-subtraction algorithm for unseen videos (BSUV-Net) based on a fully-convolutional neural network. The input to our network consists of the current frame and two background frames captured at different time scales along with their semantic segmentation maps. In order to reduce the chance of overfitting, we also introduce a new data-augmentation technique which mitigates the impact of illumination difference between the background frames and the current frame. On the CDNet-2014 dataset, BSUV-Net outperforms stateof-the-art algorithms evaluated on unseen videos in terms of several metrics including F-measure, recall and precision.

We propose two supervised methods for people counting using an overhead fisheye camera. As opposed to standard cameras, fisheye cameras offer a large field of view and, when mounted overhead, reduce occlusions. However, methods developed for standard cameras perform poorly on fisheye images since they do not account for the radial image geometry. Furthermore, no large-scale fisheye-image datasets with radially-aligned bounding box annotations are available for training. We adapt YOLOv3 trained on standard images for people counting in fisheye images. In one method, YOLOv3 is applied to 24 rotated, overlapping windows and the results are post-processed to produce a people count. In another method, YOLOv3 is applied to windows of interest extracted by background subtraction. For evaluation, we collected and annotated an indoor fisheye-image dataset that we make public. Experiments on this dataset show that our methods reduce the people counting MAE of two natural benchmarks by over 60%.

Hand-specific on-skin (HSoS) gestures are a trending interaction modality yet there is a gap in the field regarding users' preferences about these gestures. Thus, we conducted a user-elicitation study collecting 957 gestures from 19 participants for 26 commands. Results indicate that (1) users use one hand as a reference object, (2) load different meanings to different parts of the hand, (3) give importance to hand-properties rather than the skin properties and (4) hands can turn into self-interfaces. Moreover, according to users' subjective evaluations, (5) exclusive gestures are less tiring than the intuitive ones. We present users' subjective evaluations regarding these and present a 33-element taxonomy to categorize them. Furthermore, we present two user-defined gesture sets; the intuitive set including users' first choices and natural-feeling gestures, and the exclusive set which includes more creative gestures indigenous to this modality. Our findings can inspire and guide designers and developers of HSoS.

We observe an increasing interest on usage of full-body interaction in games. However, human-to-human social touch interaction has not been implemented as a sophisticated gaming apparatus. To address this, we designed the Sensation, a device for detecting touch patterns between players, and introduce the game, Shape Destroy, which is a collaborative game designed to be played with social touch. To understand if usage of social touch has a meaningful contribution to the overall player experience in collaborative games we conducted a user study with 30 participants. Participants played the same game using i) the Sensation and ii) a gamepad, and completed a set of questionnaires aimed at measuring the immersion levels. As a result, the collected data and our observations indicated an increase in general, shared, ludic and affective involvement with significant differences. Thus, human-to-human touch can be considered a promising control method for collaborative physical games.