The infrastructure built in the City of Oulu provides rich information about the city environment and objects moving in it. We utilize this infrastructure in building an IoT system for data-intensive smart city services; by collecting data from real city environment and developing analysis methods for these data. We are building Smart City Traffic Pilot on top of the infrastructure to provide the functionality to collect the data and perform the analysis. Based on this experience, we present in this article requirements for data-intensive smart city services. Moreover, we describe four implemented use cases for utilizing rich data sources available in the smart city: situational picture, driving coach, real time reasoning, and mobile code. A lively collaboration between a large number of different actors is essential in realizing these use cases. Finally, we discuss how the use cases fulfill the requirements and the lessons we have learnt.

Demand for new efficient methods for processing large-scale heterogeneous data in real-time is growing. Currently, one key challenge in Big Data is performing low-latency analysis with real-time data. In vehicle traffic, continuous high speed data streams generate large data volumes. Harnessing new technologies is required to benefit from all the potential this data withholds. This work studies the state-of-the-art in distributed and parallel computing, storage, query and ingestion methods, and evaluates tools for periodical and real-time analysis of heterogeneous data. We also introduce a Big Data cloud platform with ingestion, analysis, storage and data query APIs to provide programmable environment for analytics system development and evaluation.

Better tools for content-based access of video are needed to improve access to time-continuous video data. Particularly information about linear TV broadcast programs has been available in a form limited to program guides that provide short manually described overviews of the program content. Recent development in digitalization of TV broadcasting and emergence of web-based services for catch-up and on-demand viewing bring out new possibilities to access data. In this paper we introduce our data mining system and accompanying services for summarizing Finnish DVB broadcast streams from seven national channels. We describe how data mining of novelty concepts can be extracted from DVB subtitles to augment web-based "Catch-Up TV Guide" and "Novelty Cloud" TV services. Furthermore, our system allows accessing media fragments as Picture Quotes via generated word lists and provides content-based recommendations to find new programs that have content similar to the user selected programs. Our index consists of over 180 000 programs that are used to recommend relevant programs. The service has been under development and available online since 2010. It has registered over 5000 user sessions.

Content-based test video corpora usually builds on top of professional material or controlled settings. However, recent years have shown strong increase in user-generated content on the web. The increase in content volume creates challenges in accessibility and utility of the video content. In order to improve the utility of the user-generated videos, better automation for content-based descriptions are needed. Proper test sets are required to develop robust methods for content analysis. Detecting people from video is a common feature that is often seen in both in science and in commercial services. Unfortunately there is a lack of test data for person tracking from consumer videos. In this paper, we introduce a novel video dataset to accommodate this shortage. The dataset is done with two consumer-priced devices: a handheld camcorder and a mobile phone. Both devices were used to store material in indoor and outdoor settings with different attention levels from the people being filmed. The dataset comes with ground truth data that includes person head trajectories and other people marked in the background in MPEG-7-based metadata model. We give description of this metadata model and publish an annotation tool we used for creating the ground truth data. Also, we provide experimental results as a benchmark for all those who would like to use our dataset.

This paper introduces a scalable solution for distributing content-based video analysis tasks using the emerging MapReduce programming model. Scalable and efficient solutions are needed for this type of tasks, as the number of multimedia content is growing at an increasing rate. We present a novel implementation utilizing the popular Apache Hadoop MapReduce framework for both analysis job scheduling and video data distribution. We employ face detection as a case example because it represents a popular visual content analysis task. The main contribution of this paper is the performance evaluation of distribution models for video content processing in various configurations. In our experiments, we have compared the performance of our video data distribution method against two alternatives solutions on a seven node cluster. Hadoop's performance overhead in video content analysis was also evaluated. We found Hadoop to be a data efficient solution with minimal computational overhead for the face detection task.

IP networks allow constructing versatile device configurations for multimedia streaming. However, the stochastic nature of the packet-switched data transmission may complicate IP-based implementations of some conventional applications such as analog wired transmission of synchronized multi-channel audio. This paper introduces a multimedia streaming system based on the synchronization of multiple playback clients as a dasiaswarmpsila. The proposed dasiaswarm synchronizationpsila mechanism is based on precise clock synchronization with the PTP protocol and adjusting the client-specific sampling rates according to the true playback rates of other clients. A streamlined version of the RTP protocol is employed to minimize playout delay. The proposed system is empirically evaluated in wired Ethernet LAN and in wireless IEEE 802.11g LAN. The experimental results show that in the Ethernet network the proposed streaming system is able to achieve very precise synchronization.

Peer-to-peer service-oriented community coordinated multimedia (SCCM) is envisioned as a novel paradigm in which the user consumes multiple media through requesting multimedia-intensive Web services via diversity display devices, converged networks, and heterogeneous platforms within a virtual, open and collaborative community in context of a novel concept of multimedia applications as Web services. A generic P2P SCCM scenario is created and examined first. The SCCM model is designed with respect to service architecture, a tunneled hierarchical P2P model and metamodel framework. The progress with the development of content annotation services is presented.

Community coordinated multimedia (CCM) envisions the paradigm of consuming multiple media via diversity display devices, converged networks, and heterogeneous platforms within a virtual, open and collaborative community. This paper applies service orientation approach and Web services technology to establish a Service-oriented CCM architecture (SCCM) for tackling requirements of scalability, discoverability, composibility, layered abstraction, QoS and agility in distributed collaborative multimedia management. A generic CCM scenario is examined. CCM service model is designed with service-orientation principles. A prototype of CCM is implemented with supporting multimedia publish, streaming, and viewing within SCCM.

Human experience is being extended and enhanced by collaboratively consuming electronic and networked content and multimedia-intensive services. This technical phenomenon is addressed by our generalized Community Coordinated Multimedia model in this paper. Content-driven and service-driven models are identified and a converged CCM model is introduced. Characteristics of Community Coordinated Multimedia are specified and challenges for developing a technological framework are presented.

Interactive experiments on video retrieval systems need to address the problem of internal validity, i.e. how much the test users' experience affects the retrieval effectiveness. This paper compares the semantic retrieval performance of novice users and expert system developers. The test system utilizes cluster-temporal browsing, which combines chronological video structure and computation of similarities into single interface. Interactive experiments with eight test users were carried out in a database of ~80 hours of multilingual news video from TRECVID 2005 benchmark. A cluster-temporal browser was found to improve the retrieval effectiveness by 12% with novice system users. Expert users were able to achieve 18% better performance than the novice users. Additionally, manual search experiments demonstrated that search performance can be improved by 19-25% when a plain text search is supplemented with content-based features

In this paper, we test statistically the effect of content-based browsing in generic video retrieval. Using TRECVID 2004 and 2005 experiments, we demonstrate that content-based browsing improves retrieval over sequential queries and relevance feedback. Two user groups, novices and system developers have been used in the experiments on large and multilingual video collections. Novice users were found to achieve improvement in search effectiveness with cluster-temporal browsing by statistically significant amount. System developers did not have statistically significant difference between the different system configurations.

This paper studies the performance sensitivity of the variations in different factors on SNoW and SVM based face detection techniques. These factors include combinations of training set, pixel transitions in sliding window, pre-processing methods (rotation, scaling, light correction, masking filter), and bootstrapping in negative training patterns. It was found that performance of the two detection methods is significantly sensitive to these factors.

This study describes experiments on automatic detection of semantic concepts, which are textual descriptions about the digital video content. The concepts can be further used in content-based categorization and access of digital video repositories. Temporal gradient correlograms, temporal color correlograms and motion activity low-level features are extracted from the dynamic visual content of a video shot. Semantic concepts are detected with an expeditious method that is based on the selection of small positive example sets and computational low-level feature similarities between video shots. Detectors using several feature and fusion operator configurations are tested in 60-hour news video database from TRECVID 2003 benchmark. Results show that the feature fusion based on ranked lists gives better detection performance than fusion of normalized low-level feature spaces distances. Best performance was obtained by pre-validating the configurations of features and rank fusion operators. Results also show that minimum rank fusion of temporal color and structure provides comparable performance

The paper describes cluster-temporal browsing of news video databases. Cluster-temporal browsing combines content similarities and temporal adjacency into a single representation. Visual, conceptual and lexical features are used to organize and view similar shot content. Interactive experiments with eight test users have been carried out using a database of roughly 60 hours of news video. Results indicate improvements in browsing efficiency when automatic speech recognition transcripts are incorporated into browsing by visual similarity. The cluster-temporal browsing application received positive comments from the test users and performed well in overall comparison with interactive video retrieval systems in TRECVID 2003 evaluation.