We introduce the novel research problem of task recognition in daily life. We recognize tasks such as project management, planning, meal-breaks, communication, documentation, and family care. We capture Cyber, Physical, and Social (CPS) activities of 17 participants over four weeks using device-based sensing, app activity logging, and an experience sampling methodology. Our cohort includes students, casual workers, and professionals, forming the first real-world context-rich task behaviour dataset. We model CPS activities across different task categories, results highlight the importance of considering the CPS feature sets in modelling, especially work-related tasks.

One of the core challenges in open-plan workspaces is to ensure a good level of concentration for the workers while performing their tasks. Hence, being able to infer concentration levels of workers will allow building designers, managers, and workers to estimate what effect different open-plan layouts will have and to find an optimal one. In this article, we present an ambient-physical system to investigate the concentration inference problem. Specifically, we deploy a series of pervasive sensors to capture various ambient and physical signals related to perceived concentration at work. The practicality of our system has been tested on two large open-plan workplaces with different designs and layouts. The empirical results highlight promising applications of pervasive sensing in occupational concentration inference, which can be adopted to enhance the capabilities of modern workplaces.

Intelligent assistants can serve many purposes, including entertainment (e.g. playing music), home automation, and task management (e.g. timers, reminders). The role of these assistants is evolving to also support people engaged in work tasks, in workplaces and beyond. To design truly useful intelligent assistants for work, it is important to better understand the work tasks that people are performing. Based on a survey of 401 respondents' daily tasks and activities in a work setting, we present a classification of work-related tasks, and analyze their key characteristics, including the frequency of their self-reported tasks, the environment in which they undertake the tasks, and which, if any, electronic devices are used. We also investigate the cyber, physical, and social aspects of tasks. Finally, we reflect on how intelligent assistants could influence and help people in a work environment to complete their tasks, and synthesize our findings to provide insight on the future of intelligent assistants in support of amplifying personal productivity.

A fundamental challenge in real-time labelling of activity data is user burden. The Experience Sampling Method (ESM) is widely used to obtain such labels for sensor data. However, in an in-situ deployment, it is not feasible to expect users to precisely label the start and end time of each event or activity. For this reason, time-point based experience sampling (without an actual start and end time) is prevalent. We present a framework that applies multi-instance and semi-supervised learning techniques to perform to predict user annotations from multiple mobile sensor data streams. Our proposed framework estimates users' annotations in ESM-based studies progressively, via an interactive pipeline of co-training and active learning. We evaluate our work using data collected from an in-the-wild data collection.

In this paper, we focus on simultaneous inference of transportation modes and human activities in daily life via modelling and inference from multivariate time series data, which are streamed from off-the-shelf mobile sensors (e.g. embedded in smartphones) in real-world dynamic environments. The transportation mode will be inferred from the structured hierarchical contexts associated with human activities. Through our mobile context recognition system, an accurate and robust solution can be obtained to infer transportation mode, human activity and their associated contexts (e.g. whether the user is in moving or stationary environment) simultaneously. There are many challenges in analysing and modelling human mobility patterns within urban areas due to the ever-changing environments of mobile users. For instance, a user could stay at a particular location and then travel to various destinations depending on the tasks they carry within a day. Consequently, there is a need to reduce the reliance on location-based sensors (e.g. GPS), since they consume a significant amount of energy on smart devices, for the purpose of intelligent mobile sensing (i.e. automatic inference of transportation mode, human activity and associated contexts). Nevertheless, our system is capable of outperforming the simplistic approach that only considers independent classifications of multiple context label sets on data streamed from low-energy sensors.

In this paper, we focus on designing and developing ProMETheus, an intelligent system for meeting minutes generated from audio data. The first task in ProMETheus is to recognize the speakers from noisy audio data. Speaker recognition algorithm is used to automatically identify who is speaking according to the speech in an audio data. Naturally, speech recognition will transcribe speakers' audio to text so that ProMETheus can generate the complete meeting text with speakers' name chronologically. In order to show the subject of the meeting and the agreed action, we use text summarization algorithm that can extract meaningful key phrases and summary sentences from the complete meeting text. In addition, sentiment analysis for meeting text of different speakers can make the agreed action more humane due to calculating the relevance score of each course by the sentiment and attitude in text tone. The ProMETheus is capable of accurately summarizing the meeting and analyzing the agreed action. Our robust system is evaluated on a real-world audio meeting dataset that involves multiple speakers in each meeting session.

In this paper, we focus on developing a model and system for predicting the city foot traffic. We utilise historical records of pedestrian counts captured with thermal and laser-based sensors installed at multiple locations throughout the city. A robust prediction system is proposed to cope with various temporal foot traffic patterns. The empirical evaluation of our experiment shows that the proposed ARIMA model is effective in modelling both weekdays and weekend patterns, outperforming other state-of-art models for short-term prediction of pedestrian counts. The model is capable of accurately predicting pedestrian numbers up to 16 days in advance, on multiple look-ahead times. Our system is evaluated with a real-world sensor dataset supplied by the City of Melbourne.

In this paper, we present Ubiquitous data Exploration (UTE), a mobile sensor data collection, annotation and exploration platform. Our platform facilitates rapid prototyping of data mining experiments by using a flexible and do-it-yourself approach. The platform allows researchers to quickly design and deploy applications on mobile devices in order to record sensor data and the corresponding ground-truth information. The platform is supported by a web interface for designing data collection experiments, synchronizing and storing the sensor data with the corresponding labels, and sharing data.

Multi-activity recognition in the urban environment is a challenging task. This is largely attributed to the influence of urban dynamics, the variety of the label sets, and the heterogeneous nature of sensor data that arrive irregularly and at different rates. One of the first tasks in multi-activity recognition is temporal segmentation. A common temporal segmentation method is the sliding window approach with a fixed window size, which is widely used for single activity recognition. In order to recognise multiple activities from heterogeneous sensor streams, we propose a new time windowing technique that can optimally extract segments with multiple activity labels. The mixture of activity labels causes the impurity in the corresponding temporal segment. Hence, larger window size imposes higher impurity in temporal segments while increasing class separability. In addition, the combination of labels from multiple activity label sets (i.e. number of unique multi-activity) may decrease as impurity increases. Naturally, these factors will affect the performance of classification task. In our proposed technique, the optimal window size is found by gaining the balance between minimising impurity and maximising class separability in temporal segments. As a result, it accelerates the learning process for recognising multiple activities (such as higher level and atomic human activities under different environment contexts) in comparison to laborious tasks of sensitivity analysis. The evaluation was validated by experiments on a real-world dataset for recognising multiple human activities in a smart environment.

There is an emerging focus on real-time data stream analysis on mobile devices. A wide range of data stream processing applications are targeted to run on mobile handheld devices with limited computational capabilities such as patient monitoring, driver monitoring, providing real-time analysis and visualization for emergency and disaster management, real-time optimization for courier pick-up and delivery etc. There are many challenges in visualization of the analysis/data stream mining results on a mobile device. These include coping with the small screen real-estate and effective presentation of highly dynamic and real-time analysis. This paper proposes a generic theory for visualization on small screens that we term Adaptive Clutter Reduction ACR. Based on ACR, we have developed and experimentally validated a novel data stream clustering result visualization technique that we term Clutter-Aware Clustering Visualizer (CACV). Experimental results on both synthetic and real datasets using the Google Andriod platform are presented proving the effectiveness of the proposed techniques.