Expressive digital drawing requires nuanced motor control, subtle variations in pressure, velocity, and rhythm that convey affect and style. While experts develop this embodied fluency through years of practice, novices struggle to produce marks that match their intentions, creating a gap between vision and execution. We propose motor-mediated creativity: treating motor training as integral to digital expression. Our system, Motus, instantiates this through structured practice of expressive primitives, expert-referenced feedback, and ideation prompts that encourage exploration. We report a two-stage investigation. A formative study characterized: (a) novice challenges in motor fluency, (b) examined how different feedback types, including corrective feedback, helped participants understand their mistakes, (c) how prompts, generic or embodied, support engagement with abstract expressive content. A controlled evaluation then linked fluency gains to subjective and expert ratings of expressiveness. Together, our findings show that scaffolding motor skills is a viable strategy for enhancing expressive agency in digital drawing.

Lifelog images captured automatically by wearable cam-eras serve as effective cues that induce Autobiographic Memory Recall (AMR) of social interactions. This is very useful for personalized memory interventions. However, manual selection of images for such therapy imposes significant load on the caregivers who need to browse through a voluminous collection of images. To reduce this load, auto-mated tools that identify moments involving significant engagement of the camera wearer in social interactions are needed. To achieve this, we reannotate images extracted from public lifelog datasets for the presence of non-verbal social signals and the perceived engagement of the lifelogger during interactions. We use this data to develop models and explore how social signals and the detected intensity of social engagement are helpful for predicting AMR. We show that understanding visual social engagement can enhance AMR prediction, demonstrating the potential of the models in reducing caregivers’ effort.

Accurate swimming performance monitoring has traditionally relied on wearable sensors, which can disrupt natural technique and are impractical in competitive settings. In this paper, we present a fully vision-based system for automatic swimmer analysis using overhead drone footage, removing the need for any wearable device or underwater equipment. By fine-tuning pose estimation models for aerial aquatic conditions, our approach robustly extracts full-body swimmer skeletons even under challenging scenarios such as splashes and partial occlusions. From these poses, we classify swimming strokes, compute instantaneous speed, estimate lap times, and count individual strokes. Unlike existing methods, our system provides scalable, unobtrusive, and infrastructure-free tracking. Evaluated on real-world drone-captured swimming competition data, our method achieves a median speed estimation error below 4% (under 0.05 m/s), a median lap time error of just 0.03s, and stroke count errors typically under one stroke per lap.

Continuous tracking of eye movement dynamics plays a significant role in developing a broad spectrum of human-centered applications, such as cognitive skills (visual attention and working memory) modeling, human-machine interaction, biometric user authentication, and foveated rendering. Recently neuromorphic cameras have garnered significant interest in the eye-tracking research community, owing to their sub-microsecond latency in capturing intensity changes resulting from eye movements. Nevertheless, the existing approaches for event-based eye tracking suffer from several limitations: dependence on RGB frames, label sparsity, and training on datasets collected in controlled lab environments that do not adequately reflect real-world scenarios. To address these limitations, in this paper, we propose a dynamic graph-based approach that uses a neuromorphic event stream captured by Dynamic Vision Sensors (DVS) for high-fidelity tracking of pupillary movement. More specifically, first, we present EyeGraph, a large-scale multi-modal near-eye tracking dataset collected using a wearable event camera attached to a head-mounted device from 40 participants -- the dataset was curated while mimicking in-the-wild settings, accounting for varying mobility and ambient lighting conditions. Subsequently, to address the issue of label sparsity, we adopt an unsupervised topology-aware approach as a benchmark. To be specific, (a) we first construct a dynamic graph using Gaussian Mixture Models (GMM), resulting in a uniform and detailed representation of eye morphology features, facilitating accurate modeling of pupil and iris. Then (b) apply a novel topologically guided modularity-aware graph clustering approach to precisely track the movement of the pupil and address the label sparsity in event-based eye tracking. We show that our unsupervised approach has comparable performance against the supervised approaches while consistently outperforming the conventional clustering approaches.

The use of social media has made it easy to communicate and share information over the internet. However, it also brings issues such as data privacy leakage, which can be exploited by recipients with malicious intentions to harm the sender. In this paper, we propose a deep neural network that analyzes user’s image for privacy sensitive content and automatically locates sensitive regions for obfuscation. Our approach relies solely on image level annotations and learns to (a) predict an overall privacy score, (b) detect sensitive attributes and (c) demarcate the sensitive regions for obfuscation, in a given input image. We validated the performance of our proposed method on three large datasets, VISPR, PASCAL VOC 2012 and MS COCO 2014, in terms of privacy score, attribute prediction and obfuscation performance. On the VISPR dataset, we achieved a Pearson correlation of 0.88 and a Spearman correlation of 0.86, outperforming previous methods. On PASCAL VOC 2012 and MS COCO 2014, our model achieved a mean IOU of 71.5% and 43.9% respectively, and is among the state-of-the-art techniques using weakly supervised semantic segmentation learning.

In this demonstration, we present a fully data driven solution to retrieve passengers’ actual paths within a metro system that are not captured by an Automated Fare Collection (AFC) system. The majority of public transit systems employ AFC systems with smart cards, which record the exact origin, destination, admission time, and exit time of each passenger’s metro trip. Our solution uses AFC data to first infer travel times and route preferences and then estimates the passengers’ travel paths for all trips to provide a statistical view of passengers’ crowdedness inside a metro network over time.

Passenger satisfaction is extremely important for the success of a public transportation system. Many studies have shown that passenger satisfaction strongly depends on the time they have to wait at the bus stop (waiting time) to get on a bus. To be specific, user satisfaction drops faster as the waiting time increases. Therefore, service providers want to provide a bus to the waiting passengers within a threshold to keep them satisfied. It is a two-pronged problem: (a) to satisfy more passengers the transport planner may increase the frequency of the buses, and (b) in turn, the increased frequency may impact the service operational costs. To address it, we propose PASS and COST as the two variants that satisfy different optimization criteria mentioned above. The optimization goal of PASS is the number of satisfied passengers while the optimization goal of COST is the number of passengers served per unit of driving time. Consequently, PASS utilizes resources to the maximum to satisfy the highest number of passengers, while COST optimizes for both passenger satisfaction and operational costs. Accordingly, we propose two algorithms to solve PASS and COST respectively and evaluate their performance based on real passenger demand data-set.

Built-in pervasive cameras have become an integral part of mobile/wearable devices and enabled a wide range of ubiquitous applications with their ability to be "always-on". In particular, life-logging has been identified as a means to enhance the quality of life of older adults by allowing them to reminisce about their own life experiences. However, the sensitive images captured by the cameras threaten individuals' right to have private social lives and raise concerns about privacy and security in the physical world. This threat gets worse when image recognition technologies can link images to people, scenes, and objects, hence, implicitly and unexpectedly reveal more sensitive information such as social connections. In this paper, we first examine life-log images obtained from 54 older adults to extract (a) the artifacts or visual cues, and (b) the context of the image that influences an older life-logger's ability to recall the life events associated with a life-log image. We call these artifacts and contextual cues "stimuli". Using the set of stimuli extracted, we then propose a set of obfuscation strategies that naturally balances the trade-off between reminiscability and privacy (revealing social ties) while selectively obfuscating parts of the images. More specifically, our platform yields privacy-utility tradeoff by compromising, on average, modest 13.4% reminiscability scores while significantly improving privacy guarantees -- around 40% error in cloud estimation.

In mobile crowd-sourcing systems, simply relying on people to opportunistically select and perform tasks typically leads to drawbacks such as low task acceptance/completion rates and undesirable spatial skews. In this paper, we utilize data from TASKer, a campus-based mobile crowd-sourcing platform, to empirically study and discover whether and how various context-aware notification strategies can help overcome such drawbacks. We first study worker interactions, in the absence of any notifications, to discover some spatio-temporal properties of task acceptance and completion. Based on these insights, we then experimentally demonstrate the effectiveness of two novel, non-personal, context-driven notification strategies, comparing the outcomes to two different baselines (no-notification and random-notification). Finally, using the data from the random-notification mechanism, we derive a classification model, incorporating several novel contextual features, that can predict a worker's responsiveness to notifications with high accuracy. Our work extends the crowd-sourcing literature by emphasizing the power of smart notifications for greater worker engagement.

Visual privacy concerns associated with image sharing is a critical issue that need to be addressed to enable safe and lawful use of online social platforms. Users of social media platforms often suffer from no guidance in sharing sensitive images in public, and often face with social and legal consequences. Given the recent success of visual attention based deep learning methods in measuring abstract phenomena like image memorability, we are motivated to investigate whether visual attention based methods could be useful in measuring psychophysical phenomena like “privacy sensitivity”. In this paper we propose PrivAttNet - a visual attention based approach, that can be trained end-to-end to estimate the privacy sensitivity of images without explicitly detecting sensitive objects and attributes present in the image. We show that our PrivAttNet model outperforms various SOTA and baseline strategies - a 1.6 fold reduction in L1 - error over SOTA and 7%-10% improvement in Spearman-rank correlation between the predicted and ground truth sensitivity scores. Additionally, the attention maps from PrivAttNet are found to be useful in directing the users to the regions that are responsible for generating the privacy risk score.

While analysis of urban commuting data has a long and demonstrated history of providing useful insights into human mobility behavior, such analysis has been performed largely in offline fashion and to aid medium-to-long term urban planning. In this work, we demonstrate the power of applying predictive analytics on real-time mobility data, specifically the smart-card generated trip data of millions of public bus commuters in Singapore, to create two novel and "live" smart city services. The key analytical novelty in our work lies in combining two aspects of urban mobility: (a) conformity: which reflects the predictability in the aggregated flow of commuters along bus routes, and (b) regularity: which captures the repeated trip patterns of each individual commuter. We demonstrate that the fusion of these two measures of behavior can be performed at city-scale using our BuScope platform, and can be used to create two innovative smart city applications. The Last-Mile Demand Generator provides O(mins) lookahead into the number of disembarking passengers at neighborhood bus stops; it achieves over 85% accuracy in predicting such disembarkations by an ingenious combination of individual-level regularity with aggregate-level conformity. By moving driverless vehicles proactively to match this predicted demand, we can reduce wait times for disembarking passengers by over 75%. Independently, the Neighborhood Event Detector uses outlier measures of currently operating buses to detect and spatiotemporally localize dynamic urban events, as much as 1.5 hours in advance, with a localization error of ~450 meters.

In this article, we investigate effective ways of utilizing crowdworkers in providing various urban services. The task recommendation platform that we design can match tasks to crowdworkers based on workers’ historical trajectories and time budget limits, thus making recommendations personal and efficient. One major challenge we manage to address is the handling of crowdworker’s trajectory uncertainties. In this article, we explicitly allow multiple routine routes to be probabilistically associated with each worker. We formulate this problem as an integer linear program whose goal is to maximize the expected total utility achieved by all workers. We further exploit the separable structures of the formulation and apply the Lagrangian relaxation technique to scale up computation. Numerical experiments have been performed over the instances generated using the realistic public transit dataset in Singapore. The results show that we can find significantly better solutions than the deterministic formulation, and in most cases we can find solutions that are very close to the theoretical performance limit. To demonstrate the practicality of our approach, we deployed our recommendation engine to a campus-scale field trial, and we demonstrate that workers receiving our recommendations incur fewer detours and complete more tasks, and are more efficient against workers relying on their own planning (25% more for top workers who receive recommendations). This is achieved despite having highly uncertain worker trajectories. We also demonstrate how to further improve the robustness of the system by using a simple multi-coverage mechanism.

This paper establishes the power of dynamic collaborative task completion among workers for urban mobile crowd-sourcing. Collaboration is defined via the notion of peer referrals, whereby a worker who has accepted a location-specific task, but is unlikely to visit that location, offloads the task to a willing friend. Such a collaborative framework might be particularly useful for task bundles, especially for bundles that have higher geographic dispersion. The challenge, however, comes from the high similarity observed in the spatio-temporal pattern of task completion among friends. Using extensive real-world crowd-sourcing studies conducted over 7 weeks and 1000+ workers on a campus-based crowd-sourcing platform, we quantify the effect of such "task completion homophily", and show that incorporating such peer-preferences can improve worker-specific models of task preferences by over 30%. We then show that such collaborative offloading works in spite of such spatio-temporal similarity, primarily because workers refer tasks to their close friends, who in turn perform such peer-requested tasks (with over 95% completion rate) even if they experience detours that are significantly larger (often more than twice) than what they normally tolerate for platform-recommended tasks.

Mobile crowd-tasking markets are growing at an unprecedented rate with increasing number of smartphone users. Such platforms differ from their online counterparts in that they demand physical mobility and can benefit from smartphone processors and sensors for verification purposes. Despite the importance of such mobile crowd-tasking markets, little is known about the labor supply dynamics and mobility patterns of the users.

While mobile crowd-sourcing has become a game-changer for many urban operations, such as last mile logistics and municipal monitoring, we believe that the design of such crowd-sourcing strategies must better accommodate the real-world behavioral preferences and characteristics of users. To provide a real-world testbed to study the impact of novel mobile crowd-sourcing strategies, we have designed, developed and experimented with a real-world mobile crowd-tasking platform on the SMU campus, called TA&Sslash;Ker. We enhanced the TA$Ker platform to support several new features (e.g., task bundling, differential pricing and cheating analytics) and experimentally investigated these features via a two-month deployment of TA$Ker, involving 900 real users on the SMU campus who performed over 30,000 tasks. Our studies (i) show the benefits of bundling tasks as a combined package, (ii) reveal the effectiveness of differential pricing strategies and (iii) illustrate key aspects of cheating (false reporting) behavior observed among workers.

Recommender systems are widely used by online retailers to promote products and content that are most likely to be of interest to a specific customer. In such systems, users often implicitly or explicitly rate products they have consumed, and some form of collaborative filtering is used to find other users with similar tastes to whom the products can be recommended. While users can benefit from more targeted and relevant recommendations, they are also exposed to greater risks of privacy loss, which can lead to undesirable financial and social consequences. The use of obfuscation techniques to preserve the privacy of user ratings is well studied in the literature. However, works on obfuscation typically assume that all users uniformly apply the same level of obfuscation. In a heterogeneous environment, in which users adopt different levels of obfuscation based on their comfort level, the different levels of obfuscation may impact the users in the system in a different way. In this work we consider such a situation and make the following contributions: (a) using an offline dataset, we evaluate the privacy-utility trade-off in a system where a varying portion of users adopt the privacy preserving technique. Our study highlights the effects that each user's choices have, not only on their own experience but also on the utility that other users will gain from the system, and (b) we propose Privacy Canary, an interactive system that enables users to directly control the privacy-utility trade-off of the recommender system to achieve a desired accuracy while maximizing privacy protection, by probing the system via a private (i.e., undisclosed to the system) set of items. We evaluate the performance of our system with an off-line recommendations dataset, and show its effectiveness in balancing a target recommender accuracy with user privacy, compared to approaches that focus on a fixed privacy level.

We consider stored content being streamed to a resource-poor device (such as a sensor node or a mobile phone), and address the issue of authenticating such content in realtime at the receiver. Per-packet digital signatures incur high computational cost, while per-block signatures impose high delays. A Merkle hash tree combines the benefits of the two by having a single signature per-block (at the root of the tree), while allowing immediate per-packet verification by following a hash-path logarithmic in the number of packets. In this paper we explore how the structure of the Merkle tree can be adapted to improve playback performance for streaming content. We make three specific contributions: First, we develop a new unbalanced authentication tree structure called the α-leaf tree that is a generalisation of the Merkle tree. We derive several key properties of this tree, highlighting the impact of the imbalance parameter α. Second, we present a theoretical model to quantify the benefits of our unbalanced tree structure in reducing startup delays for streaming applications by optimally readjusting the burden of authentication across packets. Third, we validate via simulation the suitability of our scheme to two representative applications, namely audio streaming to a low-cost sensor device and video streaming to a mobile phone, and demonstrate that startup delays can be reduced without affecting stall rates. We believe our authentication tree structure is of importance both theoretically, as a generalisation of the Merkle hash tree, as well as practically, for applications requiring real-time verification of streaming content.