This paper introduces ASGaze, a novel gaze tracking system using the RGB camera of smartphones. ASGaze improves the accuracy of existing methods and uniquely tracks gaze points on various surfaces, including phone screens, computer displays, and non-electronic surfaces like whiteboards or paper - a situation that is challenging for existing methods. To achieve this, we revisit the 3D geometric eye model, commonly used in high-end commercial trackers, and it has the potential to achieve our goals. To avoid the high cost of commercial solutions, we identify three fundamental issues when processing the eye model with an RGB camera, including how to accurately extract iris boundary that is the meta-information in our design, how to remove ambiguity from iris boundary to gaze point transformation, and how to map gaze points onto the target surface. Furthermore, as we consider deploying ASGaze in real-world applications, two additional challenges should be addressed: how to automatically and accurately annotate the training dataset to reduce manual labor and time costs, and how to accelerate the inference speed of ASGaze on mobile devices to improve user experience. We propose effective techniques to resolve these issues. Our prototype and experiments on three tracking surfaces demonstrate significant performance gains.

Power governing is a critical component of modern mobile devices, reducing heat generation and extending device battery life. A popular technology of power governing is dynamic voltage and frequency scaling (DVFS), which adjusts the operating frequency of a processor to balance its performance and energy consumption. With the emergence of diverse workloads on mobile devices, traditional DVFS methods that do not consider workload characteristics become suboptimal. Recent application-oriented methods propose dedicated and effective DVFS governors for individual application tasks. Since their approach is only tailored to the targeted task, performance drops significantly when other tasks run concurrently, which is however common on today's mobile devices. In this paper, our key insight is that hardware meta-data, widely used in existing DVFS designs, has great potential to enable capable workload awareness and task concurrency adaptability for DVFS, but they are underexplored. We find that workload characteristics can be described in a hyperspace composed of multiple dimensions derived from these metadata to form a novel workload contextual indicator to profile task dynamics and concurrency. On this basis, we propose a meta-state metric to capture this relationship and design a new solution, GearDVFS. We evaluate it for a rich set of application tasks, and it outperforms state-of-the-art methods.

This paper introduces ASGaze, a new gaze tracking system designed using the common RGB camera from mobile phones. In addition to improving the accuracy of existing RGB camera-based gaze tracking methods, a novelty of ASGaze is that it can be configured to track gaze points on various surface areas commonly required in different applications, such as mobile phone screens, computer displays or even non-electronic surfaces like whiteboards or paper - a situation that is difficult for existing RGB camera-based methods to handle. To achieve the design of ASGaze, we revisit the 3D geometric model of the eye, which is widely adopted by high-end and commercial gaze trackers, and it has the potential to achieve our design goals. To avoid the high cost of commercial solutions, we identify three key issues to be addressed when processing the eye model with an RGB camera, including how to first accurately extract eye iris boundary that is the meta-information in our gaze tracking design, and then how to remove gaze ambiguity from iris boundary to gaze point transformation, and finally how to precisely map gaze points to the target tracking surface. In this paper, we propose a series of effective techniques to address these issues. We develop a prototype system and conduct extensive experiments on three different typical tracking surfaces to show promising performance gains compared to the recent solution.

We revisit the problem of iris tracking with RGB cameras, aiming to obtain iris contours from captured images of eyes. We find the reason that limits the performance of the state-of-the-art method in more general non-cooperative environments, which prohibits a wider adoption of this useful technique in practice. We believe that because the iris boundary could be inherently unclear and blocked, as its pixels occupy only an extremely limited percentage of those on the entire image of the eye, similar to the stars hidden in fireworks, we should not treat the boundary pixels as one class to conduct end-to-end recognition directly. Thus, we propose to learn features from iris and sclera regions first, and then leverage entropy to sketch the thin and sharp iris boundary pixels, where we can trace more precise parameterized iris contours. In this work, we also collect a new dataset by smartphone with 22 K images of eyes from video clips. We annotate a subset of 2 K images, so that label propagation can be applied to further enhance the system performance. Extensive experiments over both public and our own datasets show that our method outperforms the state-of-the-art method. The results also indicate that our method can improve the coarsely labeled data to enhance the iris contour’s accuracy and support the downstream application better than the prior method.

In this poster, we present our recent work, a wearable system for achieving real-time 3D arm skeleton. We have coped with the major challenge that the skeleton of each arm is determined from the locations of the elbow and wrist, whereas a wearable device only senses a single point from the wrist. Result shows that the potential solution space is huge. This underconstrained nature fundamentally challenges the achievement of accurate and real-time arm skeleton tracking. In this study, we propose Hidden Markov Model (HMM) state reorganization and hierarchical search two methods to improve the heavyweight computation of the state-of-art arm tracking model and achieve real-time tracking even on mobile phone.