At ByteDance, where we execute over a million Spark jobs and handle 500PB of shuffled data daily, ensuring resource efficiency is paramount for cost savings. However, achieving optimization of resource efficiency in large-scale production environments poses significant challenges. Drawing from our practical experiences, we have identified three key issues critical to addressing resource efficiency in real-world production settings: 1 slow I/Os leading to excessive CPU and memory idleness, 2 coarse-grained resource control causing wastage, and 3 sub-optimal job configurations resulting in low utilization. To tackle these issues, we propose a resource efficiency governance framework for Spark workloads. Specifically, 1 we devise the multi-mechanism shuffle services, including Enhanced External Shuffle Service (ESS) and Cloud Shuffle Service (CSS), where CSS employs a push-based approach to enhance I/O efficiency through sequential reading. 2 We modify the Spark configuration parameter protocol, allowing for fine-grained resource control by introducing several new parameters such as milliCores and memoryBurst, as well as supporting operators with additional spill modes. 3 We design a two-stage configuration autotuning method, comprising rule-based and algorithm-based tuning, providing more reliable Spark configuration optimizations. By deploying these techniques on millions of Spark jobs in production over the last two years, we have achieved over 22% CPU utilization increase, 5% memory utilization increase, and 10% shuffle block time ratio decrease, effectively saving millions of CPU cores and petabytes of memory daily.

Often, emotional disorders are overlooked due to their lack of awareness, resulting in potential mental issues. Recent advances in sensing and inference technology provide a viable path to wearable facial-expression-based emotion recognition. However, most prior work has explored only laboratory settings and few platforms are geared towards end-users in everyday lives or provide personalized emotional suggestions to promote self-regulation. We present EmoGlass, an end-to-end wearable platform that consists of emotion detection glasses and an accompanying mobile application. Our single-camera-mounted glasses can detect seven facial expressions based on partial face images. We conducted a three-day out-of-lab study (N=15) to evaluate the performance of EmoGlass. We iterated on the design of the EmoGlass application for effective self-monitoring and awareness of users’ daily emotional states. We report quantitative and qualitative findings, based on which we discuss design recommendations for future work on sensing and enhancing awareness of emotional health.

Repetitive tasks widely exist in applied fields of human-computer interaction. One underestimated example is parcel scanning, which has consistent operation difficulty but comprises multiple processes (e.g., label seeking and scanning, result confirming, and parcel relocating), involving respective cognitive requirements. Many devices are developed to facilitate repetitive operations, but few are to reduce fluctuating cognitive load throughout task processes. We present the eye-tracking augmented reality headset that integrates foveated vision detection and smooth pursuit of eye tracking and investigate how it can reduce cognitive load in the repetitive task. In total, 33 participants completed a set of parcel scanning tasks with the headset and their visual and cognitive performance were assessed. The results show that the headset maintained high scanning efficiency and lower cognitive load across the tasks with varying difficulties and it significantly reduced the participants’ cognitive load during the processes of barcode seeking and scanning and result confirmation. The headset demonstrated good usability and ease of use. Implications for how the case study result could be used in generalizing applications are discussed.