In human activity recognition (HAR), activity labels have typically been encoded in one-hot format, which has a recent shift towards using textual representations to provide contextual knowledge. Here, we argue that HAR should be anchored to physical motion data, as motion forms the basis of activity and applies effectively across sensing systems, whereas text is inherently limited. We propose SKELAR, a novel HAR framework that pretrains activity representations from skeleton data and matches them with heterogeneous HAR signals. Our method addresses two major challenges: (1) capturing core motion knowledge without context-specific details. We achieve this through a self-supervised coarse angle reconstruction task that recovers joint rotation angles, invariant to both users and deployments; (2) adapting the representations to downstream tasks with varying modalities and focuses. To address this, we introduce a self-attention matching module that dynamically prioritizes relevant body parts in a data-driven manner. Given the lack of corresponding labels in existing skeleton data, we establish MASD, a new HAR dataset with IMU, WiFi, and skeleton, collected from 20 subjects performing 27 activities. This is the first broadly applicable HAR dataset with time-synchronized data across three modalities. Experiments show that SKELAR achieves the state-of-the-art performance in both full-shot and few-shot settings. We also demonstrate that SKELAR can effectively leverage synthetic skeleton data to extend its use in scenarios without skeleton collections.

Web and mobile systems show constant distribution shifts due to the evolvement of services, users, and threats, severely degrading the performance of threat detection models trained on prior distributions. Fast model adaptation with minimal new data is essential for maintaining reliable security measures. A key challenge in this context is the lack of ground truth, which undermines the ability of existing solutions to align classes across shifted distributions. Moreover, the limited new data often fails to represent the underlying distribution, providing sparse and potentially noisy information for adaptation. In this paper, we propose REACT, a novel framework that adapts the model using a few unlabeled data and contextual insights. We leverage the inherent data imbalance in threat detection and meta-train weights on diverse unlabeled subsets to generalize common patterns across distributions, eliminating the reliance on labels for alignment. REACT decomposes a neural network into two complementary components: meta weights as a shared foundation of general knowledge, and residual adaptive weights as adjustments for specific shifts. To compensate for the limited availability of new data, REACT trains a hypernetwork to predict adaptive weights based on data and contextual information, enabling knowledge sharing across distributions. The meta weights and the hypernetwork are updated alternately, maximizing both generalization and adaptability. Extensive experiments across multiple datasets and models demonstrate that REACT improves AUROC by 14.85% over models without adaptation, outperforming the state-of-the-art.

Inferring contextual information such as demographics from historical transactions is valuable to public agencies and businesses. Existing methods are data-hungry and do not work well when the available records of transactions are sparse. We consider here specifically inference of demographic information using limited historical grocery transactions from a few random trips that a typical business or public service organization may see. We propose a novel method calledDemoMotifto build a network model from heterogeneous data and identify subgraph patterns (i.e., motifs) that enable us to infer demographic attributes. We then design a novel motif context selection algorithm to find specific node combinations significant to certain demographic groups. Finally, we learn representations of households using these selected motif instances as context, and employ a standard classifier (e.g., SVM) for inference. For evaluation purposes, we use three real-world consumer datasets, spanning different regions and time periods in the U.S. We evaluate the framework for predicting three attributes: ethnicity, seniority of household heads, and presence of children. Extensive experiments and case studies demonstrate thatDemoMotifis capable of inferring household demographics using only a small number (e.g., fewer than 10) of random grocery trips, significantly outperforming the state-of-the-art.

Chenyang An, Zhibo Chen, Qihao Ye, Emily First, Letian Peng, Jiayun Zhang, Zihan Wang, Sorin Lerner, Jingbo Shang. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2024.

Real-world deployment of federated learning requires orchestrating clients with widely varied compute resources, from strong enterprise-grade devices in data centers to weak mobile and Web-of-Things devices. Prior works have attempted to downscale large models for weak devices and aggregate shared parts among heterogeneous models. A typical architectural assumption is that there are equally many strong and weak devices. In reality, however, we often encounter resource skew where a few (1 or 2) strong devices hold substantial data resources, alongside many weak devices. This poses challenges-the unshared portion of the large model rarely receives updates or gains benefits from weak collaborators.

The temporal patterns of code submissions, denoted as work rhythms, provide valuable insight into the work habits and productivity in software development. In this paper, we investigate the work rhythms in software development and their effects on technical performance by analyzing the profiles of developers and projects from 110 international organizations and their commit activities on GitHub. Using clustering, we identify four work rhythms among individual developers and three work rhythms among software projects. Strong correlations are found between work rhythms and work regions, seniority, and collaboration roles. We then define practical measures for technical performance and examine the effects of different work rhythms on them. Our findings suggest that moderate overtime is related to good technical performance, whereas fixed office hours are associated with receiving less attention. Furthermore, we survey 92 developers to understand their experience with working overtime and the reasons behind it. The survey reveals that developers often work longer than required. A positive attitude towards extended working hours is associated with situations that require addressing unexpected issues or when clear incentives are provided. In addition to the insights from our quantitative and qualitative studies, this work sheds light on tangible measures for both software companies and individual developers to improve the recruitment process, project planning, and productivity assessment.

Human mobility trajectories are fundamental resources for analyzing mobile behaviors in urban computing applications. However, these trajectories, typically collected from location-based services, often suffer from sparsity and irregularity in time. To support the development of mobile applications, there is a need to recover or estimate missing locations of unobserved time slots in these trajectories at a fine-grained spatial-temporal resolution. Existing methods for trajectory recovery rely on either individual user trajectories or collective mobility patterns from all users. The potential to combine individual and collective patterns for precise trajectory recovery remains unexplored. Additionally, current methods are sensitive to the heterogeneous temporal distributions of the observable trajectory segments. In this paper, we propose CLMove (where CL stands for contrastive learning), a novel model designed to capture multilevel mobility patterns and enhance robustness in trajectory recovery. CLMove features a two-stage location encoder that captures collective and individual mobility patterns. The graph neural network based networks in CLMove explore location transition patterns within a single trajectory and across various user trajectories. We also design a trajectory-level contrastive learning task to improve the robustness of the model. Extensive experimental results on three representative real-world datasets demonstrate that our CLMove model consistently outperforms state-of-the-art methods in terms of trajectory recovery accuracy.

Current human activity recognition (HAR) techniques regard activity labels as integer class IDs without explicitly modeling the semantics of class labels. We observe that different activity names often have shared structures. For example, "open door" and "open fridge" both have "open" as the action; "kicking soccer ball" and "playing tennis ball" both have "ball" as the object. Such shared structures in label names can be translated to the similarity in sensory data and modeling common structures would help uncover knowledge across different activities, especially for activities with limited samples. In this paper, we propose SHARE, a HAR framework that takes into account shared structures of label names for different activities. To exploit the shared structures, SHARE comprises an encoder for extracting features from input sensory time series and a decoder for generating label names as a token sequence. We also propose three label augmentation techniques to help the model more effectively capture semantic structures across activities, including a basic token-level augmentation, and two enhanced embedding-level and sequence-level augmentations utilizing the capabilities of pre-trained models. SHARE outperforms state-of-the-art HAR models in extensive experiments on seven HAR benchmark datasets. We also evaluate in few-shot learning and label imbalance settings and observe even more significant performance gap.

The context about trips and users from mobility data is valuable for mobile service providers to understand their customers and improve their services. Existing inference methods require a large number of labels for training, which is hard to meet in practice. In this paper, we study a more practical yet challenging setting—contextual inference using mobility data with minimal supervision (i.e., a few labels per class and massive unlabeled data). A typical solution is to apply semi-supervised methods that follow a self-training framework to bootstrap a model based on all features. However, using a limited labeled set brings high risk of overfitting to self-training, leading to unsatisfactory performance. We propose a novel collaborative distillation framework STCOLAB. It sequentially trains spatial and temporal modules at each iteration following the supervision of ground-truth labels. In addition, it distills knowledge to the module being trained using the logits produced by the latest trained module of the other modality, thereby mutually calibrating the two modules and combining the knowledge from both modalities. Extensive experiments on two real-world datasets show STCOLAB achieves significantly more accurate contextual inference than various baselines.

Traditional federated classification methods, even those designed for non-IID clients, assume that each client annotates its local data with respect to the same universal class set. In this paper, we focus on a more general yet practical setting, non-identical client class sets, where clients focus on their own (different or even non-overlapping) class sets and seek a global model that works for the union of these classes. If one views classification as finding the best match between representations produced by data/label encoder, such heterogeneity in client class sets poses a new significant challenge-local encoders at different clients may operate in different and even independent latent spaces, making it hard to aggregate at the server. We propose a novel framework, FedAlign1, to align the latent spaces across clients from both label and data perspectives. From a label perspective, we leverage the expressive natural language class names as a common ground for label encoders to anchor class representations and guide the data encoder learning across clients. From a data perspective, during local training, we regard the global class representations as anchors and leverage the data points that are close/far enough to the anchors of locally-unaware classes to align the data encoders across clients. Our theoretical analysis of the generalization performance and extensive experiments on four real-world datasets of different tasks confirm that FedAlign outperforms various state-of-the-art (non-IID) federated classification methods.

Sensors measuring real-life physical processes are ubiquitous in today's interconnected world. These sensors inherently bear noise that often adversely affects the performance and reliability of the systems they support. Classic filtering approaches introduce strong assumption on the time or frequency characteristics of sensory measurements, while learning-based denoising approaches typically rely on using ground truth clean data to train a denoising model, which is often challenging or prohibitive to obtain for many real-world applications. We observe that in many scenarios, the relationships between different sensor measurements (e.g., location and acceleration) are analytically described by laws of physics (e.g., second-order differential equation). By incorporating such physics constraints, we can guide the denoising process to improve performance even in the absence of ground truth data. In light of this, we design a physics-informed denoising model that leverages the inherent algebraic relationships between different measurements governed by the underlying physics. By obviating the need for ground truth clean data, our method offers a practical denoising solution for real-world applications. We conducted experiments in various domains, including inertial navigation, CO2 monitoring, and HVAC control, and achieved state-of-the-art performance compared with existing denoising methods. Our method can denoise data in real time (4ms for a sequence of 1s) for low-cost noisy sensors and produces results that closely align with those from high-precision, high-cost alternatives, leading to an efficient, cost-effective approach for more accurate sensor-based systems.

Online developer communities like GitHub allow a massive number of developers to collaborate. However, the openness of the communities makes them vulnerable to different types of malicious attacks, since attackers can easily join these communities and interact with legitimate users. In this work, we propose GitSec, a deep learning-based solution for detecting malicious accounts in online developer communities. GitSec distinguishes malicious accounts from legitimate ones based on the account profiles, dynamic activity characteristics, as well as social interactions. First, GitSec introduces two user activity sequences and applies a parallel neural network design with an attention mechanism to process the sequences. Second, GitSec constructs two graphs to represent the interactions between users according to their repository operations. Especially, graph neural networks and structural hole theory are employed to deal with the two constructed graphs. Third, GitSec makes use of the descriptive features to enhance the detection performance. The final judgement is made by a decision maker implemented by a supervised machine learning-based classifier. Based on the real-world data of GitHub users, our comprehensive evaluations show that GitSec achieves a better performance than state-of-the-art solutions, with an AUC value of 0.916.

Visible light communication (VLC) based on unmanned aerial vehicles (UAVs) can simultaneously transmit data and lighting, which has been considered as a promising technology for the next generation wireless networks. In this paper, we construct a system consisting of UAV elements to fairly communicate with receiving plane. However, the unreasonable layout of UAVs may lead to the uneven distribution of the received optical power on the same receiving plane, which cannot guarantee the fairness of the communication between the UAVs and receiving plane. Besides, the transmission power of the UAVs has direct effects on the strength of the received optical signal in VLC communication. Therefore, we formulate an optical power coverage optimization problem (OPCOP) to achieve more uniform received optical power coverage by jointly considering the positions and appropriate power adjustment factor of transmission power of UAVs. Then, an improved cuckoo search with c haotic solution initialization operation and m utation mechanisms (ICSCM) algorithm is proposed to solve the formulated optimization problem. ICSCM introduces the chaotic solution initialization operation for increasing the performance of initial solution, and employs two mutation mechanisms, which are mutation operator of differential evolution (DE) algorithm and Gaussian perturbation to enhance the exploration ability of conventional cuckoo search (CS). Simulations are conducted and the results verify that the received optical power of receivers distributed on the same receiving plane obtained by ICSCM can be more uniform than other comparison methods.

Due to more flexible mobility, better line-of-sight (LOS) and faster on-demand deployment, unmanned aerial vehicles (UAVs) play a unique role for assisting post-disaster rescues, which often require UAVs to perform computationintensive rescue missions. However, UAVs generally have inherent limited computational capacity and battery storage, which makes it challenging to complete the heavy computing tasks within short period of time during the complicated postdisaster recovery. To overcome this issue, we introduce the vehicular fog computing (VFC) system in which a UAV splits and assigns the heavy tasks to the ground vehicles. First, to evaluate the performance of the VFC-assisted UAV network task offloading, the task processing latency and energy consumption are incorporated into a system utility construction. Moreover, we propose a joint UAV and vehicular task assignment scheme (JUVTAS) with the aim of optimizing the performance of the network. Specifically, we propose a genetic algorithminvasive weed optimization (GA-IWO) algorithm to achieve the approximately optimal task assignment strategy. The GA-IWO algorithm combines the global search ability of genetic algorithm and the local search ability of invasive weed optimization to achieve a better optimization performance. Simulation results show that the proposed JUVTAS is able to effectively reduce the latency and energy consumption for task processing. Moreover, JUVTAS achieves superior performance compared to several conventional methods.

As one of the biggest online developer communities, GitHub supports interactions between millions of developers around the world. This paper focuses on detecting and analyzing the interactions on GitHub from several perspectives. First, from a global viewpoint, we build an interaction graph based on GitHub Events to investigate the general structure and attributes of the GitHub interaction network. Second, from the perspective of important users on GitHub, we pay particular attention to those who bridge social circles by spanning across communities. Concretely, we apply the structural hole theory to identify and explore the structural hole spanners in this interaction network. Last but not least, we compare structural hole spanners with ordinary users in several aspects and excavate distinctions between them. To our best knowledge, this is the first study that applies structural hole theory to the interaction graph of a leading social network. It not only provides a unique standpoint on user interactions, but also gives a comprehensive inspection of structural hole spanners.

Online developer communities like GitHub provide services such as distributed version control and task management, which allow a massive number of developers to collaborate online. However, the openness of the communities makes themselves vulnerable to different types of malicious attacks, since the attackers can easily join and interact with legitimate users. In this work, we formulate the malicious account detection problem in online developer communities, and propose GitSec, a deep learning-based solution to detect malicious accounts. GitSec distinguishes malicious accounts from legitimate ones based on the account profiles as well as dynamic activity characteristics. On one hand, GitSec makes use of users' descriptive features from the profiles. On the other hand, GitSec processes users' dynamic behavioral data by constructing two user activity sequences and applying a parallel neural network design to deal with each of them, respectively. An attention mechanism is used to integrate the information generated by the parallel neural networks. The final judgement is made by a decision maker implemented by a supervised machine learning-based classifier. Based on the real-world data of GitHub users, our extensive evaluations show that GitSec is an accurate detection system, with an F1-score of 0.922 and an AUC value of 0.940.