Modern courses require students to use multiple digital learning platforms, but traditional learning analytics often focuses on single, specific platforms. Thus, there is a limited understanding of how students integrate information across sources. We present an alternative approach using a browser extension, which collects behavior information across multiple web-based learning platforms simultaneously. We tested the extension in a college-level introductory statistics course with 27 students over 15 weeks. Using the information collected by the extension, we found that students navigate multi-platform learning environments in diverse ways, which can inform curriculum design and the effectiveness of learning platforms.

Large language models (LLMs) have enhanced conventional recommendation models via user profiling, which generates representative textual profiles from users' historical interactions. However, their direct application to session-based recommendation (SBR) remains challenging due to severe session context scarcity and poor scalability. In this paper, we propose SPRINT, a scalable SBR framework that incorporates reliable and informative intents while ensuring high efficiency in both training and inference. SPRINT constrains LLM-based profiling with a global intent pool and validates inferred intents based on recommendation performance to mitigate noise and hallucinations under limited context. To ensure scalability, LLMs are selectively invoked only for uncertain sessions during training, while a lightweight intent predictor generalizes intent prediction to all sessions without LLM dependency at inference time. Experiments on real-world datasets show that SPRINT consistently outperforms state-of-the-art methods while providing more explainable recommendations.

Recent advances in generative recommenders adopt a two-stage paradigm: items are first tokenized into semantic IDs using a pretrained tokenizer, and then large language models (LLMs) are trained to generate the next item via sequence-to-sequence modeling. However, these two stages are optimized for different objectives: semantic reconstruction during tokenizer pretraining versus user interaction modeling during recommender training. This objective misalignment leads to two key limitations: (i) suboptimal static tokenization, where fixed token assignments fail to reflect diverse usage contexts; and (ii) discarded pretrained semantics, where pretrained knowledge—typically from language model embeddings—is overwritten during recommender training on user interactions. To address these limitations, we propose to learn DEcomposed COntextual Token Representations (DECOR), a unified framework that preserves pretrained semantics while enhancing the adaptability of token embeddings. DECOR introduces contextualized token composition to refine token embeddings based on user interaction context, and decomposed embedding fusion that integrates pretrained codebook embeddings with newly learned collaborative embeddings. Experiments on three real-world datasets demonstrate that DECOR consistently outperforms state-of-the-art baselines in recommendation performance. Our code is available at https://github.com/yliuaa/DECOR.git.

The growing prevalence of false information originating from human–LLM mixed generation sources presents new challenges for veracity assessment, as different generation sources (e.g., human or LLM) exhibit distinct semantic patterns and retrieval needs. In particular, LLM-generated false information is often more fluent, persuasive, and difficult to detect than traditional human-written content. Existing retrieval-augmented generation (RAG) methods for veracity assessment often use a uniform retrieval strategy for all inputs, limiting their flexibility in adapting to varying content characteristics. Current adaptive RAG methods primarily focus on general input difficulty or model confidence, overlooking the distinct semantic characteristics and retrieval needs of different generation sources. We propose MoARE, a Mixture of Adaptive Retrieval Experts framework that dynamically determines whether to retrieve and how much evidence to retrieve for each post. MoARE leverages a Mixture-of-Experts (MoE) network trained via reinforcement learning to balance veracity assessment accuracy and time cost—without requiring knowledge of the input's generation source. Experiments on recent human–LLM mixed datasets demonstrate that MoARE outperforms state-of-the-art RAG baselines, achieving higher accuracy with lower time cost.

Media bias significantly shapes public perception by reinforcing stereotypes and exacerbating societal divisions. Prior research has often focused on isolated media bias dimensions such as political bias or racial bias , neglecting the complex interrelationships among various bias dimensions across different topic domains. Moreover, models trained on existing media bias benchmarks exhibit substantial performance degradation when applied to recent social media content. This shortfall primarily arises because these benchmarks do not adequately reflect the rapidly evolving nature of social media content, which is characterized by shifting user behaviors and emerging trends. To this end, we introduce a new dataset collected from YouTube and Reddit over the past five years. To efficiently and reliably annotate our collected dataset, we propose U ncertainty E stimation-guided A nnotation R efinement ( UEAR ), a novel human-LLM collaborative annotation scheme. UEAR estimates the uncertainties of LLM-based annotations through a two-way reasoning process and refines uncertain annotations from LLM with human inputs. Our final dataset contains human-LLM joint annotations of YouTube and Reddit content across multiple bias dimensions (e.g., gender, racial) and diverse topic domains (e.g., politics, sports), capturing the complex interplay of biases across societal sectors. We evaluate the annotation performance of UEAR on a manually annotated test set, demonstrating that UEAR can reliably generate multidimensional bias identification annotations with a 0.9082 macro F1 score, with limited human refinement. Through our statistical analysis of the generated annotations, we identify significant differences in bias expression patterns and intra-domain bias correlations across different domains. The code and data are made publicly available.

Drought has become a critical global threat with significant societal impact. Existing drought monitoring solutions primarily focus on assessing drought severity using quantitative measurements, overlooking the diverse societal impact of drought from human-centric perspectives. Motivated by the collective intelligence on social media and the computational power of AI, this paper studies a novel problem of socially informed AI-driven drought estimation that aims to leverage social and news media information to jointly estimate drought severity and its societal impact. Two technical challenges exist: 1) How to model the implicit temporal dynamics of drought societal impact. 2) How to capture the social-physical interdependence between the physical drought condition and its societal impact. To address these challenges, we develop SIDE, a socially informed AI-driven drought estimation framework that explicitly quantifies the societal impact of drought and effectively models the social-physical interdependency for joint severity-impact estimation. Experiments on real-world datasets from California and Texas demonstrate SIDE's superior performance compared to state-of-the-art baselines in accurately estimating drought severity and its societal impact. SIDE offers valuable insights for developing human-centric drought mitigation strategies to foster sustainable and resilient communities.

Students struggle with accurately assessing their own performance, especially given little training to do so.We propose an AI-powered training tool to help students improve "metacognitive calibration, " or the ability to accurately predict their own learning, potentially enhancing learning outcomes by enabling students' use of metacognitioninformed learning behaviors.We present results from a randomized controlled trial (N = 133) assessing the effectiveness of the tool in a college-level computer-based learning environment.The AIdriven tool significantly improved learning gains compared to the control group by 8.9% (t = -2.384,p = .019),and this effect was significantly mediated by learning behaviors.Overconfident students who received the intervention showed significantly greater metacognitive calibration improvement than the control group by 4.1% (t = 2.001, p = .049).These insights highlight the value of AIpowered metacognitive calibration training and the importance of promoting specific metacognition-informed learning behaviors in computer-based learning.

With the advancement of personalized image generation technologies, concerns about forgery attacks that infringe on portrait rights and privacy are growing. To address these concerns, protection perturbation algorithms have been developed to disrupt forgery generation. However, the protection algorithms would become ineffective when forgery attackers apply purification techniques to bypass the protection. To address this issue, we present a novel approach, Anti-Tamper Perturbation (ATP). ATP introduces a tamper-proof mechanism within the perturbation. It consists of protection and authorization perturbations, where the protection perturbation defends against forgery attacks, while the authorization perturbation detects purification-based tampering. Both protection and authorization perturbations are applied in the frequency domain under the guidance of a mask, ensuring that the protection perturbation does not disrupt the authorization perturbation. This design also enables the authorization perturbation to be distributed across all image pixels, preserving its sensitivity to purification-based tampering. ATP demonstrates its effectiveness in defending forgery attacks across various attack settings through extensive experiments, providing a robust solution for protecting individuals' portrait rights and privacy. Our code is available at: https://github.com/Seeyn/Anti-Tamper-Perturbation .

The scaling of inference computation has unlocked the potential of long-context large language models (LLMs) across diverse settings. For knowledge-intensive tasks, the increased compute is often allocated to incorporate more external knowledge. However, without effectively utilizing such knowledge, solely expanding context does not always enhance performance. In this work, we investigate inference scaling for retrieval augmented generation (RAG), exploring the combination of multiple strategies beyond simply increasing the quantity of knowledge, including in-context learning and iterative prompting. These strategies provide additional flexibility to scale test-time computation (e.g., by increasing retrieved documents or generation steps), thereby enhancing LLMs’ ability to effectively acquire and utilize contextual information. We address two key questions: (1) How does RAG performance benefit from the scaling of inference computation when optimally configured? (2) Can we predict the optimal test-time compute allocation for a given budget by modeling the relationship between RAG performance and inference parameters? Our observations reveal that increasing inference computation leads to nearly linear gains in RAG performance when optimally allocated, a relationship we describe as the inference scaling laws for RAG. Building on this, we further develop the computation allocation model to estimate RAG performance across different inference configurations. The model predicts optimal inference parameters under various computation constraints, which align closely with the experimental results. By applying these optimal configurations, we demonstrate that scaling inference compute on long-context LLMs achieves up to 58.9% gains on benchmark datasets compared to standard RAG.

This paper studies a critical problem of explainable public health misinformation detection on social media, where clear explanations are essential for enhancing user understanding and trust, surpassing the limitations of black-box misinformation detection results. To tackle this problem, there is a growing trend of leveraging collective intelligence from diverse intelligence sources, such as deep neural networks (DNNs), human intelligence, and large language models (LLMs). However, integrating hybrid intelligence from different sources remains a challenge: DNNs excel in accurate and efficient classification, crowd workers provide contextual understanding and readable explanations, and LLMs offer extensive domain knowledge and advanced language generation. Moreover, current crowdsourcing and human-AI collaboration methods mainly focus on aggregating misinformation detection labels using traditional measures like consistency, often overlooking more complex and challenging inputs like textual explanations. We propose SynthX, a collective intelligence framework that incorporates a holistic prompting design to harness the language and reasoning capabilities of LLMs for synthesizing diverse detection and explanation results. It also integrates a novel estimation theory-LLM hybrid approach to assess the varying reliability of detection results from different intelligence sources. Our evaluation on a real-world social media misinformation dataset demonstrates that SynthX consistently outperforms a rich set of state-of-the-art baselines in both detection accuracy and explanation quality.

This paper studies a bidirectional human-AI collaborative student performance prediction problem to enhance equitable online education, aligning with the United Nations' Sustainable Development Goal (SDG) of ensuring inclusive and equitable quality education for all. The goal is to leverage collaborative intelligence to generate accurate and fair student outcome predictions from behavioral data, ensuring equitable estimation for underrepresented populations. Current fair AI solutions often fail to mitigate demographic bias in the absence of student demographic data, while human-AI collaborative approaches frequently overlook human cognitive biases, leading to inaccurate predictions. We develop CollabDebias, a novel bidirectional human-AI collaborative framework that utilizes the complementary strengths of AI and humans to mitigate the AI demographic bias and human cognitive bias. To address AI demographic bias, we propose an uncertainty learning-based bias identification method and a reliability-aware human-AI integration approach. To reduce human cognitive bias, we design uncertainty-aware visualization of AI decision area and attention mechanism. Experimental results on an online course demonstrate CollabDebias's effectiveness in improving student performance prediction accuracy and fairness.

Large multimodal models (LMMs) exhibit enhanced capabilities in understanding and generating both textual and visual content. By leveraging item metadata, LMMs are also applied for recommendation and demonstrate improvements across diverse scenarios. However, the majority of existing methods explore static item attributes without considering additional contextual information (e.g., price, brand). Moreover, overlooking the interaction between the retrieval and ranking stages may lead to suboptimal solutions for fine-grained recommendations. In this work, we introduce PRIME: preference-optimized retrieval and ranking for efficient multimodal recommendation. PRIME operates in two stages: (i) a lightweight retriever identifies potential candidate items; (ii) an LMM learns to rank the retrieved candidates with detailed user history and multimodal features (e.g., text and image attributes). These features are incorporated into a carefully designed prompt, facilitating fine-grained transition patterns for user preference understanding. To optimize the inference efficiency of PRIME, we introduce verbalizer-based inference, which computes ranking scores for all candidate items in a single forward pass. Furthermore, we employ the LMM ranker to provide feedback on sampled candidate sets, enabling online preference optimization that refines the retriever model and improves the alignment between retrieval and ranking. As a result, PRIME can capture subtle user intentions and efficiently rank candidate items with minimal inference costs. Extensive experiments show the effectiveness and efficiency of PRIME, which consistently achieves superior performance over baseline methods.

Recent advances in large language models (LLMs) have introduced latent reasoning as a promising alternative to autoregressive reasoning. By performing internal computation with hidden states from previous steps, latent reasoning benefit from more informative features rather than sampling a discrete chain-of-thought (CoT) path. Yet latent reasoning approaches are often incompatible with LLMs, as their continuous paradigm conflicts with the discrete nature of autoregressive generation. Moreover, these methods rely on CoT traces for training and thus fail to exploit the inherent reasoning patterns of LLMs. In this work, we explore latent reasoning by leveraging the intrinsic capabilities of LLMs via reinforcement learning (RL). To this end, we introduce hybrid reasoning policy optimization (HRPO), an RL-based hybrid latent reasoning approach that (1) integrates prior hidden states into sampled tokens with a learnable gating mechanism, and (2) initializes training with predominantly token embeddings while progressively incorporating more hidden features. This design maintains LLMs' generative capabilities and incentivizes hybrid reasoning using both discrete and continuous representations. In addition, the hybrid HRPO introduces stochasticity into latent reasoning via token sampling, thereby enabling RL-based optimization without requiring CoT trajectories. Extensive evaluations across diverse benchmarks show that HRPO outperforms prior methods in both knowledge- and reasoning-intensive tasks. Furthermore, HRPO-trained LLMs remain interpretable and exhibit intriguing behaviors like cross-lingual patterns and shorter completion lengths, highlighting the potential of our RL-based approach and offer insights for future work in latent reasoning.

The proliferation of fake news on social media platforms disproportionately impacts vulnerable populations, eroding trust, exacerbating inequality, and amplifying harmful narratives. Detecting fake news in multimodal contexts-where deceptive content combines text and images-is particularly challenging due to the nuanced interplay between modalities. Existing multimodal fake news detection methods often emphasize cross-modal consistency but ignore the complex interactions between text and visual elements, which may complement, contradict, or independently influence the predicted veracity of a post. To address these challenges, we present Modality Interactive Mixture-of-Experts for Fake News Detection (MIMoE-FND), a novel hierarchical Mixture-of-Expert framework designed to enhance multimodal fake news detection by explicitly modeling modality interactions through an interaction gating mechanism. Our approach models modality interactions by evaluating two key aspects of modality interactions: unimodal prediction agreement and semantic alignment. The hierarchical structure of MIMoE-FND allows for distinct learning pathways tailored to different fusion scenarios, adapting to the unique characteristics of each modality interaction. By tailoring fusion strategies to diverse modality interaction scenarios, MIMoE-FND provides a more robust and nuanced approach to multimodal fake news detection. We evaluate our approach on three real-world benchmarks spanning two languages, demonstrating its superior performance compared to state-of-the-art methods. By enhancing the accuracy and interpretability of fake news detection, MIMoE-FND offers a promising tool to mitigate the spread of misinformation, with potential to better safeguard vulnerable communities against its harmful effects.

In this paper, we study the problem of AI explanation of misinformation, where the goal is to identify explanation designs that help improve users' misinformation detection abilities and their overall user experiences. Our work is motivated by the limitation of current Explainable AI (XAI) approaches, which predominantly focus on content explanations that elucidate the linguistic features and sentence structures of the misinformation. To address this limitation, we explore various explanations beyond content explanation, such as ''social explanation'' that considers the broader social context surrounding misinformation, as well as a ''combined explanation'' where both the content and social explanations are presented in scenarios that are either aligned or misaligned with each other. To evaluate the comparative effectiveness of these AI explanations, we conduct two online crowdsourcing experiments in COVID-19 (Study 1 on Prolific) and Politics domains (Study 2 on MTurk). Our results show that AI explanations are generally effective in aiding users to detect misinformation, with effectiveness significantly influenced by the alignment between content and social explanations. We also find that the order in which explanation types are presented-specifically, whether a content or social explanation comes first-can influence detection accuracy, with differences found between the COVID-19 and Political domains. This work contributes towards more effective design of AI explanations, fostering a deeper understanding of how different explanation types and their combinations influence misinformation detection.

With the prevalence of social media and short video sharing platforms (e.g., TikTok, YouTube Shorts), the proliferation of healthcare misinformation has become a widespread and concerning issue that threatens public health and undermines trust in mass media. This paper focuses on an important problem of detecting multimodal healthcare misinformation in short videos on TikTok. Our objective is to accurately identify misleading healthcare information that is jointly conveyed by the visual, audio, and textual content within the TikTok short videos. Three critical challenges exist in solving our problem: i) how to effectively extract information from distractive and manipulated visual content in short videos? ii) How to efficiently identify the interrelation of the heterogeneous visual and speech content in short videos? iii) How to accurately capture the complex dependency of the densely connected sequential content in short videos? To address the above challenges, we developMultiTec, a multimodal detector that explicitly explores the audio and visual content in short videos to investigate both the sequential relation of video elements and their inter-modality dependencies to jointly detect misinformation in healthcare videos on TikTok. To the best of our knowledge, MultiTec is the first modality-aware dual-attentive short video detection model for multimodal healthcare misinformation on TikTok. We evaluate MultiTec on two real-world healthcare video datasets collected from TikTok. Evaluation results show that MultiTec achieves substantial performance gains compared to state-of-the-art baselines in accurately detecting misleading healthcare short videos.

The rapid propagation of misinformation poses substantial risks to public interest.To combat misinformation, large language models (LLMs) are adapted to automatically verify claim credibility.Nevertheless, existing methods heavily rely on the embedded knowledge within LLMs and / or black-box APIs for evidence collection, leading to subpar performance with smaller LLMs or upon unreliable context.In this paper, we propose retrieval augmented fact verification through the synthesis of contrasting arguments (RAFTS).Upon input claims, RAFTS starts with evidence retrieval, where we design a retrieval pipeline to collect and re-rank relevant documents from verifiable sources.Then, RAFTS forms contrastive arguments (i.e., supporting or refuting) conditioned on the retrieved evidence.In addition, RAFTS leverages an embedding model to identify informative demonstrations, followed by in-context prompting to generate the prediction and explanation.Our method effectively retrieves relevant documents as evidence and evaluates arguments from varying perspectives, incorporating nuanced information for fine-grained decisionmaking.Combined with informative in-context examples as prior, RAFTS achieves significant improvements to supervised and LLM baselines without complex prompts.We demonstrate the effectiveness of our method through extensive experiments, where RAFTS can outperform GPT-based methods with a significantly smaller 7B LLM 1 .

Federated Recommendation (FR) emerges as a novel paradigm that enables privacy-preserving recommendations. However, traditional FR systems usually represent users/items with discrete identities (IDs), suffering from performance degradation due to data sparsity and heterogeneity in FR. On the other hand, Large Language Models (LLMs) as recommenders have proven effective across various recommendation scenarios. Yet, LLM-based recommenders encounter challenges such as incomplete recommendation and potential hallucination, compromising their performance in real-world scenarios. To this end, we propose GPT-FedRec, a federated recommendation framework leveraging ChatGPT and a novel hybrid Retrieval Augmented Generation (RAG) mechanism. GPT-FedRec is a two-stage solution. The first stage is a hybrid retrieval process, mining ID-based user patterns and text-based item features. Next, in the second stage, the results returned by hybrid retrieval are converted into text prompts and fed into GPT for re-ranking. Under GPT-FedRec, the privacy of both local training data and global test data is well protected, as there is no data exchange across any clients or the global server. For test users, GPT-FedRec executes inference only on the global server: given the historical data of a test user, GPT-FedRec performs hybrid retrieval and GPT-based re-ranking, without exposing test data to any other clients. Our proposed hybrid retrieval mechanism and LLM-based re-ranking aim to extract generalized features from data and exploit pretrained knowledge within LLM, overcoming data sparsity and heterogeneity in FR. Finally, the RAG nature of GPT-FedRec also prevents LLM hallucination, improving the recommendation performance for real-world users. Experimental results on diverse benchmark datasets demonstrate the superior performance of GPT-FedRec against state-of-the-art baseline methods. Our code is available at https://github.com/huiminzeng/GPT-FedRec.git.

While sequential recommendation achieves significant progress on capturing user-item transition patterns, transferring such large-scale recommender systems remains challenging due to the disjoint user and item groups across domains. In this paper, we propose a vector quantized meta learning for transferable sequential recommenders (MetaRec). Without requiring additional modalities or shared information across domains, our approach leverages user-item interactions from multiple source domains to improve the target domain performance. To solve the input heterogeneity issue, we adopt vector quantization that maps item embeddings from heterogeneous input spaces to a shared feature space. Moreover, our meta transfer paradigm exploits limited target data to guide the transfer of source domain knowledge to the target domain (i.e., learn to transfer). In addition, MetaRec adaptively transfers from multiple source tasks by rescaling meta gradients based on the source-target domain similarity, enabling selective learning to improve recommendation performance. To validate the effectiveness of our approach, we perform extensive experiments on benchmark datasets, where MetaRec consistently outperforms baseline methods by a considerable margin.

Federated learning (FL) emerges as a promising solution to train machine learning (ML) models from distributed data sources. In FL, the heterogeneous and imbalanced data distribution of local clients could severely hurt the fairness of the aggregated global model. In this paper, we identify two key obstacles of developing fair FL models w.r.t. the global distribution: the domain shifts from local clients to the global data distribution and the fairness heterogeneity across local clients. Therefore, considering these two obstacles, we present a novel fairness-aware FL training framework Robust-Fair Domain Smoothing (RFDS) to address the bias issue of FL models from a unique domain-shifting perspective. In particular, we design two novel components to build RFDS: 1) local robust-fair training, and 2) reference domain smoothing. Local robust-fair training aims to train robust-fair local models whose fairness is robust against the domain shifts from local distributions to the global distribution. Reference domain smoothing reduces the heterogeneity of fairness across clients to improve the fairness of the aggregated global model. We further provide a theoretical analysis to show the connection between the domain discrepancy of local data distributions and the heterogeneity of fairness across clients. Empirical evaluation results on multiple real-world datasets show that RFDS achieves promising performance gains in improving demographic fairness compared to state-of-the-art baselines.

A fundamental issue in healthcare misinformation detection is the lack of timely resources (e.g., medical knowledge, annotated data), making it challenging to accurately detect emergent healthcare misinformation at an early stage. In this paper, we develop a crowdsourcing-based early healthcare misinformation detection framework that jointly exploits the medical expertise of expert crowd workers and adapts the medical knowledge from a source domain (e.g., COVID-19) to detect misleading posts in an emergent target domain (e.g., Mpox, Polio). Two important challenges exist in developing our solution: (i) How to leverage the complex and noisy knowledge from the source domain to facilitate the detection of misinformation in the target domain? (ii) How to effectively utilize the limited amount of expert workers to correct the inapplicable knowledge facts in the source domain and adapt the corrected facts to examine the truthfulness of the posts in the emergent target domain? To address these challenges, we develop CrowdAdapt, a crowdsourcing-based domain adaptive approach that effectively identifies and adapts relevant knowledge facts from the source domain to accurately detect misinformation in the target domain. Evaluation results from two real-world case studies demonstrate the superiority of CrowdAdapt over state-of-the-art baselines in accurately detecting emergent healthcare misinformation.

Drought poses significant challenges to sustainability across various sectors in our society, leading to substantial consequences on agriculture, environments, ecosystems, public health, and socioeconomic stability. While prior work has studied the impacts of drought using professionally measured data sources, the societal perspectives of drought impacts remain largely under-explored. In this work, we present SocialDrought, a novel and comprehensive dataset to facilitate research on the societal impacts of drought. In particular, SocialDrought consists of three major components: 1) over 1.5 million social media posts, 2) over 1,400 news articles collected and verified by domain experts, and 3) over 31,000 meteorological records from the U.S. Drought Monitor about drought severity. In addition, we also introduce an online analytical platform that enables interactive and real-time data exploration to gain timely insights into the societal impacts of drought. Our interdisciplinary dataset integrates both conventional meteorological data and unconventional social and news media data to provide a holistic understanding of drought impacts. SocialDrought opens new opportunities to study the societal impacts of drought through the lens of social and news media.

Zhenrui Yue, Huimin Zeng, Yimeng Lu, Lanyu Shang, Yang Zhang, Dong Wang. Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 2024.

Huimin Zeng, Zhenrui Yue, Dong Wang. Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 2024.

Much existing literature on fair recommendation (i.e., group fairness) leverages users' demographic attributes (e.g., gender) to develop fair recommendation methods. However, in real-world scenarios, due to privacy concerns and convenience considerations, users may not be willing to share their demographic information with the system, which limits the application of many existing methods. Moreover, sequential recommendation (SR) models achieve state-of-the-art performance compared to traditional collaborative filtering (CF) recommenders, and can represent users solely using user-item interactions (user-free). This leaves a wrong impression that SR models are free from group unfairness by design. In this work, we explore a critical question: how can we build a fair sequential recommendation system without even knowing user demographics? To address this problem, we propose Agnostic FairSeqRec (A-FSR): a model-agnostic and demographic-agnostic debiasing framework for sequential recommendation without requiring users' demographic attributes. Firstly, A-FSR reduces the correlation between the potential stereotypical patterns in the input sequences and final recommendations via Dirichlet neighbor smoothing. Secondly, A-FSR estimates an under-represented group of sequences via a gradient-based heuristic, and implicitly moves training focus towards the under-represented group by minimizing a distributionally robust optimization (DRO) based objective. Results on real-world datasets show that A-FSR achieves significant improvements on group fairness in sequential recommendation, while outperforming other state-of-the-art baselines.

State-of-the-art sequential recommendation relies heavily on self-attention-based recommender models. Yet such models are computationally expensive and often too slow for real-time recommendation. Furthermore, the self-attention operation is performed at a sequence-level, thereby making low-cost incremental inference challenging. Inspired by recent advances in efficient language modeling, we propose linear recurrent units for sequential recommendation (LRURec). Similar to recurrent neural networks, LRURec offers rapid inference and can achieve incremental inference on sequential inputs. By decomposing the linear recurrence operation and designing recursive parallelization in our framework, LRURec provides the additional benefits of reduced model size and parallelizable training. Moreover, we optimize the architecture of LRURec by implementing a series of modifications to address the lack of non-linearity and improve training dynamics. To validate the effectiveness of our proposed LRURec, we conduct extensive experiments on multiple real-world datasets and compare its performance against state-of-the-art sequential recommenders. Experimental results demonstrate the effectiveness of LRURec, which consistently outperforms baselines by a significant margin. Results also highlight the efficiency of LRURec with our parallelized training paradigm and fast inference on long sequences, showing its potential to further enhance user experience in sequential recommendation.

This paper studies a critical problem of emergent health misinformation detection, aiming to mitigate the spread of misinformation in emergent health domains to support well-informed healthcare decisions towards a Web for good health. Our work is motivated by the lack of timely resources (e.g., medical knowledge, annotated data) during the initial phases of an emergent health event or topic. In this paper, we develop a multi-source domain adaptive framework that jointly exploits medical knowledge and annotated data from different high-resource source domains (e.g., cancer, COVID-19) to detect misleading posts in an emergent target domain (e.g., mpox, polio). Two important challenges exist in developing our solution: 1) how to accurately detect the partially misleading and unverifiable content in an emergent target domain? 2) How to identify the conflicting knowledge facts from different source domains to accurately detect emergent misinformation in the target domain? To address these challenges, we develop MMAdapt, a multi-source multi-class domain adaptive misinformation detection framework that effectively explores diverse knowledge facts from different source domains to accurately detect not only the outright misleading but also the partially misleading or unverifiable posts on the Web. Extensive experimental results on four real-world misinformation datasets demonstrate that MMAdapt substantially outperforms state-of-the-art baselines in accurately detecting misinformation in an emergent health domain.

This paper develops a symbiotic human-AI collective learning framework that explores the complementary strengths of both AI and crowdsourced human intelligence to address a novel Web-based healthcare-policy-adherence assessment (WebHA) problem. In particular, the objective of the WebHA problem is to automatically assess people's public health policy adherence during emergent global health crisis events (e.g., COVID-19, MonkeyPox) by exploring massive social media imagery data. Recent advances in human-AI systems exhibit a significant potential in addressing the intricate imagery-based classification problems like WebHA by leveraging the collective intelligence of both humans and AI. This paper aims to address the limitation of existing human-AI systems that often rely heavily on human intelligence to improve AI model performance while overlooking the fact that humans themselves can be fallible and prone to errors. To address the above limitation, this paper develops SymLearn, a symbiotic human-AI co-learning framework that leverages human intelligence to troubleshoot and fine-tune the AI model while using AI models to guide human crowd workers to reduce the inherent human errors in their labels. Extensive experiments on two real-world WebHA applications show that SymLearn clearly outperforms the state-of-the-art baselines by improving WebHA performance and reducing crowd response delay.

This article overviews the state of the art, research challenges, and future directions in our vision: integrating social explanation into explainable artificial intelligence (XAI) to combat misinformation. In our context, “social explanation” is an explanatory approach that reveals the social aspect of misinformation by analyzing sociocontextual cues, such as user attributes, user engagement metrics, diffusion patterns, and user comments. Our vision is motivated by the research gap in the existing XAI that tends to overlook the broader social context in which misinformation spreads. In this article, we first define social explanation, demonstrating it through examples, enabling technologies, and real-world applications. We then outline the unique benefits social explanation brings to the fight against misinformation and discuss the challenges that make our vision complex. The significance of this article lies in introducing the “social explanation” concept in XAI, which has been underexplored in the previous literature. Also, we demonstrate how social explanations can be effectively employed to tackle misinformation and promote collaboration across diverse fields by drawing upon interdisciplinary techniques spanning from computer science, social computing, human–computer interaction, to psychology. We hope that this article will advance progress in the field of XAI and contribute to the ongoing efforts to counter misinformation.

With emerging online topics as a source for numerous new events, detecting unseen / rare event types presents an elusive challenge for existing event detection methods, where only limited data access is provided for training.To address the data scarcity problem in event detection, we propose MetaEvent, a meta learningbased framework for zero-and few-shot event detection.Specifically, we sample training tasks from existing event types and perform meta training to search for optimal parameters that quickly adapt to unseen tasks.In our framework, we propose to use the clozebased prompt and a trigger-aware soft verbalizer to efficiently project output to unseen event types.Moreover, we design a contrastive meta objective based on maximum mean discrepancy (MMD) to learn class-separating features.As such, the proposed MetaEvent can perform zero-shot event detection by mapping features to event types without any prior knowledge.In our experiments, we demonstrate the effectiveness of MetaEvent in both zero-shot and few-shot scenarios, where the proposed method achieves state-of-the-art performance in extensive experiments on benchmark datasets Few-Event and MAVEN.

With emerging topics (e.g., COVID-19) on social media as a source for the spreading misinformation, overcoming the distributional shifts between the original training domain (i.e., source domain) and such target domains remains a non-trivial task for misinformation detection.This presents an elusive challenge for early-stage misinformation detection, where a good amount of data and annotations from the target domain is not available for training.To address the data scarcity issue, we propose MetaAdapt, a meta learning based approach for domain adaptive few-shot misinformation detection.MetaAdapt leverages limited target examples to provide feedback and guide the knowledge transfer from the source to the target domain (i.e., learn to adapt).In particular, we train the initial model with multiple source tasks and compute their similarity scores to the meta task.Based on the similarity scores, we rescale the meta gradients to adaptively learn from the source tasks.As such, MetaAdapt can learn how to adapt the misinformation detection model and exploit the source data for improved performance in the target domain.To demonstrate the efficiency and effectiveness of our method, we perform extensive experiments to compare MetaAdapt with state-of-the-art baselines and large language models (LLMs) such as LLaMA, where MetaAdapt achieves better performance in domain adaptive few-shot misinformation detection with substantially reduced parameters on real-world datasets.

Robust out-domain uncertainty estimation has gained growing attention for its capacity of providing adversary-resistant uncertainty estimates on out-domain samples. However, existing work on robust uncertainty estimation mainly focuses on evasion attacks that happen during test time. The threat of poisoning attacks against uncertainty models is largely unexplored. Compared to evasion attacks, poisoning attacks do not necessarily modify test data, and therefore, would be more practical in real-world applications. In this work, we systematically investigate the robustness of state-of-the-art uncertainty estimation algorithms against data poisoning attacks, with the ultimate objective of developing robust uncertainty training methods. In particular, we focus on attacking the out-domain uncertainty estimation. Under the proposed attack, the training process of models is affected. A fake high-confidence region is established around the targeted out-domain sample, which originally would have been rejected by the model due to low confidence. More fatally, our attack is clean-label and targeted: it leaves the poisoned data with clean labels and attacks a specific targeted test sample without degrading the overall model performance. We evaluate the proposed attack on several image benchmark datasets and a real-world application of COVID-19 misinformation detection. The extensive experimental results on different tasks suggest that the state-of-the-art uncertainty estimation methods could be extremely vulnerable and easily corrupted by our proposed attack.

Demographic fairness has become a critical objective when developing modern visual models for identity-sensitive applications, such as face attribute recognition (FAR). While great efforts have been made to improve the fairness of the models, the investigation on the adversarial robustness of the fairness (e.g., whether the fairness of the models could still be maintained under potential malicious fairness attacks) is largely ignored. Therefore, this paper explores the adversarial robustness of demographic fairness in FAR applications from both attacking and defending perspectives. In particular, we firstly present a novel fairness attack, who aims at corrupting the demographic fairness of face attribute classifiers. Next, to mitigate the effect of the fairness attack, we design an efficient defense algorithm called robust-fair training. With this defense, face attribute classifiers learn how to combat the bias introduced by the fairness attack. As such, the face attribute classifiers are not only trained to be fair, but the fairness is also robust. Our extensive experimental results show the effectiveness of both our proposed attack and defense methods across various model architectures and FAR applications. We believe our work could be strong baselines for future work on robust-fair AI models.

This paper focuses on the AI-based damage assessment (ADA) applications that leverage state-of-the-art AI techniques to automatically assess the disaster damage severity using online social media imagery data, which aligns well with the ''disaster risk reduction'' target under United Nations' Sustainable Development Goals (UN SDGs). This paper studies an ADA model generality problem where the objective is to address the limitation of current ADA solutions that are often optimized only for a single disaster event and lack the generality to provide accurate performance across different disaster events. To address this limitation, we work with domain experts and local community stakeholders in disaster response to develop CollabGeneral, a subjective logic-driven crowd-AI collaborative learning framework that integrates AI and crowdsourced human intelligence into a principled learning framework to address the ADA model generality problem. Extensive experiments on four real-world ADA datasets demonstrate that CollabGeneral consistently outperforms the state-of-the-art baselines by significantly improving the ADA model generality across different disasters.

Social sensing has become a critical source to obtain timely observations of emergent crisis events at an unprecedented scale by exploring the social media data contributed by common citizens. This paper focuses on an AI-powered crisis situation awareness (ACSA) application in social sensing that aims to obtain accurate situation awareness of emergent crisis events by leveraging advanced AI techniques and social sensing data. In particular, we study a cross-event quality-of-service (C-QoS) problem in ACSA applications where the goal is to address the limitation of current AI models that are often optimized only for a single crisis event and lack the generality to provide desirable QoS across different events. This paper explores the integration of AI and crowdsourced human intelligence as a solution to address the limitation in tackling the problem of C-QoS. However, two critical challenges exist: 1) it is challenging to optimize the ACSA model’s C-QoS without sacrificing its specificity on each studied crisis event; 2) it is non-trivial to integrate the diversified AI and human intelligence to optimize C-QoS in ACSA applications. To combat these challenges, we introduce CrossGenerat, a subjective logic-driven human-AI collective learning framework that jointly leverages the specificity of AI and the generality of human intelligence to provide robust and accurate ACSA performance across different types of crisis events. Through evaluations performed on two real-world ACSA applications, CrossGeneral exhibits superior performance compared to state-of-the-art baselines by substantially enhancing the C-QoS of ACSA models.

Web-based disaster response (WebDR) is emerging as a pervasive approach to acquire real-time situation awareness of disaster events by collecting timely observations from the Web (e.g., social media). This paper studies a class-wise inequality problem in WebDR applications where the objective is to address the limitation of current WebDR solutions that often have imbalanced classification performance across different classes. To address such a limitation, this paper explores the collaborative strengths of the diversified yet complementary biases of AI and crowdsourced human intelligence to ensure a more balanced and accurate performance for WebDR applications. However, two critical challenges exist: 1) it is difficult to identify the imbalanced AI results without knowing the ground-truth WebDR labels a priori; ii) it is non-trivial to address the class-wise inequality problem using potentially imperfect crowd labels. To address the above challenges, we develop CollabEquality, an inequality-aware crowd-AI collaborative learning framework that carefully models the inequality bias of both AI and human intelligence from crowdsourcing systems into a principled learning framework. Extensive experiments on two real-world WebDR applications demonstrate that CollabEquality consistently outperforms the state-of-the-art baselines by significantly reducing class-wise inequality while improving the WebDR classification accuracy.

The widespread misinformation on the Web has raised many concerns with serious societal consequences. In this paper, we study a critical type of online misinformation, namely fauxtography, where the image and associated text of a social media post jointly convey a questionable or false sense. In particular, we focus on a sparse semi-supervised fauxtography detection problem, which aims to accurately identify fauxtography by only using the sparsely annotated ground truth labels of social media posts. Our problem is motivated by the key limitation of current fauxtography detection approaches that often require a large amount of expensive and inefficient manual annotations to train an effective fauxtography detection model. We identify two key technical challenges in solving the problem: 1) it is non-trivial to train an accurate detection model given the sparse fauxtography annotations, and 2) it is difficult to extract the heterogeneous and complicated fauxtography features from the multi-modal social media posts for accurate fauxtography detection. To address the above challenges, we propose ContrastFaux, a multi-view contrastive learning framework that jointly explores the sparse fauxtography annotations and the cross-modal fauxtography feature similarity between the image and text in multi-modal posts to accurately detect fauxtography on social media. Evaluation results on two social media datasets demonstrate that ContrastFaux consistently outperforms state-of-the-art deep learning and semi-supervised learning fauxtography detection baselines by achieving the highest fauxtography detection accuracy.

Fauxtography is a category of multi-modal posts that spread misleading information on various big data online social platforms that generate billions of posts on a daily basis (e.g., Facebook, Twitter, Reddit). A fauxtography post usually consists of an image, a text description and comments from its readers. In this paper, we focus on explaining fauxtography posts by identifyingwhatspecific component andwhythat component of a post leads to the fauxtography (i.e., duo explanations). This problem is motivated by the limitations of current fauxtography detection solutions that only focus on the detection but ignore the importantexplanationaspect of their results. Two critical challenges exist in solving our problem: i) it is difficult to accurately identify the “guilty” component of a fauxtography post given the fact that different components of the post and their associations could all lead to the fauxtography; ii) it is expensive and time-consuming to obtain a good training set with fine-grained labels of fauxtography posts in terms of explainability, making it challenging to develop fully supervised explainable solutions. To address the above challenges, we develop aDuoExplainableFauxtography Detection Framework under aConstrained Supervision (DExFC) to generate duo explanations from both content and comment parts of fauxtography posts. We evaluate the DExFC by creating real-world datasets from different online social media platforms (Twitter and Reddit). The results show that DExFC not only detects the fauxtography posts more accurately than the state-of-the-art solutions but also provides well-justified explanations to its results without the full supervision.

With the proliferation of smart devices and widespread Internet connectivity, social sensing is advancing as a pervasive sensing paradigm where experiences shared by individuals on social platforms (e.g., Twitter and Facebook) are analyzed to interpret the physical world. In this article, we introduce CovidTrak, a vision of social intelligence-empowered contact tracing that aims to scrutinize the knowledge derived using social sensing to track Coronavirus Disease 2019 (COVID-19) infections among the general public. Contact tracing is known to be an effective technique for detecting and monitoring persons who may have been exposed to individuals infected with any communicable disease. While a good number of contact tracing schemes are existent today (e.g., in-person and phone interviews, paper forms, email and web-based questionnaires, and smartphone apps), they often require active user participation and might miss certain cases of social interactions that go off-the-records but still lead to COVID-19 transmission. By contrast, social sensing provides an alternative avenue for spontaneously determining such contacts by harnessing the rich experiences and information conveyed by people on social data platforms (e.g., a group photograph tweeted from a house party with a potential contact). As such, CovidTrak can form a powerful basis to combat the COVID-19 pandemic. The vision of CovidTrak intends to answer the following questions: 1) how to bolster the privacy and security of the online users while determining their contacts? 2) how to collect relevant social signals that indicate in-person encounters among people? 3) how to reliably process the vast amount of noisy data from social platforms to identify chains of transmission? 4) how to handle the scarcity of location metadata in the incoming data? 5) how to effectively communicate crucial contact information to concerned individuals? and 6) how to model and handle the responses of the common people toward contact information? We envision unexplored opportunities to leverage multidisciplinary techniques to address the above questions and develop effective future CovidTrak schemes.

This paper focuses on an important problem of early misinformation detection in an emergent health domain on social media. Current misinformation detection solutions often suffer from the lack of resources (e.g., labeled datasets, sufficient medical knowledge) in the emerging health domain to accurately identify online misinformation at an early stage. To address such a limitation, we develop a knowledge-driven domain adaptive approach that explores a good set of annotated data and reliable knowledge facts in a source domain (e.g., COVID-19) to learn the domain-invariant features that can be adapted to detect misinformation in the emergent target domain with little ground truth labels (e.g., Monkeypox). Two critical challenges exist in developing our solution: i) how to leverage the noisy knowledge facts in the source domain to obtain the medical knowledge related to the target domain? ii) How to adapt the domain discrepancy between the source and target domains to accurately assess the truthfulness of the social media posts in the target domain? To address the above challenges, we develop KAdapt, a knowledge-driven domain adaptive early misinformation detection framework that explicitly extracts rel-evant knowledge facts from the source domain and jointly learns the domain-invariant representation of the social media posts and their relevant knowledge facts to accurately identify misleading posts in the target domain. Evaluation results on five real-world datasets demonstrate that KAdapt significantly outperforms state-of-the-art baselines in terms of accurately detecting misleading Monkeypox posts on social media.

In the real-world application of COVID-19 misinformation detection, a fundamental challenge is the lack of the labeled COVID data to enable supervised end-to-end training of the models, especially at the early stage of the pandemic. To address this challenge, we propose an unsupervised domain adaptation framework using contrastive learning and adversarial domain mixup to transfer the knowledge from an existing source data domain to the target COVID-19 data domain. In particular, to bridge the gap between the source domain and the target domain, our method reduces a radial basis function (RBF) based discrepancy between these two domains. Moreover, we leverage the power of domain adversarial examples to establish an intermediate domain mixup, where the latent representations of the input text from both domains could be mixed during the training process. Extensive experiments on multiple real-world datasets suggest that our method can effectively adapt misinformation detection systems to the unseen COVID-19 target domain with significant improvements compared to the state-of-the-art baselines.

Modern machine learning (ML) is one of the prevailing tools for big data applications of face attribute recognition. However, due to the commonly observed imbalanced distribution of the training data, well-trained models could suffer severely from undesired performance bias across different demographic groups. Motivated by the fact that neural networks could be extremely sensitive to adversarial examples, we argue that there exists the possibility of properly leveraging adversarial examples to address the imbalanced data distribution, and guiding the training convergence towards the direction of improved fairness. That is, we propose to use adversarial examples to alleviate the performance bias issue from the origin: the data source. In this paper, we present a novel adversarial training framework that generates adversarial features in the latent space to automatically balance the distribution of training features and adjust the deep classification layers of the face attribute classifiers to be more fair. Extensive experimental results on the CelebA face dataset show that our method is able to boost the model fairness more effectively compared to the state-of-the-art adversarial debiasing algorithms.

Despite recent progress in improving the performance of misinformation detection systems, classifying misinformation in an unseen domain remains an elusive challenge. To address this issue, a common approach is to introduce a domain critic and encourage domain-invariant input features. However, early misinformation often demonstrates both conditional and label shifts against existing misinformation data (e.g., class imbalance in COVID-19 datasets), rendering such methods less effective for detecting early misinformation. In this paper, we propose contrastive adaptation network for early misinformation detection (CANMD). Specifically, we leverage pseudo labeling to generate high-confidence target examples for joint training with source data. We additionally design a label correction component to estimate and correct the label shifts (i.e., class priors) between the source and target domains. Moreover, a contrastive adaptation loss is integrated in the objective function to reduce the intra-class discrepancy and enlarge the inter-class discrepancy. As such, the adapted model learns corrected class priors and an invariant conditional distribution across both domains for improved estimation of the target data distribution. To demonstrate the effectiveness of the proposed CANMD, we study the case of COVID-19 early misinformation detection and perform extensive experiments using multiple real-world datasets. The results suggest that CANMD can effectively adapt misinformation detection systems to the unseen COVID-19 target domain with significant improvements compared to the state-of-the-art baselines.

Question answering (QA) has demonstrated impressive progress in answering questions from customized domains. Nevertheless, domain adaptation remains one of the most elusive challenges for QA systems, especially when QA systems are trained in a source domain but deployed in a different target domain. In this work, we investigate the potential benefits of question classification for QA domain adaptation. We propose a novel framework: Question Classification for Question Answering (QC4QA). Specifically, a question classifier is adopted to assign question classes to both the source and target data. Then, we perform joint training in a self-supervised fashion via pseudo-labeling. For optimization, inter-domain discrepancy between the source and target domain is reduced via maximum mean discrepancy (MMD) distance. We additionally minimize intra-class discrepancy among QA samples of the same question class for fine-grained adaptation performance. To the best of our knowledge, this is the first work in QA domain adaptation to leverage question classification with self-supervised adaptation. We demonstrate the effectiveness of the proposed QC4QA with consistent improvements against the state-of-the-art baselines on multiple datasets.

Modern smart sensor-based energy management systems leverage non-intrusive load monitoring (NILM) to predict and optimize appliance load distribution in real-time. NILM, or energy disaggregation, refers to the decomposition of electricity usage conditioned on the aggregated power signals (i.e., smart sensor on the main channel). Based on real-time appliance power prediction using sensory technology, energy disaggregation has great potential to increase electricity efficiency and reduce energy expenditure. With the introduction of transformer models, NILM has achieved significant improvements in predicting device power readings. Nevertheless, transformers are less efficient due to O(l2) complexity w.r.t. sequence length l. Moreover, transformers can fail to capture local signal patterns in sequence-to-point settings due to the lack of inductive bias in local context. In this work, we propose an efficient localness transformer for non-intrusive load monitoring (ELTransformer). Specifically, we leverage normalization functions and switch the order of matrix multiplication to approximate self-attention and reduce computational complexity. Additionally, we introduce localness modeling with sparse local attention heads and relative position encodings to enhance the model capacity in extracting short-term local patterns. To the best of our knowledge, ELTransformer is the first NILM model that addresses computational complexity and localness modeling in NILM. With extensive experiments and quantitative analyses, we demonstrate the efficiency and effectiveness of the the proposed ELTransformer with considerable improvements compared to state-of-the-art baselines.

Question answering (QA) has recently shown impressive results for answering questions from customized domains.Yet, a common challenge is to adapt QA models to an unseen target domain.In this paper, we propose a novel selfsupervised framework called QADA for QA domain adaptation.QADA introduces a novel data augmentation pipeline used to augment training QA samples.Different from existing methods, we enrich the samples via hidden space augmentation.For questions, we introduce multi-hop synonyms and sample augmented token embeddings with Dirichlet distributions.For contexts, we develop an augmentation method which learns to drop context spans via a custom attentive sampling strategy.Additionally, contrastive learning is integrated in the proposed self-supervised adaptation framework QADA.Unlike existing approaches, we generate pseudo labels and propose to train the model via a novel attention-based contrastive adaptation method.The attention weights are used to build informative features for discrepancy estimation that helps the QA model separate answers and generalize across source and target domains.To the best of our knowledge, our work is the first to leverage hidden space augmentation and attention-based contrastive adaptation for self-supervised domain adaptation in QA.Our evaluation shows that QADA achieves considerable improvements on multiple target datasets over state-of-the-art baselines in QA domain adaptation.

In this paper, we study an explainable COVID-19 misinformation detection problem where the goal is to accurately identify COVID-19 misleading posts on social media and explain the posts with natural language explanations (NLEs). Our problem is motivated by the limitations of current explainable misinformation detection approaches that cannot provide NLEs for COVID-19 posts due to the lack of sufficient professional COVID-19 knowledge for supervision. To address such a limitation, we develop CEA-COVID, a crowd-expert-AI framework that jointly exploits the common logical reasoning ability of online crowd workers and the professional knowledge of COVID-19 experts to effectively generate NLEs for detecting and explaining COVID-19 misinformation. We evaluate CEA-COVID using two public COVID-19 misinformation datasets on social media. Results demonstrate that CEA-COVID outperforms existing explainable misinformation detection models in terms of both explainability and detection accuracy.

In many applications with real-world consequences, it is crucial to develop reliable uncertainty estimation for the predictions made by the AI decision systems. Targeting at the goal of estimating uncertainty, various deep neural network (DNN) based uncertainty estimation algorithms have been proposed. However, the robustness of the uncertainty returned by these algorithms has not been systematically explored. In this work, to raise the awareness of the research community on robust uncertainty estimation, we show that state-of-the-art uncertainty estimation algorithms could fail catastrophically under our proposed adversarial attack despite their impressive performance on uncertainty estimation. In particular, we aim at attacking out-domain uncertainty estimation: under our attack, the uncertainty model would be fooled to make high-confident predictions for the out-domain data, which they originally would have rejected. Extensive experimental results on various benchmark image datasets show that the uncertainty estimated by state-of-the-art methods could be easily corrupted by our attack.

In this paper, we focus on the quality of information service (QoIS) of COVID-19-related information on social media. Our goal is to provide reliable COVID-19 information service by accurately detecting the misleading COVID-19 posts on social media by exploring the community-contributed COVID-19 fact data (CCFD) from different social media platforms. In particular, CCFD refers to the fact-checking reports that are submitted to each social media platform by its users and fact-checking professionals. Our work is motivated by the observation that CCFD often contains useful COVID-19 knowledge facts (e.g., "COVID-19 is not a flu") that can effectively facilitate the identification of misleading COVID-19 social media posts. However, CCFD is often private to the individual social media platform that owns it due to the data privacy concerns such as data copyright of CCFD and user profile information of CCFD contributors. In this paper, we leverage the CCFD from different social media platforms to accurately detect COVID19 misinformation while effectively protecting the privacy of CCFD. Two critical challenges exist in solving our problem: 1) how to generate privacy-aware COVID-19 knowledge facts from the platform-specific CCFD? 2) How to effectively integrate the privacy-aware COVID-19 knowledge facts from different social media platforms to correctly assess the truthfulness of a COVID19 post? To address these challenges, we develop CoviDKG, a COVID-19 distributed knowledge graph framework that constructs a set of CCFD-based knowledge graphs on individual social media platform and exchanges the privacy-aware COVID19 knowledge facts across different platforms to effectively detect misleading COVID-19 posts. We evaluate CoviDKG on two real-world social media datasets and the results show that CoviDKG achieves significant performance gains compared to state-of-the-art baselines in accurately detecting misleading COVID-19 posts on social media.

While sequential recommender systems achieve significant improvements on capturing user dynamics, we argue that sequential recommenders are vulnerable against substitution-based profile pollution attacks. To demonstrate our hypothesis, we propose a substitution-based adversarial attack algorithm, which modifies the input sequence by selecting certain vulnerable elements and substituting them with adversarial items. In both untargeted and targeted attack scenarios, we observe significant performance deterioration using the proposed profile pollution algorithm. Motivated by such observations, we design an efficient adversarial defense method called Dirichlet neighborhood sampling. Specifically, we sample item embeddings from a convex hull constructed by multi-hop neighbors to replace the original items in input sequences. During sampling, a Dirichlet distribution is used to approximate the probability distribution in the neighborhood such that the recommender learns to combat local perturbations. Additionally, we design an adversarial training method tailored for sequential recommender systems. In particular, we represent selected items with one-hot encodings and perform gradient ascent on the encodings to search for the worst case linear combination of item embeddings in training. As such, the embedding function learns robust item representations and the trained recommender is resistant to test-time adversarial examples. Extensive experiments show the effectiveness of both our attack and defense methods, which consistently outperform baselines by a significant margin across model architectures and datasets.

Groundwater contamination poses serious threats to public health and environmental sustainability. In this paper, we study the smart groundwater contamination sensing problem that aims at accurately estimating the nitrate concentration in groundwater via a crowdsensing approach. Existing solutions often require professional groundwater collection and high-quality measurement of groundwater properties, making the data collection process time-consuming and unscalable. In this work, we leverage the approximate nitrate concentration measured by crowd sensors (i.e., participants from well-dependent communities) to accurately estimate nitrate concentration in groundwater samples. Two critical challenges exist in developing the crowdsensing-based groundwater contamination estimation solution: i) the spatial irregularity of the crowdsensing groundwater contamination data, and ii) the hidden temporal dependency of groundwater contamination on the anthropogenic context. To address the above challenges, we develop SmartWaterSens, a context-aware graph neural network framework that explicitly models the irregular spatial relations of crowdsensing groundwater contamination data and its relevant anthropogenic context to accurately estimate groundwater nitrate concentration. We evaluate the SmartWaterSens framework through a crowdsensing nitrate contamination dataset collected from a real-world case study in well-dependent communities in Northern Indiana, United States. The evaluation results not only show the effectiveness of SmartWaterSens in accurately estimating nitrate concentration but also demonstrate the viability of crowdsensing for community-level groundwater quality monitoring.

Human face images represent a rich set of visual information for online social media platforms to optimize the machine learning (ML)/AI models in their data-driven facial applications (e.g., face detection, face recognition). However, there exists a growing privacy concern from social media users to share their online face images that will be annotated by unknown crowd workers and analyzed by ML/AI researchers in the model training and optimization process. In this paper, we focus on a privacy-aware face recognition problem where the goal is to empower the facial applications to train their face recognition models with images shared by social media users while protecting the identity of the users. Our problem is motivated by the limitation of current privacy-aware face recognition approaches that mainly prevent algorithmic attacks by manipulating face images but largely ignore the potential privacy leakage related to human activities (e.g., crowdsourcing annotation). To address such limitations, we develop FaceCrowd, a web crowdsourcing based face partition approach to improve the performance of current face recognition models by designing a novel crowdsourced partial face graph generated from privacy-preserved social media face images. We evaluate the performance of FaceCrowd using two real-world human face datasets that consist of large-scale human face images. The results show that FaceCrowd not only improves the accuracy of the face recognition models but also effectively protects the identity information of the social media users who share their face images.

This paper focuses on a critical problem of explainable multimodal COVID-19 misinformation detection where the goal is to accurately detect misleading information in multimodal COVID-19 news articles and provide the reason or evidence that can explain the detection results. Our work is motivated by the lack of judicious study of the association between different modalities (e.g., text and image) of the COVID-19 news content in current solutions. In this paper, we present a generative approach to detect multimodal COVID-19 misinformation by investigating the cross-modal association between the visual and textual content that is deeply embedded in the multimodal news content. Two critical challenges exist in developing our solution: 1) how to accurately assess the consistency between the visual and textual content of a multimodal COVID-19 news article? 2) How to effectively retrieve useful information from the unreliable user comments to explain the misinformation detection results? To address the above challenges, we develop a duo-generative explainable misinformation detection (DGExplain) framework that explicitly explores the cross-modal association between the news content in different modalities and effectively exploits user comments to detect and explain misinformation in multimodal COVID-19 news articles. We evaluate DGExplain on two real-world multimodal COVID-19 news datasets. Evaluation results demonstrate that DGExplain significantly outperforms state-of-the-art baselines in terms of the accuracy of multimodal COVID-19 misinformation detection and the explainability of detection explanations.

The proliferation of social media has promoted the spread of misinformation that raises many concerns in our society. This paper focuses on a critical problem of explainable COVID-19 misinformation detection that aims to accurately identify and explain misleading COVID-19 claims on social media. Motivated by the lack of COVID-19 relevant knowledge in existing solutions, we construct a novel crowdsource knowledge graph based approach to incorporate the COVID-19 knowledge facts by leveraging the collaborative efforts of expert and non-expert crowd workers. Two important challenges exist in developing our solution: i) how to effectively coordinate the crowd efforts from both expert and non-expert workers to generate the relevant knowledge facts for detecting COVID-19 misinformation; ii) How to leverage the knowledge facts from the constructed knowledge graph to accurately explain the detected COVID-19 misinformation. To address the above challenges, we develop HC-COVID, a hierarchical crowdsource knowledge graph based framework that explicitly models the COVID-19 knowledge facts contributed by crowd workers with different levels of expertise and accurately identifies the related knowledge facts to explain the detection results. We evaluate HC-COVID using two public real-world datasets on social media. Evaluation results demonstrate that HC-COVID significantly outperforms state-of-the-art baselines in terms of the detection accuracy of misleading COVID-19 claims and the quality of the explanations.

Human-centric fauxtography is a category of multi-modal posts that spread misleading information on online information distribution and sharing platforms such as online social media. The reason of a human-centric post being fauxtography is closely related to its multi-modal content that consists of diversified human and non-human subjects with complex and implicit relationships. In this paper, we focus on an explainable fauxtography detection problem where the goal is to accurately identify and explain why a human-centric social media post is fauxtography (or not). Our problem is motivated by the limitations of current fauxtography detection solutions that focus primarily on the detection task but ignore the important aspect of explaining their results (e.g., why a certain component of the post delivers the misinformation). Two important challenges exist in solving our problem: 1) it is difficult to capture the implicit relations and attributions of different subjects in a fauxtography post given the fact that many of such knowledge is shared between different crowd workers; 2) it is not a trivial task to create a multi-modal knowledge graph from crowd workers to identify and explain human-centric fauxtography posts with multi-modal contents. To address the above challenges, we develop CrowdGraph, a crowdsourcing based multi-modal knowledge graph approach to address the explainable fauxtography detection problem. We evaluate the performance of CrowdGraph by creating a real-world dataset that consists of human-centric fauxtography posts from Twitter and Reddit. The results show that CrowdGraph not only detects the fauxtography posts more accurately than the state-of-the-arts but also provides well-justified explanations to the detection results with convincing evidence.

Disaster damage assessment (DDA) has emerged as an important application in disaster response and management, which aims to assess the damage severity of an affected area by leveraging AI (e.g., deep learning) techniques to examine the imagery data posted on social media during a disaster event. In this paper, we focus on a crowd-guided neural architecture searching (NAS) problem in DDA applications. Our goal is to leverage human intelligence from crowdsourcing systems to guide the discovery of the optimal neural network architecture in the design space to achieve the desirable damage assessment performance. Our work is motivated by the limitation that the deep neural network architectures in current DDA solutions are mainly designed by AI experts, which is known to be both time-consuming and error-prone. Two critical technical challenges exist in solving our problem: i) it is challenging to design a manageable NAS space for crowd-based solutions; ii) it is non-trivial to transfer the imperfect crowd knowledge to effective decisions in identifying the optimal neural network architecture of a DDA application. To address the above challenges, we develop CrowdNAS, a crowd-guided NAS framework that develops novel techniques inspired by AI, crowdsourcing, and estimation theory to address the NAS problem. The evaluation results from two real-world DDA applications show that CrowdNAS consistently outperforms the state-of-the-art AI-only, crowd-AI, and NAS baselines by achieving the highest classification accuracy in the damage assessment while maintaining a low computational cost under various evaluation scenarios.

AI-based smart urban sensing (ASUS) has emerged as a scalable and pervasive application paradigm in smart city planning and management that aims to automatically assess the physical status of the urban environments by leveraging AI techniques and massive urban sensing data. In this paper, we focus on a crowd-driven neural network (NN) hyperparameter optimization problem in ASUS applications. Our goal is to utilize the human intelligence from crowdsourcing systems to identify the optimal NN hyperparameter configuration for an ASUS model. Our work is motivated by the observation that the hyperparameters of current ASUS models are often manually configured by the AI specialists, which is known to be both error-prone and suboptimal. Two key technical challenges exist in solving our problem: i) it is challenging to effectively translate the highly complex NN hyperparameter optimization problem in AI to a simplified problem that can be solved by crowd workers without extensive AI expertise; ii) it is difficult to identify the optimal hyperparameter configuration in the large hyperparameter search space given the blackbox nature of the AI model. To address the above challenges, we develop CrowdOptim, a crowd-AI collaborative learning framework that integrates the techniques from crowdsourcing, hyperparameter optimization, and estimation theory to address the crowd-driven NN hyperparameter optimization problem in ASUS applications. The evaluation results from two real-world ASUS applications (i.e., smart city infrastructure monitoring (SCIM) and urban environment cleanliness assessment (UECA)) show that CrowdOptim consistently outperforms the state-of-the-art deep convolutional networks, crowd-AI, and hyperparameter optimization baselines in achieving the ASUS application objectives under various evaluation scenarios.

A key challenge in scientific simulation is that the simulation outputs often require intensive I/O and storage space to store the results for effective post hoc analysis. This article focuses on aquality-aware adaptive temporal data selection and reconstructionproblem where the goal is to adaptively select simulation data samples at certain key timesteps in situ and reconstruct the discarded samples with quality assurance during post hoc analysis. This problem is motivated by the limitation of current solutions that a significant amount of simulation data samples are either discarded or aggregated during the sampling process, leading to inaccurate modeling of the simulated phenomena. Two unique challenges exist: 1) the sampling decisions have to be made in situ and adapted to the dynamics of the complex scientific simulation data; 2) the reconstruction error must be strictly bounded to meet the application requirement. To address the above challenges, we developDeepSample, an error-controlled convolutional neural network framework, that jointly integrates a set of coherent multi-branch deep decoders to effectively reconstruct the simulation data with rigorous quality assurance. The results on two real-world scientific simulation applications show that DeepSample significantly outperforms other state-of-the-art methods on both sampling efficiency and reconstructed simulation data quality.

Cultural heritage sites are precious and fragile resources that hold significant historical, esthetic, and social values in our society. However, the increasing frequency and severity of natural and man-made disasters constantly strike the cultural heritage sites with significant damages. In this article, we focus on a cultural heritage damage assessment (CHDA) problem where the goal is to accurately locate the damaged area of a cultural heritage site using the imagery data posted on social media during a disaster event by exploring the collective strengths of both AI and human intelligence from crowdsourcing systems. Unlike other infrastructure-based solutions, social media platforms provide a more pervasive and scalable solution to acquire timely cultural heritage damage information during disaster events. Our work is motivated by the limitation of current AI solutions that fail to accurately model the complex cultural heritage damage due to the lack of essential human cultural knowledge to differentiate various damage types and identify the actual causes of the damage. Two critical technical challenges exist in solving our problem: 1) it is challenging to effectively detect the problematic cultural heritage damage estimation of AI in the absence of ground truth labels and 2) it is nontrivial to acquire accurate cultural background knowledge from the potentially unreliable crowd workers to effectively address the failure cases of AI. To address the above-mentioned challenges, we develop CollabLearn, an uncertainty-aware crowd-AI collaborative assessment system that explicitly explores the human intelligence from crowdsourcing systems to identify and fix AI failure cases and boost the damage assessment accuracy in CHDA applications. The evaluation results on real-world datasets show that CollabLearn consistently outperforms both the state-of-the-art AI-only and crowd-AI hybrid baselines in accurately assessing the damage of several world-renowned cultural heritage sites in recent disaster events.

Motivated by the state-of-the-art optical sensing and image processing technologies, remote urban sensing (RUS) has emerged as a powerful sensing paradigm to capture abundant visual information about the urban environment for intelligent city monitoring, planning, and management. In this article, we focus on a classification and super-resolution coupling (CSC) problem in RUS applications, where the goal is to explore the interdependence between two critical tasks (i.e.,classificationandsuper-resolution) to concurrently boost the performance of both the tasks. Two fundamental challenges exist in solving our problem: 1) it is challenging to obtain accurate classification results and generate high-quality reconstructed images without knowing either of thema prioriand 2) the noise embedded in the image data could be amplified infinitely by the complex interdependence and coupling between the two tasks. To address these challenges, we developSCLearn, a novel deep convolutional neural network architecture, to couple the classification task with the super-resolution task in an integrated learning framework toconcurrentlyboost the performance of both the tasks. The evaluation results on a real-world RUS application over two different cities in Europe (Barcelona and Berlin) show that SCLearn consistently outperforms the state-of-the-art baselines by simultaneously achieving better land usage classification accuracy and higher reconstructed image quality under various application scenarios.

Social sensing has emerged as a pervasive and scalable sensing paradigm to obtain timely information of the physical world from "human sensors". In this paper, we study a new extremely-sparse disaster damage assessment (DBA) problem in social sensing. The objective is to automatically assess the damage severity of affected areas in a disaster event by leveraging the imagery data reported on online social media with extremely sparse training data (e.g., only 1% of the data samples have labels). Our problem is motivated by the limitation of current DDA solutions that often require a significant amount of high-quality training data to learn an effective DDA model. We identify two critical challenges in solving our problem: i) it remains to be a fundamental challenge on how to effectively train a reliable DDA model given the lack of sufficient damage severity labels; ii) it is a difficult task to capture the excessive and fine-grained damage-related features in each image for accurate damage assessment. In this paper, we propose ContrastDDA, a deep contrastive learning approach to address the extremely-sparse DDA problem by designing an integrated contrastive and augmentative neural network architecture for accurate disaster damage assessment using the extremely sparse training samples. The evaluation results on two real-world DDA applications demonstrate that ContrastDDA clearly outperforms state-of-the-art deep learning and semi-supervised learning baselines with the highest DDA accuracy under different application scenarios.

Human face images represent a rich set of visual data information that is utilized by various big data driven human facial applications. However, the performance of these applications is usually biased towards the majority demographic group due to the data imbalance issue. In this paper, we focus on a fair human face data exchange problem where the goal is to exchange visual features of human face images between different human face datasets and obtain a set of augmented datasets that improve the fairness and performance of human facial applications. Our problem is motivated by the limitations of current fairness approaches that only focus on a single human face dataset from a particular application and require a large amount of pre-annotated demographic attribute labels to develop fair human facial models. To address these limitations, we develop ExgFair, a crowdsourcing-based fair data exchange framework to generate a set of augmented fair face image datasets by leveraging the crowdsourced demographic attribute labels of human face images. We evaluate ExgFair using a set of real-world human face image datasets with different demographic distributions. The results show that ExgFair not only reduces demographic biases of the datasets but also improves the accuracy of human facial applications trained on the augmented fair datasets.

This paper studies an emerging and important problem of identifying misleading COVID-19 short videos where the misleading content is jointly expressed in the visual, audio, and textual content of videos. Existing solutions for misleading video detection mainly focus on the authenticity of videos or audios against AI algorithms (e.g., deepfake) or video manipulation, and are insufficient to address our problem where most videos are user-generated and intentionally edited. Two critical challenges exist in solving our problem: i) how to effectively extract information from the distractive and manipulated visual content in TikTok videos? ii) How to efficiently aggregate heterogeneous information across different modalities in short videos? To address the above challenges, we develop TikTec, a multimodal misinformation detection framework that explicitly exploits the captions to accurately capture the key information from the distractive video content, and effectively learns the composed misinformation that is jointly conveyed by the visual and audio content. We evaluate TikTec on a real-world COVID- 19 video dataset collected from TikTok. Evaluation results show that TikTec achieves significant performance gains compared to state-of-the-art baselines in accurately detecting misleading COVID-19 short videos.

Social sensing is emerging as an effective and pervasive sensing paradigm to collect timely data and observations from human sensors. This paper focuses on the problem of COVID-19 misinformation detection on social media. Our work is motivated by the lack of COVID-specific knowledge in current misinformation detection solutions, which is critical to assess the truthfulness of social media claims about the emerging COVID-19 disease. In this paper, we leverage human intelligence on a crowdsourcing platform to obtain essential knowledge facts for detecting the COVID-19 misinformation on social media. Two critical challenges exist in solving our problem: i) how to efficiently acquire accurate and timely knowledge that is both inclusive and specific to COVID-19? ii) How to effectively coordinate the efforts from both expert and non-expert workers to detect COVID-19 misinformation? To address these challenges, we develop FakeSens, a social sensing based crowd knowledge graph approach that explicitly explores the knowledge facts specific to COVID-19 and models the reliability of different types of crowd workers to capture the misleading COVID-19 claims. Evaluation results on a real-world dataset show that FakeSens significantly outperforms state-of-the-art baselines in accurately detecting misleading claims of COVID-19 on social media.

Social airborne sensing (SAS) is emerging as a new sensing paradigm that leverages the complementary aspects of social sensing and airborne sensing (i.e., UAVs) for reliable information collection. In this paper, we present HeteroSAS, a heterogeneous resource management framework for "all-in-the-air" SAS in disaster response applications. Current SAS approaches use UAVs to only capture data, but carry out computation on ground-based processing nodes that may be unavailable in disaster scenarios and thus consider a single model of UAV along with only one type of task (i.e., data capture). In this paper, we explore the opportunity to exploit the complementary strengths of different UAV models to accomplish all stages of sensing tasks (i.e., data capturing, maneuvering, and computation) exclusively "in-the-air". However, several challenges exist in developing such a resource management framework: i) handling the uncertain social signals in presence of the heterogeneity of UAVs and tasks; and ii) adapting to constantly changing cyber-physical-social environments. The HeteroSAS framework addresses these challenges by building a novel resource management framework that observes the environment and learns the optimal strategy for each UAV using techniques from multi-agent reinforcement learning, game theory, and ensemble learning. The evaluation with a real-world case study shows that HeteroSAS outperforms the state-of-the-art in terms of detection effectiveness, deadline hit rate, and robustness on heterogeneity.

This paper studies a critical problem of identifying offensive meme posts on online social media where images are superimposed with deliberately altered or crafted captions to deliver offensive information. Existing solutions often ignore the implicit relationship between visual and textual contents in the meme and their implied meanings, which are critical to accurately detect offensive memes. Two important challenges exist in solving the problem: i) how to effectively incorporate human commonsense knowledge to capture the symbolic or implied meaning of the meme contents that implicitly deliver offensive information? ii) How to accurately identify the cross-modal knowledge-based relations between entities in both the visual and textual content of the meme that jointly insinuate offensive messages? To address the above challenges, we develop KnowMeme, a knowledge-enriched graph neural network solution that leverages knowledge facts from human commonsense knowledge to effectively detect offensive meme posts on online social media. Evaluation results show that KnowMeme achieves significant performance gains compared to the state-of-the-art baseline methods in accurately detecting offensive memes.

Social sensing has emerged as a pervasive and scalable sensing paradigm to collect observations of the physical world from human sensors. A key advantage of social sensing is its infrastructure-free nature. In this paper, we focus on a streaming urban infrastructure monitoring (Streaming UIM) problem in social sensing. The goal is to automatically detect the urban infrastructure damages from the streaming imagery data posted on social media by exploring the collective power of both AI and human intelligence from crowdsourcing systems. Our work is motivated by the limitation of current AI and crowdsourcing solutions that either fail in many critical time-sensitive UIM application scenarios or are not easily generalizable to monitor the damage of different types of urban infrastructures. We identify two critical challenges in solving our problem: i) it is difficult to dynamically integrate AI and crowd intelligence to effectively identify and fix the failure cases of AI solutions; ii) it is non-trivial to obtain accurate human intelligence from unreliable crowd workers in streaming UIM applications. In this paper, we propose StreamCollab, a streaming crowd-AI collaborative system that explores the collaborative intelligence from AI and crowd to solve the streaming UIM problem. The evaluation results on a real-world urban infrastructure imagery dataset collected from social media demonstrate that StreamCollab consistently outperforms both state-of-the-art AI and crowd-AI baselines in UIM accuracy while maintaining the lowest computational cost.

The advance of mobile sensing and edge computing has brought new opportunities for abnormal health detection (AHD) systems where edge devices such as smartphones and wearable sensors are used to collect people's health information and provide early alerts for abnormal health conditions such as stroke and depression. The recent development of federated learning (FL) allows participants to collaboratively train powerful AHD models while keeping their health data private to local devices. This paper targets at addressing a critical challenge of adapting FL to train AHD models, where the participants' health data is highly imbalanced and contains biased class distributions. Existing FL solutions fail to address the class imbalance issue due to the strict privacy requirements of participants as well as the heterogeneous resource constraints of their edge devices. In this work, we propose FedSens, a new FL framework dedicated to address the class imbalance problem in AHD applications with explicit considerations of participant privacy and device resource constraints. We evaluate FedSens using a real-world edge computing testbed on two real-world AHD applications. The results show that FedSens can significantly improve the accuracy of AHD models in the presence of severe class imbalance with low energy cost to the edge devices.

In the domain of smart city applications, anomalistic crowd investigation in public places is a crucial task for the efficient management of unforeseen crowds. Many of the current approaches for crowd monitoring utilize physical sensor-based tracking (e.g., cellular signals, cameras, RFID beacons) to detect abrupt crowds. Unfortunately, these solutions are highly application-specific and are able to only detect the mere presence of crowds without determining their causes that may reveal the actual story behind gatherings that occur suddenly. In this paper, we explore the opportunity to meld two complementary sensing paradigms: social sensing and unmanned aerial vehicle (UAV)-based physical sensing to scrutinize abrupt crowd events occurring in public places. However, two technical challenges exist in developing our solution: i) handling the data sparsity across social sensing platforms and within video footage from UAVs; and ii) the accurate inference of the cause of crowd events from the collected data. To address these challenges, we develop Unravel, an intelligent social airborne sensing (SAS) framework that leverages interdisciplinary techniques from natural language processing (NLP), object recognition, and estimation theory for a comprehensive anomalistic crowd investigation. The evaluation results with a real-world anomalous crowd instance dataset show that the Unravel framework outperforms the state-of-the-art baselines in accurately validating the existence of anomalistic crowd instances in public areas and deriving their causes.

Human face image is a large category of visual information utilized by various human facial data services (e.g., face recognition, face generation, face attribute prediction). However, the quality of data services (QoDS) on human face datasets is usually biased towards the majority demographic group due to the data imbalance issue. In this paper, we focus on a fair human face dataset sampling problem where the goal is to sample a sub-dataset from the original dataset to reduce its bias by leveraging crowd intelligence to infer the demographic labels of face images (e.g., male or female, old or young). Our problem is motivated by the limitations of current fair data sampling solutions that require pre-annotated demographic labels to sample a fair dataset. Two important challenges exist in solving our problem: 1) it is extremely time-consuming and expensive to assign crowd workers to annotate demographic labels of all images in a large-scale facial dataset; 2) it is not a trivial task to improve the fairness of the sampled sub-dataset (with fewer data samples) without sacrificing the accuracy performance of data services on such dataset. To address the above challenges, we develop FairCrowd, a fair crowdsourcing-based data sampling framework that leverages an efficient batch-level demographic label inference model and a joint fair-accuracy-aware data shuffling method. We evaluate the performance of FairCrowd through a large-scale real-world face image dataset that consists of celebrity faces from a diversified set of demographic groups. The results show that FairCrowd not only reduces demographic bias but also improves the accuracy of data services trained on the sub-dataset generated by FairCrowd, leading to a more desirable QoDS of the application.

Streaming disaster damage assessment (DDA) aims to automatically assess the damage severity of affected areas in a disaster event on the fly by leveraging the streaming imagery data about the disaster on social media. In this paper, we focus on a dynamic optimal neural architecture searching (NAS) problem in streaming DDA applications. Our goal is to dynamically determine the optimal neural network architecture that accurately estimates the damage severity for each newly-arrived image in the stream by leveraging human intelligence from the crowdsourcing systems. Our work is motivated by the observations that the neural network architectures in current DDA solutions are mainly designed by AI experts, which often leads to non-negligible costs and errors given the dynamic nature of the streaming DDA applications and the lack of real-time annotations of the massive social media data inputs. In this paper, we develop CD-NAS, a crowd-driven dynamic NAS framework that is inspired by novel techniques from AI, crowdsourcing, and estimation theory to address the dynamic optimal NAS problem. The evaluation results from a real-world streaming DDA application show that CD-NAS consistently outperforms the state-of-the-art AI and NAS baselines by achieving the highest disaster damage assessment accuracy.

Image-driven smart urban sensing (ISUS) has emerged as a powerful sensing paradigm to capture abundant visual information about the urban environment for intelligent city monitoring, planning, and management. In this paper, we focus on a Classification and Super-resolution Coupling (CSC) problem in ISUS applications, where the goal is to explore the interdependence between two critical tasks (i.e., classification and super-resolution) to concurrently boost the Quality of Service (QoS) of both tasks. Previous efforts often focus on solving the two tasks individually and ignore the opportunity to explore the interdependence between them. In this paper, we develop SuperClass, a deep duo-task learning framework, to effectively integrate the classification and super-resolution tasks into a holistic network design that jointly optimizes the QoS of both tasks. The evaluation results on a real-world ISUS application show that SuperClass consistently outperforms state-of-the-art baselines by simultaneously achieving better land usage classification accuracy and higher reconstructed image quality.

The proliferation of online Location-Based Social Networks (LBSN) has offered unprecedented opportunities for understanding fine-grained spatio-temporal behaviors of users and developing new location-aware applications. In this article, we focus on the problem of “Sparse User Check-in Venue Prediction,” where the goal is to predict the next venue LBSN users will visit by exploiting their sparse online check-in traces and the latent decision contexts. While efforts have been made to predict users’ check-in traces on a LBSN, several important challenges still exist. First, check-in traces contributed by LBSN users are often too sparse to provide sufficient evidence for a reliable prediction, especially when the prediction space is huge (e.g., hundreds of thousands of venues in large cities). Second, the user's decision context on which venue to visit next is often latent and has not been incorporated by current venue prediction models. Third, the dynamic and non-deterministic dependency between check-ins is either ignored or replaced by a simplified “consecutiveness” assumption in existing solutions, leading to sub-optimal prediction results. In this article, we develop aContext-aware Sparse Check-in Venue Prediction (CSCVP)scheme inspired by natural language processing techniques to address the above challenges. In particular, CSCVP predicts the venue category information and explores the similarity between users to address data sparsity challenge by significantly reducing the prediction space. It also leverages the Probabilistic Latent Semantic Analysis (PLSA) model to incorporate the user decision context into the prediction model. Finally, we develop a novel Temporal Adaptive Ngram (TA-Ngram) model in CSCVP to capture the dynamic and non-deterministic dependency between check-ins. We evaluate CSCVP using three real-world LBSN datasets. The results show that our scheme significantly improves accuracy (30.9 percent improvement) of the state-of-the-art user check-in venue prediction solutions.

The recent advances of mobile sensing and artificial intelligence (AI) have brought new revolutions in disaster response applications. One example is disaster scene assessment (DSA) which leverages computer vision techniques to assess the level of damage severity of the disaster events from images provided by eyewitnesses on social media. The assessment results are critical in prioritizing the rescue operations of the response teams. While AI algorithms can significantly reduce the detection time and manual labeling cost in such applications, their performance often falls short of the desired accuracy. Our work is motivated by the emergence of crowdsourcing platforms (e.g., Amazon Mechanic Turk, Waze) that provide unprecedented opportunities for acquiring human intelligence for AI applications. In this paper, we develop an interactive Disaster Scene Assessment (iDSA) scheme that allows AI algorithms to directly interact with humans to identify the salient regions of the disaster images in DSA applications. We also develop new incentive designs and active learning techniques to ensure reliable, timely, and cost-efficient responses from the crowdsourcing platforms. Our evaluation results on real-world case studies during Nepal and Ecuador earthquake events demonstrate that iDSA can significantly outperform state-of-the-art baselines in accurately assessing the damage of disaster scenes.

With the ubiquitous network connectivity and the proliferation of mobile devices, people are increasingly consuming digital contents from social media driven music sharing platforms (e.g., YouTube, Soundcloud). In this paper, we study a novel problem of connotation-aware music retrieval that focuses on the connotation which expresses the implicit feeling or emotion beyond the explicit content in artworks. Our goal is to automatically retrieve relevant music on social media based on the connotation of visual inputs (e.g., images, photos) provided by the users. The problem is challenging as it requires the accurate identification of the implicit connotation from both images and music pieces, and the precise matching of the identified connotation across different data modalities. We develop a connotation-aware music retrieval (CaMR) framework to address the above challenges. Evaluation results from a real-world social media dataset demonstrate that the CaMR framework can retrieve music that is highly relevant to the connotation of the input image.

Social media sensing has emerged as a powerful sensing paradigm to collect the observations of the physical world by exploring the “wisdom of crowd”. In this paper, we focus on a migratable disaster damage assessment problem in social media sensing applications. Our goal is to accurately identify the damage severity of affected areas in an unfolding disaster event using unlabeled social media data feeds (e.g., image posts on social media). Two fundamental challenges exist in solving our problem: i) different disaster events often have distinct characteristics (e.g., damage types, affected areas) that cannot be easily migrated; ii) it is non-trivial to modify a damage assessment model from a previous event to adapt to a new event without using the labeled data from the new event. To address the above challenges, we develop SocialTrans, a hybrid deep transfer learning framework, to enable effective model migration for accurate damage assessment without using any training data from the studied disaster event. The evaluation results on four real-world disaster events show that SocialTrans consistently outperforms the state-of-the-art baselines in accurately assessing the damage level of disasters.

Fauxtography is a category of multi-modal posts that spreads misleading information on various online social platforms (e.g., Facebook, Twitter, Reddit). A fauxtography post usually consists of an image, a text description and comments from its readers. In this paper, we focus on an explainable fauxtography detection problem where the goal is to explain which a specific component of a post leads to the fauxtography decision. This problem is motivated by the limitations of current fauxtography detection solutions that only focus on the detection but ignore the important explanation aspect of their results. Two critical challenges exist in solving our problem: i) it is difficult to accurately identify the "guilty" component of a fauxtography post given the fact that different components of the post and their associations could all lead to the fauxtography; ii) it is expensive and time-consuming to obtain a good training set with fine-grained labels of fauxtography posts in terms of explainability, making the corresponding solutions weakly supervised in nature. To address the above challenges, we develop ExFaux, an end-to-end graph-based fauxtography explanation framework, to effectively explain which part of the post contributes to its fauxtography. We evaluate the ExFaux by creating a real-world dataset from online social media (Twitter and Reddit). The results show that ExFaux not only detects the fauxtography posts more accurately than the state-of-the-arts but also provides well-justified explanations to its results.

The recent advance of the federated learning (FL) has brought new opportunities for privacy-aware distributed machine learning (ML) applications to train a powerful ML model without accessing the private training data of the participants. In this paper, we focus on addressing a novel fair classification problem in FL where the model trained by FL displays discriminatory bias towards particular demographic groups. Addressing the fairness issue in a FL framework posts three critical challenges: fairness and performance trade-offs, restricted information, and constrained coordination. To address these challenges, we develop FairFL, a fair federated learning framework dedicated to reducing the bias in privacy-sensitive ML applications. It consists of a principled deep multi-agent reinforcement learning framework and a secure information aggregation protocol that optimizes both the accuracy and the fairness of the learned model while respecting the strict privacy constraints of the clients. Evaluation results on real-world applications showed that FairFL can achieve significant performance gains in both fairness and accuracy of the learned model compared to state-of-the-art baselines.

Forest fires cause irreversible damages worldwide every year. Monitoring wildfire propagation is thus a vital task in mitigating forest fires. While computational model-based wildfire prediction methods provide reasonable accuracy in monitoring wildfire behavior, they are often limited due to the lack of constant availability of real-time meteorological data. In contrast, social-media-driven drone sensing (SDS) is emerging as a new sensing paradigm that detects the early signs of forest fires from online social media feeds and drives the drones for reliable sensing. However, due to the scarcity of social media data in remote regions and limited flight times of drones, SDS solutions often underperform in large-scale forest fires. In this paper, we present CompDrone, a wildfire monitoring framework that exploits the collective strengths of computational wildfire modeling and SDS for reliable wildfire monitoring. Two critical challenges exist to integrate computational modeling and SDS together: i) limited availability of social signals in the regions of a forest fire; and ii) predicting the regions of fire where the drones should be dispatched to. To solve the above challenges, the CompDrone framework leverages techniques from cellular automata, constrained optimization, and game theory. The evaluation results using a real-world wildfire dataset show that CompDrone outperforms the state-of-the-art schemes in effectively predicting wildfire propagation.

Social media sensing has emerged as a new disaster response application paradigm to collect real-time observations from online social media users about the disaster status. Due to the noisy nature of social media data, the task of identifying trustworthy information (referred to as "truth discovery") has been a crucial task in social media sensing. However, existing truth discovery solutions often fall short of providing accurate results in disaster response applications due to the spread of misinformation and difficulty of an efficient verification in such scenarios. In this paper, we present SocialDrone, a novel closed-loop social-physical active sensing framework that integrates social media and unmanned aerial vehicles (UAVs) for reliable disaster response applications. In SocialDrone, signals emitted from the social media are distilled to drive the drones to target areas to verify the emergency events. The verification results are then taken back to improve the sensing and distillation process on social media. The SocialDrone framework introduces several unique challenges: i) how to drive the drones using the unreliable social media signals? ii) How to ensure the system is adaptive to the high dynamics from both the physical world and social media? iii) How to incorporate real-world constraints (e.g., the deadlines of events, limited number of drones) into the framework? The SocialDrone addresses these challenges by building a novel integrated social-physical sensing system that leverages techniques from game theory, constrained optimization, and reinforcement learning. The evaluation results on a real-world disaster response application show that SocialDrone significantly outperforms state-of-the-art truth discovery schemes and drone-only solutions by providing more effective disaster response.

Recent advances in remote sensing open up unprecedented opportunities to obtain a rich set of visual features of objects on the earth's surface. In this paper, we focus on a single-image super-resolution (SISR) problem in remote sensing, where the objective is to generate a reconstructed satellite image of high quality (i.e., a high spatial resolution) from a satellite image of relatively low quality. This problem is motivated by the lack of high quality satellite images in many remote sensing applications (e.g., due to the cost of high resolution sensors, communication bandwidth constraints, and historic hardware limitations). Two important challenges exist in solving our problem: i) it is not a trivial task to reconstruct a satellite image of high quality that meets the human perceptual requirement from a single low quality image; ii) it is challenging to rigorously quantify the uncertainty of the results of an SISR scheme in the absence of ground truth data. To address the above challenges, we develop PQA-CNN, a perceptual quality-assured conventional neural network framework, to reconstruct a high quality satellite image from a low quality one by designing novel uncertainty-driven neural network architectures and integrating an uncertainty quantification model with the framework. We evaluate PQA-CNN on a real-world remote sensing application on land usage classifications. The results show that PQA-CNN significantly outperforms the state-of-the-art super-resolution baselines in terms of accurately reconstructing high-resolution satellite images under various evaluation scenarios.

Forecasting traffic accidents at a fine-grained spatial scale is essential to provide effective precautions and improve traffic safety in smart urban sensing applications. Current solutions primarily rely on complete historical traffic accident records and/or accurate real-time traffic sensor data for traffic risk prediction. These solutions are prone to various limitations (e.g., facility availability, privacy and legal constraints). In this paper, we address those limitations by exploring two types of widely available and complementary data sources: social media sensing and remote sensing. However, three important technical challenges exist: i) the social media data and remote sensing data often have different data modalities and characteristics, which makes it difficult to effectively fuse them for traffic risk forecasting; ii) the social media data is often noisy, providing fuzzy evidence to capture the spatial-temporal dynamics of traffic accident occurrences; iii) the remote sensing data often contains complex visual features, making it a non-trivial task to explore the accident occurrence correlation between different locations. To address the above challenges, we propose GraphCast, a multi-modal graph neural network framework to accurately forecast the traffic risks in a city by jointly exploring the social media sensing and remote sensing paradigms. The evaluation results on a real-world case study in New York City show that our GraphCast scheme significantly outperforms the state-of-the-art conventional and deep learning baselines in accurately forecasting the traffic risks in a city.

Recent advances in remote sensing provide a powerful and scalable sensing paradigm to capture abundant visual information about the urban environments. We refer to such a sensing paradigm as remote urban sensing. In this paper, we focus on a migratable satellite image super-resolution problem in remote urban sensing applications. Our goal is to reconstruct satellite images of a high resolution in a target area where the high-resolution training data is not available by transferring a super-resolution model learned in a source area where such data is available. This problem is motivated by the limitation of current solutions that primarily rely on a rich set of high-resolution satellite images in the studied area that are not always available. Two important challenges exist in solving our problem: i) the target and source areas often have very different urban characteristics that prevent the direct application of a super-resolution model learned from the source area to the target area; ii) it is not a trivial task to ensure effective model migration with desirable quality without sufficient high quality training data. To address the above challenges, we develop TransRes, a deep adversarial transfer learning framework, to effectively reconstruct high-resolution satellite images without requiring any ground-truth training data from the studied area. We evaluate the TransRes framework using the real-world satellite imagery data collected from three different cities in Europe. The results show that TransRes consistently outperforms the state-of-the-art baselines by achieving the lowest perception errors under various application scenarios.

With the advance in mobile computing, Internet of Things, and ubiquitous wireless connectivity, social sensing-based edge computing (SSEC) has emerged as a new computation paradigm where people and their personally owned devices collect sensor measurements from the physical world and process them at the edge of the network. This article focuses on a privacy-aware task allocation problem where the goal is to optimize the computation task allocation in SSEC systems while respecting the users' customized privacy settings. It introduces a novel game-theoretic privacy-aware task allocation (G-PATA) framework to achieve the goal. G-PATA includes: 1) a bottom-up game-theoretic model to generate the maximum payoffs at end devices while satisfying the end user's privacy settings and 2) a top-down incentive scheme to adjust the rewards for the tasks to ensure that the task allocation decisions made by end devices meet the Quality-of-Service (QoS) requirements of the applications. Furthermore, the framework incorporates an efficient load balancing and iteration reduction component to adapt to the dynamic changes in status and privacy configurations of end devices. The G-PATA framework was implemented on a real-world edge computing platform that consists of heterogeneous end devices (Jetson TX1 and TK1 boards, and Raspberry Pi3). We compare G-PATA with state-of-the-art task allocation schemes through two real-world social sensing applications. The results show that G-PATA significantly outperforms existing approaches under various privacy settings (our scheme achieved as much as 47% improvements in delay reduction for the application and 15% more payoffs for end devices compared to the baselines).

Social sensing is a new big data application paradigm for Cyber-Physical Systems (CPS), where a group of individuals volunteer (or are recruited) to report measurements or observations about the physical world at scale. A fundamental challenge in social sensing applications lies in discovering the correctness of reported observations and reliability of data sources without prior knowledge on either of them. We refer to this problem as truth discovery. While prior studies have made progress on addressing this challenge, two important limitations exist: (i) current solutions did not fully explore the uncertainty aspect of human reported data, which leads to sub-optimal truth discovery results; (ii) current truth discovery solutions are mostly designed as sequential algorithms that do not scale well to large-scale social sensing events. In this paper, we develop a Scalable Uncertainty-Aware Truth Discovery (SUTD) scheme to address the above limitations. The SUTD scheme solves a constraint estimation problem to jointly estimate the correctness of reported data and the reliability of data sources while explicitly considering the uncertainty on the reported data. To address the scalability challenge, the SUTD is designed to run a Graphic Processing Unit (GPU) with thousands of cores, which is shown to run two to three orders of magnitude faster than the sequential truth discovery solutions. In evaluation, we compare our SUTD scheme to the state-of-the-art solutions using three real world datasets collected from Twitter: Paris Attack, Oregon Shooting, and Baltimore Riots, all in 2015. The evaluation results show that our new scheme significantly outperforms the baselines in terms of both truth discovery accuracy and execution time.

A user's localness (i.e., whether a user is a local resident in a city or not) and a venue's local attractiveness (i.e., the likelihood of a venue to attract local people) are important information for many location-based applications related with Cyber-Physical Systems (CPS), such as participatory sensing, urban planning, traffic control and localized travel recommendations. Previous effort has been devoted to geo-locating users in a city using supervised learning approaches, which depend on the availability of high quality training datasets. However, it is difficult to obtain such training datasets in the real-world CPS applications due to the issue of privacy. In this work, we develop an unsupervised approach, called a Physical-Social-Aware Inference (PSAI) scheme, to jointly infer a user's localness and a venue's local attractiveness by exploring both the physical and social information embedded in the location-based social networks (LBSN). We further implement a parallel PSAI framework on the platform of a Graphic Processing Unit (GPU) to enhance its ability to process large-scale data. Our extensive experiments on the real-world LBSN datasets demonstrate the effectiveness and efficiency of the PSAI scheme compared to the state-of-the-art baselines.

Social sensing has emerged as a new sensing paradigm to observe the physical world by exploring the "wisdom of crowd" on social media. This paper focuses on the abnormal traffic event localization problem using social media sensing. Two critical challenges exist in the state-of-the-arts: i) "content-only inference": the limited and unstructured content of a social media post provides little clue to accurately infer the locations of the reported traffic events; ii) "informal and scarce data": the language of the social media post (e.g., tweet) is informal and the number of the posts that report the abnormal traffic events is often quite small. To address the above challenges, we develop SyntaxLoc, a syntax-based probabilistic learning framework to accurately identify the location entities by exploring the syntax of social media content. We perform extensive experiments to evaluate the SyntaxLoc framework through real world case studies in both New York City and Los Angeles. Evaluation results demonstrate significant performance gains of the SyntaxLoc framework over state-of-the-art baselines in terms of accurately identifying the location entities that can be directly used to locate the abnormal traffic events.

With the increasing popularity of portable devices with cameras (e.g., smartphones and tablets) and ubiquitous Internet connectivity, travelers can share their instant experience during the travel by posting photos they took to social media platforms. In this paper, we present a new image-driven poetry recommender system that takes a traveler's photo as input and recommends classical poems that can enrich the photo with aesthetically pleasing quotes from the poems. Three critical challenges exist to solve this new problem: i) how to extract the implicit artistic conception embedded in both poems and images? ii) How to identify the salient objects in the image without knowing the creator's intent? iii) How to accommodate the diverse user perceptions of the image and make a diversified poetry recommendation? The proposed iPoemRec system jointly addresses the above challenges by developing heterogeneous information network and neural embedding techniques. Evaluation results from real-world datasets and a user study demonstrate that our system can recommend highly relevant classical poems for a given photo and receive significantly higher user ratings compared to the state-of-the-art baselines.

Road traffic accidents are a major challenge in urban transportation systems. An effective countermeasure to address this problem is to accurately forecast the traffic risks in a city before accidents actually happen. Current traffic accident prediction solutions largely rely on accurate data collected from infrastructure-based sensors, which is not always available due to various resource constraints or privacy and legal concerns. In this paper, we address this limitation by exploring social sensing, a new sensing paradigm that uses humans as sensors to report the states of the physical world. In particular, we consider two types of publicly available social sensing data sources: social media data (e.g., traffic posts on Twitter) and open city data (e.g., traffic data from the city web portal). In this paper, we develop the RiskCast, an inductive multi-view learning approach to accurately forecast the traffic risk by exploiting the social sensing data under a principled co-regularization framework. The evaluation results on a real world dataset from New York City show that RiskCast significantly outperforms the state-of-the-art baselines in forecasting the traffic risks in a city.

With the increasing popularity of online video platforms (e.g., YouTube, Vimeo), the spread of hateful videos and the lack of rigorous hateful content control have become a critical issue. This paper focuses on the problem of identifying online hatred-vulnerable videos where the videos themselves do not contain any hateful content but unexpectedly trigger hateful comments from the audience. It is suboptimal to simply treat the hatred-vulnerable videos as hateful ones and remove them from the sharing platforms. This will discourage the uploaders of such videos from sharing valid and informative videos in the future. However, treating these hatred-vulnerable videos as hatred-free ones will provide undesirable opportunities for hateful users to spread their toxic comments and extreme ideology. In this paper, we develop VulnerCheck, an end-to-end supervised learning approach to effectively classify hatred-vulnerable videos from hateful and hatred-free ones by exploring the structure and semantics features of audience's comment networks. VulnerCheck is content-agnostic in the sense that it does not analyze the content of the video and is therefore robust against sophisticated content creators who craft hateful videos to bypass the current content censorship. We evaluate VulnerCheck on a real-world dataset collected from YouTube. Results demonstrate that our scheme is both effective and efficient in identifying hatred-vulnerable videos and significantly outperforms the state-of-the-art baselines.

Urban land usage classification is a critical task in big data based smart city applications that aim to understand the social-economic land functions and physical land attributes in urban environments. This paper focuses on a migratable urban land usage classification problem using remote sensing data (i.e., satellite images). Our goal is to accurately classify the land usage of locations in a target city where the ground truth land usage data is not available by leveraging a classification model from a source city where such data is available. This problem is motivated by the limitation of current solutions that primarily rely on a rich set of ground-truth data for accurate model training, which encounters high annotation costs. Two important challenges exist in solving our problem: i) the target and source cities often have different urban characteristics that prevent the direct application of a model learned from the source city to the target city; ii) the complex visual features in satellite images make it non-trivial to “translate” the images from the target city to the source city for an accurate classification. To address the above challenges, we develop TransLand, an adversarial transfer learning framework to translate the satellite images from the target city to the source city for accurate land usage classification. We evaluate our scheme on the real-world satellite imagery and land usage datasets collected from live different cities in Europe. The results show that TransLand significantly outperforms the state-of-the-art land usage classification baselines in classifying the land usage of locations in a city.

This paper focuses on the migratable traffic risk estimation problem in intelligent transportation systems using the social (human-centric) sensing. The goal is to accurately estimate the traffic risk of a target area where the ground truth traffic accident reports are not available by leveraging an estimation model from a source area where such data is available. Two important challenges exist. The first challenge lies in the discrepancy between source and target areas (e.g., layouts, road conditions, and local regulations) and such discrepancy would prevent a direct application of a model from the source area to the target area. The second challenge lies in the difficulty of identifying all potential features in the migratable traffic risk estimation problem and decide the importance of identified features due to the lack of ground truth labels in the target area. To address these challenges, we develop DeepRisk, a social sensing based migratable traffic risk estimation scheme using deep transfer learning techniques. The evaluation results on a real world dataset in New York City show the DeepRisk significantly outperforms the state-of-the-art baselines in accurately estimating the traffic risk of locations in a city.

While autonomous unmanned aerial vehicles (UAVs) have attained a reputable stance in modern disaster response applications, their practical adoptions are impeded due to various constraints (e.g., requiring manual input, battery life). In this paper, we develop a novel spatiotemporal aware drone sensing system that is driven by harnessing social media signals, a process known as social sensing. Social sensing has emerged as a new sensing paradigm where humans act as "sensors" to report their observations about the physical world. However, maneuvering drones with "social signals" introduces a new realm of challenges. The first challenge is to drive the drones by leveraging noisy and unreliable social media signals. The second challenge is to optimize the drone deployment by exploring the highly dynamic and latent correlations among event locations. In this paper, we present CollabDrone that devises a new spatiotemporal correlation inference model and game-theoretic drone dispatching mechanism to address the above challenges. The evaluation results on a real-world case study show that CollabDrone significantly outperforms current drone and social sensing baselines in terms of accuracy and deadline hit rate.

This paper overviews the state of the art, research challenges, and future opportunities in an emerging research direction: Social Sensing based Edge Computing (SSEC). Social sensing has emerged as a new sensing application paradigm where measurements about the physical world are collected from humans or from devices on their behalf. The advent of edge computing pushes the frontier of computation, service, and data along the cloud-to-things continuum. The merging of these two technical trends generates a set of new research challenges that need to be addressed. In this paper, we first define the new SSEC paradigm that is motivated by a few underlying technology trends. We then present a few representative real-world case studies of SSEC applications and several key research challenges that exist in those applications. Finally, we envision a few exciting research directions in future SSEC. We hope this paper will stimulate discussions of this emerging research direction in the community.

Artificial Intelligence (AI) has been widely adopted in many important application domains such as speech recognition, computer vision, autonomous driving, and AI for social good. In this paper, we focus on the AI-based damage assessment applications where deep neural network approaches are used to automatically identify damage severity of impacted areas from imagery reports in the aftermath of a disaster (e.g., earthquake, hurricane, landslides). While AI algorithms often significantly reduce the detection time and labor cost in such applications, their performance sometimes falls short of the desired accuracy and is considered to be less reliable than domain experts. To exacerbate the problem, the black-box nature of the AI algorithms also makes it difficult to troubleshoot the system when their performance is unsatisfactory. The emergence of crowdsourcing platforms (e.g., Amazon Mechanic Turk, Waze) brings about the opportunity to incorporate human intelligence into AI algorithms. However, the crowdsourcing platform is also black-box in terms of the uncertain response delay and crowd worker quality. In this work, we propose the CrowdLearn, a crowd-AI hybrid system that leverages the crowdsourcing platform to troubleshoot, tune, and eventually improve the black-box AI algorithms by welding crowd intelligence with machine intelligence. The system is specifically designed for deep learning-based damage assessment (DDA) applications where the crowd tend to be more accurate but less responsive than machines. Our evaluation results on a real-world case study on Amazon Mechanic Turk demonstrate that CrowdLearn can provide timely and more accurate assessments to natural disaster events than the state-of-the-art AI-only and human-AI integrated systems.

Autonomous unmanned aerial vehicles (UAVs) have become an important tool for efficient disaster response. Despite the virtues of UAVs in disaster response applications, various limitations (e.g., requiring manual input, finite battery life) hinder their mass adoption. In contrast, social sensing is emerging as a new sensing paradigm that utilizes signals provided by "human sensors" to gather awareness of the events occurring in the physical world. Despite being inherently broader in scope, a shortcoming of social sensing is the reliability of the sensing data that are contributed by humans. In this paper, we introduce the concept of jointly exploiting the reliability of drones and the scope of social sensing to efficiently uncover the truthful events during disasters. However, such a tight integration of social and physical sensing introduces several technical challenges. The first challenge is satisfying the conflicting objectives of event coverage of the application and energy conservation of drones. The second challenge is adapting to the dynamics of the physical world and social media. In this paper, we present a Social-media-driven Energy-Aware Drone (SEAD) sensing framework to address the above challenges. In particular, we develop a reinforcement learning-based drone dispatching scheme that adapts to the physical and social environments and launches an appropriate proportion of drones for event exploration. We further utilize a bottom-up game-theoretic task allocation approach to guide drones effectively to the event locations. The evaluation with a real-world disaster case study show that SEAD noticeably outperforms state-of-the-art baselines in terms of detection effectiveness and energy efficiency.

With the advance of mobile computing, Internet of Things, and 5G networks, social sensing based edge computing (SSEC) systems have emerged as a new computation paradigm where people and their personally owned devices collect and process sensing measurements about the physical world at the edge of networks. In this paper, we focus on the task allocation problem in SSEC where rational edge devices are motivated by incentives to collectively accomplish the computation tasks in the system. Several unique challenges exist to solve this problem: (i) the edge devices often do not share the complete context information (e.g., CPU, memory usage) in the task allocation process due to privacy concerns; (ii) the edge devices are rational actors who may have competing objectives with the application; (iii) the application server and edge devices are usually owned by different entities, making the coordination in task allocation more challenging. This paper develops a novel integrated Top-Down and Bottom-Up (TDBU) task allocation framework to address these challenges. In particular, TDBU incorporates abottom-up game-theoretic model that allows the edge devices to specify their task preferences in a way that maximizes their payoffs. It also incorporates atop-down control model that ensures the performance of the applications using control theory. The TDBU was implemented on a real-world edge computing testbed that consists of heterogeneous devices (Jetson TX1, TK1 boards, Raspberry Pi3). We compared the performance of TDBU with state-of-the-art baselines through a real-world social sensing application. The results showed that our solution significantly outperformed the baselines in various application settings.

Vehicular sensor networks (VSNs) have become a reliable sensing instrument utilizing sensors built into cars. However, VSNs have limited sensing scopes since car drivers only opportunistically discover new events. Conversely, social sensing is protruding as a new sensing paradigm where measurements about the physical world are collected from humans. In this paper, we develop SocialCar, a task allocation framework for social media driven vehicular network sensing systems that exploits the collective powers of social sensing and VSNs for a robust sensing application. However, integrating VSNs with social sensing introduces a new set of challenges. The first challenge is leveraging the noisy and unreliable social signals to route vehicles to the regions of interest. The second challenge is the inconsistent availability attributed to car drivers being rational actors causing churn in the system. To address the above challenges, this paper introduces a bottom-up game-theoretic task preference model along with a top-down dynamic incentive control mechanism to allow car drivers to be selectively dispatched to event locations and verify the truthfulness of reported events. The results on a real-world disaster recovery case study show that SocialCar significantly outperforms current VSNs based solutions in detection effectiveness and deadline hit rate.

Modern Internet of Things (IoT) systems are increasingly leveraging deep neural networks (DNNs) with the goal of enabling intelligence at the edge of the network. While applying DNNs can greatly improve the accuracy of autonomous decisions and inferences, a significant challenge is that DNNs are traditionally designed and developed for advanced hardware (e.g., GPU clusters) and can not easily meet the real time requirements when deployed in a resource-constrained edge computing environment. While many systems have been proposed to facilitate deep learning at the edge, a key limitation lies in the under-utilization of the parallelizable GPU resources of edge nodes (e.g., IoT devices). In this paper, we propose EdgeBatch, a collaborative intelligent edge computing framework that minimizes the delay and energy consumption of executing DNN tasks at the edge by sharing idle GPU resources among privately owned IoT devices. EdgeBatch develops 1) a stochastic task batching mechanism that identifies the optimal batching strategy for the GPUs on IoT devices given uncertain task arrival times, and 2) a dynamic task offloading scheme that coordinates the collaboration among edge nodes to optimize the utilization of idle GPU resources in the system. We implemented EdgeBatch on a real-world edge computing testbed that consists of heterogeneous IoT devices (Jetson TX2, TX1, TK1, and Raspberry Pi3s). The results show that EdgeBatch achieved significant performance gains in terms of both the end-to-end delay and energy savings compared to the state-of-the-art baselines.

Social sensing based Edge Computing (SSEC) is an emerging application paradigm for rich context awareness in sensing applications in which people collaborate in both data acquisition and processing at the edge of the network. While keeping people in the loop can be an immense benefit, the unreliability and churn introduced can pose a dangerous stability threat for long-running SSEC applications. In the past this problem was addressed by offloading these responsibilities to more reliable server hardware in the cloud or a fog layer, however, this does not take full advantage of the power at the edge. In this paper we address the issue of reliable edge computing on dynamic high-churn edge systems. We develop a deviceless pipeline based approach (DPA) to establish workflows in which stages of the analysis pipeline are completed on edge devices, and any devices that leave the system can be replaced without data loss. We evaluate the performance of our system on a real-world edge computing system performing an object detection application and demonstrate that it can provide significant performance gains over traditional computation offloading schemes in terms of throughput and error recovery.

Identifying trustworthy information in the presence of noisy data contributed by numerous unvetted sources from online social media (e.g., Twitter, Facebook, and Instagram) has been a crucial task in the era of big data. This task, referred to as truth discovery, targets at identifying the reliability of the sources and the truthfulness of claims they make without knowing either a priori. In this work, we identified three important challenges that have not been well addressed in the current truth discovery literature. The first one is “misinformation spread” where a significant number of sources are contributing to false claims, making the identification of truthful claims difficult. For example, on Twitter, rumors, scams, and influence bots are common examples of sources colluding, either intentionally or unintentionally, to spread misinformation and obscure the truth. The second challenge is “data sparsity” or the “long-tail phenomenon” where a majority of sources only contribute a small number of claims, providing insufficient evidence to determine those sources' trustworthiness. For example, in the Twitter datasets that we collected during real-world events, more than 90 percent of sources only contributed to a single claim. Third, many current solutions are not scalable to large-scale social sensing events because of the centralized nature of their truth discovery algorithms. In this paper, we develop a Scalable and Robust Truth Discovery (SRTD) scheme to address the above three challenges. In particular, the SRTD scheme jointly quantifies both the reliability of sources and the credibility of claims using a principled approach. We further develop a distributed framework to implement the proposed truth discovery scheme using Work Queue in an HTCondor system. The evaluation results on three real-world datasets show that the SRTD scheme significantly outperforms the state-of-the-art truth discovery methods in terms of both effectiveness and efficiency.

Combating copyright infringing multimedia content has arisen as a critical undertaking in online video sharing platforms, such as YouTube and Twitch. In contrast to the traditional copyright detection problem that studies the static content (e.g., music, films, digital documents), the proposed system focuses on a much more challenging problem: detecting copyright infringements in live video streams. This is motivated by the observation that a large amount of copyright-infringing videos bypass the detector while many legal videos are taken down by mistake. In this paper, we present an end-to-end system that is dedicated to combating the copyright infringements in live video streams. The system to be demonstrated consists of 1) a web front-end for user interaction and customized video query, 2) a scalable and real-time video crawling system that can collect video metadata, live chat messages, and visual content of the live video streams on video sharing platforms, and 3) a novel supervised copyright detection engine that leverages the live chat messages of the audience to detect the copyright infringement of live videos.

Social media sensing has emerged as a new application paradigm to collect observations from online social media users about the physical environment. A fundamental problem in social media sensing applications lies in estimating the evolving truth of the measured variables and the reliability of data sources without knowing either of them a priori. This problem is referred to as dynamic truth discovery. Two major limitations exist in current truth discovery solutions: i) existing solutions cannot effectively address the missing truth problem where the measured variables do not have any reported measurements from the data sources; ii) the latent correlations among the measured variables were not fully captured and utilized in current solutions. In this paper, we proposed a Reliable Missing Truth Finder (RMTF) to address the above limitations in social media sensing applications. In particular, we develop a novel data-driven technique to identify the lagged and latent correlations among measured variables, and incorporate such correlation information into a holistic spatiotemporal inference model to infer the missing truth. We evaluated the RMTF using the real-world Twitter data feeds. The results show that the RMTF scheme significantly outperforms the state-of-the-art truth discovery solutions by correctly inferring the missing truth of the measured variables.

With the increasing popularity of online video sharing platforms (such as YouTube and Twitch), the detection of content that infringes copyright has emerged as a new critical problem in online social media. In contrast to the traditional copyright detection problem that studies the static content (e.g., music, films, digital documents), this paper focuses on a much more challenging problem: one in which the content of interest is from live videos. We found that the state-of-the-art commercial copyright infringement detection systems, such as the ContentID from YouTube, did not solve this problem well: large amounts of copyright-infringing videos bypass the detector while many legal videos are taken down by mistake. In addressing the copyright infringement detection problem for live videos, we identify several critical challenges: i) live streams are generated in real-time and the original copyright content from the owner may not be accessible; ii) streamers are getting more and more sophisticated in bypassing the copyright detection system (e.g., by modifying the title, tweaking the presentation of the video); iii) similar video descriptions and visual contents make it difficult to distinguish between legal streams and copyright-infringing ones. In this paper, we develop a crowdsourcing-based copyright infringement detection (CCID) scheme to address the above challenges by exploring a rich set of valuable clues from live chat messages. We evaluate CCID on two real world live video datasets collected from YouTube. The results show our scheme is significantly more effective and efficient than ContentID in detecting copyright-infringing live videos on YouTube.

With the ever-increasing number of road traffic accidents worldwide, the road traffic safety has become a critical problem in intelligent transportation systems. A key step towards improving the road traffic safety is to identify the locations where severe traffic accidents happen with a high probability so the precautions can be applied effectively. We refer to this problem as risky traffic location identification. While previous efforts have been made to address similar problems, two important limitations exist: i) data availability: many cities (especially in developing countries) do not maintain a publicly accessible database for the traffic accident records in a city, which makes it difficult to accurately estimate the accidents in the city; ii) location accuracy: many self-reported traffic accidents (e.g., social media posts from common citizens) are not associated with the exact GPS locations due to the privacy concerns. To address these limitations, this paper develops the RiskSens, a multi-view learning approach to identifying the risky traffic locations in a city by jointly exploring the social and remote sensing data. We evaluate RiskSens using a real world dataset from New York. The evaluation results show that RiskSens significantly outperforms the state-of-the- art baselines in identifying risky traffic locations in a city.

With the increasing popularity of online social media (e.g., Facebook, Twitter, Reddit), the detection of misleading content on social media has become a critical undertaking. This paper focuses on an important but largely unsolved problem: detecting fauxtography (i.e., social media posts with misleading images). We found that the existing literature falls short in solving this problem. In particular, current solutions either focus on the detection of fake images or misinformed texts of a social media post. However, they cannot solve our problem because the detection of fauxtography depends not only on the truthfulness of the images and the texts but also on the information they deliver together on the posts. In this paper, we develop the FauxBuster, an end-to-end supervised learning scheme that can effectively track down fauxtography by exploring the valuable clues from user's comments of a post on social media. The FauxBuster is content-free in that it does not rely on the analysis of the actual content of the images, and hence is robust against malicious uploaders who can intentionally modify the presentation and description of the images. We evaluate FauxBuster on real-world data collected from two mainstream social media platforms - Reddit and Twitter. Results show that our scheme is both effective and efficient in addressing the fauxtography problem.

Copyright infringement detection is a critical problem in large-scale online video sharing systems: the copyright-infringing videos must be correctly identified and removed from the system to protect the copyright of the content owners. This paper focuses on a challenging problem of detecting copyright infringement in live video streams. The problem is particularly difficult because i) streamers can be sophisticated and modify the title or tweak the presentation of the video to bypass the detection system; ii) legal videos and copyright-infringing ones may have very similar visual content and descriptions. We found current commercial copyright detection systems did not address this problem well: a large amount of copyrighted content bypasses the detection system while legal streams are taken down by mistake. In this paper, we develop the StreamGuard, an unsupervised Bayesian network based copyright infringement detection system that addresses the above challenges by leveraging live chat messages from the audience. We evaluate StreamGuard on real-world live video streams collected from YouTube. The results show that StreamGuard is effective and efficient in identifying the copyright-infringing videos.

Social sensing has emerged as a new application paradigm in networked sensing where data is collected from humans or devices on their behalf. This paper focuses on the opinion characterization problem in social sensing where the goal is to accurately characterize opinion attributes of the participants (e.g., analyze the sentiments, understand the opinion bias) from their sensor measurements. Several important challenges exist in solving the opinion characterization problem. First, human sensors often generate unstructured data (e.g., text, image, video) in which the opinion attributes are deeply embedded. Second, human sensors naturally generate measurements in different data modalities which encode the opinion attributes differently. Third, the possible imbalance between different data modalities may lead to potential bias in the opinion characterization results. To address the above challenges, this paper develops a Multi-View Opinion Characterization (MVOC) scheme to accurately characterize opinion attributes using a multi-view subspace learning approach. We evaluate the MVOC scheme through the real-world social sensing task of classifying the sentiments of reports from Twitter users. The evaluation results show that our scheme significantly outperforms the state-of-the-art baselines in solving the opinion characterization problem.

Social sensing has emerged as a new sensing paradigm where human sensors collectively report measurements about the physical world. This paper focuses on the cost-sensitive task allocation problem in social sensing where the goal is to effectively allocate sensing tasks to the human sensors to meet the desirable data quality requirement of the applications while minimizing the sensing cost. While recent progress has been made to tackle the cost-sensitive task allocation problem, an important challenge has not been well addressed, namely "real time task allocation", the task allocation schemes need to respond quickly to the potential large dynamics of the measured variables in social sensing. To address this challenge, this paper presents a Cost-Sensitive Task Allocation (CSTA) scheme inspired by techniques from online learning. The preliminary results show that our new scheme significantly outperforms the-state-of-the-art baselines.

Social sensing has emerged as a new sensing paradigm where humans (or devices on their behalf) collectively report measurements about the physical world. This paper focuses on an optimized task allocation problem in multi- attribute social sensing applications where the goal is to effectively allocate the tasks of collecting multiple attributes of the measured variables to human sensors while respecting the application's budget constraints. While recent progress has been made to tackle the optimized task allocation problem, two important challenges have not been well addressed. The first challenge is "online task allocation": the task allocation schemes need to respond quickly to the potentially large dynamics of the measured variables (e.g., temperature, noise, traffic) in social sensing. Delayed task allocation may lead to inaccurate sensing results and/or unnecessarily high sensing costs. The second challenge is the "multi-attribute constrained optimization": minimizing the overall sensing error given the dependencies and constraints of multiple attributes of the measured variables is a non-trivial problem to solve. To address the above challenges, this paper develops an Online Optimized Multi-attribute Task Allocation (OO-MTA) scheme inspired by techniques from machine learning and information theory. We evaluate the OO-MTA scheme using an urban sensing dataset collected from a real-world social sensing application. The evaluation results show that OO- MTA scheme significantly outperforms the state-of-the-art baselines in terms of the sensing accuracy.

Privacy on the Internet has become a major concern in recent years and is an ethical dilemma for Internet based applications. Social Sensing based Edge Computing (SSEC)is a category of distributed applications in which privately owned devices at the “edge” of the network participate in data collection and processing. Traditionally in SSEC, it has been difficult to manage privacy without making a tradeoff with performance or accuracy. In this paper, we present Privacy-aware Edge Computing (PEC), a framework that utilizes ring signatures for implementing social sensing applications with full privacy and without sacrificing data quality. Under PEC, privacy is managed by the edge devices without the need of a third party trusted authority. We evaluate the performance of PEC on a realworld edge computing platform and through extensive simulation studies. The results demonstrate that PEC achieves a high level of anonymity with a constrained cryptographic overhead.

Social Sensing has emerged as a new distributed application paradigm in networked sensing where streaming data is constantly collected from humans or devices on their behalf. A key challenge in social sensing is that the applications are often both data-intensive and delay-sensitive, which often require extensive resources in the cloud to support the data processing and analytics tasks. To address this problem, this paper focuses on a quality-aware online adaptive sampling problem where the goal is to judiciously sample a small set of representative sensing measurements (i.e., representative set) that effectively represent the key information of the social sensing data stream. However, two technical challenges exist in identifying such a representative set: (i) “quality-aware quantification”: data collected in the social sensing applications is often unstructured (e.g., text, image, video) where the key information is so deeply embedded in the raw data that can not be easily quantified to ensure the desirable quality of the selected representative set; (ii) “online adaptive sampling”: the sampling decision has to be made in real-time and be adaptive to the dynamics of the streaming data in social sensing. To address these challenges, this paper develops a Light-weight and Quality-aware Online Adaptive Sampling (LQOAS) scheme to dynamically identify the representative set of measurements from the streaming social sensing data using a submodular maximization approach. We evaluate the LQOAS scheme using a real world social sensing dataset from Twitter. The evaluation results show that the LQOAS scheme significantly outperforms the state-of-the-art sampling schemes.

In crowdsensing applications, participants (crowd sensors) work collectively to report their measurements about the physical world. This paper focuses on the optimized online task allocation problem in cost-sensitive crowdsensing applications where the goal is to dynamically allocate the sensing tasks to participants to meet the requirement of the applications while minimizing the sensing costs. Recent progress has been made to tackle the task allocation problem in crowdsensing. However, two important challenges have not been well addressed: i) “physical dynamics”: the values of the measured variables in crowdsensing often change significantly over time and space. It is essential for the task allocation schemes to adapt to such changes efficiently to optimize the task allocation process; ii) “crowd irregularity”: the number of participants in crowdsensing is often smaller than the number of desirable sensing locations and not all crowd sensors contribute data all the time (e.g., due to incentive or budget constraints). To address the above challenges, this paper develops an Online Optimized Task Allocation (OO-TA) scheme inspired by techniques from information theory and online learning. We evaluate the OO-TA scheme using a dataset collected from a real-world crowdsensing application. The evaluation results show that OO-TA scheme significantly outperforms the state-of-the-art baselines in terms of both effectiveness and efficiency.

Collaborative sensing has emerged as a new sensing paradigm where sensors collaboratively report measurements about the phenomena or events in the physical world. This paper focuses on a robust state prediction problem in collaborative sensing where the goal is to provide an accurate prediction on the dynamic status of the physical environment (e.g., air quality index, temperature, traffic) based on the incomplete and noisy data contributed by collaborative sensors. While significant progress has been made to study the state prediction problem, we identify two fundamental challenges that have not been well addressed by the current literature. The first challenge is "latent spatial dynamics": the spatial correlations among measured variables are highly dynamic and the features that affect such dynamics may not be directly observable from the sensing data. The second challenge is "incomplete and noisy data": a significant amount of measurements are missing in collaborative sensing due to the prohibitively high cost of deploying sensors for full spatial-temporal coverage. The collaborative sensors are also known to be unreliable in many applications and can easily contribute to noisy measurements. To address these challenges, this paper develops a Context-Aware Collaborative Sensing Prediction (CACSP) scheme inspired by techniques from latent semantic analysis and statistics. We evaluate our scheme through two real-world collaborative sensing applications and the results show that CACSP significantly outperforms the state-of-the-art baselines.

Social sensing has emerged as a new sensing application paradigm where measurements about the physical world are collected from humans or devices on their behalf. A key limitation in the current social sensing solution space is that data processing and analytics are often done in a "backend" mode (e.g., on dedicated servers or commercial cloud platforms). Such mode ignores the rich processing capability of increasingly powerful edge devices (e.g., mobile phones and nodes in the Internet of Things). Exploiting such edge devices in the social sensing setting introduces new challenges to real-time resource management. In this work, we develop a Bottom-up Game-theoretic Task Allocation (BGTA) framework to solve the critical problem of allocating real-time social sensing tasks to self-aware and non-cooperative edge computing nodes. In particular, we address two important challenges in solving this problem. The first one is "conflicting interest" where the objectives of applications and edge nodes may be at odds with each other. The second challenge is "asymmetric and incomplete information" where the application is often unaware of the detailed status (e.g., energy profile, utilization, CPU frequency) and compliance level of the edge nodes. To address these challenges, we first design a non-cooperative task allocation game model to address the conflicting objectives of the applications and edge nodes. We then develop a decentralized Fictitious Play scheme to allow each edge node to make its own decision on which task to execute in a non-cooperative context. Finally, we design a dynamic incentive mechanism to ensure the decisions made by the edge nodes meet objectives of the application. We implement a system prototype deployed on Nvidia Jetson TX1 and Jetson TK1 boards and evaluate our task allocation framework using two real-world social sensing applications. The results show that our scheme can well satisfy Quality of Service (QoS) requirement of the applications while providing optimized payoffs to edge nodes compared to the state-of-the-art baselines.

In this demo, we present a Social Sensing Edge Computing (SSEC) system that supports real-time task allocation on non-cooperative and heterogeneous edge devices. We evaluate the system on two real-time social sensing applications and demonstrate the energy savings as well as the Quality of Service (QoS) of the SSEC system. The demonstrated framework complements the traditional centralized social sensing solutions by providing a more scalable and responsive computation paradigm.

Social sensing has become a new crowdsourcing application paradigm where humans function as sensors to report their observations about the physical world. While many previous studies in social sensing focus on the problem of ascertaining the reliability of data sources and the truthfulness of their reported claims (often known as truth discovery), this paper investigates a new problem of hypothesis validation where the goal is to validate some high-level statements (referred to as hypotheses) from the low-level statements (referred to as claims) embedded in the social sensing data. The truthfulness of hypotheses cannot be directly obtained from the truth discovery results and two key challenges are involved in solving the hypothesis validation problem: (i) how to match the hypotheses generated by end users to the relevant claims generated by social sensors? (ii) How to accurately validate the truthfulness of the hypotheses given the unknown reliability of data sources and unvetted truthfulness of the claims? This paper proposes a Reliable Hypothesis Validation (RHV) scheme to address the above challenges. In particular, we develop a critical claim selection approach to match the hypotheses with the relevant claims and derive an optimal solution to validate their truthfulness by exploring the complex relationship between hypotheses and claims. The performance of RHV scheme is evaluated on three datasets collected from real- world social sensing applications. The results show that the RHV scheme significantly outperformed the state-of-the-art baselines in terms of validating the truthfulness of hypotheses.

Research in the field of Social Sensing Edge Computing (SSEC) faces a challenge in that most systems are developed on a small set of hardware and are never tested on large-scale networks. To address this issue, we present a Social Sensing Edge Computing Simulation Platform (SSEC SP) that is used to evaluate how SSEC systems scale to millions of devices. We demonstrate the platform by simulating several SSEC configurations to optimize Quality of Service (QoS) for large numbers of devices, while asserting that simulated results are consistent with the hardware implementations.

Identifying a set of diversified users who are local residents in a city is an important task for a wide spectrum of applications such as target ads of local business, surveys and interviews, and personalized recommendations. While many previous studies have investigated the problem of identifying the local users in a given area using online social network information (e.g., geotagged posts), few methods have been developed to solve the diversified user identification problem. In this paper, we propose a new analytical framework, Diversified Local Users Finder (DLUF), to accurately identify a set of diversified local users using a principled approach. In particular, the DLUF scheme first defines a new distance metric that measures the diversity between local users from physical dimension. The DLUF scheme then provides a solution to find the set of local users with maximum diversity. The performance of DLUF scheme is compared to several representative baselines using two real world datasets obtained from Foursquare application. We observe that the DLUF scheme accurately identifies the local users with a great diversity and significantly outperforms the compared baselines.

Social media sensing has emerged as a new big data application paradigm to collect observations and claims about the measured variables in physical environment from common citizens. A fundamental problem in social media sensing applications lies in estimating the evolving truth of claims and the reliability of data sources without knowing either of them a priori, which is referred to as dynamic truth discovery. We identified two critical challenges that are not fully addressed by solutions from current literature. The first challenge is “physical constraint-awareness” where the transition of truth is constrained by some physical rules that must be followed to ensure correct estimation of the evolving truth. The second one is “noisy and incomplete data” where the social media sensing data is sparse in nature and contains a lot of rumors and misinformation, making it difficult to capture the constantly evolving truth of measured variables. In this paper, we developed a new Constraint-Aware Dynamic Truth Discovery (CA-DTD) scheme to address the above challenges. To address the physical constraint-awareness challenge, CA-DTD develops a new constraint-aware Hidden Markov Model to effectively infer the evolving truth of measured variables by incorporating physical constraints. To address the noisy and incomplete data challenge, CA-DTD fuses sensing observations from online social media with information from traditional news media using a principled approach. We evaluate CA-DTD scheme using two real-world social media sensing data traces and the results show that CA-DTD significantly outperforms the state-of-the-art baselines.

Point-of-Interest (POI) recommendation is an important application in Location-based Social Networks (LBSN). The category prediction problem is to predict the next POI category that users may visit. The predicted category information is critical in large-scale POI recommendation because it can significantly reduce the prediction space and improve the recommendation accuracy. While efforts have been made to address the POI category prediction problem, several important challenges still exist. First, existing solutions did not fully explore the temporal dependency (e.g., “long range dependency”) of users' check-in traces. Second, the hidden contextual information associated with each check-in point has been underutilized. In this work, we propose a Context-Aware POI Category Prediction (CAP-CP) scheme using Natural Language Processing (NLP) models. In particular, to address temporal dependency challenge, we develop a novel Temporal Adaptive Ngram (TA-Ngram) model to capture the dynamic dependency between check-in points. To address the challenge of hidden context incorporation, CAP-CP leverages the Probabilistic Latent Semantic Analysis (PLSA) model to infer the semantic implications of the context variables in the prediction model. Empirical results on a real world dataset show that our scheme can effectively improve the performance of the state-of-the-art POI recommendation solutions.

Social sensing has emerged as a new data collection paradigm in networked sensing applications where humans are used as "sensors" to report their observations about the physical world. While many previous studies in social sensing focus on the problem of ascertaining the reliability of data sources and the correctness of their reported claims (often known as truth discovery), this paper investigates a new problem of critical source selection. The goal of this problem is to identify a subset of critical sources that can help effectively reduce the computational complexity of the original truth discovery problem and improve the accuracy of the analysis results. In this paper, we propose a new scheme, Critical Sources Selection (CSS) scheme, to find the critical set of sources by explicitly exploring both dependency and speak rate of sources. We evaluated the performance of our scheme and compared it to the state-of-the-art baselines using two data traces collected from a real world social sensing application. The results showed that our scheme significantly outperforms the baselines by finding more truthful information at a faster speed.

With the rapid growth of online social media and ubiquitous Internet connectivity, social sensing has emerged as a new crowdsourcing application paradigm of collecting observations (often called claims) about the physical environment from humans or devices on their behalf. A fundamental problem in social sensing applications lies in effectively ascertaining the correctness of claims and the reliability of data sources without knowing either of them a priori, which is referred to as truth discovery. While significant progress has been made to solve the truth discovery problem, some important challenges have not been well addressed yet. First, existing truth discovery solutions did not fully solve the dynamic truth discovery problem where the ground truth of claims changes over time. Second, many current solutions are not scalable to large-scale social sensing events because of the centralized nature of their truth discovery algorithms. Third, the heterogeneity and unpredictability of the social sensing data traffic pose additional challenges to the resource allocation and system responsiveness. In this paper, we developed a Scalable Streaming Truth Discovery (SSTD) solution to address the above challenges. In this paper, we developed a Scalable Streaming Truth Discovery (SSTD) solution to address the above challenges. In particular, we first developed a dynamic truth discovery scheme based on Hidden Markov Models (HMM) to effectively infer the evolving truth of reported claims. We further developed a distributed framework to implement the dynamic truth discovery scheme using Work Queue in HTCondor system. We also integrated the SSTD scheme with an optimal workload allocation mechanism to dynamically allocate the resources (e.g., cores, memories) to the truth discovery tasks based on their computation requirements. We evaluated SSTD through real world social sensing applications using Twitter data feeds. The evaluation results on three real-world data traces (i.e., Boston Bombing, Paris Shooting and College Football) show that the SSTD scheme is scalable and outperforms the state-of-the-art truth discovery methods in terms of both effectiveness and efficiency.

Crowdsourcing has emerged as an important data collection paradigm in participatory and human-centric sensing applications. While many crowdsourcing studies focus on sensing and recovering the status of the physical world, this paper investigates the problem of profiling the crowd sensors (i.e., humans). In particular, we study the problem of accurately inferring the home locations of people from the noisy and sparse crowdsourcing data they contribute. In this study, we propose a semi-supervised framework, Where Are You From (WAYF), to accurately infer the home locations of people by explicitly exploring the localness of people and the dependency between people based on their check-in behaviors under a rigorous analytical framework. We perform extensive experiments to evaluate the performance of our scheme and compared it to the state-of-the-art techniques using three real world data traces collected from Foursquare. The results showed the effectiveness of our scheme in accurately profiling the home locations of people.

Social sensing has emerged as a new application paradigm in networked sensing communities where a colossal amount of observations about the physical world are contributed by people or devices they use. Our work solves a critical challenge in social sensing applications where the goal is to estimate the reliability of social sensors and the truthfulness of observed variables (typically known as claims) with little prior knowledge on either of them. This challenge is referred to as truth discovery. An important limitation in the previous truth discovery solutions is that they assume the relationship between source reliability and claim truthfulness can be represented by simplified functions (e.g., linear, quadratic and binomial). This assumption leads to suboptimal truth discovery results because the exact relational dependency between sources and claims is often unknown a priori. In this paper, we show that a neural network approach can learn the complex relational dependency better than the previous truth discovery methods. In particular, we develop a multi-layer neural network model that solves the truth discovery problem in social sensing without any assumption on the prior knowledge of the source-claim relational dependency distribution. The performance of our model is evaluated through two real-world events using data crawled from Twitter. The evaluation results show that our neural network approach significantly outperforms previous truth discovery methods.

Inferring the localness of people is to classify people who are local residents in a city from people who visit the city by analyzing online check-in points that are contributed by online users. This information is critical for the urban planning, user profiling, and localized recommendation systems. Supervised learning approaches have been developed to infer the location of people in a city by assuming the availability of high-quality training datasets with complete geotemporal information. In this article, we develop an unsupervised model to accurately identify local people in a city by using the incomplete online check-in data that are publicly available. In particular, we develop an incomplete geotemporal expectation maximization (IGT-EM) scheme, which incorporates a set of hidden variables to represent the localness of people and a set of estimation parameters to represent the likelihood of venues to attract local and nonlocal people, respectively. Our solution can accurately classify local people from nonlocal nones without requiring any training data. We also implement a parallel IGT-EM algorithm by leveraging the computing power of a graphic processing unit (GPU) that consists of 2,496 cores. In the evaluation, we compare our new approach with the existing solutions through four real-world case studies using data from the New York City, Chicago, Boston, and Washington, DC. The results show that our approach can identify the local people and significantly outperform the compared baselines in estimation accuracy and execution time.

The localness inference problem is to identify whether a person is a local resident in a city or not and the likelihood of a venue to attract local people. This information is critical for many applications such as targeted ads of local business, urban planning, localized news and travel recommendations. While there are prior work on geo-locating people in a city using supervised learning approaches, the accuracy of those techniques largely depends on a high quality training dataset, which is difficult and expensive to obtain in practice. In this study, we propose to exploit spatial-temporal-social constraints from noisy online social media data to solve the localness inference problem using an unsupervised approach. The spatial-temporal constraint represents the correlations between people and venues they visit and the social constraint represents social connections between people. In particular, we develop a Spatial-Temporal-Social-Aware (STSA) inference framework to jointly infer i) the localness of a person and ii) the local attractiveness of a venue without requiring any training data. We evaluate the performance of STSA scheme using three real-world datasets collected from Foursquare. Experimental results show that STSA scheme outperforms the state-of-the-art techniques by significantly improving the estimation accuracy.

Users' home location is important information for many advanced information services in big data applications (e.g., localized recommendation, target ads of local business and urban planning). In this paper, we study the problem of accurately inferring the home locations of people from the noisy and sparse data they voluntarily share on online social media. Previous studies have developed supervised learning approaches to predict a person's home location in a city. However, the accuracy of these techniques largely depends on a high quality training dataset, which is difficult and expensive to obtain in practice. In this study, we propose a new analytical framework, Unsupervised Home Location Inference (UHLI), to accurately infer the home locations of people using a set of principle approaches. In particular, the UHLI scheme addresses the critical challenges of using sparse and noisy online social media data and derives an optimal solution to the home location inference problem. We evaluated the performance of our scheme and compared it to the state-of-the-art baselines using three real world data traces collected from Foursquare. The results showed that our scheme can accurately infer the home location of people and significantly outperform the state-of-the-art baselines.

In the big data era, it's important to identify trustworthy information from an influx of noisy data contributed by unvetted sources from online social media (e.g., Twitter, Instagram, Flickr). This task is referred to as truth discovery which aims at identifying the reliability of the sources and the truthfulness of claims they make without knowing either of them a priori. There are two important challenges that have not been well addressed in current truth discovery solutions. The first one is “misinformation spread” where a majority of sources are contributing to false claims, making the identification of truthful claims difficult. The second challenge is “data sparsity” where sources contribute a small number of claims, providing insufficient evidence to accomplish the truth discovery task. In this paper, we developed a Robust Truth Discovery (RTD) scheme to address the above two challenges. In particular, the RTD scheme explicitly quantifies different degrees of attitude that a source may express on a claim and incorporates the historical contributions of a source using a principled approach. The evaluation results on two real world datasetsshow that the RTD scheme significantly outperforms the state-of-the-art truth discovery methods.

Crowdsourcing has become an emerging data collection paradigm for smart city applications. A new category of crowdsourcing-based urban anomaly reporting systems have been developed to enable pervasive and real-time reporting of anomalies in cities (e.g., noise, illegal use of public facilities, urban infrastructure malfunctions). An interesting challenge in these applications is how to accurately predict an anomaly in a given region of the city before it happens. Prior works have made significant progress in anomaly detection. However, they can only detect anomalies after they happen, which may lead to significant information delay and lack of preparedness to handle the anomalies in an efficient way. In this paper, we develop a Crowdsourcing-based Urban Anomaly Prediction Scheme (CUAPS) to accurately predict the anomalies of a city by exploring both spatial and temporal information embedded in the crowdsourcing data. We evaluated the performance of our scheme and compared it to the state-of-the-art baselines using four real-world datasets collected from 311 service in the city of New York. The results showed that our scheme can predict different categories of anomalies in a city more accurately than the baselines.

This paper presents an unsupervised approach to accurately discover interesting places in a city from location-based social sensing applications, a new sensing application paradigm that collects observations of physical world from Location-based Social Networks (LBSN). While there are alarge amount of prior works on personalized Point of Interests (POI) recommendation systems, they used supervised learning approaches that did not work for users who have little or no historic (training) data. In this paper, we focused on an interesting place discovery problem where the goal is to accurately discover the interesting places in a city that average people may have strong interests to visit (e.g., parks, museums, historic sites, etc.) using unsupervised approaches. In particular, we develop a new Physical-Social-aware Interesting Place Discovery (PSIPD) scheme which jointly exploits the location's physical dependency and the visitor's social dependency to solve the interesting place discovery problem using an unsupervised approach. We compare our solution with state-of-the-art baselines using two real world data traces from LBSN. The results showed that our approach achieved significant performance improvements compared to all baselines in terms of both estimation accuracy and ranking performance.

This paper develops a new principled framework to solve a hardness-aware truth discovery problem in social sensing applications. Social sensing has emerged as a new application paradigm where a large crowd of social sensors (humansor devices on their behalf) are recruited to or voluntarily report observations about the physical environment at scale. These observations may be either true or false, and hence are viewed as binary claims. A fundamental problem in social sensing applications lies in ascertaining the correctness of claims and the reliability of data sources. We refer to this problem as truth discovery. Significant efforts were made to address the truth discovery problem, but an important dimension of the problem has not been fully exploited: hardness of claims (how challenging a claim is to be made). A common assumption made in the previous work is that they assumed all claims areof the same degree of hardness. However, in real world social sensing applications, simply ignoring the hardness differences between claims could easily lead to suboptimal truth discovery results. In this paper, we develop a new hardness-aware truth discovery scheme that explicitly considers different hardness degrees of claims into a rigorous analytical framework. The new truth discovery scheme solves a maximum likelihood estimation problem to determine both the claim correctness and the source reliability. We compare our hardness-aware scheme with the state-of-the-art baselines through three real world case studies(Baltimore Riots, Paris Attack and Oregon Shootings, all in 2015) using Twitter data feeds. The evaluation results showed that our new scheme outperforms all compared baselines and significantly improves the truth discovery accuracy in social sensing applications.

This work is motivated by the emergence of social sensing as a new paradigm of collecting observations about the physical environment from humans or devices on their behalf. These observations may be true or false, and hence are viewed as binary claims. A fundamental problem in social sensing applications lies in ascertaining the correctness of claims and the reliability of data sources without knowing either of them a priori. We refer to this problem as truth discovery. Prior works have made significant progress to addressing the truth discovery problem, but two significant limitations exist: (i) they ignored the fact that claims reported in social sensing applications can be either relevant or irrelevant to the topic of interests. (ii) They either assumed the data sources to be independent or the source dependency graphs can be represented as a set of disjoint trees. These limitations led to suboptimal truth discovery results. In contrast, this paper presents the first social sensing framework that explicitly incorporates the topic relevance feature of claims and arbitrary source dependency graphs into the solutions of truth discovery problem. The new framework solves a multidimensional maximum likelihood estimation problem to jointly estimate the truthfulness and topic relevance of claims as well as the reliability and topic awareness of sources. We compared our new scheme with the state-of-the-art truth discovery solutions using three real world data traces collected from Twitter in the aftermath of Paris Shooting event (2015), Hurricane Arthur (2014) and Boston Bombing event (2013) respectively. The evaluation results showed that our schemes significantly outperform the compared baselines by identifying more relevant and truthful claims in the truth discovery results.

This work is motivated by the need to provide reliable information recommendation to users in social sensing. Social sensing has become an emerging application paradigm that uses humans as sensors to observe and report events in the physical world. These human sensed observations are often viewed as binary claims (either true or false). A fundamental challenge in social sensing is how to ascertain the credibility of claims and the reliability of sources without knowing either of them a priori. We refer to this challenge as truth discovery. While prior works have made progress on addressing this challenge, an important limitation exists: they did not explore the mood sensitivity aspect of the problem. Therefore, the claims identified as correct by current solutions can be completely biased in regards to the mood of human sources and lead to useless or even misleading recommendations. In this paper, we present a new analytical model that explicitly considers the mood sensitivity feature in the solution of truth discovery problem. The new model solves a multi-dimensional estimation problem to jointly estimate the correctness and mood neutrality of claims as well as the reliability and mood sensitivity of sources. We compare our model with state-of-the-art truth discovery solutions using four real world datasets collected from Twitter during recent disastrous and emergent events: Brussels Bombing, Paris Attack, Oregon Shooting, Baltimore Riots, which occurred in 2015 and 2016. The results show that our model has significant improvements over the compared baselines by finding more correct and mood neutral claims.

This paper develops a new principled framework to solve an emotional-aware truth discovery problem in social sensing applications. Social sensing has emerged as a new application paradigm of cyber-physical systems with humans-in-the-loop where a large crowd of social sensors (humans or devices on their behalf) are recruited to or spontaneously report observations about the physical environment at scale. A fundamental problem in social sensing applications lies in ascertaining the correctness of the reported observations (often called claims) and the reliability of data sources. We refer to this problem as truth discovery. While significant efforts have been made to address the truth discovery problem, an important aspect of the problem has not been fully explored in previous studies: how to deal with emotional claims. A common assumption made in the previous works is that all claims are assumed to be factual (i.e., either true or false). However, unlike physical sensors, humans are more likely to incorporate personal emotions and sentiments in the reported observations (e.g., tweets, blogs), which can easily confuse the current truth discovery solutions and lead to inaccurate results. In this paper, we develop a new emotional-aware truth discovery scheme that explicitly incorporates emotional information of human reported data into an analytical framework. The new truth discovery scheme solves a maximum likelihood estimation problem to determine both the claim correctness and the source reliability. We compare our emotional-aware scheme with the state-of-the-art baselines through three real world case studies using Twitter data feeds. The evaluation results showed that our new scheme outperforms all compared baselines and significantly improves the truth discovery accuracy in social sensing applications.

This paper presents a link weight based maximum likelihood estimation framework to solve the truth discovery problem in social sensing applications. Social sensing has emerged as a new paradigm of data collection, where a group of individuals collect and share observations or measurements about the physical world at scale. A key challenge in social sensing applications lies in ascertaining the correctness of reported observations from unvetted data sources with unknown reliability. We refer to this problem as truth discovery. In this paper, we develop a new link weight based truth discovery scheme that solves the truth discovery problem by explicitly considering different degrees of confidence that sources may express on the reported data. The preliminary results show that our new scheme significantly outperforms the-state-of-the-art baselines and improves the accuracy of the truth estimation results in social sensing applications.

This paper presents a new principled framework for exploiting time-sensitive information to improve the truth discovery accuracy in social sensing applications. This work is motivated by the emergence of social sensing as a new paradigm of collecting observations about the physical environment from humans or devices on their behalf. These observations maybe true or false, and hence are viewed as binary claims. A fundamental problem in social sensing applications lies in ascertaining the correctness of claims and the reliability of data sources. We refer to this problem as truth discovery. In this paper, we develop a new time-sensitive truth discovery scheme that explicitly incorporates the source responsiveness and the claim lifespan into a rigorous analytical framework. The preliminary results showed that our new scheme outperforms all compared baselines and significantly improves the truth discovery accuracy in social sensing applications.

This paper develops a new principled framework for exploiting time-sensitive information to improve the truth discovery accuracy in social sensing applications. This work is motivated by the emergence of social sensing as a new paradigm of collecting observations about the physical environment from humans or devices on their behalf. These observations maybe true or false, and hence are viewed as binary claims. A fundamental problem in social sensing applications lies in ascertaining the correctness of claims and the reliability of data sources. We refer to this problem as truth discovery. Time is a critical dimension that needs to be carefully exploited in the truth discovery solutions. In this paper, we develop a new time-sensitive truth discovery scheme that explicitly incorporates the source responsiveness and the claim lifespan into a rigorous analytical framework. The new truth discovery scheme solves a maximum likelihood estimation problem to determine both the claim correctness and the source reliability. We compare our time-sensitive scheme with the state-of-the-art baselines through an extensive simulation study and a real world case study. The evaluation results showed that our new scheme outperforms all compared baselines and significantly improves the truth discovery accuracy in social sensing applications.

This paper presents a demonstration of our SECON 2015 paper using Twitter based case studies for social sensing applications. Social sensing has emerged as a new paradigm of data collection, where a group of individuals volunteer (or are recruited) to share certain observations or measurements about the physical world. A key challenge in social sensing applications lies in ascertaining the correctness of reported observations from unvetted data sources with unknown reliability. We refer to this problem as truth estimation. In this paper, we showed a demo of a new confidence-aware truth estimation scheme that explicitly considers different degrees of confidence that sources express on the reported data. In the demo session: the participants will have a chance to (i) play with the tool on some historic datasets we have collected from Twitter; (ii) send live queries to Twitter and perform real-time truth estimation analysis in the events of their interests.

This paper presents a spatial-temporal aware analytical framework to solve the truth finding problem in social sensing applications. Social sensing has emerged as a new big data application paradigm of collecting observations about the physical environment from social sensors (e.g., humans) or devices on their behalf. The collected observations may be true or false, and hence are viewed as binary claims. A fundamental challenge in social sensing applications lies in accurately ascertaining the correctness of claims and the reliability of data sources without knowing either of them a priori. This challenge is referred to as truth finding. Significant efforts have been made to address this challenge but two important features were largely missing in the state-of-the-arts solutions: when and where the claims are reported by a source. In this paper, we develop a new spatial-temporal aware truth finding scheme to explicitly incorporate the time information of a claim and location information of a source into a rigorous analytical framework. The new truth finding scheme solves a constraint optimization problem to determine both the source reliability and claim correctness. We evaluated the spatial-temporal aware truth finding scheme through an extensive simulation study. The initial results show that our new scheme outperforms all the compared state-of-the-art baselines and significantly improves the truth finding accuracy in social sensing applications.

This paper presents a confidence-aware maximum likelihood estimation framework to solve the truth estimation problem in social sensing applications. Social sensing has emerged as a new paradigm of data collection, where a group of individuals volunteer (or are recruited) to share certain observations or measurements about the physical world. A key challenge in social sensing applications lies in ascertaining the correctness of reported observations from unvetted data sources with unknown reliability. We refer to this problem as truth estimation. The prior works have made significant efforts to solve this problem by developing various truth estimation algorithms. However, an important limitation exists: they assumed a data source makes all her/his observations with the same degree of confidence, which may not hold in many real-world social sensing applications. In this paper, we develop a new confidence-aware truth estimation scheme that removes this limitation by explicitly considering different degrees of confidence that sources express on the reported data. The new truth estimation scheme solves a maximum likelihood estimation problem to determine both the correctness of collected data and the reliability of data sources. We compare our confidence-aware scheme with the state-of-the-art baselines through both an extensive simulation study and three real world case studies based on Twitter. The evaluation shows that our new scheme outperforms all compared baselines and significantly improves the accuracy of the truth estimation results in social sensing applications.

The paper presents a new model for crowd-sensing applications, where humans are used as the sensing sources to report information regarding the physical world. In contrast to previous work on the topic, we consider a model where the sources in question are polarized. Such might be the case, for example, in political disputes and in situations involving different communities with largely dissimilar beliefs that color their interpretation and reporting of physical world events. Reconstructing accurate ground truth is more complicated when sources are polarized. The paper describes an algorithm that significantly improves the quality of reconstruction results in the presence of polarized sources. For evaluation, we recorded human observations from Twitter for four months during a recent Egyptian uprising against the former president. We then used our algorithm to reconstruct a version of events and compared it to other versions produced by state of the art algorithms. Our analysis of the data set shows the presence of two clearly defined camps in the social network that tend of propagate largely disjoint sets of claims (which is indicative of polarization), as well as third population whose claims overlap subsets of the former two. Experiments show that, in the presence of polarization, our reconstruction tends to align more closely with ground truth in the physical world than the existing algorithms.

This paper complements the large body of social sensing literature by developing means for augmenting sensing data with inference results that "fill-in" missing pieces. Unlike trend-extrapolation methods, we focus on prediction in disaster scenarios where disruptive trend changes occur. A set of prediction heuristics (and a standard trend extrapolation algorithm) are compared that use either predominantly-spatial or predominantly-temporal correlations for data extrapolation purposes. The evaluation shows that none of them do well consistently. This is because monitored system state, in the aftermath of disasters, alternates between periods of relative calm and periods of disruptive change (e.g., aftershocks). A good prediction algorithm, therefore, needs to intelligently combine time-based data extrapolation during periods of calm, and spatial data extrapolation during periods of change. The paper develops such an algorithm. The algorithm is tested using data collected during the New York City crisis in the aftermath of Hurricane Sandy in November 2012. Results show that consistently good predictions are achieved. The work is unique in addressing the bi-modal nature of damage propagation in complex systems subjected to stress, and offers a simple solution to the problem.

Today's cyber-physical systems (CPS) increasingly operate in social spaces. Examples include transportation systems, disaster response systems, and the smart grid, where humans are the drivers, survivors, or users. Much information about the evolving system can be collected from humans in the loop, a practice that is often called crowd-sensing. Crowd-sensing has not traditionally been considered a CPS topic, largely due to the difficulty in rigorously assessing its reliability. This paper aims to change that status quo by developing a mathematical approach for quantitatively assessing the probability of correctness of collected observations (about an evolving physical system), when the observations are reported by sources whose reliability is unknown. The paper extends prior literature on state estimation from noisy inputs, that often assumed unreliable sources that fall into one or a small number of categories, each with the same (possibly unknown) background noise distribution. In contrast, in the case of crowd-sensing, not only do we assume that the error distribution is unknown but also that each (human) sensor has its own possibly different error distribution. Given the above assumptions, we rigorously estimate data reliability in crowd-sensing systems, hence enabling their exploitation as state estimators in CPS feedback loops. We first consider applications where state is described by a number of binary variables, then extend the approach trivially to multivalued variables. The approach also extends prior work that addressed the problem in the special case of systems whose state does not change over time. Evaluation results, using both simulation and a real-life case-study, demonstrate the accuracy of the approach.

This article addresses the challenge of truth discovery from noisy social sensing data. The work is motivated by the emergence of social sensing as a data collection paradigm of growing interest, where humans perform sensory data collection tasks. Unlike the case with well-calibrated and well-tested infrastructure sensors, humans are less reliable, and the likelihood that participants' measurements are correct is often unknown a priori. Given a set of human participants of unknown trustworthiness together with their sensory measurements, we pose the question of whether one can use this information alone to determine, in an analytically founded manner, the probability that a given measurement is true. In our previous conference paper, we offered the first maximum likelihood solution to the aforesaid truth discovery problem for corroborating observations only. In contrast, this article extends the conference paper and provides the first maximum likelihood solution to handle the cases where measurements from different participants may be conflicting . The article focuses on binary measurements. The approach is shown to outperform our previous work used for corroborating observations, the state-of-the-art fact-finding baselines, as well as simple heuristics such as majority voting.

This paper presents a streaming approach to solve the truth estimation problem in crowdsourcing applications. We consider a category of crowdsourcing applications where a group of individuals volunteer (or are recruited to) share certain observations or measurements about the physical world. Examples include reporting locations of gas stations that remain operational after a natural disaster or reporting locations of potholes on city streets. We call such applications social sensing. Ascertaining the correctness of reported observations is a key challenge in such applications, referred to as the truth estimation problem. This problem is made difficult by the fact that the reliability of individual sources is usually unknown a priori, since any concerned citizen may, in principle, participate. Moreover, the timescales of crowdsourcing campaigns of interest can be as small as a few hours or days, which does not offer enough history for a reputation system to converge. Instead, recent prior work, including our own, developed fact-finding algorithms to solve this problem by iteratively assessing the credibility of sources and their claims in the absence of reputation scores. Such algorithms, however, operate on the entire dataset of reported observations in a batch fashion, which makes them less suited to applications where new observations arrive continually. In this paper, we describe a streaming fact-finder that recursively updates previous estimates based on new data. The recursive algorithm solves an expectation maximization (EM) problem to determine the odds of correctness of different observations. We compare the performance of our recursive EM algorithm to a batch EM algorithm, as well as to several state-of-art fact-finders through extensive simulations. We also demonstrate convergence of the recursive algorithm to the results of the batch version through a real social sensing experiment. Our evaluation shows that the proposed approach can process data streams much more efficiently while keeping the truth estimation accuracy close to that of the (much slower) batch algorithm. Ours is therefore the first fact-finder developed with explicit consideration to the continuous update needs of crowd-sourcing applications.

This paper develops and evaluates algorithms for exploiting physical constraints to improve the reliability of social sensing. Social sensing refers to applications where a group of sources (e.g., individuals and their mobile devices) volunteer to collect observations about the physical world. A key challenge in social sensing is that the reliability of sources and their devices is generally unknown, which makes it non-trivial to assess the correctness of collected observations. To solve this problem, the paper adopts a cyber-physical approach, where assessment of correctness of individual observations is aided by knowledge of physical constraints on both sources and observed variables to compensate for the lack of information on source reliability. We cast the problem as one of maximum likelihood estimation. The goal is to jointly estimate both (i) the latent physical state of the observed environment, and (ii) the inferred reliability of individual sources such that they are maximally consistent with both provenance information (who claimed what) and physical constraints. We evaluate the new framework through a real-world social sensing application. The results demonstrate significant performance gains in estimation accuracy of both source reliability and observation correctness.

In this demo, a learning system, called Metis, is presented that extrapolates missing pieces in participatory sensing data. The work addresses the challenge of incomplete coverage in participatory sensing applications, where lack of complete control over participant mobility and sensing patterns may create coverage gaps in space and in time. Metis learns the underlying spatiotemporal patterns of the measured phenomenon from available incomplete observations, and uses these patterns to infer missing data. We describe the overall system design and demonstrate the system using data collected during the New York City gas crisis in the aftermath of Hurricane Sandy.

Two goals of network science are to (i) uncover fundamental properties of phenomena modeled as networks, and to (ii) explore novel use of networks as models for a diverse range of systems and phenomena in order to improve our understanding of such systems and phenomena. This paper advances the latter direction by casting credibility estimation in social sensing applications as a network science problem, and by presenting a network model that helps understand the fundamental accuracy trade-offs of a credibility estimator. Social sensing refers to data collection scenarios, where observations are collected from (possibly unvetted) human sources. We call such observations claims to emphasize that we do not know whether or not they are factually correct. Predictable, scalable and robust estimation of both source reliability and claim correctness, given neither in advance, becomes a key challenge given the unvetted nature of sources and lack of means to verify their claims. In a previous conference publication, we proposed a maximum likelihood approach to jointly estimate both source reliability and claim correctness. We also derived confidence bounds to quantify the accuracy of such estimation. In this paper, we cast credibility estimation as a network science problem and offer systematic sensitivity analysis of the optimal estimator to understand its fundamental accuracy trade-offs as a function of an underlying network topology that describes key problem space parameters. It enables assured social sensing, where not only source reliability and claim correctness are estimated, but also the accuracy of such estimates is correctly predicted for the problem at hand.

This demonstration illustrates an information aggregation and summarization service for social sensing applications. Social sensing, using mobile phones and other networked devices in the possession of individuals, has gained significant popularity in recent years. In some cases, the information collected is structured, such as numeric data from temperature sensors, accelerometers, or GPS devices. Aggregate statistical properties, such as expected values, standard deviations, and outliers, can be easily computed, and can be used to summarize the data set. In other cases, however, the collection includes unstructured data types such as text or images with textual annotations. The concepts of expected values and outliers are harder to define, yet it is still important to be able to aggregate and summarize the data. We demonstrate a system which can automatically summarize real-time textual data common to social sensing applications. Specifically, we focus on text messages that describe events in the environment. The output of our service provides a reliable summary of observations that can be used in many contexts from military intelligence to participatory sensing campaigns.

This paper addresses the challenge of truth discovery from noisy social sensing data. The work is motivated by the emergence of social sensing as a data collection paradigm of growing interest, where humans perform sensory data collection tasks. A challenge in social sensing applications lies in the noisy nature of data. Unlike the case with well-calibrated and well-tested infrastructure sensors, humans are less reliable, and the likelihood that participants' measurements are correct is often unknown a priori. Given a set of human participants of unknown reliability together with their sensory measurements, this paper poses the question of whether one can use this information alone to determine, in an analytically founded manner, the probability that a given measurement is true. The paper focuses on binary measurements. While some previous work approached the answer in a heuristic manner, we offer the first optimal solution to the above truth discovery problem. Optimality, in the sense of maximum likelihood estimation, is attained by solving an expectation maximization problem that returns the best guess regarding the correctness of each measurement. The approach is shown to outperform the state of the art fact-finding heuristics, as well as simple baselines such as majority voting.

This paper estimates new confidence bounds on source reliability in social sensing applications. Scalable and robust estimation of source reliability is a key challenge in social sensing where humans or human-operated sensors act as data sources. In order to assess correctness of data, the reliability of sources must first be assessed, yet this is complicated when sources are not a priori known and vetted, but rather can opt in at will, for example, by downloading a sensing application on their mobile device. In our previous work, we developed a maximum likelihood source reliability estimator and approximately quantified confidence in its estimation based on an asymptotic Cramer-Rao lower bound (CRLB). In this paper we show that the asymptotic bound fails to track estimation performance when the number of sources is small. We derive the real CRLB to accurately characterize estimation performance for scenarios where the asymptotic bound fails. We study the limitations of the real and asymptotic CRLBs and show the trade-offs they offer between computational complexity and estimation scalability. We also evaluate the robustness of these bounds to changes in the number of sources. The results offer an understanding of attainable estimation accuracy of source reliability in social sensing applications that rely on un-vetted sources whose reliability is not known in advance.

At CPSWeek 2011, the authors presented a demonstration of Apollo, a fact-finder for participatory sensing that ranks archived human-centric and sensor data by credibility. The current demonstration significantly extends our previous work by allowing Apollo to operate on live streaming data; in this case, live Twitter feeds. As the role of humans as sensors increases in emerging sensing applications, a principled approach becomes necessary to address the problem of ascertaining the veracity of sources and observations made by them. Participatory and social sensing applications may use potentially unreliable or unverified sources, such as a phone-based sensing application that grows virally in a large un-vetted population, a disaster-response application, where conflicting damage assessment reports may come from large numbers of different volunteers, or a military application, where friendly observers at a remote location may make hard-to-verify claims about local events. Apollo analyzes noisy data that increasingly plagues human-centric sensing to determine which items of information are more likely to be true.

This paper develops a software service for dynamic performance optimization and control in performance-sensitive systems. The next generation of performance-sensitive systems is expected to be more distributed and dynamic. They will have multiple "knobs” that affect performance and resource allocation. However, relying on the conglomeration of independent knob controls can become increasingly suboptimal. The problem lies in performance composability or lack thereof; a challenge that arises because individual optimizations in performance-sensitive systems generally do not compose well when combined. Performance adaptation in such systems needs to be carefully designed and implemented by holistically considering performance composability in order to achieve desired system performance. A flexible supporting software layer is therefore needed to easily apply different holistic performance management techniques. In this paper, we develop a software service, called OptiTuner, that monitors the current performance and the resource availability in performance-sensitive systems and allows easy implementation of different performance management schemes based on theoretical concepts of constrained optimization and feedback control. In order to show the efficacy of OptiTuner, we apply it to implement three holistic energy minimization techniques in a real-time web server farm comprising 18 machines. Using an industry standard e-Business benchmark, TPC-W, we demonstrate that the three approaches save up to 40 percent of total energy cost compared to the baseline approaches that do not holistically optimize the cost.

This paper derives energy-optimal batching periodsfor asynchronous multistage data processing on sensor nodes in the sense of minimizing energy consumption while meeting end-to-end deadlines. Batching the processing of (sensor) data maximizes processor sleep periods, hence minimizing the wakeup frequency and the corresponding overhead. The algorithm is evaluated on mPlatform, a next-generation heterogeneous sensor node platform equipped with both a low-end microcontroller(MSP430) and a higher-end embedded systems processor (ARM). Experimental results show that the total energy consumption of mPlatform, when processing data flowsat their optimal batching periods, is up to 35% lower than that for uniform period assignment. Moreover, processing data at the appropriate processor can use as much as 80% less energy than running the same task set on the ARM alone and 25% less energy than running the taskset on the MSP430 alone.

In this paper, we present LiteView, an experimental toolkit for end-user diagnosis of communication path problems in sensor network systems. This toolkit provides an on-site, interactive environment that helps answering questions related to the instantaneous communication environments. For example, it allows users to identify broken links or asymmetric links, which are likely to become traffic bottlenecks. It also allows users to identify traffic hotspots by collecting round-trip delays of arbitrary pairs of nodes. Compared to the previous work on sensor network troubleshooting, this toolkit has two key differences. First, its utilities are generic and application independent, allowing users to easily transfer experiences in managing one deployment to another. Second, its utilities are interactive, allowing users to quickly obtain the current status, optimize the deployment decisions, and observe their immediate effects. The utilities provided in this paper are thus promising to identify and solve communication path problems.