Foundation models are powerful technologies: how they are released publicly directly shapes their societal impact. In this position paper, we focus on *open* foundation models, defined here as those with broadly available model weights (e.g., Llama 3, Stable Diffusion XL). We identify five distinctive properties (e.g., greater customizability, poor monitoring) that mediate their benefits and risks. Open foundation models present significant benefits, with some caveats, that span innovation, competition, the distribution of decision-making power, and transparency. To understand their risks of misuse, we design a risk assessment framework for analyzing their *marginal risk*. Across several misuse vectors (e.g., cyberattacks, bioweapons), we find that current research is insufficient to effectively characterize the marginal risk of open foundation models relative to pre-existing technologies. The framework helps explain why the marginal risk is low in some cases, clarifies disagreements about misuse risks by revealing that past work has focused on different subsets of the framework with different assumptions, and articulates a way forward for more constructive debate. Overall, our work helps support a more grounded assessment of the societal impact of open foundation models by outlining what research is needed to empirically validate their theoretical benefits and risks.

Non-autoregressive models and, in particular, Connectionist Temporal Classification (CTC) models have been the most popular approaches towards mispronunciation detection and diagnosis (MDD) task. In this paper, we identify two important knowledge gaps in MDD that have not been well studied in existing MDD research. First, CTC-based MDD models often assume conditional independence in the predicted phonemes, and therefore prominent mispronunciation patterns are underutilized. Second, existing MDD approaches are constrained to use training data from a language-specific phoneme set, and therefore cannot distinguish accented sounds that are not present in the predefined phoneme set. In this paper, we propose a set of autoregressive phonetic Recurrent Neural Network Transducer (RNN-T) MDD models that are capable of capturing temporal dependence of mispronunciation patterns. We further devise an extended phoneme set and weakly supervised training strategy to allow the model to distinguish similar sounding phonemes from different languages. We evaluate the proposed method on the public L2-ARCTIC dataset. Results have shown the proposed phonetic RNN-T model achieves significant improvements in false acceptance rate compared to state-of-the-art methods.

Fauxtography is a category of multi-modal posts that spread misleading information on various big data online social platforms that generate billions of posts on a daily basis (e.g., Facebook, Twitter, Reddit). A fauxtography post usually consists of an image, a text description and comments from its readers. In this paper, we focus on explaining fauxtography posts by identifyingwhatspecific component andwhythat component of a post leads to the fauxtography (i.e., duo explanations). This problem is motivated by the limitations of current fauxtography detection solutions that only focus on the detection but ignore the importantexplanationaspect of their results. Two critical challenges exist in solving our problem: i) it is difficult to accurately identify the “guilty” component of a fauxtography post given the fact that different components of the post and their associations could all lead to the fauxtography; ii) it is expensive and time-consuming to obtain a good training set with fine-grained labels of fauxtography posts in terms of explainability, making it challenging to develop fully supervised explainable solutions. To address the above challenges, we develop aDuoExplainableFauxtography Detection Framework under aConstrained Supervision (DExFC) to generate duo explanations from both content and comment parts of fauxtography posts. We evaluate the DExFC by creating real-world datasets from different online social media platforms (Twitter and Reddit). The results show that DExFC not only detects the fauxtography posts more accurately than the state-of-the-art solutions but also provides well-justified explanations to its results without the full supervision.

Human-centric fauxtography is a category of multi-modal posts that spread misleading information on online information distribution and sharing platforms such as online social media. The reason of a human-centric post being fauxtography is closely related to its multi-modal content that consists of diversified human and non-human subjects with complex and implicit relationships. In this paper, we focus on an explainable fauxtography detection problem where the goal is to accurately identify and explain why a human-centric social media post is fauxtography (or not). Our problem is motivated by the limitations of current fauxtography detection solutions that focus primarily on the detection task but ignore the important aspect of explaining their results (e.g., why a certain component of the post delivers the misinformation). Two important challenges exist in solving our problem: 1) it is difficult to capture the implicit relations and attributions of different subjects in a fauxtography post given the fact that many of such knowledge is shared between different crowd workers; 2) it is not a trivial task to create a multi-modal knowledge graph from crowd workers to identify and explain human-centric fauxtography posts with multi-modal contents. To address the above challenges, we develop CrowdGraph, a crowdsourcing based multi-modal knowledge graph approach to address the explainable fauxtography detection problem. We evaluate the performance of CrowdGraph by creating a real-world dataset that consists of human-centric fauxtography posts from Twitter and Reddit. The results show that CrowdGraph not only detects the fauxtography posts more accurately than the state-of-the-arts but also provides well-justified explanations to the detection results with convincing evidence.

Natural language understanding (NLU) is one of the most critical components in goal-oriented dialog systems and enables innovative Big Data applications such as intelligent voice assistants (IVA) and chatbots. While recent advances in deep learning-based NLU models have achieved significant improvements in terms of accuracy, most existing works are monolingual or bilingual. In this work, we propose and experiment with techniques to develop multilingual NLU models. In particular, we first propose a purely language-agnostic multilingual NLU framework using a multilingual BERT (mBERT) encoder, a joint decoder design for intent classification and s lot filling tasks, and a novel co-appearance regularization technique. Then three distinct language-aware multilingual NLU approaches are proposed including using language code as explicit input; using language-specific parameters during decoding; and using implicit language identification as an auxiliary task. We show results for a large-scale, commercial IVA system trained on a various set of intents with huge vocabulary sizes, as well as on a public multilingual NLU dataset. We performed experiments in explicit consideration of code-mixing and language dissimilarities which are practical concerns in large-scale real-world IVA systems. We have found that language-aware designs can improve NLU performance when language dissimilarity and code-mixing exist. The empirical results together with our proposed architectures provide important insights towards designing multilingual NLU systems.

Social airborne sensing (SAS) is emerging as a new sensing paradigm that leverages the complementary aspects of social sensing and airborne sensing (i.e., UAVs) for reliable information collection. In this paper, we present HeteroSAS, a heterogeneous resource management framework for "all-in-the-air" SAS in disaster response applications. Current SAS approaches use UAVs to only capture data, but carry out computation on ground-based processing nodes that may be unavailable in disaster scenarios and thus consider a single model of UAV along with only one type of task (i.e., data capture). In this paper, we explore the opportunity to exploit the complementary strengths of different UAV models to accomplish all stages of sensing tasks (i.e., data capturing, maneuvering, and computation) exclusively "in-the-air". However, several challenges exist in developing such a resource management framework: i) handling the uncertain social signals in presence of the heterogeneity of UAVs and tasks; and ii) adapting to constantly changing cyber-physical-social environments. The HeteroSAS framework addresses these challenges by building a novel resource management framework that observes the environment and learns the optimal strategy for each UAV using techniques from multi-agent reinforcement learning, game theory, and ensemble learning. The evaluation with a real-world case study shows that HeteroSAS outperforms the state-of-the-art in terms of detection effectiveness, deadline hit rate, and robustness on heterogeneity.

The advance of mobile sensing and edge computing has brought new opportunities for abnormal health detection (AHD) systems where edge devices such as smartphones and wearable sensors are used to collect people's health information and provide early alerts for abnormal health conditions such as stroke and depression. The recent development of federated learning (FL) allows participants to collaboratively train powerful AHD models while keeping their health data private to local devices. This paper targets at addressing a critical challenge of adapting FL to train AHD models, where the participants' health data is highly imbalanced and contains biased class distributions. Existing FL solutions fail to address the class imbalance issue due to the strict privacy requirements of participants as well as the heterogeneous resource constraints of their edge devices. In this work, we propose FedSens, a new FL framework dedicated to address the class imbalance problem in AHD applications with explicit considerations of participant privacy and device resource constraints. We evaluate FedSens using a real-world edge computing testbed on two real-world AHD applications. The results show that FedSens can significantly improve the accuracy of AHD models in the presence of severe class imbalance with low energy cost to the edge devices.

The proliferation of online Location-Based Social Networks (LBSN) has offered unprecedented opportunities for understanding fine-grained spatio-temporal behaviors of users and developing new location-aware applications. In this article, we focus on the problem of “Sparse User Check-in Venue Prediction,” where the goal is to predict the next venue LBSN users will visit by exploiting their sparse online check-in traces and the latent decision contexts. While efforts have been made to predict users’ check-in traces on a LBSN, several important challenges still exist. First, check-in traces contributed by LBSN users are often too sparse to provide sufficient evidence for a reliable prediction, especially when the prediction space is huge (e.g., hundreds of thousands of venues in large cities). Second, the user's decision context on which venue to visit next is often latent and has not been incorporated by current venue prediction models. Third, the dynamic and non-deterministic dependency between check-ins is either ignored or replaced by a simplified “consecutiveness” assumption in existing solutions, leading to sub-optimal prediction results. In this article, we develop aContext-aware Sparse Check-in Venue Prediction (CSCVP)scheme inspired by natural language processing techniques to address the above challenges. In particular, CSCVP predicts the venue category information and explores the similarity between users to address data sparsity challenge by significantly reducing the prediction space. It also leverages the Probabilistic Latent Semantic Analysis (PLSA) model to incorporate the user decision context into the prediction model. Finally, we develop a novel Temporal Adaptive Ngram (TA-Ngram) model in CSCVP to capture the dynamic and non-deterministic dependency between check-ins. We evaluate CSCVP using three real-world LBSN datasets. The results show that our scheme significantly improves accuracy (30.9 percent improvement) of the state-of-the-art user check-in venue prediction solutions.

The recent advances of mobile sensing and artificial intelligence (AI) have brought new revolutions in disaster response applications. One example is disaster scene assessment (DSA) which leverages computer vision techniques to assess the level of damage severity of the disaster events from images provided by eyewitnesses on social media. The assessment results are critical in prioritizing the rescue operations of the response teams. While AI algorithms can significantly reduce the detection time and manual labeling cost in such applications, their performance often falls short of the desired accuracy. Our work is motivated by the emergence of crowdsourcing platforms (e.g., Amazon Mechanic Turk, Waze) that provide unprecedented opportunities for acquiring human intelligence for AI applications. In this paper, we develop an interactive Disaster Scene Assessment (iDSA) scheme that allows AI algorithms to directly interact with humans to identify the salient regions of the disaster images in DSA applications. We also develop new incentive designs and active learning techniques to ensure reliable, timely, and cost-efficient responses from the crowdsourcing platforms. Our evaluation results on real-world case studies during Nepal and Ecuador earthquake events demonstrate that iDSA can significantly outperform state-of-the-art baselines in accurately assessing the damage of disaster scenes.

With the ubiquitous network connectivity and the proliferation of mobile devices, people are increasingly consuming digital contents from social media driven music sharing platforms (e.g., YouTube, Soundcloud). In this paper, we study a novel problem of connotation-aware music retrieval that focuses on the connotation which expresses the implicit feeling or emotion beyond the explicit content in artworks. Our goal is to automatically retrieve relevant music on social media based on the connotation of visual inputs (e.g., images, photos) provided by the users. The problem is challenging as it requires the accurate identification of the implicit connotation from both images and music pieces, and the precise matching of the identified connotation across different data modalities. We develop a connotation-aware music retrieval (CaMR) framework to address the above challenges. Evaluation results from a real-world social media dataset demonstrate that the CaMR framework can retrieve music that is highly relevant to the connotation of the input image.

Fauxtography is a category of multi-modal posts that spreads misleading information on various online social platforms (e.g., Facebook, Twitter, Reddit). A fauxtography post usually consists of an image, a text description and comments from its readers. In this paper, we focus on an explainable fauxtography detection problem where the goal is to explain which a specific component of a post leads to the fauxtography decision. This problem is motivated by the limitations of current fauxtography detection solutions that only focus on the detection but ignore the important explanation aspect of their results. Two critical challenges exist in solving our problem: i) it is difficult to accurately identify the "guilty" component of a fauxtography post given the fact that different components of the post and their associations could all lead to the fauxtography; ii) it is expensive and time-consuming to obtain a good training set with fine-grained labels of fauxtography posts in terms of explainability, making the corresponding solutions weakly supervised in nature. To address the above challenges, we develop ExFaux, an end-to-end graph-based fauxtography explanation framework, to effectively explain which part of the post contributes to its fauxtography. We evaluate the ExFaux by creating a real-world dataset from online social media (Twitter and Reddit). The results show that ExFaux not only detects the fauxtography posts more accurately than the state-of-the-arts but also provides well-justified explanations to its results.

The recent advance of the federated learning (FL) has brought new opportunities for privacy-aware distributed machine learning (ML) applications to train a powerful ML model without accessing the private training data of the participants. In this paper, we focus on addressing a novel fair classification problem in FL where the model trained by FL displays discriminatory bias towards particular demographic groups. Addressing the fairness issue in a FL framework posts three critical challenges: fairness and performance trade-offs, restricted information, and constrained coordination. To address these challenges, we develop FairFL, a fair federated learning framework dedicated to reducing the bias in privacy-sensitive ML applications. It consists of a principled deep multi-agent reinforcement learning framework and a secure information aggregation protocol that optimizes both the accuracy and the fairness of the learned model while respecting the strict privacy constraints of the clients. Evaluation results on real-world applications showed that FairFL can achieve significant performance gains in both fairness and accuracy of the learned model compared to state-of-the-art baselines.

Forest fires cause irreversible damages worldwide every year. Monitoring wildfire propagation is thus a vital task in mitigating forest fires. While computational model-based wildfire prediction methods provide reasonable accuracy in monitoring wildfire behavior, they are often limited due to the lack of constant availability of real-time meteorological data. In contrast, social-media-driven drone sensing (SDS) is emerging as a new sensing paradigm that detects the early signs of forest fires from online social media feeds and drives the drones for reliable sensing. However, due to the scarcity of social media data in remote regions and limited flight times of drones, SDS solutions often underperform in large-scale forest fires. In this paper, we present CompDrone, a wildfire monitoring framework that exploits the collective strengths of computational wildfire modeling and SDS for reliable wildfire monitoring. Two critical challenges exist to integrate computational modeling and SDS together: i) limited availability of social signals in the regions of a forest fire; and ii) predicting the regions of fire where the drones should be dispatched to. To solve the above challenges, the CompDrone framework leverages techniques from cellular automata, constrained optimization, and game theory. The evaluation results using a real-world wildfire dataset show that CompDrone outperforms the state-of-the-art schemes in effectively predicting wildfire propagation.

Social media sensing has emerged as a new disaster response application paradigm to collect real-time observations from online social media users about the disaster status. Due to the noisy nature of social media data, the task of identifying trustworthy information (referred to as "truth discovery") has been a crucial task in social media sensing. However, existing truth discovery solutions often fall short of providing accurate results in disaster response applications due to the spread of misinformation and difficulty of an efficient verification in such scenarios. In this paper, we present SocialDrone, a novel closed-loop social-physical active sensing framework that integrates social media and unmanned aerial vehicles (UAVs) for reliable disaster response applications. In SocialDrone, signals emitted from the social media are distilled to drive the drones to target areas to verify the emergency events. The verification results are then taken back to improve the sensing and distillation process on social media. The SocialDrone framework introduces several unique challenges: i) how to drive the drones using the unreliable social media signals? ii) How to ensure the system is adaptive to the high dynamics from both the physical world and social media? iii) How to incorporate real-world constraints (e.g., the deadlines of events, limited number of drones) into the framework? The SocialDrone addresses these challenges by building a novel integrated social-physical sensing system that leverages techniques from game theory, constrained optimization, and reinforcement learning. The evaluation results on a real-world disaster response application show that SocialDrone significantly outperforms state-of-the-art truth discovery schemes and drone-only solutions by providing more effective disaster response.

Recent advances in remote sensing open up unprecedented opportunities to obtain a rich set of visual features of objects on the earth's surface. In this paper, we focus on a single-image super-resolution (SISR) problem in remote sensing, where the objective is to generate a reconstructed satellite image of high quality (i.e., a high spatial resolution) from a satellite image of relatively low quality. This problem is motivated by the lack of high quality satellite images in many remote sensing applications (e.g., due to the cost of high resolution sensors, communication bandwidth constraints, and historic hardware limitations). Two important challenges exist in solving our problem: i) it is not a trivial task to reconstruct a satellite image of high quality that meets the human perceptual requirement from a single low quality image; ii) it is challenging to rigorously quantify the uncertainty of the results of an SISR scheme in the absence of ground truth data. To address the above challenges, we develop PQA-CNN, a perceptual quality-assured conventional neural network framework, to reconstruct a high quality satellite image from a low quality one by designing novel uncertainty-driven neural network architectures and integrating an uncertainty quantification model with the framework. We evaluate PQA-CNN on a real-world remote sensing application on land usage classifications. The results show that PQA-CNN significantly outperforms the state-of-the-art super-resolution baselines in terms of accurately reconstructing high-resolution satellite images under various evaluation scenarios.

Forecasting traffic accidents at a fine-grained spatial scale is essential to provide effective precautions and improve traffic safety in smart urban sensing applications. Current solutions primarily rely on complete historical traffic accident records and/or accurate real-time traffic sensor data for traffic risk prediction. These solutions are prone to various limitations (e.g., facility availability, privacy and legal constraints). In this paper, we address those limitations by exploring two types of widely available and complementary data sources: social media sensing and remote sensing. However, three important technical challenges exist: i) the social media data and remote sensing data often have different data modalities and characteristics, which makes it difficult to effectively fuse them for traffic risk forecasting; ii) the social media data is often noisy, providing fuzzy evidence to capture the spatial-temporal dynamics of traffic accident occurrences; iii) the remote sensing data often contains complex visual features, making it a non-trivial task to explore the accident occurrence correlation between different locations. To address the above challenges, we propose GraphCast, a multi-modal graph neural network framework to accurately forecast the traffic risks in a city by jointly exploring the social media sensing and remote sensing paradigms. The evaluation results on a real-world case study in New York City show that our GraphCast scheme significantly outperforms the state-of-the-art conventional and deep learning baselines in accurately forecasting the traffic risks in a city.

Recent advances in remote sensing provide a powerful and scalable sensing paradigm to capture abundant visual information about the urban environments. We refer to such a sensing paradigm as remote urban sensing. In this paper, we focus on a migratable satellite image super-resolution problem in remote urban sensing applications. Our goal is to reconstruct satellite images of a high resolution in a target area where the high-resolution training data is not available by transferring a super-resolution model learned in a source area where such data is available. This problem is motivated by the limitation of current solutions that primarily rely on a rich set of high-resolution satellite images in the studied area that are not always available. Two important challenges exist in solving our problem: i) the target and source areas often have very different urban characteristics that prevent the direct application of a super-resolution model learned from the source area to the target area; ii) it is not a trivial task to ensure effective model migration with desirable quality without sufficient high quality training data. To address the above challenges, we develop TransRes, a deep adversarial transfer learning framework, to effectively reconstruct high-resolution satellite images without requiring any ground-truth training data from the studied area. We evaluate the TransRes framework using the real-world satellite imagery data collected from three different cities in Europe. The results show that TransRes consistently outperforms the state-of-the-art baselines by achieving the lowest perception errors under various application scenarios.

With the advance in mobile computing, Internet of Things, and ubiquitous wireless connectivity, social sensing-based edge computing (SSEC) has emerged as a new computation paradigm where people and their personally owned devices collect sensor measurements from the physical world and process them at the edge of the network. This article focuses on a privacy-aware task allocation problem where the goal is to optimize the computation task allocation in SSEC systems while respecting the users' customized privacy settings. It introduces a novel game-theoretic privacy-aware task allocation (G-PATA) framework to achieve the goal. G-PATA includes: 1) a bottom-up game-theoretic model to generate the maximum payoffs at end devices while satisfying the end user's privacy settings and 2) a top-down incentive scheme to adjust the rewards for the tasks to ensure that the task allocation decisions made by end devices meet the Quality-of-Service (QoS) requirements of the applications. Furthermore, the framework incorporates an efficient load balancing and iteration reduction component to adapt to the dynamic changes in status and privacy configurations of end devices. The G-PATA framework was implemented on a real-world edge computing platform that consists of heterogeneous end devices (Jetson TX1 and TK1 boards, and Raspberry Pi3). We compare G-PATA with state-of-the-art task allocation schemes through two real-world social sensing applications. The results show that G-PATA significantly outperforms existing approaches under various privacy settings (our scheme achieved as much as 47% improvements in delay reduction for the application and 15% more payoffs for end devices compared to the baselines).

Social sensing has emerged as a new sensing paradigm to observe the physical world by exploring the "wisdom of crowd" on social media. This paper focuses on the abnormal traffic event localization problem using social media sensing. Two critical challenges exist in the state-of-the-arts: i) "content-only inference": the limited and unstructured content of a social media post provides little clue to accurately infer the locations of the reported traffic events; ii) "informal and scarce data": the language of the social media post (e.g., tweet) is informal and the number of the posts that report the abnormal traffic events is often quite small. To address the above challenges, we develop SyntaxLoc, a syntax-based probabilistic learning framework to accurately identify the location entities by exploring the syntax of social media content. We perform extensive experiments to evaluate the SyntaxLoc framework through real world case studies in both New York City and Los Angeles. Evaluation results demonstrate significant performance gains of the SyntaxLoc framework over state-of-the-art baselines in terms of accurately identifying the location entities that can be directly used to locate the abnormal traffic events.

With the increasing popularity of portable devices with cameras (e.g., smartphones and tablets) and ubiquitous Internet connectivity, travelers can share their instant experience during the travel by posting photos they took to social media platforms. In this paper, we present a new image-driven poetry recommender system that takes a traveler's photo as input and recommends classical poems that can enrich the photo with aesthetically pleasing quotes from the poems. Three critical challenges exist to solve this new problem: i) how to extract the implicit artistic conception embedded in both poems and images? ii) How to identify the salient objects in the image without knowing the creator's intent? iii) How to accommodate the diverse user perceptions of the image and make a diversified poetry recommendation? The proposed iPoemRec system jointly addresses the above challenges by developing heterogeneous information network and neural embedding techniques. Evaluation results from real-world datasets and a user study demonstrate that our system can recommend highly relevant classical poems for a given photo and receive significantly higher user ratings compared to the state-of-the-art baselines.

Road traffic accidents are a major challenge in urban transportation systems. An effective countermeasure to address this problem is to accurately forecast the traffic risks in a city before accidents actually happen. Current traffic accident prediction solutions largely rely on accurate data collected from infrastructure-based sensors, which is not always available due to various resource constraints or privacy and legal concerns. In this paper, we address this limitation by exploring social sensing, a new sensing paradigm that uses humans as sensors to report the states of the physical world. In particular, we consider two types of publicly available social sensing data sources: social media data (e.g., traffic posts on Twitter) and open city data (e.g., traffic data from the city web portal). In this paper, we develop the RiskCast, an inductive multi-view learning approach to accurately forecast the traffic risk by exploiting the social sensing data under a principled co-regularization framework. The evaluation results on a real world dataset from New York City show that RiskCast significantly outperforms the state-of-the-art baselines in forecasting the traffic risks in a city.

With the increasing popularity of online video platforms (e.g., YouTube, Vimeo), the spread of hateful videos and the lack of rigorous hateful content control have become a critical issue. This paper focuses on the problem of identifying online hatred-vulnerable videos where the videos themselves do not contain any hateful content but unexpectedly trigger hateful comments from the audience. It is suboptimal to simply treat the hatred-vulnerable videos as hateful ones and remove them from the sharing platforms. This will discourage the uploaders of such videos from sharing valid and informative videos in the future. However, treating these hatred-vulnerable videos as hatred-free ones will provide undesirable opportunities for hateful users to spread their toxic comments and extreme ideology. In this paper, we develop VulnerCheck, an end-to-end supervised learning approach to effectively classify hatred-vulnerable videos from hateful and hatred-free ones by exploring the structure and semantics features of audience's comment networks. VulnerCheck is content-agnostic in the sense that it does not analyze the content of the video and is therefore robust against sophisticated content creators who craft hateful videos to bypass the current content censorship. We evaluate VulnerCheck on a real-world dataset collected from YouTube. Results demonstrate that our scheme is both effective and efficient in identifying hatred-vulnerable videos and significantly outperforms the state-of-the-art baselines.

Urban land usage classification is a critical task in big data based smart city applications that aim to understand the social-economic land functions and physical land attributes in urban environments. This paper focuses on a migratable urban land usage classification problem using remote sensing data (i.e., satellite images). Our goal is to accurately classify the land usage of locations in a target city where the ground truth land usage data is not available by leveraging a classification model from a source city where such data is available. This problem is motivated by the limitation of current solutions that primarily rely on a rich set of ground-truth data for accurate model training, which encounters high annotation costs. Two important challenges exist in solving our problem: i) the target and source cities often have different urban characteristics that prevent the direct application of a model learned from the source city to the target city; ii) the complex visual features in satellite images make it non-trivial to “translate” the images from the target city to the source city for an accurate classification. To address the above challenges, we develop TransLand, an adversarial transfer learning framework to translate the satellite images from the target city to the source city for accurate land usage classification. We evaluate our scheme on the real-world satellite imagery and land usage datasets collected from live different cities in Europe. The results show that TransLand significantly outperforms the state-of-the-art land usage classification baselines in classifying the land usage of locations in a city.

This paper focuses on the migratable traffic risk estimation problem in intelligent transportation systems using the social (human-centric) sensing. The goal is to accurately estimate the traffic risk of a target area where the ground truth traffic accident reports are not available by leveraging an estimation model from a source area where such data is available. Two important challenges exist. The first challenge lies in the discrepancy between source and target areas (e.g., layouts, road conditions, and local regulations) and such discrepancy would prevent a direct application of a model from the source area to the target area. The second challenge lies in the difficulty of identifying all potential features in the migratable traffic risk estimation problem and decide the importance of identified features due to the lack of ground truth labels in the target area. To address these challenges, we develop DeepRisk, a social sensing based migratable traffic risk estimation scheme using deep transfer learning techniques. The evaluation results on a real world dataset in New York City show the DeepRisk significantly outperforms the state-of-the-art baselines in accurately estimating the traffic risk of locations in a city.

While autonomous unmanned aerial vehicles (UAVs) have attained a reputable stance in modern disaster response applications, their practical adoptions are impeded due to various constraints (e.g., requiring manual input, battery life). In this paper, we develop a novel spatiotemporal aware drone sensing system that is driven by harnessing social media signals, a process known as social sensing. Social sensing has emerged as a new sensing paradigm where humans act as "sensors" to report their observations about the physical world. However, maneuvering drones with "social signals" introduces a new realm of challenges. The first challenge is to drive the drones by leveraging noisy and unreliable social media signals. The second challenge is to optimize the drone deployment by exploring the highly dynamic and latent correlations among event locations. In this paper, we present CollabDrone that devises a new spatiotemporal correlation inference model and game-theoretic drone dispatching mechanism to address the above challenges. The evaluation results on a real-world case study show that CollabDrone significantly outperforms current drone and social sensing baselines in terms of accuracy and deadline hit rate.

This paper overviews the state of the art, research challenges, and future opportunities in an emerging research direction: Social Sensing based Edge Computing (SSEC). Social sensing has emerged as a new sensing application paradigm where measurements about the physical world are collected from humans or from devices on their behalf. The advent of edge computing pushes the frontier of computation, service, and data along the cloud-to-things continuum. The merging of these two technical trends generates a set of new research challenges that need to be addressed. In this paper, we first define the new SSEC paradigm that is motivated by a few underlying technology trends. We then present a few representative real-world case studies of SSEC applications and several key research challenges that exist in those applications. Finally, we envision a few exciting research directions in future SSEC. We hope this paper will stimulate discussions of this emerging research direction in the community.

Artificial Intelligence (AI) has been widely adopted in many important application domains such as speech recognition, computer vision, autonomous driving, and AI for social good. In this paper, we focus on the AI-based damage assessment applications where deep neural network approaches are used to automatically identify damage severity of impacted areas from imagery reports in the aftermath of a disaster (e.g., earthquake, hurricane, landslides). While AI algorithms often significantly reduce the detection time and labor cost in such applications, their performance sometimes falls short of the desired accuracy and is considered to be less reliable than domain experts. To exacerbate the problem, the black-box nature of the AI algorithms also makes it difficult to troubleshoot the system when their performance is unsatisfactory. The emergence of crowdsourcing platforms (e.g., Amazon Mechanic Turk, Waze) brings about the opportunity to incorporate human intelligence into AI algorithms. However, the crowdsourcing platform is also black-box in terms of the uncertain response delay and crowd worker quality. In this work, we propose the CrowdLearn, a crowd-AI hybrid system that leverages the crowdsourcing platform to troubleshoot, tune, and eventually improve the black-box AI algorithms by welding crowd intelligence with machine intelligence. The system is specifically designed for deep learning-based damage assessment (DDA) applications where the crowd tend to be more accurate but less responsive than machines. Our evaluation results on a real-world case study on Amazon Mechanic Turk demonstrate that CrowdLearn can provide timely and more accurate assessments to natural disaster events than the state-of-the-art AI-only and human-AI integrated systems.

Autonomous unmanned aerial vehicles (UAVs) have become an important tool for efficient disaster response. Despite the virtues of UAVs in disaster response applications, various limitations (e.g., requiring manual input, finite battery life) hinder their mass adoption. In contrast, social sensing is emerging as a new sensing paradigm that utilizes signals provided by "human sensors" to gather awareness of the events occurring in the physical world. Despite being inherently broader in scope, a shortcoming of social sensing is the reliability of the sensing data that are contributed by humans. In this paper, we introduce the concept of jointly exploiting the reliability of drones and the scope of social sensing to efficiently uncover the truthful events during disasters. However, such a tight integration of social and physical sensing introduces several technical challenges. The first challenge is satisfying the conflicting objectives of event coverage of the application and energy conservation of drones. The second challenge is adapting to the dynamics of the physical world and social media. In this paper, we present a Social-media-driven Energy-Aware Drone (SEAD) sensing framework to address the above challenges. In particular, we develop a reinforcement learning-based drone dispatching scheme that adapts to the physical and social environments and launches an appropriate proportion of drones for event exploration. We further utilize a bottom-up game-theoretic task allocation approach to guide drones effectively to the event locations. The evaluation with a real-world disaster case study show that SEAD noticeably outperforms state-of-the-art baselines in terms of detection effectiveness and energy efficiency.

With the advance of mobile computing, Internet of Things, and 5G networks, social sensing based edge computing (SSEC) systems have emerged as a new computation paradigm where people and their personally owned devices collect and process sensing measurements about the physical world at the edge of networks. In this paper, we focus on the task allocation problem in SSEC where rational edge devices are motivated by incentives to collectively accomplish the computation tasks in the system. Several unique challenges exist to solve this problem: (i) the edge devices often do not share the complete context information (e.g., CPU, memory usage) in the task allocation process due to privacy concerns; (ii) the edge devices are rational actors who may have competing objectives with the application; (iii) the application server and edge devices are usually owned by different entities, making the coordination in task allocation more challenging. This paper develops a novel integrated Top-Down and Bottom-Up (TDBU) task allocation framework to address these challenges. In particular, TDBU incorporates abottom-up game-theoretic model that allows the edge devices to specify their task preferences in a way that maximizes their payoffs. It also incorporates atop-down control model that ensures the performance of the applications using control theory. The TDBU was implemented on a real-world edge computing testbed that consists of heterogeneous devices (Jetson TX1, TK1 boards, Raspberry Pi3). We compared the performance of TDBU with state-of-the-art baselines through a real-world social sensing application. The results showed that our solution significantly outperformed the baselines in various application settings.

Vehicular sensor networks (VSNs) have become a reliable sensing instrument utilizing sensors built into cars. However, VSNs have limited sensing scopes since car drivers only opportunistically discover new events. Conversely, social sensing is protruding as a new sensing paradigm where measurements about the physical world are collected from humans. In this paper, we develop SocialCar, a task allocation framework for social media driven vehicular network sensing systems that exploits the collective powers of social sensing and VSNs for a robust sensing application. However, integrating VSNs with social sensing introduces a new set of challenges. The first challenge is leveraging the noisy and unreliable social signals to route vehicles to the regions of interest. The second challenge is the inconsistent availability attributed to car drivers being rational actors causing churn in the system. To address the above challenges, this paper introduces a bottom-up game-theoretic task preference model along with a top-down dynamic incentive control mechanism to allow car drivers to be selectively dispatched to event locations and verify the truthfulness of reported events. The results on a real-world disaster recovery case study show that SocialCar significantly outperforms current VSNs based solutions in detection effectiveness and deadline hit rate.

Modern Internet of Things (IoT) systems are increasingly leveraging deep neural networks (DNNs) with the goal of enabling intelligence at the edge of the network. While applying DNNs can greatly improve the accuracy of autonomous decisions and inferences, a significant challenge is that DNNs are traditionally designed and developed for advanced hardware (e.g., GPU clusters) and can not easily meet the real time requirements when deployed in a resource-constrained edge computing environment. While many systems have been proposed to facilitate deep learning at the edge, a key limitation lies in the under-utilization of the parallelizable GPU resources of edge nodes (e.g., IoT devices). In this paper, we propose EdgeBatch, a collaborative intelligent edge computing framework that minimizes the delay and energy consumption of executing DNN tasks at the edge by sharing idle GPU resources among privately owned IoT devices. EdgeBatch develops 1) a stochastic task batching mechanism that identifies the optimal batching strategy for the GPUs on IoT devices given uncertain task arrival times, and 2) a dynamic task offloading scheme that coordinates the collaboration among edge nodes to optimize the utilization of idle GPU resources in the system. We implemented EdgeBatch on a real-world edge computing testbed that consists of heterogeneous IoT devices (Jetson TX2, TX1, TK1, and Raspberry Pi3s). The results show that EdgeBatch achieved significant performance gains in terms of both the end-to-end delay and energy savings compared to the state-of-the-art baselines.

Social sensing based Edge Computing (SSEC) is an emerging application paradigm for rich context awareness in sensing applications in which people collaborate in both data acquisition and processing at the edge of the network. While keeping people in the loop can be an immense benefit, the unreliability and churn introduced can pose a dangerous stability threat for long-running SSEC applications. In the past this problem was addressed by offloading these responsibilities to more reliable server hardware in the cloud or a fog layer, however, this does not take full advantage of the power at the edge. In this paper we address the issue of reliable edge computing on dynamic high-churn edge systems. We develop a deviceless pipeline based approach (DPA) to establish workflows in which stages of the analysis pipeline are completed on edge devices, and any devices that leave the system can be replaced without data loss. We evaluate the performance of our system on a real-world edge computing system performing an object detection application and demonstrate that it can provide significant performance gains over traditional computation offloading schemes in terms of throughput and error recovery.

Identifying trustworthy information in the presence of noisy data contributed by numerous unvetted sources from online social media (e.g., Twitter, Facebook, and Instagram) has been a crucial task in the era of big data. This task, referred to as truth discovery, targets at identifying the reliability of the sources and the truthfulness of claims they make without knowing either a priori. In this work, we identified three important challenges that have not been well addressed in the current truth discovery literature. The first one is “misinformation spread” where a significant number of sources are contributing to false claims, making the identification of truthful claims difficult. For example, on Twitter, rumors, scams, and influence bots are common examples of sources colluding, either intentionally or unintentionally, to spread misinformation and obscure the truth. The second challenge is “data sparsity” or the “long-tail phenomenon” where a majority of sources only contribute a small number of claims, providing insufficient evidence to determine those sources' trustworthiness. For example, in the Twitter datasets that we collected during real-world events, more than 90 percent of sources only contributed to a single claim. Third, many current solutions are not scalable to large-scale social sensing events because of the centralized nature of their truth discovery algorithms. In this paper, we develop a Scalable and Robust Truth Discovery (SRTD) scheme to address the above three challenges. In particular, the SRTD scheme jointly quantifies both the reliability of sources and the credibility of claims using a principled approach. We further develop a distributed framework to implement the proposed truth discovery scheme using Work Queue in an HTCondor system. The evaluation results on three real-world datasets show that the SRTD scheme significantly outperforms the state-of-the-art truth discovery methods in terms of both effectiveness and efficiency.

Combating copyright infringing multimedia content has arisen as a critical undertaking in online video sharing platforms, such as YouTube and Twitch. In contrast to the traditional copyright detection problem that studies the static content (e.g., music, films, digital documents), the proposed system focuses on a much more challenging problem: detecting copyright infringements in live video streams. This is motivated by the observation that a large amount of copyright-infringing videos bypass the detector while many legal videos are taken down by mistake. In this paper, we present an end-to-end system that is dedicated to combating the copyright infringements in live video streams. The system to be demonstrated consists of 1) a web front-end for user interaction and customized video query, 2) a scalable and real-time video crawling system that can collect video metadata, live chat messages, and visual content of the live video streams on video sharing platforms, and 3) a novel supervised copyright detection engine that leverages the live chat messages of the audience to detect the copyright infringement of live videos.

Social media sensing has emerged as a new application paradigm to collect observations from online social media users about the physical environment. A fundamental problem in social media sensing applications lies in estimating the evolving truth of the measured variables and the reliability of data sources without knowing either of them a priori. This problem is referred to as dynamic truth discovery. Two major limitations exist in current truth discovery solutions: i) existing solutions cannot effectively address the missing truth problem where the measured variables do not have any reported measurements from the data sources; ii) the latent correlations among the measured variables were not fully captured and utilized in current solutions. In this paper, we proposed a Reliable Missing Truth Finder (RMTF) to address the above limitations in social media sensing applications. In particular, we develop a novel data-driven technique to identify the lagged and latent correlations among measured variables, and incorporate such correlation information into a holistic spatiotemporal inference model to infer the missing truth. We evaluated the RMTF using the real-world Twitter data feeds. The results show that the RMTF scheme significantly outperforms the state-of-the-art truth discovery solutions by correctly inferring the missing truth of the measured variables.

With the increasing popularity of online video sharing platforms (such as YouTube and Twitch), the detection of content that infringes copyright has emerged as a new critical problem in online social media. In contrast to the traditional copyright detection problem that studies the static content (e.g., music, films, digital documents), this paper focuses on a much more challenging problem: one in which the content of interest is from live videos. We found that the state-of-the-art commercial copyright infringement detection systems, such as the ContentID from YouTube, did not solve this problem well: large amounts of copyright-infringing videos bypass the detector while many legal videos are taken down by mistake. In addressing the copyright infringement detection problem for live videos, we identify several critical challenges: i) live streams are generated in real-time and the original copyright content from the owner may not be accessible; ii) streamers are getting more and more sophisticated in bypassing the copyright detection system (e.g., by modifying the title, tweaking the presentation of the video); iii) similar video descriptions and visual contents make it difficult to distinguish between legal streams and copyright-infringing ones. In this paper, we develop a crowdsourcing-based copyright infringement detection (CCID) scheme to address the above challenges by exploring a rich set of valuable clues from live chat messages. We evaluate CCID on two real world live video datasets collected from YouTube. The results show our scheme is significantly more effective and efficient than ContentID in detecting copyright-infringing live videos on YouTube.

With the ever-increasing number of road traffic accidents worldwide, the road traffic safety has become a critical problem in intelligent transportation systems. A key step towards improving the road traffic safety is to identify the locations where severe traffic accidents happen with a high probability so the precautions can be applied effectively. We refer to this problem as risky traffic location identification. While previous efforts have been made to address similar problems, two important limitations exist: i) data availability: many cities (especially in developing countries) do not maintain a publicly accessible database for the traffic accident records in a city, which makes it difficult to accurately estimate the accidents in the city; ii) location accuracy: many self-reported traffic accidents (e.g., social media posts from common citizens) are not associated with the exact GPS locations due to the privacy concerns. To address these limitations, this paper develops the RiskSens, a multi-view learning approach to identifying the risky traffic locations in a city by jointly exploring the social and remote sensing data. We evaluate RiskSens using a real world dataset from New York. The evaluation results show that RiskSens significantly outperforms the state-of-the- art baselines in identifying risky traffic locations in a city.

With the increasing popularity of online social media (e.g., Facebook, Twitter, Reddit), the detection of misleading content on social media has become a critical undertaking. This paper focuses on an important but largely unsolved problem: detecting fauxtography (i.e., social media posts with misleading images). We found that the existing literature falls short in solving this problem. In particular, current solutions either focus on the detection of fake images or misinformed texts of a social media post. However, they cannot solve our problem because the detection of fauxtography depends not only on the truthfulness of the images and the texts but also on the information they deliver together on the posts. In this paper, we develop the FauxBuster, an end-to-end supervised learning scheme that can effectively track down fauxtography by exploring the valuable clues from user's comments of a post on social media. The FauxBuster is content-free in that it does not rely on the analysis of the actual content of the images, and hence is robust against malicious uploaders who can intentionally modify the presentation and description of the images. We evaluate FauxBuster on real-world data collected from two mainstream social media platforms - Reddit and Twitter. Results show that our scheme is both effective and efficient in addressing the fauxtography problem.

Copyright infringement detection is a critical problem in large-scale online video sharing systems: the copyright-infringing videos must be correctly identified and removed from the system to protect the copyright of the content owners. This paper focuses on a challenging problem of detecting copyright infringement in live video streams. The problem is particularly difficult because i) streamers can be sophisticated and modify the title or tweak the presentation of the video to bypass the detection system; ii) legal videos and copyright-infringing ones may have very similar visual content and descriptions. We found current commercial copyright detection systems did not address this problem well: a large amount of copyrighted content bypasses the detection system while legal streams are taken down by mistake. In this paper, we develop the StreamGuard, an unsupervised Bayesian network based copyright infringement detection system that addresses the above challenges by leveraging live chat messages from the audience. We evaluate StreamGuard on real-world live video streams collected from YouTube. The results show that StreamGuard is effective and efficient in identifying the copyright-infringing videos.

Social sensing has emerged as a new application paradigm in networked sensing where data is collected from humans or devices on their behalf. This paper focuses on the opinion characterization problem in social sensing where the goal is to accurately characterize opinion attributes of the participants (e.g., analyze the sentiments, understand the opinion bias) from their sensor measurements. Several important challenges exist in solving the opinion characterization problem. First, human sensors often generate unstructured data (e.g., text, image, video) in which the opinion attributes are deeply embedded. Second, human sensors naturally generate measurements in different data modalities which encode the opinion attributes differently. Third, the possible imbalance between different data modalities may lead to potential bias in the opinion characterization results. To address the above challenges, this paper develops a Multi-View Opinion Characterization (MVOC) scheme to accurately characterize opinion attributes using a multi-view subspace learning approach. We evaluate the MVOC scheme through the real-world social sensing task of classifying the sentiments of reports from Twitter users. The evaluation results show that our scheme significantly outperforms the state-of-the-art baselines in solving the opinion characterization problem.

Social sensing has emerged as a new sensing paradigm where humans (or devices on their behalf) collectively report measurements about the physical world. This paper focuses on an optimized task allocation problem in multi- attribute social sensing applications where the goal is to effectively allocate the tasks of collecting multiple attributes of the measured variables to human sensors while respecting the application's budget constraints. While recent progress has been made to tackle the optimized task allocation problem, two important challenges have not been well addressed. The first challenge is "online task allocation": the task allocation schemes need to respond quickly to the potentially large dynamics of the measured variables (e.g., temperature, noise, traffic) in social sensing. Delayed task allocation may lead to inaccurate sensing results and/or unnecessarily high sensing costs. The second challenge is the "multi-attribute constrained optimization": minimizing the overall sensing error given the dependencies and constraints of multiple attributes of the measured variables is a non-trivial problem to solve. To address the above challenges, this paper develops an Online Optimized Multi-attribute Task Allocation (OO-MTA) scheme inspired by techniques from machine learning and information theory. We evaluate the OO-MTA scheme using an urban sensing dataset collected from a real-world social sensing application. The evaluation results show that OO- MTA scheme significantly outperforms the state-of-the-art baselines in terms of the sensing accuracy.

Privacy on the Internet has become a major concern in recent years and is an ethical dilemma for Internet based applications. Social Sensing based Edge Computing (SSEC)is a category of distributed applications in which privately owned devices at the “edge” of the network participate in data collection and processing. Traditionally in SSEC, it has been difficult to manage privacy without making a tradeoff with performance or accuracy. In this paper, we present Privacy-aware Edge Computing (PEC), a framework that utilizes ring signatures for implementing social sensing applications with full privacy and without sacrificing data quality. Under PEC, privacy is managed by the edge devices without the need of a third party trusted authority. We evaluate the performance of PEC on a realworld edge computing platform and through extensive simulation studies. The results demonstrate that PEC achieves a high level of anonymity with a constrained cryptographic overhead.

Social Sensing has emerged as a new distributed application paradigm in networked sensing where streaming data is constantly collected from humans or devices on their behalf. A key challenge in social sensing is that the applications are often both data-intensive and delay-sensitive, which often require extensive resources in the cloud to support the data processing and analytics tasks. To address this problem, this paper focuses on a quality-aware online adaptive sampling problem where the goal is to judiciously sample a small set of representative sensing measurements (i.e., representative set) that effectively represent the key information of the social sensing data stream. However, two technical challenges exist in identifying such a representative set: (i) “quality-aware quantification”: data collected in the social sensing applications is often unstructured (e.g., text, image, video) where the key information is so deeply embedded in the raw data that can not be easily quantified to ensure the desirable quality of the selected representative set; (ii) “online adaptive sampling”: the sampling decision has to be made in real-time and be adaptive to the dynamics of the streaming data in social sensing. To address these challenges, this paper develops a Light-weight and Quality-aware Online Adaptive Sampling (LQOAS) scheme to dynamically identify the representative set of measurements from the streaming social sensing data using a submodular maximization approach. We evaluate the LQOAS scheme using a real world social sensing dataset from Twitter. The evaluation results show that the LQOAS scheme significantly outperforms the state-of-the-art sampling schemes.

In crowdsensing applications, participants (crowd sensors) work collectively to report their measurements about the physical world. This paper focuses on the optimized online task allocation problem in cost-sensitive crowdsensing applications where the goal is to dynamically allocate the sensing tasks to participants to meet the requirement of the applications while minimizing the sensing costs. Recent progress has been made to tackle the task allocation problem in crowdsensing. However, two important challenges have not been well addressed: i) “physical dynamics”: the values of the measured variables in crowdsensing often change significantly over time and space. It is essential for the task allocation schemes to adapt to such changes efficiently to optimize the task allocation process; ii) “crowd irregularity”: the number of participants in crowdsensing is often smaller than the number of desirable sensing locations and not all crowd sensors contribute data all the time (e.g., due to incentive or budget constraints). To address the above challenges, this paper develops an Online Optimized Task Allocation (OO-TA) scheme inspired by techniques from information theory and online learning. We evaluate the OO-TA scheme using a dataset collected from a real-world crowdsensing application. The evaluation results show that OO-TA scheme significantly outperforms the state-of-the-art baselines in terms of both effectiveness and efficiency.

Collaborative sensing has emerged as a new sensing paradigm where sensors collaboratively report measurements about the phenomena or events in the physical world. This paper focuses on a robust state prediction problem in collaborative sensing where the goal is to provide an accurate prediction on the dynamic status of the physical environment (e.g., air quality index, temperature, traffic) based on the incomplete and noisy data contributed by collaborative sensors. While significant progress has been made to study the state prediction problem, we identify two fundamental challenges that have not been well addressed by the current literature. The first challenge is "latent spatial dynamics": the spatial correlations among measured variables are highly dynamic and the features that affect such dynamics may not be directly observable from the sensing data. The second challenge is "incomplete and noisy data": a significant amount of measurements are missing in collaborative sensing due to the prohibitively high cost of deploying sensors for full spatial-temporal coverage. The collaborative sensors are also known to be unreliable in many applications and can easily contribute to noisy measurements. To address these challenges, this paper develops a Context-Aware Collaborative Sensing Prediction (CACSP) scheme inspired by techniques from latent semantic analysis and statistics. We evaluate our scheme through two real-world collaborative sensing applications and the results show that CACSP significantly outperforms the state-of-the-art baselines.

Social sensing has emerged as a new sensing application paradigm where measurements about the physical world are collected from humans or devices on their behalf. A key limitation in the current social sensing solution space is that data processing and analytics are often done in a "backend" mode (e.g., on dedicated servers or commercial cloud platforms). Such mode ignores the rich processing capability of increasingly powerful edge devices (e.g., mobile phones and nodes in the Internet of Things). Exploiting such edge devices in the social sensing setting introduces new challenges to real-time resource management. In this work, we develop a Bottom-up Game-theoretic Task Allocation (BGTA) framework to solve the critical problem of allocating real-time social sensing tasks to self-aware and non-cooperative edge computing nodes. In particular, we address two important challenges in solving this problem. The first one is "conflicting interest" where the objectives of applications and edge nodes may be at odds with each other. The second challenge is "asymmetric and incomplete information" where the application is often unaware of the detailed status (e.g., energy profile, utilization, CPU frequency) and compliance level of the edge nodes. To address these challenges, we first design a non-cooperative task allocation game model to address the conflicting objectives of the applications and edge nodes. We then develop a decentralized Fictitious Play scheme to allow each edge node to make its own decision on which task to execute in a non-cooperative context. Finally, we design a dynamic incentive mechanism to ensure the decisions made by the edge nodes meet objectives of the application. We implement a system prototype deployed on Nvidia Jetson TX1 and Jetson TK1 boards and evaluate our task allocation framework using two real-world social sensing applications. The results show that our scheme can well satisfy Quality of Service (QoS) requirement of the applications while providing optimized payoffs to edge nodes compared to the state-of-the-art baselines.

In this demo, we present a Social Sensing Edge Computing (SSEC) system that supports real-time task allocation on non-cooperative and heterogeneous edge devices. We evaluate the system on two real-time social sensing applications and demonstrate the energy savings as well as the Quality of Service (QoS) of the SSEC system. The demonstrated framework complements the traditional centralized social sensing solutions by providing a more scalable and responsive computation paradigm.

Social sensing has become a new crowdsourcing application paradigm where humans function as sensors to report their observations about the physical world. While many previous studies in social sensing focus on the problem of ascertaining the reliability of data sources and the truthfulness of their reported claims (often known as truth discovery), this paper investigates a new problem of hypothesis validation where the goal is to validate some high-level statements (referred to as hypotheses) from the low-level statements (referred to as claims) embedded in the social sensing data. The truthfulness of hypotheses cannot be directly obtained from the truth discovery results and two key challenges are involved in solving the hypothesis validation problem: (i) how to match the hypotheses generated by end users to the relevant claims generated by social sensors? (ii) How to accurately validate the truthfulness of the hypotheses given the unknown reliability of data sources and unvetted truthfulness of the claims? This paper proposes a Reliable Hypothesis Validation (RHV) scheme to address the above challenges. In particular, we develop a critical claim selection approach to match the hypotheses with the relevant claims and derive an optimal solution to validate their truthfulness by exploring the complex relationship between hypotheses and claims. The performance of RHV scheme is evaluated on three datasets collected from real- world social sensing applications. The results show that the RHV scheme significantly outperformed the state-of-the-art baselines in terms of validating the truthfulness of hypotheses.

Research in the field of Social Sensing Edge Computing (SSEC) faces a challenge in that most systems are developed on a small set of hardware and are never tested on large-scale networks. To address this issue, we present a Social Sensing Edge Computing Simulation Platform (SSEC SP) that is used to evaluate how SSEC systems scale to millions of devices. We demonstrate the platform by simulating several SSEC configurations to optimize Quality of Service (QoS) for large numbers of devices, while asserting that simulated results are consistent with the hardware implementations.

Social media sensing has emerged as a new big data application paradigm to collect observations and claims about the measured variables in physical environment from common citizens. A fundamental problem in social media sensing applications lies in estimating the evolving truth of claims and the reliability of data sources without knowing either of them a priori, which is referred to as dynamic truth discovery. We identified two critical challenges that are not fully addressed by solutions from current literature. The first challenge is “physical constraint-awareness” where the transition of truth is constrained by some physical rules that must be followed to ensure correct estimation of the evolving truth. The second one is “noisy and incomplete data” where the social media sensing data is sparse in nature and contains a lot of rumors and misinformation, making it difficult to capture the constantly evolving truth of measured variables. In this paper, we developed a new Constraint-Aware Dynamic Truth Discovery (CA-DTD) scheme to address the above challenges. To address the physical constraint-awareness challenge, CA-DTD develops a new constraint-aware Hidden Markov Model to effectively infer the evolving truth of measured variables by incorporating physical constraints. To address the noisy and incomplete data challenge, CA-DTD fuses sensing observations from online social media with information from traditional news media using a principled approach. We evaluate CA-DTD scheme using two real-world social media sensing data traces and the results show that CA-DTD significantly outperforms the state-of-the-art baselines.

Point-of-Interest (POI) recommendation is an important application in Location-based Social Networks (LBSN). The category prediction problem is to predict the next POI category that users may visit. The predicted category information is critical in large-scale POI recommendation because it can significantly reduce the prediction space and improve the recommendation accuracy. While efforts have been made to address the POI category prediction problem, several important challenges still exist. First, existing solutions did not fully explore the temporal dependency (e.g., “long range dependency”) of users' check-in traces. Second, the hidden contextual information associated with each check-in point has been underutilized. In this work, we propose a Context-Aware POI Category Prediction (CAP-CP) scheme using Natural Language Processing (NLP) models. In particular, to address temporal dependency challenge, we develop a novel Temporal Adaptive Ngram (TA-Ngram) model to capture the dynamic dependency between check-in points. To address the challenge of hidden context incorporation, CAP-CP leverages the Probabilistic Latent Semantic Analysis (PLSA) model to infer the semantic implications of the context variables in the prediction model. Empirical results on a real world dataset show that our scheme can effectively improve the performance of the state-of-the-art POI recommendation solutions.

With the rapid growth of online social media and ubiquitous Internet connectivity, social sensing has emerged as a new crowdsourcing application paradigm of collecting observations (often called claims) about the physical environment from humans or devices on their behalf. A fundamental problem in social sensing applications lies in effectively ascertaining the correctness of claims and the reliability of data sources without knowing either of them a priori, which is referred to as truth discovery. While significant progress has been made to solve the truth discovery problem, some important challenges have not been well addressed yet. First, existing truth discovery solutions did not fully solve the dynamic truth discovery problem where the ground truth of claims changes over time. Second, many current solutions are not scalable to large-scale social sensing events because of the centralized nature of their truth discovery algorithms. Third, the heterogeneity and unpredictability of the social sensing data traffic pose additional challenges to the resource allocation and system responsiveness. In this paper, we developed a Scalable Streaming Truth Discovery (SSTD) solution to address the above challenges. In this paper, we developed a Scalable Streaming Truth Discovery (SSTD) solution to address the above challenges. In particular, we first developed a dynamic truth discovery scheme based on Hidden Markov Models (HMM) to effectively infer the evolving truth of reported claims. We further developed a distributed framework to implement the dynamic truth discovery scheme using Work Queue in HTCondor system. We also integrated the SSTD scheme with an optimal workload allocation mechanism to dynamically allocate the resources (e.g., cores, memories) to the truth discovery tasks based on their computation requirements. We evaluated SSTD through real world social sensing applications using Twitter data feeds. The evaluation results on three real-world data traces (i.e., Boston Bombing, Paris Shooting and College Football) show that the SSTD scheme is scalable and outperforms the state-of-the-art truth discovery methods in terms of both effectiveness and efficiency.

Users' home location is important information for many advanced information services in big data applications (e.g., localized recommendation, target ads of local business and urban planning). In this paper, we study the problem of accurately inferring the home locations of people from the noisy and sparse data they voluntarily share on online social media. Previous studies have developed supervised learning approaches to predict a person's home location in a city. However, the accuracy of these techniques largely depends on a high quality training dataset, which is difficult and expensive to obtain in practice. In this study, we propose a new analytical framework, Unsupervised Home Location Inference (UHLI), to accurately infer the home locations of people using a set of principle approaches. In particular, the UHLI scheme addresses the critical challenges of using sparse and noisy online social media data and derives an optimal solution to the home location inference problem. We evaluated the performance of our scheme and compared it to the state-of-the-art baselines using three real world data traces collected from Foursquare. The results showed that our scheme can accurately infer the home location of people and significantly outperform the state-of-the-art baselines.

In the big data era, it's important to identify trustworthy information from an influx of noisy data contributed by unvetted sources from online social media (e.g., Twitter, Instagram, Flickr). This task is referred to as truth discovery which aims at identifying the reliability of the sources and the truthfulness of claims they make without knowing either of them a priori. There are two important challenges that have not been well addressed in current truth discovery solutions. The first one is “misinformation spread” where a majority of sources are contributing to false claims, making the identification of truthful claims difficult. The second challenge is “data sparsity” where sources contribute a small number of claims, providing insufficient evidence to accomplish the truth discovery task. In this paper, we developed a Robust Truth Discovery (RTD) scheme to address the above two challenges. In particular, the RTD scheme explicitly quantifies different degrees of attitude that a source may express on a claim and incorporates the historical contributions of a source using a principled approach. The evaluation results on two real world datasetsshow that the RTD scheme significantly outperforms the state-of-the-art truth discovery methods.