Machine learning models often make predictions based on biased features such as gender, race, and other social attributes, posing significant fairness risks, especially in societal applications, such as hiring, banking, and criminal justice. Traditional approaches to addressing this issue involve retraining or fine-tuning neural networks with fairness-aware optimization objectives. However, these methods can be impractical due to significant computational resources, complex industrial tests, and the associated CO2 footprint. Additionally, regular users often fail to fine-tune models because they lack access to model parameters. In this paper, we introduce the Inference-Time Rule Eraser (Eraser), a novel method designed to address fairness concerns by removing biased decision-making rules from deployed models during inference without altering model weights. We begin by establishing a theoretical foundation for modifying model outputs to eliminate biased rules through Bayesian analysis. Next, we present a specific implementation of Eraser that involves two stages: (1) distilling the biased rules from the deployed model into an additional patch model, and (2) removing these biased rules from the output of the deployed model during inference. Extensive experiments validate the effectiveness of our approach, showcasing its superior performance in addressing fairness concerns in AI systems.

The Internet has extended the physical boundary of people's social circles to manage an inordinate number of online friends. It is recognized that only a fraction of these online friends are also known with each other in offline circumstances, i.e., the offline friends. An important type of offline friend, onsite offline friend, is defined and addressed in this paper. We explores the possibility of utilizing users' online photo sharing-related behaviors and network topologies to analyze and distinguish between online and onsite offline friendships. Different from traditional social science studies which rely on survey-based data, we employ users' tagged people on the shared Instagram photos as the ground-truth for onsite offline friends. This enables a large-scale and objective analysis and experimental evaluation, which compares between different factors and identifies the features that are key to onsite offline friend identification.

Image tweets are becoming a prevalent form of socialmedia, but little is known about their content — textualand visual — and the relationship between the two mediums.Our analysis of image tweets shows that while visualelements certainly play a large role in image-text relationships, other factors such as emotional elements, also factor into the relationship. We develop Visual-Emotional LDA (VELDA), a novel topic model to capturethe image-text correlation from multiple perspectives (namely, visual and emotional). Experiments on real-world image tweets in both Englishand Chinese and other user generated content, show that VELDA significantly outperforms existingmethods on cross-modality image retrieval. Even in other domains where emotion does not factor in imagechoice directly, our VELDA model demonstrates good generalization ability, achieving higher fidelity modeling of such multimedia documents.

We study how users of multiple online social networks (OSNs) employ and share information by studying a common user pool that use six OSNs -- Flickr, Google+, Instagram, Tumblr, Twitter, and YouTube. We analyze the temporal and topical signature of users' sharing behaviour, showing how they exhibit distinct behaviorial patterns on different networks. We also examine cross-sharing (i.e., the act of user broadcasting their activity to multiple OSNs near-simultaneously), a previously-unstudied behaviour and demonstrate how certain OSNs play the roles of originating source and destination sinks.

In social media, users have contributed enormous behavior data online which can be leveraged for user modeling and conduct personalized services. Temporal user modeling, which incorporates the timestamp of these behavior data and understands users' interest evolution, have attracted attention recently. With the recognition that user interests are vulnerable to transient events, many current temporal user modeling solutions propose to first identify the transient events and then consider the identified events into user behavior modeling. In this work, in the context of microblogs, we propose a unified probabilistic framework to simultaneously model the process of transient event detection and temporal user tweeting. The outputs of the framework include: (1) one long-term topic space spanning over general categories, (2) one short-term topic space for each time interval corresponding to the transient events, and (3) users' interest distributions over the long- and short-term topic spaces. Qualitative and quantitative experimental evaluation are conducted on a large-scale Twitter dataset, with more than 2 million users and 0.3 billion tweets. The promising results demonstrate the advantage of the proposed topic models.

In the context of social media services, data shortage has severally hindered accurate user modeling and practical personalized applications. This paper is motivated to leverage the user data distributed in disparate online social networks (OSN) to make up for the data shortage in user modeling, which we refer to as “cross-OSN user modeling.” Generally, the data that the same user distributes in different OSNs consist of both behavior data (i.e., interaction with multimedia items) and social data (i.e., interaction between users). This paper focuses on the following two challenges: 1) how to aggregate the users' cross-OSN interactions with multimedia items of the same modality, which we call cross-OSN homogeneous behaviors, and 2) how to integrate users' cross-OSN social data with behavior data. Our proposed solution to address the challenges consist of two corresponding components as follows. 1) Homogeneous behavior quantification, where homogeneous user behaviors are quantified based on their importance in reflecting user preferences. After quantification, the examined cross-OSN user behaviors are aggregated to construct a unified user-item interaction matrix. 2) Local social regularization, where the cross-OSN social data is integrated as regularization in matrix factorization-based user modeling at local topic level. The proposed cross-OSN user modeling solution is evaluated in the application of personalized video recommendation. Carefully designed experiments on self-collected Google+ and YouTube datasets have validated its effectiveness and the advantage over single-OSN-based methods.

Social media platforms now allow users to share images alongside their textual posts. These image tweets make up a fast-growing percentage of tweets, but have not been studied in depth unlike their text-only counterparts. We study a large corpus of image tweets in order to uncover what people post about and the correlation between the tweet's image and its text. We show that an important functional distinction is between visually-relevant and visually-irrelevant tweets, and that we can successfully build an automated classifier utilizing text, image and social context features to distinguish these two classes, obtaining a macro F1 of 70.5%.

Increasingly developed social sharing websites like Flickr and Youtube allow users to create, share, annotate, and comment medias. The large-scale user-generated metadata not only facilitate users in sharing and organizing multimedia content, but provide useful information to improve media retrieval and management. Personalized search serves as one of such examples where the web search experience is improved by generating the returned list according to the modified user search intents. In this paper, we exploit the social annotations and propose a novel framework simultaneously considering the user and query relevance to learn to personalized image search. The basic premise is to embed the user preference and query-related search intent into user-specific topic spaces. Since the users' original annotation is too sparse for topic modeling, we need to enrich users' annotation pool before user-specific topic spaces construction. The proposed framework contains two components: 1) a ranking-based multicorrelation tensor factorization model is proposed to perform annotation prediction, which is considered as users' potential annotations for the images; 2) we introduce user-specific topic modeling to map the query relevance and user preference into the same user-specific topic space. For performance evaluation, two resources involved with users' social activities are employed. Experiments on a large-scale Flickr dataset demonstrate the effectiveness of the proposed method.