The utilization of automated depression detection significantly enhances early intervention for individuals experiencing depression. Despite numerous proposals on automated depression detection using recorded clinical interview videos, limited attention has been paid to considering the hierarchical structure of the interview questions. In clinical interviews for diagnosing depression, clinicians use a structured questionnaire that includes routine baseline questions and follow-up questions to assess the interviewee's condition. This paper introduces HiQuE (Hierarchical Question Embedding network), a novel depression detection framework that leverages the hierarchical relationship between primary and follow-up questions in clinical interviews. HiQuE can effectively capture the importance of each question in diagnosing depression by learning mutual information across multiple modalities. We conduct extensive experiments on the widely-used clinical interview data, DAIC-WOZ, where our model outperforms other state-of-the-art multimodal depression detection models and emotion recognition models, showcasing its clinical utility in depression detection.

Daeun Lee, Hyolim Jeon, Sejung Son, Chaewon Park, Ji hyun An, Seungbae Kim, Jinyoung Han. Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 2024.

Aphasia, a language disorder resulting from brain damage, requires accurate identification of specific aphasia types, such as Broca's and Wernicke's aphasia, for effective treatment.However, little attention has been paid to developing methods to detect different types of aphasia.Recognizing the importance of analyzing co-speech gestures for distinguish aphasia types, we propose a multimodal graph neural network for aphasia type detection using speech and corresponding gesture patterns.By learning the correlation between the speech and gesture modalities for each aphasia type, our model can generate textual representations sensitive to gesture information, leading to accurate aphasia type detection.Extensive experiments demonstrate the superiority of our approach over existing methods, achieving stateof-the-art results (F1 84.2%).We also show that gesture features outperform acoustic features, highlighting the significance of gesture expression in detecting aphasia types.We provide the codes for reproducibility purposes 1 .

As influencers play considerable roles in social media marketing, companies increase the budget for influencer marketing. Hiring effective influencers is crucial in social influencer marketing, but it is challenging to find the right influencers among hundreds of millions of social media users. In this paper, we propose InfluencerRank that ranks influencers by their effectiveness based on their posting behaviors and social relations over time. To represent the posting behaviors and social relations, the graph convolutional neural networks are applied to model influencers with heterogeneous networks during different historical periods. By learning the network structure with the embedded node features, InfluencerRank can derive informative representations for influencers at each period. An attentive recurrent neural network finally distinguishes highly effective influencers from other influencers by capturing the knowledge of the dynamics of influencer representations over time. Extensive experiments have been conducted on an Instagram dataset that consists of 18,397 influencers with their 2,952,075 posts published within 12 months. The experimental results demonstrate that InfluencerRank outperforms existing baseline methods. An in-depth analysis further reveals that all of our proposed features and model components are beneficial to discover effective influencers.

Bipolar disorder (BD) is closely associated with an increased risk of suicide. However, while the prior work has revealed valuable insight into understanding the behavior of BD patients on social media, little attention has been paid to developing a model that can predict the future suicidality of a BD patient. Therefore, this study proposes a multi-task learning model for predicting the future suicidality of BD patients by jointly learning current symptoms. We build a novel BD dataset clinically validated by psychiatrists, including 14 years of posts on bipolar-related subreddits written by 818 BD patients, along with the annotations of future suicidality and BD symptoms. We also suggest a temporal symptom-aware attention mechanism to determine which symptoms are the most influential for predicting future suicidality over time through a sequence of BD posts. Our experiments demonstrate that the proposed model outperforms the state-of-the-art models in both BD symptom identification and future suicidality prediction tasks. In addition, the proposed temporal symptom-aware attention provides interpretable attention weights, helping clinicians to apprehend BD patients more comprehensively and to provide timely intervention by tracking mental state progression.

Detecting depression based on non-verbal behaviors has received great attention. However, most prior work on detecting depression mainly focused on detecting depressed individuals in laboratory settings, which are difficult to be generalized in practice. In addition, little attention has been paid to analyzing the non-verbal behaviors of depressed individuals in the wild. Therefore, in this paper, we present a multimodal depression dataset, D-Vlog, which consists of 961 vlogs (i.e., around 160 hours) collected from YouTube, which can be utilized in developing depression detection models based on the non-verbal behavior of individuals in real-world scenario. We develop a multimodal deep learning model that uses acoustic and visual features extracted from collected data to detect depression. Our proposed model employs the cross-attention mechanism to effectively capture the relationship across acoustic and visual features, and generates useful multimodal representations for depression detection. The extensive experimental results demonstrate that the proposed model significantly outperforms other baseline models. We believe our dataset and the proposed model are useful for analyzing and detecting depressed individuals based on non-verbal behavior.

Interpretability is an important property for visual mod-els as it helps researchers and users understand the in-ternal mechanism of a complex model. However, gener-ating semantic explanations about the learned representation is challenging without direct supervision to produce such explanations. We propose a general framework, La-tent Visual Semantic Explainer (LaViSE), to teach any ex-isting convolutional neural network to generate text de-scriptions about its own latent representations at the filter level. Our method constructs a mapping between the vi-sual and semantic spaces using generic image datasets, using images and category names. It then transfers the map-ping to the target domain which does not have semantic la-bels. The proposedframework employs a modular structure and enables to analyze any trained network whether or not its original training data is available. We show that our method can generate novel descriptions for learned filters beyond the set of categories defined in the training dataset and perform an extensive evaluation on multiple datasets. We also demonstrate a novel application of our method for unsupervised dataset bias analysis which allows us to auto-matically discover hidden biases in datasets or compare dif-ferent subsets without using additional labels. The dataset and code are made public to facilitate further research.11https://github.com/YuYang0901/LaViSE

The transparency issue of sponsorship disclosure in advertising posts has become a significant problem in influencer marketing. Although influencers are urged to comply with the regulations governing sponsorship disclosure, a considerable number of influencers fail to disclose sponsorship properly in paid advertisements. In this paper, we propose a learning-to-rank based model, Sponsored Post Detector (SPoD), to detect undisclosed sponsorship of social media posts by learning various aspects of the posts such as text, image, and the social relationship among influencers and brands. More precisely, we exploit image objects and contextualized information to obtain the representations of the posts and also utilize Graph Convolutional Networks (GCNs) on a network which consists of influencers, brands, and posts with embed social media attributes. We further optimize the model by conducting manifold regularization based on temporal information and mentioned brands in posts. The extensive studies and experiments are conducted on sampled real-world Instagram datasets containing 1,601,074 posts, which mention 26,910 brands, published over 6 years by 38,113 influencers. Our experimental results demonstrate that SPoD significantly outperforms the existing baseline methods in discovering sponsored posts on social media.

Influencer marketing has become a key marketing method for brands in recent years. Hence, brands have been increasingly utilizing influencers’ social networks to reach niche markets, and researchers have been studying various aspects of influencer marketing. However, brands have often suffered from searching and hiring the right influencers with specific interests/topics for their marketing due to a lack of available influencer data and/or limited capacity of marketing agencies. This paper proposes a multimodal deep learning model that uses text and image information from social media posts (i) to classify influencers into specific interests/topics (e.g., fashion, beauty) and (ii) to classify their posts into certain categories. We use the attention mechanism to select the posts that are more relevant to the topics of influencers, thereby generating useful influencer representations. We conduct experiments on the dataset crawled from Instagram, which is the most popular social media for influencer marketing. The experimental results show that our proposed model significantly outperforms existing user profiling methods by achieving 98% and 96% accuracy in classifying influencers and their posts, respectively. We release our influencer dataset of 33,935 influencers labeled with specific topics based on 10,180,500 posts to facilitate future research.

Brand mentioning is a type of word-of-mouth advertising method where a brand name is disclosed by social media users in posts. Recently, brand mentioning by influencers has raised great attention because of the strong viral effects on the huge fan base of influencers. In this paper, we study the brand mentioning practice of influencers. More specifically, we analyze a brand mentioning social network built on 18,523 Instagram influencers and 804,397 brand mentioning posts. As a result, we found four inspiring findings: (i) most influencers mention only a few brands in their posts; (ii) popular influencers tend to mention only popular brands while micro-influencers do not have a preference on brand popularity; (iii) audience have highly similar reactions to sponsored and non-sponsored posts; and (iv) compared to non-sponsored posts, sponsored brand mentioning posts favor fewer usertags and more hashtags with longer captions to exclusively promote the specific products. Moreover, we propose a neural network-based model to classify the sponsorship of posts utilizing network embedding and social media features. The experimental results show that our model achieves 80% accuracy and significantly outperforms baseline methods.

A proof-of-work blockchain adopts an incentive-driven design to encourage people to participate in the network. Miners provide computing resources and services in exchange for incentives such as static block rewards and transaction fees collected from blockchain users. However, our findings suggest that the current reward scheme may not encourage miners to process user transactions. A non-cooperative strategy that submits a block with no transaction can be more rewarded than a cooperative strategy. As a consequence, the non-cooperative strategy can prevail over the cooperative strategy, decrease the system throughput, and distort credit distribution. We particularly choose Ethereum project as a subject since it is a general-purpose smart contract platform. By investigating the past two years of its ledger history, we find network propagation and block processing delays are the most significant factors that cause miners to choose the non-cooperative strategy. From this finding, we develop a more accurate statistical model for a block discovery time, as well as a reward matrix. We then derive the condition that either strategy has no additional gain, which also helps to estimate whether the transaction fee is underpriced or not. Simulation results show that the non-cooperative strategy is no longer dominant under the revised reward scheme.

Delay Tolerant Network (DTN) routing protocols have been studied by many researchers for areas that are densely populated but have no Internet access. Many researchers have studied and proposed DTN protocols that rely on social metrics such as individual encountering frequency or centrality of a node. In this paper, we propose a DTN routing protocol, Hitchhiker, that utilizes the power of the dense crowd. Hitchhiker uses a distributed method of (i) self-clustering of mobile nodes, (ii) disseminating a packet within a local wireless subnetwork to find its intended destination, and (iii) relaying the message to the border nodes to reach the destination. We use real world human trace from two social media platforms, Twitter and Instagram, to emulate people's movement in the simulation. We find that Hitchhiker exhibits a success rate of delivery comparable to other popular DTN protocols, and also achieves sufficiently low network overhead.

We conduct comprehensive measurements on the current practice of content bundling to understand the structural patterns of torrents and the participant behaviors of swarms on one of the largest BitTorrent portals: The Pirate Bay. From the datasets of the 120K torrents and 14.8M peers, we investigate what constitutes torrents and how users participate in swarms from the perspective of bundling, across different content categories: Movie, TV, Porn, Music, Application, Game and E-book. In particular, we focus on: (1) how prevalent content bundling is, (2) how and what files are bundled into torrents, (3) what motivates publishers to bundle files, and (4) how peers access the bundled files. We find that over 72% of BitTorrent torrents contain multiple files, which indicates that bundling is widely used for file sharing. We reveal that profit-driven BitTorrent publishers who promote their own web sites for financial gains like advertising tend to prefer to use the bundling. We also observe that most files (94%) in a bundle torrent are selected by users and the bundle torrents are more popular than the single (or non-bundle) ones on average. Overall, there are notable differences in the structural patterns of torrents and swarm characteristics (i) across different content categories and (ii) between single and bundle torrents.

Despite the increasing interest in content bundling in BitTorrent systems, there are still few empirical studies on the bundling practice in real BitTorrent communities. In this paper, we conduct comprehensive measurements on one of the largest BitTorrent portals: The Pirate Bay. From the torrents data set collected for 38 days from April to May, 2010, we study how prevalent bundling is and how many files are bundled in a torrent, across different types of contents shared: Movie, Porn, TV, Music, Application, E-book, and Game.

As mobile Internet environments are becoming dominant, how to revamp P2P operations for mobile hosts is gaining more and more attention. In this paper, we carry out empirical traffic measurement of BitTorrent service in various settings (static, bus and subway) in commercial WiMAX networks. To this end, we analyze the connectivity among peers, the download throughput/stability, and the signaling overhead of mobile WiMAX hosts in comparison to a wired (Ethernet) host. We find out the drawbacks of BitTorrent operations in mobile Internet are characterized by lower connection ratio, unstable connections amongst peers, and higher control message overhead.