Variants of Persian (Farsi, Dari, and Tajiki) are spoken by more than 110 million people worldwide. Despite the growing interest in broadening the coverage of Natural Language Processing (NLP) methods, Persian has received limited attention. This survey addresses this gap by reviewing over 200 peer-reviewed studies published across the last two decades, documenting more than 40 publicly available corpora, and analysing linguistic challenges, pipelines, and applications across approaches ranging from rule-based methods to Large Language Models (LLMs). Given the rapid advancement of LLMs, we study the state of Persian-specific models across different tasks, review the available LLMs for Persian, assess their performance and limitations, and provide a comprehensive discussion of their strengths and gaps. Our analysis shows that although attention to Persian NLP has increased and notable progress has been made, critical gaps remain—particularly the scarcity of standardised benchmarks and gold corpora in tasks such as toxicity and safety, bias and fairness, and explainability and reasoning. Future progress requires expanding culturally grounded datasets, establishing robust evaluation frameworks, and fostering open-source collaboration and shared-task participation to accelerate the development of Persian NLP.

Social media platforms like Twitter provide rich data that can offer insights into various aspects of users' behavior. In this study, we explore the potential of emoji usage as a means for demographic prediction. Leveraging a Twitter dataset of 18,689 users, annotated with gender and ethnicity labels, we analyze the proportion of tweets containing emoji across different demographic groups. We identify significant variations in emoji usage, with women utilizing emoji more frequently than men and users of African descent displaying a higher tendency for emoji usage compared to users of European descent. Moreover, we investigate the most distinctive emoji for each group, revealing intriguing patterns that are closely tied to the cultural and demographic backgrounds of users. Building upon these findings, we employ machine learning models with different feature extraction techniques to predict users' gender and ethnicity. Our results demonstrate the predictive power of emoji, outperforming traditional text-based features. Furthermore, our study provides evidence that emoji usage can be a valuable resource for inferring user demographic characteristics on social media platforms, contributing to our understanding of user behavior in digital environments.

On behalf of all the members of the organizing Committee, we are pleased to welcome all of you to IEEE/ACM ASONAM 2022.

In this paper we present the second version of our multilingual tweet classification tool. ClassStrength v2 classifies tweets into 14 categories (Sports, Music, News&Politics etc.) using a distant supervision approach. The new version extends the initial set of five languages to ten (English, French, German, Chinese, Japanese, Arabic, Russian, Spanish, Portuguese and Polish). In addition, the classification models of each language get automatically updated every month to allow accurate classification over time. Our experimentation showed that the larger the time gap between the tweet and the data used for training the model, the worse the performance, which motivated for creating an adaptive version of ClassStrength that get its models updated periodically.

We analyze fifteen Twitter user geolocation models and two baselines comparing how they are evaluated. Our results demonstrate that the choice of effectiveness metric can have a substantial impact on the conclusions drawn from an experiment. We show that for general evaluations, a range of metrics should be reported to ensure that a complete picture of system effectiveness is conveyed.

Predicting the stance of social media users on a topic can be challenging, particularly for users who never express explicit stances. Earlier work has shown that using users' historical or non-relevant tweets can be used to predict stance. We build on prior work by making use of users' interaction elements, such as retweeted accounts and mentioned hashtags, to compute the similarities between users and to classify new users in a user similarity feature space. We show that this approach significantly improves stance prediction on two datasets that differ in terms of language, topic, and cultural background.

In this paper we present our multilingual tweet classification tool. ClassStrength provides a set of classification models in different languages that classify tweets into 14 general-purpose categories, including: sports, politics, entertainment, comedy, etc. Our classifier uses a distant-supervision approach for creating training data in any available language on Twitter. The classifier uses a soft-classification scheme, where it generates a likelihood score for a tweet to match each of the 14 categories. The initial version of our tool covers five languages, namely: English, Arabic, French, German, and Russian. More languages are to be covered in next releases. The classification model created for each language is generated from hundreds of thousands of training tweets. Our evaluation to the classifier shows superior accuracy compared to standard manual methods. Our reported accuracy is 84% based on crowd preferences over a balanced test set of English tweets covering all 14 classes.

Many adult content websites incorporate social networking features. Although these are popular, they raise significant challenges, including the potential for users to "catfish", i.e., to create fake profiles to deceive other users. This paper takes an initial step towards automated catfish detection. We explore the characteristics of the different age and gender groups, identifying a number of distinctions. Through this, we train models based on user profiles and comments, via the ground truth of specially verified profiles. When applying our models for age and gender estimation to unverified profiles, 38% of profiles are classified as lying about their age, and 25% are predicted to be lying about their gender. The results suggest that women have a greater propensity to catfish than men. Our preliminary work has notable implications on operators of such online social networks, as well as users who may worry about interacting with catfishes.

TweetMogaz is a news portal platform that generates news reports from social media content. It uses an adaptive information filtering technique for tracking tweets relevant to news topics, such as politics and sports in some regions. Relevant tweets for each topic are used to generate a comprehensive report about public reaction toward events happening. Showing a news report about an entire topic may be suboptimal for some users, since users prefer story-oriented presentation. In this demonstration, we present a technique for identifying stories within a stream of microblogs on a given topic. Detected tweets on a news story are used to generate a dynamic pseudo-article that gets its content updated in real-time based on trends on Twitter. Pseudo-article consists of a title, front-page image, set of tweets on the story, and links to external news articles. The platform is running live and tracks news on hot topics including Egyptian politics, Syrian conflict, and international sports.

Over the past years, Twitter has earned a growing reputation as a hub for communication, and events advertisement and tracking. However, several recent research studies have shown that Twitter users (and microblogging platforms' users in general) are increasingly posting microblogs containing questions seeking answers from their readers. To help those users answer or route their questions, the problem of question identification in tweets has been studied over English tweets; up to our knowledge, no study has attempted it over Arabic (not to mention dialectal Arabic) tweets.

Twitter is currently one of the largest social hubs for users to spread and discuss news. For most of the top news stories happening, there are corresponding discussions on social media. In this demonstration TweetMogaz is presented, which is a platform for microblog search and filtering. It creates a real-time comprehensive report about what people discuss and share around news happening in certain regions. TweetMogaz reports the most popular tweets, jokes, videos, images, and news articles that people share about top news stories. Moreover, it allows users to search for specific topics. A scalable automatic technique for microblog filtering is used to obtain relevant tweets to a certain news category in a region. TweetMogaz.com demonstrates the effectiveness of our filtering technique for reporting public response toward news in different Arabic regions including Egypt and Syria in real-time.

The use of social media has profoundly affected social and political dynamics in the Arab world. In this paper, we explore the Arabic microblogs retrieval. We illustrate some of the challenges associated with Arabic microblog retrieval, which mainly stem from the use of different Arabic dialects that vary in lexical selection, morphology, and phonetics and lack orthographic and spelling conventions. We present some of the required processing for effective retrieval such as improved letter normalization, elongated word handling, stopword removal, and stemming

Searching social content in general and microblogs (aka tweets) in particular has been basic and limited, especially for time-sensitive topics. The currently implemented microblog search on sites such as Twitter is based on simple word matching and retrieves the most recent microblogs that match a given query. Furthermore, a user may obtain hundreds or perhaps thousands of microblogs in response to a given query, leading to information overload. We present a new multidimensional microblog search tool that generates a comprehensive report from microblogs instead of a flat list of recent/relevant microblogs for a given query. Reports may include tag-clouds, topic time series, and most popular and funny microblogs, etc. The tool can be configured for monitoring time-sensitive topics using a set of predefined queries. We demonstrate our system on Arabic and English microblog collections. Additionally, we show a special configuration of the system for monitoring the 2012 Egyptian presidential elections.

Queries in patent prior art search are full patent applications and much longer than standard ad hoc search and web search topics. Standard information retrieval (IR) techniques are not entirely effective for patent prior art search because of ambiguous terms in these massive queries. Reducing patent queries by extracting key terms has been shown to be ineffective mainly because it is not clear what the focus of the query is. An optimal query reduction algorithm must thus seek to retain the useful terms for retrieval favouring recall of relevant patents, but remove terms which impair IR effectiveness. We propose a new query reduction technique decomposing a patent application into constituent text segments and computing the Language Modeling (LM) similarities by calculating the probability of generating each segment from the top ranked documents. We reduce a patent query by removing the least similar segments from the query, hypothesising that removal of these segments can increase the precision of retrieval, while still retaining the useful context to achieve high recall. Experiments on the patent prior art search collection CLEF-IP 2010 show that the proposed method outperforms standard pseudo-relevance feedback (PRF) and a naive method of query reduction based on removal of unit frequency terms (UFTs).

Topics in prior-art patent search are typically full patent applications and relevant items are patents often taken from sources in different languages. Cross language patent retrieval (CLPR) technologies support searching for relevant patents across multiple languages. As such, CLPR requires a translation process between topic and document languages. The most popular method for crossing the language barrier in cross language information retrieval (CLIR) in general is machine translation (MT). High quality MT systems are becoming widely available for many language pairs and generally have higher effectiveness for CLIR than dictionary based methods. However for patent search, using MT for translation of the very long search queries requires significant time and computational resources. We present a novel MT approach specifically designed for CLIR in general and CLPR in particular. In this method information retrieval (IR) text pre-processing in the form of stop word removal and stemming are applied to the MT training corpus prior to the training phase of the MT system. Applying this step leads to a significant decrease in the MT computational and resource requirements in both the training and translation phases. Experiments on the CLEF-IP 2010 CLPR task show the new technique to be 5 to 23 times faster than standard MT for query translation, while maintaining statistically indistinguishable IR effectiveness. Furthermore the new method is significantly better than standard MT when only limited translation training resources are available.

Decompounding has been found to improve information retrieval (IR) effectiveness in general domains for languages such as German or Dutch. We investigate if cross-language patent retrieval can profit from decompounding. This poses two challenges: i) There may be few resources such as parallel corpora available for training an machine translation system for a compounding language. ii) Patents have a specific writing style and vocabulary ("patentese"), which may affect the performance of decompounding and translation methods. Experiments on data from the CLEF-IP 2010 task show that decompounding patents for translation can overcome out-of-vocabulary problems (OOV) and that decompounding improves IR performance significantly for small training corpora.

Information retrieval (IR) evaluation scores are generally designed to measure the effectiveness with which relevant documents are identified and retrieved. Many scores have been proposed for this purpose over the years. These have primarily focused on aspects of precision and recall, and while these are often discussed with equal importance, in practice most attention has been given to precision focused metrics. Even for recall-oriented IR tasks of growing importance, such as patent retrieval, these precision based scores remain the primary evaluation measures. Our study examines different evaluation measures for a recall-oriented patent retrieval task and demonstrates the limitations of the current scores in comparing different IR systems for this task. We introduce PRES, a novel evaluation metric for this type of application taking account of recall and the user's search effort. The behaviour of PRES is demonstrated on 48 runs from the CLEF-IP 2009 patent retrieval track. A full analysis of the performance of PRES shows its suitability for measuring the retrieval effectiveness of systems from a recall focused perspective taking into account the user's expected search effort.

Due to the existence of large numbers of legacy documents (such as old books and newspapers), improving retrieval effectiveness for OCR'ed documents continues to be an important problem. This article compares the effect of OCR error correction with and without language modeling and the effect of query garbling with weighted structured queries on the retrieval of OCR degraded Arabic documents. The results suggest that moderate error correction does not yield statistically significant improvement in retrieval effectiveness when indexing and searching using n-grams. Also, reversing error correction models to perform query garbling in conjunction with weighted structured queries yields improved retrieval effectiveness. Lastly, using very good error correction that utilizes language modeling yields the best improvement in retrieval effectiveness.