Chatbots have shown promise as tools to scale qualitative data collection. Recent advances in Large Language Models (LLMs) could accelerate this process by allowing researchers to easily deploy sophisticated interviewing chatbots. We test this assumption by conducting a large-scale user study (n=399) evaluating 3 different chatbots, two of which are LLM-based and a baseline which employs hard-coded questions. We evaluate the results with respect to participant engagement and experience, established metrics of chatbot quality grounded in theories of effective communication, and a novel scale evaluating ''richness'' or the extent to which responses capture the complexity and specificity of the social context under study. We find that, while the chatbots were able to elicit high-quality responses based on established evaluation metrics, the responses rarely capture participants' specific motives or personalized examples, and thus perform poorly with respect to richness. We further find low inter-rater reliability between LLMs and humans in the assessment of both quality and richness metrics. Our study offers a cautionary tale for scaling and evaluating qualitative research with LLMs.

Peer-to-Peer (P2P) cryptocurrency exchanges are two-sided marketplaces, similar to eBay, where individuals can offer to sell cryptocurrencies in exchange for payment. Due to disintermediation, these marketplaces trade off increased privacy for higher risk (e.g., scams/fraud). Although these marketplaces use feedback systems to encourage healthier transactions, anecdotal evidence suggests that feedback often fails to capture vendor-associated risks. This work documents the online safety of cryptocurrency P2P marketplaces, identifies underlying issues in feedback-based reputation systems, and proposes improved mechanisms for predicting/monitoring risky accounts. We collect data from two cryptocurrency marketplaces, Paxful and LocalCoinSwap (LCS) for 12 months (06/2022--06/2023). The data includes over 396,000 listings, 67,000 vendors, and 4.7 million feedback for Paxful; and about 52,000 listings, 14,000 users, and 146,000 feedback for LCS.First, we show that the current feedback system does not sufficiently convey enough information about risky vendors, and is susceptible to reputation manipulation through user collusion and automation. Second, combining various publicly available information, we build machine learning models to predict account suspension, and achieve a 0.86 F1-score and 0.93 AUC for Paxful. Third, while our models appear to have limited transferability across markets, we identify which features most help account suspension across platforms. Finally, we perform a month-long online evaluation to show that our models are significantly more successful than mere feedback-based reputation schemes at predicting which users will be suspended in the future.

The web and social media platforms have drastically changed how investors produce and consume financial advice. Historically, individual investors were often relying on newsletters and related prospectus backed by the reputation and track record of their issuers. Nowadays, financial advice is frequently offered online, by anonymous or pseudonymous parties with little at stake. As such, a natural question is to investigate whether these modern financial “influencers” operate in good faith, or whether they might be misleading their followers intentionally. To start answering this question, we obtained data from a very large cryptocurrency derivatives exchange, from which we derived individual trading positions. Some of the investors on that platform elect to link to their Twitter profiles. We were thus able to compare the positions publicly espoused on Twitter with those actually taken in the market. We discovered that 1) staunchly “bullish” investors on Twitter often took much more moderate, if not outright opposite, positions in their own trades when the market was down, 2) their followers tended to align their positions with bullish Twitter outlooks, and 3) moderate voices on Twitter (and their own followers) were on the other hand far more consistent with their actual investment strategies. In other words, while social media advice may attempt to foster a sense of camaraderie among people of like-minded beliefs, the reality is that this is merely an illusion, which may result in financial losses for people blindly following advice.

The expanding accessibility and appeal of investing have attracted millions of new retail investors. As such, investment discussion boards became the de facto communities where traders create, disseminate, and discuss investing ideas. These communities, which can provide useful information to support investors, have anecdotally also attracted a wide range of misbehavior – toxicity, spam/fraud, and reputation manipulation. This paper is the first comprehensive analysis of online misbehavior in the context of investment communities. We study TradingView, the largest online communication platform for financial trading. We collect 2.76M user profiles with their corresponding social graphs, 4.2M historical article posts, and 5.3M comments, including information on nearly 4 000 suspended accounts and 17 000 removed comments. Price fluctuations seem to drive abuse across the platform and certain types of assets, such as “meme” stocks, attract disproportionate misbehavior. Suspended user accounts tend to form more closely-knit communities than those formed by non-suspended accounts; and paying accounts are less likely to be suspended than free accounts even when posting similar levels of content violating platform policies. We conclude by offering guidelines on how to adapt content moderation efforts to fit the particularities of online investment communities.

Asset discovery is fundamental to any organization's cybersecurity efforts. Indeed, one must accurately know which assets belong to an IT infrastructure before the infrastructure can be secured. While practitioners typically rely on a relatively small set of well-known techniques, the academic literature on the subject is voluminous. In particular, the Internet measurement research community has devised a number of asset discovery techniques to support many measurement studies over the past five years. In this paper, we systematize asset discovery techniques by constructing a framework that comprehensively captures how network identifiers and services are found. We extract asset discovery techniques from recent academic literature in security and networking and place them into the systematized framework. We then demonstrate how to apply the framework to several case studies of asset discovery workflows, which could aid research reproducibility. These case studies further suggest opportunities for researchers and practitioners to uncover and identify more assets than might be possible with traditional techniques.