Given the vast amount of unstructured financial text data available today, there is a high demand for reliable, quality annotations to facilitate robust model development. However, traditional methods can often be expensive and time-inefficient. In this study, we investigate annotations for emotion, sentiment, and cognitive dissonance generated by the large language models (LLMs), GPT-3.5 and GPT-4, for quarterly earnings conference calls and compare them against human annotations obtained via traditional methods. We also investigate different prompt engineering choices on LLM annotation quality, experimenting with 4 styles of prompts centered around varying the amount of contextual information given and how it is presented to the models. Our results show the GPT models are not only more consistent and reliable than human annotators, but also provide annotations in a more cost- and time-efficient manner.

A significant amount of resources have been used in both academia and industry to study the impact of financial text on company perception and performance. In order to mitigate potential adverse outcomes, companies have begun to regulate word usage based on perceived sentiment, making conventional text-based analysis less reliable. To address this, we present a multimodal bidirectional Long Short-Term Memory (LSTM) framework augmented with a cross-attention fusion mechanism trained on audio and text data obtained from quarterly earnings conferences calls. The framework is applied to two tasks: financial restatement prediction and market movement prediction. We compare the proposed model against several baseline methods and find that while it does not achieve superior performance, our results show that utilizing multimodal data leads to a substantial increase in model accuracy for restatement prediction. Furthermore, we gain insight on the effectiveness of semantic-and emotion-related features towards these tasks.