Spectral information plays a crucial role in many domains, including remote sensing, cultural heritage analysis, food inspection, and material appearance modeling. Spectral measurements, such as hyperspectral imaging, provide a powerful means of acquiring this information but often require expensive equipment and time-consuming capture procedures.

The rate of terror attacks has surged over the past decade, resulting in the tragic and senseless loss or alteration of numerous lives. Offenders behind mass shootings, bombings, or other domestic terrorism incidents have historically exhibited warning signs on social media before carrying out actual incidents. However, due to inadequate and comprehensive police procedures, authorities and social media platforms are often unable to detect these early indicators of intent. To tackle this issue, we aim to create a multimodal model capable of predicting sentiments simultaneously from both images (i.e., social media photos) and text (i.e., social media posts), generating a unified prediction. The proposed method involves segregating the image and text components of an online post and utilizing a captioning model to generate sentences summarizing the image’s contents. Subsequently, a sentiment analyzer evaluates this caption, or description, along with the original post’s text to determine whether the post is positive (i.e., concerning) or negative (i.e., benign). This undertaking represents a significant step toward implementing the developed system in real-world scenarios.