The opaqueness of modern digital advertising, exemplified by large platforms such as Meta Ads , raises concerns regarding their control over audience targeting, pricing structures, and ad relevancy assessments. Locked in place by network effects, these natural monopolies attract countless advertisers who rely on subjective intuition, with billions of dollars lost on ineffective social media advertisements. The platforms’ algorithms rely on huge amounts of data unavailable to advertisers, and the algorithms themselves are opaque too, so advertisers often cannot make informed decisions. To promote transparency and help individual advertisers, we first propose novel ways to optimize advertising strategies, predicting click-through rates of novel advertising content based on the content itself. However, advertisers face both opaqueness and a vast abundance of data: a large platform has so many competitor ads that it is hard to derive meaningful insights. Drawing inspiration from the success of Large Language Models (LLM), we propose a system that merges multimodal LLMs and pretrained AI models with an emphasis on digital marketing and advertising data analysis. Leveraging the capabilities of LLMs and incorporating explainability features, including modern text-image models, we aim to improve efficiency and produce synergy between human marketers and AI systems.

Digital Advertising is historically one of the most developed areas where Machine Learning and AI have been applied since its origination. From smart bidding to creative content generation and DCO, AI is well-demanded in the modern digital marketing industry and partially serves as a backbone of most of the state-of-the-art computational advertising systems, making them impossible for the AI tech and the programmatic systems to exist apart from one another. At the same time, given the drastic growth of the available AI technology nowadays, the issue of responsible AI utilization as well as the balance between the opportunity of deploying AI systems and the possible borderline etic and privacy-related consequences are still yet to be discussed comprehensively in both business and research communities. Particularly, an important issue of automatic User Profiling use in modern Programmatic systems like Meta Ads as well as the need for responsible application of the creative assessment models to fit into the business etic guidelines is yet to be described well. Therefore, in this talk, we are going to discuss the technology behind modern programmatic bidding and content scoring systems and the responsible application of AI by SoMin.ai to manage the Advertising targeting and Creative Validation process.

In this technical demonstration, we present SoCraft, a framework to build an advertiser-level multimedia ad content scoring platform for Meta Ads. The system utilizes a multimodal deep neural architecture to score and evaluate advertised content on Meta using both high- and low-level features of its contextual data such as text, image, targeting, and ad settings. In this demo, we present two deep models, SoDeep and SoWide, and validate the effectiveness of SoCraft with a successful real-world case study in Singapore.

In this technical demonstration, we present SMILEY, a voice-guided virtual assistant. The system utilizes a deep neural architecture ContraCLIP to manipulate facial attributes using voice instructions, allowing for deeper speaker engagement and smoother customer experience when being used in the "virtual concierge" scenario. We validate the effectiveness of SMILEY and ContraCLIP via a successful real-world case study in Singapore and a large-scale quantitative evaluation.

In this technical demonstration, we showcase the World's first personality-driven marketing content generation platform, called SoMin.ai. The platform combines deep multi-view personality profiling framework and style generative adversarial networks facilitating the automatic creation of content that appeals to different human personality types. The platform can be used for enhancement of the social networking user experience as well as for content marketing routines. Guided by the MBTI personality type, automatically derived from a user social network content, SoMin.ai generates new social media content based on the preferences of other users with a similar personality type aiming at enhancing the user experience on social networking venues as well diversifying the efforts of marketers when crafting new content for digital marketing campaigns. The real-time user feedback to the platform via the platform's GUI fine-tunes the content generation model and the evaluation results demonstrate the promising performance of the proposed multi-view personality profiling framework when being applied in the content generation scenario. By leveraging content generation at a large scale, marketers will be able to execute more effective digital marketing campaigns at a lower cost.

Venue category recommendation is an essential application for the tourism and advertisement industries, wherein it may suggest attractive localities within close proximity to users' current location. Considering that many adults use more than three social networks simultaneously, it is reasonable to leverage on this rapidly growing multi-source social media data to boost venue recommendation performance. Another approach to achieve higher recommendation results is to utilize group knowledge, which is able to diversify recommendation output. Taking into account these two aspects, we introduce a novel cross-network collaborative recommendation framework C3R, which utilizes both individual and group knowledge, while being trained on data from multiple social media sources. Group knowledge is derived based on new cross-source user community detection approach, which utilizes both inter-source relationship and the ability of sources to complement each other. To fully utilize multi-source multi-view data, we process user-generated content by employing state-of-the-art text, image, and location processing techniques. Our experimental results demonstrate the superiority of our multi-source framework over state-of-the-art baselines and different data source combinations. In addition, we suggest a new approach for automatic construction of inter-network relationship graph based on the data, which eliminates the necessity of having pre-defined domain knowledge.

Learning user attributes from mobile social media is a fundamental basis for many applications, such as personalized and targeting services. A large and growing body of literature has investigated the user attributes learning problem. However, far too little attention has been paid to jointly consider the dual heterogeneities of user attributes learning by harvesting multiple social media sources. In particular, user attributes are complementarily and comprehensively characterized by multiple social media sources, including footprints from Foursqare, daily updates from Twitter, professional careers from Linkedin, and photo posts from Instagram. On the other hand, attributes are inter-correlated in a complex way rather than independent to each other, and highly related attributes may share similar feature sets. Towards this end, we proposed a unified model to jointly regularize the source consistency and graph-constrained relatedness among tasks. As a byproduct, it is able to learn the attribute-specific and attribute-sharing features via graph-guided fused lasso penalty. Besides, we have theoretically demonstrated its optimization. Extensive evaluations on a real-world dataset thoroughly demonstrated the effectiveness of our proposed model.

Wellness is a widely popular concept that is commonly applied to fitness and self-help products or services. Inference of personal wellness--related attributes, such as body mass index (BMI) category or disease tendency, as well as understanding of global dependencies between wellness attributes and users’ behavior, is of crucial importance to various applications in personal and public wellness domains. At the same time, the emergence of social media platforms and wearable sensors makes it feasible to perform wellness profiling for users from multiple perspectives. However, research efforts on wellness profiling and integration of social media and sensor data are relatively sparse. This study represents one of the first attempts in this direction. Specifically, we infer personal wellness attributes by utilizing our proposed multisource multitask wellness profile learning framework—WellMTL—which can handle data incompleteness and perform wellness attributes inference from sensor and social media data simultaneously. To gain insights into the data at a global level, we also examine correlations between first-order data representations and personal wellness attributes. Our experimental results show that the integration of sensor data and multiple social media sources can substantially boost the performance of individual wellness profiling.

User profile learning, such as mobility and demographic profile learning, is of great importance to various applications. Meanwhile, the rapid growth of multiple social platforms makes it possible to perform a comprehensive user profile learning from different views. However, the research efforts on user profile learning from multiple data sources are still relatively sparse, and there is no large-scale dataset released towards user profile learning. In our study, we contribute such benchmark and perform an initial study on user mobility and demographic profile learning. First, we constructed and released a large-scale multi-source multi-modal dataset from three geographical areas. We then applied our proposed ensemble model on this dataset to learn user profile. Based on our experimental results, we observed that multiple data sources mutually complement each other and their appropriate fusion boosts the user profiling performance.