Understanding human emotions from images is a challenging yet essential task for vision-language models. While recent efforts have fine-tuned vision-language models to enhance emotional awareness, most approaches rely on global visual representations and fail to capture the nuanced and multi-faceted nature of emotional cues. Furthermore, most existing approaches adopt instruction tuning, which requires costly dataset construction and involves training a large number of parameters, thereby limiting their scalability and efficiency. To address these challenges, we propose MASP, a novel framework for Multi-Aspect guided emotion reasoning with Soft Prompt tuning in vision-language models. MASP explicitly separates emotion-relevant visual cues via multi-aspect cross-attention modules and guides the language model using soft prompts, enabling efficient and scalable task adaptation without modifying the base model. Our method achieves state-of-the-art performance on various emotion recognition benchmarks, demonstrating that the explicit modeling of multi-aspect emotional cues with soft prompt tuning leads to more accurate and interpretable emotion reasoning in vision-language models.

The quality of online wellness communities is difficult to explain using simple growth metrics. However, an empirical understanding of the structural and relational factors that define quality has proven elusive. To fill this gap, we analyzed over 17 million posts from 56 Reddit wellness communities between 2023 and 2024 by applying the sociability-usability framework, measuring user-based factors (e.g., User Retention, Gini Coefficient) and content-based factors (e.g., Interactivity, Quasi-Quality Index), while also examining the impact of identity (Linguistic Distinctiveness and Topic Drift). Our analysis confirmed that quality depends on a stable core user base and meaningful reciprocal exchanges rather than conversation volume. A strong identity creates a trade-off, enhancing internal cohesion while acting as a barrier to new user inflows. Overall, we challenge the conventional ‘growth=success’ paradigm, establishing structural sustainability as the core mechanism for defining quality in online wellness communities and offering a new framework for their design and evaluation.

Recent advances in AI-generated image models significantly impact the digital art domain, raising concerns about the misuse of AI-generated imagery. To address the limitations of existing detection approaches, which lack pixel-level localization and robustness to diverse generation methods, we introduce CLIP-SegFusion, a multitask learning framework for synthetic image detection. Built upon the CLIP model for feature extraction, CLIP-SegFusion jointly performs image-level classification and pixel-level segmentation by integrating a segmentation module and an attention-guided feature fusion mechanism. This enables sophisticated authenticity verification and explicit localization of the manipulated regions. In our extensive experiments, CLIP-SegFusion demonstrates strong performance, achieving an average F1 score of \(92.63\%\) on the in-distribution (ID) dataset and \(83.66\%\) on the out-of-distribution (OOD) datasets, while outperforming existing state-of-the-art approaches on OOD benchmarks by a margin of \(5.80\%\). The experimental results further demonstrate that multi-scale feature modeling and the joint optimization of classification and segmentation objectives significantly enhance the robustness and generalization capability of the model in complex artistic image scenarios. Our code is available at: https://github.com/dxlabskku/CLIP-SegFusion.

Video summarization aims to selectively extract important events and contexts embedded in a video and reconstruct them into a concise video while preserving the original content. Previous studies employed attention-based approaches to capture long-term global dependencies or convolutional neural networks to learn local frame-level patterns separately. However, these approaches tend to be biased toward either global or local information, limiting their ability to achieve high summarization performance. We propose a new video summarization architecture, the Multi-Scale Convolution-Attention Residual Fusion (MARs), which integrates convolutional modules with multi-head self-attention mechanisms. The proposed model captures inter-frame variations and fine-grained local features through convolutional modules while simultaneously learning global contextual information across the entire video using attention-based modules. Furthermore, additional designs, such as temporal-difference embedding, multi-scale convolution, and positional encoding, are incorporated to enhance the summarization performance. Experimental results on two benchmark datasets demonstrate that the proposed model outperforms existing methods. Ablation studies further validate the contribution of each module, confirming the effectiveness of the proposed architecture. Codes are available at https://github.com/dxlabskku/MARs.

Human Action Recognition (HAR) in real-world scenarios is significantly challenged by unseen domain shifts, such as variations in the camera viewpoint, illumination, lighting, or background. Although recent advancements in video domain generalization are promising for HAR by introducing models that are robust to these shifts, existing methods often fall short. They typically depend on a single modality or employ static frame-level fusion approaches, which inherently limit the capture of multi-scale temporal dependencies and the alignment of the asynchronous modalities frequently present in video data. To address these limitations, we propose Multimodal Alignment and Distillation for Domain Generalization (MAD-DG), a novel framework that synchronizes asynchronous modalities through a segment-label aligned temporal binding window with a contrastive alignment mechanism. We further incorporate an online self-distillation temporal module that captures multi-scale temporal relationships and learns robust, domain-invariant representations. Extensive experiments demonstrate that MAD-DG achieves state-of-the-art performance and exhibits better generalization capabilities across both single- and multi-source domain generalization settings. Our source code is available at https://github.com/dxlabskku/MAD-DG.

Previous Click-Through Rate (CTR) prediction models rely on enumerating high-order feature combinations up to a fixed order, limiting expressiveness and scalability. Recent studies explored arbitrary-order interaction modeling through two major paradigms: log-based and graph-based methods. However, both paradigms suffer from inherent weaknesses: log-based methods lack stability, and graph-based methods lack generalizability, as both attempt to model overly diverse combinations of features, many of which may be noisy or redundant. This observation provokes a central question: What if only a small set of core interactions is sufficient? To explore this, we progressively mask feature interactions and find that removing up to 90% of them results in negligible performance degradation. This suggests that most interactions are unnecessary. Motivated by this finding, we propose SCV: Selectively Crossing Vectors, a CTR prediction framework that reformulates feature interaction learning as a sparse edge selection task over a globally shared feature-interaction graph. By modeling feature interactions over a globally learned graph and dynamically fusing expert outputs in an instance-aware manner, the SCV effectively leverages global consistency and local adaptability. We further introduce a label-biased self-distillation objective to mitigate the effects of noisy supervision and stabilize training. Experiments on public CTR benchmarks show that SCV achieves state-of-the-art performance while reducing computational cost by up to 66%, validating the effectiveness of globally sparse yet locally adaptive interaction modeling. All codes are available at: https://github.com/bw-99/scv.

Short video platforms like YouTube Shorts and TikTok face significant copyright compliance challenges, as infringers frequently embed arbitrary background music (BGM) to obscure original soundtracks (OST) and evade content originality detection. To tackle this issue, we propose a novel pipeline that integrates Music Source Separation (MSS) and cross-modal video-music retrieval (CMVMR). Our approach effectively separates arbitrary BGM from the original OST, enabling the restoration of authentic video audio tracks. To support this work, we introduce two domain-specific datasets: OASD-20K for audio separation and OSVAR-160 for pipeline evaluation. OASD-20K contains 20,000 audio clips featuring mixed BGM and OST pairs, while OSVAR160 is a unique benchmark dataset comprising 1,121 video and mixed-audio pairs, specifically designed for short video restoration tasks. Experimental results demonstrate that our pipeline not only removes arbitrary BGM with high accuracy but also restores OSTs, ensuring content integrity. This approach provides an ethical and scalable solution to copyright challenges in user-generated content on short video platforms.

The burgeoning advancement of digital technologies, especially the emergence of the metaverse, holds significant potential to transform education. Despite this promise, little research has explored user perspectives in this context. This study addresses this gap by proposing a novel research model that integrates the Diffusion of Innovation theory with the Extended Unified Theory of Acceptance and Use of Technology to evaluate users' attitudes and intentions toward adopting metaverse education platforms. Structural equation modelling was applied to a sample of 892 participants, identifying key factors influencing the adoption of these platforms. The results show that attitudes and perceived usefulness are primary determinants of adoption, with trialability and hedonic motivation exerting positive effects, while complexity and compatibility have negative effects. Interestingly, observability and social influence were found to have negligible effects on attitudes. These findings provide both practical and theoretical insights into the factors driving the adoption of metaverse platforms in education.

Recently, sentiment analysis research has made significant improvements in addressing sentiment and subjectivity within textual content. The advent of multimodal deep learning techniques has further broadened this scope, enabling the integration of diverse modalities such as voice and image features alongside text. However, despite these advancements, the analysis of the Korean language remains challenging due to its inherently agglutinative nature and linguistic ambiguity, primarily examined at the sentence level. To effectively address this challenge, we propose a novel Multimodal Sentimental Deep Learning Framework for Korean (MSDLF-K), which can examine not only Korean text but also its associated speech. Our framework, MSDLF-K, integrates spectrograms and waveforms from Korean voice data with embedding vectors derived from script sentences, creating a unified multimodal representation. This approach facilitates the identification of both shared and unique features within the latent space, thereby offering valuable insights into their respective impacts on sentiment analysis performance. To validate the efficacy of MSDLF-K, we conducted a set of experiments using the emotion speech synthesis dataset. Our findings demonstrate that MSDLF-K achieves a remarkable accuracy of 79.0% in valence and 81.7% in arousal for emotion classification, metrics previously unexplored in the literature. Furthermore, empirical analysis reveals the significant influence of multimodal representations, encompassing both text and voice, on enhancing emotion analysis performance. In summary, our study not only presents a pioneering solution for sentiment analysis in the Korean language but also underscores the importance of incorporating multimodal approaches for more comprehensive and accurate sentiment analysis across diverse linguistic contexts.

Photo restoration technology enables preserving visual memories in photographs. However, physical prints are vulnerable to various forms of deterioration, ranging from physical damage to loss of image quality, etc. While restoration by human experts can improve the quality of outcomes, it often comes at a high price in terms of cost and time for restoration. In this work, we present the AI-based photo restoration framework composed of multiple stages, where each stage is tailored to enhance and restore specific types of photo damage, accelerating and automating the photo restoration process. By integrating these techniques into a unified architecture, our framework aims to offer a one-stop solution for restoring old and deteriorated photographs. Furthermore, we present a novel old photo restoration dataset because we lack a publicly available dataset for our evaluation.

Limited training data is a long-standing problem for video emotion analysis (VEA). Existing works leverage the power of large-scale image datasets for transferring while failing to extract the temporal correlation of affective cues in the video. Inspired by psychology research and empirical theory, we verify that the degree of emotion may vary in different segments of the video, thus introducing the sen-timent complementary and emotion intrinsic among temporal segments. We propose an MAE-style method for learning robust affective representation of videos via masking, termed MART. First, we extract the affective cues of the lexicon and verify the extracted one by computing its matching score with video content, in terms of sentiment and emotion scores alongside the temporal dimension. Then, with the verified cues, we propose masked affective modeling to re-cover temporal emotion distribution. We present temporal affective complementary learning that pulls the complementary part and pushes the intrinsic one of masked multimodal features, where the constraint is set with cross-modal attention among features to mask the video and recover the degree of emotion among segments. Extensive experiments on five benchmarks show the superiority of our method in video sentiment analysis, video emotion recognition, multimodal sentiment analysis, and multimodal emotion recognition.

In online chatting, people increasingly prefer using stickers to supplement or replace text for replies, as sticker images can express vivid and varied emotions. The Sticker Response Selection (SRS) task aims to predict the sticker image that is most relevant to the history dialogue. Previous researches explore the semantic similarity between context and stickers, overlooking both unimodal and cross-modal emotional information. In this paper, we propose a 'Perceive before Respond' (PBR) training paradigm. PBR perceives sticker emotions through a knowledge distillation module. Variety representations of each emotion category are acquired from the large-scale sticker emotion recognition dataset and distilled into our model to enhance emotion comprehension. We further distinguish stickers with similar subject elements under the same topic. We perform contrastive learning at both inter- and intra-topic levels to obtain discriminative and diverse sticker representations. In addition, we improve the hard negative sampling method for image-text matching based on cross-modal sentiment association, conducting hard sample mining from both semantic similarity and sentiment polarity similarity for sticker-to-dialogue and dialogue-to-sticker. Extensive experiments verify the effectiveness of each proposed component. Ablation experiments on different backbone networks demonstrate the generality of our approach. Our code is released on https://github.com/wuyou-xia/Perceive-before-Respond.

The increasing utilization of newly emerging networks (e.g., private-5G) across industries underscores the need for accurate traffic prediction to manage network resources effectively. However, rapidly emerging networks face challenges in accurate prediction due to limited training data at the early stage and fluctuation in traffic load at the maintenance stage. In response, we propose ISOML (Inter-Service Online Meta-Learning), a novel traffic prediction pipeline designed for newly emerging networks. ISOML utilizes meta-learning to address data scarcity and employs the EWC (Elastic Weight Consolidation) for online learning to learn dynamics of traffic patterns. Experimental validation in real-world datasets demonstrates the efficacy of ISOML in predicting traffic for emerging network environments.

Abstract Social media has entrenched itself as an indispensable marketing tool. We introduce a quantitative approach to predicting the popularity of social media posts within the café and bakery sector. Employing Multi‐Pop , a multimodal popularity prediction model that harnesses both images and text from post content, it utilizes the features of posts that significantly influence their popularity on one of the most widely used platforms, Instagram. By focusing solely on post‐content features and excluding user information, we analysed 8765 Instagram posts from the cafe and bakery domain, revealing that our model attains a superior accuracy rate of 82.0% compared with existing popularity prediction methods. Furthermore, the study identifies hashtags and post captions as exerting a greater impact on post popularity than images. This research furnishes valuable insights, particularly for small business owners and individual entrepreneurs, by introducing novel computational and empirical methodologies for Instagram marketing strategy and post popularity prediction, thereby enhancing the comprehension of social media marketing dynamics.

Pipeline inspection methods are continuously being developed, given that the safe maintenance and operation of heat transport pipelines is one of the most important issues in Korea. However, owing to the limitations of traditional inspection methods (e.g., excavation work), new solutions for underground heat-transport pipelines are required. Therefore, we propose DeepAUP, which is a deep neural network framework for detecting abnormalities in underground heat-transport pipelines. With datasets collected from actual environments using two sensors, DeepAUP effectively identified abnormal underground heat-transport pipelines, with more than 99% accuracy. Based on the experimental findings, several practical implications, as well as notable limitations, are examined. Note to Practitioners—This study was motivated by the several issues related to checking and maintaining underground heat-transport pipelines. Although there are several cornerstones for addressing this issue, they involve notable limitations, including economic and environmental concerns. Most existing studies indicate that expensive additional facilities, such as hoop strain sensors, are required to explore underground pipeline abnormalities. Thus, this study suggests a unique approach using deep learning called DeepAUP, which is expected to significantly reduce the effort required from practitioners. In future studies, we plan to employ a more efficient framework and a real-time detection system in the urban areas of South Korea.

With the growing ubiquity and broad usage, emojis are widely used as a universal visual language, which complements the intentions and emotions beyond the textual data. Despite the critical role of representing emotion, existing emojis neglect the subtle and complex properties of human emotion in that only countable and finite face emojis exist in a categorical manner. In this article, we propose a novel approach to facial emoji generation, which can control the emotional degree of generated emojis for more complex and detailed usage on online conversations. In other words, we develop a new emotion-aware emoji generative adversarial network, which is capable of generating an emoji that expresses a given emotion distribution. In this way, our approach aims to map fine-grained emotions to expressive emojis. Both quantitative and qualitative evaluation demonstrate that our approach can successfully generate highquality emoji-like images by representing a wide range of emotions. To the best of our knowledge, this is the first approach to use the deep generative model from the standpoint of the emoji’s emotional role, which can further promote more interactive and effective online communication.

The Billboard chart is a clear barometer for measuring a song's success in the music industry. Therefore, a number of artists and affiliated marketers in the music industry have attempted to determine how to emerge at the top of the chart. In the current study, artist-fan interactions on social media are examined as one of the possible indicators to predict the success of songs on the Billboard Hot 100 chart. The performance of a song on the Billboard chart was predicted based on the artist-fan interaction using the artist-fan dataset composed of posts, comments, and quote tweets, their sentimental levels, and the interaction styles of each post. Overall, the XGBoost model with the quote-tweet interaction data exhibited the highest classification performance (F1-score: 80.75% on Top 1 label), showing that the interaction features extracted from quote-tweets show the strongest relevance to a song's success. We present a simplified approach for observing and understanding public perception for the entertainment industry, specifically for the music industry, through social media interactions. We also suggest the facilitation of artist-fan interactions on social media with similar functions of quote-tweet function on Twitter as a valid strategy to make songs more successful.

Visual sentiment analysis aims to predict human emotional responses to visual stimuli. It has attracted considerable attention owing to the increasing popularity of online image sharing. Most researchers have focused on improving emotion recognition using holistic and local information derived from given images. Relatively less attention has been paid to the semantic information of objects in images, which influences human emotional responses to the images. Therefore, we propose a novel object semantic attention network (OSANet) that attempts to unravel the semantic information of objects in images that contribute to emotion detection. The OSANet combines both global representation and semantic information of objects to predict the emotion elicited by a given image. First, the holistic features that represent the entire image are extracted using convolutional blocks. Subsequently, the object-level semantic information is obtained from pre-trained word embedding and then weighted according to the relative importance of the object using the attention mechanism. Notably, a new loss function to address the subjectivity of sentiment analysis is introduced, which improves the performance of the emotion detection task. Extensive experiments on three image emotion datasets demonstrated the superiority and interpretability of the OSANet. The results show that the OSANet outperforms extant image emotion detection frameworks.

The global information and communication technologies (ICT) industry considers the Internet of Things (IoT) to be a promising and innovative area for development. The IoT concept encompasses both traditional products and services as well as new services. However, there is always user resistance whenever new products or innovative services are introduced in the market, and only a few studies have considered the concept of user resistance in understanding users' perspectives toward IoT environments. Thus, this study employed the concept of user resistance behaviour to explain the continual usage intention of IoT services. The results, using structural equation modelling with an integrated research model, confirm the roles of perceived benefits, risk, and three external factors – privacy concerns, trust, and ease of use – in influencing the intention to use IoT services or products. Moreover, the relationships between intention, resistance attitude, perceived benefits, and risk were also investigated. The findings can provide a foundation for using user resistance behaviour to explain the future of IoT-based services.

A recommendation system saves the user the trouble of searching for information and analyzes their profile to recommend the most suitable content. A variety of techniques, including content-based, collaborative, and knowledge-based techniques, have been proposed for performing recommendations. Recommendation systems are employed to suggest content, such as books, music, and videos; furthermore, they are often used in e-commerce. In particular, the South Korean film industry recommends movies using a collaborative filtering method based on genres, which is commonly used in film recommendation systems. However, this method can be ineffective when users initially encounter film recommendation services or have specific preferences with respect to movies, such as preferences regarding actors or directors. This motivated us to propose an actor-based recommendation system using the content-based filtering that considers actors’ filmography information and the genres of 509 South Korean movies. The effectiveness and performance of the suggested system are evaluated in comparison with those of a traditional genre-oriented recommendation system.

Although automobile navigation systems have conventionally utilised visual and auditory modalities to deliver information, recent advancement in tactile technology has introduced the possibility of integrating tactile actuators into navigation systems. To empirically test the effect of tactile navigational cues, we conducted simulated driving experiments in which participants (N = 96) were exposed to four sets of information modalities (visual, visual + auditory, visual + tactile, and visual + auditory + tactile). The results indicate that multisensory systems including tactile cues allow participants to respond faster to unexpected road events and impart greater satisfaction with the overall driving experience. The implications and limitations of these findings are discussed.

With rapid improvements in communication technologies and infrastructure, the Internet of Things (IoT) has become a promising sector within the global information and communication technology industry. Various fields are employing the concept of IoT for their traditional products and services for convenient use by consumers. In the construction industry, the majority of leading companies and organizations are using IoT technologies in a smart home environment. However, only few studies have focused on the user experience of IoT technologies in such an environment. Thus, the current study explores the key determinants of user acceptance of IoT technologies in a smart home environment, and investigates a research model integrated with five potential user factors and a technology acceptance model. The results of the collected data, which were investigated using a structural equation modeling method, show that three positive motivations, compatibility, connectedness, and control, and a negative hindrance, cost, are significant determinants of the technology acceptance behavior of users. The current study can serve as a foundation for future studies on improving IoT technologies in a smart home environment by considering the user experience.

This study proposes an integrated research model for investigating driver adoption of car navigation systems. We consider the potential causal connections between core cognitive and psychological factors and driver intention to use these systems. We extracted possible factors that may significantly affect the perceived usability of car navigation systems from in-depth interviews with two groups of individuals: an expert group and a driver group. Data collected from N = 1045 drivers via an online survey were analysed by structural equation modelling. The results showed that the service & display quality components of the systems were the most significant determinants of driver attitude and intention to use car navigation systems. Two other factors, namely attitude and perceived usefulness, also had impacts on driver intention. Moreover, both satisfaction and service & display quality were affected by perceived system reliability, while usefulness was affected by both perceived locational accuracy and satisfaction. Satisfaction also significantly affected perceived ease of use. In addition, we introduced new external variables to the technology acceptance model (TAM) and validated the causal connections proposed by the original TAM. The present study provides valuable insights into the core factors that significantly affect driver perspectives of and intention to use car navigation systems.

Remote pointing devices like the Wii remote have a wide range of applications and are becoming more important for the manipulation of and interactions with information on a distant display such as smart TVs. Because remote pointing devices are used without external support, however, muscular tremors and motional disparity between the display and motor space can result in usability problems of mouse jitters and instability. In this research, a solution is proposed to those problems using feedforward technology, where a user is provided with predictive information in multisensory modalities while approaching a target. Also, gender effect on the user experience of remote pointing devices is examined. By conducting two experiments and a survey, it was found that the feedforward signal plays a role of precue and is more effective than the typical feedback. It was also found that the modality variations in feedforward were impacted by the gender of the user. The findings can be used to improve user interfaces for remote pointing controllers.

Do individual differences in dispositional behavioral tendencies, such as immersive tendency and need to belong, play a significant role in human-robot interaction? To answer this question, the present study conducted a 2 x 2 between-subjects experiment to examine the effects of immersive tendency (high vs. low) and need to belong (high vs. low) on individuals' perceptions of a social robot. Preliminary data analyses revealed that participants with a higher level of immersive tendency and need to belong showed greater attachment and trust towards the robot, and were more satisfied with their relationship with the robot than participants with a lower level of immersive tendency and need to belong. In addition, participants with a higher level of immersive tendency experienced greater feelings of social presence. Implications of notable findings are discussed.

A 2 x 2 between-subjects experiment was conducted to examine the effects of the type of artificial agent (robot vs. computer) and the role of the agent (ally vs. enemy) on people's perceptions and evaluations of the agent when playing a video game. Participants perceived that playing the game with a robot was more enjoyable and easier than playing with a computer. Regardless of the agent type, participants reported that playing the game was more enjoyable when the agent played as an ally rather than as their opponent. Implications of notable findings are discussed.

This study explores the effect of the robot's appearance vs. behavior (voice and gestures) on the way it is perceived as a machine-like instead of a human-like robot. A between-subjects experiment with four conditions was conducted. Results suggest that both the robot's behavior and appearance are important but, if they are contradictory, the robot's behavior is more powerful than the robot's appearance in the perception of the robot as more machine-like or human-like.