Traditional video reasoning segmentation methods rely on supervised fine-tuning, which limits generalization to out-of-distribution scenarios and lacks explicit reasoning. To address this, we propose VideoSeg-R1, the first framework to introduce reinforcement learning into video reasoning segmentation. It adopts a decoupled architecture that formulates the task as joint referring image segmentation and video mask propagation. It comprises three stages: (1) A hierarchical text-guided frame sampler to emulate human attention; (2) A reasoning model that produces spatial cues along with explicit reasoning chains; and (3) A segmentation-propagation stage using SAM2 and XMem. A task difficulty-aware mechanism adaptively controls reasoning length for better efficiency and accuracy. Extensive evaluations on multiple benchmarks demonstrate that VideoSeg-R1 achieves state-of-the-art performance in complex video reasoning and segmentation tasks.

Recognizing speaker intent in long audio dialogues among speakers has a wide range of applications, but is a non-trivial AI task due to complex inter-dependencies in speaker utterances and scarce annotated data. To address these challenges, an end-to-end framework, namely DialogGraph-LLM, is proposed in the current work. DialogGraph-LLM combines a novel Multi-Relational Dialogue Attention Network (MR-DAN) architecture with multimodal foundation models (e.g., Qwen2.5-Omni-7B) for direct acoustic-to-intent inference. An adaptive semi-supervised learning strategy is designed using LLM with a confidence-aware pseudo-label generation mechanism based on dual-threshold filtering using both global and class confidences, and an entropy-based sample selection process that prioritizes high-information unlabeled instances. Extensive evaluations on the proprietary MarketCalls corpus and the publicly available MIntRec 2.0 benchmark demonstrate DialogGraph-LLM’s superiority over strong audio and text-driven baselines. The framework demonstrates strong performance and efficiency in intent recognition in real world scenario audio dialogues, proving its practical value for audio-rich domains with limited supervision. Our code is available at https://github.com/david188888/DialogGraph-LLM

Correctly identifying an individual's social context from passively worn sensors holds promise for delivering just-in-time adaptive interventions (JITAIs) to treat social anxiety. In this study, we present results using passively collected data from a within-subjects experiment that assessed physiological responses across different social contexts (i.e., alone vs. with others), social phases (i.e., pre- and post-interaction vs. during an interaction), social interaction sizes (i.e., dyadic vs. group interactions), and levels of social threat (i.e., implicit vs. explicit social evaluation). Participants in the study ($N=46$) reported moderate to severe social anxiety symptoms as assessed by the Social Interaction Anxiety Scale ($\geq$34 out of 80). Univariate paired difference tests, multivariate random forest models, and cluster analyses were used to explore physiological response patterns across different social and non-social contexts. Our results suggest that social context is more reliably distinguishable than social phase, group size, or level of social threat, and that there is considerable variability in physiological response patterns even among distinguishable contexts. Implications for real-world context detection and future deployment of JITAIs are discussed.

Deciphering sentiments or emotions in face-to-face human interactions is an inherent capability of human intelligence, and thus a natural goal of artificial intelligence. The proliferation of multimedia data in video sites gives rise to multimodal sentiment analysis in various applications and research fields such as movie and product review, opinion polling, and affective computing. In order to improve the performance of multimodal sentiment analysis task, this paper proposes a novel neural network with multiple stacked attention mechanism (MSAM) on multimodal data containing texts, video, and audio at an utterance level. We conduct experiments using two benchmark datasets, namely CMU Multi-modal Opinion-level Sentiment Intensity (CMU-MOSI) corpus, and CMU Multimodal Opinion Sentiment and Emotion Intensity (CMU-MOSEI) corpus. Compared with a comprehensive set of state-of-the-art baselines, the evaluation results demonstrate the effectiveness of our proposed MSAM network.

The Internet of Things (IoT) boom has revolutionized almost every corner of people’s daily lives: healthcare, environment, transportation, manufacturing, supply chain, and so on. With the recent development of sensor and communication technology, IoT artifacts, including smart wearables, cameras, smartwatches, and autonomous systems can accurately measure and perceive their surrounding environment. Continuous sensing generates massive amounts of data and presents challenges for machine learning. Deep learning models (e.g., convolution neural networks and recurrent neural networks) have been extensively employed in solving IoT tasks by learning patterns from multi-modal sensory data. Graph neural networks (GNNs), an emerging and fast-growing family of neural network models, can capture complex interactions within sensor topology and have been demonstrated to achieve state-of-the-art results in numerous IoT learning tasks. In this survey, we present a comprehensive review of recent advances in the application of GNNs to the IoT field, including a deep dive analysis of GNN design in various IoT sensing environments, an overarching list of public data and source codes from the collected publications, and future research directions. To keep track of newly published works, we collect representative papers and their open-source implementations and create a Github repository at GNN4IoT.

During pandemics, effective interventions require monitoring the problem at different scales and understanding the various tradeoffs between efficacy, privacy, and economic burden. To address these challenges, we propose a framework where we perform Bayesian change-point analysis on aggregate behavior markers extracted from mobile sensing data collected during the COVID-19 pandemic. Results generated by 598 participants for up to four months reveal rich insights: We observe an increase in smartphone usage around February 10th, followed by an increase in email usage around February 27th and, finally, a large reduction in participant’s mobility around March 13th. These behavior changes overlapped with important news events and government directives such as the naming of COVID-19, a spike in the number of reported cases in Europe, and the declaration of national emergency by President Trump. We also show that our detected change points align with changes in large scale external sources, including number of COVID-19 tweets, COVID-19 search traffic, and a large-scale foot traffic data collected by SafeGraph, providing further validation of our method. Our results show promise towards the feasibility of using mobile sensing to understand communities’ responses to public health interventions.

Mental health disorders, such as generalized anxiety disorder and depression, are prevalent in modern society. Early detection of mental illness is essential to minimize the negative consequences of long-term psychological discomfort and behavioral dysfunction. As a diverse set of embedded sensors in smart mobile devices becomes commonplace, passively and continuously collected mobile sensing data are increasingly being used to develop machine learning based tools for early-stage disease diagnosis. In the training process of machine learning models, self-reported results from ecological momentary assessments (EMAs) are usually employed to provide supervisions. However, complete responses of these high-frequency surveys in the wild are impractical due to heavy user burden and low user engagement. Without the availability of EMA responses in low level of label granularity, the annotations in the high level can only provide weak supervisions. To leverage the vast majority of unannotated data in different levels of granularity, in this paper, we propose an end-to-end graph neural network algorithm called semi-supervised Graph Instance Transformer (SS-GIT) based on multiple instance learning and contrastive self-supervised learning to predict early signs of generalized anxiety disorder and depression under the weak supervisions. Using a mobile sensing dataset that we collected from around 1,300 participants in the wild, our empirical results demonstrate improved performance when compared to the existing state-of-the-art baseline graph neural networks for mental health inference. On average, our proposed model outperforms the best baseline model by 8.8% on Fl-score, 6.7% on ROC-AUC, and 7.2% on PR-AUC.

In order to research the multi-year spatial-temporal distribution of Harmful Algal Bloom (HAB) in the South Yellow Sea of China, NDVI method was used to extract HAB from MODIS images after Cloud removal by a comprehensive threshold method explored in this paper. Then, the growth law and spatial-temporal distribution of HAB are analyzed by the method of standard deviation ellipse and superposition analysis. It shows that the periphery of the radial sand ridge area off the coast of Jiangsu Province is the main birthplace of the HAB. Entering the breeding period, the northward drift speed as well as the diffusion speed of HAB accelerates. Until late June, it invades the coast of Shandong Province. In the early stage of HAB growth, preventing the spreading along the northeast-southwest direction will effectively restrain the HAB.

Delivering treatment recommendations via pervasive electronic devices such as mobile phones has the potential to be a viable and scalable treatment medium for long-term health behavior management. But active experimentation of treatment options can be time-consuming, expensive and altogether unethical in some cases. There is a growing interest in methodological approaches that allow an experimenter to learn and evaluate the usefulness of a new treatment strategy before deployment. We present the first development of a treatment recommender system for emotion regulation using real-world historical mobile digital data from n = 114 high socially anxious participants to test the usefulness of new emotion regulation strategies. We explore a number of offline contextual bandits estimators for learning and propose a general framework for learning algorithms. Our experimentation shows that the proposed doubly robust offline learning algorithms performed significantly better than baseline approaches, suggesting that this type of recommender algorithm could improve emotion regulation. Given that emotion regulation is impaired across many mental illnesses and such a recommender algorithm could be scaled up easily, this approach holds potential to increase access to treatment for many people. We also share some insights that allow us to translate contextual bandit models to this complex real-world data, including which contextual features appear to be most important for predicting emotion regulation strategy effectiveness.

Continuous passive sensing using smartphone embedded sensors can drain the battery quickly, interrupting other usages of the device. In order to improve the energy efficiency in continuous mobile sensing applications, we propose a new adaptive sensing framework using reinforcement learning to optimize the sensing timing. We model our adaptive sensing problem as a Markov Decision Process and dynamically change the sensing timing of targeted sensor(s) so that they are only operating in desired contexts (e.g. collect accelerometer data only when the phone is moving). Using accelerometer data continuously collected from 220 participants for over two weeks, we show that our approach is able to save the energy while attaining high accuracy and data coverage. Specifically, our strategy attains energy saving of 62.4 % at an accuracy of 80.9% and data coverage of 67.4%, which outperforms two baseline strategies, including a random strategy and a strategy using learning automata technique.