The rapid advancement of deep learning and the exponential growth of multimodal data have led to increased attention on multimodal emotion analysis and comprehension in affect computing. While existing multimodal works have achieved notable results in emotion recognition, several challenges remain. First, the scarcity of public large-scale multimodal emotion datasets is attributed to the high cost of manual annotation and the subjectivity of handcrafted labels. Second, most approaches only focus on learning emotion category information, disregarding the crucial evaluation indicator of emotion intensity, which hampers the development of fine-grained emotion recognition. Third, a significant emotion semantic discrepancy exists in different modalities, and current methodologies struggle to bridge the cross-modal gap and effectively utilize a vast amount of unlabeled emotion data, hindering the production of high-quality pseudolabels and superior classification performance. To address these challenges, based on the multitask learning architecture, we propose a novel semisupervised fine-grained emotion recognition model SMEIR-net for multimodal emotion and intensity recognition. Concretely, in semisupervised learning (SSL) phase, we design multistage self-training and consistency regularization paradigm to generate high-quality pseudolabels. Then, in supervised learning phase, we leverage multimodal transformer fusion and adversarial learning to eliminate the cross-modal semantic discrepancy. Extensive experiments are conducted on three benchmark datasets, namely RAVDESS, eNTERFACE, and Lombard-GRID, to evaluate the proposed model. The series sets of experimental results demonstrate that our SSL model successfully utilizes multimodal data and available labels to transfer emotion and intensity information from labeled to unlabeled datasets. Moreover, the corresponding evaluation metrics demonstrate that the utilize high-quality pseudolabels can achieve superior emotion and intensity classification performance, which outperforms other state-of-the-art baselines under the same condition.

In recent years, there has been a significant surge in commercial demand for citation graph-based tasks, such as patent analysis, social network analysis, and recommendation systems. Graph Neural Networks (GNNs) are widely used for these tasks due to their remarkable performance in capturing topological graph information. However, GNNs’ output results are highly dependent on the composition of local neighbors within the topological structure. To address this issue, we identify two types of neighbors in a citation graph: explicit neighbors based on the topological structure and implicit neighbors based on node features. Our primary motivation is to clearly define and visualize these neighbors, emphasizing their importance in enhancing graph neural network performance. We propose a Correlation-aware Network (CNet) to re-organize the citation graph and learn more valuable informative representations by leveraging these implicit and explicit neighbors. Our approach aims to improve graph data augmentation and classification performance, with the majority of our focus on stating the importance of using these neighbors, while also introducing a new graph data augmentation method. We compare CNet with state-of-the-art (SOTA) GNNs and other graph data augmentation approaches acting on GNNs. Extensive experiments demonstrate that CNet effectively extracts more valuable informative representations from the citation graph, significantly outperforming baselines. The code is available on public GitHub. 1

We study the forecasting problem for traffic with dynamic, possibly periodical, and joint spatial-temporal dependency between regions. Given the aggregated inflow and outflow traffic of regions in a city from time slots 0 to$t - 1$, we predict the traffic at time$t$for any region. Prior arts in the area often considered the spatial and temporal dependencies in a decoupled manner, or were rather computationally intensive in training with a large number of hyper-parameters which needed tuning. We propose ST-TIS, a novel, lightweight and accurateSpatial-TemporalTransformer withinformation fusion and regionsampling for traffic forecasting. ST-TIS extends the canonical Transformer with information fusion and region sampling. The information fusion module captures the complex spatial-temporal dependency between regions. The region sampling module is to improve the efficiency and prediction accuracy, cutting the computation complexity for dependency learning from$O(n^{2})$to$O(n\sqrt{n})$, where$n$is the number of regions. With far fewer parameters than state-of-the-art deep learning models, ST-TIS's offline training is significantly faster in terms of tuning and computation (with a reduction of up to$90\%$on training time and network parameters). Notwithstanding such training efficiency, extensive experiments show that ST-TIS is substantially more accurate in online prediction than state-of-the-art approaches (with an average improvement of$9.5\%$on RMSE, and$12.4\%$on MAPE compared to STDN and DSAN).

Docked bike systems have been widely deployed in many cities around the world. To the service provider, predicting the demand and supply of bikes at any station is crucial to offering the best service quality. The docked bike prediction problem is highly challenging because of the complicated joint spatial-temporal (ST) dependency as bikes are picked up and dropped off, the so-called “flows”, between stations. Prior works often considered the spatial and temporal dependencies separately using sequential network models, and based on locality assumptions. Without sufficiently capturing the joint spatial and temporal features, these approaches are not optimal for attaining the best prediction accuracy. We propose STGNN-DJD, a novel data-driven Spatial-Temporal Graph Neural Network to solve the bike demand and supply prediction problem by unifiedly embedding the Dynamic and Joint ST Dependency in two novel ST graphs. Given station locations and historical rental data on bike flow over the past time slots 0 to$t-1$, we seek to predict online the bike demand and supply at any station at time$t$. To extract joint spatial-temporal dependency, STGNN-DJD employs a graph generator to construct, at the beginning of time$t$, two graphs that embed the flow relationships between stations at various time slots (flow-convoluted graph) and dynamic demand-supply pattern correlation between stations (pattern correlation graph), respectively. Given the two spatial-temporal graphs, STGNN-DJD subsequently employs a graph neural network with novel flow-based and attention-based aggregators to generate embedding of each station for docked bike prediction. We have conducted extensive experiments on two large bike-sharing datasets. Our re-sults confirm the effectiveness of STGNN-DJD as compared with other state-of-the-art approaches, with significant improvement on RMSE and MAE (by 20%-50%). We also provide a case study on dynamic dependencies between stations and demonstrate that the locality assumption does not always hold for a docked bike system.

With the rapid advances and the penetration of the Internet of Things and sensors, a massive amount of trajectory data, given by discrete locations at certain timestamps, have been extracted or collected. Knowing the similarity between trajectories is fundamental to understanding their spatial-temporal correlation, with direct and far-reaching applications in contact tracing, companion detection, personalized marketing, etc. In this work, we consider the general and realistic sensing scenario that the locations of the trajectories may be noisy, and that these trajectories are sporadically sampled with randomness and asynchrony from the underlying continuous paths. Most of the prior work on trajectory similarity has not sufficiently considered the temporal dimension, or the issues of location noise and sporadic sampling, while others have limitations of strong assumptions such as a fixed known speed of users or the availability of a large amount of training data.We propose a novel and effective spatial-temporal measure termed STS (Spatial-Temporal Similarity) to evaluate the spatial-temporal overlap between any two trajectories. In order to account for the location noise and sporadic sampling, STS models each location in a trajectory as an observable outcome drawn from a probability distribution. With that, it efficiently reduces the need for training data by estimating a personalized spatial-temporal probability distribution of the object position from its own trajectory. Based on that, it subsequently computes the co-location probability and hence derives the similarity of any two trajectories. We have conducted extensive experiments to evaluate STS using real large-scale indoor (mall) and outdoor (taxi) datasets. Our results show that STS is substantially more accurate and robust than the state-of-the-art approaches, with an improvement of 63% on precision and 85% on mean rank.

Abstract In emergency situation, mass panic often causes more causalities than the disaster itself. The crowd emotional model could be used to simulate how crowd behavior in emergency scenarios and be helpful for developing crowd evacuation plans in emergency situations. However, existing crowd emotional models usually set model parameters in an empirical manner and are not validated by real cases. In this paper, a crowd emotional model is proposed to simulate the crowd movement in outdoor emergency situations. First of all, the crowd entropy and the movement difference are proposed to describe the emotional impact of the crowd scene on the agents. The perception of vision and hearing are considered, and the calculation formulas of the agent's emotional intensity and crowd emotional contagion are proposed. By calculating individual trajectories in the real video, the cumulative differences between the movements of the real crowd and the corresponding virtual crowd are analyzed. At last, a multi‐parameter optimization method is implemented by the differential evolution algorithm. To verify the parameters in models, three videos which are generated from three real cases, including explosion attack, shooting incident, and crowd disturbance are selected for experimental verification. The results showed that the proposed model could be a feasible method for optimizing parameters to simulate the emergency scenario.

The importance of effective surgical wound care cannot never be underestimated. Poorly managing surgical wounds may cause many serious complications. Thus, it raises the necessity to develop a patient-friendly self-care system which can help both patients and medical professionals to ensure the state of the surgical wounds without any special medical equipment. In this paper, a surgical wound assessment system for self-care is proposed. The proposed system is designed to enable patients capture surgical wound images of themselves by using a mobile device and upload these images for analysis. Combining image-processing and machine-learning techniques, the proposed method is composed of four phases. First, images are segmented into superpixels where each superpixel contains the pixels in the similar color distribution. Second, these superpixels corresponding to the skin are identified and the area of connected skin superpixels is derived. Third, surgical wounds will be extracted from this area based on the observation of the texture difference between skin and wounds. Lastly, state and symptoms of surgical wound will be assessed. Extensive experimental results are conducted. With the proposed method, more than 90% state assessment results are correct and more than 91% symptom assessment results consistent with the actual diagnosis. Moreover, case studies are provided to show the advantage and limitation of this system. These results show that this system could perform well in the practical self-care scenario.

Public transportation is beating heart of a city. Understanding how citizens utilize public transportation can be used to optimize many applications such as traffic planning, crowd flow prediction, and location-based marketing. However, obtaining how citizens used transportation is not a trivial task. It is almost not possible to ask citizens to report their exact location and their transportation mode; moreover, there are usually various public transportation that move along the similar paths. These increase challenges to identify people's transport modes. To address these issues, this paper proposes Public Transport General Framework (PTGF) to identify people's transport modes by their cellular data in both offline and online manners. Regarding the offline phase, given historical cellular data of people and urban transportation networks, PTGF derives cellular data into trajectories, to match each trajectory to public transportation networks to find the most possible transport modes for sub-trajectories of a trajectory. In the online phase, given streaming trajectories, PTGF identifies the transport modes of each location by an LSTM which are trained by historical trajectories with transport modes annotated in the offline phase. Extensive experiments are conducted by using both synthetic and real datasets. The experimental results show that the accuracy of PTGF in offline phase around 80% and that in online phase F1-score around 0.7, which could prove that the effectiveness of the proposed framework PTGF.

As the web information exponentially grows and the needs of users become more specific, traditional general web search engines are not able to perfectly satisfy the nowadays user requirement. Vertical search engines have emerged in various domains, which more focus on specific segments of online content, including local, shopping, medical information, travel search, etc. Vertical search engines start attracting more attention while relevance ranking in different vertical search engines is becoming the key technology. In addition, vertical search results are often slotted into general Web search results. Hence, designing effective ranking functions for vertical search has become practically important to improve users' experience in both web search and vertical search. The workshop bring together researchers from IR, ML, NLP, and other areas of computer and information science, who are working on or interested in this area. It provides a forum for the researchers to identify the issues and the challenges, to share their latest research results, to express a diverse range of opinions about this topic, and to discuss future directions.

In this paper, we tackle the problem of discovering movement-based communities of users, where users in the same community have similar movement behaviors. Note that the identification of movement-based communities is beneficial to location-based services and trajectory recommendation services. Specifically, we propose a framework to mine movement-based communities which consists of three phases: 1) constructing trajectory profiles of users, 2) deriving similarity between trajectory profiles, and 3) discovering movement-based communities. In the first phase, we design a data structure, called the Sequential Probability tree (SP-tree), as a user trajectory profile. SP-trees not only derive sequential patterns, but also indicate transition probabilities of movements. Moreover, we propose two algorithms: BF (standing for breadth-first) and DF (standing for depth-first) to construct SP-tree structures as user profiles. To measure the similarity values among users’ trajectory profiles, we further develop a similarity function that takes SP-tree information into account. In light of the similarity values derived, we formulate an objective function to evaluate the quality of communities. According to the objective function derived, we propose a greedy algorithm Geo-Cluster to effectively derive communities. To evaluate our proposed algorithms, we have conducted comprehensive experiments on two real data sets. The experimental results show that our proposed framework can effectively discover movement-based user communities.

The advance of object tracking technologies leads to huge volumes of spatio-temporal data collected in the form of trajectory data stream. In this study, we investigate the problem of discovering object groups that travel together (i.e., traveling companions) from trajectory stream. Such technique has broad applications in the areas of scientific study, transportation management and military surveillance. To discover traveling companions, the monitoring system should cluster the objects of each snapshot and intersect the clustering results to retrieve moving-together objects. Since both clustering and intersection steps involve high computational overhead, the key issue of companion discovery is to improve the algorithm's efficiency. We propose the models of closed companion candidates and smart intersection to accelerate data processing. A new data structure termed traveling buddy is designed to facilitate scalable and flexible companion discovery on trajectory stream. The traveling buddies are micro-groups of objects that are tightly bound together. By only storing the object relationships rather than their spatial coordinates, the buddies can be dynamically maintained along trajectory stream with low cost. Based on traveling buddies, the system can discover companions without accessing the object details. The proposed methods are evaluated with extensive experiments on both real and synthetic datasets. The buddy-based method is an order of magnitude faster than existing methods. It also outperforms other competitors with higher precision and recall in companion discovery.

To conserve energy, sensor nodes with similar readings can be grouped such that readings from only the representative nodes within the groups need to be reported. However, efficiently identifying sensor groups and their representative nodes is a very challenging task. In this paper, we propose a centralized algorithm to determine a set of representative nodes with high energy levels and wide data coverage ranges. Here, the data coverage range of a sensor node is considered to be the set of sensor nodes that have reading behaviors very close to the particular sensor node. To further reduce the extra cost incurred in messages for selection of representative nodes, a distributed algorithm is developed. Furthermore, maintenance mechanisms are proposed to dynamically select alternative representative nodes when the original representative nodes run low on energy, or cannot capture spatial correlation within their respective data coverage ranges. Using experimental studies on both synthesis and real data sets, our proposed algorithms are shown to effectively and efficiently provide approximate data collection while prolonging the network lifetime.

A Cyber-Physical System (CPS) integrates physical devices (e.g., sensors, cameras) with cyber (or informational)components to form a situation-integrated analytical system that responds intelligently to dynamic changes of the real-world scenarios. One key issue in CPS research is trustworthiness analysis of the observed data: Due to technology limitations and environmental influences, the CPS data are inherently noisy that may trigger many false alarms. It is highly desirable to sift meaningful information from a large volume of noisy data. In this paper, we propose a method called Tru-Alarm which finds out trustworthy alarms and increases the feasibility of CPS. Tru-Alarm estimates the locations of objects causing alarms, constructs an object-alarm graph and carries out trustworthiness inferences based on linked information in the graph. Extensive experiments show that Tru-Alarm filters out noises and false information efficiently and guarantees not missing any meaningful alarms.

In recent years, the global position system (GPS) is widely used in technical products, such as navigation devices, GPS loggers, PDAs and mobile phones. Hence, traffic data collection platforms are proposed to collect GPS data points for traffic monitoring. In traffic data collection platforms, each vehicle equips with GPS modules and the wireless communication interfaces, such as 3G or WiFi networks, and the GPS data sensed (e.g., the speed and the position) are sent to the server. One challenge issue is that if a significant number of vehicles upload their GPS data points at the same time, it is possible that the wireless network cannot offer enough network resources for simultaneous network connections. This paper proposes a framework MDC (standing for Model-based Data Collection) to reduce the amount of data transmission and the number of vehicles reporting their GPS data points. The MDC framework is executed at the server and vehicle side collaboratively. In the vehicle side, given a series of GPS data points, model functions are derived to represent the raw GPS data points. Hence, each vehicle could report some coefficients that describe its movements instead of reporting all position information. Since vehicles move along with road segments that are usually a set of line segments, algorithm LR (standing for Liner Regression) is proposed to determine a set of line functions to represent movements of vehicles. By observing the spatial-temporal locality in traffic data, algorithm KR (standing for Kernel Regression) is developed to derive a set of kernel functions to model a series of speed readings sensed. Moreover, with the spatial-temporal locality feature in traffic data, an in-network aggregation mechanism are proposed to determine a set of groups and for each group, only one vehicle needs to report traffic data, thereby further reducing the number of simultaneous connections. Experimental results show that MDC can collect traffic data effectively and the efficiently.

Prior works have shown that probabilistic suffix trees (PST) could predict accurately the moving behaviors of objects for prediction-based object tracking sensor networks. However, maintaining PSTs for objects incurs a considerable amount of storage spaces for resource-constrained sensor nodes. In this paper, we derive a distance function between two PSTs and propose an algorithm to determine the similarity between them. By the distance between PSTs, we propose a clustering algorithm to partition objects with similar moving behaviors into groups. Furthermore, for each group, one PST is selected to predict movements of objects within one group. Experimental results show that our proposed approaches not only effectively reduce the storage cost but also provide good prediction accuracy.