Recent advances in video generation techniques have given rise to an emerging paradigm of generative video coding for Ultra-Low Bitrate (ULB) scenarios by leveraging powerful generative priors. However, most existing methods are limited by domain specificity (e.g., facial or human videos) or excessive dependence on high-level text guidance, which tend to inadequately capture fine-grained motion details, leading to unrealistic or incoherent reconstructions. To address these challenges, we propose Trajectory-Guided Generative Video Coding (dubbed T-GVC), a novel framework that bridges low-level motion tracking with high-level semantic understanding. T-GVC features a semantic-aware sparse motion sampling pipeline that extracts pixel-wise motion as sparse trajectory points based on their semantic importance, significantly reducing the bitrate while preserving critical temporal semantic information. In addition, by integrating trajectory-aligned loss constraints into diffusion processes, we introduce a training-free guidance mechanism in latent space to ensure physically plausible motion patterns without sacrificing the inherent capabilities of generative models. Experimental results demonstrate that T-GVC outperforms both traditional and neural video codecs under ULB conditions. Furthermore, additional experiments confirm that our framework achieves more precise motion control than existing text-guided methods, paving the way for a novel direction of generative video coding guided by geometric motion modeling.

The Segment Anything Model (SAM) excels at generating high-quality object masks with various prompts but struggles in ultra-low light. We developed EP-SAM (Edge-Detection Prompt SAM) with a Low Light Edge-Detection Network (LLEN), offering strong robustness and lightweight performance in ultra-low light. Using edge information as prompts, EP-SAM achieves high-precision segmentation in low-light videos.By combining motion estimation with reference frame optimization, the initial frame can predict the next 29 frames, reducing inference time by over 80% and computational complexity by 86%. Tests show LLEN accurately extracts edges even with about 80 photons per pixel, enabling EP-SAM to produce precise masks and significantly outperform SAM and SAM-2. EP-SAM improves mean Intersection over Union (mIoU) by 5.85% over SAM on the CamVid dataset. Video demos: https://github.com/wzt22thu/EP-SAM/releases/tag/DEMO.

Autonomous parking has become a critical application in automatic driving research and development. Parking operations often suffer from limited space and complex environments, requiring accurate perception and precise maneuvering. Traditional rule-based parking algorithms struggle to adapt to diverse and unpredictable conditions, while learning-based algorithms lack consistent and stable performance in various scenarios. Therefore, a hybrid approach is necessary that combines the stability of rule-based methods and the generalizability of learning-based methods. Recently, reinforcement learning (RL) based policy has shown robust capability in planning tasks. However, the simulation-to-reality (sim-to-real) transfer gap seriously blocks the real-world deployment. To address these problems, we employ a hybrid policy, consisting of a rule-based Reeds-Shepp (RS) planner and a learningbased reinforcement learning (RL) planner. A real-time LiDARbased Occupancy Grid Map (OGM) representation is adopted to bridge the sim-to-real gap, leading the hybrid policy can be applied to real-world systems seamlessly. We conducted extensive experiments both in the simulation environment and real-world scenarios, and the result demonstrates that the proposed method outperforms pure rule-based and learningbased methods. The real-world experiment further validates the feasibility and efficiency of the proposed method.

Large language models (LLMs) excel at generating code from natural language (NL) descriptions. However, the plain textual descriptions are inherently ambiguous and often fail to capture complex requirements like intricate system behaviors, conditional logic, and architectural constraints; implicit data dependencies in service-oriented architectures are difficult to infer and handle correctly.To bridge this gap, we propose a novel step-by-step code generation framework named UML2Dep by leveraging unambiguous formal specifications of complex requirements. First, we introduce an enhanced Unified Modeling Language (UML) sequence diagram tailored for service-oriented architectures. This diagram extends traditional visual syntax by integrating decision tables and API specifications, explicitly formalizing structural relationships and business logic flows in service interactions to rigorously eliminate linguistic ambiguity. Second, recognizing the critical role of data flow, we introduce a dedicated data dependency inference (DDI) task. DDI systematically constructs an explicit data dependency graph prior to actual code synthesis. To ensure reliability, we formalize DDI as a constrained mathematical reasoning task through novel prompting strategies, aligning with LLMs’ excellent mathematical strengths. Additional static parsing and dependency pruning further reduce context complexity and cognitive load associated with intricate specifications, thereby enhancing reasoning accuracy and efficiency.Experimental results on our in-house industrial datasets demonstrate the effectiveness of the proposed framework. Specifically, our framework achieves strong performance, with 89.97% recall, 95.06% precision, and 92.33% F1 score on the DDI task. Furthermore, the integration of UML2Dep into the code generation pipeline also improves practical deployment, increasing compilation pass rate by 8.83% and unit test pass rate by 11.66%.

Gradient-based meta-learning (GBML) algorithms can quickly adapt to new tasks by transferring the learned meta-knowledge while assuming that all tasks come from the same distribution (in-distribution, ID). However, in the real world, they often grapple with an out-of-distribution (OOD) generalization challenge, where tasks stem from diverse distributions. OOD exacerbates discrepancies in task gradient magnitudes and directions, posing a formidable challenge for GBML in optimizing meta-knowledge by minimizing the sum of task gradients in each minibatch. To address this problem, we propose RotoGBML, a novel approach designed to homogenize OOD task gradients. RotoGBML employs reweighted vectors to dynamically balance diverse magnitudes to a standardized scale and uses rotation matrices to align conflicting directions. To reduce overhead, we homogenize gradients with the features rather than network parameters. Additionally, to circumvent the impact of non-causal features (e.g., backgrounds), we propose an Invariant Self-Information (ISI) module to extract invariant causal features (e.g., the outlines of objects). Finally, task gradients are homogenized based on these invariant causal features. Experiments demonstrate that RotoGBML outperforms state-of-the-art methods across various few-shot benchmarks.

Depth estimation problem for the perspective vision systems has been extensively studied in the literature and the depth estimation can be achieved under the assumption that the exact camera motion parameters are available. However, in practice, it is hard to obtain the camera motion parameters exactly. For instance, the exact camera velocity is always unavailable and instead only the roughly estimated one can be obtained. This introduces the significant difficulties to achieve the depth estimation with partially uncertain camera motion parameters. In this article, we consider the depth estimation problem for the perspective vision system in the case that the camera angular velocities are accurate, while partial camera linear velocities are contaminated by some disturbances. This problem is theoretically formulated and solved for the first time in this article by proposing a new depth observer in the presence of the partially uncertain camera linear velocities. Local exponential convergence of the depth and disturbance estimates is achieved in presence of the partially uncertain camera linear velocities such that the estimation errors of states and disturbances converge to zero for constant disturbances or to a small error bound for bound time-varying disturbances. Simulations and experiments are carried out to verify the performance of the proposed observer.

Network motif is a kind of frequently occurring subgraph that reflects local topology in graphs. Although network motif has been studied in graph analytics, e.g., social network and biological network, it is yet unclear whether network motif is useful for analyzing online transaction network that is generated in applications such as instant messaging and e-commerce. In this work, we analyze online transaction networks from the perspective of network motif. We define vertex features based on size-2 and size-3 motifs, and introduce motif-based centrality measurements. We further design motif-based vertex embedding that integrates weighted motif counts and centrality measurements. Afterward, we implement a distributed framework for motif detection in large-scale online transaction networks. To understand the effectiveness of motif for analyzing online transaction network, we study the statistical distribution of motifs in various kinds of graphs in Tencent and assess the benefit of motif-based embedding in a range of downstream graph analytical tasks. Empirical results show that our proposed method can efficiently find motifs in large-scale graphs, help interpretability, and benefit downstream tasks.

Recommender systems aim to accurately and actively provide users with potentially interesting items (products, information or services). Deep reinforcement learning has been successfully applied to recommender systems, but still heavily suffer from data sparsity and cold-start in real-world tasks. In this work, we propose an effective way to address such issues by leveraging the pervasive social networks among users in the estimation of action-values (Q). Specifically, we develop a Social Attentive Deep Q-network (SADQN) to approximate the optimal action-value function based on the preferences of both individual users and social neighbors, by successfully utilizing a social attention layer to model the influence between them. Further, we propose an enhanced variant of SADQN, termed SADQN++, to model the complicated and diverse trade-offs between personal preferences and social influence for all involved users, making the agent more powerful and flexible in learning the optimal policies. The experimental results on real-world datasets demonstrate that the proposed SADQNs remarkably outperform the state-of-the-art deep reinforcement learning agents, with reasonable computation cost.

Information sharing behavior and social link building behavior have shown strong correlation on social media. The aim of this paper is to explore this correlation for simultaneously modeling and predicting sharing behavior in information diffusion sequences and linking behavior in social network, which correspond to information diffusion prediction and social link prediction problems. To achieve this goal, we propose a joint user representation learning model to characterize the two correlated behaviors in a shared latent space. The proposed model learns user representations via two maximum likelihood estimation objectives defined on observed information diffusion sequences and social network structure respectively and incorporates them in a unified framework. A multi-task learning algorithm is designed for efficient model optimization. Based on the learned representations, the model can be directly applied to predicting diffusion processes and inferring unobserved social links at the same time. We evaluate the proposed model on two real social media datasets with extensive experiments. The model consistently achieves significant improvements over the state-of-the-art approaches on diffusion prediction and link prediction tasks. The better robustness of our model in further ablation studies demonstrates that capturing the behavior correlation in the shared representation space is beneficial.

For autonomous vehicles integrating onto road-ways with human traffic participants, it requires understanding and adapting to the participants’ intention by responding in predictable ways. This paper proposes a reinforcement learning based negotiation-aware motion planning framework, which adopts RL to adjust the driving style of the planner by dynamically modifying the prediction horizon length of the motion planner in real time adaptively. The framework models the interaction between the autonomous vehicle and other traffic participants as a Markov Decision Process. A temporal sequence of occupancy grid maps are taken as inputs for RL module to embed an implicit intention reasoning. Curriculum learning is employed to enhance the training efficiency and the robustness of the algorithm. We applied our method to narrow lane navigation in both simulation and real world to demonstrate that the proposed method outperforms the common alternative due to its advantage in alleviating the social dilemma problem with proper negotiation skills.

In this work, we establish a context graph from both conversation utterances and external knowledge, and develop a novel graph-based encoder to better understand the conversation context. Specifically, the encoder fuses the information in the context graph stage-by-stage and provides global context-graph-aware representations of each node in the graph to facilitate knowledge-grounded response generation. On a large-scale conversation corpus, we validate the effectiveness of the proposed approach and demonstrate the benefit of knowledge in conversation understanding.

In this article, we aim at developing neighborhood-based neural models for link prediction. We design a novel multispace neighbor attention mechanism to extract universal neighborhood features by capturing latent importance of neighbors and selectively aggregate their features in multiple latent spaces. Grounded on this mechanism, we propose two link prediction models, i.e., self neighborhood attention network (SNAN), which predicts the link of two nodes by encoding and matching their respective neighborhood information, and its extension cross neighborhood attention network (CNAN), where we additionally design a cross neighborhood attention to directly capture structural interactions between two nodes. Another key novelty of this work is that we propose an adversarial learning framework, where a negative sample generator is devised to improve the optimization of the proposed link prediction models by continuously providing highly informative negative samples in the adversarial game. We evaluate our models with extensive experiments on 12 benchmark data sets against 14 popular and state-of-the-art link prediction approaches. The results strongly demonstrate the significant and universal superiority of our models on various types of networks. The effectiveness and robustness of the proposed attention mechanism and adversarial learning framework are also verified by detailed ablation studies.

Nowadays, social media has become a popular channel for people to exchange opinions through the user-generated text. Exploring the mechanisms about how customers' opinions towards products are influenced by friends, and further predicting their future opinions have attracted great attention from corporate administrators and researchers. Various influence models have already been proposed for the opinion prediction problem. However, they largely formulate opinions as derived sentiment categories or values but ignore the role of the content information. Besides, existing models only make use of the most recently received information without taking into consideration the long-term historical communication. To keep track of user opinion behaviors and infer user opinion influence from the historical exchanged textual information, we develop a content-based sequential opinion influence framework. Based on this framework, two opinion sentiment prediction models with alternative prediction strategies are proposed. In the experiments conducted on three Twitter datasets, the proposed models outperform other popular influence models. An interesting finding based on a further analysis of user characteristic is that an individuals influence is correlated to her/his style of expressions.

Interactions between neighborhoods of two target nodes are often regarded as important clues for link prediction. In this paper, we propose a novel link prediction neural model named Neighborhood Interaction Attention Network (NIAN), which is able to automatically learn comprehensive neighborhood interaction features and predict links in an end-to-end way. The proposed model mainly consists of two attention layers. A node-level attention is designed to extract latent structure features of nodes in target neighborhoods. Based on the latent node features, a neighborhood-level attention is proposed to learn neighborhood interaction features by considering different importance of pair-wise interactions. The superiority of NIAN is demonstrated by extensive experiments on 6 benchmark datasets against 12 popular and state-of-the-art approaches.

A series of recent studies formulated the diffusion prediction problem as a sequence prediction task and proposed several sequential models based on recurrent neural networks. However, non-sequential properties exist in real diffusion cascades, which do not strictly follow the sequential assumptions of previous work. In this paper, we propose a hierarchical diffusion attention network (HiDAN), which adopts a non-sequential framework and two-level attention mechanisms, for diffusion prediction. At the user level, a dependency attention mechanism is proposed to dynamically capture historical user-to-user dependencies and extract the dependency-aware user information. At the cascade (i.e., sequence) level, a time-aware influence attention is designed to infer possible future user's dependencies on historical users by considering both inherent user importance and time decay effects. Significantly higher effectiveness and efficiency of HiDAN over state-of-the-art sequential models are demonstrated when evaluated on three real diffusion datasets. The further case studies illustrate that HiDAN can accurately capture diffusion dependencies.

While deep reinforcement learning has been successfully applied to recommender systems, it is challenging and unexplored to improve the performance of deep reinforcement learning recommenders by effectively utilizing the pervasive social networks. In this work, we develop a Social Attentive Deep Q-network (SADQN) agent, which is able to provide high-quality recommendations during user-agent interactions by leveraging social influence among users. Specifically, SADQN is able to estimate action-values not only based on the users' personal preferences, but also based on their social neighbors' preferences by employing a particular social attention layer. The experimental results on three real-world datasets demonstrate that SADQN significantly improves the performance of deep reinforcement learning agents that overlook social influence.

In this article, we aim at developing a user representation learning model to solve the information diffusion prediction problem in social media. The main idea is to project the diffusion users into a continuous latent space as the role-based (sender and receiver) representations, which capture unique diffusion characteristics of users. The model learns the role-based representations based on a cascade modeling objective that aims at maximizing the likelihood of observed cascades, and employs the matrix factorization objective of reconstructing structural proximities as a regularization on representations. By jointly embedding the information of cascades and network, the learned representations are robust on different diffusion data. We evaluate the proposed model on three real-world datasets. The experimental results demonstrate the better performance of the proposed model than state-of-the-art diffusion embedding and network embedding models and other popular graph-based methods.

With the growing volume of publications in the Computer Science (CS) discipline, tracking the research evolution and predicting the future research trending topics are of great importance for researchers to keep up with the rapid progress of research. Within a research area, there are many top conferences that publish the latest research results. These conferences mutually influence each other and jointly promote the development of the research area. To predict the trending topics of mutually influenced conferences, we propose a correlated neural influence model, which has the ability to capture the sequential properties of research evolution in each individual conference and discover the dependencies among different conferences simultaneously. The experiments conducted on a scientific dataset including conferences in artificial intelligence and data mining show that our model consistently outperforms the other state-of-the-art methods. We also demonstrate the interpretability and predictability of the proposed model by providing its answers to two questions of concern, i.e., what the next rising trending topics are and for each conference who the most influential peer is.

In this paper, we propose a novel sequential neural network with structure attention to model information diffusion. The proposed model explores both sequential nature of an information diffusion process and structural characteristics of user connection graph. The recurrent neural network framework is employed to model the sequential information. The attention mechanism is incorporated to capture the structural dependency among users, which is defined as the diffusion context of a user. A gating mechanism is further developed to effectively integrate the sequential and structural information. The proposed model is evaluated on the diffusion prediction task. The performances on both synthetic and real datasets demonstrate its superiority over popular baselines and state-of-the-art sequence-based models.

Session-based recommendation performance has been significantly improved by Recurrent Neural Networks (RNN). However, existing RNN-based models do not expose the global knowledge of frequent click patterns or consider variability of sequential behaviors in sessions. In this paper, we propose a novel Variational Recurrent Model (VRM), which employs the stochastic latent variable to capture the knowledge of frequent click patterns and impose variability for the sequential behavior modeling. A stochastic generative process of session sequence is specified, where the latent variable modulates the generation of session sequences in RNN. We further extend VRM to a Conditional Variational Recurrent Model (CVRM) by considering additional information (e.g., focused category in sessions) as the generative condition. When evaluated on a public benchmark dataset, VRM and its extension clearly demonstrate their superiority over popular baselines and state-of-the-art models.

Exploring the mechanism that explains how a user's opinion changes under the influence of his/her neighbors is of practical importance (e.g., for predicting the sentiment of his/her future opinion) and has attracted wide attention from both enterprises and academics.Though various opinion influence models have been proposed for opinion prediction, they only consider users' personal identities, but ignore their social identities with which people behave to fit the expectations of the others in the same group. In this work, we explore users' dual identities, including both personal identities and social identities to build a more comprehensive opinion influence model for a better understanding of opinion behaviors. A novel joint learning framework is proposed to simultaneously model opinion dynamics and detect social identity in a unified model. The effectiveness of the proposed approach is demonstrated through the experiments conducted on Twitter datasets

In this paper, we propose a predictive network representation learning (PNRL) model to solve the structural link prediction problem. The proposed model defines two learning objectives, i.e., observed structure preservation and hidden link prediction. To integrate the two objectives in a unified model, we develop an effective sampling strategy to select certain edges in a given network as assumed hidden links and regard the rest network structure as observed when training the model. By jointly optimizing the two objectives, the model can not only enhance the predictive ability of node representations but also learn additional link prediction knowledge in the representation space. Experiments on four real-world datasets demonstrate the superiority of the proposed model over the other popular and state-of-the-art approaches.

Nowadays, social media has become a popular platform for companies to understand their customers. It provides valuable opportunities to gain new insights into how a person’s opinion about a product is influenced by his friends. Though various approaches have been proposed to study the opinion formation problem, they all formulate opinions as the derived sentiment values either discrete or continuous without considering the semantic information. In this paper, we propose a Content-based Social Influence Model to study the implicit mechanism underlying the change of opinions. We then apply the learned model to predict users’ future opinions. The advantages of the proposed model is the ability to handle the semantic information and to learn two influence components including the opinion influence of the content information and the social relation factors. In the experiments conducted on Twitter datasets, our model significantly outperforms other popular opinion formation models.

Micro-blog has been increasingly used for the public to express their opinions, and for organizations to detect public sentiment about social events or public policies. In this article, we examine and identify the key problems of this field, focusing particularly on the characteristics of innovative words, multi-media elements, and hierarchical structure of Chinese “Weibo.” Based on the analysis, we propose a novel approach and develop associated theoretical and technological methods to address these problems. These include a new sentiment word mining method based on three wording metrics and point-wise information, a rule set model for analyzing sentiment features of different linguistic components, and the corresponding methodology for calculating sentiment on multi-granularity considering emoticon elements as auxiliary affective factors. We evaluate our new word discovery and sentiment detection methods on a real-life Chinese micro-blog dataset. Initial results show that our new diction can improve sentiment detection, and they demonstrate that our multi-level rule set method is more effective, with the average accuracy being 10.2% and 1.5% higher than two existing methods for Chinese micro-blog sentiment analysis. In addition, we exploit visualization techniques to study the relationships between online sentiment and real life. The visualization of detected sentiment can help depict temporal patterns and spatial discrepancy.

Micro-blog has been increasingly used for the public to express their opinions, and for organisations to detect public sentiment about social events. In contrast to the effort and progress made in English-based micro-blog analysis, research on Chinese micro-blog received relatively little attention. In this paper we examine and identify the key problems of this field, focusing particularly on the characteristics of innovative words, emoticon elements and hierarchical structure of Chinese “Weibo”. Based on the analysis we propose and develop associated theoretical and technological methods to address these problems. These include the development of new sentiment word mining method based on three wording standards and point-wise metrics, a rule set model for analyzing sentiment features of different linguistic components, and the corresponding methodology for calculating sentiment on multi-granularity considering emoticon elements. We use original Chinese tweets from a dataset of Sina Weibo to test and evaluate our new word discovery and sentiment detection methods. Initial results show that our new diction can improve sentiment detection, and demonstrate that our multi-level rule set method is more effective by giving 10.2% and 1.5% higher average accuracy than two existing methods for Chinese micro-blog sentiment analysis. In addition, we exploit visualisation techniques to study the relationships between online sentiment and real life, which can help depict the correlation between public emotions and events.

The newly emerging event-based social networks (EBSNs) connect online and offline social interactions, offering a great opportunity to understand behaviors in the cyber-physical space. While existing efforts have mainly focused on investigating user behaviors in traditional social network services (SNS), this paper aims to exploit individual behaviors in EBSNs, which remains an unsolved problem. In particular, our method predicts activity attendance by discovering a set of factors that connect the physical and cyber spaces and influence individual's attendance of activities in EBSNs. These factors, including content preference, context (spatial and temporal) and social influence, are extracted using different models and techniques. We further propose a novel Singular Value Decomposition with Multi-Factor Neighborhood (SVD-MFN) algorithm to predict activity attendance by integrating the discovered heterogeneous factors into a single framework, in which these factors are fused through a neighborhood set. Experiments based on real-world data from Douban Events demonstrate that the proposed SVD-MFN algorithm outperforms the state-of-the-art prediction methods.