Video object detection has made significant progress in recent years thanks to convolutional neural networks (CNNs) and vision transformers (ViTs). Typically, CNNs excel at capturing local features but struggle to model global representations. Conversely, ViTs are adept at capturing long-range global features but face challenges in representing local feature details. Off-the-shelf video object detection methods solely rely on CNNs or ViTs to conduct feature aggregation, which hampers their capability to simultaneously leverage global and local information, thereby resulting in limited detection performance. In this paper, we propose a Transformer-GraphFormer Blender Network (TGBFormer) for video object detection, with three key technical improvements to fully exploit the advantages of transformers and graph convolutional networks while compensating for their limitations. First, we develop a spatial-temporal transformer module to aggregate global contextual information, constituting global representations with long-range feature dependencies. Second, we introduce a spatial-temporal GraphFormer module that utilizes local spatial and temporal relationships to aggregate features, generating new local representations that are complementary to the transformer outputs. Third, we design a global-local feature blender module to adaptively couple transformer-based global representations and GraphFormer-based local representations. Extensive experiments demonstrate that our TGBFormer establishes new state-of-the-art results on the ImageNet VID dataset. Particularly, our TGBFormer achieves 86.5% mAP while running at around 41.0 FPS on a single Tesla A100 GPU.

Shihan Dou, Yan Liu, Haoxiang Jia, Enyu Zhou, Limao Xiong, Junjie Shan, Caishuang Huang, Xiao Wang, Xiaoran Fan, Zhiheng Xi, Yuhao Zhou, Tao Ji, Rui Zheng, Qi Zhang, Tao Gui, Xuanjing Huang. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2024.

Shihan Dou, Enyu Zhou, Yan Liu, Songyang Gao, Wei Shen, Limao Xiong, Yuhao Zhou, Xiao Wang, Zhiheng Xi, Xiaoran Fan, Shiliang Pu, Jiang Zhu, Rui Zheng, Tao Gui, Qi Zhang, Xuanjing Huang. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2024.

Large Language Models (LLMs) have showcased remarkable capabilities in following human instructions. However, recent studies have raised concerns about the robustness of LLMs for natural language understanding (NLU) tasks when prompted with instructions combining textual adversarial samples. In this paper, drawing inspiration from recent works that LLMs are sensitive to the design of the instructions, we utilize instructions in code style, which are more structural and less ambiguous, to replace typically natural language instructions. Through this conversion, we provide LLMs with more precise instructions and strengthen the robustness of LLMs. Moreover, under few-shot scenarios, we propose a novel method to compose in-context demonstrations using both clean and adversarial samples (adversarial context method) to further boost the robustness of the LLMs. Experiments on eight robustness datasets show that our method consistently outperforms prompting LLMs with natural language, for example, with gpt-3.5-turbo on average, our method achieves an improvement of 5.68% in test set accuracy and a reduction of 5.66 points in Attack Success Rate (ASR).

The success of AI assistants based on language models (LLMs) hinges crucially on Reinforcement Learning from Human Feedback (RLHF), which enables the generation of responses more aligned with human preferences. As universal AI assistants, there's a growing expectation for them to perform consistently across various domains. However, previous work shows that Reinforcement Learning (RL) often exploits shortcuts to attain high rewards and overlooks challenging samples. This focus on quick reward gains undermines both the stability in training and the model's ability to generalize to new, unseen data. In this work, we propose a novel approach that can learn a consistent policy via RL across various data groups or domains. Given the challenges associated with acquiring group annotations, our method automatically classifies data into different groups, deliberately maximizing performance variance. Then, we optimize the policy to perform well on challenging groups. Lastly, leveraging the established groups, our approach adaptively adjusts the exploration space, allocating more learning capacity to more challenging data and preventing the model from over-optimizing on simpler data. Experimental results indicate that our approach significantly enhances training stability and model generalization.

The success of AI assistants based on Language Models (LLMs) hinges on Reinforcement Learning from Human Feedback (RLHF) to comprehend and align with user intentions. However, traditional alignment algorithms, such as PPO, are hampered by complex annotation and training requirements. This reliance limits the applicability of RLHF and hinders the development of professional assistants tailored to diverse human preferences. In this work, we introduce Linear Alignment, a novel algorithm that aligns language models with human preferences in one single inference step, eliminating the reliance on data annotation and model training. Linear alignment incorporates a new parameterization for policy optimization under divergence constraints, which enables the extraction of optimal policy in a closed-form manner and facilitates the direct estimation of the aligned response. Extensive experiments on both general and personalized preference datasets demonstrate that linear alignment significantly enhances the performance and efficiency of LLM alignment across diverse scenarios.

In this paper, we propose R$^3$: Learning Reasoning through Reverse Curriculum Reinforcement Learning (RL), a novel method that employs only outcome supervision to achieve the benefits of process supervision for large language models. The core challenge in applying RL to complex reasoning is to identify a sequence of actions that result in positive rewards and provide appropriate supervision for optimization. Outcome supervision provides sparse rewards for final results without identifying error locations, whereas process supervision offers step-wise rewards but requires extensive manual annotation. R$^3$ overcomes these limitations by learning from correct demonstrations. Specifically, R$^3$ progressively slides the start state of reasoning from a demonstration’s end to its beginning, facilitating easier model exploration at all stages. Thus, R$^3$ establishes a step-wise curriculum, allowing outcome supervision to offer step-level signals and precisely pinpoint errors. Using Llama2-7B, our method surpasses RL baseline on eight reasoning tasks by $4.1$ points on average. Notably, in program-based reasoning, 7B-scale models perform comparably to larger models or closed-source models with our R$^3$.

SongYang Gao, Shihan Dou, Yan Liu, Xiao Wang, Qi Zhang, Zhongyu Wei, Jin Ma, Ying Shan. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2023.

Xiao Wang, Shihan Dou, Limao Xiong, Yicheng Zou, Qi Zhang, Tao Gui, Liang Qiao, Zhanzhan Cheng, Xuanjing Huang. Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2022.

Intermodal road-rail transportation Systems (IRRTS) is a typical complex giant system which is dilïîcult for traditional management approaches to form effective solutions with dynamic adaptability. In the physical world, simple equipment upgrading and transport optimization can't satisfy the demand of IRRTS for high efficiency and low cost. Thus, a framework of parallel IRRTS based on ACP approach including artificial IRRTS, computational experiments and parallel execution is proposed. The artificial IRRTS is the foundation of parallel IRRTS, which can be constructed based on multi-agent modeling. In this paper, the structure of the IRRTS is analyzed in detail, while eight types of agents are designed to model all of the entities in an IRRTS, and the artificial IRRTS is constructed considering the interactions among all types of agents. Therefore, the management and control for IRRTS in physical world can be tested and computed through the artificial one, and the systems' operation efficiency will be improved.

Nowadays, the problem of Internet services accessibility has become a hot topic with more and more cyber attacks and censorship. This has prompted the rapid development of jamming-resistance infrastructure consisting of multiple dynamic access points such as proxies, anonymous communication nodes and covert communication nodes. However, the channel between user and access point has become an emerging attacking target for the adversary. Once the channel is detected and identified by the adversary, the channel will be interrupted. Though users can require new access points from the infrastructure and resume the communication to their destination, the service quality will be downgraded dramatically due to the time-consuming bootstrapping process. In this paper, a moving target framework is proposed to improve the network service accessibility, which combines both time-consuming bootstrapping and frequent and low-cost channel refreshing operations. The constant channel refreshing operations is imported to limit the adversary's ability of detecting and blocking communications, thus to make the lifespan of effective communication longer. With theoretical and simulating analysis, the optimized refreshing strategy is proposed, which can help improve service accessibility.

Text corpus and link network are interrelated data in social networks. Discovering the inner relationship between these two kinds of data can help better understand the evolution mechanism underneath social networks. Moreover, social networks exhibit unique characteristics such as sparse and noisy in both text and link data. Thus, it is imperative to combine both text and link data to complement and correct mining results. However, previous work did not explore a uniform generative model that can unveil their inner relationship probably because of the difficulty to harness the heterogenous data in social networks. To address this issue, in this paper we present a generative model Topic Block that clearly pinpoints the latent concept underlying the text corpus and link network, i.e., User inner interests. In our generative model, user inner interests guide the generation of the topic and community distributions underlying the text corpus and link data. We can infer the topic and community distributions based on the user inner interests through both content and topology information. Compared to existing popular models, our method experimentally outperforms on three real world social network data sets.

As one of the most popular anonymous communication systems, Tor has become a research hotspot in this area. Recently, Tor nodes from Tor families (referred to as family nodes) have played an increasingly important role and caused significant influence on the Tor network. However, existing research about Tor mostly focuses on the entire Tor network without much consideration about the difference between family nodes and the others. In order to analyze family nodes' contribution to the entire Tor network as well as their influence, this paper distinguishes family nodes from the others, and gives an empirical analysis of family nodes based on the live Tor network data of 3 years. Results show that, family nodes compose a small but full functional subset of Tor nodes; and compared with the other Tor nodes, they can provide relatively stable and high-performance service to Tor users. Furthermore, family nodes naturally form a hot area in the Tor network, relaying increasingly high-density traffic through a small number of nodes. Compared with random node targets, selective attacks focusing on family nodes can cause serious availability downgrade of the Tor network with much lower cost.

Tor is a worldwide publicly deployed low-latency anonymity system. In order to prevent observers from telling where the data came from and where it's going, data packets on the Tor network take a pathway through several intermediate relays. However, nodes selection to build such a pathway is oblivious to Internet routing, so anonymity guarantees can break down in cases where an attacker can correlate traffic across the entry- and exit-segments of a Tor circuit. Although many works have been done to avoid this kind of collusion attack, recent researches [18] indicated that some Internet exchanges (IXes) locating at the entry- and exit-transmission paths in Tor network (that are the paths from the client to the chosen entry node and from the chosen exit node to the destination) are still possible to perform a correlation attack. However, few works have been done to suggest and verify modifications to Tor's path selection algorithm that would help clients avoid an IX-level observer. In this paper, we first, based on the entry-exit pairs chosen by Tor's path selection algorithm, demonstrated that the probability of a single IX observing both ends of an anonymous Tor connection is greater than previously thought. And then, we proposed and evaluated the effectiveness of a simple IX-awareness path selection algorithm that help to resist IX-level attackers.

With the worldwide increasing of Internet censorship, censorship-resistance technology has attracted more and more attentions, some famous systems, such as Tor and JAP, have been deployed to provide public service for censorship-resistance. However, these systems all rely on dedicated infrastructure and entry points for service accessibility. The network infrastructure and entry points may become the target of censorship attack. In this paper, a UGC-based method is proposed (called user-generated content based covert communication, UGC3) for covert communication in a friends-to-friends (F2F) manner. It uses existing infrastructures (i.e., UGC sites ) to form a fully distributed overlay network. An efficient resource discovery algorithm is proposed to negotiate the rendezvous point. Analysis shows that this method is able to circumvent internet censorship with user repudiation and fault tolerance.