The capability of the reward model (RM) is crucial for the success of Reinforcement Learning from Human Feedback (RLHF) in aligning with human preferences. However, as training progresses, the output space distribution of the policy model shifts. The RM, initially trained on responses sampled from the output distribution of the early policy model, gradually loses its ability to distinguish between responses from the newly shifted distribution. This issue is further compounded when the RM, trained on a specific data distribution, struggles to generalize to examples outside of that distribution. These two issues can be united as a challenge posed by the shifted distribution of the environment. To surmount this challenge, we introduce MetaRM, a novel method leveraging meta-learning to adapt the RM to the shifted environment distribution. MetaRM optimizes the RM in an alternating way, by preserving both the preferences of the original preference pairs, as well as maximizing discrimination power over new examples of the shifted distribution. Extensive experiments demonstrate that MetaRM can iteratively enhance the performance of human preference alignment by improving the RM's capacity to identify subtle differences in samples of shifted distributions.

The recent surge in jailbreaking attacks has revealed significant vulnerabilities in Large Language Models (LLMs) when exposed to malicious inputs. While various defense strategies have been proposed to mitigate these threats, there has been limited research into the underlying mechanisms that make LLMs vulnerable to such attacks. In this study, we suggest that the self-safeguarding capability of LLMs is linked to specific activity patterns within their representation space. Although these patterns have little impact on the semantic content of the generated text, they play a crucial role in shaping LLM behavior under jailbreaking attacks. Our findings demonstrate that these patterns can be detected with just a few pairs of contrastive queries. Extensive experimentation shows that the robustness of LLMs against jailbreaking can be manipulated by weakening or strengthening these patterns. Further visual analysis provides additional evidence for our conclusions, providing new insights into the jailbreaking phenomenon. These findings highlight the importance of addressing the potential misuse of open-source LLMs within the community.

Name prefix lookup is a core characteristic of name-based packet forwarding in ICN and also URL redirections in CDN. This operation is essentially a longest prefix match on URL-like names, which faces significant challenges in high-speed processing due to large set, length-unbounded names and per-packet lookup need. In this paper, we propose a scheme to support generic hierarchical name lookups on millions of prefixes with fast, constant and guaranteed performance using ternary content-addressable memory (TCAM). Our work has three contributions. First, a scheme, namely PtCAM, is proposed to exploit TCAM for name lookup using a binary Patricia trie. Second, several approaches are proposed to enhance efficiency and scalability. Third, experiments on various name datasets are carried out. The results show that our scheme has the potential to achieve high-speed lookup performance nearly equal to the given TCAM throughput. Notably, PtCAM can be implemented on TCAM-based IP linecards, which means that industrial commercial off-the-shelf devices can be directly involved in the advancement of information-centric innovation and deployment.

Wenhao Liu, Xiaohua Wang, Muling Wu, Tianlong Li, Changze Lv, Zixuan Ling, Zhu JianHao, Cenyuan Zhang, Xiaoqing Zheng, Xuanjing Huang. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2024.

Muling Wu, Wenhao Liu, Xiaohua Wang, Tianlong Li, Changze Lv, Zixuan Ling, Zhu JianHao, Cenyuan Zhang, Xiaoqing Zheng, Xuanjing Huang. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2024.

Named Data Networking (NDN) presents strong bandwidth benefits on edge networks because its in-network caching design is probably fit for popularity and locality. However, separate NDN edge regions cannot be efficiently connected over IP backbone world, slowing down the deployment of NDN edges. Due to the difference of stateful NDN and stateless IP in forwarding principles, the coexistence of two protocols is difficult and challenging in the network layer. In this paper, we propose iGate, an NDN gateway design for tunneling over IP world, to bridge two kinds of protocols in the same layer. iGate translates protocols between stateful NDN and stateless IP by two temporary tables. It considers many factors about gateway efficiency, like table tunning, minimal NDN integration, TCP/UDP alternatives. We evaluate the performance of iGate with a real implementation on Linux and the results show that it can achieve more than 85.03% translation efficiency and less than 29% protocol conversion delay compared with IP VPN which uses the typical tunneling technology over IP network. Our work presents a practical and highly-efficient solution to assemble NDN edges into IP world, and can be applied to other similar Information-Centric Networking (ICN) designs.

Large-scale search engines need to answer thousands of queries per second over billions of documents, which is typically done by querying a large inverted index. Many highly optimized integer encoding techniques are applied to compress the inverted index and reduce the query processing time. In this paper, we propose a new grammar-based inverted index compression scheme, which can improve the performance of both index compression and query processing.