Learning reward functions remains the bottleneck to equip a robot with a broad repertoire of skills. Large Language Models (LLM) contain valuable task-related knowledge that can potentially aid in the learning of reward functions. However, the proposed reward function can be imprecise, thus ineffective which requires to be further grounded with environment information. We proposed a method to learn rewards more efficiently in the absence of humans. Our approach consists of two components: We first use the LLM to propose features and parameterization of the reward, then update the parameters through an iterative self-alignment process. In particular, the process minimizes the ranking inconsistency between the LLM and the learnt reward functions based on the execution feedback. The method was validated on 9 tasks across 2 simulation environments. It demonstrates a consistent improvement in training efficacy and efficiency, meanwhile consuming significantly fewer GPT tokens compared to the alternative mutation-based method.

We present ClothesNet: a large-scale dataset of 3D clothes objects with information-rich annotations. Our dataset consists of around 4400 models covering 11 categories annotated with clothes features, boundary lines, and keypoints. ClothesNet can be used to facilitate a variety of computer vision and robot interaction tasks. Using our dataset, we establish benchmark tasks for clothes perception, including classification, boundary line segmentation, and keypoint detection, and develop simulated clothes environments for robotic interaction tasks, including rearranging, folding, hanging, and dressing. We also demonstrate the efficacy of our ClothesNet in real-world experiments. Supplemental materials and dataset are available on our project webpage at https://sites.google.com/view/clothesnet.

Model watermarking has become an important solution to protect the intellectual property right (IPR) of deep neural networks (DNN) models. However, there are few researches on the IPR protection of generative adversarial networks (GAN), which are widely used to generate photorealistic images. In the current backdoor-based watermarking method for GAN models, the trigger pattern of the watermark is easy to be detected and invalidated by the adversary, which may fail to achieve IPR protection. To address this drawback, we propose a new GAN model watermarking method, where an invisible backdoor based on steganography is injected into the target GAN model as a watermark. Experimentally, the proposed method effectively trades off the performance of GAN on original task and the robustness of watermarking for removal attacks. Moreover, the generated triggers can effectively resist being detected by attackers.

URL redirection has become an important tool for adversaries to cover up their malicious campaigns. In this paper, we conduct the first large-scale measurement study on how adversaries leverage URL redirection to circumvent security checks and distribute malicious content in practice. To this end, we design an iteratively running framework to mine the domains used for malicious redirections constantly. First, we use a bipartite graph-based method to dig out the domains potentially involved in malicious redirections from real-world DNS traffic. Then, we dynamically crawl these suspicious domains and recover the corresponding redirection chains from the crawler’s performance log. Based on the collected redirection chains, we analyze the working mechanism of various malicious redirections, involving the abused modes and methods, and highlight the pervasiveness of node sharing. Notably, we find a new redirection abuse, redirection fluxing, which is abused to enhance the concealment of malicious sites by introducing randomness into the redirection. Our case studies reveal the adversary’s preference for abusing JavaScript methods to conduct redirection, even by introducing time-delay and fabricating user clicks to simulate normal users.

Malicious domain detection is of great significance for cybersecurity. Most prior works detect malicious domains based on individual features, which are only related to the attributes of domains themselves and can be easily changed to avoid detection. To solve the problem, we propose a novel system called MalPortrait, which combines individual features and association information of domains to detect malicious domains. In MalPortrait, we show the association information among domains by a domain association graph where vertices represent domains and edges connect domains resolved to the same IP. Based on the graph, we combine individual features (e.g., string-based, network-based) of each domain and its association information to generate new features. Compared with individual features, the new features are harder to be tampered with and can help determine whether a domain is malicious from a more comprehensive perspective. We evaluate MalPortrait on the passive DNS traffic collected from real-world large ISP networks. Our experimental results show that MalPortrait can accurately identify malicious domain names with a precision of 96.8% and a recall of 95.5%. Compared with prior works, MalPortrait performs better and hardly relies on additional knowledge (e.g., IP reputation, Domain whois).

Many botmasters use domain generation algorithms (DGA) to generate a host of malicious algorithmically- generated domains (mAGDs) and then choose several mAGDs for actual command and control (C2) communication. The botmasters use different seeds (e.g., timestamp) to generate different mAGDs, which makes the communication mechanism resilient to blacklisting. Thus the botmasters can hide C2 channels very well. If we can quickly detect these mAGDs from DNS traffic, we will effectively block the communication. In this paper, we propose a novel system, called Pontus, to detect mAGDs from DNS traffic. Pontus extract features exclusively from the individual domain names. We compare Pontus with the state-of-the-art system and find that Pontus improves the precision by at least 4.7%.

Domain-squatting abuse refers to the premeditated attempt by an attacker to register perceptively confusing domain names thereby tricking visitors into querying them. There are totally five squatting types have been investigated so far, namely typo-squatting, bit-squatting, homograph-squatting, sound-squatting, and combo-squatting. Existing researches only focus on one specific squatting type and never explore the relationship among them. In this paper, we perform the first comprehensive measurement study of domain-squatting abuse. We select 786 the most queried domains, and hunt for squatting abuses against them in ISP-level DNS traffic. We find that although typo-squatting accounts for most of squatting domains, combo-squatting are able to attract more traffic. Our further case studies show that parking ads is still the most important way for attackers to make profits. The only exception is combo-squatting, in which squatters tend to leverage the reputation of squatted domains to develop their own business. It is worth noting that some squatting domains are even used to deliver malware. Moreover, the Alexa ranks of certain squatting domains have already surpassed the original domains. These results clearly call for the need to better protect the intellectual property of domain names.

More Internet services tend to collect the one-time information from clients via DNS queries. Notably, the uncertainty of such transient information makes these domain names be queried only once in their lifetime. This type of domain is called disposable domain. Although they are not malicious, the efficiency of DNS infrastructures will still be affected by their ever-increasing number. In this paper, we propose Vogers, a linguistics-based stacking model, to detect the disposable domains. Our evaluation demonstrates that Vogers decreases the false positive rate by more than 19%, compared with the prior art, while maintaining the true positive rate above 98.9%.