Watermarking approaches are proposed to identify if text being circulated is human-or large language model-(LLM) generated.The stateof-the-art watermarking strategy of Kirchenbauer et al. (2023a) biases the LLM to generate specific ("green") tokens.However, determining the robustness of this watermarking method under finite (low) edit budgets is an open problem.Additionally, existing attack methods fail to evade detection for longer text segments.We overcome these limitations, and propose Self Color Testing-based Substitution (SCTS), the first "color-aware" attack.SCTS obtains color information by strategically prompting the watermarked LLM and comparing output tokens frequencies.It uses this information to determine token colors, and substitutes green tokens with non-green ones.In our experiments, SCTS successfully evades watermark detection using fewer number of edits than related work.Additionally, we show both theoretically and empirically that SCTS can remove the watermark for arbitrarily long watermarked text.

Linear scalarization, i.e., combining all loss functions by a weighted sum, has been the default choice in the literature of multi-task learning (MTL) since its inception. In recent years, there is a surge of interest in developing Specialized Multi-Task Optimizers (SMTOs) that treat MTL as a multi-objective optimization problem. However, it remains open whether there is a fundamental advantage of SMTOs over scalarization. In fact, heated debates exist in the community comparing these two types of algorithms, mostly from an empirical perspective. To approach the above question, in this paper, we revisit scalarization from a theoretical perspective. We focus on linear MTL models and study whether scalarization is capable of fully exploring the Pareto front. Our findings reveal that, in contrast to recent works that claimed empirical advantages of scalarization, scalarization is inherently incapable of full exploration, especially for those Pareto optimal solutions that strike the balanced trade-offs between multiple tasks. More concretely, when the model is under-parametrized, we reveal a multi-surface structure of the feasible region and identify necessary and sufficient conditions for full exploration. This leads to the conclusion that scalarization is in general incapable of tracing out the Pareto front. Our theoretical results partially answer the open questions in Xin et al. (2021), and provide a more intuitive explanation on why scalarization fails beyond non-convexity. We additionally perform experiments on a real-world dataset using both scalarization and state-of-the-art SMTOs. The experimental results not only corroborate our theoretical findings, but also unveil the potential of SMTOs in finding balanced solutions, which cannot be achieved by scalarization.