The entropic-regularized optimal transport (OT) has gained massive attention in machine learning due to its ability to provide scalable solutions for OT-based tasks. However, most of the existing algorithms, including the Sinkhorn algorithm and its extensions, suffer from relatively slow convergence in many cases. More recently, some second-order methods have been proposed based on the idea of Hessian sparsification. Despite their promising results, they have two major issues: first, there is limited theoretical understanding on the effect of sparsification; second, in cases where the transport plan is dense, Hessian sparsification does not perform well. In this paper, we propose a new quasi-Newton method to address these problems. First, we develop new theoretical analyses to understand the benefits of Hessian sparsification, which lays the foundation for highly flexible sparsification schemes. Then we introduce an additional low-rank term in the approximate Hessian to better handle the dense case. Finally, the convergence properties of the proposed algorithm are rigorously analyzed, and various numerical experiments are conducted to demonstrate its improved performance in solving large-scale OT problems.

The rapid development of LLMs has raised concerns about their potential misuse, leading to various watermarking schemes that typically offer high detectability. However, existing watermarking techniques often face trade-off between watermark detectability and generated text quality. In this paper, we introduce Learning to Watermark (LTW), a novel selective watermarking framework that leverages multi-objective optimization to effectively balance these competing goals. LTW features a lightweight network that adaptively decides when to apply the watermark by analyzing sentence embeddings, token entropy, and current watermarking ratio. Training of the network involves two specifically constructed loss functions that guide the model toward Pareto-optimal solutions, thereby harmonizing watermark detectability and text quality. By integrating LTW with two baseline watermarking methods, our experimental evaluations demonstrate that LTW significantly enhances text quality without compromising detectability. Our selective watermarking approach offers a new perspective for designing watermarks for LLMs and a way to preserve high text quality for watermarks. The code is publicly available at: https://github.com/fattyray/learning-to-watermark

Abstract Content‐based image retrieval (CBIR) is the problem of searching for items in an image database that are similar to the query image. Most of the existing image retrieval methods are trained based on metric learning loss functions (e.g. contrastive loss or triplet loss), however, which require the use of hard sample mining strategies (HMS) to better train the model. The HMS implies that picking out hard positive or negative samples increases the complexity of model training and requires a large amount of additional training time. To address this issue, lessons from recent work are leveraged on representation learning and a model called GS is proposed that combines the state‐of‐the‐art Generalized‐Mean (GeM) pooling and the smoothed average precision (AP). The entire network can be learned end‐to‐end by approximating the non‐differentiable AP function to a differentiable one‐without mining hard samples, only image‐level annotations. A model named GSA is also presented which achieves excellent retrieval performance jointly trained by two various loss functions. Experimental results validate the effectiveness of the proposed approach and demonstrate the competitive performance on a common standard image retrieval dataset (Revisited Oxford and Paris).