Abstract Trajectory data offers significant potential for personalized services and behavioral analysis, but also raises substantial privacy concerns. However, centralized privacy protection strategies are susceptible to single-point failures, often fail to accommodate user-specific privacy needs, and can lack precision in local privacy protection. To address these concerns, this work utilizes the decentralized philosophy of Web 3.0. A novel strategy is proposed, the Blackhole Model-based Distributed Trajectory Privacy-Preserving Strategy (BlackHole-DTP). First, the Distributed Deep Learning Trajectory Training Algorithm with Multi-head Attention and Variational Adversarial Autoencoders is introduced to improve the simulation of temporal and semantic information in trajectory data, thereby enhancing data utility and accuracy. Additionally, a Local Differential Privacy Algorithm based on the Data Blackhole is designed that dynamically adjusts privacy protection levels according to user requirements. This algorithm incorporates principles inspired by general relativity to determine perturbation values. Finally, experimental results demonstrate that BlackHole-DTP provides superior privacy protection, achieving significantly lower success rates in reconstruction and re-identification attacks compared to baseline models. Specifically, under high-precision requirements, the attack success rate is reduced by $\sim $30%.

Pre-trained vision-language models are able to interpret visual concepts and language semantics. Prompt learning, a method of constructing prompts for text encoders or image encoders, elicits the potentials of pre-trained models and readily adapts them to new scenarios. Compared to fine-tuning, prompt learning enables the model to achieve comparable or better performance using fewer trainable parameters. Besides, prompt learning freezes the pre-trained model and avoids the catastrophic forgetting issue in the fine-tuning. Continuous prompts inserted into the input of every transformer layer (i.e. deep prompts) can improve the performances of pre-trained models on downstream tasks. For i-th transformer layer, the inserted prompts replace previously inserted prompts in the (i − 1)-th layer. Although the self-attention mechanism contextualizes newly inserted prompts for the current layer and embeddings from the previous layer’s output, removing all inserted prompts from the previous layer inevitably loses information contained in the continuous prompts. In this work, we propose Modular Prompt Learning (MPL) that is designed to promote the preservation of information contained in the inserted prompts. We evaluate the proposed method on base-to-new generalization and cross-dataset tasks. On average of 11 datasets, our method achieves 0.7% performance gain on the base-to-new generalization task compared to the state-of-the-art method. The largest improvement on the individual dataset is 10.7% (EuroSAT dataset). Our code is available at https://github.com/Zhenhan-Huang/Modular-Prompt-Learning.

Enterprises have a growing need to identify relevant tables in data lakes; e.g. tables that are unionable, joinable, or subsets of each other. Tabular neural models can be help-ful for such data discovery tasks. In this paper, we present TabSketchFM, a neural tabular model for data discovery over data lakes. First, we propose novel pre-training: a sketch-based approach to enhance the effectiveness of data discovery in neural tabular models. Second, we finetune the pretrained model for identifying unionable, joinable, and subset table pairs and show significant improvement over previous tabular neural models. Third, we present a detailed ablation study to highlight which sketches are crucial for which tasks. Fourth, we use these finetuned models to perform table search; i.e., given a query table, find other tables in a corpus that are unionable, joinable, or that are subsets of the query. Our results demonstrate significant improvements in F1 scores for search compared to state-of-the-art techniques. Finally, we show significant transfer across datasets and tasks establishing that our model can generalize across different tasks and over different data lakes.

Prompt learning is an effective way to exploit the potential of large-scale pre-trained foundational models. Continuous prompts parameterize context tokens in prompts by turning them into differentiable vectors. Deep continuous prompts insert prompts not only in the input but also in the intermediate hidden representations. Manually designed deep continuous prompts exhibit a remarkable improvement compared to the zero-shot pre-trained model on downstream tasks. How to automate the continuous prompt design is an underexplored area, and a fundamental question arises, is manually designed deep prompt strategy optimal? To answer this question, we propose a method dubbed differentiable prompt learning (DPL). The DPL method is formulated as an optimization problem to automatically determine the optimal context length of the prompt to be added to each layer, where the objective is to maximize the performance. We test the DPL method on the pre-trained CLIP. We empirically find that by using only limited data, our DPL method can find deep continuous prompt configuration with high confidence. The performance on the downstream tasks exhibits the superiority of the automatic design: our method boosts the average test accuracy by 2.60% on 11 datasets compared to baseline methods. Besides, our method focuses only on the prompt configuration (i.e. context length for each layer), which means that our method is compatible with the baseline methods that have sophisticated designs to boost the performance. We release our code in https://github.com/Zhenhan-Huang/Differentiable-Prompt-Learn.

While Swarm User Interfaces (SUIs) have succeeded in enriching tangible interaction experiences, their limitations in autonomous action planning have hindered the potential for personalized and dynamic interaction generation in tabletop games. Based on the AI-Gadget Kit we developed, this paper explores how to integrate LLM-driven agents to enable SUIs to execute interaction tasks within tabletop games. After defining the design space of this kit, we elucidate the method for designing agents that can extend the meta-actions of SUIs to motion planning. Furthermore, we introduce an add-on prompt method that simplifies the design process for four interaction relationships in tabletop games. Lastly, we present an example that illustrates the potential of AI-Gadget Kit to construct personalized complex interactions in SUI tabletop games.