Dataset distillation (DD) condenses large datasets into compact yet informative substitutes, preserving performance comparable to the original dataset while reducing storage, transmission costs, and computational consumption. However, previous DD methods mainly focus on distilling information from images, often overlooking the semantic information inherent in the data. The disregard for context hinders the model's generalization ability, particularly in tasks involving complex datasets, which may result in illogical outputs or the omission of critical objects. In this study, we integrate vision-language methods into DD by introducing text prototypes to distill language information and collaboratively synthesize data with image prototypes, thereby enhancing dataset distillation performance. Notably, the text prototypes utilized in this study are derived from descriptive text information generated by an open-source large language model. This framework demonstrates broad applicability across datasets without pre-existing text descriptions, expanding the potential of dataset distillation beyond traditional image-based approaches. Compared to other methods, the proposed approach generates logically coherent images containing target objects, achieving state-of-the-art validation performance and demonstrating robust generalization. Source code and generated data are available in https://github.com/zou-yawen/Dataset-Distillation-via-Vision-Language-Category-Prototype/

Multi-task learning (MTL) for dense prediction has shown promising results but still faces challenges in balancing shared representations with task-specific specialization. In this paper, we introduce a novel Fine-Grained Mixture of Experts (FGMoE) architecture that explores MoE-based MTL models through a combination of three key innovations and fine-tuning. First, we propose intra-task experts that partition along intermediate hidden dimensions of MLPs, enabling finer decomposition of task information while maintaining parameter efficiency. Second, we introduce shared experts that consolidate common information across different contexts of the same task, reducing redundancy, and allowing routing experts to focus on unique aspects. Third, we design a global expert that facilitates adaptive knowledge transfer across tasks based on both input feature and task requirements, promoting beneficial information sharing while preventing harmful interference. In addition, we use the fine-tuning approach to improve parameter efficiency only by training the parameters of the decoder. Extensive experimental results show that the proposed FGMoE uses fewer parameters and significantly outperforms current MoE-based competitive MTL models on two dense prediction datasets (i.e., NYUD-v2, PASCAL-Context) in various metrics.

Free-hand sketches serve as efficient tools for creativity and communication, yet expressing ideas clearly through sketches remains challenging for untrained individuals. Optimizing sketches through text guidance can enhance individuals' ability to effectively convey their ideas and improve overall communication efficiency. While recent advancements in Artificial Intelligence Generated Content (AIGC) have been notable, research on optimizing free-hand sketches remains relatively unexplored. In this paper, we introduce SketchRefiner, an innovative method designed to refine rough sketches from various categories into polished versions guided by text prompts. SketchRefiner utilizes a latent diffusion model with ControlNet to guide a differentiable rasterizer in optimizing a set of Bézier curves. We extend the score distillation sampling (SDS) loss and introduce a joint semantic loss to encourage sketches aligned with given text prompts and free-hand sketches. Additionally, we propose a fusion attention-map stroke initialization strategy to improve the quality of refined sketches. Furthermore, SketchRefiner provides users with fine-grained control over text guidance. Through extensive experiments, we demonstrate that our method can generate accurate and aesthetically pleasing refined sketches that closely align with input text prompts and sketches.

Dataset distillation offers a lightweight synthetic dataset for fast network training with promising test accuracy. To imitate the performance of the original dataset, most approaches employ bi-level optimization and the distillation space relies on the matching architecture. Nevertheless, these approaches either suffer significant computational costs on large-scale datasets or experience performance decline on cross-architectures. We advocate for designing an economical dataset distillation framework that is independent of the matching architectures. With empirical observations, we argue that constraining the consistency of the real and synthetic image spaces will enhance the cross-architecture generalization. Motivated by this, we introduce Dataset Distillation via Disentangled Diffusion Model (D4M), an efficient framework for dataset distillation. Compared to architecture-dependent methods, D4M employs latent diffusion model to guarantee consistency and incorporates label information into category prototypes. The distilled datasets are versatile, eliminating the need for repeated generation of distinct datasets for various architectures. Through comprehensive experiments, D4M demonstrates superior performance and robust generalization, surpassing the SOTA methods across most aspects.