Recent advancements in text-to-image models, such as Stable Diffusion, show significant demographic biases. Existing de-biasing techniques rely heavily on additional training, which imposes high computational costs and risks of compromising core image generation functionality. This hinders them from being widely adopted to real-world applications. In this paper, we explore Stable Diffusion’s overlooked potential to reduce bias without requiring additional training. Through our analysis, we uncover that initial noises associated with minority attributes form "minority regions" rather than scattered. We view these "minority regions" as opportunities in SD to reduce bias. To unlock the potential, we propose a novel de-biasing method called ‘weak guidance,’ carefully designed to guide a random noise to the minority regions without compromising semantic integrity. Through analysis and experiments on various versions of SD, we demonstrate that our proposed approach effectively reduces bias without additional training, achieving both efficiency and preservation of core image generation functionality.

Recent advances in video generation have enabled high-fidelity video synthesis from user provided prompts. However, existing models and benchmarks fail to capture the complexity and requirements of professional video generation. Towards that goal, we introduce Stable Cinemetrics, a structured evaluation framework that formalizes filmmaking controls into four disentangled, hierarchical taxonomies: Setup, Event, Lighting, and Camera. Together, these taxonomies define 76 fine-grained control nodes grounded in industry practices. Using these taxonomies, we construct a benchmark of prompts aligned with professional use cases and develop an automated pipeline for prompt categorization and question generation, enabling independent evaluation of each control dimension. We conduct a large-scale human study spanning 10+ models and 20K videos, annotated by a pool of 80+ film professionals. Our analysis, both coarse and fine-grained reveal that even the strongest current models exhibit significant gaps, particularly in Events and Camera-related controls. To enable scalable evaluation, we train an automatic evaluator, a vision-language model aligned with expert annotations that outperforms existing zero-shot baselines. SCINE is the first approach to situate professional video generation within the landscape of video generative models, introducing taxonomies centered around cinematic controls and supporting them with structured evaluation pipelines and detailed analyses to guide future research.

Diffusion models create data from noise by inverting the forward paths of data towards noise and have emerged as a powerful generative modeling technique for high-dimensional, perceptual data such as images and videos. Rectified flow is a recent generative model formulation that connects data and noise in a straight line. Despite its better theoretical properties and conceptual simplicity, it is not yet decisively established as standard practice. In this work, we improve existing noise sampling techniques for training rectified flow models by biasing them towards perceptually relevant scales. Through a large-scale study, we demonstrate the superior performance of this approach compared to established diffusion formulations for high-resolution text-to-image synthesis. Additionally, we present a novel transformer-based architecture for text-to-image generation that uses separate weights for the two modalities and enables a bidirectional flow of information between image and text tokens, improving text comprehension, typography, and human preference ratings. We demonstrate that this architecture follows predictable scaling trends and correlates lower validation loss to improved text-to-image synthesis as measured by various metrics and human evaluations. Our largest models outperform state-of-the-art models. Stability AI is considering making experimental data, code, and model weights publicly available.

Multimodal datasets are a critical component in recent breakthroughs such as CLIP, Stable Diffusion and GPT-4, yet their design does not receive the same research attention as model architectures or training algorithms. To address this shortcoming in the machine learning ecosystem, we introduce DataComp, a testbed for dataset experiments centered around a new candidate pool of 12.8 billion image-text pairs from Common Crawl. Participants in our benchmark design new filtering techniques or curate new data sources and then evaluate their new dataset by running our standardized CLIP training code and testing the resulting model on 38 downstream test sets. Our benchmark consists of multiple compute scales spanning four orders of magnitude, which enables the study of scaling trends and makes the benchmark accessible to researchers with varying resources. Our baseline experiments show that the DataComp workflow leads to better training sets. Our best baseline, DataComp-1B, enables training a CLIP ViT-L/14 from scratch to 79.2% zero-shot accuracy on ImageNet, outperforming OpenAI's CLIP ViT-L/14 by 3.7 percentage points while using the same training procedure and compute. We release \datanet and all accompanying code at www.datacomp.ai.

Today’s deep neural networks require substantial computation resources for their training, storage, and inference, which limits their effective use on resource-constrained devices. Many recent research activities explore different options for compressing and optimizing deep models for the Internet of Things (IoT). Our work so far has focused on two important aspects of deep neural network compression: class-dependent model compression and explainable compression. We shortly summarize our contributions and conclude with an outline of our future research directions.

Network control in microgrids is an active research area driven by a steady increase in energy demand, the necessity to minimize the environmental footprint, yet achieve socioeconomic benefits and ensure sustainability. Reducing deviation of the predicted energy consumption from the actual one, softening peaks in demand and filling in the troughs, especially at times when power is more affordable and clean, present challenges for the demand-side response. In this paper, we present a hierarchical energy system architecture with embedded control. This architecture pushes prediction models to edge devices and executes local control loops to address the challenge of managing demand-side response locally. We employ a two-step approach: At an upper level of hierarchy, we adopt a conventional machine learning pipeline to build load prediction models using automated domain-specific feature extraction and selection. Given historical data, these models are then used to label prediction failure events that force the operator to use backup energy sources to stabilize the network. On a lower level of hierarchy, computed labels are used to train impact models realized by LSTM networks running on edge devices to infer the probability that the power consumption of the player contributes to the upper level prediction failure event. The system is evaluated on clustered and aggregated energy traces from a public data set of academic buildings. The results show the benefits of the proposed hierarchical energy system architecture in terms of impact prediction with 55% accuracy. This allows minimizing the number of prediction failure events by 11.69 % by executing targeted local control.

Waveform design plays an important role in compressed sensing radar (CSR). The performance of the target scene recovery in CSR scenarios depends on the mutual coherence of the sensing matrix, which is determined by the radar transmission waveform. In this study, the authors propose a waveform optimisation method for pulse‐train scenario through solving the matrix nearness problem between the Gram matrix and the identity matrix. This method has a simple closed‐form solution and simulation results show that the proposed method is more efficient than the current waveforms such as linear frequency modulation (LFM), Gaussian and Alltop sequences in terms of mutual coherence, recovery errors and recovery percentages.