Recent text-to-image diffusion models leverage cross-attention layers, which have been effectively utilized to enhance a range of visual generative tasks. However, our understanding of cross-attention layers remains somewhat limited. In this study, we introduce a mechanistic interpretability approach for diffusion models by constructing Head Relevance Vectors (HRVs) that align with human-specified visual concepts. An HRV for a given visual concept has a length equal to the total number of cross-attention heads, with each element representing the importance of the corresponding head for the given visual concept. To validate HRVs as interpretable features, we develop an ordered weakening analysis that demonstrates their effectiveness. Furthermore, we propose concept strengthening and concept adjusting methods and apply them to enhance three visual generative tasks. Our results show that HRVs can reduce misinterpretations of polysemous words in image generation, successfully modify five challenging attributes in image editing, and mitigate catastrophic neglect in multi-concept generation. Overall, our work provides an advancement in understanding cross-attention layers and introduces new approaches for fine-controlling these layers at the head level.

This paper presents a fully integrated dual-output switched-capacitor DC-DC converter with dynamic four-phase operation. The presented converter is derived from an existing topology known as the algorithmic voltage-feed-in and it maintains the voltage conversion ratio (VCR) reconfiguration capability of the original topology. By using a selective pair of VCRs, the presented converter generates two output voltages with an arbitrary ratio with high efficiency. Additionally, efficiency is maintained for various combinations of load currents via dynamic phase skipping and frequency modulation. A test chip fabricated using a 0.18 pm CMOS process exhibits conversion efficiencies higher than 89% for all VCR combinations with a 0.5 mA load current.

BERT-based Neural Ranking Models (NRMs) can be classified according to how the query and document are encoded through BERT's self-attention layers - bi-encoder versus cross-encoder. Bi-encoder models are highly efficient because all the documents can be pre-processed before the query time, but their performance is inferior compared to cross-encoder models. Both models utilize a ranker that receives BERT representations as the input and generates a relevance score as the output. In this work, we propose a method where multi-teacher distillation is applied to a cross-encoder NRM and a bi-encoder NRM to produce a bi-encoder NRM with two rankers. The resulting student bi-encoder achieves an improved performance by simultaneously learning from a cross-encoder teacher and a bi-encoder teacher and also by combining relevance scores from the two rankers. We call this method TRMD (Two Rankers and Multi-teacher Distillation). In the experiments, TwinBERT and ColBERT are considered as baseline bi-encoders. When monoBERT is used as the cross-encoder teacher, together with either TwinBERT or ColBERT as the bi-encoder teacher, TRMD produces a student bi-encoder that performs better than the corresponding baseline bi-encoder. For [email protected], the maximum improvement was 11.4%, and the average improvement was 6.8%. As an additional experiment, we considered producing cross-encoder students with TRMD, and found that it could also improve the cross-encoders.

In this paper a framework for off-line handwritten numeral string recognition based on stroke grouping is proposed. In our approach, strokes are aligned into a sequence of strokes and then a segmentation process is performed to partition strokes in the sequence into possible-digits, that is, groups of strokes which may be a digit. As a result of stroke grouping, grouping-hypotheses, which imply possible segmentation, are generated. An input numeral string is recognized by a dynamic programming scheme, in which the best grouping-hypothesis with maximum matching score is chosen. The framework also provides a systematic way of reducing computational complexity by embedding external knowledge into the framework. The experimental results to evaluate the proposed framework are shown.