We propose new static word embeddings optimised for sentence semantic representation.We first extract word embeddings from a pretrained Sentence Transformer, and improve them with sentence-level principal component analysis, followed by either knowledge distillation or contrastive learning.During inference, we represent sentences by simply averaging word embeddings, which requires little computational cost.We evaluate models on both monolingual and cross-lingual tasks and show that our model substantially outperforms existing static models on sentence semantic tasks, and even surpasses a basic Sentence Transformer model (SimCSE) on a text embedding benchmark.Lastly, we perform a variety of analyses and show that our method successfully removes word embedding components that are not highly relevant to sentence semantics, and adjusts the vector norms based on the influence of words on sentence semantics.

This paper describes an experimental switching system employing a nonprocedural language. It presents a study of the appropriate architecture for implementing the language, and examines the system's performance characteristics. Furthermore, it identifies the critical part of the architecture, and provides two strategies for throughput improvement and indicates their effects.