In this paper we introduce SimTDE, a simple knowledge distillation framework to compress sentence embeddings transformer models with minimal performance loss and significant size and latency reduction. SimTDE effectively distills large and small transformers via a compact token embedding block and a shallow encoding block, connected with a projection layer, relaxing dimension match requirement. SimTDE simplifies distillation loss to focus only on token embedding and sentence embedding. We evaluate on standard semantic textual similarity (STS) tasks and entity resolution (ER) tasks. It achieves 99.94% of the state-of-the-art (SOTA) SimCSE-Bert-Base performance with 3 times size reduction and 96.99% SOTA performance with 12 times size reduction on STS tasks. It also achieves 99.57% of teacher's performance on multi-lingual ER data with a tiny transformer student model of 1.4M parameters and 5.7MB size. Moreover, compared to other distilled transformers SimTDE is 2 times faster at inference given similar size and still 1.17 times faster than a model 33% smaller (e.g. MiniLM). The easy-to-adopt framework, strong accuracy and low latency of SimTDE can widely enable runtime deployment of SOTA sentence embeddings.

Distribution forecast can quantify forecast uncertainty and provide various forecast scenarios with their corresponding estimated probabilities. Accurate distribution forecast is crucial for planning – for example when making production capacity or inventory allocation decisions. We propose a practical and robust distribution forecast framework that relies on backtest-based bootstrap and adaptive residual selection. The proposed approach is robust to the choice of the underlying forecasting model, accounts for uncertainty around the input covariates, and relaxes the independence between residuals and covariates assumption. It reduces the Absolute Coverage Error by more than 63% compared to the classic bootstrap approaches and by 2%−32% compared to a variety of State-of-the-Art deep learning approaches on in-house product sales data and M4-hourly competition data.

We initiate the study of the Stam region, defined as the subset of the positive orthant in ℝ(2n-1) that arises from considering the entropy powers of subset sums of n independent random vectors in a Euclidean space of finite dimension. We show that the class of fractionally superadditive set functions provides an outer bound to the Stam region, resolving a conjecture of Barron and Madiman. On the other hand, the entropy power of a sum of independent random vectors is not supermodular in any dimension. We also develop some qualitative properties of the Stam region, showing for instance that its closure is a logarithmically convex cone.

It is shown that the entropy power of a sum of independent random vectors, seen as a set function, is fractionally superadditive. This resolves a conjecture of the first author and A. R. Barron, and implies in particular all previously known entropy power inequalities for independent random variables. It is also shown that, for general dimension, the entropy power of a sum of independent random vectors is not supermodular.

We discuss the off-line and on-line aspects of trajectory planning in bearings-only localization. Assuming that there are m(ges 1) moveable sensors (e.g. UAVs), which fly in closed trajectories, the aim is to determine the optimal shape of the trajectory. We investigate the properties of closed optimal trajectories in the off-line problem and show that these solutions are invariant under a scaling transformation of the problem parameters. This result is used to numerically derive a set of solutions for the normalized parameters. These solutions are then used in a stochastic search algorithm which randomly explores the trajectories but spends the largest amount of time in the optimal trajectory

An indirect, self-calibrating, easy to install, and robust joint angle sensing method for heavy-duty manipulators is presented in this paper. This method is suitable for the harsh working environment of these machines where conventional contact-type angle sensors cannot be deployed, or problems are associated with their use. The approach is based on processing the outputs of a pair of biaxial accelerometers placed very close to the joint axis on the adjacent links. In the proposed technique, joint angles are obtained without integrating the accelerometer outputs to avoid measurement error accumulation over a long period of time. Two calibration procedures are also described for accelerometers to ensure the accuracy of their measurements. The experimental results attest to the efficiency and accuracy of the new angle sensing mechanism.

We present a system for rendering planar rigid-body motion by means of a redundant parallel mechanism. The device design, the control architecture and the passive virtual environment simulation are presented. The system is used to compare various virtual walls and friction models proposed for haptic applications. In addition, the reset-integrator dry friction model proposed by Haessig and Friedland (1991) is implemented in a haptic interface for the first time.