Abstract As a ubiquitous method in natural language processing, word embeddings are extensively employed to map semantic properties of words into a dense vector representation. They capture semantic and syntactic relations among words, but the vectors corresponding to the words are only meaningful relative to each other. Neither the vector nor its dimensions have any absolute, interpretable meaning. We introduce an additive modification to the objective function of the embedding learning algorithm that encourages the embedding vectors of words that are semantically related to a predefined concept to take larger values along a specified dimension, while leaving the original semantic learning mechanism mostly unaffected. In other words, we align words that are already determined to be related, along predefined concepts. Therefore, we impart interpretability to the word embedding by assigning meaning to its vector dimensions. The predefined concepts are derived from an external lexical resource, which in this paper is chosen as Roget’s Thesaurus. We observe that alignment along the chosen concepts is not limited to words in the thesaurus and extends to other related words as well. We quantify the extent of interpretability and assignment of meaning from our experimental results. Manual human evaluation results have also been presented to further verify that the proposed method increases interpretability. We also demonstrate the preservation of semantic coherence of the resulting vector space using word-analogy/word-similarity tests and a downstream task. These tests show that the interpretability-imparted word embeddings that are obtained by the proposed framework do not sacrifice performances in common benchmark tests.

We consider the problem of representing a finite energy signal with a finite number of samples. When the signal is interpolated via sinc function from the samples, there will be a certain reconstruction error since only a finite number of samples are used. Without making any additional assumptions, we derive a lower bound for this error. This error bound depends on the number of samples but nothing else, and is thus represented as a universal curve of error versus number of samples. Furthermore, the existence of a function that achieves the bound shows that this is the tightest such bound possible.

We consider the transmission of a Gaussian vector source over a multidimensional Gaussian channel where a random or a fixed subset of the channel outputs are erased. Within the setup where the only encoding operation allowed is a linear unitary transformation on the source, we investigate the minimum mean-square error (MMSE) performance, both in average, and also in terms of guarantees that hold with high probability as a function of the system parameters. Under the performance criterion of average MMSE, necessary conditions that should be satisfied by the optimal unitary encoders are established and explicit solutions for a class of settings are presented. For random sampling of signals that have a low number of degrees of freedom, we present MMSE bounds that hold with high probability. Our results illustrate how the spread of the eigenvalue distribution and the unitary transformation contribute to these performance guarantees. The performance of the discrete Fourier transform (DFT) is also investigated. As a benchmark, we investigate the equidistant sampling of circularly wide-sense stationary signals, and present the explicit error expression that quantifies the effects of the sampling rate and the eigenvalue distribution of the covariance matrix of the signal. These findings may be useful in understanding the geometric dependence of signal uncertainty in a stochastic process. In particular, unlike information theoretic measures such as entropy, we highlight the basis dependence of uncertainty in a signal with another perspective. The unitary encoding space restriction exhibits the most and least favorable signal bases for estimation.

In this paper, we study the effect of limited amplitude resolution (pixel depth) in super-resolution problem. The problem we address differs from the standard super-resolution problem in that amplitude resolution is considered as important as spatial resolution. We study the trade-off between the pixel depth and spatial resolution of low resolution (LR) images in order to obtain the best visual quality in the reconstructed high resolution (HR) image. The proposed framework reveals great flexibility in terms of pixel depth and number of LR images in super-resolution problem, and demonstrates that it is possible to obtain target visual qualities with different measurement scenarios including images with different amplitude and spatial resolutions.

Linear canonical transforms (LCTs) are a family of integral transforms with wide application in optical, acoustical, electromagnetic, and other wave propagation problems. The Fourier and fractional Fourier transforms are special cases of LCTs. We present the exact relation between continuous and discrete LCTs (which generalizes the corresponding relation for Fourier transforms), and also express it in terms of a new definition of the discrete LCT (DLCT), which is independent of the sampling interval. This provides the foundation for approximately computing the samples of the LCT of a continuous signal with the DLCT. The DLCT in this letter is analogous to the DFT and approximates the continuous LCT in the same sense that the DFT approximates the continuous Fourier transform. We also define the bicanonical width product which is a generalization of the time-bandwidth product.

We give a precise mathematical formulation of some measurement problems arising in optics, which is also applicable in a wide variety of other contexts. In essence the measurement problem is an estimation problem in which data collected by a number of noisy measurement probes are combined to reconstruct an unknown realization of a random process f(x) indexed by a spatial variable x epsi Rkfor some k ges 1. We wish to optimally choose and position the probes given the statistical characterization of the process f(x) and of the measurement noise processes. We use a model in which we define a cost function for measurement probes depending on their resolving power. The estimation problem is then set up as an optimization problem in which we wish to minimize the mean-square estimation error summed over the entire domain of f subject to a total cost constraint for the probes. The decision variables are the number of probes, their positions and qualities. We are unable to offer a solution to this problem in such generality; however, for the metrical problem in which the number and locations of the probes are fixed, we give complete solutions for some special cases and an efficient numerical algorithm for computing the best trade-off between measurement cost and mean-square estimation error. A novel aspect of our formulation is its close connection with information theory; as we argue in the paper, the mutual information function is the natural cost function for a measurement device. The use of information as a cost measure for noisy measurements opens up several direct analogies between the measurement problem and classical problems of information theory, which are pointed out in the paper.

Diffraction and holography are fertile areas for application of signal theory and processing. Recent work on 3DTV displays has posed particularly challenging signal processing problems. Various procedures to compute Rayleigh-Sommerfeld, Fresnel and Fraunhofer diffraction exist in the literature. Diffraction between parallel planes and tilted planes can be efficiently computed. Discretization and quantization of diffraction fields yield interesting theoretical and practical results, and allow efficient schemes compared to commonly used Nyquist sampling. The literature on computer-generated holography provides a good resource for holographic 3DTV related issues. Fast algorithms to compute Fourier, Walsh-Hadamard, fractional Fourier, linear canonical, Fresnel, and wavelet transforms, as well as optimization-based techniques such as best orthogonal basis, matching pursuit, basis pursuit etc., are especially relevant signal processing techniques for wave propagation, diffraction, holography, and related problems. Atomic decompositions, multiresolution techniques, Gabor functions, and Wigner distributions are among the signal processing techniques which have or may be applied to problems in optics. Research aimed at solving such problems at the intersection of wave optics and signal processing promises not only to facilitate the development of 3DTV systems, but also to contribute to fundamental advances in optics and signal processing theory.

Two important signal processing problems in the display side of a holographic 3DTV are the computation of the diffraction field of a 3D object from its abstract representation, and determination of the best display configuration to synthesize some intended light distribution. To solve the former problem, we worked on the computation of 1D diffraction patterns from discrete data distributed over 2D space. The problem is solved using matrix pseudo-inversion which dominates the computational complexity. Then, the light field synthesis problem by a deflectable mirror array device (DMAD) is posed as a constrained linear optimization problem. The formulation makes direct application of common optimization algorithms quite easy. The simulations indicate that developed methods are promising.

We introduce the fractional Fourier domain decomposition for continuous and discrete signals and systems. A procedure called pruning, analogous to truncation of the singular-value decomposition, underlies a number of potential applications, among which we discuss the fast implementation of space-variant linear systems.

We propose and consolidate a definition of the discrete fractional Fourier transform which generalizes the discrete Fourier transform (DFT) in the same sense that the continuous fractional Fourier transform (FRT) generalizes the continuous ordinary Fourier Transform. This definition is based on a particular set of eigenvectors of the DFT which constitutes the discrete counterpart of the set of Hermite-Gaussian functions. The fact that this definition satisfies all the desirable properties expected of the discrete FRT, supports our confidence that it will be accepted as the definitive definition of this transform.

It is possible to obtain either exact realizations or useful approximations of linear systems or matrix-vector products arising in many different applications, by synthesizing them in the form of repeated or multi-channel filtering operations in fractional Fourier domains, resulting in much more efficient implementations with acceptable reduction in accuracy. By varying the number and configuration of the filter blocks, which may take the form of arbitrary flow graphs, it is possible to trade off between accuracy and efficiency in the desired manner. The proposed scheme constitutes a systematic way of exploiting the information inherent in the regularity or structure of a given linear system or matrix, even when that structure is not readily apparent.

We show that there is a limit to the total number of bits per second,B, of information that can flow in a simple digital electrical interconnection that is set only by the ratio of the lengthlof the interconnection to the total cross-sectional dimensionAof the interconnect wiring—the “aspect ratio” of the interconnection. This limit is largely independent of the details of the design of the electrical lines. The limit is approximatelyB∼BoA/l2bits/s, withBo∼ 1015(bit/s) for high-performance strip lines and cables, ∼1016for small on-chip lines, and ∼1017–1018for equalized lines. Because the limit is scale-invariant, neither growing nor shrinking the system substantially changes the limit. Exceeding this limit requires techniques such as repeatering, coding, and multilevel modulation. Such a limit will become a problem as machines approach Tb/s information bandwidths. The limit will particularly affect architectures in which one processor must talk reasonably directly with many others. We argue that optical interconnects can solve this problem since they avoid the resistive loss physics that gives this limit.

The extended versions of common Laplace and Fourier transforms are given. This is achieved by defining a new function f/sub e/(p), p/spl epsiv/C related to the function to be transformed f(t), t/spl epsiv/R. Then f/sub e/(p) is transformed by an integral whose path is defined on an inclined line on the complex plane. The slope of the path is the parameter of the extended definitions which reduce to common transforms with zero slope. Inverse transforms of the extended versions are also defined. These proposed definitions, when applied to filtering in complex ordered fractional Fourier stages, significantly reduce the required computation.

We consider the Cohen (1989) class of time-frequency distributions, which can be obtained from the Wigner distribution by convolving it with a kernel characterizing that distribution. We show that the time-frequency distribution of the fractional Fourier transform of a function is a rotated version of the distribution of the original function, if the kernel is rotationally symmetric. Thus, the fractional Fourier transform corresponds to rotation of a relatively large class of time-frequency representations (phase-space representations), confirming the important role this transform plays in the study of such representations.

The ordinary Fourier transform is suited best for analysis and processing of time-invariant signals and systems. When we are dealing with time-varying signals and systems, filtering in fractional Fourier domains might allow us to estimate signals with smaller minimum mean square error (MSE). We derive the optimal fractional Fourier domain filter that minimizes the MSE for given non-stationary signal and noise statistics, and time-varying distortion kernel. We present an example for which the MSE is reduced by a factor of 50 as a result of filtering in the fractional Fourier domain, as compared to filtering in the conventional Fourier or time domains. We also discuss how the fractional Fourier transformation can be computed in O(N log N) time, so that the improvement in performance is achieved with little or no increase in computational complexity.

Wavelet kernels in the form of complex exponentials with linear instantaneous frequencies are discussed. These wavelets do not satisfy the commonly used admissibility condition; furthermore, they are neither band-, nor space-limited functions and they are not absolutely integrable. It is shown, however, that the wavelet transforms using these kernels are still invertable and the inversion formulas are given. Such kernels are closely related to wave propagation related phenomena like the diffraction and the holography, and therefore, the obtained mathematical results coincide to well known optical applications. The kernels may be useful in space-depth analysis of data obtained from diffraction related imaging techniques.>