This paper studies the multiple-input single-output free-space optical intensity channel with signal-independent additive Gaussian noise and subject to both an average- and a peak-intensity constraint. Closed-form expressions for the asymptotic high-signal-to-noise-ratio (high-SNR) capacity and for the corresponding capacity-achieving input distribution are presented. Moreover, several practical modulation schemes are proposed that approach the capacity over a wide range of practical SNR values.

We reformulate the Equal Coding Theorem in sensory neural encoding with ON- and OFF-neurons as a channel capacity problem. We then present a capacity-based proof of the Equal Coding Theorem, and generalize it to neurons with different firing probabilities. We also briefly discuss the biological implications of this generalization.

Consider a multiple-input single-output system, where the nonnegative, peak-limited inputs ${X_1}, \ldots ,{X_{{n_{\text{T}}}}} \in [0,\mathcal{A}]$ are subject to first- and second-moment sum-constraints on all antennas. The paper characterizes all probability distributions that can be induced for the "channel image," which is given by the inner product of the input vector with a given channel vector. Key to this result is the description of input vectors that achieve a given deterministic channel image with the smallest energy, where "energy" of an input vector refers to a weighted sum of its one- and two-norms. Minimum-energy input vectors have an interesting structure: depending on the desired channel image, some of the weakest antennas are silenced, and the remaining antennas are chosen according to a shifted and amplitude-constrained beamforming rule.

We study rate-distortion problems of a Poisson process using a group theoretic approach. By describing a realization of a Poisson point process with either point timings or inter-point intervals and by choosing appropriate distortion measures, we establish rate-distortion problems of a homogeneous Poisson process as ball-or sphere-covering problems for realizations of the hyperoctahedral group in $\mathbb{R}^{n}$. Specifically, the realizations we investigate are a hypercube and a hyperoctahedron. Thereby we unify three known rate-distortion problems of a Poisson process (with different distortion measures, but resulting in the same rate-distortion function) with the Laplacian-$\ell_{1}$ rate-distortion problem.

The geometric interpretation of sphere covering describing the rate distortion problem of a Gaussian source with the squared-error distortion measure is generalized to a Laplacian source and the ℓ1-distortion measure. Using additional constraints on the distortion measure, sphere covering is further generalized to exponential sources and to Poisson point processes.

This paper studies the capacity of a general multiple-input multiple-output (MIMO) free-space optical intensity channel under a per-input-antenna peak-power constraint and a total average-power constraint over all input antennas. The focus is on the scenario with more transmit than receive antennas. In this scenario, different input vectors can yield identical distributions at the output, when they result in the same image vector under multiplication by the channel matrix. We first determine the most energy-efficient input vectors that attain each of these image vectors. Based on this, we derive an equivalent capacity expression in terms of the image vector, and establish new lower and upper bounds on the capacity of this channel. The bounds match when the signal-to-noise ratio (SNR) tends to infinity, establishing the high-SNR asymptotic capacity. We also characterize the low-SNR slope of the capacity of this channel.

This paper investigates the capacity of block fading optical intensity channels with more transmit than receive antennas under different assumptions on the transmitter's channel state information (CSI). Lower and upper bounds on the capacities are derived using the entropy power inequality (EP!) and a dual expression for capacity. Our lower bounds for perfect and partial CSI utilize a transmit-antenna cooperation strategy based on minimum-energy signaling, which we proposed recently. For perfect CSI, this lower bound matches the upper bound asymptotically in the high signal-to-noise ratio (SNR) regime. For imperfect CSI, our lower bound is close to its perfect-CSI counterpart.

An extension from the pairwise Hamming distance to the r-wise Hamming distance is presented. It can be used to fully characterize the maximum-likelihood decoding (MLD) error of an arbitrary code over the binary erasure channel (BEC). By noting that good codes always have large minimum r-wise Hamming distances for all r, a new design criterion for a code is introduced: the minimum r-wise Hamming distance. We then prove an upper bound for the minimum r-wise Hamming distance of an arbitrary code, called the generalized Plotkin bound, and provide a class of (nonlinear) codes that achieve the bound for every r.

This paper investigates the capacity of the multiple- input multiple-output free-space optical intensity channel under a per-input-antenna peak-power constraint and a total average-power constraint over all input antennas. Our work considers the setup with more transmit than receive antennas, and characterizes capacity as an alternative optimization problem over the distribution of the input vector times the channel matrix. This alternative capacity expression is then used to obtain upper and lower bounds on the capacity, which match asymptotically in the high signal-to-noise ratio regime.

This paper investigates fundamental properties of nonlinear binary codes by looking at the codebook matrix not row-wise (codewords), but column-wise. The family of weak flip codes is presented and shown to contain many beautiful properties. In particular the subfamily fair weak flip codes, which goes back to Shannon et al. and which was shown to achieve the error exponent with a fixed number of codewords M, can be seen as a generalization of linear codes to an arbitrary number of codewords. The fair weak flip codes are related to binary nonlinear Hadamard codes. Based on the column-wise approach to the codebook matrix, the r-wise Hamming distance is introduced as a generalization to the well-known and widely used (pairwise) Hamming distance. It is shown that the minimum r-wise Hamming distance satisfies a generalized r-wise Plotkin bound. The r-wise Hamming distance structure of the nonlinear fair weak flip codes is analyzed and shown to be superior to many codes. In particular, it is proven that the fair weak flip codes achieve the r-wise Plotkin bound with equality for all r. In the second part of this paper, these insights are applied to a binary erasure channel with an arbitrary erasure probability 0 <; δ <; 1. An exact formula for the average error probability of an arbitrary (linear or nonlinear) code using maximum likelihood decoding is derived and shown to be expressible using only the r-wise Hamming distance structure of the code. For a number of codewords M satisfying M ≤ 4 and an arbitrary finite blocklength n, the globally optimal codes (in the sense of minimizing the average error probability) are found. For M = 5 or M = 6 and an arbitrary finite blocklength n, the optimal codes are conjectured. For larger M, observations regarding the optimal design are presented, e.g., that good codes have a large r-wise Hamming distance structure for all r. Numerical results validate our code design criteria and show the superiority of our best found nonlinear weak flip codes compared with the best linear codes.

This paper derives upper and lower bounds on the capacity of the multiple-input single-output free-space optical intensity channel with signal-independent additive Gaussian noise subject to both an average-intensity and a peak-intensity constraint. In the limit where the signal-to-noise ratio (SNR) tends to infinity, the asymptotic capacity is specified, while in the limit where the SNR tends to zero, the exact slope of the capacity is given.

This paper provides several asymptotic capacity results for the multiple-input multiple-output free-space optical intensity channel in the regime of high signal-to-noise ratio (SNR). For the case where the channel matrix has full column rank, the asymptotic capacity is derived assuming a peak-power constraint on each transmit antenna, or an average-power constraint on the total power across all transmit antennas, or both. For multiple-input and single-output channels, the asymptotic high-SNR capacity is derived when either only the total average power is constrained, or only the per-antenna peak power is constrained, or both but with the average-power constraint being sufficiently loose.

This paper derives the asymptotic capacity for the multiple-input single-output free-space optical intensity channel in the regime of high signal-to-noise ratio (SNR). The asymptotic result is proven via upper and lower bounds on capacity at finite SNR.

The exact value of the average error probability of an arbitrary code (linear or nonlinear) using maximum likelihood decoding is studied on binary erasure channels (BECs) with arbitrary erasure probability 03.

A rigorous performance analysis of Fano coding is presented, providing an upper bound on the average codeword length of binary and ternary Fano codes for an arbitrary discrete memoryless source. The performance bound is slightly better than Shannon's well-known bound for Shannon coding. As a by-product a novel general lower bound on Shannon entropy is derived that might be of interest also in a different context. This bound is expressed with the help of variational distance and provides a special case of a reverse Pinsker inequality.

The memoryless additive inverse Gaussian noise channel model describing communication based on the exchange of chemical molecules in a drifting liquid medium is investigated for the situation of simultaneously an average-delay and a peak-delay constraint. Analytical upper and lower bounds on its capacity in bits per molecule use are presented. These bounds are shown to be asymptotically tight, i.e., for the delay constraints tending to infinity with their ratio held constant (or for the drift velocity of the fluid tending to infinity), the asymptotic capacity is derived precisely. Moreover, characteristics of the capacity-achieving input distribution are derived that allow accurate numerical computation of capacity. The optimal input appears to be a mixed continuous and discrete distribution.

An analytic expression for the asymptotic capacity of noncoherent single-input multiple-output regular fading channels with memory and with partial receiver side information is derived and is shown to remain unchanged by causal or acausal side information at the transmitter and by a noiseless feedback link. In particular, the corresponding fading numbers are identical. Furthermore, the asymptotic capacity of a single-input single-output nonregular Gaussian fading channel with memory is investigated, and it is shown that the prelog is unaffected by noiseless feedback.

In this paper, we re-introduce from our previous work [1] a new family of nonlinear codes, called weak flip codes, and show that its subfamily fair weak flip codes belongs to the class of equidistant codes, satisfying that any two distinct codewords have identical Hamming distance. It is then noted that the fair weak flip codes are related to the binary nonlinear Hadamard codes as both code families maximize the minimum Hamming distance and meet the Plotkin upper bound under certain blocklengths. Although the fair weak flip codes have the largest minimum Hamming distance and achieve the Plotkin bound, we find that these codes are by no means optimal in the sense of average error probability over binary symmetric channels (BSC). In parallel, this result implies that the equidistant Hadamard codes are also nonoptimal over BSCs. Such finding is in contrast to the conventional code design that aims at the maximization of the minimum Hamming distance. The results in this paper are proved by examining the exact error probabilities of these codes on BSCs, using the column-wise analysis on the codebook matrix.

The channel capacity of a noncoherent single-input multiple-output regular fading channel with memory and with feedback is investigated. The fading process is assumed to be a general stationary and ergodic random process of finite energy and finite differential entropy rate. The feedback is assumed to be noisefree (i.e., it is of infinite capacity), but causal. It is reported that the asymptotic capacity grows double-logarithmically in the power and that the second term in the asymptotic expansion, the fading number, is unchanged with respect to the same channel without feedback.

Optimal block-codes (in the sense of minimum average error probability, using maximum likelihood decoding) with a small number of codewords are investigated for the binary asymmetric channel (BAC), including the two special cases of the binary symmetric channel (BSC) and the Z-channel (ZC), both with arbitrary cross-over probabilities. For the ZC, the optimal code structure for an arbitrary finite blocklength is derived in the cases of two, three, and four codewords and conjectured in the case of five codewords. For the BSC, the optimal code structure for an arbitrary finite blocklength is derived in the cases of two and three codewords and conjectured in the case of four codewords. For a general BAC, the best codebooks under the assumption of a threshold decoder are derived for the case of two codewords. The derivation of these optimal codes relies on a new approach of constructing and analyzing the codebook matrix not rowwise (codewords), but columnwise. This new tool leads to an elegant definition of interesting code families that is recursive in the blocklength n and admits their exact analysis of error performance. This allows for a comparison of the average error probability between all possible codebooks.

A very recent and new model describing communication based on the exchange of chemical molecules in a drifting liquid medium is investigated and new analytical upper and lower bounds on the capacity are presented. The bounds are asymptotically tight, i.e., if the average-delay constraint is loosened to infinity or if the drift velocity of the liquid medium tends to infinity, the corresponding asymptotic capacities are derived precisely.

This paper investigates a channel model describing optical communication based on intensity modulation. It is assumed that the main distortion is caused by additive Gaussian noise, however, with a noise variance depending on the current signal strength. Both the high-power and low-power asymptotic capacities under simultaneously both a peak-power and an average-power constraint are derived. The high-power results are based on a new firm (nonasymptotic) lower bound and a new asymptotic upper bound. The upper bound relies on a dual expression for channel capacity and the notion of capacity-achieving input distributions that escape to infinity. The lower bound is based on a new lower bound on the differential entropy of the channel output in terms of the differential entropy of the channel input. The low-power results make use of a theorem by Prelov and van der Meulen.

Block-codes with a very small number of codewords are Investigated for the two special binary memoryless channels, the binary symmetric channel (BSC) and the Z-channel (ZC). The optimal (In the sense of minimum average error probability, using maximum likelihood decoding) code structure Is derived for the cases of two, three, and four codewords and an arbitrary blocklength. It Is shown that for two possible messages, on a BSC, the so-called flip codes of type t are optimal for any t, while on a ZC, the flip code of type 0 is optimal. For codes with three or four messages It Is shown that the so-called weak flip codes of some given type are optimal where the type depends on the blocklength. For all cases an algorithm Is presented that constructs an optimal code for blocklength n recursively from an optimal code of length n - 1. For the ZC a recursive optimal code design Is conjectured In the case of live possible messages. The derivation of these optimal codes relies heavily on a new approach of constructing and analyzing the code-matrix not row-wise (codewords), but column-wise. Moreover, these results also prove that the minimum Hamming distance might be the wrong design criterion for optimal codes even for very symmetric channels like the BSC.

The sum-rate capacity of a noncoherent memoryless multiple-access Rician fading channel is investigated under three different categories of power constraints: individual per user peak-power constraints, individual per user average-power constraints, or a global power-sharing average-power constraint. Upper and lower bounds on the sum-rate capacity are derived, and it is shown that at high signal-to-noise ratio the sum-rate capacity only grows double-logarithmically in the available power. The asymptotic behavior of capacity is then analyzed in detail and the exact asymptotic expansion is derived including its second term, the so-called fading number. It is shown that the fading number is identical to the fading number of the single-user Rician fading channel that is obtained when only the user seeing the best channel is transmitting and all other users are switched off at all times. This pessimistic result holds independently of the type of power constraint that is imposed.

The large-inputs asymptotic capacity of a peak-power and average-power limited discrete-time Poisson channel is derived using a new firm (nonasymptotic) lower bound and an asymptotic upper bound. The upper bound is based on the dual expression for channel capacity and the notion of capacity-achieving input distributions that escape to infinity. The lower bound is based on a lower bound on the entropy of a conditionally Poisson random variable in terms of the differential entropy of its conditional mean.

Upper and lower bounds are derived on the capacity of the free-space optical intensity channel. This channel has a nonnegative input (representing the transmitted optical intensity), which is corrupted by additive white Gaussian noise. To preserve the battery and for safety reasons, the input is constrained in both its average and its peak power. For a fixed ratio of the allowed average power to the allowed peak power, the difference between the upper and the lower bound tends to zero as the average power tends to infinity and their ratio tends to one as the average power tends to zero. When only an average power constraint is imposed on the input, the difference between the bounds tends to zero as the allowed average power tends to infinity, and their ratio tends to a constant as the allowed average power tends to zero.

The fading number of a general (not necessarily Gaussian) regular multiple-input multiple-output (MIMO) fading channel with arbitrary temporal and spatial memory is derived. The channel is assumed to be noncoherent, i.e., neither receiver nor transmitter have knowledge about the channel state, but they only know the probability law of the fading process. The fading number is the second term in the asymptotic expansion of channel capacity when the signal-to-noise ratio (SNR) tends to infinity. It is related to the border of the high-SNR region with double-logarithmic capacity growth.

New upper and lower bounds are presented on the capacity of the free-space optical intensity channel. This channel is characterized by inputs that are nonnegative (representing the transmitted optical intensity) and by outputs that are corrupted by additive white Gaussian noise (because in free space the disturbances arise from many independent sources). Due to battery and safety reasons the inputs are simultaneously constrained in both their average and peak power. For a fixed ratio of the average power to the peak power the difference between the upper and the lower bounds tends to zero as the average power tends to infinity, and the ratio of the upper and lower bounds tends to one as the average power tends to zero. The case where only an average-power constraint is imposed on the input is treated separately. In this case, the difference of the upper and lower bound tends to 0 as the average power tends to infinity, and their ratio tends to a constant as the power tends to zero.

The fading number of a general (not necessarily Gaussian) regular multiple-input multiple-output (MIMO) fading channel with arbitrary temporal and spatial memory is derived. The channel is assumed to be non-coherent, i.e., neither receiver nor transmitter have knowledge about the channel state, but they only know the probability law of the fading process. The fading number is the second term in the asymptotic expansion of channel capacity when the signal-to-noise ratio (SNR) tends to infinity. It is shown that the fading number can be achieved by an input that is the product of two independent processes: a stationary and circularly symmetric direction- (or unit-) vector process whose distribution needs to be chosen such that it maximizes the fading number, and a non-negative magnitude process that is independent and identically distributed (IID) and that escapes to infinity. Additionally, in the more general context of an arbitrary stationary channel model satisfying some weak conditions on the channel law, it is shown that the optimal input distribution is stationary apart from some edge effects.

In this correspondence, we derive the fading number of multiple-input multiple-output (MIMO) flat-fading channels of general (not necessarily Gaussian) regular law without temporal memory. The channel is assumed to be noncoherent, i.e., neither receiver nor transmitter have knowledge about the channel state, but they only know the probability law of the fading process. The fading number is the second term, after the double-logarithmic term, of the high signal-to-noise ratio (SNR) expansion of channel capacity. Hence, the asymptotic channel capacity of memoryless MIMO fading channels is derived exactly. The result is then specialized to the known cases of single-input-multiple-output (SIMO), multiple-input single-output (MISO), and single-input-single-output (SISO) fading channels, as well as to the situation of Gaussian fading.

We derive new upper and lower bounds on the fading number of non-coherent multiple-input single-output (MISO) fading channels of general (not necessarily Gaussian) regular law with spatial and temporal memory. The fading number is the second term, after the double-logarithmic term, of the high signal-to-noise ratio (SNR) expansion of channel capacity. In case of an isotropically distributed fading vector it is proven that the upper and lower bound coincide, i.e., the general MISO fading number with memory is known precisely. The upper and lower bounds show that a type of beam-forming is asymptotically optimal

We derive new upper and lower bounds on the fading number of multiple-input single-output (MISO) fading channels of general (not necessarily Gaussian) regular law with spatial and temporal memory. The fading number is the second term, after the double-logarithmic term, of the high signal-to-noise ratio (SNR) expansion of channel capacity.In case of an isotropically distributed fading vector it is proven that the upper and lower bound coincide, i.e., the general MISO fading number with memory is known precisely.The upper and lower bounds show that a type of beam-forming is asymptotically optimal.

We derive the fading number of stationary and ergodic (not necessarily Gaussian) single-input multiple-output (SIMO) fading channels with memory. This is the second term, after the double-logarithmic term, of the high signal-to-noise ratio (SNR) expansion of channel capacity. The transmitter and receiver are assumed to be cognizant of the probability law governing the fading but not of its realization. It is demonstrated that the fading number is achieved by independent and identically distributed (i.i.d.) circularly symmetric inputs of squared magnitude whose logarithm is uniformly distributed over an SNR-dependent interval. The upper limit of the interval is the logarithm of the allowed transmit power, and the lower limit tends to infinity sublogarithmically in the SNR. The converse relies inter alia on a new observation regarding input distributions that escape to infinity. Lower and upper bounds on the fading number for Gaussian fading are also presented. These are related to the mean squared-errors of the one-step predictor and the one-gap interpolator of the fading process respectively. The bounds are computed explicitly for stationary mth-order autoregressive AR(m) Gaussian fading processes.

A technique is proposed for the derivation of upper bounds on channel capacity. It is based on a dual expression for channel capacity where the maximization (of mutual information) over distributions on the channel input alphabet is replaced with a minimization (of average relative entropy) over distributions on the channel output alphabet. We also propose a technique for the analysis of the asymptotic capacity of cost-constrained channels. The technique is based on the observation that under fairly mild conditions capacity achieving input distributions "escape to infinity." The above techniques are applied to multiple-antenna flat-fading channels with memory where the realization of the fading process is unknown at the transmitter and unknown (or only partially known) at the receiver. It is demonstrated that, for high signal-to-noise ratio (SNR), the capacity of such channels typically grows only double-logarithmically in the SNR. To better understand this phenomenon and the rates at which it occurs, we introduce the fading number as the second-order term in the high-SNR asymptotic expansion of capacity, and derive estimates on its value for various systems. It is suggested that at rates that are significantly higher than the fading number, communication becomes extremely power inefficient, thus posing a practical limit on practically achievable rates. Upper and lower bounds on the fading number are also presented. For single-input-single-output (SISO) systems the bounds coincide, thus yielding a complete characterization of the fading number for general stationary and ergodic fading processes. We also demonstrate that for memoryless multiple-input single-output (MISO) channels, the fading number is achievable using beam-forming, and we derive an expression for the optimal beam direction. This direction depends on the fading law and is, in general, not the direction that maximizes the SNR on the induced SISO channel. Using a new closed-form expression for the expectation of the logarithm of a noncentral chi-square distributed random variable we provide some closed-form expressions for the fading number of some systems with Gaussian fading, including SISO systems with circularly symmetric stationary and ergodic Gaussian fading. The fading number of the latter is determined by the fading mean, fading variance, and the mean squared error in predicting the present fading from its past; it is not directly related to the Doppler spread. For the Rayleigh, Ricean, and multiple-antenna Rayleigh-fading channels we also present firm (nonasymptotic) upper and lower bounds on channel capacity. These bounds are asymptotically tight in the sense that their difference from capacity approaches zero at high SNR, and their ratio to capacity approaches one at low SNR.

It has recently been shown that at high signal-to-noise ratios (SNR) the capacity of multi-antenna systems over flat fading channels (without receiver or transmitter side-information) typically grows only double-logarithmically in the SNR. Here we further refine the analysis and study the "fading number" /spl chi/, which we define as the limit of the difference between channel capacity and log(1+log(1+SNR)). It is suggested that at high SNR, i.e., at rates that significantly exceed the fading number, a capacity increase of one bit per channel use requires the squaring of the SNR, or equivalently, the doubling of the SNR as expressed in decibels. In this loose sense, the fading number can be viewed as the channel limiting rate for power-efficient communication. Note, however, that the fading number may be negative. While the use of multiple antennas does not typically change the double-logarithmic asymptotic dependence of channel capacity on the SNR, multiple antennas do typically increase the fading number, albeit at times (e.g., in the Rayleigh fading case) only in an additive way that grows only logarithmically with the number of antennas.