Probabilistic amplitude shaping (PAS) is on track to become the de facto coded modulation standard for communication systems aiming to operate close to channel capacity at high transmission rates. The essential component of PAS that breeds this widespread interest is the amplitude shaping block, through which the channel input distribution is controlled. This block is responsible for converting bit strings into amplitude sequences with certain properties, e.g., fixed composition, limited energy, limited energy variation, etc. Recently, band-trellis enumerative sphere shaping (B-ESS) was introduced as an amplitude shaping technique that achieves limited energy variations which is useful in optical communication scenarios. B-ESS operates based on a trellis diagram in which sequences with high energy variations are pruned. In this work, we study the implementation of B-ESS. We first show that thanks to the trellis structure obtained by this pruning, B-ESS can be implemented with very low storage complexity. The trellis computation is shown to be reduced to a set of recursive multiplications with a scalar factor. Then we show that this scalar factor can be adjusted such that the trellis computation is further simplified and realized with only binary shifts. This shift-based B-ESS (1) can be implemented for arbitrarily long blocklengths without incurring an increase in complexity, and (2) can operate in a streaming mode similar to convolutional coding.

Recent years have seen renewed attention to arithmetic coding (AC). This is thanks to the use of AC for distribution matching (DM) to control the channel input distribution in probabilistic amplitude shaping. There are two main problems inherent to AC: (1) its required arithmetic precision grows linearly with the input length, and (2) high-precision multiplications and divisions are required. Here, we introduce a multiplication-free AC-based DM technique via three lookup tables (LUTs) which solves both problems above. These LUTs are used to approximate the high-precision multiplications and divisions by additions and subtractions. The required precision of our approach is shown to grow logarithmically with the input length. We prove that this approximate technique maintains the invertibility of DM. At an input length of 1024 symbols, the proposed technique achieves negligible rate loss (< 0.01 bit/sym) against the full-precision DM, while requiring less than 4 kilobytes of storage.

We consider a discrete memoryless cloud radio access network in which K users communicate with a remote destination through 2 relays in a cascade. The relays are oblivious in the sense that they operate without knowledge of the users’ codebooks. We focus on a scenario where the first and second relays are connected through a finite-capacity error-free link, while the second relay is connected to the remote destination via an infinite-capacity link. We establish the capacity region in this case, and show that it is achieved via a compress-and-forward scheme with successive decoding. Finally, the extension to Gaussian networks is discussed.

In line-of-sight (LOS) massive MIMO, there is a nonnegligible probability that the channel vectors of some users become correlated. In these correlated scenarios, nonlinear precoders can be used instead of linear precoders at the cost of high computational complexity. To reduce the complexity of nonlinear precoders, hybrid linear and nonlinear precoders have been suggested in 5G New Radio (NR). In this paper, we find the probability that there is at least one pair of correlated users and we find the average number of correlated users. We propose a hybrid linear and nonlinear precoder (HLNP) with max-min power control for which the served users are divided into two groups. By employing a proposed modified Tomlinson-Harashima Precoding (THP), we design and combine the transmit vectors of the two groups such that inter-group interference is removed. Simulation results show that by employing HLNP instead of zero-forcing, the required transmit power to assure a given average block error rate (BLER) with 95% probability is reduced. For a 64-antennas BS, when modified THP is used for 3 out of 10 users in HLNP, the transmit power is reduced by up to 4.70 dB to assure an average BLER of 10−2using 16QAM and 64QAM constellations with NR low-density parity-check codes.

We present RESCURE, a security solution built on software, which retrofits Internet of Things (IoT) devices to secure ones. RESCURE exploits the entropy originating from the random variations of silicon (transistors) during manufacturing and generates a unique unforgeable root key and an identity per device. In this way, root key and identity are inseparable from the IoT hardware. To achieve lifetime reliability (reproducibility) and security (randomness) for root key and identity, we apply error correcting and randomness amplification algorithms to the signals derived from silicon. RESCURE supports certificates which are able to prove the device identity and authenticity. RESCURE supports multiple keys derivation (private keys or private/public key pairs) and End-to-End security. In this way an IoT device is able to communicate securely and independently with multiple actors (e.g., Service Providers). It supports secure storage so it is able to encrypt sensitive data such as application keys, sensitive data or software Intellectual Properties (IP). Finally, the entire device software is protected by secure boot and secure software update mechanisms allowing for malware-free software execution and renewable security and features. RESCURE has been prototyped on an ST32L4 device and its performance is presented across real use case scenarios covering the entire life cycle of the device. It is a low-cost solution for all the devices manufacturers that want to achieve high standard security without redesigning the hardware of their IoT product.

Probabilistic amplitude shaping (PAS) combines an outer shaping layer with an inner, systematic forward error correction (FEC) layer to close the shaping gap. Proposed for PAS, constant composition distribution matching (CCDM) produces amplitude sequences with a fixed empirical distribution. We show that CCDM suffers from high rate losses for small block lengths, and we propose to use Enumerative Sphere Shaping (ESS) instead. ESS minimizes the rate loss at any block length. Furthermore, we discuss the computational complexity of ESS and demonstrate that it is significantly smaller than shell mapping (SM), which is another method to perform sphere shaping. We then study the choice of design parameters for PAS. Following Wachsmann et al., we show that for a given constellation and target rate, there is an optimum balance between the FEC code rate and the entropy of the Maxwell-Boltzmann distribution that minimizes the gap-to-capacity. Moreover, we demonstrate how to utilize the non-systematic convolutional code from IEEE 802.11 in PAS. Simulations over the additive white Gaussian noise (AWGN) and frequency-selective channels exhibit that ESS is up to 1.6 and 0.7 dB more energy-efficient than uniform signaling at block lengths as small as 96 symbols, respectively, with convolutional and low-density parity-check (LDPC) codes.

The dependency between the Gaussianity of the input distribution for the additive white Gaussian noise (AWGN) channel and the gap-to-capacity is discussed. We show that a set of particular approximations to the Maxwell- Boltzmann (MB) distribution virtually closes most of the shaping gap. We relate these symbol-level distributions to bit-level distributions, and demonstrate that they correspond to keeping some of the amplitude bit-levels uniform and independent of the others. Then we propose partial enumerative sphere shaping (P-ESS) to realize such distributions in the probabilistic amplitude shaping (PAS) framework. Simulations over the AWGN channel exhibit that shaping 2 amplitude bits of 16-ASK have almost the same performance as shaping 3 bits, which is 1.3 dB more power- efficient than uniform signaling at a rate of 3 bit/symbol. In this way, required storage and computational complexity of shaping are reduced by factors of 6 and 3, respectively.

Internet-of-Things (IoT) devices are usually small, low cost, and have limited resources, which makes them vulnerable to physical and cloning attacks. To secure IoT devices, physical unclonable functions (PUFs) are relatively new security primitives used for device authentication and device-specific secret-key generation. In this paper, we focus on designing a robust construction to derive secret keys from static randomaccess memory (SRAM)-PUFs, which enjoy the uniqueness and randomness properties stemming from the manufacturing variations of SRAM memory cells. We make use of a polar code construction. Based on the fact that SRAM memory can often be found in today's IoT devices, and since polar codes have been selected as error-correction technique in the fifth generation standard, this makes the proposed scheme a promising candidate for reducing the extra cost and securing resource-constrained IoT devices. In this paper, we propose a novel construction method to eliminate the effect of noise and bias in SRAM-PUFs. We shall prove that the secrecy leakage of the helper data about the secretkey can be made negligible due to polarization and proper code construction design. Results show that the proposed scheme provides a significant improvement of the reliability (achieve a failure probability below 10-6) and of the realizable secret-key rate, which is also evaluated by the theoretical analysis. In addition, the proposed scheme provides the possibility to tradeoff complexity, secrecy, and reliability with the same code construction for different IoT applications.

We introduce the multiple enrollment scheme for SRAM-physical unclonable functions (PUFs). During each enrollment, the binary power-on values of the SRAM are observed, and a corresponding key and helper data are generated. Each key can later be reconstructed from an additional observation and the helper data. The helper data do not reveal information about the keys to an attacker. It is our goal to use the additional enrollments to consecutively increase the entropy of the generated key material. We analyze two alternative settings. First, we present a regular setting, where each additional key is independent of all previous keys. Second, we introduce a key-replacement setting, where instead of an additional independent key, a new key (of increased length) is generated that replaces the old key. We characterize the capacity regions for both the settings. We show that the total achievable secret-key rate is equal to the mutual information between all enrollment observations and a single (reconstruction) observation. We derive our results based on a statistical model for the SRAM-PUF that has been proposed in the literature. This model implies a permutation symmetry property of the SRAM-PUF which plays a key role in our proofs.

Enumerative sphere shaping of N-dimensional constellations is discussed. It is proven that a finite-precision number representation is suitable for use in two enumerative indexing algorithms: Enumerative sphere shaping and Divide & Conquer (D&C) shaping. This representation decreases the storage complexities of these methods significantly. D&C is the basis of the well-known shell mapping algorithm and thus our approximations also apply there.

We use an SRAM Physical Unclonable Function (PUF) to generate secret keys for authentication purposes. During enrollment, an encoder generates a secret key and corresponding helper data based on an SRAM-PUF observation vector. Later, the device identity is verified by its ability to reconstruct the same key. We define the multiple enrollment forgetful setting. Here, during each consecutive enrollment the previous key is replaced by a new (larger) key based on a new observation of the SRAM-PUF. Furthermore, additional helper data is published after each enrollment. We show that all helper messages together do not reveal information about the relevant secret. Furthermore, the achievable secret-key rate increases with each enrollment, up to a limit that depends on the statistics of the source. For the SRAM-PUF this limit is given by the mutual information between the observation variable and the cell-state.

Physical unclonable functions (PUFs) are relatively new security primitives used for device authentication and device-specific secret key generation. In this paper we focus on SRAM- PUFs. The SRAM-PUFs enjoy uniqueness and randomness properties stemming from the intrinsic randomness of SRAM memory cells, which is a result of manufacturing variations. This randomness can be translated into the cryptographic keys thus avoiding the need to store and manage the device cryptographic keys. Therefore these properties, combined with the fact that SRAM memory can be often found in today's IoT devices, make SRAM-PUFs a promising candidate for securing and authentication of the resource-constrained IoT devices. PUF observations are always affected by noise and environmental changes. Therefore secret- generation schemes with helper data are used to guarantee reliable regeneration of the PUF-based secret keys. Error correction codes (ECCs) are an essential part of these schemes. In this work, we propose a practical error correction construction for PUF-based secret generation that are based on polar codes. The resulting scheme can generate 128-bit keys using 1024 SRAM-PUF bits and 896 helper data bits and achieve a failure probability of 10^{-9} or lower for a practical SRAM-PUFs setting with bit error probability of 15%. The method is based on successive cancellation combined with list decoding and hash-based checking that makes use of the hash that is already available at the decoder. In addition, an adaptive list decoder for polar codes is investigated. This decoder increases the list size only if needed.

Fuzzy commitment and syndrome-based schemes are two well-known helper data schemes used to bind and generate, respectively, a secret key to/from SRAM-PUF observations. To allow the decoder to reconstruct this secret key from a new (verification) observation of an SRAM-PUF, an encoder has to generate so-called helper data. This helper data is a function of an SRAM-PUF enrollment observation and, in case of fuzzy commitment, the secret key. The helper data is assumed to be public and thus must leak no information about the secret key. It is known that both schemes can achieve secrecy capacity equal to the mutual information between enrollment and verification SRAM-PUF observations at zero secrecy leakage, when the observations are unbiased and a single enrollment is performed. We study here the situation when multiple SRAM-PUF observations are used to create multiple secret keys. First, we introduce a symmetry property for multiple SRAM-PUF observations. For such symmetric SRAM-PUFs, we show that, in both helper data schemes, the helper data corresponding to multiple SRAM-PUF observations provide no information about any of the secret keys.

A constellation shaping scheme is proposed. The motivation is to decrease the required transmit power for a specified spectral efficiency. Instead of imposing a non-uniform distribution on the constellation or using nonuniformly spaced symbols, a sphere constraint is employed on the n-dimensional signal space. An efficient algorithm called enumerative amplitude shaping is given to find and index all signal points in the sphere. A comparison with a prominent probabilistic shaping algorithm is provided. The enumerative approach achieves the target rate more efficiently for small block lengths. To introduce error correction, the convolutional encoder used in IEEE Std 802.11 is combined with the shaper in a novel way. Instead of utilizing a larger constellation in combination with shaping, a higher code rate is used by puncturing. Gains up to 1.61 dB are observed for the rates 3 to 6 bits/2-D with 64- and 256-QAM schemes in AWGN channels. The contribution of puncturing in these gains is discussed.

Visible light communications (VLC) with light-emitting diodes (LEDs) has attracted applications, such as data communications, lighting control, and light interaction. In this paper, we propose a system by which two LED devices are used for two-way VLC while also providing illumination. We consider Manchester encoding with on-off keying for transmission. A reception scheme in which the LEDs themselves are used as receivers by sensing in the OFF periods is considered. A synchronization scheme to achieve frame- and symbol-level synchronization is also proposed. A prototype of the proposed system is designed and the communication performance is evaluated.

Visible light communications (VLC) with light emitting diodes (LEDs) has attracted applications like data communications, lighting control and interaction. In this paper, we propose a system in which two LED devices are used for two-way VLC while also providing illumination. We consider a redundant run-length limited encoding for data transmission. A reception scheme in which the LEDs themselves are used as receivers by sensing in the off period is considered. A prototype of the proposed system is designed and the communication performance is evaluated. An advantage of the proposed system is that no additional light sensors are needed as the LEDs themselves are used as communication transceivers.

In an information-theoretic framework, we introduce a two-stage decoding scheme capable of achieving identification capacity to address search and memory complexities in large-scale identification systems. This two-stage decoding procedure is accomplished as follows. For a given query, at the first stage, a list of cluster indices is estimated. Then, at the second stage, refinement checks are performed for all the members of the clusters to produce a single index. The first result this paper presents is the achievable rate quadruple region that specifies necessary conditions that the two-stage decoding scheme should satisfy to be able to achieve the identification capacity. The rest of this paper is designated to investigate various achievable rate quadruples in which the proposed two-stage identification setup can reduce the search complexity with respect to conventional identification setups.

In this paper, we analyze two types of biometric identification systems with protected templates that also support authentication. In the first system, two terminals observe biometric enrollment and identification sequences of a number of individuals. It is the goal of these terminals to form a common secret for the sequences belonging to the same individual by interchanging public (helper) messages of all individuals such that the information leakage about the secrets from these helper messages is negligible. These secret keys are used for authentication purposes. Moreover, the second terminal should be able to establish the identity of an individual based on the presented biometric identification sequence and helper messages. It is important to realize that biometric data are unique for individuals and cannot be replaced if compromised. Therefore, the helper messages should contain as little as possible information about the biometric data. In the second setting, we consider the first terminal does not generate secret keys from biometric sequences of individuals but chooses them uniformly at random. These keys are conveyed to the second terminal by communicating the corresponding helper messages. In this paper, we determine the fundamental tradeoffs between secret-key, identification, and privacy-leakage rates for both biometric settings.

In biometric privacy-preserving authentication systems that are based on key-binding, two terminals observe two correlated biometric sequences. The first terminal selects a secret key, which is independent of the biometric data, binds this secret key to the observed biometric sequence and communicates it to the second terminal by sending a public message. This message should only contain a negligible amount of information about the secret key, but also leak as little as possible about the biometric data. Current approaches to realize such biometric systems use fuzzy commitment with codes that, given a secret-key rate, can only achieve the corresponding privacy-leakage rate equal to one minus this secret-key rate. However, the results in Willems and Ignatenko [2009] indicate that lower privacy leakage can be achieved if vector quantization is used at the encoder. In this paper we study the use of convolutional and turbo codes applied in fuzzy commitment and its modifications that realize this.

An ultrasonic sensor that employs moving target processing based on differential echo processing and occupant tracking is considered for indoor occupant presence detection. We present a simple statistical model to analyze the presence detection performance of such a sensor. The probability distributions of the differential power signal are first obtained under vacancy and occupancy conditions. We then study the influence of occupant tracking and derive an upper bound on the probability of false alarm. Experimental data is used to verify that the presented analytical statistical models match actual distributions of the differential power.

In this paper, we introduce a new generalized scheme to resolve the trade-off between the identification rate, search and memory complexities in large-scale identification systems. The main contribution of this paper consists in a special database organization based on assigning entries of a database to a set of predefined and possibly overlapping clusters, where the cluster representative points are generated based on statistics of both entries of the database and queries. The decoding procedure is accomplished in two stages: At the first stage, a list of clusters related to the query is estimated, then refinement checks are performed to all members of these clusters to produce a unique index at the second stage. The proposed scheme generalizes several practical searching in identification systems as well as makes it possible to approach a new achievable region of search- memory complexity trade-off.

Starting from the Poisson model we present a channel model for optical communications, called the Square Root (SR) Channel, in which the noise is additive Gaussian with constant variance. Initially, we prove that for large peak or average power, the transmission rate of a Poisson Channel when coding and decoding methods for the SR Channel are used, converges to the capacity of the Poisson Channel. Then, we derive bounds and asymptotic expressions for the capacity of the SR Channel, and compare them to those of other optical channel models. Finally, signal-independent noise sources are discussed.

Noncoherent pulse-position modulation (PPM) with simple channel codes has the potential to realize ultra-low power (ULP) wireless design. In this paper, we develop a Wagnerlike decoding rule for single-parity-check and high-rate Reed-Solomon (RS) coded PPM schemes by simply `flipping' the most unreliable received PPM symbol(s) to obtain a good balance between performance and coding complexity. The proposed algorithm can be considered as a list decoding algorithm that first generates a candidate codeword list based on the algebraic structure of the code before applying soft decisions to decode. This approach can result in more power-efficient realizations of the studied schemes. It is shown that our decoding approach can achieve near maximum likelihood decoding performance based on the trellis, while having a significantly lower decoding complexity. In addition, by exploiting the inherent advantage of PPM transmission, it is possible to reduce the candidate list to further simplify the decoding for RS-coded PPM without losing coding gain. This makes the proposed scheme more attractive for ULP communications.

We present models for authentication in biometric settings. We could determine the trade-off between false-acceptance exponents and the so-called privacy-leakage rate. In this way we extend the Ahlswede-Csiszar secret-key generation result [1993] and more specifically the secret-key rate vs. privacy-leakage rate trade-off, studied by Ignatenko and Willems [2009].

In this paper we analyze secret generation in biometric identification systems with protected templates. This problem is closely related to the study of the biometric identification capacity of Willems et al. 2003 and O'Sullivan and Schmid 2002 and the common randomness generation of Ahlswede and Csiszár 1993. In our system two terminals observe biometric enrollment and identification sequences of a number of individuals. It is the goal of these terminals to form a common secret for the sequences that belong to the same individual by interchanging public (helper) messages for all individuals in such a way that the information leakage about the secrets from these helper messages is negligible. It is important to realize that biometric data are unique for individuals and cannot be replaced if compromised. Therefore the helper messages should contain as little as possible information about the biometric data. On the other hand, the second terminal has to establish the identity of the individual who presented his biometric sequence, based on the helper data produced by the first terminal. In this paper we determine the fundamental tradeoff between secret-key rate, identification rate and privacy-leakage rate in biometric identification systems.

In 1999, Juels and Wattenberg introduced the fuzzy commitment scheme. This scheme is a particular realization of a binary biometric secrecy system with chosen secret keys. It became a popular technique for designing biometric secrecy systems, since it is convenient and easy to implement using standard error-correcting codes. This paper investigates privacy- and secrecy-leakage in fuzzy commitment schemes. The analysis is carried out for four cases of biometric data statistics, i.e., memoryless totally symmetric, memoryless input-symmetric, memoryless, and stationary ergodic. First, the achievable regions are determined for the cases when data statistics are memoryless totally symmetric and memoryless input-symmetric. For the general memoryless and stationary ergodic cases, only outer bounds for the achievable rate-leakage regions are provided. These bounds, however, are sharpened for systematic parity-check codes. Given the achievable regions (bounds), the optimality of fuzzy commitment is assessed. The analysis shows that fuzzy commitment is only optimal for the memoryless totally symmetric case if the scheme operates at the maximum secret-key rate. Moreover, it is demonstrated that for the general memoryless and stationary ergodic cases, the scheme leaks information on both the secret and biometric data.

Ahlswede and Csiszar [1993] introduced the concept of secret sharing. In their source model two terminals observe two correlated sequences. It is the objective of the terminals to form a common secret by interchanging a public message (helper data) in such a way that the secrecy leakage is negligible. In a biometric setting, where the sequences correspond to the enrollment and authentication data, respectively, it is crucial that the public message leaks as little information as possible about the biometric data, since compromised biometric data cannot be replaced. We investigated the fundamental trade-offs for four biometric settings. The first one is the standard (Ahlswede-Csiszar) secret generation setting, for which we determined the secret-key vs. privacy-leakage rate region. Here leakage corresponds to the mutual information between helper data and biometric enrollment sequence. In the second setting the secret is not generated by the terminals but independently chosen, and transmitted using a public message. Again we determined the region of achievable rate-leakage pairs. In setting three and four we consider zero-leakage, i.e. the public message contains only a negligible amount of information about the secret and about the biometric enrollment sequence. To achieve this a private key is needed, which can be observed only by the terminals. We considered again both secret generation and secret transmission and determined for both cases the region of achievable secret-key vs. private-key rate pairs.

We study two-stage search procedures for biometric identification systems in an information-theoretical setting. Our main conclusion is that clustering based on vector-quantization achieves the optimum trade-off between the number of clusters (cluster rate) and the number of individuals within a cluster (refinement rate). The notion of excess rate is introduced, a parameter which relates to the amount of clusters to which the individuals belong. We demonstrate that noisier observation channels lead to larger excess rates.

This paper addresses privacy leakage in biometric secrecy systems. Four settings are investigated. The first one is the standard Ahlswede-Csiszar secret-generation setting in which two terminals observe two correlated sequences. They form a common secret by interchanging a public message. This message should only contain a negligible amount of information about the secret, but here, in addition, we require it to leak as little information as possible about the biometric data. For this first case, the fundamental tradeoff between secret-key and privacy-leakage rates is determined. Also for the second setting, in which the secret is not generated but independently chosen, the fundamental secret-key versus privacy-leakage rate balance is found. Settings three and four focus on zero-leakage systems. Here the public message should only contain a negligible amount of information on both the secret and the biometric sequence. To achieve this, a private key is needed, which can only be observed by the terminals. For both the generated-secret and the chosen-secret model, the regions of achievable secret-key versus private-key rate pairs are determined. For all four settings, the fundamental balance is determined for both unconditional and conditional privacy leakage.

We revisit the information-theoretic analysis of bit-interleaved coded modulation (BICM) by modeling the BICM decoder as a mismatched decoder. The mismatched decoding model is well defined for finite, yet arbitrary, block lengths, and naturally captures the channel memory among the bits belonging to the same symbol. We give two independent proofs of the achievability of the BICM capacity calculated by Caire, where BICM was modeled as a set of independent parallel binary-input channels whose output is the bitwise log-likelihood ratio. Our first achievability proof uses typical sequences, and shows that due to the random coding construction, the interleaver is not required. The second proof is based on the random coding error exponents with mismatched decoding, where the largest achievable rate is the generalized mutual information. Moreover, the generalized mutual information of the mismatched decoder coincides with the infinite-interleaver BICM capacity. We show that the error exponent—and hence the cutoff rate—of the BICM mismatched decoder is upper-bounded by that of coded modulation and may thus be lower than in the infinite-interleaved model; for binary reflected Gray mapping in Gaussian channels the loss in error exponent is small. We also consider the mutual information appearing in the analysis of iterative decoding of BICM with extrinsic information transfer (EXIT) charts: if the symbol metric has knowledge of the transmitted symbol, EXIT mutual information admits a representation as a pseudo-generalized mutual information, which is in general not achievable. A different symbol decoding metric, for which the extrinsic side information refers to the hypothesized symbol, induces a generalized mutual information lower than the coded modulation capacity. In this case, perfect extrinsic side information turns the mismatched-decoder error exponent into that of coded modulation.

We revisit the information-theoretic analysis of bit-interleaved coded modulation (BICM) by modeling the BICM decoder as a mismatched decoder. The mismatched-decoding model is well-defined for finite, yet arbitrary, block lengths, and captures the channel memory among the bits belonging to the same symbol. The generalized mutual information of the mismatched decoder coincides with the infinite-interleaver BICM capacity, where BICM is modeled as a set of independent parallel binary-input channels whose output is the bitwise log-likelihood ratio. The error exponent —and hence the cutoff rate— of the BICM mismatched decoder is upper bounded by that of coded modulation and may thus be lower than in the infinite-interleaved model. For binary reflected Gray mapping in Gaussian channels the loss in error exponent is small.

Two low-complexity methods are proposed for sequential probability assignment for binary independent and identically distributed (i.i.d.) individual sequences with empirical distributions whose governing parameters are known to be bounded within a limited interval. The methods can be applied to different problems where fast accurate estimation of the maximizing sequence probability is very essential to minimizing some loss. Such applications include applications in finance, learning, channel estimation and decoding, prediction, and universal compression. The application of the new methods to universal compression is studied, and their universal coding redundancies are analyzed. One of the methods is shown to achieve the minimax redundancy within the inner region of the limited parameter interval. The other method achieves better performance on the region boundaries and is more robust numerically to outliers. Simulation results support the analysis of both methods. While non-asymptotically the gains may be significant over standard methods that maximize the probability over the complete parameter simplex, asymptotic gains are in second order. However, these gains translate to meaningful significant factor gains in other applications, such as financial ones. Moreover, the methods proposed generate estimators that are constrained within a given interval throughout the complete estimation process which are essential to applications such as sequential binary channel crossover estimation. The results for the binary case lay the foundation to studying larger alphabets.

Repetition-based retransmission is used in Alamouti-modulation [1998] for 2 times 2 MIMO systems. We propose to use instead of ordinary repetition so-called dasiascaled repetitionpsila together with rotation. It is shown that the rotated and scaled Alamouti code has a hard-decision performance which is only slightly worse than that of the Golden code [2005], the best known 2 times 2 space-time code. Decoding the Golden code requires an exhaustive search over all the codewords (or sphere decoding for higher spectral efficiencies), while our rotated and scaled Alamouti code can be decoded with an acceptable complexity.

Shannon wrote in 1948: rdquoThe semantic aspects of communication are irrelevant to the engineering problemrdquo. He demonstrated indeed that the information generated by a source depends only on its statistics and not on the meaning of the source output. The authors derived the fundamental limits for semantic compaction, transmission and compression systems recently. These systems have the property that the codewords are semantic however, i.e. close to the source sequences. In the present article we determine the minimum distortion for semantic partial transmission systems. In these systems only a quantized version of each source source symbol is transmitted to the receiver. It should be noted that our achievability proof is based on weak instead of strong typicality. This is unusual for Gelfand-Pinsker [1980] related setups as e.g. semantic coding and embedding.

The wideband regime of bit-interleaved coded modulation (BICM) in Gaussian channels is studied. The Taylor expansion of the coded modulation capacity for generic signal constellations at low signal-to-noise ratio (SNR) is derived and used to determine the corresponding expansion for the BICM capacity. Simple formulas for the minimum energy per bit and the wideband slope are given. BICM is found to be suboptimal in the sense that its minimum energy per bit can be larger than the corresponding value for coded modulation schemes. The minimum energy per bit using standard Gray mapping on$M$-PAM or$M^2$-QAM is given by a simple formula and shown to approach${-}$0.34 dB as$M$increases. Using the low SNR expansion, a general tradeoff between power and bandwidth in the wideband regime is used to show how a power loss can be traded off against a bandwidth gain.

The problem of transmission of information over arbitrarily permuted parallel channels is studied here. The transmitter does not know over which channel a certain code-sequence will actually be transmitted, however the receiver knows how the sequences are permuted. The permutation is arbitrary but constant during the (simultaneous) transmission of the code-sequences via the parallel channels. It is shown first that the sum of the capacities of each channel is achievable for such a communication system in the special case where the capacity achieving input distributions of all channels are identical. More important is that this sum-capacity can also be achieved using a single channel code for all channels combined with a sequential decoding method. The construction of a rate-matching code based on maximum distance separable (MDS) codes turns out to be crucial. Finally, the case where the parallel channels have different capacity-achieving input distributions is investigated. Also for this case the capacity is determined. Again, this capacity is achievable with a sequential decoding procedure.

We focus here on two secure biometric systems (a common randomness based scheme and a fuzzy commitment scheme) and discuss their privacy preserving properties. We derive bounds on the privacy leakage in these schemes. We also show the relation between employed error-correction and leakage on biometric information, and between privacy and security for the fuzzy commitment scheme.

We propose methods to estimate the secrecy-rate of fuzzy sources (e.g. biometrics and physical unclonable functions (PUFs)) using context-tree weighting. In this paper we focus on PUFs. In order to show that our estimates are realistic we first generalize Maurer's (1993) result to the ergodic case. Then we focus on the fact that the entropy of a stationary two-dimensional structure is a limit of a series of conditional entropies, a result by Anastassiou and Sakrison (1982). We extend this result to the conditional entropy of one two-dimensional structure given another one. Finally we show that the general CTW-method approaches the source entropy also in the two-dimensional stationary case. We further extend this result to the two-dimensional conditional entropy. Based on the obtained results we do several measurements on (our) optical PUFs. These measurements allow us to conclude that a secrecy-rate of 0.3 bit/location is possible

In Bell Syst. Tech. J., 1948, Shannon wrote: "The semantic aspects of communication are irrelevant to the engineering problem." Indeed Shannon demonstrated that the information that is generated by a source depends only on the statistics of the source, and not on the meaning of the source output. By contrast with this we investigate here whether in a compaction system the codewords can be meaningful just like the source output sequences. We require the codeword to be close to the source sequence for some given distortion measure. For so-called semantic compaction systems we determine the fundamental limits for the i.i.d. case. Moreover we consider semantic transmission systems. These systems have the property that the codewords, i.e. the channel input sequences, are close to the source sequence. Finally we investigate semantic compression. A semantic compression system is actually a vector quantizer for which the codeword, i.e. the index to the reproduction vector, resembles the source sequence. Also for semantic transmission and semantic compression we determine the fundamental limits for the i.i.d. case

Gray-scale signals can be represented as sequences of integer-valued symbols. If such a symbol has alphabet {0,1,...,2/sup B/-1} it can be represented by B binary digits. To embed information in these sequences, we are allowed to distort the symbols. The distortion measure that we consider here is squared error, however, errors larger than m are not allowed. The embedded message must be recoverable with error probability zero. In this setup, there is a so-called "rate-distortion function" that tells us what the largest embedding rate is, given a certain distortion level and parameter m. First, we determine this rate-distortion function for m=1 and for m/spl rarr//spl infin/. Next we compare the performance of "low-bits modulation" to the rate-distortion function for m/spl rarr//spl infin/. Then embedding codes are proposed based on i) ternary Hamming codes and on the ii) ternary Golay code. We show that all these codes are optimal in the sense that they achieve the smallest possible distortion at a given rate for fixed block length for any m.

This work presents numerical algorithms for the computation of the capacity for channels with noncausal transmitter side information (the Gel'fand-Pinsker problem) and the rate-distortion function for source coding with decoder side information (the Wyner-Ziv problem). The algorithms are based on the reformulation of the mutual information expressions in terms of Shannon strategies.

Three dependent users are physically separated but communicate with each other via a satellite. Each user generates data which it stores locally. In addition, each user sends a message to the satellite. The satellite processes the messages received from the users and broadcasts one common message to all three users. Each user must be capable of reconstructing the data of the other two users based upon the broadcast message and its own stored data. Our problem is to determine the minimum amount of information which must be transmitted to and from the satellite. The solution to this problem is obtained for the case where subsequent data triples that are produced by the users are independent and identically distributed. The three symbols within each triple are assumed to be dependent. Crucial for the solution is an achievability proof that involves cascaded Slepian-Wolf (1973) source coding.

Summary form only given. We consider the coding of a binary IID source using variable-to-fixed length (VF) source codes. The goal is to design "good" codes of low complexity. A VF code maps variable length source sequences (segments) into fixed length code sequences (codewords). We conclude that Petry codes are an efficient implementation of Tunstall codes and moreover that by approximating the enumeration an even better trade-off between redundancy and complexity is achieved.

This paper discusses a switching method which can be used to combine two sequential universal source coding algorithms. The switching method treats these two algorithms as black-boxes and can only use their estimates of the probability distributions for the consecutive symbols of the source sequence. Three weighting algorithms based on this switching method are presented. Empirical results show that all three weighting algorithms give a performance better than the performance of the source coding algorithms they combine.

First we modify the basic (binary) context-tree weighting method such that the past symbols x/sub 1-D/, x/sub 2-D/, ..., x/sub 0/ are not needed by the encoder and the decoder. Then we describe how to make the context-tree depth D infinite, which results in optimal redundancy behavior for all tree sources, while the number of records in the context tree is not larger than 2T-1. Here T is the length of the source sequence. For this extended context-tree weighting algorithm we show that with probability one the compression ratio is not larger than the source entropy for source sequence length T/spl rarr//spl infin/ for stationary and ergodic sources.

Two weighting procedures are presented for compaction of output sequences generated by binary independent sources whose unknown parameter may occasionally change. The resulting codes need no knowledge of the sequence length T, i.e., they are strongly sequential, and also the number of parameter changes is unrestricted. The additional-transition redundancy of the first method was shown to achieve the Merhav lower bound, i.e., log T bits per transition. For the second method we could prove that additional-transition redundancy is not more than 3/2 log T bits per transition, which is more than the Merhav bound; however, the storage and computational complexity of this method are also more interesting than those of the first method. Simulations show that the difference in redundancy performance between the two methods is negligible.

Context weighting procedures are presented for sources with models (structures) in four different classes. Although the procedures are designed for universal data compression purposes, their generality allows application in the area of classification.

Describes a sequential universal data compression procedure for binary tree sources that performs the "double mixture." Using a context tree, this method weights in an efficient recursive way the coding distributions corresponding to all bounded memory tree sources, and achieves a desirable coding distribution for tree sources with an unknown model and unknown parameters. Computational and storage complexity of the proposed procedure are both linear in the source sequence length. The authors derive a natural upper bound on the cumulative redundancy of the method for individual sequences. The three terms in this bound can be identified as coding, parameter, and model redundancy, The bound holds for all source sequence lengths, not only for asymptotically large lengths. The analysis that leads to this bound is based on standard techniques and turns out to be extremely simple. The upper bound on the redundancy shows that the proposed context-tree weighting procedure is optimal in the sense that it achieves the Rissanen (1984) lower bound.>

is well known that the performance of a data receiver for an intersymbol interference (ISI) channel can depend strongly on the detection delay δ. For a discrete-time communication system, this paper derives a lower bound on the bit-error probability as a function of δ. This “restricted delay bound” is governed by a “restricted-delay distance” d(δ). In many instances, it improves upon Forney’s bound, which is governed by the minimum distance dmin. For instance, for partial-response channels, d(δ) does not converge to dmineven as δ-∝. For channels without spectral zeros, a finite detection delay suffices for d(δ) to coincide with dmin. For all finite δ, d(δ) is determined by a finite number of error patterns and may be computed in a straightforward manner. Unlike dmin, d(δ) depends on the phase characteristics of the channel. Minimum phase is proved to maximize d(δ). The lower bound is generalized to discrete-time channels with colored noise and to continuous-time channels. The effect of transforming a continuous-time channel into a discrete-time channel is discussed. Transformation via a matched filter, as in the IS1 canceller and a Viterbi detector due to Ungerboeck and MacKechnie, is shown to result in poor restricted-delay properties. Implications of these results are illustrated by means of examples.

It is shown that the modified Lawrence algorithm is universal over the class of binary memoryless sources and that the rate converges asymptotically optimally fast to the source entropy. It is proven that no codes exist that have a better asymptotic performance. The asymptotic bounds show that universal variable-to-fixed-length codes can have a significantly lower redundancy than universal fixed-to-variable-length codes with the same number of codewords.>

If in a transmission the inputs of a single-output two-way channel exhibit some interdependence, this dependence must have been created during earlier transmissions. The idea that no more dependence can be consumed than is produced is used to obtain new upper bounds to the capacity region of the discrete memoryless single-output two-way channel. With these upper bounds it is shown that C.E. Shannon' (1961) inner bound region is the capacity region for channels in a certain class, and the Zhang-Berger-Schalkwijk upper bound (1986) for Blackwell's multiplying channel is improved upon.>

A novel universal data compression algorithm is described. This algorithm encodes L source symbols at a time. An upper limit for the number of bits per source symbol is given for the class of binary stationary sources. In the author's analysis, a property of repetition times turns out to be of crucial importance.>

It is proved that the Slepian-Wolf data compression theorem still holds when both encoders are operating totally asynchronously. In addition it is shown that in this case the Wyner-Ahlswede-Korner source-coding theorem holds.>

Petry's efficient and optimal variable to fixed-length source code for discrete memoryless sources was described by Schalkwijk. By extending this coding technique we are able to give an algorithm for Markov sources that is easy to implement. We can bound the loss of efficiency as a function of the code complexity and the mismatch between the source and the code. Rates arbitrarily close to the source entropy are shown to be achievable. In this sense the codes introduced are optimal.

The capacity regions are determined for various communication situations in which one or both encoders for a multiple access channel crib from the other encoder and learn the channel input(s) (to be) emitted by this encoder. Most of the achievability proofs in this paper hinge upon the new concept of backward decoding. Also, the notion of Shannon strategies seems to be of crucial importance. It is demonstrated that in some situations parts of the total cooperation line are achievable. Moreover, it is proved that if the encoders and the decoder are allowed to be nondeterministic, the capacity regions are not increased.

For the binary erasure multiple access channel, van der Meulen showed in his survey paper that the symmetrical rate pair(0.79113,0.79113)is achievable in the ease of feedback. Here we prove that this rate point is on the boundary of the feedback capacity region. Subsequently we apply this result to demonstrate the fact that the feedback capacity region of the product of two multiple access channels can be strictly larger than the (Minkowski)sum Of the feedback capacity regions for the separate channels.

We introduce the communication situation in which the encoders of a multiple access channel are partially cooperating. These encoders are connected by communication links with finite capacities, which permit both encoders to communicate with each other. First we give a general definition of such a communication process (conference). Then, by proving a converse and giving an achievability proof, we establish the capacity region of the multiple access channel with partially cooperating encoders. It turns out that the optimal conference is very simple.

The region Cover and Leung found for the discrete memoryless multiple access channel with feedback to both encoders is proved achievable also with feedback to only one encoder. The novel ideas of nonrandom partitions and restricted decoding are used to avoid list coding techniques.

The capacity region of a class of discrete memoryless multiple access channels with feedback is determined, including as a special case the channel considered by Gaarder and Wolf.