We present a collective behavior in the optimal aggregation of noisy observations of a source. The quality of estimation of the source state involves a difficult tradeoff between sensing quality which increases by increasing the number of sensors, and aggregation quality which decreases if the number of sensors is too large. We analytically study the optimal strategy for large scale aggregation, and obtain an explicit and exact result by introducing a basic model. We show that larger scale aggregation always outperforms smaller scale aggregation at higher noise levels, while below a critical value of noise, there exist moderate scale aggregation levels at which optimal estimation is realized. We also examine the practical tradeoff between the above two aggregation strategies by applying an iterative encoding to linear codes.

We consider the problem of distributed sensing in a noisy environment; the CEO problem. Here individual sensor readings are encoded and transmitted independently by multiple sensors with a limited combined data rate. We present a scaling analysis of a semi-practical system using the sparse matrix codes, and show that the analysis is consistent with a general argument based on an existence of the rate distortion function. This approach well describes the tradeoff between reducing errors due to environmental noise and increasing errors due to lossy coding as the number of sensors increases, showing threshold behavior for optimal number of sensors

This paper provides the asymptotic analysis for the sparse matrix codes in the CEO problem. In this problem, a firm's chief executive officer (CEO) is interested in the data sequence which cannot be observed directly. Therefore, the CEO deploys a team of L agents who encodes his/her noisy observation of the data sequence without sharing any information. The CEO then collects all the L codeword sequences to recover the data sequence, where the combined data rate R at which the agents can communicate with the CEO is limited. In our scenario, each agent is supposed to use his/her LDPC-like code for lossy compression, while the CEO estimates each data bit by a majority vote of the L reproductions. The replica ansatz and the central limit theorem allow us to derive an analytical description of the problem in the case of large L. Here, the expected error frequency can be numerically evaluated for a given R, indicating that the optimum decentralization strategy depends largely on the bandwidth, as well as the observation noise level

We study an ill-posed linear inverse problem, where a binary sequence will be reproduced using a sparse matrix of LDPC type. We propose an iterative algorithm based on belief propagation technique. Numerical results show that the algorithm can empirically saturate the theoretical limit for the sparse construction of our codes, which also is very close to the rate-distortion function itself.

The Slepian-Wolf system is analyzed using statistical physics. In our framework, two correlated sequences are separately compressed into shorter codewords employing linear codes, which will be jointly decoded by a practical algorithm. Our analysis shows that the achievable rate region is described as phase transitions, assuming any kind of correlation.

We applied statistical mechanics to an inverse problem of linear mapping to investigate the physics of optimal lossy compressions. We used the replica symmetry breaking technique with a toy model to demonstrate Shannon’s result. The rate distortion function, which is widely known as the theoretical limit of the compression with a fidelity criterion, is derived. Numerical study shows that sparse constructions of the model provide suboptimal compressions.

This paper presents a new and fast segmentation algorithm, capable of incorporating expert's knowledge and expertise in a straightforward manner, to detect DNA spots in a restriction landmark genomic scanning (RLGS) profile. A RLGS profile is a gray scale image obtained in a X-ray film containing hundreds of dark spots corresponding to certain DNA fragments labeled by radioactive markers. Extracting these DNA spots is a very challenging task due to their fuzzy boundaries against noisy background regions having non-uniform intensity variations. The proposed algorithm, utilizing a support vector machine, is found to be successful to detect and extract DNA spots in a RLGS profile.