Online federated learning (FL) enables geographically distributed devices to learn a global shared model from locally available streaming data. Most online FL literature considers a best case scenario regarding the participating clients and the communication channels. However, these assumptions are often not met in real-world applications. Asynchronous settings can reflect a more realistic environment, such as heterogeneous client participation due to available computational power and battery constraints, as well as delays caused by communication channels or straggler devices. Further, in most applications, energy efficiency must be taken into consideration. Using the principles of partial-sharing-based communications, we propose a communication-efficient asynchronous online FL (PAO-Fed) strategy. By reducing the communication load of the participants, the proposed method renders participation more accessible and efficient. In addition, the proposed aggregation mechanism accounts for random participation, handles delayed updates, and mitigates their effect on accuracy. We study the first- and second-order convergence of the proposed PAO-Fed method and obtain an expression for its steady-state mean square deviation. Finally, we conduct comprehensive simulations to study the performance of the proposed method on both synthetic and real-life data sets. The simulations reveal that in asynchronous settings, the proposed PAO-Fed is able to achieve the same convergence properties as that of the online federated stochastic gradient while reducing the communication by 98%.

This article presents communication-efficient approaches to federated learning (FL) for resource-constrained devices with access to streaming data. In particular, we first propose a partial-sharing-based framework for online federated learning (PSO-Fed), wherein clients update local models from a stream of data and exchange tiny fractions of the model with the server, reducing the communication overhead. In contrast to classical FL approaches, the proposed strategy provides clients who are not part of a global iteration with the freedom to update local models whenever new data arrives. Furthermore, by devising a client-side innovation check, we also propose an event-triggered PSO-Fed (ETPSO-Fed) that further reduces the computational burden of clients while enhancing communication efficiency. We implement the above-mentioned frameworks in the context of kernel regression, where clients perform local learning employing random Fourier features (RFFs)-based kernel least mean squares. In addition, we examine the mean and mean-square convergence of the proposed PSO-Fed. Finally, we conduct experiments to determine the efficacy of the proposed frameworks. Our results show that PSO-Fed and ETPSO-Fed can compete with Online-Fed while requiring significantly less communication overhead. Simulations demonstrate an 80% reduction in PSO-Fed and an 84.5% reduction in ETPSO-Fed communication overhead compared to Online-Fed. Notably, the proposed PSO-Fed strategies show good resilience against model-poisoning attacks without involving additional mechanisms.

Federated learning (FL) literature typically assumes that each client has a fixed amount of data, which is unrealistic in many practical applications. Some recent works introduced a framework for online FL (Online-Fed) wherein clients perform model learning on streaming data and communicate the model to the server; however, they do not address the associated communication overhead. As a solution, this paper presents a partial-sharing-based online federated learning framework (PSO-Fed) that enables clients to update their local models using continuous streaming data and share only portions of those updated models with the server. During a global iteration of PSO-Fed, non-participant clients have the privilege to update their local models with new data. Here, we consider a global task of kernel regression, where clients use a random Fourier features-based kernel LMS on their data for local learning. We examine the mean convergence of the PSO-Fed for kernel regression. Experimental results show that PSO-Fed can achieve competitive performance with a significantly lower communication overhead than Online-Fed.

Many assumptions in the federated learning literature present a best-case scenario that can not be satisfied in most real-world applications. An asynchronous setting reflects the realistic environment in which federated learning methods must be able to operate reliably. Besides varying amounts of non-IID data at participants, the asynchronous setting models heterogeneous client participation due to available computational power and battery constraints and also accounts for delayed communications between clients and the server. To reduce the communication overhead associated with asynchronous online federated learning (ASO-Fed), we use the principles of partial-sharing-based communication. In this manner, we reduce the communication load of the participants and, therefore, render participation in the learning task more accessible. We prove the convergence of the proposed ASO-Fed and provide simulations to analyze its behavior further. The simulations reveal that, in the asynchronous setting, it is possible to achieve the same convergence as the federated stochastic gradient (Online-FedSGD) while reducing the communication tenfold.

Recent years have bore witness to the proliferation of distributed filtering techniques, where a collection of agents communicating over an ad-hoc network aim to collaboratively estimate and track the state of a system. These techniques form the enabling technology of modern multi-agent systems and have gained great importance in the engineering community. Although most distributed filtering techniques come with a set of stability and convergence criteria, the conditions imposed are found to be unnecessarily restrictive. The paradigm of stability and convergence in distributed filtering is revised in this manuscript. Accordingly, a general distributed filter is constructed and its estimation error dynamics is formulated. The conducted analysis demonstrates that conditions for achieving stable filtering operations are the same as those required in the centralized filtering setting. Finally, the concepts are demonstrated in a Kalman filtering framework and validated using simulation examples.

This paper presents a novel approach to assessing some of the electric vehicles(EVs) critical environmental conditions without relying on physical sensors. The proposed approach employs distributed estimation in networked vehicles to estimate parameters that characterize the EV's environmental conditions which impact the range of an EV. The parameters to be estimated are the highly uncertain quantities such as wind speed and rolling resistance. Key to this approach is the setmembership filtering (SMF) method, which is quite different from the traditional adaptive filtering methods. The SMF aims to meet a specification on the filtering error magnitude, instead of minimizing the time average or ensemble average of the squared errors. Moreover, the SMF algorithms feature datadependent selective update of parameter estimates, which is very useful in estimating the environmental parameters of networked vehicles. The estimation schemes developed here use a vehicle longitudinal model and commonly available vehicle sensor signals. Simulations are performed to demonstrate the advantages of the proposed algorithms in terms of computational complexity, residual parameter uncertainties, and steadystate mean-squared errors in comparison to existing algorithms.

A step-based tutoring system is being developed to improve student learning in elementary linear circuit analysis courses. The approach is based on the known importance of viewing worked examples or solutions in alternation with problem-solving exercises of the same type, to avoid excessive cognitive loads when learning a new skill. The system can generate an unlimited supply of circuit problems (with randomly varying topologies) similar to those in textbooks and corresponding fully explained solutions for both DC, AC, and transient analysis. Solution methods and topics include voltage and current division, node and mesh analysis, superposition, Thévenin and Norton equivalents, Bode plots, and Laplace analysis. Special pedagogical devices are used to reduce extraneous cognitive load when using the system, such as color-coding nodes and equation terms. Qualitative conceptual topics such as series and parallel connections are given special emphasis. The system has been used to date by over 2860 students in 71 class sections at 10 different colleges and universities of widely varying types, with generally very favorable student ratings. Evaluation has included a series of randomized, controlled trials in both laboratory and classroom settings to rigorously assess student learning gains as well as the impact of the materials on student motivation and preferences. Learning improvements of 0.72 std. deviations are found in comparison to a publisher-based homework system for node analysis, and 0.97 std. deviations for identification of elements in series and parallel compared to paper exercises.

This paper presents a dynamic pricing and energy management framework for electric vehicle (EV) charging service providers. To set the charging prices, the service providers faces three uncertainties: the volatility of wholesale electricity price, intermittent renewable energy generation, and spatial-temporal EV charging demand. The main objective of our work here is to help charging service providers to improve their total profits while enhancing customer satisfaction and maintaining power grid stability, taking into account those uncertainties. We employ a linear regression model to estimate the EV charging demand at each charging station, and introduce a quantitative measure for customer satisfaction. Both the greedy algorithm and the dynamic programming (DP) algorithm are employed to derive the optimal charging prices and determine how much electricity to be purchased from the wholesale market in each planning horizon. Simulation results show that DP algorithm achieves an increased profit (up to 9%) compared to the greedy algorithm (the benchmark algorithm) under certain scenarios. Additionally, we observe that the integration of a low-cost energy storage into the system can not only improve the profit, but also smooth out the charging price fluctuation, protecting the end customers from the volatile wholesale market.

The development, expansion, and assessment of a step-based computer-aided tutoring system to teach linear circuit analysis is described. Circuit problems and fully worked, error-free solutions to same are created automatically and randomly with any desired characteristics, varying both the layout and numerical values. Students therefore have access to an unlimited source of both examples and practice problems of variable difficulty. A unique feature is that students input each stage of their work in a variety of forms including redrawn circuit diagrams, equations, matrix equations, numerical answers, multiple choice answers, and sketches of time-dependent waveforms. Students receive immediate feedback on the correctness of each step of their work. The solutions use analytical methods, rather than the numerical methods used in a tool like PSPICE. A total of 15 tutorials are now available, including DC and AC circuits and two on Laplace transforms. The system has been used by over 1950 students to date. A randomized, controlled laboratory-based study finds a statistically significant learning gain of 1.21 standard deviations compared to working conventional textbook problems. Another randomized, controlled classroom study found a strong preference of students for this system over a widely-used commercial publisher-based homework system, as well as higher scores.

We develop a fully-distributed iterative algorithm for finding a model-distributed least-squares solution of systems of linear equations over sensor networks. Here, model-distributed means the solution vector is distributed across the network rather than being replicated at each node. For this purpose, we devise a dual regularized least-squares problem via a suitable decomposition of the normal equations associated with the original problem. The resultant dual problem can be solved in a fully-decentralized and iterative manner by means of the diffusion-based Pareto optimization strategy. We verify the usefulness of the proposed algorithm via both theoretical analysis and numerical examples.

This paper presents a multi-stage approach to the placement of charging stations under the scenarios of different electric vehicle (EV) penetration rates. The EV charging market is modeled as the oligopoly. A consumer behavior based approach is applied to forecast the charging demand of the charging stations using a nested logit model. The impacts of both the urban road network and the power grid network on charging station planning are also considered. At each planning stage, the optimal station placement strategy is derived through solving a Bayesian game among the service providers. To investigate the interplay of the travel pattern, the consumer behavior, urban road network, power grid network, and the charging station placement, a simulation platform (The EV Virtual City 1.0) is developed using Java on Repast.We conduct a case study in the San Pedro District of Los Angeles by importing the geographic and demographic data of that region into the platform. The simulation results demonstrate a strong consistency between the charging station placement and the traffic flow of EVs. The results also reveal an interesting phenomenon that service providers prefer clustering instead of spatial separation in this oligopoly market.

We consider the downlink of a wireless cellular network where the base stations are equipped with multiple antennas and operate in the same frequency band. Since scheduling changes the spatial transmit signal processing with each time slot, information from neighboring base stations is required for data encoding. This can, in theory, be accomplished by a high-capacity backhaul network through which the base stations exchange channel state information (CSI) and other control signals. In reality, however, the temporal granularity of the scheduler does not allow for timely distribution of CSI among base stations. We propose a two-phase scheduler which optimizes the precoding in the first phase and allows the users to feed back their instantaneous interference power in the second phase. For the single-user case, we present a practical scheme that combines two-phase scheduling with precoders that maximize the signal-to-leakage-plus-noise ratio. If the users feed back the interference power together with a supported rate, communication between base stations can be limited to integers. By comparing the performance to multi-user two-phase scheduling with dirty paper coding and to algorithms that share CSI among base stations we show that two-phase scheduling is a technically and practically feasible solution to deal with non-stationary intercell interference.

We purpose and analyze an off-line randomized sensor selection strategy for sequential hypothesis testing problem constrained with sensor measurement costs. Within the framework of Wald's approximation, the sequential probability ratio test (SPRT) with sensor selection is designed for minimizing the expected total measurement cost subject to reliability and sensor usage constraints. In the case of symmetric hypotheses, we introduce a quantity, called efficiency, of a sensor and show that it is critical to the sensor selection in SPRT. Furthermore, an algorithm with linear time complexity is proposed to obtain the optimal sensor selection probabilities.

We consider the downlink of a cellular wireless network where the base stations are equipped with multiple transmit antennas and operate in the same frequency band. Each base station serves one user per orthogonal frequency-division multiplexing (OFDM) sub-carrier by means of spatial precoding. Due to temporal scheduling, the active users change rapidly in time and consequently, intercell interference becomes non-stationary limiting the achievable network sum-rate. Under the assumption of perfect channel knowledge at each base station, we optimize both precoding and scheduling in order to maximize the achievable network sum-rate. The solution, however, unveils that both precoding and scheduling can be performed locally at each base station without the need for CSI to be exchanged between base stations if they are solely based on average channel measurements.

This paper proposes selective update and cooperation strategies for parameter estimation in distributed adaptive sensor networks. A set-membership filtering approach is employed that results in reduced complexity for updating parameter estimates at each network node, a significant reduction in information exchange between cooperating nodes, and an optimal strategy to obtain consensus estimates. The proposed strategies and the estimation algorithm offer a new way to explore cooperation in adaptive distributed sensor networks.

Both fast scheduling and spatial signal processing have proven to be capacity-increasing methods in wireless communication systems. However, when applied in the downlink of a cellular network, the combination of both leads to non- stationary intercell interference. If the base stations do not cooperate, either they have to encode the data very conservatively to gain robustness or the non-stationary fluctuations of the interference powers lead to frequent outages, both of which strongly impair the average achievable throughput. On the other hand, base station cooperation increases complexity and delays, contradicting the desire for fast scheduling algorithms. In this paper, we propose a scheme that makes average channel state information available to all base stations via low-rate back haul communication, whereas high-rate inter-base-station communication is limited to B lceillog2Krceil-bit integers, K being the number of users in each of the B cells. Simulations show that for slow fading channels, the proposed algorithm preserves most of the per cell sum-rate of other beamforming and dirty- paper coding approaches that have unlimited-capacity back haul links. Furthermore, when out-of-cell information is outdated the proposed algorithm even outperforms those.

This paper proposes a clustering approach to parameter estimation in distributed sensor networks. The proposed approach is an alternative to the conventional centralized and decentralized approaches. This is made possible by the unique adaptive estimation architecture, U-SHAPE, stemming from set-membership adaptive filtering. At the expense of a slightly degraded mean-square error performance (comparing to the least-squares approach), the proposed approach offers improved data processing flexibility in a distributed sensor network, reduced signal processing hardware and reduced communication bandwidth and power requirements.

Carrier frequency synchronization is critical to the quality of signal reception in OFDM systems. This paper presents an approximate maximum-likelihood (ML) carrier frequency offset (CFO) estimation scheme based on time-domain channel estimates which retain the CFO information in the form of phase rotation. The proposed ML CFO estimate is investigated under static as well as time-varying fading channels. Statistical properties of the estimator are examined and Cramer-Rao lower bound (CRLB) is derived. Theoretical analysis and numerical simulations show that the proposed CFO estimator renders excellent performance with lower computational complexity. The proposed CFO estimate also has an advantage of allowing for more flexible pilot patterns.

We consider amulti-usermultiple-inputsingle-output(MU-MISO) scenario, where the decentralized users are served by a centralized transmitter with multiple channel inputs via frequency selective vector channels. For this broadcast setup, we jointly design the transmitter and receivers based on a minimum summeansquareerror(MSE) criterion. Contrary to previous work in this field, we do not restrict the receivers to be scalar weights but employfiniteimpulseresponse(FIR) receive filters. Since the sum MSE minimization has no closed-form solution neither for linear preceding nor forTomlinson-Harashimaprecoding(THP), we propose to use an alternating optimization and prove the convergence of the resulting iterative algorithm. The simulations show that the obtained linear and nonlinear preceding solutions with FIR receivers clearly outperform the state-of-the-art precoders with scalar receivers.

This paper presents an opportunistic multiple-input multiple-output (MIMO) scheme that achieves increased throughput of multi-user downlink transmission in frequency-selective channels. Developed for OFDM systems, the main idea of the proposed scheme is to find the optimal subset of users for each subchannel and optimal transmit beamforming vector which would maximize the sum-rate. Assuming perfectly known channel information, an optimal solution is derived and a suboptimal solution with much lower complexity that provides comparable results to the optimal solution is also presented. The proposed scheme outperforms the existing FDMA-based and SDMA-based schemes significantly in terms of sum-rate by efficiently exploiting the time-varying nature of radio channels and multi-user diversity

The benefits of multiuser diversity for fading chanels are well established in the literature. What is not well documented is the effort required to obtain this CSI information at the scheduler for a frequency division duplex (FDD) system. This paper investigates the practical aspects of the feedback transfer process for a discrete rate-adaptive multiuser system including multi-access, feedback delay and feedback channel imperfections due to fading and additive noise. We compute the optimal power and bandwidth allocated for channel estimation and show that tradeoffs exist between these resources and the number of users depending on the feedback delay. It is also shown that quantization and feedback errors may increase the outage region as the number of users increases. This in effect reduces the multiuser diversity gain expected in ideal feedback channels.

Carrier frequency offset (CFO) is a major contributor to the inter-carrier interference in OFDM systems. This effect becomes more severe when compounded by the presence of Doppler fading in wireless channels. This paper presents two maximum-likelihood CFO estimation schemes, one in frequency domain and another in time-domain, both under Doppler fading. The properties of the estimators are analyzed and simulation results showing the performance gain of the proposed estimators over other estimation schemes are presented. Apart from improved accuracy, the time-domain maximum likelihood estimator features significantly reduced complexity

This letter analyzes the performance of an adaptive modulation system, taking into account additive noise and fading on the feedback channel. It is shown that these feedback channel imperfections could significantly degrade the throughput gains of adaptive modulation over nonadaptive transmission. Specifically, feedback errors can result in an outage region in the low signal-to-noise ratio region. Two feedback receivers are proposed: one is based on the finite-state Markov channel model; and the other is a generalized Bayesian receiver. These receivers reduce the outage region due to feedback errors, and they can complement or be used as alternatives to error-control coding schemes.

The paper considers adaptive transmission systems with imperfect feedback channels. It is shown that if the average SNR for the forward channel is known to the transmitter, the feedback detection can be modeled as a classical multiple hypotheses testing problem. Though maximum a posteriori detection is optimal compared to maximum likelihood (in the sense of minimum probability of error) from a feedback communication link viewpoint, the overall effect on adaptive system performance shows that ML detection could be better in the high SNR range. The MAP scheme is generalized to Bayesian detection by defining a cost function matrix, which assigns (unequal) weights to the feedback decision error probabilities. The result is close to ideal bit error rate (BER) performance with only a small drop in spectral efficiency.

This paper considers set-membership filtering (SMF) when the error-bound specification is hard to determine. Improper error-bound specification could cause overbounding or underbounding, both of which can result in degraded performance for SMF algorithms. This paper introduces a novel variable error bound and presents a different SMF criterion. It is shown that the recursive algorithm derived from this new set-membership filtering criterion has less risk of overbounding/underbounding and outperforms conventional SMF algorithms with one fixed error-bound specification, particularly when insufficient knowledge is available to determine the bound. The proposed algorithm is more suitable to time-variant environments. Frequency-domain equalization for broadband wireless communications is used as an example to illustrate the proposed criterion and recursive solution. Simulation results that show convergence performance and tracking of a time-variant channel are presented.

In linear filtering, the set-membership normalized least mean squares (SM-NLMS) algorithm has been shown to exhibit desirable features of selective update and optimized variable step size. In this paper, a kernel approach to the SM-NLMS algorithm is presented that makes it feasible to address nonlinear problems. An online greedy approximation technique to achieve sparsity is discussed. Simulation results are presented for two practical problems: equalization of nonlinear inter-symbol interference (ISI) channels and predistortion of nonlinear high power amplifiers (HPA).

In adaptive modulation systems, it is widely assumed that the feedback channel is error-free by employing sophisticated error-control codes and ARQ signaling. However, since feedback is sent over a wireless channel, a heavy price may be paid to justify this assumption. In this paper, we first analyze system performance, given additive noise and fading on the feedback channel. Then, by utilizing the finite-state Markov channel model for a slowly fading Rayleigh channel, we design differential and full-state feedback schemes to correct most of the errors caused by an AWGN channel. For a fading feedback channel, a Markov-based scheme is designed with a threshold level, below which feedback decisions are deemed unreliable. The resulting system performance in terms of the spectral efficiency and bit error rate (BER) is comparable to the ideal case of an error-free feedback channel.

This paper describes a location system using an asynchronous distributed sensor network. The sensors' clocks operate independently and are not explicitly synchronized. The motion of the source, or the object, between two emitted pulses will cause the inter-arrival time of these two pulses at each sensor to vary. This inter-arrival time is a function of the source location and motion and is independent of the clock offsets among sensors. Thus, the independently calculated time-difference-of-arrival (TDOA) of successive pulses at each sensor provides a reliable source of information to estimate the source position as well as its motion. The Cramer-Rao lower bound (CRLB) is derived to assess the performance of the proposed method and to show that its performance can approach that of a synchronous network given a sufficient number of sensors and moderate source movement. Some sub-optimal but effective source trajectory tracking methods are proposed which further relax the assumption of source motion between consecutive pulses. The simulation results show that the proposed estimation and tracking methods are quite good.

This paper presents a novel multiple antenna transmission scheme that depends on channel state information. The proposed approach uses low-rate feedback to adaptively estimate the downlink fading channel at the base-station. The optimal transmission scheme given channel state information can in turn be determined from these channel estimates. It is shown that the channel estimation requires selective feedback of the adaptive filter prediction errors, that can be computed at the mobile by receiving a beamformed pilot signal. To this end, a randomized channel inversion pilot beamforming scheme is developed that makes possible this feedback of prediction errors possible. Further, this method is shown to be relatively insensitive to feedback quantization effects for a given average feedback rate. Simulation results indicate that the feedback scheme provides significant closed-loop beamforming gains with low bit rate feedback even for large number of transmit antennas.

An opportunistic transmission scheduling scheme is proposed to make better use of the multiuser diversity gain in a MIMO system. The performance of a MIMO link is quantified by its information-theoretic capacity. Even though the base station has no knowledge of the transmission channel, the proposed algorithm generates a transmission randomizing matrix according to the distribution of the channel, so that it is likely that some user is near the "water-filling" configuration. The mobile users measure and feedback the channel quality. Based on this information, the base station schedules transmission to the user whose instantaneous channel capacity is the largest. Simulation results show that, over slow varying channels, or over Rician channels with a large K factor, opportunistic transmission scheduling can improve system performance in terms of channel capacity.

This paper examines a downlink transmit beamforming scheme we proposed recently (see Nagaraj, S. and Huang, Y.F., Asilomar Conference on Signals, Systems and Computers, 2000). The idea is based on the use of an adaptive channel estimator at the base-station and requires the mobile selectively to feed back the value of the prediction error. This paper proposes a joint maximum-likelihood and set-membership filtering algorithm for adaptive channel estimation that provides robustness against incorrect feedback due to mobile decision errors. The amount of power and bandwidth saving possible with this scheme is quantified via an uplink capacity analysis.

This paper presents the blind equalization problem in the novel deterministic framework of set-membership filtering (SMF). An adaptive algorithm based on a deterministic version of the constant modulus criterion is derived to seek the set-membership blind equalization solution. The proposed set-membership constant modulus (SM-CM) algorithm incorporates selection of data-dependent optimal step-size at every time and a selective update strategy. The performance and features of the proposed SM-CM algorithm are compared with those of the conventional CMA. Results indicate that the SM-CM algorithm outperforms the CMA in convergence speed, steady-state performance and average computational complexity. Furthermore, unlike the conventional CMA, the proposed algorithm does not require the step-size to be set heuristically.

In this paper, we formulate the continuous l/sub /spl infin// minimax filtering problem, which has several practical applications in optimal worst-case filter design. Given a continuous minimax problem, we show that a discrete l/sub /spl infin// minimax filtering problem can be constructed that yields the same solution as the original continuous problem with probability one. Several practical applications of the continuous l/sub /spl infin// minimax filter are also presented.

This paper studies an application of turbo codes to compressed image/video transmission and presents an approach to improving error control performance through joint channel and source decoding (JCSD). The proposed approach to JCSD includes error-free source information feedback, error-detected source information feedback, and the use of channel soft values (CSV) for source signal postprocessing. These feedback schemes are based on a modification of the extrinsic information passed between the constituent maximum a posteriori probability (MAP) decoders in a turbo decoder. The modification is made according to the source information obtained from the source signal processor. The CSVs are considered as reliability information on the hard decisions and are further used for error recovery in the reconstructed signals. Applications of this joint decoding technique to different visual source coding schemes, such as spatial vector quantization, JPEG coding, and MPEG coding, are examined. Experimental results show that up to 0.6 dB of channel SNR reduction can be achieved by the joint decoder without increasing computational cost for various channel coding rates.

This paper introduces the notion of nonlinear MMSE multiuser detection and shows that MMSE signal estimates followed by single user detectors yield MAP (or minimum probability of error) decisions for CDMA signals. Iterative solutions are proposed for nonlinear MMSE estimation. If interferers' codes are assumed to be known by the signal estimators for each user, the MMSE solution is shown to be a fixed point which is reached in one iteration, but is computationally intractable. If interferers' codes are not assumed to be known, the resulting iterative multiuser detector (called random-code NMIC) is shown to be of the same order of complexity as a conventional multistage interference canceler. Furthermore, the nonlinear MMSE solution is shown to be a fixed point of the random-code NMIC. Particular solutions are presented for the cases of BPSK and M-ary orthogonal spread spectrum systems.

This paper presents a lossy image compression scheme that employs a generalized rank-ordered prediction filter for pyramidal image coding. The proposed prediction method renders significantly reduced variance of the quantizer input. Consequently, the quality of the decompressed image is much enhanced due to the greatly reduced quantization distortion. Both analytical and simulation results show that the proposed scheme yields high-quality performance.

The joint channel-source decoding scheme for using turbo codes to protect compressed image data proposed by Peng, Huang, Costello and Stevenson (see, Proc. 1998 IEEE International Symposium on Circuits and Systems, Monterey, California, 1998) is modified for subband coded images. Two different modifications are presented and compared. The factors affecting the performance of the schemes are studied. Both modified schemes show superiority over a separate decoding system.

This paper is intended to investigate the bit rate allocation between source coding and channel coding rates when turbo codes are used to protect vector quantized images. The experimental results show that an appropriate bit rate allocation can lead to significant gains in the quality of the reconstructed images compared to an arbitrary bit rate allocation scheme.

Set-membership identification (SMI) theory is extended to the more general problem of linear-in-parameters filtering by defining a set-membership specification, as opposed to a bounded noise assumption. This sets the framework for several important filtering problems that are not modeled by a "true" unknown system with bounded noise, such as adaptive equalization, to exploit the unique advantages of SMI algorithms. A recursive solution for set membership filtering is derived that resembles a variable step size normalized least mean squares (NLMS) algorithm. Interesting properties of the algorithm, such as asymptotic cessation of updates and monotonically non-increasing parameter error, are established. Simulations show significant performance improvement in varied environments with a greatly reduced number of updates.

Compressed images transmitted over noisy channels are extremely sensitive to bit errors. This necessitates the application of error control channel coding to the compressed representation before transmission. This paper presents an image transmission system which takes advantage of the superior performance of turbo codes, an important new class of parallel concatenated codes. Several aspects of the application of turbo codes to image transmission are studied, including comparison to a previous image transmission system using convolutional codes. Experimental results for several channel signal-to-noise ratios show that, in the same SNR range, turbo codes achieve much better performance with less decoding complexity than convolutional codes and that similar performance can be achieved at much lower channel SNRs. Studies also show that the use of feedback from an outer Reed-Solomon code to aid turbo decoding results in further improvement.

This paper presents a lossy image compression scheme which employs a pyramidal coding technique with segmentation based on the local activity levels. A novel prediction method is introduced, rendering significantly reduced variance of the quantizer input. Consequently, the coding efficiency is much enhanced due to the greatly reduced quantization distortion. Both theoretical analysis and simulation results show that the proposed scheme yields high quality performance. A reconstruction PSNR of 30.5 dB is achieved at a bit rate of 0.181 bpp for a 512×512 image "Lenna" before entropy coding.

The sensitivity of the compressed image representation to bit errors requires application of a channel code before transmission over noisy channels. To prevent the uncontrolled degradation caused by a channel error, an error controlling channel code is applied to the compressed representation before transmission. Many image communication systems have constraints on bandwidth, power and time which prohibit transmission of uncompressed raw image data. Compressed image formats, however, are extremely sensitive to bit errors which can seriously degrade the quality of the image at the receiver. A new list-based iterative trellis decoder is proposed which accepts feedback from a post-processor which can detect channel errors in the reconstructed image. Experimental results are shown which indicate the new decoder provides significant improvement over the standard Viterbi decoder.

This paper presents an image communication system with improved decoding of compressed image information. A convolutional code protects the compressed image information from channel noise while a Reed-Solomon outer code gives additional protection to the critical image header information. A post-processor detects uncorrected channel errors in the reconstructed image and feeds error location information to a list-based iterative trellis decoder. This list-based decoder provides significant improvement in image quality. Experimental results are given for varying channel SNR and for varying bit rate.

Unlike the conventional stochastic approach, an unsupervised learning algorithm is viewed as a deterministic system. A new concept of time-average invariance is introduced, which is a property of deterministic signals, but plays the role of stochastic signals that are stationary and ergodic. As such, deterministic-based analysis can be used for stochastic-like signals. Consequently, the complexity of convergence analysis is significantly reduced. The simplicity of the main theorem also suggests the possibility for the design of unsupervised learning algorithms. Two examples are given for illustration.

An unsupervised neural network is constructed for the problem of blind signal separation. It is designed based on the condition that the outputs of the neural network are independent. A study of the stability of the neural network in the sense of expectation is presented. A stability condition on the system matrix A is obtained. Simulation studies show that this neural network is robust. It can separate two signals with strength ratio 100:1.>

One of the major obstacles to the use of vector quantization techniques in practice is the huge design complexity. Huang and Huang (1991, 1992, 1993) and Huang (1991) developed a tree-structured polytopal vector quantization scheme referred to as the principal component vector quantization algorithm (PCVQA). It has been shown that the design complexity of PCVQA is only linearly proportional to the codebook size. The present paper examines various techniques to further reduce the implementational complexity for PCVQA so that the algorithm is suitable for real-time codebook retransmission. In addition, an extension of PCVQA is developed to exploit both inter-block and intra-block redundancy and to deal with high definition images.>

This paper addresses a signal detection problem based on a class of quantizers. The known moments of the underlying noise distribution are used to index a set of quantization points that have been predetermined under an assumed noise model. The fact that lower order moments are easy to obtain and that they are required in the implementation and analysis of most threshold detectors makes this approach quite appealing. The performance of the resulting quantizers is shown to be relatively insensitive to variations in the underlying noise distribution and to small deviations of the presumed moments. The detector's performance in the finite-sample-size case is investigated, and the distribution resulting in the highest false alarm rate is described for both symmetric and asymmetric noise cases. By setting the test threshold according to the worst distribution, a lower bound on the detector's performance is guaranteed.>

Fundamental issues concerning the capability of multilayer perceptrons with one hidden layer are investigated. The studies are focused on realizations of functions which map from a finite subset of E(n) into E(d). Real-valued and binary-valued functions are considered. In particular, a least upper bound is derived for the number of hidden neurons needed to realize an arbitrary function which maps from a finite subset of E(n ) into E(d). A nontrivial lower bound is also obtained for realizations of injective functions. This result can be applied in studies of pattern recognition and database retrieval. An upper bound is given for realizing binary-valued functions that are related to pattern-classification problems.

An analysis of error propagation in the OBE (optimal bounding ellipsoid) algorithm shows that roundoff errors made at some instant of time would result in bounded errors in the subsequent estimates. A modified forgetting factor determination formula that is less sensitive to rounding errors is proposed. It is shown through simulations that the OBE algorithm performs better than the RLS (recursive least squares) algorithm in small wordlength environments. The improvement is attributed to differences in the recursion of the P matrix and the smaller number of updates of the OBE algorithm. The performance of the factored implementations of the RLS and the OBE algorithms is comparable.>

A new recursive estimation algorithm which has been used for the parameter estimation of ARX models is extended to the more general ARMAX case. Sufficient conditions for the convergence of the algorithm are derived. Simulation results show that consistent parameter estimates are obtained for a large range of model parameters.

Continual updating of estimates required by most recursive estimation schemes often involves redundant usage of information and may result in system instabilities in the presence of bounded output disturbances. An algorithm which eliminates these difficulties is investigated. Based on a set theoretic assumption, the algorithm yields modified least-squares estimates with a forgetting factor. It updates the estimates selectively depending on whether the observed data contain sufficient information. The information evaluation required at each step involves very simple computations. In addition, the parameter estimates are shown to converge asymptotically, at an exponential rate, to a region around the true parameter.