Studies on backdoor attacks in recent years suggest that an adversary can compromise the integrity of a deep neural network (DNN) by manipulating a small set of training samples.Our analysis shows that such manipulation can make the backdoor model converge to a bad local minima, i.e., sharper minima as compared to a benign model.Intuitively, the backdoor can be purified by re-optimizing the model to smoother minima.However, a naïve adoption of any optimization targeting smoother minima can lead to sub-optimal purification techniques hampering the clean test accuracy.Hence, to effectively obtain such re-optimization, inspired by our novel perspective establishing the connection between backdoor removal and loss smoothness, we propose Fisher Information guided Purification (FIP), a novel backdoor purification framework.Proposed FIP consists of a couple of novel regularizers that aid the model in suppressing the backdoor effects and retaining the acquired knowledge of clean data distribution throughout the backdoor removal procedure through exploiting the knowledge of Fisher Information Matrix (FIM).In addition, we introduce an efficient variant of FIP, dubbed as Fast FIP, which reduces the number of tunable parameters significantly and obtains an impressive runtime gain of almost 5×.Extensive experiments show that the proposed method achieves state-of-the-art (SOTA) performance on a wide range of backdoor defense benchmarks: 5 different tasks-Image Recognition, Object Detection, Video Action Recognition, 3D point Cloud, Language Generation; 11 different datasets including ImageNet, PASCAL VOC, UCF101; diverse model architectures spanning both CNN and vision transformer; 14 different backdoor attacks, e.g., Dynamic, WaNet, LIRA, ISSBA, etc.Our code is available in this GitHub Repository.

Deep learning models are susceptible to contrived adversarial examples, even in the decision-based black-box setting where the attacker has access to the model’s decisions only. Developing more efficient and practical attacks help in better understanding the limitations of deep models. It is important that attacks are crafted with limited queries to avoid suspicion. Since the required number of queries increase with dimensions, low-dimensional embeddings are attractive. This low query budget constraint is a bottleneck for learning-based and data-driven attacks which rely heavily on querying the model. We propose LSDAT, an image-agnostic non-data-driven decision-based black-box attack that exploits low-rank and sparse decomposition (LSD) of images to dramatically reduce the queries and improve fooling rates compared to existing methods. LSDAT crafts perturbations in the low-dimensional subspace formed by the sparse component of the input image and that of a target adversarial image to obtain query-efficiency. A viable perturbation is obtained by traversing the path between the input and adversarial sparse components. Theoretical analyses are provided to justify the functionality of LSDAT. Unlike other competitors (e.g., FFT), LSD works directly in the image domain to guarantee that non-$\ell _{2}$constraints, such as sparsity, are satisfied. LSDAT offers better control over the number of queries and is computationally efficient as it performs sparse decomposition of the input and adversarial images only once to generate all queries. Four variants of LSDAT are presented for different scenarios including a pure black-box attack where no queries are allowed. We demonstrate$\ell _{0}$,$\ell _{2}$and$\ell _{\infty} $bounded attacks with LSDAT to evince its efficiency compared to baseline attacks in diverse low-query budget scenarios. LSDAT obtains 15 to 20% improvement in fooling ResNet-50 while using far fewer queries than competing methods in a similar setting.

This paper presents a new method for user localization in single-site massive Multiple-Input-Multiple-Output (MIMO) systems, which circumvents the need for large labeled datasets typically required for training data-driven models. Instead, the proposed model utilizes a limited set of geo-tagged Channel State Information (CSI) samples for training. The approach combines a Fully-Connected Auto-Encoder (FC-AE) with a Gaussian Process Regression (GPR) model. The GPR model is efficient, as it requires only a minimal amount of labeled data for training, although it presents challenges in computational complexity. To address this complexity, the FC-AE is introduced, which encodes the Angle-Delay Profile (ADP) transformation of the CSI data. The training dataset for the FC-AE is crafted by employing data augmentation techniques on a small collection of unlabeled data. The simulation results demonstrate that FC-AE is scenario-independent and adaptable to new scenarios with similar ADP characteristics. Additionally, our FC-AE-GPR model surpasses the performance of the Convolutional Neural Network model and the non-parametric grid search method when provided with limited labeled data, applicable in both indoor and outdoor settings.

Deep neural networks (DNNs) in the wireless communication domain have been shown to be hardly generalizable to scenarios where the train and test datasets follow a different distribution. This lack of generalization poses a significant hurdle to the practical utilization of DNNs in wireless communication. In this paper, we propose a generalizable deep learning approach for millimeter wave (mmWave) beam selection using sub-6 GHz channel state information (CSI) measurements, referred to as PARAMOUNT. First, we provide a detailed discussion on physical aspects of the electromagnetic wave scattering in the mmWave and sub-6 GHz bands. Based on this discussion, we develop the augmented discrete angle delay profile (ADADP) which is a novel linear transformation for the sub-6 GHz CSI that extracts the angle-delay attributes and provides a semantic visual representation of the multi-path clusters. Next, we introduce a convolutional neural network (CNN) structure that can learn the signatures of the path clusters in the sub-6 GHz ADADP representation and transform it to mmWave band beam indices. We demonstrate by extensive simulations on several different datasets that PARAMOUNT can generalize beyond the training dataset which is mainly due to transfer learning principles that allow transferring information from previously learned tasks to the learning of new unseen tasks.

Unsupervised domain adaptation (UDA) approaches focus on adapting models trained on a labeled source domain to an unlabeled target domain. In contrast to UDA, source-free domain adaptation (SFDA) is a more practical setup as access to source data is no longer required during adaptation. Recent state-of-the-art (SOTA) methods on SFDA mostly focus on pseudo-label refinement based self-training which generally suffers from two issues: i) inevitable occurrence of noisy pseudo-labels that could lead to early training time memorization, ii) refinement process requires maintaining a memory bank which creates a significant burden in resource constraint scenarios. To address these concerns, we propose C-SFDA, a curriculum learning aided self-training framework for SFDA that adapts efficiently and reliably to changes across domains based on selective pseudo-labeling. Specifically, we employ a curriculum learning scheme to promote learning from a restricted amount of pseudo labels selected based on their reliabilities. This simple yet effective step successfully prevents label noise propagation during different stages of adaptation and eliminates the need for costly memory-bank based label refinement. Our extensive experimental evaluations on both image recognition and semantic segmentation tasks confirm the effectiveness of our method. C-SFDA is also applicable to online test-time domain adaptation and outperforms previous SOTA methods in this task.

Supervised deep learning methods require a large repository of annotated data; hence, label noise is inevitable. Training with such noisy data negatively impacts the generalization performance of deep neural networks. To combat label noise, recent state-of-the-art methods employ some sort of sample selection mechanism to select a possibly clean subset of data. Next, an off-the-shelf semi-supervised learning method is used for training where rejected samples are treated as unlabeled data. Our comprehensive analysis shows that current selection methods disproportionately select samples from easy (fast learnable) classes while rejecting those from relatively harder ones. This creates class imbalance in the selected clean set and in turn, deteriorates performance under high label noise. In this work, we propose UNICON, a simple yet effective sample selection method which is robust to high label noise. To address the disproportionate selection of easy and hard samples, we introduce a Jensen-Shannon divergence based uniform selection mechanism which does not require any probabilistic modeling and hyperparameter tuning. We complement our selection method with contrastive learning to further combat the memorization of noisy labels. Extensive experimentation on multiple benchmark datasets demonstrates the effectiveness of UNICON; we obtain an 11.4% improvement over the current state-of-the-art on CIFAR100 dataset with a 90% noise rate. Our code is publicly available.11https://github.com/nazmul-karim170/UNICON-Noisy-Label

Even though convolutional neural networks (CNNs) can classify objects in images very accurately, it is well known that the attention of the network may not always be on the semantically important regions of the scene. It has been observed that networks often learn background textures, which are not relevant to the object of interest. In turn this makes the networks susceptible to variations and changes in the background which may negatively affect their performance.We propose a new three-step training procedure called split training to reduce this bias in CNNs for object recognition using Infrared imagery and Color (RGB) data. Our split training procedure has three steps. First, a baseline model is trained to recognize objects in images without background, and the activations produced by the higher layers are observed. Next, a second network is trained using Mean Square Error (MSE) loss to produce the same activations, but in response to the objects embedded in background. This forces the second network to ignore the background while focusing on the object of interest. Finally, with layers producing the activations frozen, the rest of the second network is trained using cross-entropy loss to classify the objects in images with background. Our training method outperforms the traditional training procedure in both a simple CNN architecture, as well as for deep CNNs like VGG and DenseNet, and learns to mimic human vision which focuses more on shape and structure than background with higher accuracy.

Deep visual models are known to be vulnerable to adversarial attacks. The last few years have seen numerous techniques to compute adversarial inputs for these models. However, there are still under-explored avenues in this critical research direction. Among those is the estimation of adversarial textures for 3D models in an end-to-end optimization scheme. In this paper, we propose such a scheme to generate adversarial textures for 3D models that are highly transferable and invariant to different camera views and lighting conditions. Our method makes use of neural rendering with explicit control over the model texture and background. We ensure transferability of the adversarial textures by employing an ensemble of robust and non-robust models. Our technique utilizes 3D models as a proxy to simulate closer to real-life conditions, in contrast to conventional use of 2D images for adversarial attacks. We show the efficacy of our method with extensive experiments.

This paper presents a novel antenna configuration to measure directions of multiple signal sources at the receiver in a THz mobile network via a single channel measurement. Directional communication is an intrinsic attribute of THz wireless networks and the knowledge of direction should be harvested continuously to maintain link quality. Direction discovery can potentially impose an immense burden on the network that limits its communication capacity exceedingly. To utterly mitigate direction discovery overhead, we propose a novel technique called spectrum shaping capable of measuring direction, power, and relative distance of propagation paths via a single measurement. We demonstrate that the proposed technique is also able to measure the transmitter antenna orientation. We evaluate the performance of the proposed design in several scenarios and show that the introduced technique performs similar to a large array of antennas while attaining a much simpler hardware architecture. Results show that the spectrum shaping with only two antennas placed 0.5 mm, 5 mm, and 1 cm apart performs direction of arrival estimation similar to a much more complex uniform linear array equipped with 7, 60, and 120 antennas, respectively.

The goal of out-of-distribution (OOD) detection is to handle the situations where the test samples are drawn from a different distribution than the training data. In this paper, we argue that OOD samples can be detected more easily if the training data is embedded into a low-dimensional space, such that the embedded training samples lie on a union of 1-dimensional subspaces. We show that such embedding of the in-distribution (ID) samples provides us with two main advantages. First, due to compact representation in the feature space, OOD samples are less likely to occupy the same region as the known classes. Second, the first singular vector of ID samples belonging to a 1-dimensional subspace can be used as their robust representative. Motivated by these observations, we train a deep neural network such that the ID samples are embedded onto a union of 1-dimensional subspaces. At the test time, employing sampling techniques used for approximate Bayesian inference in deep learning, input samples are detected as OOD if they occupy the region corresponding to the ID samples with probability 0. Spectral components of the ID samples are used as robust representative of this region. Our method does not have any hyperparameter to be tuned using extra information and it can be applied on different modalities with minimal change. The effectiveness of the proposed method is demonstrated on different benchmark datasets, both in the image and video classification domains.

This paper presents a new map-assisted localization approach utilizing Chanel State Information (CSI) in Massive Multiple-Input Multiple-Output (MIMO) systems. Map-assisted localization is an environment-aware approach in which the communication system has information regarding the surrounding environment. By combining radio frequency ray tracing parameters of the multipath components (MPC) with the environment map, it is possible to accomplish localization. Unfortunately, in real-world scenarios, ray tracing parameters are typically not explicitly available. Thus, additional complexity is added at a base station to obtain this information. On the other hand, CSI is a common communication parameter, usually estimated for any communication channel. In this work, we leverage the already available CSI data to propose a novel map-assisted CSI localization approach, referred to as MAP-CSI. We show that Angle-of-Departure (AoD) and Time-of-Arrival (ToA) can be extracted from CSI and then be used in combination with the environment map to localize the user. We perform simulations on a public MIMO dataset and show that our method works for both line-of-sight (LOS) and non-line-of-sight (NLOS) scenarios. We compare our method to the state-of-the-art (SoA) method that uses the ray tracing data. Using MAP-CSI, we accomplish an average localization error of 1.8 m in LOS and 2.8 m in mixed (combination of LOS and NLOS samples) scenarios. On the other hand, SoA ray tracing has an average error of 1.0 m and 2.2 m, respectively, but requires explicit AoD and ToA information to perform the localization task.

Current face image retrieval solutions are limited, since they treat different facial attributes the same and cannot incorporate user’s preference for a subset of attributes in their search criteria. This paper introduces a new face image retrieval framework, where the input face query is augmented by both an adjustment vector that specifies the desired modifications to the facial attributes, and a preference vector that assigns different levels of importance to different attributes. For example, a user can ask for retrieving images similar to a query image, but with a different hair color, and no preference for absence/presence of eyeglasses in the results. To achieve this, we propose to disentangle the semantics, corresponding to various attributes, by learning a set of sparse and orthogonal basis vectors in the latent space of StyleGAN. Such basis vectors are then employed to decompose the dissimilarity between face images in terms of dissimilarity between their attributes, assign preference to the attributes, and adjust the attributes in the query. Enforcing sparsity on the basis vectors helps us to disentangle the latent space and adjust each attribute independently from other attributes, while enforcing orthogonality facilitates preference assignment and the dissimilarity decomposition. The effectiveness of our approach is illustrated by achieving state-of-the-art results for the face image retrieval task.

This paper presents a data-driven localization framework with high precision in time-varying complex multi-path environments, such as dense urban areas and indoors, where GPS and model-based localization techniques come short. We consider the angle-delay profile (ADP), a linear transformation of channel state information (CSI), in massive MIMO systems and show that ADPs preserve users' motion when stacked temporally. We discuss that given a static environment, future frames of ADP time-series are predictable employing a video frame prediction algorithm. We express that a deep convolutional neural network (DCNN) can be employed to learn the background static scattering environment. To detect foreground changes in the environment, corresponding to path blockage or addition, we introduce an algorithm taking advantage of the trained DCNN. Furthermore, we present DyLoc, a data-driven framework to recover distorted ADPs due to foreground changes and to obtain precise location estimations. We evaluate the performance of DyLoc in several dynamic scenarios employing DeepMIMO dataset [1] to generate geo-tagged CSI datasets for indoor and outdoor environments. We show that previous DCNN-based techniques fail to perform with desirable accuracy in dynamic environments, while DyLoc pursues localization precisely. Moreover, simulations show that as the environment gets richer in terms of the number of multipath, DyLoc gets more robust to foreground changes.

Augmenting neural networks with skip connections, as introduced in the so-called ResNet architecture, surprised the community by enabling the training of networks of more than 1,000 layers with significant performance gains. This paper deciphers ResNet by analyzing the effect of skip connections, and puts forward new theoretical results on the advantages of identity skip connections in neural networks. We prove that the skip connections in the residual blocks facilitate preserving the norm of the gradient, and lead to stable back-propagation, which is desirable from optimization perspective. We also show that, perhaps surprisingly, as more residual blocks are stacked, the norm-preservation of the network is enhanced. Our theoretical arguments are supported by extensive empirical evidence. Can we push for extra norm-preservation? We answer this question by proposing an efficient method to regularize the singular values of the convolution operator and making the ResNet's transition layers extra norm-preserving. Our numerical investigations demonstrate that the learning dynamics and the classification performance of ResNet can be improved by making it even more norm preserving. Our results and the introduced modification for ResNet, referred to as Procrustes ResNets, can be used as a guide for training deeper networks and can also inspire new deeper architectures.

Along with the success of deep neural network (DNN) models, rise the threats to the integrity of these models. A recent threat is the Trojan attack where an attacker interferes with the training pipeline by inserting triggers into some of the training samples and trains the model to act maliciously only for samples that contain the trigger. Since the knowledge of triggers is privy to the attacker, detection of Trojan networks is challenging. Existing Trojan detectors make strong assumptions about the types of triggers and attacks. We propose a detector that is based on the analysis of the intrinsic DNN properties; that are affected due to the Trojan insertion process. For a comprehensive analysis, we develop Odyssey, the most diverse dataset to date with over 3,000 clean and Trojan models. Odyssey covers a large spectrum of attacks; generated by leveraging the versatility in trigger designs and source to target class mappings. Our analysis results show that Trojan attacks affect the classifier margin and shape of decision boundary around the manifold of clean data. Exploiting these two factors, we propose an efficient Trojan detector that operates without any knowledge of the attack and significantly outperforms existing methods. Through a comprehensive set of experiments we demonstrate the efficacy of the detector on cross model architectures, unseen Triggers and regularized models.

Convolutional neural networks (CNNs) have become a key asset to most of fields in AI. Despite their successful performance, CNNs suffer from a major drawback. They fail to capture the hierarchy of spatial relation among different parts of an entity. As a remedy to this problem, the idea of capsules was proposed by Hinton. In this paper, we propose the SubSpace Capsule Network (SCN) that exploits the idea of capsule networks to model possible variations in the appearance or implicitly-defined properties of an entity through a group of capsule subspaces instead of simply grouping neurons to create capsules. A capsule is created by projecting an input feature vector from a lower layer onto the capsule subspace using a learnable transformation. This transformation finds the degree of alignment of the input with the properties modeled by the capsule subspace.We show that SCN is a general capsule network that can successfully be applied to both discriminative and generative models without incurring computational overhead compared to CNN during test time. Effectiveness of SCN is evaluated through a comprehensive set of experiments on supervised image classification, semi-supervised image classification and high-resolution image generation tasks using the generative adversarial network (GAN) framework. SCN significantly improves the performance of the baseline models in all 3 tasks.

Finding a small subset of data whose linear combination spans other data points, also called column subset selection problem (CSSP), is an important open problem in computer science with many applications in computer vision and deep learning. There are some studies that solve CSSP in a polynomial time complexity w.r.t. the size of the original dataset. A simple and efficient selection algorithm with a linear complexity order, referred to as spectrum pursuit (SP), is proposed that pursuits spectral components of the dataset using available sample points. The proposed non-greedy algorithm aims to iteratively find K data samples whose span is close to that of the first K spectral components of entire data. SP has no parameter to be fine tuned and this desirable property makes it problem-independent. The simplicity of SP enables us to extend the underlying linear model to more complex models such as nonlinear manifolds and graph-based models. The nonlinear extension of SP is introduced as kernel-SP (KSP). The superiority of the proposed algorithms is demonstrated in a wide range of applications.

A wavelet network (WN) is a feed-forward neural network that uses wavelets as activation functions for the neurons in its hidden layer. By predetermining the wavelet positions and dilations, the WN can turn into a linear regression model. The common approach for the construction of these WN families is to use least-squares type algorithms. In this letter, we propose a novel approach by formulating a WN as a sparse linear regression problem, which we call a sparse wavelet network (SWN). In this WN, the problem of calculating the unknown inner parameters of the network becomes that of finding the sparse solution of an under-determined system of linear equations. Our sparse solution algorithm is a non-convex sparse relaxation approach inspired by smoothed L0 (SL0), a distinguished sparse recovery algorithm. The proposed SWN can be applied as a tool for the prediction and identification of dynamical systems.

Collaborative estimation of a sparse vector x by M potential measurements is considered. Each measurement is the projection of x obtained by a regressor, i.e., ym1/4 amTx. The problem of selecting K sensor measurements from a set of M potential sensors is studied where K ≫ M and K is less than the dimension of x. In other words, we aim to reduce the problem to an under-determined system of equations in which a sparse solution is desired. This paper suggests selecting sensors in a way that the reduced matrix construct a well conditioned measurement matrix. Our criterion is based on E-optimality, which is highly related to the restricted isometry property that provides some guarantees for sparse solution obtained by ℓ1minimization. The proposed basic E-optimal selection is vulnerable to outlier and noisy data. The robust version of the algorithm is presented for distributed selection for big data sets. Moreover, an online implementation is proposed that involves partially observed measurements in a sequential manner. Our simulation results show the proposed method outperforms the other criteria for collaborative spectrum sensing in cognitive radio networks (CRNs). Our suggested selection method is evaluated in machine learning applications. It is used to pick up the most informative features/data. Specifically, the proposed method is exploited for face recognition with partial training data.

The goal of data selection is to capture the most structural information from a set of data. This paper presents a fast and accurate data selection method, in which the selected samples are optimized to span the subspace of all data. We propose a new selection algorithm, referred to as iterative projection and matching (IPM), with linear complexity w.r.t. the number of data, and without any parameter to be tuned. In our algorithm, at each iteration, the maximum information from the structure of the data is captured by one selected sample, and the captured information is neglected in the next iterations by projection on the null-space of previously selected samples. The computational efficiency and the selection accuracy of our proposed algorithm outperform those of the conventional methods. Furthermore, the superiority of the proposed algorithm is shown on active learning for video action recognition dataset on UCF-101; learning using representatives on ImageNet; training a generative adversarial network (GAN) to generate multi-view images from a single-view input on CMU Multi-PIE dataset; and video summarization on UTE Egocentric dataset.

Recently, the promising aspects of compressive sensing have inspired new circuit-level approaches for their efficient realization within the literature. However, most of these recent advances involving novel sampling techniques have been proposed without considering hardware and signal constraints. Additionally, traditional hardware designs for generating non-uniform sampling clock incur large area overhead and power dissipation. Herein, we propose a novel non-uniform clock generator called Adaptive Quantization Rate (AQR) generator using Magnetic Random Access Memory (MRAM)-based stochastic oscillator devices. Our proposed AQR generator provides ~25-fold reduction in area, on average, while offering ~6-fold reduced power dissipation, on average, compared to the state-of-the-art non-uniform clock generators.

Utilization of directional antennas is a promising solution for efficient spectrum sensing and accurate source localization and tracking. Spectrum sensors equipped with directional antennas should constantly scan the space in order to track emitting sources and discover new activities in the area of interest. In this paper, we propose a new formulation that unifies received-signal-strength (RSS) and direction of arrival (DoA) in a compressive sensing (CS) framework. The underlying CS measurement matrix is a function of beamforming vectors of sensors and is referred to as the propagation matrix. Comparing to the omni-directional antenna case, our employed propagation matrix provides more incoherent projections, an essential factor in the compressive sensing theory. Based on the new formulation, we optimize the antenna beams, enhance spectrum sensing efficiency, track active primary users accurately and monitor spectrum activities in an area of interest. In many practical scenarios there is no fusion center to integrate received data from spectrum sensors. We propose the distributed version of our algorithm for such cases. Experimental results show a significant improvement in source localization accuracy, compared with the scenario when sensors are equipped with omni-directional antennas. Applicability of the proposed framework for dynamic radio cartography is shown. Moreover, comparing the estimated dynamic RF map over time with the ground truth demonstrates the effectiveness of our proposed method for accurate signal estimation and recovery.

In this letter, we propose a sparsity promoting feedback acquisition and reconstruction scheme for sensing, encoding and subsequent reconstruction of spectrally sparse signals. In the proposed scheme, the spectral components are estimated utilizing a sparsity-promoting, sliding-window algorithm in a feedback loop. Utilizing the estimated spectral components, a level signal is predicted and sign measurements of the prediction error are acquired. The sparsity promoting algorithm can then estimate the spectral components iteratively from the sign measurements. Unlike many batch-based compressive sensing algorithms, our proposed algorithm gradually estimates and follows slow changes in the sparse components utilizing a sliding-window technique. We also consider the scenario in which possible flipping errors in the sign bits propagate along iterations (due to the feedback loop) during reconstruction. We propose an iterative error correction algorithm to cope with this error propagation phenomenon considering a binary-sparse occurrence model on the error sequence. Simulation results show effective performance of the proposed scheme in comparison with the literature.

Reliable and efficient spectrum sensing through dynamic selection of a subset of spectrum sensors is studied. The problem of selecting K sensor measurements from a set of M potential sensors is considered where K ≪ M. In addition, K may be less than the dimension of the unknown variables of estimation. Through sensor selection, we reduce the problem to an under-determined system of equations with potentially infinite number of solutions. However, the sparsity of the underlying data facilitates limiting the set of solutions to a unique solution. Sparsity enables employing the emerging compressive sensing technique, where the compressed measurements are selected from a large number of potential sensors. This paper suggests selecting sensors in a way that the reduced system of equations constructs a well-conditioned measurement matrix. Our criterion for sensor selection is based on E-optimalily, which is highly related to the restricted isometry property that provides some guarantees for sparse solution obtained by ℓ1minimization. Moreover, the proposed framework exploits a feedback mechanism to evolve the selected sensors dynamically over time. The evolution aims to maximize the reliability of the sensed spectrum.

In this paper, the problem of target localization in the presence of outlying sensors is tackled. This problem is important in practice because in many real-world applications the sensors might report irrelevant data unintentionally or maliciously. The problem is formulated by applying robust statistics techniques on squared range measurements and two different approaches to solve the problem are proposed. The first approach is computationally efficient; however, only the objective convergence is guaranteed theoretically. On the other hand, the whole-sequence convergence of the second approach is established. To enjoy the benefit of both approaches, they are integrated to develop a hybrid algorithm that offers computational efficiency and theoretical guarantees. The algorithms are evaluated for different simulated and real-world scenarios. The numerical results show that the proposed methods meet the Cr'amer-Rao lower bound (CRLB) for a sufficiently large number of measurements. When the number of the measurements is small, the proposed position estimator does not achieve CRLB though it still outperforms several existing localization methods.

The problem of signal detection using a flexible and general model is considered. Owing to applicability and flexibility of sparse signal representation and approximation, it has attracted a lot of attention in many signal processing areas. In this study, the authors propose a new detection method based on sparse decomposition in a union of subspaces model. Their proposed detector uses a dictionary that can be interpreted as a bank of matched subspaces. This improves the performance of signal detection, as it is a generalisation for detectors. Low‐rank assumption for the desired signals implies that the representations of these signals in terms of some proper bases would be sparse. Their proposed detector exploits sparsity in its decision rule. They demonstrate the high efficiency of their method in the cases of voice activity detection in speech processing.

A generalization of unequal error protection (UEP) rateless codes and distributed rateless codes for distributed relay networks is proposed. We consider a two-hop relaying network where a single source transmits UEP rateless coded data to a destination via multiple relays. At the relays, the UEP rateless coded symbols are re-encoded by distributed rateless codes (DRC) to minimize the redundancy at the second hop. Previously introduced UEP rateless codes (URC) have supported a limited number of importance classes, mostly two classes, and targeted importance levels suited to a specific application. In this paper, however, we formulate an optimization problem to provide optimal URC in terms of the symbol error rate (SER) given any specific number and strengths of importance levels. Next, another optimization is proposed to obtain DRC with the minimum SER at a given overhead, which utilizes information of common symbols among the relays. The optimization methods are based on the AND-OR tree analysis and sequential quadratic programming. In addition, we evaluate the minimum achievable end-to-end symbol error rates over the wireless relay networks.

Several studies on fountain codes have proposed degree distribution optimization schemes to maximize symbol recovery rate. However, if the number of transmitted coded symbols is limited or the channel erasure probability is high, it may be impossible that a user recovers all of the data symbols regardless of degree distribution employed by the source. In this study, we focus on a new system model where one source transmits fountain-coded symbols to multiple users who already possess some data symbols and coded symbols. Assuming that each user can transmit a feedback packet containing its own state information before the source transmits coded symbols, we propose two types of degree distribution design schemes that are suitable for the system model. Simulation results demonstrate the efficiency of our proposed schemes by comparing with conventional methods in terms of symbol recovery rate and full recovery rate.

In this work, we are interested in the compressive sensing of sparse signals whose significant coefficients are distributed asymmetrically with respect to zero. To properly address this problem, we develop a framework utilizing a two-state normal and skew normal mixture density as the prior distribution of the signal. The significant and insignificant coefficients of the signal are represented by skew normal and normal distributions, respectively. A novel approximate message passing-based algorithm is developed to estimate the signal from its compressed measurements. A fast gradient-based estimator is designed to infer the density of each state. Experiment results on simulated data and two real-world tests, i.e., multi-input multi-output (MIMO) communication system and weather sensor network, confirm that our proposed technique is powerful in exploiting asymmetrical feature, and outperforms many sophisticated methods.

In Wireless Sensor Networks (WSN), interactions of sensors to provide service to a specific task is a significant issue. Services provided by WSNs include collecting data from the environment and aggregating them to address queries, or processing the collected data and using the result to adjust environmental parameters such as temperature and moisture. In WSNs, a number of nodes may be simultaneously needed by some application, and these nodes usually need to interact with one another while running in parallel. It is desirable in WSNs that enough resources be assigned to the applications so that a typical application can acquire its needed resources as fast as possible. Distances among sensors assigned to an application typically constitute an important factor in saving communication energy. This paper concerns introducing a clustering algorithm to enhance the efficiency of resource assignment by reducing overall distances among cooperating sensors. In the proposed algorithm, clusters are formed with different sizes to assign just enough sensors to requesting application. Cluster sizes are determined based on an input file to the algorithm that contains the initial number of required clusters of each size. Several increasingly more inclusive versions of the proposed algorithm were studied that are reported in this paper. A performance analysis is provided that shows the proposed clustering algorithm outperforms the Means clustering algorithm from different perspectives.

Enabling the spectrum handoff for multimedia applications in cognitive radio networks (CRNs) is challenging, due to multiple interruptions from primary users (PUs), contentions among secondary users (SUs), and heterogenous Quality-of-Experience (QoE) requirements. In this paper, we propose a learning-based and QoE-driven spectrum handoff scheme to maximize the multimedia users' satisfaction. We develop a mixed preemptive and non-preemptive resume priority (PRP/NPRP) M/G/1 queueing model for modeling the spectrum usage behavior for prioritized multimedia applications. Then, a mathematical framework is formulated to analyze the performance of SUs. We apply the reinforcement learning to our QoE-driven spectrum handoff scheme to maximize the quality of video transmissions in the long term. The proposed learning scheme is asymptotically optimal, model-free, and can adaptively perform spectrum handoff for the changing channel conditions and traffic load. Experimental results demonstrate the effectiveness of the proposed queueing model for prioritized traffic in CRNs, and show that the proposed learning-based QoE-driven spectrum handoff scheme improves quality of video transmissions.

Efficient spectrum sensing is one of the key features that allows the implementation of fully agile cognitive radio networks. In this paper, we present an efficient coordinated spectrum sensing algorithm for wideband large-scale cognitive radio networks. Our approach is based on clustering secondary users according to their spectrum sensing results and performing the spectrum sensing tasks collaboratively within each cluster. In addition, the clusters can collaborate with each other to achieve an optimal distributed spectrum sensing across the network. We set up a cognitive radio framework and evaluate our proposed algorithm using numerical simulations. We show that the proposed algorithm increases the successful channel sensing rate at a reasonable computational cost.

LT codes are capacity achieving and flexible rateless codes that need a single-bit feedback to inform the encoder of the successful decoding. However, this weak feedback channel remains unused when the transmission is in progress. In addition, although LT codes are asymptotically capacity achieving, their performance significantly degrades for short block lengths. Consequently, we propose LT Codes with Alternating Feedback (LT-AF Codes) that considerably improve the performance of LT codes for short-block lengths when belief propagation decoder is in use. In our proposed scheme, the decoder alternatively issues two types of feedbacks based on the dependencies of the still undecoded received output symbols and the number of decoded input symbols. We propose two methods to form the latter type of feedback with a trade-off in their complexity and performance.

In this paper, we study compressive sampling of difference frames in videos and introduce a novel reconstruction method that exploits the structural characteristic, i.e., clustered sparsity in difference frames. Our method, referred to as structured reweighted ℓ1minimization (SRL1), estimates the signal support and adjusts the weights associated with the signal coefficients in a weighted ℓ1minimization in an iterative fashion. For the signal support estimation we propose local exploration and global purification steps to promote the clustered sparsity in difference frames. It is shown that by exploiting the clustered sparsity, isolated non-zero noise could be eliminated, and undiscovered signal coefficients could be retrieved. It should be noted that these steps are done based on the clustered sparsity, rather than the exact signal support distribution. This makes our method robust and distinct from many state-of-the-art algorithms. Experimental results show the effectiveness of our method.

Existing rateless codes have low intermediate symbol recovery rates (ISRR). Therefore, we first design new rateless codes with close to optimal ISRR employing genetic algorithms. Next, we assume an estimate of the channel erasure rate is available and propose an algorithm to further improve the ISRR of the designed codes.

In distributed rateless coding, multiple disjoint sources need to deliver their rateless coded symbols (where a symbol may contain a single bit or thousands of bits) to a common destination via a single relay. In this paper, we propose and design novel distributed rateless codes called DU-rateless codes that can provide Unequal Error Protection (UEP) for disjoint sources with unequal data lengths on erasure channels. To design DU-rateless codes, we tune the coding parameters at each source and propose to smartly combine the encoded symbols at the relay. We analyze DU-rateless codes employing And-Or tree analysis technique and leverage our analysis to design several sets of codes for various setups employing the-state-of-the-art multi-objective genetic algorithms. We evaluate the performance of the designed codes using numerical simulations and discuss their advantages.

In this paper, we propose CStorage a fully distributed and efficient data storage scheme for wireless sensor networks (WSNs) based on compressive sensing (CS) techniques. CStorage requires much smaller number of transmissions compared to existing algorithms by exploiting the compressibility of the natural signals along with the broadcast property of wireless channels. In CStorage, after the probabilistic readings dissemination phase, each node obtains one compressed sample (measurement) of the network's readings. Later, a data collector can query a small number measurements and recover all sensors' readings employing CS. We find the optimal parameters of CStorage and evaluate its performance.

The cooperation among the secondary users or among the secondary users and the primary users have proven to bring many benefits to both primary and secondary users in cognitive radio networks. In this paper, cooperative relaying in cognitive radio networks is studied for rateless coding error-control mechanism. For a pair of primary transmitter and receiver, we find the region in which any secondary or cognitive user can cooperate with the primary user in a constructive way. A queueing theoretical model is employed to describe the underlying system parameters and to create a mathematical framework for the analysis. The throughput performance of the secondary users is maximized based on the input traffic of the primary users. Using the unique properties of rateless codes, a low-complexity relay selection mechanism is also introduced. Through the experimental analysis, we show that the proposed rateless-coding-based cooperative mechanism can perform similar to the more complicated and more costly selective-repeat automatic repeat request algorithm. The numerical analysis using the Rapid Analysis of Queueing Systems (RAQS) software package is provided to support the findings.

When multiple sources of data need to transmit their rateless coded symbols through a single relay to a common destination, a distributed rateless code instead of several separate conventional rateless codes can be employed to encode the input symbols to increase the transmission efficiency and flexibility. In this paper, we propose distributed rateless codes (DU-rateless) that can provide unequal error protection (UEP) for distributed sources with different data block lengths and different importance levels. We analyze our proposed DU-rateless code employing And-Or tree analysis technique. Next, we design several sets of optimum DU-rateless codes for various setups employing multi-objective genetic algorithms and evaluate their performances.

LT codes are modern and efficient rateless forward error correction (FEC) codes with close to channel capacity performance. Nevertheless, in intermediate range where the number of received encoded symbols is less than the number of source symbols, LT codes have very low recovery rates. In this paper, we propose a novel algorithm which significantly increases the intermediate recovery rate of LT codes, while it preserves the codes' close to channel capacity performance. To increase the intermediate recovery rate, our proposed algorithm rearranges the transmission order of the encoded symbols exploiting their structure, their transmission history, and an estimate of the channel's erasure rate. We implement our algorithm for conventional LT codes, and numerically evaluate its performance.

We investigate the problem of reliable and energy-efficient one-to-all broadcasting in multihop wireless networks, and propose fractional transmission scheme (FTS) - a low-complexity and scalable broadcasting scheme. FTS exploits the broadcasting nature of wireless channels and random encoding of rateless codes to reduce energy consumption while ensuring reliable delivery of packets to all nodes in the network. In the proposed scheme, different neighbors of a node share the responsibility of transmitting packets by sending only a fraction of encoded packets required by the node to successfully receive the data sent by the source. A detailed analysis of the performance of FTS is presented for grid and random deployment networks. Further, extensive simulations compare our scheme with present energy-efficient methods such as random linear coding, multipoint relaying, dominant pruning, and broadcast incremental power scheme. Simulations reveal that FTS offers good performance and adaptability at a low computational cost.

In this paper, we design several degree distributions for rateless codes with optimum intermediate packet recovery rates. In rateless coding, the employed degree distribution significantly affects the packet recovery rate. Each degree distribution is designed based on the number of message packets, k, and desired coding overhead, ¿, which is the ratio of the number of received packets, n, to k, i.e., ¿ = n/k. Previously designed degree distributions are tuned for full recovery of the entire source packets for ¿'s slightly larger than 1, and as a consequence, they show very small packet recovery rates for ¿ < 1. Hence, finding degree distributions with maximal packet recovery rates in intermediate range, 0 < ¿ < 1, is of interest. We define packet recovery rates at three values of ¿ as our conflicting objective functions and employ NSGA-II multiobjective genetic algorithms optimization method to find several degree distributions with optimum packet recovery rates. We propose degree distributions for both cases of finite and infinite (asymptotic) k.

In this paper, we propose a new multicast subgraph construction algorithm and a distributed source coding-based multicast (DSCM) protocol for multihop wireless networks. The DSCM emphasize reliability, rate optimality, and energy efficiency. Both algorithms are based on local knowledge of the network. DSCM uses rateless error correcting codes to provide reliability and rate optimality, and distributed source coding to ensure the energy efficiency. We compared our scheme to energy-efficient methods such as network coding (NC) and multicast incremental power (MIP). Simulation results show DSCM performs close to these algorithms. However unlike the proposed algorithm, NC and MIP assume full knowledge of the network topology and have much higher decoding complexity than DSCM.

In this paper, we propose and evaluate two distributed protocols for finding low-cost broadcast and multicast trees in wireless networks. The constructed trees can then be used for reliable and energy-efficient data broadcast and multicast in wireless networks. The proposed schemes, referred to as broadcast decremental power (BDP) and multicast decremental power (MDP), evolve a given spanning tree of a network and form other spanning trees with lower costs of broadcast/multicast. In our schemes, the Bellman-Ford (BF) tree is considered as the initial spanning tree. Links in a network are assumed to have some cost based on parameters such as the distance between nodes, link losses, etc. We consider two different network scenarios. In the first one, nodes in the network have adjustable transmission power, and in the second one, the transmission power is fixed. Exhaustive simulation results are provided for the two different communication power scenarios and different network topologies to evaluate the proposed schemes. We show that broadcast/multicast cost is substantially improved over BF and previous well-known centralized schemes such as broadcast incremental power (BIP) and multicast incremental power (MIP), which can be implemented for the adjustable radius model. For the fixed power model, substantial improvement over BF and Network Coding (NC) is observed.

This paper introduces a novel two-phase broadcast scheme referred to as collaborative rateless broadcast (CRBcast). CRBcast is a scalable approach for reliable and energy-efficient broadcasting in a multihop wireless sensor networks that also addresses load balancing, while requiring no knowledge of network topology. CRBcast combines the energy-efficiency offered by probabilistic broadcasting (PBcast) with the reliability features offered by application-layer rateless coding. In the first phase of CRBcast, packets encoded using a rateless code are dispersed into the network based on PBcast. In the second phase, simple collaboration of neighboring nodes ensures that all nodes recover original data with a very high probability of success. Since the performance of CRBcast rests heavily on that of PBcast, first part of this paper analyzes both analytically and via simulations the probabilistic broadcasting scheme. We then study the effectiveness of CRBcast. We show that CRBcast provides both reliability and energy efficiency simultaneously. Simulation results indicate that CRBcast provides an energy savings of at least 72% and 60% in comparison with flooding and PBcast, respectively.

The problem of reliable and energy-efficient one-to-all broadcasting in multihop wireless networks is investigated in this paper and a low-complexity and scalable scheme (referred to as FTS) is proposed. This scheme utilizes rateless coding and the broadcasting nature of wireless channels to reduce the cost of broadcasting. In FTS, raw data is first encoded, and each node requires to send only a fraction of the total encoded packets. We compare our schemes with present energy-efficient methods such as Network Coding (NC), Multipoint Relaying (MPR), Broadcast Incremental Power (BIP), and Collaborative Rateless Broadcast (CRBcast). Our simulations reveal that our scheme performs well in comparison with these strategies, while having lower complexity and higher adaptability in comparison with some of them.

In this paper, we propose a scheme to construct low-density parity-check (LDPC) codes that are suitable for unequal error protection (UEP). We derive density evolution (DE) formulas for the proposed unequal error protecting LDPC ensembles over the binary erasure channel (BEC). Using the DE formulas, we optimize the codes. For the finite-length cases, we compare our codes with some other LDPC codes, the time-sharing method, and a previous work on UEP using LDPC codes. Simulation results indicate the superiority of the proposed design methodology for UEP

In this correspondence, a generalization of rateless codes is proposed. The proposed codes provide unequal error protection (UEP). The asymptotic properties of these codes under the iterative decoding are investigated. Moreover, upper and lower bounds on maximum-likelihood (ML) decoding error probabilities of finite-length LT and Raptor codes for both equal and unequal error protection schemes are derived. Further, our work is verified with simulations. Simulation results indicate that the proposed codes provide desirable UEP. We also note that the UEP property does not impose a considerable drawback on the overall performance of the codes. Moreover, we discuss that the proposed codes can provide unequal recovery time (URT). This means that given a target bit error rate, different parts of information bits can be decoded after receiving different amounts of encoded bits. This implies that the information bits can be recovered in a progressive manner. This URT property may be used for sequential data recovery in video/audio streaming

In this paper, we propose a two-phase broadcasting scheme referred as Collaborative Rateless Broadcast (CRBcast). We are particularly interested in reliability and energy efficiency of the broadcasting scheme for multi-hop wireless sensor networks. Our two-phase protocol is based on Probabilistic Broadcasting (PBcast) and an application layer rateless coding. At the first phase, the rateless-encoded packets are broadcasted based on PBcast, in which each node probabilistically relays every new received packet. The second recovery phase, which is based on simple collaborations of nodes, ensures that all nodes can recover original data. We first investigate PBcast analytically and with simulation, since the characteristics of PBcast influence CRBcast. Then, we investigate the effectiveness of CRBcast. We show that CRBcast can provide both reliability and energy efficiency. Simulation results indicate that CRBcast saves at least 72% and 60% energy in comparison with flooding and PBcast, respectively.

In this paper, we propose rateless codes that provide unequal error protection (UEP) property. We analyze the asymptotic properties of these codes under the iterative decoding algorithm. We further verify our work with simulations. The simulation results indicate that the proposed codes have strong UEP property. Moreover, the UEP property does not have a considerable drawback on the overall performance of the codes. We also discuss that the proposed codes can provide unequal recovery time (URT). This means that given a target bit error rate, different parts of information bits can be decoded after receiving different amounts of encoded bits. This implies that the information bits can be recovered in a progressive manner. This URT property may be used for sequential data recovery in video/audio streaming

A generalization of rateless codes (LT and Raptor codes) to provide unequal error protection (UEP) property is proposed in this paper. The proposed codes (UEP-LT and UEP-Raptor codes) are analyzed for the best possible performance over the binary erasure channel (BEC) in finite-length cases. We derive upper and lower bounds on the bit error probabilities under the maximum-likelihood (ML) decoding. We further verify our work with simulations. Simulation results indicate that the bounds are tight for small error rates, and the proposed codes have strong UEP property.

This correspondence introduces a framework to design and analyze low-density parity-check (LDPC) codes over nonuniform channels. We study LDPC codes for channels with nonuniform noise distributions, rate-adaptive coding, and unequal error protection. First, we propose a technique to design LDPC codes for volume holographic memory (VHM) systems for which the noise distribution is nonuniform. We show that the proposed coding scheme has an easy design procedure and results in efficient codes for holographic memories. An important property of the proposed technique is the design of the codes that have a low error floor and low variable node degrees, while maintaining performance close to the Shannon limit. We then show that punctured LDPC codes can be studied as a special case of our design methodology for nonuniform channels. Finally, we propose a method to generate LDPC codes that can provide unequal error protection in addition to having a good overall performance. Moreover, the highly protected bits can be decoded without requiring the entire word to be decoded.

In this study, we propose a scheme to construct low-density parity-check (LDPC) codes that are suitable for unequal error protection (UEP). We derive density evolution formulas for the proposed ensemble over the binary erasure channel (BEC). For the finite length case, we compare our code with some other LDPC codes and the time-sharing method. Simulation results indicate the superiority of the proposed design methodology for unequal error protection.

We propose a technique for designing low-density parity-check (LDPC) codes over non-uniform channels. In particular, we investigate LDPC codes for volume holographic memory (VHM) systems. We show that the proposed coding scheme for holographic memories has an easy design procedure and results in efficient codes. An important property of the proposed technique is that we can design simple codes whose performance is close to the Shannon limit, while they are very good in terms of the error floor effect. We also derive a capacity bound and the stability condition for the proposed codes over the binary erasure channel. We briefly discuss other applications like punctured codes, OFDM systems and multilevel coding.