Public datasets, such as KITTI, nuScenes, and Waymo, have played a key role in the research and development of autonomous vehicles and advanced driver assistance systems. However, many of these datasets fail to incorporate a full range of driving conditions; some datasets only contain clear-weather conditions, underrepresenting or entirely missing colder weather conditions such as snow or autumn scenes with bright colorful foliage. In this paper, we present the Michigan State University Four Seasons (MSU-4S) Dataset, which contains real-world collections of autonomous vehicle data from varied types of driving scenarios. These scenarios were recorded throughout a full range of seasons, and capture clear, rainy, snowy, andfall weather conditions, at varying times of day. MSU-4S contains more than 100,000 two-and three-dimensional frames for camera, lidar, and radar data, as well as Global Navigation Satellite System (GNSS), wheel speed, and steering data, all annotated with weather, time-of-day, and time-of-year. Our data includes cluttered scenes that have large numbers of vehicles and pedestrians; and it also captures industrial scenes, busy traffic thoroughfare with traffic lights and nu-merous signs, and scenes with dense foliage. While pro-viding a diverse set of scenes, our data incorporate an im-portant feature: virtually every scene and its corresponding lidar, camera, and radar frames were captured in four different seasons, enabling unparalleled object detection analysis and testing of the domain shift problem across weather conditions. In that context, we present detailed analyses for 3D and 2D object detection showing a strong domain shift effect among MSU-4S data segments collected across different conditions. MSU-4S will also enable advanced mul-timodal fusion research including different combinations of camera-lidar-radar fusion, which continues to be of strong interest for the computer vision, autonomous driving and ADAS development communities. The MSU-4S dataset is available online at https://egr.msu.edu/waves/msu4s.

Neural network-based algorithms have advanced the state-of-the-art in many perception-related tasks such as detection, tracking, and prediction, which are critically important for autonomous vehicles, ADAS, and other related technologies. However, integrating disparate neural network-based algorithms together can be challenging due to varying system-level requirements, non-standard network input and output formats, and custom network features. In this paper, we introduce RAMPS, a framework for developing a node-based system for integrating datasets, perception algorithms, and other processing modules in a single cohesive graph-based system. This system represents a novel framework architecture for constructing and evaluating perception pipelines. We first outline the design philosophy of this system, then we explain the design decisions we made in developing a functioning prototype, then outline algorithms and functions that we have already integrated into RAMPS, and lastly demonstrate some practical examples and benchmarks that show how our proposed framework operates and benefits current and future state-of-the-art perception systems. We intend to release our framework as open source software in the near future for the benefit of the autonomous vehicle, ADAS, and overall computer vision communities.

Object detection using LiDAR point clouds relies on a large amount of human-annotated samples when training the underlying detectors' deep neural networks. However, generating 3-D bounding box annotation for a large-scale dataset could be costly and time-consuming. Alternatively, unsupervised domain adaptation (UDA) enables a given object detector to operate on novel new data, with an unlabeled training dataset, by transferring the knowledge learned from training labeledsource domaindata to the new unlabeledtarget domain. Pseudolabel strategies, which involve training the 3-D object detector using target-domain predicted bounding boxes from a pretrained model, are commonly used in UDA. However, these pseudolabels often introduce noise, impacting performance. In this article, we introduce the domain adaptive LiDAR (DALI) object detection framework to address noise at both distribution and instance levels. First, a posttraining size normalization (PTSN) strategy is developed to mitigate bias in pseudolabel size distribution by identifying an unbiased scale after network training. To address instance-level noise between pseudolabels and corresponding point clouds, two pseudopoint clouds generation (PPCG) strategies, ray-constrained and constraint-free, are developed to generate pseudopoint clouds for each instance, ensuring the consistency between pseudolabels and pseudopoints during training. We demonstrate the effectiveness of our method on the publicly available and popular datasets KITTI, Waymo, and nuScenes. We show that the proposed DALI framework achieves state-of-the-art results and outperforms leading approaches on most of the domain adaptation tasks.

RGB-based pedestrian detection is a challenging task in low light and adverse weather conditions because image quality can degrade rather significantly. Including other modalities such as thermal and gated imaging sensors can, on the other hand, significantly improve the detection performance in these conditions. However, these sensors are expensive, and including them may cause design and manufacturing challenges. In this paper, we propose a new framework that utilizes Cross Modality Knowledge Distillation (CMKD) to improve the performance of RGB-only pedestrian detection in low light and adverse weather conditions. Specifically, we develop two CMKD methods that rely on feature-based knowledge distillation and adversarial training to transfer knowledge from a pedestrian detector (teacher) that is trained using multiple modalities to a single modality detector (student) that is trained using RGB images only. Experimental results using the "Seeing Through Fog" dataset show that both of our proposed methods outperform the baseline detector in terms of detection accuracy without increasing computational complexity during inference. In particular, the proposed methods reduce the performance gap between teacher and baseline models by up to 55%.

The adversarial robustness of a model is its ability to resist adversarial attacks in the form of small perturbations to input data. Universal adversarial attack methods such as Fast Sign Gradient Method (FSGM) [1] and Projected Gradient Descend (PGD) [2] are popular for LiDAR object detection, but they are often deficient compared to task-specific adversarial attacks. Additionally, these universal methods typically require unrestricted access to the model's information, which is difficult to obtain in real-world applications. To address these limitations, we present a black-box Scaling Adversarial Robustness (ScAR) method for LiDAR object detection. By analyzing the statistical characteristics of 3D object detection datasets such as KITTI, Waymo, and nuScenes, we have found that the model's prediction is sensitive to scaling of 3D instances. We propose three black-box scaling adversarial attack methods based on the available information: model-aware attack, distribution-aware attack, and blind attack. We also introduce a strategy for generating scaling adversarial examples to improve the model's robustness against these three scaling adversarial attacks. Comparison with other methods on public datasets under different 3D object detection architectures demonstrates the effectiveness of our proposed method.

Despite radar's popularity in the automotive industry, for fusion-based 3D object detection, most existing works focus on LiDAR and camera fusion. In this paper, we propose TransCAR, a Transformer-based Camera-And-Radar fusion solution for 3D object detection. Our TransCAR consists of two modules. The first module learns 2D features from surround-view camera images and then uses a sparse set of 3D object queries to index into these 2D features. The vision-updated queries then interact with each other via transformer self-attention layer. The second module learns radar features from multiple radar scans and then applies transformer decoder to learn the interactions between radar features and vision-updated queries. The cross-attention layer within the transformer decoder can adaptively learn the soft-association between the radar features and vision-updated queries instead of hard-association based on sensor calibration only. Finally, our model estimates a bounding box per query using set-to-set Hungarian loss, which enables the method to avoid non-maximum suppression. TransCAR improves the velocity estimation using the radar scans without temporal information. The superior experimental results of our TransCAR on the challenging nuScenes datasets illustrate that our TransCAR outperforms state-of-the-art Camera-Radar fusion-based 3D object detection approaches.

The area of domain adaptation has been instrumental in addressing the domain shift problem encountered by many deep learning applications. This problem arises due to the difference between the distributions of source data used for training in comparison with target data used during realistic testing scenarios. In this paper, we introduce a novel MultiScale Domain Adaptive YOLO (MS-DAYOLO) framework that employs multiple domain adaptation paths and corresponding domain classifiers at different scales of the YOLOv4 object detector. Building on our baseline multiscale DAYOLO framework, we introduce three novel deep learning architectures for a Domain Adaptation Network (DAN) that generates domain-invariant features. In particular, we propose a Progressive Feature Reduction (PFR), a Unified Classifier (UC), and an Integrated architecture. We train and test our proposed DAN architectures in conjunction with YOLOv4 using popular datasets. Our experiments show significant improvements in object detection performance when training YOLOv4 using the proposed MS-DAYOLO architectures and when tested on target data for autonomous driving applications. Moreover, MS-DAYOLO framework achieves an order of magnitude real-time speed improvement relative to Faster R-CNN solutions while providing comparable object detection performance.

Unsupervised domain adaptation (UDA) focuses on transferring knowledge learned in the labeled source domain to the unlabeled target domain. Semi-supervised learning is a proven strategy for improving UDA performance. In this paper, we propose a novel strong-weak integrated semi-supervision (SWISS) learning strategy for unsupervised domain adaptation. Under the proposed SWISSUDA framework, a strong representative set with high confidence but low diversity target domain samples and a weak representative set with low confidence but high diversity target domain samples are updated constantly during the training process. Both sets are fused randomly to generate an augmented strong-weak training batch with pseudo-labels to train the network during every iteration. Moreover, a novel adversarial logit loss is proposed to reduce the intra-class divergence between source and target domains, which is back-propagated adversarially with a gradient reverse layer between the classifier and the rest of the network. Experimental results based on two popular benchmarks, Office-Home, and DomainNet, show the effectiveness of the proposed SWISS framework with our method achieving the best performance in both Office-Home and DomainNet.

Realistic object detection scenarios, most notably the detection of objects such as pedestrians and traffic signs by autonomous vehicles, present a real challenge for state-of-the-art methods. Domain shifts further aggravate this problem when an autonomous vehicle has to operate and detect objects under more challenging conditions (e.g., rainy weather). In this paper, we demonstrate that the performance of challenging object detection scenarios can be improved rather significantly using a novel integrated Generative-model based unsupervised training and Domain Adaptation (GDA) framework. In particular, we exploit unsupervised image-to-image translation to generate annotated visuals that are representatives of a target challenging domain. Then, we use these generated annotated visuals in addition to unlabeled target domain data to train a domain adaptive object detection method. We show that using this integrated approach outperforms both methods, unsupervised image translation, and domain adaptation, when they are used separately. We evaluate the performance of the proposed GDA framework using real visuals captured by vehicles driving under rainy weather. Our simulation results show that GDA can achieve significant improvements in the detection of pedestrians and traffic signs when tested under real challenging conditions in comparison with state-of-the-art methods.

When compared to single modality approaches, fusion-based object detection methods often require more complex models to integrate heterogeneous sensor data, and use more GPU memory and computational resources. This is particularly true for camera-LiDAR based multimodal fusion, which may require three separate deep-learning networks and/or processing pipelines that are designated for the visual data, LiDAR data, and for some form of a fusion framework. In this paper, we propose Fast Camera-LiDAR Object Candidates (Fast-CLOCs) fusion network that can run high-accuracy fusion-based 3D object detection in near real-time. Fast-CLOCs operates on the output candidates before Non-Maximum Suppression (NMS) of any 3D detector, and adds a lightweight 3D detector-cued 2D image detector (3D-Q-2D) to extract visual features from the image domain to improve 3D detections significantly. The 3D detection candidates are shared with the proposed 3D-Q-2D image detector as proposals to reduce the network complexity drastically. The superior experimental results of our Fast-CLOCs on the challenging KITTI and nuScenes datasets illustrate that our Fast-CLOCs outperforms state-of-the-art fusion-based 3D object detection approaches. We will release the code upon publication.

The ability of an autonomous vehicle to perform 3D tracking is essential for safe planing and navigation in cluttered environments. The main challenges for multi-object tracking (MOT) in autonomous driving applications reside in the inherent uncertainties regarding the number of objects, when and where the objects may appear and disappear, and uncertainties regarding objects’ states. Random finite set (RFS) based approaches can naturally model these uncertainties accurately and elegantly, and they have been widely used in radar-based tracking applications. In this work, we developed an RFS-based MOT framework for 3D LiDAR data. In partiuclar, we propose a Poisson multi-Bernoulli mixture (PMBM) filter to solve the amodal MOT problem for autonomous driving applications. To the best of our knowledge, this represents a first attempt for employing an RFS-based approach in conjunction with 3D LiDAR data for MOT applications with comprehensive validation using challenging datasets made available by industry leaders. The superior experimental results of our PMBM tracker on public Waymo and Argoverse datasets clearly illustrate that an RFS-based tracker outperforms many state-of-the-art deep learning-based and Kalman filter-based methods, and consequently, these results indicate a great potential for further exploration of RFS-based frameworks for 3D MOT applications.

The area of domain adaptation has been instrumental in addressing the domain shift problem encountered by many applications. This problem arises due to the difference between the distributions of source data used for training in comparison with target data used during realistic testing scenarios. In this paper, we introduce a novel MultiScale Domain Adaptive YOLO (MS-DAYOLO) framework that employs multiple domain adaptation paths and corresponding domain classifiers at different scales of the recently introduced YOLOv4 object detector to generate domain-invariant features. We train and test our proposed method using popular datasets. Our experiments show significant improvements in object detection performance when training YOLOv4 using the proposed MSDAYOLO and when tested on target data representing challenging weather conditions for autonomous driving applications.

Multiple object tracking (MOT) is a critical module for enabling autonomous vehicles to achieve safe planing and navigation in cluttered environments. In tracking-by-detection systems, there are inevitably many false positives and misses among learning-based input detections. The challenge for MOT is to combine these detections into tracks, and filter them based on their uncertainties, states, and temporal consistency to achieve accurate and persistent tracks. In this paper, we propose to solve the 3D MOT problem for autonomous driving applications using a random finite set-based (RFS) Multiple Measurement Models filter (RFS-M3). In partiuclar, we propose multiple measurement models for a Poisson multi-Bernoulli mixture (PMBM) filter in support of different application scenarios. Our RFS-M3filter can naturally model these uncertainties accurately and elegantly. We combine the learning-based detections with our RFS-M3tracker through incorporating the detection confidence score into the PMBM prediction and update step. The superior experimental results of our RFS-M3tracker on Waymo, Argoverse and nuSceness datasets illustrate that our RFS-M3tracker outperforms state-of-the-art deep learning-based and traditional filter-based approaches. To the best of our knowledge, this represents a first successful attempt for employing an RFS-based approach in conjunction with 3D learning-based amodal detections for 3D MOT applications with comprehensive validation using challenging datasets made available by industry leaders.

Modern localization techniques allow ground vehicle robots to determine their position with centimeter-level accuracy under nominal conditions, enabling them to utilize fixed maps to navigate their environments. However, when localization measurements become unavailable, the position accuracy will drop and uncertainty will increase. While research and development on localization estimation seeks to reduce the severity of these outages, the question of what actions a robot should take under high localization uncertainty is still unresolved, and can vary on a platform-by-platform and mission-by-mission basis. In this paper, we exploit localization uncertainty measures to adapt system control parameters in real time. Offline, we optimize non-linear activation functions whose control parameters and relevant weights are trained and learned using Evolutionary Algorithm (EA). Subsequently, in real time, we apply the optimized adaptation functions to the controller look-ahead distance and intermediate linear and angular velocity commands, which we identify as the most sensitive to localization error. Evolutionary runs are conducted in which a simulated target vehicle is tasked with following a randomly generated path while minimizing cross-track error, with time varying localization uncertainty added. These runs produce situation-dependent weights for parameters to the adaptation functions, which are transferred to the physical platform, a 1:5-scale autonomous vehicle. In simulation, our system was able to reduce cross-track error, which in certain cases exceeds 250 centimeters on non-adapted systems, to below 15 centimeters on average using EA-derived weights and parameters applied to our proposed adaptation system. Evaluation on the physical platform demonstrates that without the adaptation module in place, the platform is unable to successfully follow the path; with the adaptation module, the platform automatically adjusts its velocity and look-ahead distance to compensate for localization uncertainty.

There have been significant advances in neural networks for both 3D object detection using LiDAR and 2D object detection using video. However, it has been surprisingly difficult to train networks to effectively use both modalities in a way that demonstrates gain over single-modality networks. In this paper, we propose a novel Camera-LiDAR Object Candidates (CLOCs) fusion network. CLOCs fusion provides a low-complexity multi-modal fusion framework that significantly improves the performance of single-modality detectors. CLOCs operates on the combined output candidates before Non-Maximum Suppression (NMS) of any 2D and any 3D detector, and is trained to leverage their geometric and semantic consistencies to produce more accurate final 3D and 2D detection results. Our experimental evaluation on the challenging KITTI object detection benchmark, including 3D and bird's eye view metrics, shows significant improvements, especially at long distance, over the state-of-the-art fusion based methods. At time of submission, CLOCs ranks the highest among all the fusion-based methods in the official KITTI leaderboard. We will release our code upon acceptance.

Accurate vehicle localization is arguably the most critical and fundamental task for autonomous vehicle navigation. While dense 3D point-cloud-based maps enable precise localization, they impose significant storage and transmission burdens when used in city-scale environments. In this paper, we propose a highly compressed representation for LiDAR maps, along with an efficient and robust real-time alignment algorithm for on-vehicle LiDAR scans. The proposed mapping framework, which we refer to as Feature Likelihood Acquisition Map Emulation (FLAME), requires less than 0.1% of the storage space of the original 3D point cloud map. In essence, FLAME emulates an original map through feature likelihood functions. In particular, FLAME models planar, pole and curb features. These three feature classes are long-term stable, distinct and common among vehicular roadways. Multiclass feature points are extracted from LiDAR scans through feature detection. A new multiclass-based point-to-distribution alignment method is proposed to find the association and alignment between the multiclass feature points and the FLAME map. The experimental results show that the proposed framework can achieve the same level of accuracy (less than 10cm) as the 3D point cloud based localization.

Iterative closest points (ICP) and normal distributions transform (NDT) are popular 3D point cloud registration algorithms, which have been widely used in mapping and 3D reconstruction. These algorithms provide robust methods for self- localizing an autonomous vehicle by registering real-time 3D-scans to a prior map. However, urban and suburban environments are continually changing, resulting in significant differences that impact registration algorithms. These temporal changes occur over varying time-scales, and include dynamic and ephemeral objects (such as parked cars), seasonal changes (vegetation, snow, dust), and human impacts such as construction. It is critical that a self-localization method be robust to these and other real-environment changes. Furthermore, the computational complexity of the algorithm and its stability and ability to process data in real-time when faced with such adverse conditions are important. In this paper, we present an empirical comparison of NDT and ICP and their performances for autonomous vehicle localization through a set of realistic field tests conducted in the state of Michigan over many months spanning the summer, fall, and winter months. The test sites include the campus of Michigan State University and the University of Michigan's MCity Test Facility, which is a professional purpose-built proving ground for testing autonomous vehicles and technologies. Our tests indicate that NDT possesses a better ability to handle realistic adversity conditions such as static and dynamic environmental changes, as well as being more computationally efficient.

In this paper, we consider the Direct Perception approach for autonomous driving. Previous efforts in this field focused more on feature extraction of the road markings and other vehicles in the scene rather than on the autonomous driving algorithm and its performance under realistic assumptions. Our main contribution in this paper is introducing a new, more robust, and more realistic Direct Perception framework and corresponding algorithm for autonomous driving. First, we compare the top 3 Convolutional Neural Networks (CNN) models in the feature extraction competitions and test their performance for autonomous driving. The experimental results showed that GoogLeNet performs the best in this application. Subsequently, we propose a deep learning based algorithm for autonomous driving, and we refer to our algorithm as GoogLenet for Autonomous Driving (GLAD). Unlike previous efforts, GLAD makes no unrealistic assumptions about the autonomous vehicle or its surroundings, and it uses only five affordance parameters to control the vehicle as compared to the 14 parameters used by prior efforts. Our simulation results show that the proposed GLAD algorithm outperforms previous Direct Perception algorithms both on empty roads and while driving with other surrounding vehicles.

The popular uprisings in a number of countries in the Middle East and North Africa in the Spring of 2011 were broadcasted live and enabled by local populations' access to social networking services such as Twitter and Facebook. The goal of this paper is to study the flow characteristics of the information flow of these broadcasts on Twitter. We have used language independent features of Twitter traffic to identify differences in information flows on Twitter mentioning countries experiencing some form of unrest, compared to traffic mentioning countries with peaceful political situations. We used these features to identify countries with political unstable situation. For empirical analysis, we collected several data sets of countries that were experiencing political unrest, as well as a set of countries in a control group that were not subject to such socio-political condition. Several different methods are used to model the flow of information between Twitter users in data sets as graphs, called information cascades. By using the dynamic properties of information cascades, naïve Bayes and SVM classifiers both achieve true positives rates of 100%, with false positives rates of 3% and 0%, respectively.

In this paper, we introduce a new video-streaming framework that is based on a novel packet design in conjunction with low-complexity active queue management within the network. Our proposed framework, which we call Erasable Packets within Internet Queues (EPIQ), exploits the inherent multi-priority nature of video to deliver optimal quality-of-experience to users, yet without requiring any additional overhead in terms of redundant packets or network resources such as multiple queues or maintaining state information about flows. The novelty of EPIQ is rooted at its packetization. Each EPIQ packet consists of multiple segments and where each segment carries different priority video content. Under congestion, a network node (ideally a router) can simply partially erase only small portions of the EPIQ packets instead of dropping complete packets. We show through an analytical model and extensive simulations that this new paradigm in video streaming provides improvements over multi-queue systems. These improvements are achieved while maintaining low-complexity within the network.

In this paper, we consider the problem of example based single image super-resolution. Our main contribution is introducing a new framework that makes no assumption about the structural similarity between the high-resolution (HR) and low-resolution (LR) manifolds. Instead, we use a subspace affinity measure to exploit the similarity between each HR and LR subspace. First, we train both LR and HR manifolds independently, and then, by using subspace similarity we find the closest HR subspace to each LR subspace. Each patch from the LR test image is projected onto the LR trained manifold to find the closest LR subspace. Finally, the corresponding HR subspace is selected to reconstruct the HR version of the test patch. We refer to the proposed framework Dual-Manifold Clustering and Subspace Similarity (DMCSS). The experimental results showed that DMCSS achieves clear visual improvements and an average of 1dB improvement in PSNR over state-of-the-art algorithms in this field.

Although the matrix completion paradigm provides an appealing solution to the collaborative filtering problem in recommendation systems, some major issues, such as data sparsity and cold-start problems, still remain open. In particular, when the rating data for a subset of users or items is entirely missing, commonly known as thecold-startproblem, the standard matrix completion methods are inapplicable due the non-uniform sampling of available ratings. In recent years, there has been considerable interest in dealing with cold-start users or items that are principally based on the idea of exploiting other sources of information to compensate for this lack of rating data. In this paper, we propose a novel and general algorithmic framework based on matrix completion that simultaneously exploits the similarity information among users and items to alleviate the cold-start problem. In contrast to existing methods, our proposed recommender algorithm, dubbed DecRec,decouplesthe following two aspects of the cold-start problem to effectively exploit the side information: (i) the completion of a rating sub-matrix, which is generated by excluding cold-start users/items from the original rating matrix; and (ii) the transduction of knowledge from existing ratings to cold-start items/users using side information. This crucial difference prevents the error propagation of completion and transduction, and also significantly boosts the performance when appropriate side information is incorporated. The recovery error of the proposed algorithm is analyzed theoretically and, to the best of our knowledge, this is the first algorithm that addresses the cold-start problem with provable guarantees on performance. Additionally, we also address the problem where both cold-start user and item challenges are present simultaneously. We conduct thorough experiments on real datasets that complement our theoretical results. These experiments demonstrate the effectiveness of the proposed algorithm in handling the cold-start users/items problem and mitigating data sparsity issue.

This paper investigates the network completion problem, where it is assumed that only a small sample of a network (e.g., a complete or partially observed subgraph of a social graph) is observed and we would like to infer the unobserved part of the network. In this paper, we assume that besides the observed subgraph, side information about the nodes such as the pairwise similarity between them is also provided. In contrast to the original network completion problem where the standard methods such as matrix completion is inapplicable due the non-uniform sampling of observed links, we show that by effectively exploiting the side information, it is possible to accurately predict the unobserved links. In contrast to existing matrix completion methods with side information such as shared subsapce learning and matrix completion with transduction, the proposed algorithm decouples the completion from transduction to effectively exploit the similarity information. This crucial difference greatly boosts the performance when appropriate similarity information is used. The recovery error of the proposed algorithm is theoretically analyzed based on the richness of the similarity information and the size of the observed submatrix. To the best of our knowledge, this is the first algorithm that addresses the network completion with similarity of nodes with provable guarantees. Experiments on synthetic and real networks from Facebook and Google+ show that the proposed two-stage method is able to accurately reconstruct the network and outperforms other methods.

This paper considers the problem of single image super-resolution (SR). Previous example-based SR approaches mainly focus on analyzing the co-occurrence properties of low resolution (LR) and high resolution (HR) patches via dictionary learning. In our recent work [1], a novel approach (SR via sparse subspace clustering-based linear approximation of manifold or SLAM) has been proposed. In this paper, we further improve the SLAM method by considering and analyzing each tangent subspace as one point in a Grassmann manifold to select an optimal subset of tangent spaces. Furthermore, the optimal subset is clustered hierarchically, which helps in reducing the proposed algorithm's complexity significantly while still preserving the quality of the reconstructed HR image.

We propose an example-based super-resolution method for a new framework of scalable video streaming. The proposed method is applicable to scalable video where the enhancement layer of some frames (e.g., within the same Group-of-Pictures) might be dropped due to changing network conditions. This leads to a streaming scenario that we call Inconsistent Scalable Video (ISV) streaming. In this paper, we focus on spatial ISV. In particular, at the decoder, the frames with the enhancement layer are used as the super-resolution dictionary for other video frames that their enhancement layers were dropped. Using the proposed method, the frames with dropped enhancement layer are scaled up to the original size. Our experimental results show a clear improvement over traditional interpolation based scaling.

In this paper, we describe a novel manifold-based interpolation method for sensed environmental data. Furthermore, we present initial results for applying the proposed method to estimate wind speed and direction around Lake Michigan. The proposed method is showing promising results based on the hypothesis that an environmental dataset (including longitude, latitude time, and measured parameters) can be mapped onto an underlying differential manifold. Our preliminary results show that the proposed manifold-based approach outperforms state-of-the-art interpolation and estimation methods.

A major challenge in collaborative filtering based recommender systems is how to provide recommendations when rating data is sparse or entirely missing for a subset of users or items, commonly known as the cold-start problem. In recent years, there has been considerable interest in developing new solutions that address the cold-start problem. These solutions are mainly based on the idea of exploiting other sources of information to compensate for the lack of rating data. In this paper, we propose a novel algorithmic framework based on matrix factorization that simultaneously exploits the similarity information among users and items to alleviate the cold-start problem. In contrast to existing methods, the proposed algorithm decouples the following two aspects of the cold-start problem: (a) the completion of a rating sub-matrix, which is generated by excluding cold-start users and items from the original rating matrix; and (b) the transduction of knowledge from existing ratings to cold-start items/users using side information. This crucial difference significantly boosts the performance when appropriate side information is incorporated. We provide theoretical guarantees on the estimation error of the proposed two-stage algorithm based on the richness of similarity information in capturing the rating data. To the best of our knowledge, this is the first algorithm that addresses the cold-start problem with provable guarantees. We also conduct thorough experiments on synthetic and real datasets that demonstrate the effectiveness of the proposed algorithm and highlights the usefulness of auxiliary information in dealing with both cold-start users and items.

The significance of social-enhanced recommender systems is increasing, along with its practicality, as online reviews, ratings, friendship links, and follower relationships are increasingly becoming available. In recent years, there has been an upsurge of interest in exploiting social information, such as trust and distrust relations in recommendation algorithms. The goal is to improve the quality of suggestions and mitigate the data sparsity and the cold-start users problems in existing systems. In this paper, we introduce a general collaborative social ranking model to rank the latent features of users extracted from rating data based on the social context of users. In contrast to existing social regularization methods, the proposed framework is able to simultaneously leverage trust, distrust, and neutral relations, and has a linear dependency on the social network size. By integrating the ranking based social regularization idea into the matrix factorization algorithm, we propose a novel recommendation algorithm, dubbed PushTrust. Our experiments on the Epinions dataset demonstrate that collaboratively ranking the latent features of users by exploiting trust and distrust relations leads to a substantial increase in performance, and to effectively deal with cold-start users problem.

Key frame extraction algorithms consider the problem of selecting a subset of the most informative frames from a video to summarize its content. Several applications, such as video summarization, search, indexing, and prints from video, can benefit from extracted key frames of the video under consideration. Most approaches in this class of algorithms work directly with the input video data set, without considering the underlying low-rank structure of the data set. Other algorithms exploit the low-rank component only, ignoring the other key information in the video. In this paper, a novel key frame extraction framework based on robust principal component analysis (RPCA) is proposed. Furthermore, we target the challenging application of extracting key frames from unstructured consumer videos. The proposed framework is motivated by the observation that the RPCA decomposes an input data into: 1) a low-rank component that reveals the systematic information across the elements of the data set and 2) a set of sparse components each of which containing distinct information about each element in the same data set. The two information types are combined into a single l1-norm-based non-convex optimization problem to extract the desired number of key frames. Moreover, we develop a novel iterative algorithm to solve this optimization problem. The proposed RPCA-based framework does not require shot(s) detection, segmentation, or semantic understanding of the underlying video. Finally, experiments are performed on a variety of consumer and other types of videos. A comparison of the results obtained by our method with the ground truth and with related state-of-the-art algorithms clearly illustrates the viability of the proposed RPCA-based framework.

The breach of privacy in encrypted instant messenger (IM) service is a serious threat to user anonymity. Performance of previous de-anonymization strategies was limited to 65%. We perform network de-anonymization by taking advantage of the cause-effect relationship between sent and received packet streams and demonstrate this approach on a data set of Yahoo! IM service traffic traces. An investigation of various measures of causality shows that IM networks can be breached with a hit rate of 99%. A KCI Causality based approach alone can provide a true positive rate of about 97%. Individual performances of Granger, Zhang and IGCI causality are limited owing to the very low SNR of packet traces and variable network delays.

This letter proposes a novel learning-based super-resolution method rooted in low dimensional manifold representations of high-resolution (HR) image-patch spaces. We exploit the input image and its different down-sampled scales to extract a set of training sample points using a min-max algorithm. A set of low dimensional tangent spaces is estimated from these samples using the l1 norm graph-based technique to cluster these samples into a set of manifold neighborhoods. The HR image is then reconstructed from these tangent spaces. Experimental results on standard images validate the effectiveness of the proposed method both quantitatively and perceptually.

Automatic video summarization is indispensable for fast browsing and efficient management of large video libraries. In this paper, we introduce an image feature that we refer to as heterogeneity image patch (HIP) index. The proposed HIP index provides a new entropy-based measure of the heterogeneity of patches within any picture. By evaluating this index for every frame in a video sequence, we generate a HIP curve for that sequence. We exploit the HIP curve in solving two categories of video summarization applications: key frame extraction and dynamic video skimming. Under the key frame extraction frame-work, a set of candidate key frames is selected from abundant video frames based on the HIP curve. Then, a proposed patch-based image dissimilarity measure is used to create affinity matrix of these candidates. Finally, a set of key frames is extracted from the affinity matrix using a min–max based algorithm. Under video skimming, we propose a method to measure the distance between a video and its skimmed representation. The video skimming problem is then mapped into an optimization framework and solved by minimizing a HIP-based distance for a set of extracted excerpts. The HIP framework is pixel-based and does not require semantic information or complex camera motion estimation. Our simulation results are based on experiments performed on consumer videos and are compared with state-of-the-art methods. It is shown that the HIP approach outperforms other leading methods, while maintaining low complexity.

Efficient video representation models are critical for many video analysis and processing tasks. In this paper, we present a framework based on the concept of finding the sparsest solution to model video frames. To model the spatio-temporal information, frames from one scene are decomposed into two components: (i) a common frame, which describes the visual information common to all the frames in the scene/segment, and (ii) a set of innovative frames, which depicts the dynamic behaviour of the scene. The proposed approach exploits and builds on recent results in the field of compressed sensing to jointly estimate the common frame and the innovative frames for each video segment. We refer to the proposed modeling framework by CIV (Common and Innovative Visuals). We show how the proposed model can be utilized to find scene change boundaries and extend CIV to videos from multiple scenes. Furthermore, the proposed model is robust to noise and can be used for various video processing applications without relying on motion estimation and detection or image segmentation. Results for object tracking, video editing (object removal, inpainting) and scene change detection are presented to demonstrate the efficiency and the performance of the proposed model.

In this paper, we exploit the intrinsic tensor-nature of video and propose a Tensor Video Coding (TVC) framework that is based on tensor decomposition. We develop a Progressive Canonical-decomposition Parallel-factor (PCP) framework that is tailored for video representation and coding. Our simulation results show that the proposed TVC approach outperforms state-of-the-art video coding methods that do not rely on motion estimation or compensation.

We present a novel attack on relayed instant messaging (IM) traffic that allows an attacker to infer who's talking to whom with high accuracy. This attack only requires collection of packet header traces between users and IM servers for a short time period, where each packet in the trace goes from a user to an IM server or vice-versa. The specific goal of the attack is to accurately identify a candidate set of top-k users with whom a given user possibly talked to, while using only the information available in packet header traces (packet payloads cannot be used because they are mostly encrypted). Towards this end, we propose a wavelet-based scheme, called COmmunication Link De-anonymization (COLD), and evaluate its effectiveness using a real-world Yahoo! Messenger data set. The results of our experiments show that COLD achieves a hit rate of more than 90% for a candidate set size of 10. For slightly larger candidate set size of 20, COLD achieves almost 100% hit rate. In contrast, a baseline method using time series correlation could only achieve less than 5% hit rate for similar candidate set sizes.

In this paper we address the problem of compressing a collection of still images of the same type, such as an image database of faces or any other visual objects of similar shapes. We refer to such collection of images as an image ensemble. The compression of image ensembles presents a unique set of requirements such as random access to any image within the collection without the need to reconstruct other images in the same ensemble. Such requirement is readily met by any still image compression standard, simply by encoding each image in isolation of other images within the same ensemble. However, traditional approaches of still image compression do not exploit the strong correlation that might exist among images within a given ensemble. In this paper we argue that a tensor-decomposition framework can achieve both: (a) random access to any image within an ensemble and (b) the exploitation of the correlation among all images within the same ensemble. To that end, we propose a progressive tensor-factorization approach that decomposes an image ensemble into a set of block-wise rank-one tensors. We derive and encode the rank-one tensors of an image ensemble using an optimization method for rank allocation among the original block tensors. Our simulation results show the viability of the proposed tensor framework when applied to image ensembles of faces.

This paper addresses the problem of identifying the top-k information hubs in a social network. Identifying top-k information hubs is crucial for many applications such as advertising in social networks where advertisers are interested in identifying hubs to whom free samples can be given. Existing solutions are centralized and require time stamped information about pair-wise user interactions and can only be used by social network owners as only they have access to such data. Existing distributed algorithms suffer from poor accuracy. In this paper, we propose a new algorithm to identify information hubs that preserves user privacy. Our method can identify hubs without requiring a central entity to access the complete friendship graph. We achieve this by fully distributing the computation using the Kempe-McSherry algorithm, while addressing user privacy concerns. We evaluate the effectiveness of our proposed technique using three real-world data set; The first two are Facebook data sets containing about 6 million users and more than 40 million friendship links. The third data set is from Twitter and comprises of a little over 2 million users. The results of our analysis show that our algorithm is up to 50% more accurate than existing algorithms. Results also show that the proposed algorithm can estimate the rank of the top-k information hubs users more accurately than existing approaches.

Identifying leaders and followers in online social networks is important for various applications in many domains such as advertisement, community health campaigns, administrative science, and even politics. In this paper, we study the problem of identifying leaders and followers in online social networks using user interaction information. We propose a new model, called the Longitudinal User Centered Influence (LUCI) model, that takes as input user interaction information and clusters users into four categories: introvert leaders, extrovert leaders, followers, and neutrals. To validate our model, we first apply it to a data set collected from an online social network called Everything2. Our experimental results show that our LUCI model achieves an average classification accuracy of up to 90.3% in classifying users as leaders and followers, where the ground truth is based on the labeled roles of users. Second, we apply our LUCI model on a data set collected from Facebook consisting of interactions among more than 3 million users over the duration of one year. However, we do not have ground truth data for Facebook users. Therefore, we analyze several important topological properties of the friendship graph for different user categories. Our experimental results show that different user categories exhibit different topological characteristics in the friendship graph and these observed characteristics are in accordance with the expected ones based on the general definition of the four roles.

Typical consumer digital cameras sense only one out of three color components per image pixel. The problem of demosaicing deals with interpolating those missing color components. In this paper, we present compressive demosaicing (CD), a framework for demosaicing natural images based on the theory of compressed sensing (CS). Given sensed samples of an image, CD employs a CS solver to find the sparse representation of that image under a fixed sparsifying dictionary Ψ. As opposed to state of the art CS-based demosaicing approaches, we consider a clear distinction between the interchannel (color) and interpixel correlations of natural images. Utilizing some well-known facts about the human visual system, those two types of correlations are utilized in a nonseparable format to construct the sparsifying transform Ψ. Our simulation results verify that CD performs better (both visually and in terms of PSNR) than leading demosaicing approaches when applied to the majority of standard test images.

Intra-flow network coding (NC) is an innovative technique that has potential to improve multicast performance in wireless mesh networks (WMNs) by allowing intermediate forwarding nodes (FNs) to use coding and overhearing to reduce the number of required transmissions. However the benefits of the NC technology are limited unless there are protocols to exploit its capabilities. The existing intra-flow NC-based multicast protocols are all based upon the conventional end-to-end transport principle. By such a principle, intermediate FNs are unable to accurately determine the minimum number of coded packets they should transmit in order to ensure successful data delivery to the destinations, and hence redundant packets can be injected into the network, leading to performance degradation. Furthermore, the existing protocols cannot handle the bandwidth heterogeneity of multicast receivers very well. We argue that a receiver-driven hop-by-hop transport approach is more suitable for intra-flow NC and these two techniques can create synergy by enabling cooperation among the FNs. In this paper we propose HopCaster, a novel protocol that incorporates intra-flow NC with hop-by-hop transport to achieve high-throughput reliable multicast and to solve the heterogeneous receiver issue. It completely eliminates the need for estimating the number of coded packets to be transmitted by a FN and avoids transmission of redundant packets, as well as simplifies multicast management and congestion control. Moreover, HopCaster employs a cross-layer rate adaptation mechanism that optimizes radio transmission rate in hop-by-hop multicast by taking into consideration next-hop node population changes. Our evaluations show that HopCaster outperforms the existing NC-based reliable multicast protocol.

In this paper, we tackle real-time learning of a dictionary D from compressive measurements Y of an image X. Existing dictionary learning algorithms are inapplicable because compressive samples Y = ΦX are incomplete and can be arbitrary linear combinations of different pixels. Our strategy is to learn a dictionary of the form D = ΨΘ, which represents compressible dictionaries with respect to the base dictionary Ψ. We show that our method for learning dictionaries during compressive image recovery can improve the recovery results by up to 3 dBs for general random sampling matrices.

Key frame extraction algorithms select a subset of the most informative frames from videos. Key frame extraction finds applications in several broad areas of video processing research such as video summarization, video indexing, and prints from video. In this paper, an image epitome [1][2] based method to extract key frames from unstructured consumer videos is presented. In the proposed approach, we exploit image epitome to measure dissimilarity between frames of the input video. The dissimilarity scores are further analyzed using a min-max approach to extract the desired number of key frames from the input video. The proposed approach does not require shot(s) detection, segmentation, or semantic understanding. A comparison of the results obtained by this method with the ground truth agreed by multiple judges clearly indicates the feasibility of the proposed approach.

This paper presents the design of a new paradigm for a content-aware wireless MAC layer that is optimized for wireless video (first and foremost) while targeting fairness and stability among competing video traffic, and among video and non-video traffic. Hence, we refer to the proposed MAC framework as the FAST (Fair And STable) protocol. FAST employs two parameters for each packet, a quality value and a time-to-live value. Based on these parameters, FAST is designed on a multiclass priority queuing system that classifies the incoming traffic according to the content of each traffic flow and further identifies different priorities within each video content. We develop analytical frameworks to formulate channel allocation based on video/non-video fairness and video stability requirements as a joint bandwidth maximization and scheduling optimization problem. We incorporate these frameworks to design and simulate a content-aware channel access mechanism, which utilizes video traffic content classifications and users demand in conjunction with stability and fairness requirements at the MAC layer to allocate wireless channels to individual wireless users. Our simulation results show that FAST provides significant improvements in packet-loss-ratio, delay, overall fairness, and stability parameters when compared with leading access control mechanisms over 4G/LTE environment.

The inherent many-to-one flow of traffic in wireless sensor networks (WSNs) produces a skewed distribution of energy consumption rates, leading to the early demise of those sensors that are critical to the ability of surviving nodes to communicate their measurements to the base station. Numerous previous approaches aimed at balancing the consumption of energy in wireless networks are either too complex or do not address problems unique to the flow of traffic in WSNs. In this article, we propose the use of a dynamic programming algorithm (DPA), an operational, low-complexity algorithm, used in conjunction with four different route discovery algorithms. We perform complexity analysis, statistical evaluation of changes in power consumption rates effected, and verify spatial redistribution of energy consumption of sensors in the network. Our results on multihop networks of 100 randomly placed nodes show that, on average, the two best performing variants of DPA yield a reduction of up to 28% and 36% in power consumption rate variance at the cost of raising average power consumption by 15% and 21%, respectively. Computational complexities of DPA variants range from O ( N 3 ) to O ( N 4 ), which is significantly lower than linear search of the solution space of O ( N ! N i ). Analysis by diffusion plots shows that DPA reduces power consumption of sensors that experience the highest power consumption under the shortest path routes.

In this paper, we explore the notion of using frames to project sensed colors within their inherently 3D space onto a larger number of color basis vectors. In particular, we develop a new frame design, Incoherent Color Frames (ICF), which can include an arbitrary number of incoherent color vectors. An ICF frame possesses key desired properties including the ability to sparsify colors in 3D and to decorrelate color channels utilizing a spatial-frequency selective strategy. We present a low complexity algorithm for constructing ICF frames targeted for the problem of image demosaicing. Our simulation results show that when incorporating the proposed ICF within a Compressive Demosaicing (CD) framework, significant visual improvements can be achieved when compared with traditional and Compressed Sesnsing-based demosaicing solutions.

Identifying the most influential nodes in social networks is a key problem in social network analysis. However, without a strict definition of centrality the notion of what constitutes a central node in a network changes with application and the type of commodity flowing through a network. In this paper we identify social hubs, nodes at the center of influential neighborhoods, in massive online social networks using principal component centrality (PCC). We compare PCC with eigenvector centrality's (EVC), the de facto measure of node influence by virtue of their position in a network. We demonstrate PCC's performance by processing a friendship graph of 70, 000 users of Google's Orkut social networking service and a gaming graph of 143, 020 users obtained from users of Facebook's 'Fighters Club' application.

In this paper, we develop a Reliable Energy Adept Link-layer (REAL) protocol for power-constrained networks to provide reliable data transmissions among battery-operated wireless nodes. REAL dynamically performs error recovery with respect to the overall distortions in the system and the available energy at wireless nodes. REAL employs rate-adaptive low density parity check (LDPC) codes for error recovery. We develop a theoretical model to estimate the distortion imposed by wireless channels and to compute the likelihood of successful decoding at the link-layer. Next, we present a recovery model using Markovian decision process to formulate the optimal decoding policy as a linear optimization problem. We design and implement the REAL protocol which provides system reliability with efficient energy utilization. We demonstrate experimentally that REAL achieves 5%-40% throughput and 3%-18% energy consumption improvements over channel traces with varying bit error rates (BERs) collected on 802.15.4 environment. Further, REAL shows 5-15db PSNR better video quality over various realtime video scenarios.

In this paper, we develop Prioritized Adaptive Code-Enhanced (PACE) link-layer protocol to achieve preferred data recovery order across connections, while maintaining stable and reliable data flow over a wireless network. We classify link-layer traffic arrivals into different priorities based on delay constraint and distortion associated with that traffic. We formulate the link-layer buffer as a multiclass M/G/1 priority queuing system and the decoding process by a nonhomogeneous Geometric density function. This formulation enables the determination of an optimal dynamic decoder scheduling for heterogeneous link-layer traffic. PACE employs rate-adaptive Low Density Parity Check (LDPC) codes for error recovery. We demonstrate experimentally that PACE reduces the throughput-delay cost by 20%-70% in comparison with the IEEE802.11 ARQ and Hybrid ARQ (HARQ) protocols. Further, it achieves 20%-60% throughput and 2-10dB PSNR improvements in channel bandwidth utilization and realtime video playback quality.

The utility of Compressed Sensing (CS) for demosaicing of images captured using random panchromatic color filter arrays (CFA) has been investigated in [1]. Meanwhile, most camera manufacturers employ periodic CFAs such as the popular Bayer CFA. In this paper, we derive a CS-based solution to demosaicing images captured using the general class of periodic CFAs. It is well known that periodic CFAs can be designed to effectively separate luminance and chrominance frequency bands [2, 3]. We employ this ability to reduce artifacts associated with luminance-chrominance overlap at the solver side. We show that the modified compressive demo-saicing method coupled with the additional constraint that chrominance channels have smooth surfaces achieves further improved results for most periodic CFAs.

This paper addresses the problem of identifying the top-k information hubs in a social network. Identifying top-k information hubs is crucial for many applications such as advertising in social networks where advertisers are interested in identifying hubs to whom free samples can be given. Existing solutions are centralized and require time stamped information about pair-wise user interactions and can only be used by social network owners as only they have access to such data. Existing distributed and privacy preserving algorithms suffer from poor accuracy. In this paper, we propose a new algorithm to identify information hubs that preserves user privacy. The intuition is that highly connected users tend to have more interactions with their neighbors than less connected users. Our method can identify hubs without requiring a central entity to access the complete friendship graph. We achieve this by fully distributing the computation using the Kempe-McSherry algorithm to address user privacy concerns. To the best of our knowledge, the proposed algorithm represents an arguably first attempt that (1) uses friendship graphs (instead of interaction graphs), (2) employs a truly distributed method over friendship graphs, and (3) maintains user privacy by not requiring them to disclose their friend associations and interactions, for identifying information hubs in social networks. We evaluate the effectiveness of our proposed technique using a real-world Facebook data set containing about 3.1 million users and more than 23 million friendship links. The results of our experiments show that our algorithm is 50% more accurate than existing distributed algorithms. Results also show that the proposed algorithm can estimate the rank of the top-k information hubs users more accurately than existing approaches.

Despite major developments in link-layer design to address reliability issues associated with the wireless communication (in a presence of heavy noise), these efforts fall short on many fronts. This includes a clear demonstration regarding the viability of a truly reliable link-layer capable of providing a minimal level of guaranteed sustainable flows for the higher layers. In this paper, we present an analytical and experimental study to design and implement a reliable wireless link-layer that provides sustainable flow control. We develop an experimental platform using Software Radio Defined (SDR) technology with the Universal Software Radio Peripheral (USRP) frontend to capture and measure the behavior of an error process imposed on a wireless channel. Next, we design a Reliable And StablE (RASE) link-layer protocol to provide reliability (by achieving optimal throughput) and stability (by ensuring a sustainable traffic flow) for realtime and non-realtime wireless communications. We then incorporate the RASE protocol into the SDR-USRP platform to investigate the level of throughput and realtime stability achieved in comparison with the IEEE802.11 ARQ and the FEC-based HARQ protocols. We demonstrate experimentally that RASE provides 20%-50% improved reliability. In addition, realtime video communication experiments show a 2-8dB PSNR gain in playback quality.

Due to the broadcast nature of wireless networks they have been a natural platform for applying Network Coding (NC). Wireless networks can benefit significantly from NC due to their broadcast nature and the opportunity of enhancing bandwidth utilization. In this paper, we develop Multi-Generation Mixing (MGM), which is a generalized approach for generation based network coding. With traditional generation based NC sender packets are grouped in generations where encoding and decoding are performed on packets that belong to the same generation. In scenarios where losses cause insufficient reception of encoded packets, NC losses occur. NC losses are expensive; the minimum unit of loss is the loss of one generation. The proposed MGM framework allows the encoding among generations for the purpose of enhancing NC decodability. With MGM in scenarios where insufficient number of encodings received of a generation, it is still possible to recover the generation using data encoded in other generations. We develop MGM encoding and decoding approaches, and demonstrate the improvements in performance achieved by MGM. Further, a canonical analytical model for MGM network coding is developed, and, extensive simulations over random wireless networks experiencing random packet losses are presented.

With the rise of mobile media devices, resizing an image or video to fit a screen of arbitrary size has become an important topic. In general, arbitrary resizing does not preserve the original image aspect ratio, and hence, can introduce significant visual distortion. Content-aware retargeting methods have been proposed to preserve salient features of an image or video even under arbitrary changing of the aspect ratio. While these are useful tools, little attention has been given to the development of an efficient retargeting-driven representation of images. Such representation, for example, can lead to efficient distribution of media destined for retargeting at a variety of devices with different aspect ratios. We propose a new framework, seamlets, which utilizes the discrete wavelet transform (DWT) to perform content-aware arbitrary media resizing. In essence, seamlets provide an efficient multi-resolution representation for retargeting applications. The seamlet transform elegantly generalizes the DWT and seam carving, which is an intriguing retargeting method. The result is a new image framework, seamlets, that inherits the benefits of retargeting and wavelets.

Recent studies have shown that standalone anomaly classifiers used by network anomaly detectors are unable to provide acceptable accuracies in real-world deployments. To achieve higher accuracies, Network Anomaly Detection Systems (NADSs) now use multiple classifiers whose outputs are combined to formulate an aggregate anomaly score. Judicious methods of combining these classifiers' outputs are largely unexplored. In this paper, we propose a novel information-theoretic combining method which caters for the individual classifiers' accuracies in a multi-classifier NADS. We first show that existing combining schemes designed for or adapted to the problem of multi-classifier NADS combining do not provide good accuracies because they do not use individual classifiers' detection and false alarm rates in the combining process. Furthermore, we reveal that an accurate multi-classifier NADS, in addition to catering for the mean accuracy rates, must also consider the classifiers' variances during combining. Therefore, we propose a Standard Deviation normalized Entropy of Accuracy (SDnEA) method for classifier combining. Using 9 prominent classifiers operating on two publicly-available traffic datasets, we show that around 3%-10% increase in detection rate and a 40% decrease in false alarm rate over existing combining techniques can be provided by the proposed information-theoretic NADS combining technique.

With the abundance and variety of display devices, novel image resizing techniques have become more desirable. Content-aware image resizing (retargeting) techniques have been proposed that show improvement over traditional techniques such as cropping and resampling. In particular, seam carving has gained attention as an effective solution, using simple filters to detect and preserve the high-energy areas of an image. Yet, it stands to be more robust to a variety of image types. To facilitate such improvement, we recast seam carving in a more general framework and in the context of filter banks. This enables improved filter design, and leads to a multiscale model that addresses the problem of scale of image features. We have found our generalized multiscale model to improve on the existing seam carving method for a variety of images.

A typical consumer digital camera uses a Color Filter Array (CFA) to sense only one color component per image pixel. The original three-color image is reconstructed by interpolating the missing color components. This interpolation process (known as demosaicing) corresponds to solving an under-determined system of linear equations. In this paper, we show that by replacing the traditional CFA with a random panchromatic CFA, recent results in the emerging field of Compressed Sensing (CS) can be used to solve the demosaicing problem in a novel way. Specifically, during the image reconstruction process, we exploit the fact that the multi-dimensional color of each pixel has a compressible representation in a (possibly overcomplete) color system. While adhering to the “single color per pixel sensing” constraint at the sensing stage, during the reconstruction process we utilize the inter-pixel correlation by exploiting the compressible representation of the overall image in some sparsifying bases. Depending on the CFA, sparsifying bases and the color system, we form an underdetermined system of linear equations and find the sparsest solution for the color image by utilizing a CS solver. We illustrate that, for natural images, the proposed Compressive Demosaicing (CD) framework visually outperforms leading demosaicing methods in a consistent manner; in many cases it achieves clear visible improvements in a significant way.

We consider the problem of recovering a signal/image (x) with a k-sparse representation, from hybrid (complex and real), noiseless linear samples (y) using a mixture of complex-valued sparse and real-valued dense projections within a single matrix. The proposed Hybrid Compressed Sensing (HCS) employs the complex-sparse part of the projection matrix to divide the n-dimensional signal (x) into subsets. In turn, each subset of the signal (coefficients) is mapped onto a complex sample of the measurement vector (y). Under a worst-case scenario of such sparsity-induced mapping, when the number of complex sparse measurements is sufficiently large then this mapping leads to the isolation of a significant fraction of the k non-zero coefficients into different complex measurement samples from y. Using a simple property of complex numbers (namely complex phases) one can identify the isolated non-zeros of x. After reducing the effect of the identified non-zero coefficients from the compressive samples, we utilize the real-valued dense submatrix to form a full rank system of equations to recover the signal values in the remaining indices (that are not recovered by the sparse complex projection part). We show that the proposed hybrid approach can recover a k-sparse signal (with high probability) while requiring only m ≈ 3k3√(n/2k) real measurements (where each complex sample is counted as two real measurements). We also derive expressions for the optimal mix of complex-sparse and real-dense rows within an HCS projection matrix. Further, in a practical range of sparsity ratio (k/n) suitable for images, the hybrid approach outperforms even the most complex compressed sensing frameworks (namely basis pursuit with dense Gaussian matrices). The theoretical complexity of HCS is less than the complexity of solving a full-rank system of m linear equations. In practice, the complexity can be lower than this bound.

Network coding (NC) within wireless sensor networks (WSNs) can be viewed as the mapping of efficient channel codes to the data generated within the network. In particular, this perspective of code-on-network-graphs (CNG) can be exploited to map source data generated within WSN (of size K) to a variable nodes subset in low-density parity check (LDPC) codes. The resulting fixed size symbol stream when transmitted through the network suffers erasures. At sink, an average ofzsource symbols can be recovered by employing belief propagation decoding. In this paper, we determine CNG code ensembles that achieve maximal recovery (z/K) for different erasure rates and network topological constraints corresponding to node transmission range. An analytic framework to predict code performance under transmission range constraints is developed. Additionally, necessary condition for code stability was derived using fixed-point stability analysis. Optimal solutions for a WSN with 1000 nodes are determined using differential evolution algorithm. We outline a distributed algorithm for generating a sequence of encoded symbols adhering to the designed code ensemble. The performance of the designed CNG code is demonstrated to be superior to random NC and growth code based ensembles, as well as resilient to network size and inter-connectivity variations.

One of the key factors underlying the popularity of Low-density parity-check (LDPC) code is its iterative decoding algorithm that is amenable to efficient hardware implementation. Even though different variants of LDPC iterative decoding algorithms have been studied for its error-correcting properties, an analytical basis for evaluating energy efficiency of LDPC decoders has not been reported. In this paper, we present a framework of a parameterized LDPC decoding algorithm that can be optimized to produce sparse representation of communication messages used in iterative decoding. The sparsity of messages is determined by its differential entropy and has been used as a theoretical metric for determining the energy efficiency of an iterative LDPC decoder. At the core of the proposed algorithm is margin propagation (MP) which approximates the log-sum-exp function used in conventional sum-product (SP) decoders by a piecewise linear (PWL) function. Using Monte-Carlo simulations, we demonstrate that the MP decoding leads to a significant reduction in message entropy compared to a conventional SP decoder, while incurring a negligible performance penalty (less than 0.03 dB). The proposed work therefore lays the foundation for design of parameterized LDPC decoders whose bit-error-rate performance can be effectively traded-off with respect to different energy efficiency constraints as required by different set of applications.

In this work, we explore and evaluate data reliability in wireless sensor networks (WSNs) using a novel approach that promises to increase the net network throughput with greater energy efficiency than prevalent schemes. Our results suggest that, under investigated scenarios, proposed approach can achieve reliable communication without fast depleting energy resources of individual sensor nodes. In particular, we propose a framework in which nodes collaborate, in a distributed fashion, to provide robustness against channel induced errors as data traverses the multi-hop network. Our scheme builds on two core concepts: partial decoding of incoming packets at intermediate nodes & directed diffusion of interest over the network for selective event based reliable path reinforcement. In addition, we use path diversity with selective budgeting of network energy resources to achieve data reliability. We show that the proposed scheme outperforms conventional error robustness approaches by considerable margin for a given network energy budget. We further discuss energy-distortion tradeoffs that may lead to greater network lifetime along-with maintaining a desired level of throughput. We use iteratively decodable LDPC codes to illustrate efficiency of our scheme.

Leading compressed sensing (CS) methods require m = O (k log(n)) compressive samples to perfectly reconstruct a k-sparse signal x of size n using random projection matrices (e.g., Gaussian or random Fourier matrices). For a given m, perfect reconstruction usually requires high complexity methods, such as Basis Pursuit (BP), which has complexity O(n3). Meanwhile, low-complexity greedy algorithms do not achieve the same level of performance (as BP) in terms of the quality of the reconstructed signal for the same m. In this paper, we introduce a new CS framework, which we refer to as Randomness-in-Structured Ensemble (RISE) projection. RISE projection matrices enable compressive sampling of image coefficients from random locations within the k-sparse image vector while imposing small structured overlaps. We prove that RISE-based compressed sensing requires only m = ck samples (where c is not a function of n) to perfectly recover a k-sparse image signal. For the case of n ≤ O(k2), the complexity of our solver is O(nk) which is less than the complexity of the popular greedy algorithm Orthogonal Matching Pursuit (OMP). Moreover, in practice we only need m = 2k samples to reconstruct the signal. We present simulation results that demonstrate the RISE framework's ability to recover the original image with higher than 50 dB PSNR, whereas other leading approaches (such as BP) can achieve PSNR values around 30 dB only.

This paper presents the 'Poor Man's SIMO System' (PMSS) which combines two ideas, cooperative communication and diversity combination, to reduce packet losses over links in Wireless Sensor Networks (WSN). The work is based on the IEEE 802.15.4 standard and is distinct from previous works that apply the same concepts because it foregoes the need for any changes to mote hardware. We describe a Poor Man's SIMO System protocol that governs the cooperation between receivers. Three diversity combination methods are evaluated including selection diversity, equal gain and maximal ratio combining. The latter relies on a model of the instantaneous Bit Error Rate (BER) driven by Channel State Information (CSI), i.e. Received Signal Strength Indication (RSSI) and Link Quality Indication (LQI). First, we demonstrate the PMSS on residual bit error traces in a fully reproducible manner. This is followed by an implementation of PMSS in C# on the .NET Micro Framework edition of the recently released Imote2 WSN mote platform. Both, trace based analysis and implementation demonstrate significant improvements over the single receiver baseline configuration. We deliberately verified PMSS by residual bit error traces and implementation to avoid the use of simulators that depend on abstract models of wireless channels.

In this paper we describe a practical implementation of a distributed video codec deployed on a real visual sensor platform, viz. the MicaZ/Cyclops platform. The codec supports two encoding schemes, one employs Discrete Cosine Transform (DCT) and the other operates on raw pixels. DCT scheme is more costly in terms of computational power consumption, on the other hand it leads to more compression and hence less transmission power consumption. At the same time the DCT scheme may not necessarily achieve minimal overall power consumption (computation and transmission). In this paper we show that the choice of either scheme (DCT or pixel based) depends on the tolerable distortion and power consumption. Results show that for a range of achievable video quality values the DCT scheme demonstrates less overall power consumption. On the other hand for another range of video quality the pixel scheme results in less overall power consumption.

In this paper we apply a generalized approach for practical Network Coding (NC) called Multi-Generation Mixing (MGM) in networks communicating scalable video contents. NC has been a viable approach of communication in packet loss networks. On the other hand NC losses are expensive. NC losses reduce the ability of receiver to decode packets, and hence may severely degrade the quality of recovered video. MGM employs the layering of scalable video streams to enhance the reliability of communication. MGM provides unequal protection for the different video layers such that the overall reliability of video communication is improved. With MGM instead of having enhancement layers just dependent on lower layers, enhancement layers support the recovery of lower layers. This is done by network encoding packets of lower layers in higher layers. Through extensive simulations, we show that MGM highly improves the quality of recovered video.

In this work, we propose an optimal joint bit allocation framework in order to provide unequal error protection (UEP) for scalable video coding. In the proposed scheme, video sequence is coded based on scalable video coding (SVC) extension of the H.264/AVC standard. We generate different streams according to changing quantization parameters. Each stream divides into different sub-streams in terms of time and quality scalability. These sub-streams with different significance levels are channel-coded using low density parity check (LDPC) codes with different coding rates respectively. Motivated by optimally allocating the bits to achieve resilience against channel induced corruptions, we propose an algorithm that optimally allocates the channel coding rates using a dynamic programming approach. We employ a probability model for decoding failure using LDPC codes. The results achieved clearly establish that the proposed framework can optimally allocate LDPC codes with varying coding rates to different sub-streams with the limited bandwidth.

In this work, we present a distributed framework for provision of reliability to visual data in wireless multimedia sensor networks (WMSNs). Individual sensor nodes cooperate, in a pair-wise fashion, to allocate processing resources within the network which are then used to partially decode the incoming data as it traverses the multi-hop network. We show that the proposed framework outperforms the conventional end-to-end channel coding by considerable margin in terms of energy efficiency for desired multimedia content delivery.

Real-time video communication over wireless channels is subject to information loss since wireless links are error-prone and susceptible to noise. Popular wireless link-layer protocols, such as retransmission (ARQ) based 802.11 and hybrid ARQ methods provide some level of reliability while largely ignoring the latency issue which is critical for real-time applications. Therefore, they suffer from low throughput (under high-error rates) and large waiting-times leading to serious degradation of video playback quality. In this paper, we develop an analytical framework for video communication which captures the behavior of real-time video traffic at the wireless link-layer while taking into consideration both reliability and latency conditions. Using this framework, we introduce a delay constraint packet embedded error control (DC-PEEC) protocol for wireless link-layer. DC-PEEC ensures reliable and rapid delivery of video packets by employing various channel codes to minimize fluctuations in throughput and provide timely arrival of video. In addition to theoretically analyzing DC-PEEC, the performance of the proposed scheme is analyzed by simulating real-time video communication over ldquorealrdquo channel traces collected on 802.11 b WLANs using H.264/AVC JM14.0 video codec. The experimental results demonstrate performance gains of 5-10 dB for different real-time video scenarios.

Network Coding (NC) is an emerging networking approach that improves overall throughput over packet networks. Meanwhile, traditional NC approaches have limited advantages under certain network conditions, such as sparse connectivity and high losses; this is especially true for real-time applications. In this paper, we propose and analyze a generalized approach of network coding, which is based on Multi-Generation Mixing (MGM). As we demonstrate in this paper, MGM-based NC improves the performance of real-time data communications under scenarios of sparse connectivity and high loss rates. Under such scenarios, practical network coding not only fail to achieve any improvements; on the contrary it may lead to performance degradations. The analytical as well as the simulation studies we present in this paper show major improvements that can be achieved in situations where practical network coding is not a viable option. In particular, we demonstrate major gains in PSNR video quality under MGM-based network coding.

In this work we analyze the nature of the error process in IEEE 802.15.4 low rate-wireless personal area network (LR-WPAN) channels experiencing varying levels of interference from nearby 802.11b/g networks co-habiting the unlicensed 2.4 GHz industrial-scientific-medical (ISM) band. The entire analysis is performed on a set of residual bit error traces, the data collection methodology for which is described in detail. These traces allow us to identify individual bit positions in a received packet that were in error. We use lagged phase space analysis, a technique commonly used in the analysis of chaotic systems. The analysis uncovers the stable nature of the BER process during short bursts. This simple behavior of the bit error rate (BER) can be captured by three parameters; the burst length of consecutive good packets, bad packets, and the distribution with which different BERs occur. Using collected residual bit error traces, we model the BER process by estimating the probability distributions of its parameters using maximum likelihood estimates (MLE).

Measurement of a wireless channel's memory length is of great, fundamental interest to designers of communication systems. The principal tool used for measuring the memory length of wireless channels has been the correlation function which has been shown to work well for the measurement of channel memory at the level of individual bits and symbols. However, as we demonstrate, applied to the packet-level bit-error process of 802.15.4 wireless channels, the correlation function fails to provide us with a clear answer even in a wide variety of channels. To overcome this limitation we propose the use of a new information theoretic measure. We introduce the use of relative mutual information and demonstrate its application for measuring the memory length of the BER process in sensor 802.15.4 wireless channels.

Several works have attempted to determine whether the process that introduces errors into transmissions over a wireless channels is long range dependent (LRD). Most of these efforts were focused on IEEE 802.11b/g channels. Knowledge of LRD is of significant interest to designers of wireless network systems. This experiment-based study attempts to gain a better understanding of the standardized IEEE 802.15.4 low rate-wireless personal area network (LR-WPAN) specification. We determine whether the bit, symbol and packet level error processes are LRD or not. Is the memory length of the channel measurable and finite, or is it sufficiently long to qualify it as an LRD process? We record channel behavior by capturing all transmissions at the receiver including the ones that would under normal operation be dropped and use these to generate residual error traces. We perform statistical data analysis of these traces using various tools for scenarios with and without interference from 802.11b/g sources. For the bit and symbol level error process we conclude that channel memory remains fixed at 2 bits and 2 symbols respectively regardless of channel and environment. For the packet level error process we conclude that the perception of on and off LRD is actually due to interference from co-located 802.11b/g networks.

In this paper, we propose an endpoint-based anomaly detection scheme that detects computer worms by comparing the current traffic patterns of each host to the corresponding benign traffic profile of the host. To detect deviations in the traffic patterns, we employ the information-theoretic Kullback-Leibler (K-L) divergence measure which estimates the distance between the distribution of source/destination ports engaged in current communication and that observed in the legitimate host traffic collected earlier. We use a small subset of traces obtained from endpoints in home, university, and office environments to build benign traffic profiles of studied endpoints. Endpoint traces are then infected with both real and simulated worms to examine the performance of our detection mechanism. To perform automated, real-time worm detection, we use Support Vector Machines (SVMs) that are trained using the K-L divergence values. Our results show that the proposed worm detector provides almost 100% detection with negligible false- alarm rates and significantly surpasses the accuracy of existing anomaly detectors.

Internet blackholes have emerged as very effective tools for monitoring changes in the Internet's traffic behavior. Prior studies have shown that traffic observed at a blackhole contains valuable information about emerging malware. While blackhole traffic has been effectively used for attack forensics, a systematic method of leveraging this traffic for online Internet- scale anomaly detection is not available. In this paper, we propose a novel technique to detect malware outbreaks using deviations in a robust statistical model of a blackhole's traffic. First, we introduce a novel and accurate Piecewise Poisson process Model (PPM) of traffic observed at an Internet Motion Sensor (IMS) blackhole which provides a statistical quantification of the intensity or rate of incoming traffic at a blackhole, which can in turn be used to detect malware outbreaks. After establishing the accuracy of the proposed PPM model, we develop a regression model that can characterize variations in the PPM's traffic rates. Once an accurate model of traffic rates is in place, malware outbreaks can be detected using deviations from the model's likely statistical patterns. After removing simple deterministic patterns, we observe that a blackhole's traffic rate residuals have a skewed and heavy-tailed behavior. Consequently, we employ a stable distribution that models variations in traffic rate residuals with very high accuracy. Finally, we propose an online detection mechanism that utilizes deviations from the rate residual distribution of blackhole traffic data to detect malware outbreaks. Experimental results using the IMS data for approximately one year show that the proposed mechanism accurately detects malware outbreaks in a timely manner.

In pixel-domain distributed video coding (DVC), due to the largely translational nature of motion, residue errors in the side-information frame are often clustered together. These clusterings can be exploited to reduce the number of syndrome bits required to successfully perform low density parity check (LDPC) decoding, and therefore improve the overall rate-distortion performance. We shall see that using alternate iterations of LDPC syndrome decoding and Baum-Welch channel estimation proves to be an efficient scheme for exploiting the spatial clustering of errors in pixel-domain DVC. In this paper we demonstrate that a sparser power-law based scheduling of the channel estimation iteration leads to significant reduction in estimation complexity (around 83% reduction) for a small loss in rate-distortion performance (less than 0.75 dB). This sparser scheduling of channel estimation iterations can potentially improve decoding delays.

Knowledge of the error process and related channel parameters in wireless networks is invaluable and highly instrumental in a broad range of applications. Under the IEEE 802.15.4 Low Rate-Wireless Personal Area Networks (LR- WPAN) standard, compliant devices are capable of providing two pieces of information about the channel conditions along with each received packet, the Link Quality Indication (LQI) and Received Signal Strength Indication (RSSI). Together they constitute a form of Channel State Information (CSI). This work is based on statistical and information theoretic analysis of a very extensive data set of wireless channel traffic between a transmitter and receiver, called packet traces. Data is collected in a variety of documented environments. To our knowledge, this is the first detailed trace collection effort for this type of network. The traces distinguish themselves from data sets of other studies in that they record individual bit errors as well as packets that are never detected by receivers. First, we provide a detailed analysis of the IEEE 802.15.4 wireless channel. More specifically, we provide a detailed analysis of the Bit Error Rate (BER) process at individual bit and on a packet-by-packet basis. We explore the relationship between the packet-level BER process and the LQI and RSSI processes (also observable on a packet-by- packet). The analysis shows that measurements of both LQI and RSSI provide information that allows us to reduce uncertainty about the BER. Secondly, we develop a model of the BER process that is driven by observable CSI parameters. Thirdly, we continue our analysis with measurements of channel memory at the packet and bit level. We determine that the wireless channel 2 bits memory. At the packet level we observe that the amount of channel memory is more varied.

Recent advances have shown that channel codes can be mapped onto networks to realize efficient Network Coding (NC); this has led to the emergence of Code-on-Network-Graphs (CNG). Traditional CNG approaches (e.g Decentralized Erasure Codes) focus on a generating a sequence of encoded symbols from a given input source (of size K), such that the original symbols can be recovered from any subset of the encoded symbols of size equal to or slightly larger than K. However in all cases the number of source symbols recovered falls rapidly if the number of encoded symbols received falls below K. In this paper we determine the CNG code-ensembles (under statistical toplogy constraint) which result in maximal recovery of WSN source data (for different erasure-rates), thereby minimizing the deterioration in data recovery. We also perform fixed point stability analysis on the underlying LDPC code ensemble. We then propose a distributed algorithm for generating a sequence of encoded symbols adhering to the designed code ensemble. Optimal solutions for a sensor network with 1000 nodes is determined using the Differential Evolution algorithm, and the solution sensitivity to variance in number of sensor nodes and node-interconnectivity is evaluated.

Research has shown that fluid limits of Markov processes can be used to obtain closed form expressions for the evolution of the ripple-size. In this work we extend the above analysis to generalized LT (GLT) codes, which can be used to represent LT encoding (with priorities) over multiple data segments. In our analysis, we segregate the ripple into multiple colored ripples, where each color corresponds to a segment. We derive closed form expressions for the size of each ripple. We utilize these expressions to design GLT distributions, optimized for a desired intermediate and unequal recovery.

A primary focus of popular wireless link-layer protocols is to achieve some level of reliability using ARQ or Hybrid ARQ mechanisms. However, these and other leading link-layer protocols largely ignore the stability aspect of wireless communication, and rely on higher layers to provide stable traffic flow control. This design strategy has led to a great deal of inefficiency in throughput and to other major issues (such as the well-known TCP over-wireless performance degradation phenomenon and the numerous studies in attempt to fix it). In this paper, we propose a paradigm shift where both reliability and stability are targeted using an Automatic Code Embedding (ACE) wireless link-layer protocol. To the best of our knowledge this is the first effort to develop a theoretical framework for analyzing and designing a wireless link-layer protocol that targets system stability in conjunction with reliable communication. We present two distinct analytical frameworks to determine optimal code embedding rates which ensure system reliability and stability for wide range of traffic demand. An important conclusion of our analysis is that various traffic demand can be met using a packet-by-packet code embedding rate constraint that is independent of traffic type. We demonstrate experimentally that ACE provides both rapid and reliable point-to-point wireless data transmission for realtime and non-realtime traffic over real channel traces collected on 802.11b WLAN. We also have conducted extensive TCP simulations in conjunction with ACE; and we demonstrate the high level of efficiency and stability that can be achieved for TCP over ACE, while not making any changes to TCP. Further, the implementation of ACE for real-time video communication shows performance gains of 5-10dB over IEEE ARQ schemes. More importantly, ACE is layer oblivious and requires no changes to higher or lower PHY layers.

Reliable transmission is a challenging task over wireless LANs since wireless links are known to be susceptible to errors. Although the current IEEE802.11 standard ARQ error control protocol performs relatively well over channels with very low bit error rates (BERs), this performance deteriorates rapidly as the BER increases. This paper investigates the problem of reliable transmission in a contention free wireless LAN and introduces a packet embedded error control (PEEC) protocol, which employs packet-embedded parity symbols instead of ARQ-based retransmission for error recovery. Specifically, depending on receiver feedback, PEEC adaptively estimates channel conditions and administers the transmission of (data and parity) symbols within a packet. This enables successful recovery of both new data and old unrecovered data from prior transmissions. In addition to theoretically analyzing PEEC, the performance of the proposed scheme is extensively analyzed over real channel traces collected on 802.11b WLANs. We compare PEEC performance with the performance of the IEEE802.il standard ARQ protocol as well as contemporary protocols such as enhanced ARQ and the hybrid ARQ/FEC. Our analysis and experimental simulations show that PEEC outperforms all three competing protocols over a wide range of actual 802.11b WLAN collected traces. Finally, the design and implementation of PEEC using an adaptive low-density-parity-check (A-LDPC) decoder is presented.

Packet drops caused byresidueerrors(MAC-layer errors) can severely deteriorate the wireless video quality. Prior studies have shown that this loss of quality can be circumvented by using forward error correction (FEC) to recover information from the corrupted packets. The performance of FEC encoded video streaming is critically dependent upon the choice of source and channel coding rates. In practice, the wireless channel conditions can vary significantly, thus altering the optimal rate choices. Thus, it is essential to develop an architecture which can estimate the channel capacity and utilize this estimate for rate allocation. In this paper we develop such a framework. Our contributions consist of two parts. In the first part we develop a prediction framework that leverages the received packets' signal to silence ratio (SSR) indications and MAC-layer checksum as side information to predict the operational channel capacity. In the second part, we use this prediction framework for rate allocation. The optimal rate allocation is dependent upon thechannelcapacity, the distribution of the (capacity)predictionerrorand the rate-distortion (RD) characteristics of the video source. Consequently, we propose a framework that utilizes the aforementioned statistics for RD optimal rate adaptation. We exhibit the efficacy of the proposed scheme by simulations using actual 802.11b wireless traces, an RD model for the video source and an ideal FEC model. Simulations using source RD models derived from five different popular video codecs (including H.264), show that the proposed framework provides up-to 5-dB improvements in peak signal-to-noise ratio (PSNR) when compared with conventional rate-adaptive schemes.

We propose a structural design of multidimensional two-channel filter banks with desirable numbers of vanishing moments for the analysis and synthesis banks. We use a three-step lifting scheme as opposed to the conventional two-step lifting method in order to provide more symmetry between the analysis and synthesis filters. Design examples are also provided. H ( z) M M H̃ ( z)

Performance of cross-layer protocols that recommend the relay of corrupted packets to higher layers can be improved significantly by accurately inferring/predicting the bit error rate (BER) in the packets. In practice, higher layers observe the bits only after some hard decision. Hence physical layer link-quality indications, such as the signal strength of each individual bit, are not observable at higher layers. Therefore, it is essential to identify practically observable variables, which can be used for reasonably robust channel state inference/prediction (CSI/CSP). Here, inference specifically refers to estimating the BER in an already received packet, while prediction refers to anticipating the BER in a future packet. In this paper, we note that, in practical 802.11b devices, it is possible to acquire a Signal to Silence Ratio (SSR) indication and measure the background traffic intensity (p) on a per packet basis. This paper, thus presents a measurement-based study that analyzes the utility of SSR andpas side-information for CSI/CSP. In this work, we exploit the method of types to measure the robustness of the observable side-information. Our analysis and simulations based on an extensive set of actual 802.11b traces exhibit the practical utility of the considered observable variables.

Recent wireless measurement studies have revealed the presence of high-order memory in wireless bit-error channels. However, most wireless studies continue to employ the memory-less or 1st order Gilbert bit-error channels to design, analyze and verify wireless protocols and systems. The inaccuracies incurred by ignoring high-order channel memory are largely unexplored. This paper quantifies inaccuracies incurred by ignoring bit-level wireless channel memory in the context of two simple and commonly-used protocol metrics: (i) packet good put of an abstract unreliable protocol and (ii) number of retransmissions per packet for an abstract reliable protocol. We analytically derive expected values of these metrics in terms of the parameters of four models with varying levels of memory. We then train the models using actual 802.11b bit-error traces and use the analytical expressions to estimate the metrics. Comparison of the model-based estimates with actual values of the metrics derived from the traces shows that the memory-less and 1st order models incur significant inaccuracies. The remaining two models that capture high-order memory are very accurate in their estimates of the crucial good put and retransmission metrics.

In this paper, we propose an endpoint-based joint network-host anomaly detection technique to detect self- propagating malicious codes. Our proposed technique is based on the observation that on any endpoint there exists very high correlation between benign network sessions and the keystrokes that trigger these sessions. Specifically, users generally use a few keystrokes to trigger most of the benign network sessions. On the other hand, malicious sessions originating from a compromised endpoint will not have the session-keystroke correlation. We lever-age this observation in a novel information-theoretic framework that characterizes the session-keystroke correlation in terms of their mutual information. Changes in session-keystroke mutual information are used to detect malicious codes in an automated and real-time fashion. To evaluate the proposed anomaly detector, we use actual traffic and keystroke data collected on benign and infected endpoints. We show that the proposed anomaly detector provides almost 100% detection with negligible false-alarm rates and significantly surpasses the accuracy of existing techniques.

Retransmission based schemes are not suitable for energy constrained wireless sensor networks. Hence, there is an interest in including parity bits in each packet for error control. From an information-theoretic perspective the most efficient usage of network capacity can be achieved by performing full encoding/decoding at each node and using a variable rate in accordance with the link-quality. However, such an approach represents a major burden on power-constrained sensors. In this paper, we propose a more practical approach that is based on optimally distributing iterative channel decoding over sensor networks. In such a paradigm, the guarantee with which the base station, orcollector, gets the data from a sensor is a function of the processing within the intermediate nodes between source and destination (in-network processing). There are two extreme cases: a) Complete channel decoding at each hop and b) decoding only at the final destination. In this paper, we present a novel scheme in which intermediate nodes conduct partial decoding of LDPC coded packets. In this scheme each node is assigned some number of decoding iterations. The relay node conducts LPDC decoding for that number of iterations and forwards the packet, without ensuring a complete error correction. We show that such partial processing is sufficient to improve the end-to-end reliability significantly. Additionally, we show that it is feasible to tradeoff complexity/energy usage with distortion/reliability by varying the assignment of number of iterations. Finally, we present a low-complexity dynamic programming algorithm that optimally assigns iterations within the network to facilitate operation along an optimalenergy-distortioncurve.

Network channel coding (NCC) is a framework under which intermediate router/nodes employ encoding/decoding operations to facilitate an efficient multicast delivery of video. The network usage and video distortion is a function of the channel coding rates assigned to nodes in the network. In this paper, we investigate the tradeoff between the total bandwidth usage and a global distortion measure. We propose a dynamic programming based framework to identify the optimal transmission-distortion operating points. The proposed optimal allocation of coding rates is compared with an earlier version of NCC, network embedded FEC (NEF), which optimally places codecs of a fixed channel coding rate within the network. The proposed NCC scheme is shown to achieve significantly improved transmission distortion tradeoffs.

We design a new form of the Maximum Likelihood Decoding Algorithm (MLDA) in [1] for LT codes [3] with a small fraction of dense rows. In particular, we design degree distributions using the Robust Soliton Distribution (RSD) for the proposed MLDA based decoding of LT codes. We also estimate the computational complexity of the proposed LT based MLDA. Simulation results, which show the viability of the proposed MLDA decoding of LT codes, are also presented.

We propose a new family of nonredundant geometrical image transforms that are based on wavelets and directional filter banks. We convert the wavelet basis functions in the finest scales to a flexible and rich set of directional basis elements by employing directional filter banks, where we form a nonredundant transform family, which exhibits both directional and nondirectional basis functions. We demonstrate the potential of the proposed transforms using nonlinear approximation. In addition, we employ the proposed family in two key image processing applications, image coding and denoising, and show its efficiency for these applications.

Many recently proposed cross-layer protocols for wireless video, have advocated the relay of corrupted packet to higher layers. Such protocols lead to both errors and erasures at the compressed video application layer. We generically refer to such schemes as hybrid erasure-error protocols (HEEPs). In this paper, we analyze the utility of HEEPs for efficient transmission of video over wireless channels. In order to maintain the generic nature of the deductions in this paper, we base our analysis on two (rather abstract) communication schemes for wireless video: hybrid error-erasure cross-layer design (CLD) and hybrid error-erasure cross-layer design with side-information (CLDS). We make a comparative analysis of the channel capacities of these schemes over single and multi-hop wireless channels to identify the conditions under which the HEEPs can provide improved performance over conventional (CON) protocols. In addition, we employ Reed Solomon (RS) and low-density parity check (LDPC)-code-based forward-error correction (FEC) schemes to illustrate that the improvement in capacity can easily enable an FEC scheme employed in conjunction with a HEEP to provide improved throughput. Finally we compare the performance of CON, CLD, and CLDS in terms of video quality using the H.264 video standard. The simulation results show a significant advantage for the HEEPs

Wireless multimedia studies have revealed that forward error correction (FEC) on corrupted packets yields better bandwidth utilization and lower delay than retransmissions. To facilitate FEC-based recovery, corrupted packets should not be dropped so that maximum number of packets is relayed to a wireless receiver's FEC decoder. Previous studies proposed to mitigate wireless packet drops by a partial checksum that ignored payload errors. Such schemes require modifications to both transmitters and receivers, and incur packet-losses due to header errors. In this paper, we introduce a receiver-based scheme which uses the history of active multimedia sessions to detect transmitted values of corrupted packet headers, thereby improving wireless multimedia throughput. Header detection is posed as the decision-theoretic problem of multihypothesis detection of known parameters in noise. Performance of the proposed scheme is evaluated using trace-driven video simulations on an 802.11b local area network. We show that header detection with application layer FEC provides significant throughput and video quality improvements over the conventional UDP/IP/802.11 protocol stack

We propose a novel cross-layer header estimation methodology that can be used by UDP-based wireless multimedia applications to estimate corrupted packet headers, thereby realizing significant throughput improvements. The proposed methodology requires only minor modifications to the protocol stack at the receiver while no modifications are needed to senders or intermediate nodes. We formulate header estimation as a problem of maximum-likelihood estimation of known parameters in noise. We derive likelihood functions for two wireless channel models, namely Markov and multifractal wavelet models. Our trace-driven video simulations at 2, 5.5 and 11 Mbps data rates of an 802.11b LAN demonstrate that significant improvements over normal UDP and UDP Lite can be achieved by employing header estimation with UDP.

In a multi-hop 802.15.4 Wireless Sensor Network the volume of traffic that is processed by intermediate nodes increases considerably as data approaches the PAN coordinator. This problem is referred to as the reachback problem. In this work we focus on a more fundamental implication of reachback on operational network lifetime of 802.15.4-based wireless sensor networks. This paper proposes the joint minimization of two network lifetime metrics, previously proposed for independent use, to more accurately characterize the degree of longevity of wireless sensor networks. The method uses the k-shortest simple path algorithm and a dynamic programming method rooted in operational rate-distortion (RD) theory to increase the operational lifetime of wireless sensor networks.

Recent wireless multimedia studies have revealed that forward error correction (FEC) on corrupted packets yields better bandwidth utilization and lesser delay than retransmissions. To facilitate FEC decoding at a wireless receiver, it is desirable to relay maximum number of (error-free and corrupted) packets to the receiver's application layer. To that end, most cross-layer multimedia schemes perform partial checksum only on packet headers. However, even with a partial checksum, bursty wireless errors introduce frequent header corruptions, thereby causing considerable packet drops. In this paper, we extend our work in [1], which proposed receiver-based schemes to correct a packet's critical (and corrupted) header fields. This paper: (a) poses header detection as the well-known decision theoretic problem of detecting known parameters in noise, (b) evaluates two detectors using bit-error traces collected over an 802.11b network, (c) provides throughput results for the detectors, and (d) uses video in conjunction with FEC as an example to highlight the efficacy of the proposed schemes. We show that header detection provides significant improvements in throughput and video quality over the conventional UDP/IP protocol stack.

In a previous work, we proposed a new family of nonredundant geometrical image transforms using hybrid wavelets and directional filter banks (HWD). In this paper we further develop and examine the proposed family and provide an efficient realization utilizing regular filters. Furthermore, we extend and employ the new proposed HWD transforms in two key image processing applications, coding and denoising, and demonstrate their promising results. For coding, an SPIHT-like algorithm is developed for HWD; and for denoising, a translation-invariant HWD (TIHWD) transform is constructed. Our simulations illustrate significant improvements under the proposed transforms when compared with other transform variations.

Network coding (NC) can be efficiently combined with the "physical layer broadcast" property of wireless mediums to facilitate mutual exchange of independent information. At the same time, experimental/theoretical analysis of wireless networks has shown the efficacy of cross-layer protocols that relay corrupted packets in bandwidth hungry video applications. The integration of NC-based information exchange and cross-layer (CL) protocols for wireless video is the primary theme of this work. A particular issue addressed in this paper is the impact of errors in a packet, on the performance of a network code. Thus, we identify the operating conditions under which NC, despite the presence of residue errors, is beneficial. Based on theoretical analysis and experiments using 802.11b wireless traces it is established that the combination of NC with relay of corrupted packets can perform better than (i) conventional schemes that drop all corrupted packets (ii) a scheme that deploys network coding only (iii) a scheme that only deploys a CL strategy that recovers information from corrupted packets. The proposed cross-layer exchange (CLIX) scheme significantly improves the performance of an H.264 based video codec for wireless networks.

Blind multiplicative watermarking schemes for speech signals using wavelets and discrete cosine transform are presented. Watermarked signals are modeled using a generalized Gaussian distribution (GGD) and Cauchy probability model. Detectors are developed employing generalized likelihood ratio test (GLRT) and locally most powerful (LMP) approach. The LMP scheme is used for the Cauchy distribution, while the GLRT estimates the gain factor as an unknown parameter in the GGD model. The detectors are tested using Monte Carlo simulation and results show the superiority of the proposed LMP/Cauchy detector in some experiments

Different denoising schemes show dissimilar types of artifacts. For example, certain transform-based denoising schemes could introduce artifacts in smooth regions while others eliminate texture regions. Using different schemes for denoising a noisy image, we can consider the denoising results as different estimates of the image. Through linear combination of the results, we minimize the l2norm of the error to find the optimum coefficients in a least-square-error sense. We employ the wavelet transform, contourlet transform, and adaptive 2-D Wiener filtering as our denoising schemes. Then we apply the proposed method to the denoising results of the individual schemes. This approach eliminates most of the artifacts and achieves significant improvement in the PSNR values. We also propose averaging of the denoising results as a special case of linear combination and show that it yields near-optimal performance

Radio hardware used for the reception of 802.11b frames is capable of associating a signal to silence ratio (SSR) with each received frame. If a received frame is corrupted, then these SSR indications can be used to provide robust apriori estimate of the bit error rate in the packet. In many recently proposed cross-layer protocols, for transmission of video over wireless networks, recovery of information from partially corrupted packets has shown significant utility. In this paper, based on experiments with actual 802.11b error traces, we show that the channel state information (CSI) provided by the SSR indications can be used to improve the error recovery performance of an FEC scheme employed in conjunction with a cross-layer protocol. H.264 based simulation are used to establish the efficacy of the proposed work for video applications; specifically for video over 802.11b WLAN

Network channel coding (NCC) distributes channel coding functions over network nodes participating in common or diverse communication sessions. A particular case of NCC is network embedded FEC (NEF), which has been shown to exhibit significant improvements in the performance of video streaming applications over multicast peer-to-peer (p2p) trees. The placement of NCC/NEF codecs and its utility in improving the throughput performance is in general a function of the underlying p2p graph topology. In this paper we consider two major forms of p2p topologies: (1) perfectly structured k-ary tree topologies that can be built from (virtually) ideal p2p graphs and (2) unstructured random tree topologies where new nodes randomly join as children to any of the existing peers. The two topologies represent an optimal low diameter structured p2p topology and a trivial randomly evolving sub-optimal topology, respectively. In this paper, we show the impact of key graph parameters, such as the maximum node-degree k and minimum tree-height D, on the performance of NCC in terms of NEF throughput as well as video quality for both structured and unstructured topologies. The utility of NCC/NEF for low-degree and/or less structured p2p topologies is especially highlighted by demonstrating that, embedding of additional codecs can render the performance of less structured topologies or higher diameter topologies to be almost as good as that of the very well structured low diameter topologies. We also investigate the impact of the graph properties on the placement of the NEF codecs

Abstract — High-order full-state Markov (FSM) chains have been employed to model errors and losses in many wireless studies. The complexity of this modeling paradigm is an exponential function of a random process ’ memory-length and hence the viability of FSM chains in resource-constrained wireless environments is severely limited. In this paper, we address an important yet unsettled question: What characteristics of high-order FSM chains should be captured by a low-complexity approximate model? We analytically derive vital guidelines for accurate approximation of an FSM chain of arbitrary memory-length. These guidelines lead to a novel constant-complexity model (CCM), which always comprises of five states irrespective of a process ’ memory-length. Our trace-driven evaluations for 802.11b and GSM channels demonstrate that the 5 state CCM, while providing orders of magnitude reduction in complexity, is comparable to FSM chains and better than linear-complexity models. I.

Most subsampled filter banks lack the feature of translation invariance, which is an important characteristic in denoising applications. In this paper, we study and develop new methods to convert a general multichannel, multidimensional filter bank to a corresponding translation-invariant (TI) framework. In particular, we propose a generalized algorithme à trous, which is an extension of the algorithme à trous introduced for 1-D wavelet transforms. Using the proposed algorithm, as well as incorporating modified versions of directional filter banks, we construct the TI contourlet transform (TICT). To reduce the high redundancy and complexity of the TICT, we also introduce semi-translation-invariant contourlet transform (STICT). Then, we employ an adapted bivariate shrinkage scheme to the STICT to achieve an efficient image denoising approach. Our experimental results demonstrate the benefits and potential of the proposed denoising approach. Complexity analysis and efficient realization of the proposed TI schemes are also presented.

Rate-distortion (R-D) modeling of video coders has always been an important issue in video streaming; however, few of the traditional R-D models and their performance have been closely examined in the context of scalable (FGS-like) video. To overcome this shortcoming, the first half of the paper models rate-distortion of DCT-based fine-granular scalable coders and derives a simple operational R-D model for Internet streaming applications. Experimental results demonstrate that this R-D result, an extension of the classical R-D formula, is very accurate within the domain of scalable coding methods exemplified by MPEG-4 FGS and H.264 progressive FGS. In the second half of the paper, we examine congestion control and dynamic rate-scaling algorithms that achieve smooth visual quality during streaming using the proposed R-D model. In constant bitrate (CBR) channels, our R-D based quality-control algorithm dramatically reduces PSNR variation between adjacent frames (to less than 0.1 dB in sample sequences). Since the Internet is a changing environment shared by many sources, even R-D based quality control often cannot guarantee nonfluctuating PSNR in variable-bitrate (VBR) channels without the help from an appropriate congestion controller. Thus, we apply recent utility-based congestion control methods to our problem and show how a combination of this approach and our R-D model can benefit future streaming applications

The contourlet transform, one of the recent geometrical image transforms, lacks the feature of translation invariance due to subsampling in its filter bank (FB) structure. In this paper, we develop a translation-invariant (TI) scheme of a general multi-channel multidimensional FB and apply our findings to the contourlet transform to obtain a TI contourlet transform (TICT). Further, we employ the proposed TICT for image denoising, where we show that a significant improvement in the PSNR values as well as visual results is gained. Moreover, we demonstrate that this proposed denoising scheme outperforms the TI wavelet denoising approach for most experiments. We also introduce a less-redundant variety of the TICT, where we merely make the first stage of contourlets, translation invariant. We show that this transform, which we call semi-TICT (STICT), achieves a performance near that of the TICT in image denoising.

Traditional wireless communication protocols do not relay corrupted packets towards the application layer and neither do they forward such packets over multiple hops. Such an approach can lead to a significant number of packet drops and thus a severe deterioration in performance of high bandwidth applications. Cross-layer protocols which do relay and forward corrupted packets have exhibited substantial promise to mitigate the above problem and thus their utility for wireless multimedia needs to be explored further. Moreover, there is a need to identify efficient channel coding methods for the cross-layer channel. Unlike the traditional schemes, where the channel observed at the application layer is a pure erasure channel, in the cross-layer schemes the application layer channel exhibits hybrid erasure-error impairments. Thus in this paper, we use a rather abstract link-layer model on the basis of which we compare the performance of cross-layer and conventional schemes. We identify the modifications required to be made to RS and LDPC based FEC schemes in order to use them over hybrid erasure-error channels. Finally we compare the considered schemes in terms of video quality using the emerging H.264 video standard. Our video analysis is based on employing a hybrid error-erasure channel coding FEC for the cross-layer schemes versus employing erasure recovery FEC for the traditional protocols. We show that cross-layer schemes can lead to a significant improvement in video quality.

Delay-sensitivity of real-time applications stipulates resilience against errors and losses in multimedia content. Such resilience is particularly important for bandwidth-constrained and error-prone wireless networks. Recent wireless studies have highlighted that significant improvements in bandwidth utilization and multimedia quality can be achieved if the decision to retain or drop "corrupted" packets is made at the application layer instead of medium access, network or transport layers. Such a strategy, however, necessitates that the maximum number of (good and bad) packets are relayed to the application layer while minimizing any modifications (if any) to the widely-deployed UDP/IP protocol stack. Previous studies have proposed that, while ignoring corruptions in packet payload, only packets with errors in the headers should be dropped. We introduce a receiver-based scheme which uses the history of a multimedia session to correct packet header errors, thereby improving the throughput of real-time applications over wireless local area networks (LANs). The proposed scheme is truly receiver-based and, therefore, does not require any modifications to the source or any intermediate network node. Only minor modifications are required to the protocol stack of the multimedia receiver. Specifically, the proposed scheme generates statistics based on the history of "critical" protocol header fields for active multimedia sessions. These statistics are in turn employed to (a) correct errors in the critical fields and (b) determine if the relevant packets belong to the receiver's session of interest.

Unreliable network's packet losses severely impact the playback quality of many predictive coded sources such as compressed video. Prior researches have shown various packet loss resilient coding methods to overcome such deficiency. In this paper, we propose a new packet-loss resilient coding method, interleaved source coding (ISC), based on an interleaving of predictive video coded frames transmitted over the Internet via a single erasure channel that is resilient to packet-losses. To select an interleaving pattern for a given erasure channel model, we employ a Markov decision process (MDP) and a corresponding dynamic programming algorithm. ISC improves the overall playback quality of predictive coded video frames over a lossy channel without complex modifications to standard predictive video coders. ISC eliminates the need for content distribution, path diversity routing, and related synchronization issues and hence, it provides a variable alternative to the path-diversity approaches. Simulations of a various video sequences showed that ISC significantly improves the playback quality of the predictive video over practical traces of Markov erasure channel model when compared with traditional non-interleaving predictive coding method.

Network embedded FEC (NEF) can greatly improve the quality of service for real-time applications in p2p and proxy-based overlay networks [H. Radha and Mingquan Wu, September 2004]. Under NEF, FEC encoders and decoders (codecs) are placed in selected intermediate nodes of an overlay network The NEF codecs detect and recover lost packets, within FEC blocks at earlier stages before these blocks arrive at deeper intermediate nodes or at the final leaf nodes. This approach significantly reduces the probability of receiving undecodable FEC blocks. In many multi-hop wireless networks (such as ad hoc networks), end systems also act as routers to forward packets; higher layer functions such as FEC can then be implemented within the intermediate nodes of a multi-hop wireless network. In this paper, we analysis and compare the performance of NEF and end-to-end FEC in a multi-hop wireless channel for packet-loss recovery. We consider Markov erasure channels with memory, and hence, we present a new (and a rather simple and elegant) method for evaluating any desired loss/recovery probability measure for the Gilbert channel model. Consequently, the impact of the loss burst length on the performance of NEF and end-to-end FEC over multi-hop wireless channels are thoroughly analyzed and presented.

We propose a new family of perfect reconstruction, non-redundant, and multiresolution geometrical image transforms using the wavelet transform in conjunction with modified versions of directional filter banks (DFB). In the proposed versions of DFB, we use either horizontal or vertical directional decomposition. Taking advantage of the wavelet transform that has efficient nonlinear approximation property, we add the important feature of directionality by applying the modified and regular DFB to the subbands of a few finest wavelet levels. This way we can eliminate a major portion of the artifacts usually introduced when DFB are used. The proposed hybrid wavelets and DFB (HWD) transform family provides visual and PSNR improvements over the wavelet and contourlet transforms.

We propose a multi-view image coding system in 3-D space based on an improved volumetric 3-D reconstruction. Unlike existing multi-view image coding schemes, in which the 3-D scene information is represented by a mesh model as well as the texture data, we use a 3-D voxel model to represent the 3-D scene information of the images to be encoded. Furthermore, we propose important, yet simple, improvements to current 3-D voxel models; these improvements lead to significant coding gain within our 3-D voxel model based compression system. This system provides an elegant framework for employing powerful and mature coding techniques. In particular, we employ the H.264 and 3-D SPIHT coding methods for compressing the proposed 3-D voxel models. Our simulation results clearly illustrate the efficiency and potential of the proposed 3-D voxel model based system.

Network embedded forward error correction (NEF) framework is a paradigm shift from a conventional approach of providing forward error correction (FEC) only on an end-to-end basis. Previous work on NEF has shown promise in greatly improving the decodable probability, message throughput and video quality available to the end receiver. However the utility of NEF for video applications in presence of packet loss correlation has not been evaluated as yet. In practice, packet losses are often correlated and occur in bursts. The distortion in video can be sensitive to the bursty nature of the losses. Thus in this paper we analyze the performance of NEF within random multicast distribution trees that exhibit losses-with-memory over their branches. We utilized the Gilbert model for packet losses over each link. We quantify the performance improvement of NEF over conventional end-to-end FEC in terms of improvement in video quality in presence of packet loss correlation. Embedding NEF codecs can impact the sensitivity of video quality to packet loss correlation. We explicitly evaluate as to how NEF can alter the dependence of video quality on packet loss burstiness. Finally we show that in addition to an average improvement in video quality, NEF can improve the performance in terms of quality guarantees also.

Network impairments such as delay and packet losses have severe impact on the presentation quality of many predictive video sources. Prior researches (e.g., [1]-[3][5]-[9][11]-[13]) have shown efforts to develop packet loss resilient coding methods to overcome such impairments for realtime streaming applications. Interleaved Source Coding (ISC) is one of the error resilient coding methods, which is based on an optimum interleaving of predictive video coded frames transmitted over a single erasure channel. ISC employs a Markov Decision Process (MDP) and a corresponding dynamic programming algorithm to identify the optimal interleaving pattern for a given channel model and a transmitting sequence. ISC has shown to significantly improve the overall quality of predictive video coded stream over a lossy channel without complex modifications to standard video coders [7][8]. In this paper, ISC is evaluated over channels with memory. In particular, we analyze the impact of packet correlation [15][16] of the popular Gilbert model on ISC-based packet video over a wide range of packet loss probabilities. Simulations have shown that ISC advances the traditional method as either the loss rate increases or the packet correlation decreases.

Practical mobile ad hoc systems include heterogeneous classes of mobile nodes, such as people and vehicles. The simultaneous presence and movement of these classes influence the mobility pattern of their members in a variety of random and deterministic ways. In addition, there is an inherent hierarchical and multi-resolution structure to the mobility patterns. We present a novel hierarchical mobility framework which incorporates the fact that the movement of certain classes of mobile nodes (e.g., people) is affected by their surroundings and the movement of other forms of mobile nodes. The proposed model takes into account that a mobile node's movement is neither completely random, nor a mere function of its routing decision and/or the trip's source and destination points. The proposed model is neither classified as microscopic nor macroscopic. Instead, it is categorized as a multi-scale mobility framework capable of modeling mobility scenarios of different scales, i.e. it is equally capable of modeling the movement of mobile nodes in a street intersection as it is to model the movement of nodes between a number of population centers. The proposed mobility framework integrates and extends an influence model and hierarchical graph-based representations of the mobility area to achieve this goal. Our simulation results show that this framework accurately captures the influences among different groups of mobile nodes and the constraints imposed by their surroundings.

Cross-layer protocols that are designed for wireless exhibit both errors and erasures. The nature of these error/erasure impairments and their impact on capacity is a function of the particular cross-layer scheme used. In this paper, we consider two (rather abstract) communication schemes, cross layer design (CLD) and cross layer design with side-information (CLDS). We make a comparative analysis of the channel capacities of these schemes over single and multi-hop wireless channels to identify the conditions under which the cross-layer protocols/channels can provide improved performance over traditional (pure) erasure channels. We show that cross-layer schemes like CLD lead to a capacity increase in most realistic channel scenarios (especially over a single hop) and schemes such as CLDS always lead to a capacity increase.

We present a new camera self-calibration algorithm that uses a low-complexity multistage approach. We derive a polynomial optimization function with respect to the camera intrinsic parameters, based on the equal singular value property of the essential matrix. In terms of the stability analysis of the intrinsic parameters, we propose a multistage procedure to refine the estimation. Experimental results with both synthetic and real images show the accuracy and robustness of our method.

Classical linear block codes, such as Reed-Solomon (RS)-based codes, fail to recover any lost message symbols when the total losses exceed the redundant symbols. Under such adverse channel conditions, source and/or channel rate adaptation to incorporate additional redundancy might not be a viable option. In this paper, we explore a novel method of code adaptation which alters the degree distribution in order to achieve partial recovery of information when complete recovery is not possible. In particular, we change the degree distribution by adjusting the density and irregularity of the code. First, we illustrate that, while maintaining a constant rate, a partial recovery code can be optimized by density modification. Then, we focus on the Partial Reed-Solomon (PRS) codes, which are a family of RS-based codes that are capable of achieving different levels of partial recovery by adjustment to their order. We analyze the dependence of erasure recovery of these codes on density and regularity for a given number of losses. Finally, we present results and analysis which demonstrate that, for a given number of erasures, the PRS codes of order-1 render optimal (erasure-recovery) performance.

After the emergence of numerous Internet streaming applications, rate-distortion (R-D) modeling of scalable video encoders has become an important issue. In this paper, we examine the performance of existing R-D models in scalable coders by using the example of MPEG-4 FGS and PFGS and propose a novel R-D model based on approximation theory. Experimental results demonstrate that the proposed model is very accurate and significantly outperforms those in previous work.

Traffic models play an important role in network simulation and performance analysis. This paper presents a frame-level hybrid framework for modeling variable bitrate (VBR) video traffic. To accurately capture long-range dependent (LRD) and short-range dependent (SRD) properties of video traffic, we incorporate elements of wavelet-domain analysis into classical time-domain modeling found in prior work. However, unlike previous studies, we analyze and successfully model both inter-GOP and intra-GOP correlation. Through the use of QQ plots and leaky-bucket simulations, we evaluate the accuracy of our approach and demonstrate that the autocorrelation function and the frame-size distribution of synthetic traffic match those of the original traffic very well. The leaky-bucket simulation also demonstrates that our model effectively preserves the temporal burstiness of the original video and can be used to predict buffer overflow probabilities and network packet loss.

In this paper, we first propose a new family of geometrical image transforms that decompose images both radially and angularly. Our construction comprises two stages of filter banks that are non-redundant and perfect reconstruction and therefore lead to an overall non-redundant and perfect reconstruction transform. Using the wavelet transform as the first stage, we apply directional filter banks to the wavelet coefficients in such a way to maintain the anisotropy scaling law. Furthermore, we propose a new image coding scheme based on the proposed transform, the wavelet-based contourlet transform (WBCT), using a new contourlet-based set partitioning in hierarchical trees (CSPIHT) algorithm that provides an embedded code. Due to differences in parent-child relationships between the WBCT coefficients and wavelet coefficients, under CSPIHT, we developed an elaborated repositioning algorithm for the WBCT coefficients in such a way that we could scan spatial orientation trees that are similar to the original SPIHT algorithm. Our experiments demonstrate that the proposed approach is efficient in coding images that possess mostly textures and contours. Our simulation results also show that this new coding approach is competitive to the wavelet coder in terms of the PSNR-rate curves, and is visually superior to the wavelet coder for the mentioned images.

Current adaptive FEC schemes used for video streaming applications alter the redundancy in a block of message packets to adapt to varying channel conditions. However, for many popular streaming applications, both the source-rate and the available bandwidth are constrained. In this paper, we present FEC codes that can adapt in real-time to provide higher source-packets recovery without changing the FEC block (N, K) pair constraint. The FEC code profile is changed as function of the number of losses to facilitate an improved data recovery even under severe channel conditions (e.g., number of losses within an N-packet FEC block is larger than N-K). We present a feedback based adaptive FEC scheme, which can adapt in a rate-constrained manner. We also illustrate the utility of this scheme for video streaming applications by analyzing the results of extensive video simulations and comparing our performance to adaptive Reed Solomon FEC schemes. We consider a variety of video sequences and use actual packet traces from WLAN (802.11b) and wired Internet environments. Comparison between the two schemes is conducted on the basis of message packet recovery, PSNR, model based perceptual evaluation and visual subjective evaluation. It is shown that the proposed scheme can significantly improve the video quality and in particular reduce the jerkiness in the received video.

In this paper, we analyze statistical and rate-distortion (R-D) properties of MPEG-4 fine-granular scalability (FGS), which has recently become an important scalable compression framework and a de-facto standard for Internet video streaming. We first propose a novel statistical model of DCT residue that accurately captures the properties of the input to the MPEG-4 FGS enhancement layer. Our results show that FGS residue concentrates a lot of probability mass near zero and cannot be accurately modeled by Gaussian or Laplacian distributions. We then model the distortion of each bitplane based on the proposed statistical framework and further demonstrate that our R-D model significantly outperforms current distortion models.

In this paper we present two optimum (in a rate-distortion -RD- sense) rate-control algorithms for FGS scalable video. The proposed methods, which could compliment the proposed RD FGS algorithms for bit allocation among different FGS frames, are targeted for optimum rate allocation among the different macroblock bitplanes within each FGS frame. The proposed methods are also designed to be very low in complexity to enable real-time streaming and fast off-line processing. Simulations with a variety of video sequences have been conducted and some of these simulations are presented in this paper. Based on these results, the proposed RD based rate-control algorithms provide a gain of 0.25 - 0.6 dB gain when compared with the traditional raster-scan bitplane FGS rate-control algorithm.

In this paper, we propose a novel rate-distortion (R-D) optimized disparity based coding scheme for stereo images. This new scheme efficiently integrates the coding of the disparity field with the residual image obtained via disparity estimation/compensation (DE/DC) in an R-D framework. The scheme first performs a quadtree decomposition of the target image and computes the disparity information along with the residual image for each node in the tree. An R-D based algorithm is then used for optimum bit allocation among the different quadtree nodes. The proposed scheme further jointly encodes the neighboring nodes with similar disparity information to attain higher coding gains. We present simulation results for the proposed scheme and compare these results with the performance of a fixed block size DE/DC based JPEG2000 stereo image coder. Our simulations show that the proposed scheme outperforms the fixed block size based disparity compensated JPEG2000 coder by more than 0.5 dB.

The objective of this paper is to study the performance of handoff procedures for an integrated wireless LAN/WAN infrastructure, both qualitatively and quantitatively. We make use of a proposed hierarchical mobility management scheme to evaluate the performance of a handoff procedure with a hierarchy level of one. Also, we propose a design and perform the analysis for a two-level hierarchy architecture that is applicable for integrated 802.11-based WLAN/WAN scenarios. The focus of our analysis is on performance measures that are suitable for real-time multimedia applications. In particular, we observe and analyze the improvement in handoff latencies and packet loss ratios that are obtained by employing hierarchical mobility schemes for UDP based traffic sessions under different network scenarios. We also study the scalability of the proposed schemes by evaluating the impact of the number of mobile nodes on the above performance measures.

In this paper, we analyze the errors observed at the link layer of an 802.11b network. Our analysis at all supported bitrates (i.e., 2, 5.5. and 11 Mbps) establishes that the error patterns are not memoryless, and therefore, they exhibit a certain level of temporal dependencies. Thus, we evaluate the suitability of a two-state Markov model to capture the channel behavior. Non-stationarity of the error patterns renders such a simplistic model inadequate, and hence, we consider higher order models. This formulates a key contribution of this paper, and that is, a hierarchical Markov model, which captures the non-stationarity of the channel while employing real-time application-specific considerations to determine state-transition probabilities.

In this paper, we extend our work in [Karande, S. S. and Radha, H., 2003] which employed partial Reed-Solomon (PRS) codes at coding rates near channel capacity on a binary erasure channel (BEC). We demonstrated that an appropriately designed PRS code outperforms the classical Reed-Solomon (RS) code for a performance criterion tailored for realtime applications. This paper extends this analysis for a general BEC with varying channel conditions (under/above channel capacity). We illustrate that PRS codes exhibit a graceful degradation in erasure recovery performance and, hence, are suitable for multimedia communication. Our video simulation results outline that the enhanced erasure recovery yields a profound improvement in the perceived media quality. Finally we investigate the performance of the dividend rendered by PRS codes operating above channel capacity. In particular we define a paradigm for a unique "fixed rate" adaptive FEC scheme based on PRS codes.

Inherent vulnerability of the wireless medium renders it more susceptible to errors and losses than classical wired media. In this paper, we evaluate the suitability of protocols and strategies across different layers of the stack to provide real-time services over 802.11b wireless LANs. More specifically, within the context of cross-layer design, we compare the performance of UDP with UDP lite - a proposed framework, which improves bandwidth utilization by delivering partially damaged packets to the realtime application. First, we study the high-level end-to-end throughput improvement achieved by making cross-layer modifications to support a UDP lite framework. We compare the quality of perceived media rendered by UDP (dropped packets only) and UDP lite (dropped and corrupted packets). This formulates one of the key findings of this study, that is, although UDP lite improves the overall high-level throughput by relaying corrupted packets to the real-time application, it fails to provide significant enhancement in perceived media quality. This can, in part, be attributed to the bursty nature of errors and losses that we observed at the application layer regardless of the selected transport protocol. Finally, we compare the error-recovery/concealment overhead required by UDP and UDP lite in order to deliver lossless multimedia. We conclude that the overhead required by UDP lite is considerably lower than UDP, since the received corrupted packets that are delivered by UDP lite (but not by UDP) facilitate error-recovery.

We introduce a new family of linear block codes, which we refer to as partial Reed Solomon (PRS) codes. These codes are specifically designed and optimized for real-time multimedia communication over packet-based erasure channels. Based on the constraints and flexibilities of real-time applications, we define a performance measure, message throughput (/spl tau//sub m/), which is suitable for these applications. This measure differentiates the notion of optimum codes for the target multimedia applications as compared to performance measures that are used for nonreal-time data. Based on the proposed measure, we combine the advantages of lowering the density of a code for near capacity performance with the high decoding efficiency of Reed Solomon (RS) codes, in order to design optimum PRS codes. Then, we demonstrate, through an example of a binary erasure channel (BEC), that at near-capacity coding rates, the appropriate design of a PRS code can outperform an RS-code. We extend this analysis and optimization for a general BEC over a wide range of channel conditions. Moreover, as compared with RS codes, the proposed PRS codes provide a significantly improved graceful degradation when the number of losses exceeds the number of parity symbols within the code block. This is a highly desirable feature for real-time multimedia applications.

Errors introduced by a wireless medium are more frequent and profound than contemporary wired media. Some of these errors, which are not corrected by the physical layer, result in Medium Access Control (MAC) layer bit errors and packet losses. Design of wireless protocols and applications can benefit substantially from a thorough understanding of these MAC layer impairments. This paper evaluates and proposes Markov-based stochastic chains to model the 802.11b MAC-to-MAC channel behavior for both bit errors and packet losses. We introduce an Entropy Normalized Kullback-Leibler measure to evaluate the performance of existing and new bit error and packet loss models. Based on the proposed measure, and contrary to recent results for mobile networks, we demonstrate that the traditional two-state Markov chain provides a very suitable model for the 802.11b MAC-to-MAC packet loss process. However, this simple model is not adequate for bit errors observed at the MAC layer of wireless local area networks. Consequently, we evaluate three other Markov-based chains for modeling these errors: full-state, hidden, and hierarchical Markov chains. Among these chains, we illustrate that the full-state Markov bit error model, evaluated under a wide range of order values, renders the best performance.

We study several properties of binary-feedback congestion control in rate-based applications. We first derive necessary conditions for generic binary-feedback congestion control to converge to fairness monotonically (which guarantees asymptotic stability of the fairness point) and show that AIMD is the only TCP-friendly binomial control with monotonic convergence to fairness. We then study the steady-state behavior of binomial controls with n competing flows on a single bottleneck. Our main result here shows that combined probing for new bandwidth by all flows results in significant overshoot of the available bandwidth and rapid (often super-linear as a function of n) increase in packet loss. We also show that AIMD has the best scalability and lowest packet-loss increase among all TCP-friendly binomial schemes. We conclude the paper by deriving the conditions necessary to achieve constant packet loss regardless of the number of competing flows, n, and, in both simulation and streaming experiments, examine one new scheme, called ideally scalable congestion control, with such constant packet loss.

This paper presents an extensive statistical study and analysis of the effects of channel delays in the current (best-effort) Internet on underflow events in MPEG-4 video streaming. Two types of network delays are considered: end-to-end round-trip delays and delay jitter. Our data were collected in a seven-month real-time streaming experiment, which was conducted between a number of unicast dialup clients in more than 600 major USA cities and a backbone video server. Among other findings, our analysis shows that startup delays approximately 15-20 times the average roundtrip time (RTT) are required for the client to avoid 90% of late packets caused by delay jitter. Meanwhile, startup delays of only 3-4 times the average RTT are needed to achieve lost-packet recovery rates of 90% or more. Hence, a key finding of our study is that delay jitter represents a more challenging problem for video streaming applications than round-trip delays. We also show that the probability density function (PDF) of RTT samples, which are associated with retransmitted video packets, can be modeled using a Pareto distribution. This observation indicates that the upper tail of the RTT PDF decays slower than reported in earlier studies.

Emerging real-time streaming applications often rely on rate-based flow control. However, congestion control for rate-based applications is typically dismissed as being not viable due to the common notion that "open-loop" congestion control is simply "difficult." This paper sheds new light on the performance of binomial NACK-based (i.e., rate-based) congestion control and measures the amount of "difficulty" inherently present in such protocols. Even though previous work proposed several new congestion control methods for real-time streaming, our analysis shows that traditional additive-increase, multiplicative-decrease (AIMD) schemes possess the best packet-loss scalability among all proposed TCP-friendly schemes, especially when used in rate-based applications. We further confirmed our analytical results in a number of experiments using MPEG-4 Fine-Granular Scalable (FGS) streaming over a Cisco testbed.

This paper examines the problem of real-time estimation of the capacity, which is also known as the bottleneck bandwidth, of a network path using end-to-end measurements in a video streaming application. We utilize two basic packet-pair bandwidth-sampling methods and show how they can be applied in real-time to the video traffic. We then examine the performance of both sampling methods in a number of tests conducted using an MPEG-4 congestion-controlled streaming application over a Cisco network under a variety of conditions (including high link utilization scenarios). In addition, we study two simple estimation techniques, which can be applied in real-time to the collected samples, and show their performance in the same Cisco network with each of the sampling methods. We focus here on low-complexity and simple estimators that can be used in real-time by a wide range of receivers. We find that the sampling methods combined with the best estimator maintain very accurate estimates for the majority of the corresponding session in a variety of scenarios and could in fact be used by the application for congestion control in video over the Internet.

Typically, NACK-based congestion control is dismissed as being not viable due to the common notion that "open-loop" congestion control is simply "difficult." Emerging real-time streaming applications, however, often rely on rate-based flow control and would benefit greatly from scalable NACK-based congestion control. This paper sheds new light on the performance of NACK-based congestion control and measures the amount of "difficulty" inherently present in such protocols. We specifically focus on increase-decrease (I-D) congestion control methods for real-time, rate-based streaming. First, we introduce and study several new performance measures that can be used to analyze the class of general I-D congestion control methods. These measures include monotonicity of convergence to fairness and packet-loss scalability (explained in the paper). Second, under the assumptions that the only feedback from the network is packet loss, we show that AIMD is the only TCP-friendly method with monotonic convergence to fairness. Furthermore, we find that AIMD possesses the best packet-loss scalability among all TCP-friendly binomial schemes and show how poorly all of the existing methods scale as the number of flows is increased. Third, we show that if the flows can obtain the knowledge of an additional network parameter (i.e., the bottleneck bandwidth), the scalability of AIMD can be substantially improved. We conclude the paper by studying the performance of a new scheme, called ideally-scalable congestion control (ISCC), both in simulation and a NACK-based MPEG-4 streaming application over a Cisco testbed.

Abstract – In this paper, we analyze the dynamics of a sevenmonth real-time streaming experiment, which was conducted between a number of unicast dialup clients, connecting to the Internet through access points in more than 600 major U.S. cities, and a backbone video server. During the experiment, the clients streamed low-bitrate MPEG-4 video sequences from the server over paths with more than 5,000 distinct Internet routers. We describe the methodology of the experiment and the architecture of our NACK-based streaming application, study end-to-end dynamics of 16 thousand ten-minute sessions (85 million packets), and analyze the behavior of the following network parameters: packet loss, round-trip delay, and packet reordering. We also study the impact of these parameters on the quality of real-time streaming (i.e., the number of underflow events). I.

Transmission of video over wireless and mobile networks requires a scalable solution that is capable of adapting to the varying channel conditions in real-time (bit-rate scalability). Furthermore, video content needs to be coded in a scalable fashion to match the capabilities of a variety of devices (complexity scalability). These two scalability properties provide the flexibility that is necessary to satisfy the "anywhere, anytime and anyone" network paradigm of wireless systems. MPEG-4 fine-granular-scalability (FGS) is a flexible low-complexity solution for video streaming over heterogeneous networks (e.g., the Internet and wireless networks) and is highly resilient to packet losses. However, the flexibility and packet-loss resilience come at the expense of decreased coding efficiency compared with nonscalable coding. A novel scalable video-coding framework and corresponding compression methods for wireless video streaming is introduced. Building on the FGS approach, the proposed framework, which we refer to as adaptive motion-compensation FGS (AMC-FGS), provides improved video quality of up to 2 dB. Furthermore, the new scalability structures provide the FGS framework with the flexibility to provide tradeoffs between resilience, higher coding efficiency and terminal complexity for more efficient wireless transmission.

To compensate for the unpredictability and variability in bandwidth between sender and receiver(s) over the Internet, a new scalable coding tool has recently been introduced in MPEG-4: Fine-Granular-Scalability (FGS). The FGS framework is very flexible and can adapt in real-time to the Internet bandwidth variations by supporting both SNR and temporal scalability through a single pre-encoded fine-granular stream. This paper presents a novel Internet video streaming system employing the MPEG-4 FGS temporal-SNR scalability structure. The proposed streaming system performs the trade-off between temporal-resolution and SNR improvements at transmission time, depending on the available bandwidth and possible user preferences, and not at encoding time. An important feature of the proposed system is the temporal-SNR rate-control that ensures a high visual quality for each client, depending on the available bandwidth. The proposed rate-allocation mechanism is based on available compressed-domain information (ie, motion-vectors and texture) and/or pre-stored quality (PSNR) tags determined at encoding time.

This paper presents a trace-driven simulation study of three classes of retransmission timeout (RTO) estimators in the context of low-bitrate real-time streaming over the Internet. We explore the viability of employing retransmission timeouts in NACK-based real-time streaming applications that support multiple retransmission attempts per lost packet. In such applications, real-time RTO estimation plays a major role (i.e., poor RTO estimation results in a larger number of duplicate packets and sometimes more frequent underflow events). Our study is based on trace data collected during a number of real-time streaming tests conducted between our dialup clients in all 50 states of the U.S. (including 653 major U.S. cities) and our backbone video server during a seven-month period. First, we define a generic performance measure for assessing the quality of hypothetical RTO estimators based on the samples of the round-trip delay (RTT) recorded in the trace data. Second, using this performance measure, we evaluate the class of TCP-like estimators, find the most optimal estimator given our performance measure, and establish power laws that describe the tradeoff between the optimal number of duplicate packets and the optimal timeout waiting time. Third, we introduce a new class of RTO estimators based on delay jitter and show that they perform significantly better than TCP-like estimators in NACK-based applications. Finally, we gain a major insight into the RTT process by establishing which tuning parameters of an RTO estimator make it optimal given our performance measure and our experimental data, and give our explanation of the observed phenomena.

Real-time transmission of video over networks with no QoS guarantees (e.g., the Internet) is increasingly becoming an important application area in multimedia communications. One of the key challenges in this area is maintaining continuous decoding and playback at the receiver despite severe network impairments such as high packet-loss-ratios, packet-delay-variations, and unbounded roundtrip delays. A popular solution to this problem is to introduce large buffering delays at the receiver to (partially) compensate for the network impairments. This solution, however, is not a viable option for clients with limited resources and for applications requiring minimal delays. We advocate a new approach to this challenge. We derive a new set of encoder buffer constraints taking into consideration network impairments. We show how these new constraints can significantly reduce (or possibly eliminate) underflow and overflow events at the decoder while enabling the receiver to maintain its ideal decoder buffer size and buffering-delay requirements (needed for video transmission under ideal conditions). We present the results of more than 40 hours of video transmission over the (real) Internet. These results demonstrate the advantages and limitations (asymptotic behavior) of our proposed encoder-buffer constraints.

The MPEG-4 fine-granular-scalability (FGS) has been introduced as a flexible low-complexity solution for video streaming over heterogeneous networks (eg, the Internet). A novel scalable video-coding framework and corresponding compression methods for wireless video streaming are introduced. Building upon the FGS approach, the proposed framework, which we refer to as motion-compensation FGS (MC-FGS), provides improved video quality of up to 2 dB.

Transmission of video over bandwidth varying networks like the Internet requires a highly scalable solution capable of adapting to the network condition in real-time. To address this requirement, scalable video-coding schemes with multiple enhancement layers have been proposed. However, under this multiple-layer paradigm, the transmission bit-rate of each layer has to be predetermined at encoding time. Consequently, the range of bit-rates that can be covered with these compression schemes is limited and often lower than, or different from, the desired range required at transmission time. In this paper, a novel scalable video-coding framework and a corresponding compression method for Internet video streaming are introduced. Building upon the MPEG-4 SNR fine-granular-scalability (FGS) approach, the proposed framework provides a new level of abstraction between the encoding and transmission process by supporting both SNR and temporal scalability through a single enhancement layer. Therefore, our proposed approach enables streaming systems to support full SNR, full temporal, and hybrid temporal-SNR scalability in real-time depending on the available bandwidth, packet-loss patterns, user preferences, and/or receiver complexity. Moreover, our experiments revealed that the presented FGS temporal-SNR scalability has similar or better PSNR performance than the multilayer scalability schemes. Subsequently, an Internet video streaming system employing the proposed hybrid FGS-temporal scalability structure is introduced, together with a very simple, yet effective, rate-control that performs the tradeoffs between individual image quality (SNR) and motion-smoothness in real-time. The hybrid temporal-SNR scalability presented in this paper has been adopted in the MPEG-4 standard to support video-streaming applications.

Real-time streaming of audiovisual content over the Internet is emerging as an important technology area in multimedia communications. Due to the wide variation of available bandwidth over Internet sessions, there is a need for scalable video coding methods and (corresponding) flexible streaming approaches that are capable of adapting to changing network conditions in real time. In this paper, we describe a new scalable video-coding framework that has been adopted recently by the MPEG-4 video standard. This new MPEG-4 video approach, which is known as Fine-Granular-Scalability (FGS), consists of a rich set of video coding tools that support quality (i.e., SNR), temporal, and hybrid temporal-SNR scalabilities. Moreover, one of the desired features of the MPEG-J FGS method is its simplicity and flexibility in supporting unicast and multicast streaming applications over IF.

Several embedded video coding schemes have been recently developed for multimedia streaming over IP. In particular, fine-granular-scalability (FGS) video coding has been recently adopted by the MPEG-4 standard as the core video-compression method for streaming applications. From its inception, the FGS scalability structure was designed to be packet-loss resilient especially under unequal packet-loss protection (UPP). However, since the introduction of FGS, there has not been a comprehensive study evaluating its packet-loss resilience under unrecoverable packet losses that are common in Internet streaming applications. In this paper, we evaluate two important aspects of FGS packet-loss resilience. First, we study the impact of applying UPP between the base- and enhancement-layers on FGS-based streams, and we compare equal packet-loss protection (EPP) with UPP scenarios. Second, we introduce the notion of fine-grained loss protection (FGLP), which is suitable for the FGS enhancement-layer, and we develop an analytical framework for evaluating FGLP bounds. Based on these bounds, we show the impact of applying fine-grained protection to the FGS enhancement-layer for different types of video sequences and over a wide range of bit-rates and packet-loss ratios. As illustrated by our extensive simulation results, applying 1) UPP between the base- and enhancement-layers and 2) FGLP for the FGS enhancement-layer can provide significant resilience under moderate-to-high packet-loss ratios (e.g., 5-10%). Furthermore, the merits of this new packet-loss protection technique go beyond the FGS coding scheme, because FGLP can be successfully applied to improve the resilience to packet-losses of other embedded video coding techniques.

This paper presents an efficient method for streaming video that has the attribute of fine-granular scalability (FGS), which is a video coding method currently undergoing the MPEG-4 standardization process. An overview of FGS coding is given followed by a description of some issues associated with streaming the FGS base and enhancement layers over packet-based networks such as the Internet. The overall structure of the streaming system is described, and the detailed functionality of the algorithm is presented including simulation results, showing how to regulate the transmission of FGS video over networks that have packet loss or bandwidth variations.

Fine granular scalability (FGS) has been adopted by the MPEG-4 standard as the core video-compression method for streaming applications. From its inception, the FGS scalability structure was designed to be packet-loss resilient especially under unequal packet-loss protection (UPP). However, since the introduction of FGS, there has not been a comprehensive study evaluating its packet-loss resilience under unrecoverable packet losses that are common in Internet streaming applications. We evaluate two important aspects of FGS packet-loss resilience. First, we study the impact of applying UPP between the base- and enhancement-layers of FGS-based streams, and we compare equal packet-loss protection (EPP) with UPP scenarios. Second, we introduce the notion of fine grained loss protection (FGLP), which is suitable for the FGS enhancement-layer, and we develop an analytical framework for evaluating FGLP bounds. Based on these bounds, we show the impact of applying fine-grained protection to the FGS enhancement-layer for different types of video sequences and over a wide range of bit-rates and packet-loss ratios. As illustrated by our extensive simulation results, applying (a) UPP between the base- and enhancement layers and (b) fine grained loss protection for the FGS enhancement-layer can provide significant resilience under moderate-to-high packet-loss ratios (e.g. 5-10%).

Transmission of video over bandwidth-varying networks (e.g. the Internet) requires a highly scalable solution that is capable of adapting to the network condition in real-time. To address this requirement, scalable video coding schemes with multiple enhancement-layers have been proposed. However, under this multiple-layer paradigm, the transmission bitrate of each layer has to be predetermined at encoding time. Consequently, the range of bitrates that can be covered with these compression schemes is limited and often lower-than or different from the desired range required at transmission time. In this paper, a novel scalable video-coding framework and a corresponding compression method for Internet video streaming is introduced. Building upon the MPEG-4 fine-granular-scalability (FGS) approach, the proposed framework provides a new level of abstraction between the encoding and transmission-process by supporting both SNR and temporal scalability through a single enhancement layer. Consequently, our proposed approach enables streaming systems to support full SNR, full temporal, and hybrid SNR-temporal scalability in real-time depending on available bandwidth, packet-loss patterns, user preferences, and/or receiver complexity.

We propose a new approach for creating splice points in compressed video streams. The resulting streams can be switched seamlessly, and therefore no decoder underflow or overflow events occur. The proposed variable splicing parameters (VSP) method eliminates major encoder buffer constraints and related video encoding issues associated with the standard approach for creating seamless splice points. These advantages are achieved at the expense of a minor increase in the splicing operation complexity. When using VSP splice points, the splicer has to wait a specific period of time (equal to the difference between the out point and in point decoding delays) before switching into the new stream (i.e. the stream with the in point).

This paper presents the ITU-T Study Group 15 development of H-series Recommendations that allow interworking between different audiovisual communication terminals manufactured by different equipment providers. The paper focuses on H.310 and H.321 systems for broad-band ATM environments and H.322 and H.323 systems for LAN environments where the quality of service may or may not be guaranteed. The paper first lists the Recommendations developed by the ITU-T for audiovisual communication systems and the network environments in which they may be used. It then describes the design philosophy, the network specific characteristics, and hardware trials for each system. Then it describes the communication control protocol defined in H.245 which is used commonly by different audiovisual communication systems. Finally, the paper discusses interworking scenarios for communication between the different types of terminal on different networks.

A novel approach for error concealment in block-based image and video coding systems over the ATM networks is presented. This new approach aims at reconstructing the lost pixels in the intra-coded frames by spatial interpolation of the nearest undamaged pixels with a recursive multi-directional nonlinear filtering scheme. The lost interframe information are also reconstructed by using robust nonlinear filtering of the adjacent and previous motion vector blocks. We assume that the important synchronization and header information of the compressed bit streams are packed into high priority ATM cells and an integral number of macroblocks are packed into normal cells for efficient transmission. Finally, simulation results are provided to illustrate the effectiveness of the proposed method.

For low bit-rate compression applications, segmentation-based coding methods provide, in general, high compression ratios when compared with traditional (e.g., transform and subband) coding approaches. In this paper, we present a new segmentation-based image coding method that divides the desired image using binary space partitioning (BSP). The BSP approach partitions the desired image recursively by arbitrarily oriented lines in a hierarchical manner. This recursive partitioning generates a binary tree, which is referred to as the BSP-tree representation of the desired image. The most critical aspect of the BSP-tree method is the criterion used to select the partitioning lines of the BSP tree representation, In previous works, we developed novel methods for selecting the BSP-tree lines, and showed that the BSP approach provides efficient segmentation of images. In this paper, we describe a hierarchical approach for coding the partitioning lines of the BSP-tree representation. We also show that the image signal within the different regions (resulting from the recursive partitioning) can be represented using low-order polynomials. Furthermore, we employ an optimum pruning algorithm to minimize the bit rate of the BSP tree representation (for a given budget constraint) while minimizing distortion. Simulation results and comparisons with other compression methods are also presented.

This paper presents a new multiresolution segmentation-based algorithm for image compression. High quality low bit rate image compression is achieved by recursively coding the binary space partitioning (BSP) tree representation of images with multilevel block truncation coding (BTC). Comparison with JPEG at rates below 0.25 bit/pixel shows superior performance both in terms of power signal-to-noise ratio (PSNR) and subjective image quality.

A multiresolution method for constructing a BSP (binary space partitioning) tree is introduced. This approach derives a hierarchy (pyramid) of scale-space images from the original image. In this hierarchy, a BSP tree of an image is built from other trees representing low-resolution images of the pyramid. A low-resolution image BSP tree serves as an initial guess to construct a higher-resolution image tree. Due to filtering when constructing the pyramid, details are discarded. As a result, a more robust segmentation is obtained. Moreover a significant computational advantage is achieved.>

LTS

Performance evaluation criteria are presented to provide a useful tool for comparing current digital signal processing (DSP)-based parallel machines used in image processing (IP) applications. Additionally, these criteria can be considered as a step toward a unified computation model for IP which may influence the design of future parallel architectures using distributed frame buffers. As an example, it is shown how to determine the criteria for a parallel machine with a mesh connected architecture and 2-D interleaving scheme. Benchmarks are derived as a function of key architectural parameters (e.g. the number of processors) and image parameters. The use of the criteria in determining task performance is explained.>

Representation of two and three-dimensional objects by tree structures has been used extensively in solid modeling, computer graphics, computer vision and image processing. (See for example [Mantyla] [Chen] [Hunter] [Rosenfeld] [Leonardi].) Quadtrees, which are used to represent objects in 2-D space, and octrees, which are the extension of quadtrees in 3-D space, have been studied thoroughly for applications in graphics and image processing.