In manufacturing processes, surface inspection is a key requirement for quality assessment and damage localization. Due to this, automated surface anomaly detection has become a promising area of research in various industrial inspection systems. A particular challenge in industries with large-scale components, like aircraft and heavy machinery, is inspecting large parts with very small defect dimensions. Moreover, these parts can be of curved shapes. To address this challenge, we present a 2-stage multi-modal inspection pipeline with visual and tactile sensing. Our approach combines the best of both visual and tactile sensing by identifying and localizing defects using a global view (vision) and using the localized area for tactile scanning for identifying remaining defects. To benchmark our approach, we propose a novel real-world dataset with multiple metallic defect types per image, collected in the production environments on real aerospace manufacturing parts, as well as online robot experiments in two environments. Our approach is able to identify 85% defects using Stage I and identify 100% defects after Stage II.

We consider the problem of estimating states (e.g., position and velocity) and physical parameters (e.g., friction, elasticity) from a sequence of observations when provided a dynamic equation that describes the behavior of the system. The dynamic equation can arise from first principles (e.g., Newton’s laws) and provide useful cues for learning, but its physical parameters are unknown. To address this problem, we propose a model that estimates states and physical parameters of the system using two main components. First, an autoencoder compresses a sequence of observations (e.g., sensor measurements, pixel images) into a sequence for the state representation that is consistent with physics by including a simulation of the dynamic equation. Second, an estimator is coupled with the autoencoder to predict the values of the physical parameters. We also theoretically and empirically show that using Fourier feature mappings improves generalization of the estimator in predicting physical parameters compared to raw state sequences. In our experiments on three visual and one sensor measurement tasks, our model imposes interpretability on latent states and achieves improved generalization performance for long-term prediction of system dynamics over state-of-the-art baselines.

We consider the problem of reinforcement learning when provided with (1) a baseline control policy and (2) a set of constraints that the learner must satisfy. The baseline policy can arise from demonstration data or a teacher agent and may provide useful cues for learning, but it might also be sub-optimal for the task at hand, and is not guaranteed to satisfy the specified constraints, which might encode safety, fairness or other application-specific requirements. In order to safely learn from baseline policies, we propose an iterative policy optimization algorithm that alternates between maximizing expected return on the task, minimizing distance to the baseline policy, and projecting the policy onto the constraint-satisfying set. We analyze our algorithm theoretically and provide a finite-time convergence guarantee. In our experiments on five different control tasks, our algorithm consistently outperforms several state-of-the-art baselines, achieving 10 times fewer constraint violations and 40% higher reward on average.

We address the problem of audio indexing for a class of special-case scenarios, where it is required to index an audio stream into 2 classes, namely, a target class and a background class, as arising, say in, audio surveillance and machine diagnostics. With the emphasis on dealing with limited training exemplars defining the target class in these scenarios, we propose a 2-class `audio verification' framework, where the target and background classes are modeled by GMMs and the indexing is done via a sliding window based detection. We characterize the performance of the system in terms of ROCs, EERs and visual detection plots for a set of 2 target classes and 4 background classes from a surveillance audio database and show the viability of such a system in practical applications. We highlight the robustness of the system to high levels of background-class using visual detection plots of continuous audio streams at SNRs ranging from 30 dB down to -20 dB.

Recent seismic exploration research proposes blended acquisition from simultaneous sources with temporal overlap between records to reduce survey time and improve the quality of seismic imaging. However, separating source records for traditional pre-stack processing poses significant challenges. Traditional procedures are incapable to remove interference from other sources equivalent to blended noise. We propose a new approach that enforces simultaneous source coherence across multiple domains in order to estimate the maximum likely distribution of energy amongst the sources.

Current link adaptation algorithms for IEEE 802.11 WLANs exhibit behavior which makes them unsuitable for transmission of multiple simultaneous real-time uplink video streams. First, these algorithms do not consider the properties of the video codec, and hence, are unaware of the impact of PHY rate selection on the perceptual quality of the received video. Second, they do not differentiate between channel errors and collisions, and hence severely malfunction when the collision probability is non-negligible. In this paper, we propose a link adaptation strategy that not only optimizes the perceptual quality of the received video, but also maintains network stability by preventing catastrophic failure due to collisions. We show that switching to a lower PHY rate improves the SNR/BER performance, but increases channel contention (and hence the collision probability). Then, we use this information plus knowledge of the video codec and network transport protocol to estimate the received perceptual video quality at the current and adjacent PHY rates. The PHY rate that yields the best perceptual quality is chosen for each Group of Pictures (GOP). We support the proposed algorithm through experiments with real wireless cameras on which we have implemented our algorithm.

The mean of a data set is one trivial representation of data from one class. Recently, mutual interdependence analysis (MIA) has been successfully used to extract more involved representations, or ldquomutual featuresrdquo, accounting for samples in the class. For example a mutual feature is a speaker signature under varying channel conditions or a face signature under varying illumination conditions. A mutual representation is a linear regression that is equally correlated with all samples of the input class. We present the MIA optimization criterion from the perspectives of regression, canonical correlation analysis and Bayesian estimation. This allows us to state and solve the above criterion concisely, to contrast the MIA solution to the sample mean, and to infer other properties of its closed form, unique solution under various statistical assumptions. We define a generalized MIA solution (GMIA) and apply MIA and GMIA in a text-independent speaker verification task using the NTIMIT database. Both methods show competitive performance with equal-error-rates of 7.5 % and 6.5 % respectively over 630 speakers.

We propose a distributed cross-layer congestion control algorithm that provides enhanced quality of service QoS and reliable operation for real-time uplink video over WiFi applications. Such applications are characterized by many wireless devices transmitting video at various PHY rates over a relatively congested channel. Unfortunately, today's off-the-shelf 802.11 equipment can be easily demonstrated to suffer catastrophic failure when subject to these conditions-let alone provide acceptable perceptual quality to the user. We show that in order to remedy these issues, it is preferable to apply airtime fairness with a cross-layer approach. The idea is to use a fast frame-by-frame control loop in the carrier sense multiple access/collision avoidance (CSMA/CA)-based medium access control (MAC) layer while simultaneously exploiting the powerful control loop gain attainable by performing source-rate adaptation in the application layer. We support the proposed algorithm through both simulation and experimentation with various channel and PHY rate scenarios.

Noisy or distorted video/audio training sets represent constant challenges in automated identification and verification tasks. We propose the method of Mutual Interdependence Analysis (MIA) to extract "mutual features" from a high dimensional training set. Mutual features represent a class of objects through a unique direction in the span of the inputs that minimizes the scatter of the projected samples of the class. They capture invariant properties of the object class and can therefore be used for classification. The effectiveness of our approach is tested on real data from face and speaker recognition problems. We show that "mutual faces" extracted from the Yale database are illumination invariant, and obtain identification error rates of 2.2% in leave-one-out tests for differently illuminated images. Also, "mutual speaker signatures" for text independent speaker verification achieve state-of-the- art equal error rates of 6.8% on the NTIMIT database.

We propose a distributed cross-layer congestion control algorithm that provides enhanced QoS and reliable operation for real-time uplink video over WiFi applications. Such applications are characterized by many wireless devices transmitting video at various PHY rates over a relatively congested channel. Unfortunately, today's off-the-shelf 802.11 equipment can be easily demonstrated to suffer catastrophic failure when subject to these conditions - let alone provide acceptable perceptual quality to the user. We show that in order to remedy these issues, it is preferred to use a cross-layer approach rather than a single-layer approach. The idea is to use a fast frame-by-frame control loop in the MAC layer while simultaneously exploiting the powerful control-loop gain attainable by performing source-rate adaptation in the APP layer. We support the proposed algorithm through both simulation and experimentation.

Traffic querying, road sensing and mobile content delivery are emerging application domains for vehicular networks whose performance depends on the throughput these networks can sustain. Rate adaptation is one of the key mechanisms at the link layer that determine this performance. Rate adaptation in vehicular networks faces the following key challenges: (1) due to the rapid variations of the link quality caused by fading and mobility at vehicular speeds, the transmission rate must adapt fast in order to be effective, (2) during infrequent and bursty transmission, the rate adaptation scheme must be able to estimate the link quality with few or no packets transmitted in the estimation window, (3) the rate adaptation scheme must distinguish losses due to environment from those due to hidden-station induced collision. Our extensive outdoor experiments show that the existing rate adaptation schemes for 802.11 wireless networks under utilize the link capacity in vehicular environments. In this paper, we design, implement and evaluate CARS, a novel context-aware rate selection algorithm that makes use of context information (e.g. vehicle speed and distance from neighbor) to systematically address the above challenges, while maximizing the link throughput. Our experimental evaluation in real outdoor vehicular environments with different mobility scenarios shows that CARS adapts to changing link conditions at high vehicular speeds faster than existing rate-adaptation algorithms. Our scheme achieves significantly higher throughput, up to 79%, in all the tested scenarios, and is robust to packet loss due to collisions, improving the throughput by up to 256% in the presence of hidden stations.

Current rate adaptation schemes for 802.11 exhibit sudden and severe drops in throughput in real application scenarios. Although there have been several propositions to remedy such rate adaptation failures, there has not been a thorough analysis of the causes that lead to such a "snowball effect." This paper provides an analysis of the factors that lead to the poor performance of current rate adaptation schemes in real environments. We show that current rate adaptation schemes fail because they do not differentiate between poor channel conditions and collisions as the source of transmission failures, and consequently invoke improper responses that cascade to dramatic throughput degradation. We support the analysis through experimentation with real data from a wireless video surveillance application, and provide recommendations for the next generation of WiFi rate control schemes.

Accurate noise power spectrum estimation in a noisy speech signal is a key challenge problem in speech enhancement. One state-of-the-art approach is the minima controlled recursive averaging (MCRA). This paper presents an enhanced MCRA algorithm (EMCRA), which demonstrates less speech signal leakage and faster response time to follow abrupt changes in the noise power spectrum. Experiments using real speech and noise recordings have validated the superiority of the proposed enhancements. EMCRA shows improvements both in intuitive subjective listening and objective quality measures in terms of higher output SNR and lower output distortion scores.

In this paper we present a new source separation method based on dynamic sparse source signal models. Source signals are modeled in frequency domain as a product of a Bernoulli selection variable with a deterministic but unknown spectral amplitude. The Bernoulli variables are modeled in turn by first order Markov processes with transition probabilities learned from a training database. We consider a video conferencing scenario where the mixing parameters are estimated by the video system. We obtain the MAP signal estimators and show they are implemented by a Vitterbi decoding scheme. We validate this approach by simulations using TIMIT database, and compare the separation performance of this algorithm with our previous extended DUET method

We address the problem of speech estimation as statistical estimation with "missing" data in the independent component analysis (ICA) domain. Missing components are substituted by values drawn from "similar" data in a multi-faceted ICA representation of the complete data. The paper presents the algorithm for the inference of missing data in the case of a fixed pattern of missing data. We apply our approach to the problem of bandwidth extension, or where speech is degraded by a fixed filtering process and show the capability of the algorithm to reconstruct fine missing details of the original data with little artifacts. The evaluation is done using objective distortion measures on speech samples from the NTT database

Sparse constraints on signal decompositions are justified by typical sensor data used in a variety of signal processing fields such as acoustics, medical imaging, or wireless, but moreover can lead to more effective algorithms. The specific sparseness assumption used in this work is that the maximum number of statistically independent sources active at any time and frequency point in a mixture of signals is small. This is shown to result from an assumption of sparseness of the sources themselves, and allows us to solve the maximum likelihood formulation of the noninstantaneous acoustic mixing source estimation problem. We consider an additive noise mixing model with an arbitrary number of sensors and possibly more sources than sensors, when sources satisfy the sparseness assumption above. The solution obtained is applicable to an arbitrary number of microphones and sources, but works best when the number of sources simultaneously active at any time frequency point is a small fraction of the total number of sources.

Few source separation and independent component analysis approaches attempt to deal with noisy data. We consider an additive noise mixing model with an arbitrary number of sensors and possibly more sources than sensors (the "degenerate separation problem") when sources are disjointly orthogonal. We show how disjoint orthogonality can be viewed as a limit of a stochastic voice modeling assumption. This is the basis for our approach to noisy model estimation by maximum likelihood, under the direct-path far-field assumptions. The implementation of the derived criterion involves iterating two steps: a partitioning of the time-frequency plane for separation followed by an optimization of the mixing parameter estimates. The solution is applicable to an arbitrary number of microphones and sources. Experimentally, we show the capability of the technique to separate four voices from two, four, six and eight channel recordings in the presence of strong noise.

Multi-channel techniques offer advantages in noise reduction and overall output signal quality when compared to the well studied mono approaches. We present an original multi-channel psychoacoustically motivated noise reduction algorithm that naturally extends the single channel psychoacoustic masking filter previously studied (see Gustafsson, S. et al., ICASSP, p.873-6, 1999). The optimality criterion is designed to satisfy the psychoacoustic masking principle and to minimize the signal total distortion simultaneously. In experiments on real data recorded in a noisy car environment, we show the enhanced performance of the two-channel solution in terms of artifacts and overall tradeoff between artifacts and the amount of noise removed as given by word recognition rates.

Multichannel techniques offer advantages in noise reduction and overall output signal quality when compared to the well studied mono approaches. In this paper we present an original multichannel psychoacoustically motivates noise reduction algorithm that naturally extends the single channel psychoacoustic masking filter previously studied in the literature [S. Gustafsson et al., 1999]. The optimality criterion is designed to simultaneously satisfy the psychoacoustic masking principle and minimize the signal total distortion. In experiments on real data recorded in a noisy car environment, we show the enhanced performance of the two-channel solution in terms of artifacts and overall tradeoff between artifacts and amount of noise removed as given by word recognition rates.