We present a method for training a neural network to perform image denoising without access to clean training examples or access to paired noisy training examples. Our method requires only a single noisy realization of each training example and a statistical model of the noise distribution, and is applicable to a wide variety of noise models, including spatially structured noise. Our model produces results which are competitive with other learned methods which require richer training data, and outperforms traditional non-learned denoising methods. We present derivations of our method for arbitrary additive noise, an improvement specific to Gaussian additive noise, and an extension to multiplicative Bernoulli noise.

To study open-ended coevolution, we define a complexity metric over interacting finite state machines playing formal language prediction games, and study the dynamics of populations under competitive and cooperative interactions. In the past purely competitive and purely cooperative interactions have been studied extensively, but neither can successfully and continuously drive an arms race. We present quantitative results using this complexity metric and analyze the causes of varying rates of complexity growth across different types of interactions. We find that while both purely competitive and purely cooperative coevolution are able to drive complexity growth above the rate of genetic drift, mixed systems with both competitive and cooperative interactions achieve significantly higher evolved complexity.

We examine the effect of cooperative and competitive interactions on the evolution of complex strategies in a prediction game. We extend previous work to the domain of noisy games, defining a new organism and mutation model, and an accompanying novel complexity metric. We find that a mix of cooperation and competition is the most effective in driving complexity growth, confirming prior results. We also compare our complexity metric with simpler metrics such as raw strategy size, and demonstrate the effectiveness of our metric in distinguishing true complexity from mere genetic bloat.

CNNs are poised to become integral parts of many critical systems. Despite their robustness to natural variations, image pixel values can be manipulated, via small, carefully crafted, imperceptible perturbations, to cause a model to misclassify images. We present an algorithm to process an image so that classification accuracy is significantly preserved in the presence of such adversarial manipulations. Image classifiers tend to be robust to natural noise, and adversarial attacks tend to be agnostic to object location. These observations motivate our strategy, which leverages model robustness to defend against adversarial perturbations by forcing the image to match natural image statistics. Our algorithm locally corrupts the image by redistributing pixel values via a process we term pixel deflection. A subsequent wavelet-based denoising operation softens this corruption, as well as some of the adversarial changes. We demonstrate experimentally that the combination of these techniques enables the effective recovery of the true class, against a variety of robust attacks. Our results compare favorably with current state-of-the-art defenses, without requiring retraining or modifying the CNN.

As deep neural networks (DNNs) have been integrated into critical systems, several methods to attack these systems have been developed. These adversarial attacks make imperceptible modifications to an image that fool DNN classifiers. We present an adaptive JPEG encoder which defends against many of these attacks. Experimentally, we show that our method produces images with high visual quality while greatly reducing the potency of state-of- the-art attacks. Our algorithm requires only a modest increase in encoding time, produces a compressed image which can be decompressed by an off-the-shelf JPEG decoder, and classified by an unmodified classifier.

It has long been considered a significant problem to improve the visual quality of lossy image and video compression. Recent advances in computing power together with the availability of large training data sets has increased interest in the application of deep learning CNNs to address image recognition and image processing tasks. Here, we present a powerful CNN tailored to the specific task of semantic image understanding to achieve higher visual quality in lossy compression. A modest increase in complexity is incorporated to the encoder which allows a standard, off-the-shelf JPEG decoder to be used. While JPEG encoding may be optimized for generic images, the process is ultimately unaware of the specific content of the image to be compressed. Our technique makes JPEG content-aware by designing and training a model to identify multiple semantic regions in a given image. Unlike object detection techniques, our model does not require labeling of object positions and is able to identify objects in a single pass. We present a new CNN architecture directed specifically to image compression, which generates a map that highlights semantically-salient regions so that they can be encoded at higher quality as compared to background regions. By adding a complete set of features for every class, and then taking a threshold over the sum of all feature activations, we generate a map that highlights semantically-salient regions so that they can be encoded at a better quality compared to background regions. Experiments are presented on the Kodak PhotoCD dataset and the MIT Saliency Benchmark dataset, in which our algorithm achieves higher visual quality for the same compressed size while preserving PSNR.

We present a new tool to help students make use of video recordings of traditional chalkboard and whiteboard lectures. Our tool provides a compact summary of a chalkboard lecture in the form of a set of slides, and an interactive environment for electronic note-taking in real time while watching a lecture in class or during review. Key to the processing of a chalkboard video by our system is the removal of the lecturer from the video (now no longer present to occlude the board) and segmentation of the resulting video to produce a set of slides comprising a compact presentation of the content written on the board during the lecture. Our method enhances learning in a number of ways. Students can focus on taking notes to supplement the material, rather than rote transcription. In addition, notes are aligned temporally with the lecture video and thus provide a useful way for students to index into the important parts of the video. Our system can be easily integrated into courses with a recorded lecture video component to help students organize their notes and quickly navigate the lectures from a given course for particular topics.