Pruning neural network parameters is often viewed as a means to compress models, but pruning has also been motivated by the desire to prevent overfitting. This motivation is particularly relevant given the perhaps surprising observation that a wide variety of pruning approaches increase test accuracy despite sometimes massive reductions in parameter counts. To better understand this phenomenon, we analyze the behavior of pruning over the course of training, finding that pruning's benefit to generalization increases with pruning's instability (defined as the drop in test accuracy immediately following pruning). We demonstrate that this "generalization-stability tradeoff'' is present across a wide variety of pruning settings and propose a mechanism for its cause: pruning regularizes similarly to noise injection. Supporting this, we find less pruning stability leads to more model flatness and the benefits of pruning do not depend on permanent parameter removal. These results explain the compatibility of pruning-based generalization improvements and the high generalization recently observed in overparameterized networks.

In this paper, we develop an efficient scheme for the cal- culation of derivatives within the context of Radial Ba- sis Function Finite-Difference (RBF-FD). RBF methods express functions as a linear combination of spherically symmetric basis functions on an arbitrary set of nodes. The Finite-Difference component expresses this combi- nation over a local set of nodes neighboring the point where the derivative is sought. The derivative at all points takes the form of a sparse matrix/vector multiplication (SpMV). In this paper, we consider the case of local stencils with a fixed number of nodes at each point and encode the sparse matrix in ELLPACK format. We increase the number of operations relative to memory bandwidth by interleaving the calculation of four derivatives of four different functions, or 16 different derivatives. We demonstrate a novel implementation on the Intel MIC archi- tecture, taking into account its advanced swizzling and channel interchange features. We present benchmarks on a real data set that show an almost sevenfold in- crease in speed compared to efficient implementations of a single derivative, reaching a performance of almost 140 Gflop/s in single precision. We explain the results through consideration of operation count versus memory bandwidth.

We propose a new iterative reweighted algorithm with iterative support detection (referred to as RISD) to improve the decoding performance for compressed sensing (CS). The support detection from previous iterations can be interpreted as extracting “prior information” that allows for different reweighting strategies within and without the detected support. The proposed RISD method achieves better sparsity-measurement tradeoff than both classicalL1algorithms and iteratively reweighted algorithms.

In this paper, we propose a new image quality assessment method based on a hybrid of curvelet, wavelet, and cosine transforms called hybrid no-reference (HNR) model. From the properties of natural scene statistics, the peak coordinates of the transformed coefficient histogram of filtered natural images occupy well-defined clusters in peak coordinate space, which makes NR possible. Compared to other methods, HNR has three benefits: 1) It is an NR method applicable to arbitrary images without compromising the prediction accuracy of full-reference methods; 2) as far as we know, it is the only general NR method well suited for four types of filters: noise, blur, JPEG2000, and JPEG compression; and 3) it can classify the filter types of the image and predict filter levels even when the image is results from the application of two different filters. We tested HNR on very intensive video image database (our image library) and Laboratory for Image & Video Engineering (a public library). Results are compared to the state-of-the-art methods including peak SNR, structural similarity, visual information fidelity, and so on.

Abstract In a previous paper, we explained how to translate an input script into a functional web service, independent of the script language. We extend this work by considering the automatic creation of graphical user interfaces to allow interaction between a user and the web service generated by KWATT. The key aspects of this work are three‐fold. First, comment lines inserted into the script provide hints to the interface generator regarding the interface widgets. Second, the structure of the GUI is encoded into an XML file, and third, a plugin architecture permits the interface to be the output in one of several languages. We present an example interface to illustrate the concepts. Copyright © 2009 John Wiley & Sons, Ltd.

In this paper, we propose a new general Quality Assessment method based on the curvelet transform, called Curvelet No-Reference (CNR) model, which can estimate levels of noise, blur and JPEG 2000 compression of natural images. The peak coordinate of the curvelet coefficient histogram occupies distinctive regions depending on how the image was modified from the original. During training, we associate peak positions with known filter levels. In the prediction stage, the filter levels of new images are estimated from the training data, with no access to the reference images. We tested CNR both on our own image dataset and on LIVE. Results demonstrate that CNR does a better job at predicting noise and blur levels among several methods, including SSIM and PSNR. We also present an accelerated version of CNR that does not sacrifice prediction accuracy on natural images.

Abstract We present the initial architecture and implementation of VLab, a Grid and Web‐Service‐based system for enabling distributed and collaborative computational chemistry and material science applications for the study of planetary materials. The requirements of VLab include job preparation and submission, job monitoring, data storage and analysis, and distributed collaboration. These components are divided into client entry (input file creation, visualization of data, task requests) and back‐end services (storage, analysis, computation). Clients and services communicate through NaradaBrokering, a publish/subscribe Grid middleware system that identifies specific hardware information with topics rather than IP addresses. We describe three aspects of VLab in this paper: (1) managing user interfaces and input data with JavaBeans and Java Server Faces; (2) integrating Java Server Faces with the Java CoG Kit; and (3) designing a middleware framework that supports collaboration. To prototype our collaboration and visualization infrastructure, we have developed a service that transforms a scalar data set into its wavelet representation. General adaptors are placed between the endpoints and NaradaBrokering, which serve to isolate the clients/services from the middleware. This permits client and service development independently of potential changes to the middleware. Copyright © 2007 John Wiley & Sons, Ltd.

We propose a hybrid particle and texture based approach for the visualization of time-dependent vector fields. The underlying space-time framework builds a dense vector field representation in a two-step process: 1) particle-based forward integration of trajectories in spacetime for temporal coherence, and 2) texture-based convolution along another set of paths through the spacetime for spatially correlated patterns. Particle density is controlled by stochastically injecting and removing particles, taking into account the divergence of the vector field. Alternatively, a uniform density can be maintained by placing exactly one particle in each cell of a uniform grid, which leads to particle-in-cell forward advection. Moreover, we discuss strategies of previous visualization methods for unsteady flow and show how they address issues of spatiotemporal coherence and dense visual representations. We demonstrate how our framework is capable of realizing several of these strategies. Finally, we present an efficient GPU implementation that facilitates an interactive visualization of unsteady 2D flow on Shader Model 3 compliant graphics hardware.

We propose unsteady flow advection-convolution (UFAC) as a novel visualization approach for unsteady flows. It performs time evolution governed by pathlines, but builds spatial correlation according to instantaneous streamlines whose spatial extent is controlled by the flow unsteadiness. UFAC is derived from a generic framework that provides spacetime-coherent dense representations of time dependent-vector fields by a two-step process: 1) construction of continuous trajectories in spacetime for temporal coherence; and 2) convolution along another set of paths through the above spacetime for spatially correlated patterns. Within the framework, known visualization techniques-such as Lagrangian-Eulerian advection, image-based flow visualization, unsteady flow LIC, and dynamic LIC-can be reproduced, often with better image quality, higher performance, or increased flexibility of the visualization style. Finally, we present a texture-based discretization of the framework and its interactive implementation on graphics hardware, which allows the user to gradually balance visualization speed against quality.

Data sets from large-scale simulations (up to 501/sup 3/ grid points) of mantle convection are analyzed with volume rendering of the temperature field and a new critical point analysis of the velocity field. As the Rayleigh number Ra is increased the thermal field develops increasingly thin plume-like structures along which heat is convected. These eventually break down and become turbulent. Visualization methods are used to distinguish between various models of heat conductivity and to develop an intuitive understanding of the structure of the flow.

Ocean model simulations commonly assume the ocean is hydrostatic, resulting in near zero vertical motion. The vertical motion found is typically associated with the variations of the thermocline depth over time, which are mainly a result of the development and movement of ocean fronts, eddies, and internal waves. A new technique, extended from Lagrangian-Eulerian Advection, is presented to help understand the variation of vertical motion associated with the change in thermocline depth over time. A time surface is correctly deformed in a single direction according to the flow. The evolution of the time surface is computed via a mixture of Eulerian and Lagrangian techniques. The dominant horizontal motion is textured onto the surface using texture advection, while both the horizontal and vertical motions are used to displace the surface. The resulting surface is shaded for enhanced contrast. Timings indicate that the overhead over standard 2D texture advection is no more than 12%.

A new hybrid scheme, called Lagrangian-Eulerian advection (LEA), that combines the advantages of the Eulerian and Lagrangian frameworks is applied to the visualization of dense representations of time-dependent vector fields. The algorithm encodes the particles into a texture that is then advected. By treating every particle equally, we can handle texture advection and dye advection within a single framework. High temporal and spatial correlation is achieved through the blending of successive frames. A combination of particle and dye advection enables the simultaneous visualization of streamlines, particle paths and streak-lines. We demonstrate various experimental techniques on several physical flow fields. The simplicity of both the resulting data structures and the implementation suggest that LEA could become a useful component of any scientific visualization toolkit concerned with the display of unsteady flows.

In this paper, we propose a new technique to visualize dense representations of time-dependent vector fields based on a Lagrangian-Eulerian Advection (LEA) scheme. The algorithm produces animations with high spatio-temporal correlation at interactive rates. With this technique, every still frame depicts the instantaneous structure of the flow, whereas an animated sequence of frames reveals the motion a dense collection of particles would take when released into the flow. The simplicity of both the resulting data structures and the implementation suggest that LEA could become a useful component of any scientific visualization toolkit concerned with the display of unsteady flows.

We present a novel hardware-accelerated texture advection algorithm to visualize the motion of two-dimensional unsteady flows. Making use of several proposed extensions to the OpenGL-1.2 specification, we demonstrate animations of over 65,000 particles at 2 frames/sec on an SGI Octane with EMXI graphics. High image quality is achieved by careful attention to edge effects, noise frequency, and image enhancement. We provide a detailed description of the hardware implementation, including temporal and spatial coherence techniques, dye advection techniques, and feature extraction.

Abstract While parallel computers offer significant computational performance, it is generally necessary to evaluate several programming strategies. Two programming strategies for a fairly common problem—a periodic tridiagonal solver—are developed and evaluated. Simple model calculations as well as timing results are presented to evaluate these strategies. The particular tridiagonal solver evaluated is used in many computational fluid dynamic simulation codes. The feature that makes this algorithm unique is that these simulation codes usually require simultaneous solution for multiple right‐hand‐sides (RHS) of the system of equations. Each RHS solutions is independent and thus can be computed in parallel. Thus, a Gaussian‐elimination‐type algorithm can be used in a parallel computation and more complicated approaches such as cyclic reduction are not required. The two strategies are a transpose strategy and a distributed solver strategy. For the transpose strategy, the data are moved so that a subset of all the RHS problems is solved on each of the several processors. This usually requires significant data movement between processor memories across a network. The second strategy attempts to have the algorithm follow the data across processor boundaries in a chained manner. This usually requires significantly less data movement. An approach to accomplish this second strategy in a near‐perfect load‐balanced manner is developed. In addition, an algorithm will be shown to directly transform a sequential Gaussian‐elimination‐type algorithm into the parallel, chained, load‐balanced algorithm.