Distributionally robust optimization (DRO) is a powerful technique to train robust machine learning models that perform well under distribution shifts. Compared with empirical risk minimization (ERM), DRO optimizes the expected loss under the worst-case distribution in an uncertainty set of distributions. This paper revisits the important problem of DRO with non-convex smooth loss functions. For this problem, Jin et al. (2021) showed that its dual problem is generalized $(L_0, L_1)$-smooth condition and gradient noise satisfies the affine variance condition, designed an algorithm of mini-batch normalized gradient descent with momentum, and proved its convergence and complexity. In this paper, we show that the dual problem and the gradient noise satisfy simpler yet more precise partially generalized smoothness condition and partially affine variance condition by studying the optimization variable and dual variable separately, which further yields much simpler algorithm design and convergence analysis. We develop a double stochastic gradient descent with clipping (D-SGD-C) algorithm that converges to an $\epsilon$-stationary point with $\mathcal O(\epsilon^{-4})$ gradient complexity, which matches with results in Jin et al. (2021). Our algorithm does not need to use momentum, and the proof is much simpler, thanks to the more precise characterization of partially generalized smoothness and partially affine variance noise. We further design a variance-reduced method that achieves a lower gradient complexity of $\mathcal O(\epsilon^{-3})$. Our theoretical results and insights are further verified numerically on a number of tasks, and our algorithms outperform the existing DRO method (Jin et al., 2021).

Focus measures are widely used to assess image clarity in various fields, such as photography and computer vision. However, many existing focus measures face challenges in balancing noise robustness and measurement capability. In this letter, a novel focus measure called Variance of Tight Framelet Feature (VTFF) is proposed to address this challenge. VTFF leverages the advantages of tight framelet features and variance information in feature maps to provide a robust and accurate assessment of image focus. Experimental results on both synthetic and real-world data demonstrate its superior performance compared to recent focus measures in measurement capability, noise robustness, and real-time performance.

Distributionally robust optimization (DRO) is a powerful framework for training robust models against data distribution shifts. This paper focuses on constrained DRO, which has an explicit characterization of the robustness level. Existing studies on constrained DRO mostly focus on convex loss function, and exclude the practical and challenging case with non-convex loss function, e.g., neural network. This paper develops a stochastic algorithm and its performance analysis for non-convex constrained DRO. The computational complexity of our stochastic algorithm at each iteration is independent of the overall dataset size, and thus is suitable for large-scale applications. We focus on the general Cressie-Read family divergence defined uncertainty set which includes chi^2-divergences as a special case. We prove that our algorithm finds an epsilon-stationary point with an improved computational complexity than existing methods. Our method also applies to the smoothed conditional value at risk (CVaR) DRO.

Abstract. Each coil image in a parallel magnetic resonance imaging (pMRI) system is an imaging slice modulated by the corresponding coil sensitivity. These coil images, structurally similar to each other, are stacked together as 3-dimensional (3D) image data, and their sparsity property can be explored via 3D directional Haar tight framelets. The features of the 3D image data from the 3D framelet systems are utilized to regularize sensitivity encoding (SENSE) pMRI reconstruction. Accordingly, a so-called SENSE3d algorithm is proposed to reconstruct images of high quality from the sampled [Formula: see text]-space data with a high acceleration rate by decoupling effects of the desired image (slice) and sensitivity maps. Since both the imaging slice and sensitivity maps are unknown, this algorithm repeatedly performs a slice step followed by a sensitivity step by using updated estimations of the desired image and the sensitivity maps. In the slice step, for the given sensitivity maps, the estimation of the desired image is viewed as the solution to a convex optimization problem regularized by the sparsity of its 3D framelet coefficients of coil images. This optimization problem, involving data from the complex field, is solved by a primal-dual three-operator splitting (PD3O) method. In the sensitivity step, the estimation of sensitivity maps is modeled as the solution to a Tikhonov-type optimization problem that favors the smoothness of the sensitivity maps. This corresponding problem is nonconvex and could be solved by a forward-backward splitting method. Experiments on real phantoms and in vivo data show that the proposed SENSE3d algorithm can explore the sparsity property of the imaging slices and efficiently produce reconstructed images of high quality with reduced aliasing artifacts caused by high acceleration rate, additive noise, and the inaccurate estimation of each coil sensitivity. To provide a comprehensive picture of the overall performance of our SENSE3d model, we provide the quantitative index (HaarPSI) and comparisons to some deep learning methods such as VarNet and fastMRI-UNet.

The goal of this paper is to develop a novel numerical method for efficient multiplicative noise removal. The nonlocal self-similarity of natural images implies that the matrices formed by their nonlocal similar patches are low-rank. By exploiting this low-rank prior with application to multiplicative noise removal, we propose a nonlocal low-rank model for this task and develop a proximal alternating reweighted minimization (PARM) algorithm to solve the optimization problem resulting from the model. Specifically, we utilize a generalized nonconvex surrogate of the rank function to regularize the patch matrices and develop a new nonlocal low-rank model, which is a nonconvex nonsmooth optimization problem having a patchwise data fidelity and a generalized nonlocal low-rank regularization term. To solve this optimization problem, we propose the PARM algorithm, which has a proximal alternating scheme with a reweighted approximation of its subproblem. A theoretical analysis of the proposed PARM algorithm is conducted to guarantee its global convergence to a critical point. Numerical experiments demonstrate that the proposed method for multiplicative noise removal significantly outperforms existing methods, such as the benchmark SAR-BM3D method, in terms of the visual quality of the denoised images, and of the peak-signal-to-noise ratio (PSNR) and the structural similarity index measure (SSIM) values.

We study empty container repositioning between hinterland and seaports considering both free detention time and liner carrier cooperation. Aiming at minimizing the cost of empty container repositioning, we propose a nonlinear mixed integer programming model to obtain optimal free detention time of liner carriers and optimal time to dispatch empty containers from hinterland to seaports. Meanwhile, under the cooperation of liner carriers, the optimal quantities of empty containers repositioned from surplus ports to deficit ports are derived. To solve this model, we use CPLEX solver to get the optimal results. It is shown that when a port is judged to be a deficit port in a certain period, the liner carrier will set a shorter free detention time; otherwise, a longer free period will be set at a surplus port. Sensitivity analysis is also conducted in this paper. Results show that the increase of storage cost leads to the increase of total cost, while total cost decreases with the expansion of demand for empty containers. And poor hinterland transportation condition extends free detention time, which in turn reduces the liner carriers' profits. Liner carriers should integrate hinterland transportation when they are short of empty containers.

Sparse modelling with the ℓ p norm of 0 ≤ p ≤ 1 requires the availability of the proximity operator of the ℓ p norm. The proximity operators of the ℓ 0 and ℓ 1 norms are the well‐known hard‐ and soft‐thresholding estimators, respectively. In this study, the authors give a complete study on the properties of the proximity operator of the ℓ p norm. Based on these properties, explicit formulas of the proximity operators of the ℓ 1/2 norm and ℓ 2/3 norm are derived with simple proofs; for other values of p , an iterative Newton's method is developed to compute the proximity operator of the ℓ p norm by fully exploring the available proximity operators of the ℓ 0 , ℓ 1/2 , ℓ 2/3 , and ℓ 1 norms. As applications, the proximity operator of the ℓ p norm with 0 ≤ p ≤ 1 is applied to the ℓ p ‐regularisation for compressive sensing and image restoration.

Parallel magnetic resonance imaging (pMRI) is a technique to accelerate the magnetic resonance imaging process. The problem of reconstructing an image from the collected pMRI data is ill-posed. Regularization is needed to make the problem well-posed. In this paper, we first construct a two-dimensional tight framelet system whose filters have the same support as the orthogonal Haar filters and are able to detect edges of an image in the horizontal, vertical, and $\pm 45^o$ directions. This system is referred to as directional Haar framelet (DHF). We then propose a pMRI reconstruction model whose regularization term is formed by the DHF. This model is solved by a fast proximal algorithm with low computational complexity. The regularization parameters are updated adaptively and determined automatically during the iteration of the algorithm. Numerical experiments for in-silico and in-vivo data sets are provided to demonstrate the superiority of the DHF-based model and the efficiency of our proposed algorithm for pMRI reconstruction.

In many applications, one may acquire a composition of several signals that may be corrupted by noise, and it is a challenging problem to reliably separate the components from one another without sacrificing significant details. Adding to the challenge, in a compressive sensing framework, one is given only an undersampled set of linear projections of the composite signal. In this study, the authors propose using the Dantzig selector model incorporating an overcomplete dictionary to separate a noisy undersampled collection of composite signals, and present an algorithm to efficiently solve the model. The Dantzig selector is a statistical approach to finding a solution to a noisy linear regression problem by minimising the ℓ 1 norm of candidate coefficient vectors while constraining the scope of the residuals. The Dantzig selector performs well in the recovery and separation of an unknown composite signal when the underlying coefficient vector is sparse. They propose a proximity operator‐based algorithm to recover and separate unknown noisy undersampled composite signals using the Dantzig selector. They present numerical simulations comparing the proposed algorithm with the competing alternating direction method, and the proposed algorithm is found to be faster, while producing similar quality results. In addition, they demonstrate the utility of the proposed algorithm by applying it in various applications including the recovery of complex‐valued coefficient vectors, the removal of impulse noise from smooth signals, and the separation and classification of a composition of handwritten digits.

This paper considers the problem of distributed compressive detection under a perfect secrecy constraint. More specifically, we consider the problem where the distributed inference network operates in the presence of an eavesdropper who wants to discover the state of the nature being monitored by the system. It is shown that perfect secrecy can be achieved by using cooperating trustworthy nodes that assist the Fusion Center (FC) by providing falsified data to the eavesdroppers. We also consider the problem of determining optimal system parameters which maximize the detection performance at the FC, while ensuring perfect secrecy at the eavesdropper.

In this paper, we propose an image inpainting optimization model whose objective function is a smoothed l(1) norm of the weighted nondecimated discrete cosine transform (DCT) coefficients of the underlying image. By identifying the objective function of the proposed model as a sum of a differentiable term and a nondifferentiable term, we present a basic algorithm inspired by Beck and Teboulle's recent work on the model. Based on this basic algorithm, we propose an automatic way to determine the weights involved in the model and update them in each iteration. The DCT as an orthogonal transform is used in various applications. We view the rows of a DCT matrix as the filters associated with a multiresolution analysis. Nondecimated wavelet transforms with these filters are explored in order to analyze the images to be inpainted. Our numerical experiments verify that under the proposed framework, the filters from a DCT matrix demonstrate promise for the task of image inpainting.

This paper studies a problem of image restoration that observed images are contaminated by Gaussian and impulse noise. Existing methods for this problem in the literature are based on minimizing an objective functional having the l(1) fidelity term and the Mumford-Shah regularizer. We present an algorithm on this problem by minimizing a new objective functional. The proposed functional has a content-dependent fidelity term which assimilates the strength of fidelity terms measured by the l(1) and l(2) norms. The regularizer in the functional is formed by the l(1) norm of tight framelet coefficients of the underlying image. The selected tight framelet filters are able to extract geometric features of images. We then propose an iterative framelet-based approximation/sparsity deblurring algorithm (IFASDA) for the proposed functional. Parameters in IFASDA are adaptively varying at each iteration and are determined automatically. In this sense, IFASDA is a parameter-free algorithm. This advantage makes the algorithm more attractive and practical. The effectiveness of IFASDA is experimentally illustrated on problems of image deblurring with Gaussian and impulse noise. Improvements in both PSNR and visual quality of IFASDA over a typical existing method are demonstrated. In addition, Fast_IFASDA, an accelerated algorithm of IFASDA, is also developed.

We present a variational approach to obtain high-resolution images from multiframe low-resolution video stills. The objective functional for the variational approach consists of a data fidelity term and a regularizer. The fidelity term is formed by adaptively mimickingl1andl2norms. The regularization uses thel1norm of the framelet coefficients of a high-resolution image with a geometric tight framelet system constructed in this paper. The tight framelet system has abilities to detect multi-orientation and multi-order variations of an image. A two-phase iterative method for super-resolution reconstruction is proposed to construct a high-resolution image. The first phase is to get an approximation of the solution (i.e., the ideal image) using the steepest descent method. The second phase is to enhance the sparsity of the approximate solution by using the soft thresholding operator with variable thresholding parameters. Numerical results based on both synthetic data and real videos show that our algorithm is efficient in terms of removing visual artifacts and preserving edges in restored images.

The proper selection and determination of partners is the key to the establishment of virtual logistic enterprises. With the rapid development of information technology, and the globalization and digitalization of the economy, the virtual logistic enterprises partners selection model based on CLIQUE is proposed to realize fast data mining in the bulkiness and high dimension information database of member enterprises provided by VLE information platform in order to select the candidate partners in the whole globe, or knowledge discover from indexes evaluation of partners. A simulating case demonstrates the effectiveness of the selection method of the grid-based clustering algorithm

We present a general mathematical theory for lifting frames that allows us to modify existing filters to construct new ones that form Parseval frames. We apply our theory to design nonseparable Parseval frames from separable (tensor) products of a piecewise linear spline tight frame. These new frame systems incorporate the weighted average operator, the Sobel operator, and the Laplacian operator in directions that are integer multiples of 45 degrees. A new image denoising algorithm is then proposed, tailored to the specific properties of these new frame filters. We demonstrate the performance of our algorithm on a diverse set of images with very encouraging results.

We present a general mathematical theory for lifting frames that allows us to modify existing filters to construct new ones that form Parseval frames. We apply our theory to design non-separable Parseval frames from separable (tensor) products of a piecewise linear spline tight frame. These new frame systems incorporate the weighted average operator and the Sobel operator in directions that are integer multiples of 45/spl deg/. A new image denoising algorithm is then proposed tailored to the specific properties of these new frame filters. We demonstrate the performance of our algorithm on a diverse set of images with very encouraging results.

We propose a new method to denoise a surface. This method is motivated by the Laplacian flow and the theory of tight frames. Mesh denoising is achieved by building local Wiener filtering into the detail representation of the surface. We have performed a number of experiments to assess the accuracy, advantages and limitations of our approach with very encouraging results.

Since the initial introduction of open source cluster application resources (OSCAR), this software package has been a well-accepted choice for building high performance computing systems. As it continues to be applied to mission-critical environments, high availability (HA) features therefore are needed to be included in OSCAR cluster. In this paper, we provide a HA solution for OSCAR cluster. As a widely used technique in HA solutions, component redundancy is adopted to improve the system availability. Based on the proposed architecture, we develop a detailed failure-repair model for predicting the availability of HA OSCAR cluster. stochastic reward nets (SRN) are used to model the behavior of the system. We specify our SRN model to stochastic Petri net package, describe and compute several interesting output measures that characterize availability features of HA OSCAR cluster.

In this paper, we construct a new class of deformable models using new biorthogonal wavelets, named Generalized Hermite Distributed Approximating Functional (g-HDAF) Wavelets. The scaling functions of this new family are symmetric and the corresponding wavelets op-timize their smoothness for a given number of vanishing moments. In addition, we embed these multiresolution deformable models to the physics-based deformable model framework and use them for fitting 3D range data. We have performed a number of experiments with both synthetic and real data with very encouraging results. 1

A critical component of spline smoothing is the choice of knots, especially for curves with varying shapes and frequencies in its domain. We consider a two-stage knot selection scheme for adaptively fitting splines to data subject to noise. A potential set of knots is chosen based on information from certain wavelet decompositions with the intention of placing more points where the curve shows rapid changes. The final knot selection is then made based on statistical model selection ideas. We show that the proposed method is well suited for a variety of smoothing and noise filtering needs.

This paper introduces a novel class of length-4N orthonormal scalar wavelets, and presents the theory, implementational issues, and their applications to image compression. We first give the necessary and sufficient conditions for the existence of this class. The parameterized representation of filters with different lengths are then given. Next, we derive new and efficient decomposition and reconstruction algorithms specifically tailored to this class of wavelets. We show that the proposed discrete wavelet transformations are orthogonal and have lower computational complexity than conventional octave-bandwidth transforms using Daubechies' (1989) orthogonal filters of equal length. In addition, we also verify that symmetric boundary extensions can be applied. Finally, our image compression results further confirm that improved performance can be achieved with lower computational cost.

Approximation order, linear phase symmetry, time-frequency localization, regularity, and stopband attenuation are some criteria that are widely used in wavelet filter design. We propose a new criterion called good multifilter properties (GMPs) for the design and construction of multiwavelet filters targeting image compression applications. We first provide the definition of GMPs, followed by a necessary and sufficient condition for an orthonormal multiwavelet system to have a GMP order of at least 1. We then present an algorithm to construct orthogonal multiwavelets possessing GMPs, starting from any length-2N scalar CQPs. Image compression experiments are performed to evaluate the importance of GMPs for image compression, as compared to other common filter design criteria. Our results confirmed that multiwavelets that possess GMPs not only yield superior PSNR performances, but also require much lower computations in their transforms.