Many machine learning applications deal with high-dimensional data. To make computations feasible and learning more efficient, it is often desirable to reduce the dimensionality of the input variables by finding linear combinations of the predictors that can retain as much original information as possible in the relationship between the response and the original predictors. We propose a neural network-based sufficient dimension reduction method that not only identifies the structural dimension effectively, but also improves the estimation accuracy on the central space. It takes advantage of approximation capabilities of neural networks for functions in some Barron classes and leads to reduced computation cost compared to other dimension reduction methods in the literature. Additionally, the framework can be extended to fit practical dimension reduction, making the methodology more applicable in practical settings.

The limitations of traditional statistical methods on high-dimensional gene expression data for inferencing have motivated and expanded research in developing deep learning methods to handle such data more effectively. In this paper, we look into the multi-task learning problem on gene expression inferencing and propose a method that incorporates deep learning techniques and statistical methods and effectively enhances the predictive accuracy in training neural network models for high-dimensional data analysis. Unlike many multi-task learning methods that handle only a relatively small number of tasks, our proposed method demonstrates effective learning capabilities for large-scale tasks up to thousands. In addition, this novel approach is computationally efficient and reproducible, and is capable of producing competitive results with a much simpler network structure in comparison to adversarial methods. The advantages are evident based on the results of the experiments, which are conducted on datasets from the Gene Expression Omnibus and Genotype-Tissue Expression project, both of which provide worldwide range of data resources for gene expression studies and various other data applications.

Current deep learning methods developed to address classification problems related to medical imaging for disease detection and diagnosis are primarily based on binary labels and also with limited focus on confidence calibration. Confidence estimates are closely related to classification accuracy. While existing neural networks have the capability of extending binary labels to multiclass labels, the confidence calibration procedure is generally overlooked. To address the issue, we propose a method called knowledge discriminator risk network (KDR) and a confidence calibration voting algorithm (KDR-CCV) that together enhance classification accuracy, with an emphasis on confidence calibration. Comparative studies on multiclass classification based on the Breast Imaging Reporting and Data Systems (BI-RADS) assessment categories with a dataset containing only binary labels of ultrasound images are conducted. Experimental results show KDR-CCV achieves the overall best classification performance in comparison to other methods that conform to the BI-RADS criterion in addition to the effective improvement on classification accuracy. The proposed method incorporates BI-RADS assessment and artificial intelligence from an application-based broad practice, and can be extended to other medical imaging problems.

High-dimensional linear regression with interaction effects is broadly applied in research fields such as bioinformatics and social science. In this paper, first, we investigate the minimax rate of convergence for regression estimation in high-dimensional sparse linear models with two-way interactions. Here, we derive matching upper and lower bounds under three types of heredity conditions: strong heredity, weak heredity, and no heredity. From the results: 1) A stronger heredity condition may or may not drastically improve the minimax rate of convergence. In fact, in some situations, the minimax rates of convergence are the same under all three heredity conditions; 2) The minimax rate of convergence is determined by the maximum of the total price of estimating the main effects and that of estimating the interaction effects, which goes beyond purely comparing the order of the number of non-zero main effects r1and non-zero interaction effects r2; and 3) Under any of the three heredity conditions, the estimation of the interaction terms may be the dominant part in determining the rate of convergence. This is due to either the dominant number of interaction effects over main effects or the higher interaction estimation price induced by a large ambient dimension. Second, we construct an adaptive estimator that achieves the minimax rate of convergence regardless of the true heredity condition and the sparsity indices r1, r2.

Multi-armed bandit problem is an important optimization game that requires an exploration-exploitation tradeoff to achieve optimal total reward. Motivated from industrial applications such as online advertising and clinical research, we consider a setting where the rewards of bandit machines are associated with covariates, and the accurate estimation of the corresponding mean reward functions plays an important role in the performance of allocation rules. Under a flexible problem setup, we establish asymptotic strong consistency and perform a finite- time regret analysis for a sequential randomized allocation strategy based on kernel estimation. In addition, since many nonparametric and parametric methods in supervised learning may be applied to estimating the mean reward functions but guidance on how to choose among them is generally unavailable, we propose a model combining allocation strategy for adaptive performance. Simulations and a real data evaluation are conducted to illustrate the performance of the proposed allocation strategy.

We study minimax-rate adaptive estimation for density classes indexed by continuous hyper-parameters. The classes are assumed to be partially ordered in terms of inclusion relationship. Under a mild condition on the minimax risks, we show that a minimax-rate adaptive estimator can be constructed for the classes.

This paper studies minimax aspects of nonparametric classification. We first study minimax estimation of the conditional probability of a class label, given the feature variable. This function, say f, is assumed to be in a general nonparametric class. We show the minimax rate of convergence under square L/sub 2/ loss is determined by the massiveness of the class as measured by metric entropy. The second part of the paper studies minimax classification. The loss of interest is the difference between the probability of misclassification of a classifier and that of the Bayes decision. As is well known, an upper bound on risk for estimating f gives an upper bound on the risk for classification, but the rate is known to be suboptimal for the class of monotone functions. This suggests that one does not have to estimate f well in order to classify well. However, we show that the two problems are in fact of the same difficulty in terms of rates of convergence under a sufficient condition, which is satisfied by many function classes including Besov (Sobolev), Lipschitz, and bounded variation. This is somewhat surprising in view of a result of Devroye, Gorfi, and Lugosi (see A Probabilistic Theory of Pattern Recognition, New York: Springer-Verlag, 1996).

For pt.I see ibid., vol.45, no.7, p.2271-84 (1999). We study nonparametric estimation of a conditional probability for classification based on a collection of finite-dimensional models. For the sake of flexibility, different types of models, linear or nonlinear, are allowed as long as each satisfies a dimensionality assumption. We show that with a suitable model selection criterion, the penalized maximum-likelihood estimator has a risk bounded by an index of resolvability expressing a good tradeoff among approximation error, estimation error, and model complexity. The bound does not require any assumption on the target conditional probability and can be used to demonstrate the adaptivity of estimators based on model selection. Examples are given with both splines and neural nets, and problems of high-dimensional estimation are considered. The resulting adaptive estimator is shown to behave optimally or near optimally over Sobolev classes (with unknown orders of interaction and smoothness) and classes of integrable Fourier transform of gradient. In terms of rates of convergence, the performance is the same as if one knew which of them contains the true conditional probability in advance. The corresponding classifier also converges optimally or nearly optimally simultaneously over these classes.

Probability models are estimated by use of penalized log-likelihood criteria related to Akaike (1973) information criterion (AIC) and minimum description length (MDL). The accuracies of the density estimators are shown to be related to the tradeoff between three terms: the accuracy of approximation, the model dimension, and the descriptive complexity of the model classes. The asymptotic risk is determined under conditions on the penalty term, and is shown to be minimax optimal for some cases. As an application, we show that the optimal rate of convergence is simultaneously achieved for log-densities in Sobolev spaces W/sub 2//sup s/(U) without knowing the smoothness parameter s and norm parameter U in advance. Applications to neural network models and sparse density function estimation are also provided.