In this paper, we prove various results about PAC learning in the presence of malicious noise. Our main interest is the sample size behavior of learning algorithms. We prove the first nontrivial sample complexity lower bound in this model by showing that order of ε/Δ 2 + d /Δ (up to logarithmic factors) examples are necessary for PAC learning any target class of {0,1}-valued functions of VC dimension d , where ε is the desired accuracy and η = ε/(1 + ε) - Δ the malicious noise rate (it is well known that any nontrivial target class cannot be PAC learned with accuracy ε and malicious noise rate η ≥ ε/(1 + ε), this irrespective to sample complexity). We also show that this result cannot be significantly improved in general by presenting efficient learning algorithms for the class of all subsets of d elements and the class of unions of at most d intervals on the real line. This is especialy interesting as we can also show that the popular minimum disagreement strategy needs samples of size d ε/Δ 2 , hence is not optimal with respect to sample size. We then discuss the use of randomized hypotheses. For these the bound ε/(1 + ε) on the noise rate is no longer true and is replaced by 2ε/(1 + 2ε). In fact, we present a generic algorithm using randomized hypotheses that can tolerate noise rates slightly larger than ε/(1 + ε) while using samples of size d /ε as in the noise-free case. Again one observes a quadratic powerlaw (in this case d ε/Δ 2 , Δ = 2ε/(1 + 2ε) - η) as Δ goes to zero. We show upper and lower bounds of this order.

In the k-Restricted-Focus-of-Attention (k-RFA) model, only k of the n attributes of each example are revealed to the learner, although the set of visible attributes in each example is determined by the learner.While the k-RFA model is a natural extension of the PAC model, there are also significant differences.For example, it was previously known that learnability in this model is not characterized by the VC dimension and that many PAC learning algorithms are not applicable in the k-RFA setting.In this paper we further explore the relationship between the PAC and k-RFA models, with several interesting results.First, we develop a characterization of k-RFA learnability upon which we build a general tool for proving hardness results.We then apply this and other new techniques for studying RFA learning to two particularly expressive function classes, k-decision-lists (k-DL) and k-TOP, the class of thresholds of parity functions in which each parity function takes at most k inputs.Among other results, we prove a hardness result for k-RFA learnability of k-DL, k ~n -2.In sharp contrast, an (n -1)-RFA algorithm for learning (n -l)decision-lists is presented.Similarly, we prove that 1-DL is learnable if and only if at least half of the inputs are visible in each instance.In addition, we show that there is a uniform-distribution lc-RFA learning algorithm for the class of k-decision-lists

Permiaaion to make digital

Article Learning with restricted focus of attention Share on Authors: Shai Ben-David View Profile , Eli Dichterman View Profile Authors Info & Claims COLT '93: Proceedings of the sixth annual conference on Computational learning theoryAugust 1993 Pages 287–296https://doi.org/10.1145/168304.168353Online:01 August 1993Publication History 15citation245DownloadsMetricsTotal Citations15Total Downloads245Last 12 Months1Last 6 weeks0 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteGet Access