We identify the classical perceptron algorithm with margin as a member of a broader family of large margin classifiers, which we collectively call the margitron. The margitron, (despite its) sharing the same update rule with the perceptron, is shown in an incremental setting to converge in a finite number of updates to solutions possessing any desirable fraction of the maximum margin. We also report on experiments comparing the margitron with decomposition support vector machines, cutting-plane algorithms, and gradient descent methods on hard margin tasks involving linear kernels which are equivalent to 2-norm soft margin. Our results suggest that the margitron is very competitive.

We present a family of incremental Perceptron-like algorithms (PLAs) with margin in which both the "effective" learning rate, defined as the ratio of the learning rate to the length of the weight vector, and the misclassification condition are entirely controlled by rules involving (powers of) the number of mistakes. We examine the convergence of such algorithms in a finite number of steps and show that under some rather mild conditions there exists a limit of the parameters involved in which convergence leads to classification with maximum margin. An experimental comparison of algorithms belonging to this family with other large margin PLAs and decomposition SVMs is also presented.