Spectral learning algorithms learn an unknown function by learning a spectral (e.g., Fourier) representation of the function. However, there are many possible spectral representations, none of which will be best in all situations. Consequently, it seems natural to consider how a spectral learner could make use of multiple representations when learning. This paper proposes and compares three approaches to learning from multiple spectral representations. Empirical results suggest that an ensemble approach to multi-spectrum learning, in which spectral models are learned independently in each of a set of candidate representations and then combined in a majority-vote ensemble, works best in practice.

A common approach to sentiment classification is to identify a set of sentiment-carrying words and then to use machine learning to build a classifier that can classify sentiment based on the presence/absence of those words. In this paper, we propose a Fourier-based extension of this approach. Specifically, we introduce a spectral learning algorithm that implicitly identifies sentiment-carrying words and higher-order functions of those words as it learns to assign real-valued sentiment scores to documents. The spectral learner extends the word presence model by applying Boolean logic operators (AND, OR, and XOR) to the word presence features to identify useful higher-order features. These spectral features can be used in other learning algorithms, and we show how the performance of other learning algorithms can be improved by these features. Finally, we consider the problem of determining which of a pair of reviews expresses more positive overall sentiment, and we show that the spectral learner can identify very small distinctions in sentiment with better-than-random accuracy, while larger distinctions can be correctly identified with high accuracy.

In this paper, we explore the problem of how to learn spectral (e.g., Fourier) models for classification problems. Specifically, we consider two sub-problems of spectral learning: (1) how to select the basis functions that will be included in the model and (2) how to assign coefficients to the selected basis functions. Interestingly, empirical results suggest that the most commonly used approach does not perform as well in practice as other approaches, while a method for assigning coefficients based on finding an optimal linear combination of low-order basis functions usually outperforms other approaches.

This paper presents a search algorithm for finding functions that are highly correlated with an arbitrary set of data. The functions found by the search can be used to approximate the unknown function that generated the data. A special case of this approach is a method for learning Fourier representations. Empirical results demonstrate that on typical real-world problems the most highly correlated functions can be found very quickly, while combinations of these functions provide good approximations of the unknown function.