We present an application of kernel methods to extracting relations from unstructured natural language sources. We introduce kernels defined over shallow parse representations of text, and design efficient algorithms for computing the kernels. We use the devised kernels in conjunction with Support Vector Machine and Voted Perceptron learning algorithms for the task of extracting person-affiliation and organization-location relations from text. We experimentally evaluate the proposed methods and compare them with feature-based learning algorithms, with promising results.

Clustering is a powerful technique for large-scale topic discovery from text.It involves two phases: first, feature extraction maps each document or record to a point in high-dimensional space, then clustering algorithms automatically group the points into a hierarchy of clusters.We describe an unsupervised, near-linear time text clustering system that offers a number of algorithm choices for each phase.We introduce a methodology for measuring the quality of a cluster hierarchy in terms of F-Measure, and present the results of experiments comparing different algorithms.The evaluation considers some feature selection parameters (tfidfand feature vector length) but focuses on the clustering algorithms, namely techniques from Scatter/Gather (buckshot, fractionation, and split/join) and kmeans.Our experiments suggest that continuous center adjustment contributes more to cluster quality than seed selection does.It follows that using a simpler seed selection algorithm gives a better time/quality tradeoff.We describe a refinement to center adjustment, "vector average damping," that further improves cluster quality.We also compare the near-linear time algorithms to a group average greedy agglomerative clustering algorithm to demonstrate the time/quality tradeoff quantitatively.

We describe one approach to build an automatically trainable anaphora resolution system. In this approach, we use Japanese newspaper articles tagged with discourse information as training examples for a machine learning algorithm which employs the C4.5 decision tree algorithm by Quinlan (Quinlan, 1993). Then, we evaluate and compare the results of several variants of the machine learning-based approach with those of our existing anaphora resolution system which uses manually-designed knowledge sources. Finally, we compare our algorithms with existing theories of anaphora, in particular, Japanese zero pronouns.

This paper describes a new discourse module within our multilingual NLP system. Because of its unique data-driven architecture, the discourse module is language-independent. Moreover, the use of hierarchically organized multiple knowledge sources makes the module robust and trainable using discourse-tagged corpora. Separating discourse phenomena from knowledge sources makes the discourse module easily extensible to additional phenomena.

I present a semantic analysis of collective-distributive ambiguity, and resolution of such ambiguity by model-based reasoning. This approach goes beyond Scha and Stallard [17], whose reasoning capability was limited to checking semantic types. My semantic analysis is based on Link [14, 13] and Roberts [15], where distributivity comes uniformly from a quantificational operator, either explicit (e.g. each) or implicit (e.g. the D operator). I view the semantics module of the natural language system as a hypothesis generator and the reasoner in the pragmatics module as a hypothesis filter (cf. Simmons and Davis [18]). The reasoner utilizes a model consisting of domain-dependent constraints and domain-independent axioms for disambiguation. There are two kinds of constraints, type constraints and numerical constraints, and they are associated with predicates in the knowledge base. Whenever additional information is derived from the model, the Contradiction Checker is invoked to detect any contradiction in a hypothesis using simple mathematical knowledge. CDCL (Collective-Distributive Constraint Language) is used to represent hypotheses, constraints, and axioms in a way isomorphic to diagram representations of collective-distributive ambiguity.

In this paper, we report on our use of zero morphemes in Unification-Based Combinatory Categorial Grammar. After illustrating the benefits of this approach with several examples, we describe the algorithm for compiling zero morphemes into unary rules, which allows us to use zero morphemes more efficiently in natural language processing. Then, we discuss the question of equivalence of a grammar with these unary rules to the original grammar. Lastly, we compare our approach to zero morphemes with possible alternatives.