Rubrics are widely used throughout postsecondary education as means for aiding in instruction and evaluation. However, despite their broad global adoption, very little is known about the quality of rubrics in use. We develop two measures to assess the quality of rubrics: (1) a checklist identifying criteria of high-quality rubrics based on analytic rubric design best practices and (2) a set of LMS-derived features that are hypothesized to represent structural components that are, in general, necessary but not sufficient for high quality rubrics. The validity of using the feature-generated scores as proxies for identifying rubric quality is evaluated through several means. First, the feature-generated scores are calculated for a set of external exemplary rubrics of known high quality. Second, the feature-scores for a subset of internal rubrics are compared to average human rater scores of rubric quality based on the checklist. We discuss the results, practical applications, and a larger research program surrounding the feature-generated scores.

We are interested in the problem of tracking broad topics such as "baseball" and "fashion" in continuous streams of short texts, exemplified by tweets from the microblogging service Twitter. The task is conceived as a language modeling problem where per-topic models are trained using hashtags in the tweet stream, which serve as proxies for topic labels. Simple perplexity-based classifiers are then applied to filter the tweet stream for topics of interest. Within this framework, we evaluate, both intrinsically and extrinsically, smoothing techniques for integrating "foreground" models (to capture recency) and "background" models (to combat sparsity), as well as different techniques for retaining history. Experiments show that unigram language models smoothed using a normalized extension of stupid backoff and a simple queue for history retention performs well on the task.