Principal Component Analysis (PCA) is perhaps the most popular linear projection technique for dimensionality reduction. We consider PCA under the assumption that the high-dimensional data points are equipped with Gaussian uncertainty. Several approaches to such uncertainty-aware PCA have been developed recently in the visualization community. Since PCA is a discontinuous map, a small uncertainty in the data points can result in a huge uncertainty in the projected points. We show that the uncertainty of the data points also creates uncertainty in the eigenvectors of the covariance matrix that defines the PCA projection. We present a closed-form expression to quantify eigenvector uncertainty. Based on this, we propose a 3D glyph that supports the decision whether existing solutions for uncertainty-aware PCA are sufficient, or whether a more expensive sampling-based approach is required. We apply our approach to several test data sets.