The sensitivity of various features that are characteristics of machine performance may vary significantly under different working conditions. Thus it is critical to devise a systematic feature extraction (FE) approach that provides a useful and automatic guidance on using the most effective features for machine performance prediction without human intervention. This paper proposes a locality preserving projections (LPP)-based FE approach. Different from principal component analysis (PCA) that aims to discover the global structure of the Euclidean space, LPP is capable to discover local structure of the data manifold. This may enable LPP to find more meaningful low-dimensional information hidden in the high-dimensional observations compared with PCA. The effectiveness of the proposed approach for bearing defect and severity classification is evaluated experimentally on bearing test-beds. Furthermore, a novel health assessment indication, Gaussian mixture model (GMM)-based negative log likelihood probability (NLLP) is developed to provide a comprehensible indication for quantifying bearing performance degradation. The proposed approach has shown to provide better performance than using regular features (e.g., root mean square (RMS)). The experimental results indicate potential applications of LPP-based FE and GMM as effective tools for bearing performance degradation assessment.