A new strategy for Dimensionality Reduction with the aim of providing reduced set with both original and combinations of features is studied. For this purpose, a framework called Minimum Projection error and Minimum Redundancy (MPeMR) to generate orthogonal compound (both original and combinations) features by minimizing both projection error and redundancy among them is proposed. An iterative approximation method under the proposed framework is also presented. Using different measures for projection error, redundancy and representative feature selection, three supervised and three unsupervised methods are considered. Experimental results are compared with both state-of-the-art feature selection and extraction methods, in both supervised and unsupervised cases, and proposed CFG methods outperform these methods. In each stage of the proposed approximation method, pairs of features are considered i.e., NPe is calculated based on two features and removal of redundancy is also based on two features. Instead of (2, 2), one could have considered (2, 3), or (3, 2) or any other such combination of features. Further research is focused along this direction.