Data coverage and completeness are crucial data quality dimensions for machine learning (ML) applications. Data discovery allows retrieving relevant features from unstructured data lakes to augment the training data. However, existing methods are fragmented: they often target only individual ML tasks, decouple table retrieval from feature selection, and neglect inter-feature relationships, resulting in redundant, high-dimensional data. In this paper, we introduce Matryoshka, a holistic system that discovers and selects relevant features from data lakes for downstream ML applications. Matryoshka replaces costly join materialization with compact Gram matrix sketches and performs feature selection via incrementally fitted linear proxy models with correlation-based pruning, yielding redundancy-free feature sets for regression and classification. We evaluate Matryoshka on multiple data lakes of varying size and heterogeneity. Matryoshka improves downstream prediction quality by 18.5% on average while achieving the lowest geometric mean runtime and up to 120x faster execution on join-intensive workloads.