Although a concept class may be learnt more efficiently using quantum samples as compared with classical samples in certain scenarios, quantum learners are asymptotically no more efficient than classical ones in the quantum PAC and Agnostic learning models. Lower bounds on sample complexity in these models were previously established via quantum state identification and Fourier analysis. In this paper, we derive optimal lower bounds for quantum sample complexity in both models via an information-theoretic approach. The proofs are arguably simpler, and the same ideas can potentially be used to derive optimal bounds for other problems in quantum learning theory. We then turn to a quantum analogue of the Coupon Collector problem, a classic problem from probability theory also of importance in the study of PAC learning. The quantum sample complexity of this problem has been characterised up to constant factors. First, we show that the information-theoretic approach mentioned above provably does not yield the optimal lower bound. As a by-product, we get a natural ensemble of pure states in arbitrarily high dimensions which are not easily (simultaneously) distinguishable, whereas the ensemble has close to maximal Holevo information. Second, we discover that the information-theoretic approach yields an asymptotically optimal bound for an approximation variant of the problem. Finally, we derive a sharper lower bound for the Quantum Coupon Collector problem via the generalised Holevo-Curlander bounds. All the aspects of the problem we study rest on properties of the spectrum of the associated Gram matrix, which may be of independent interest.