Human mobility data refers to records of human movements, such as cellphone traces, vehicle GPS trajectories, geo-tagged posts, and photos. While successfully mining human mobility data can benefit many applications such as city planning, transportation, urban economics, and public safety, it is very challenging to model large-scale Heterogeneous Human Mobility Data (HHMD) that are generated from different resources. In this paper, we develop a general collective learning approach to model HHMD at an individual level towards identifying and quantifying the urban forms of residential communities. Specifically, our proposed method exploits two geographic regularities among HHMD. First, we jointly capture the correlations among residential communities, urban functions, temporal effects, and user mobility patterns by analogizing communities as documents and mobility patterns as words. Also, we further combine explicit LASSO analysis and significant testing into latent representation learning as a regularization term by analogizing compatible Point-of-Interests (POIs) as the meta-data of communities. In this way, we can learn the urban forms, including a mix of functions and corresponding portfolios, of residential communities from HHDM and POIs. We further leverage these learned results to address two application problems: real estate ranking and restaurant popularity prediction. Finally, we conduct intensive evaluations with a variety of real-world data, where experimental results demonstrate the effectiveness of our proposed modeling method and its successful applications for other problems.