It is well recognized that air quality inference is of great importance for environmental protection. However, due to the limited monitoring stations and various impact factors, e.g., meteorology, traffic volume and human mobility, inference of air quality index (AQI) could be a difficult task. Recently, with the development of new ways for collecting and integrating urban, mobile, and public service data, there is a potential to leverage spatial relatedness and temporal dependencies for better AQI estimation. To that end, in this paper, we exploit a novel spatio-temporal multi-task learning strategy and develop an enhanced framework for AQI inference. Specifically, both time dependence within a single monitoring station, and spatial relatedness across all the stations will be captured, and then well trained with effective optimization to support AQI inference tasks. As air-quality related features from cross-domain data have been extracted and quantified, comprehensive experiments based on real-world datasets validate the effectiveness of our proposed framework with significant margin compared with several state-of-the-art baselines, which support the hypothesis that our spatio-temporal multi-task learning framework could better predict and interpret AQI fluctuation.