Environmental perception tasks such as object detection and free space detection based on 3+1D radar severely suffer from the disorder and sparsity of point cloud. To tackle this problem, we propose a novel Multi-Task Radar-based Single Stage Detector, termed MTRadSSD, where we adopt instance-aware sampling strategies to discover multi-class road users and propose an occupancy map tool based on kernel density estimation (KDE) to make predictions in bird’s eye view (BEV). The denoised occupancy map also plays key role in generating polygon represented free space in the scene. As a result, our elaborated sampling strategies effectively retained useful semantic information and narrowed the difference of detection performance across object categories. Meanwhile, our MTRadSSD outperforms those state-of-the-art approaches in terms of real-time requirement and detection accuracy. In detail, the proposed method achieves an satisfactory speed of ˜ View-of-Delft (VOD). With IoU thresholds 0.5/0.25/0.25 the average prediction precision (AP) of easy-level objects (cars, pedestrians and cyclists) reaches at competitive 52.2%, 61.1%, 86.3%, respectively, while mean IoU of free space is 87.8%. Especially, the occupancy map also makes difference in improving prediction precision of object orientation dramatically to averaged 64.0%.