Visual-based semantic segmentation for traffic scene plays an important role in intelligent vehicles. In this paper, we present a new real-time deep fully convolution neural network (FCNN) for pixel-wise segmentation with six channel inputs. The six channel inputs include the RGB three channel color image, the Disparity (D) image generated by stereo vision sensor, the image to describe the Height (H) of each pixel above road ground, and the image to describe the Angle (A) between each pixel normal direction and the predicted direction of gravity, which are defined as a RGB-DHA multi-feature map. The FCNN is simplified and modified based on AlexNet to meet the real-time requirements of intelligent vehicle for environmental perception. The proposed algorithm is tested and compared in Cityscapes dataset, yields global accuracies 73.4% and 22ms for $400 \times 200$ resolution image with one Titan X GPU.