Rail area detection is essential in active obstacle perception system of the train. This paper presents an efficient rail area detection method based on the convolutional neural network (CNN). The proposed method is divided into two main parts: extraction of the rail area and further optimization. First, a CNN architecture is established to achieve accurate rail area detection, enabling the pixel-level classification of the rail area. It is notable that the main improvement of our architecture is dilated cascade connection and cascade sampling. Second, an improved polygon fitting method is applied to optimize the contour of the extracted rail area and, thus, obtains a more elegant outline of the rail region. As shown by the experimental results, the excellent accuracy is obtained by using our method, i.e., 98.46% mean intersection-over-union and 99.15% mean pixel accuracy on the BH-rail-dataset, and verified the applicability of our detection method in a large-scale traffic scene video frames of Beijing metro Yanfang line and Shanghai metro line 6.