In the process of globalization, English has become an essential skill. This article provides a brief introduction to the recognition process of English pronunciation errors based on deep learning. In the recognition process, the audio features of pronunciation were combined with the video features of lip movements during pronunciation to improve error detection performance. Subsequently, simulation experiments were conducted on the error detection algorithm, and a case analysis was performed on 100 freshmen from Hui College at Hebei Normal University to verify the effectiveness of the algorithm in correcting pronunciation. The results showed that the long short-term memory (LSTM) algorithm based on audio and video converged the fastest during training and had the smallest loss function. Additionally, it achieved the highest accuracy in phoneme recognition and pronunciation error detection, while being less affected by noise interference. After using the pronunciation error detection algorithm proposed in this article for oral correction training, students' pronunciation was significantly improved.