Emotion is a property by which human beings and machines can be differentiated as machines are emotionless while human beings are not. If the emotion of a speaker is recognized then others can interact accordingly. This paper presents a new approach for recognizing all the six basic emotions (Happy, anger, fear, sadness, boredom and neutral) from the speech signals more effectively. To recognize the emotion of a speaker, pitch value and two wavelet packet feature vectors derived from speech signals are used. Principal Component Analysis (PCA) has been applied to reduce the dimension of feature vectors. Random Forest (RF) and Support Vector Machine (SVM) classifiers are trained separately based on these reduced feature vectors. The experimental results show that the accuracy of emotion recognition with Random Forest classifier is 86.11% while with SVM classifier it is 84.41%. Experimentally, it is also found that clean speech of 1 sec duration is sufficient enough to recognize emotion of the speaker.