Distracted driving is the major cause for injuries and fatalities due to road accidents. Driving is a continuous task which requires constant attention of the driver; a certain level of distraction can cause the driver lose his/her attention to the driving task which might lead to an accident. Thus, detection of distraction will help reduce the number of accidents. There has been much research conducted for automatic detection of driver distraction. Many previous approaches have employed camera based techniques. However these methods might detect the distraction rather late to warn the drivers. On the other hand, neurophysiological signals using Electroencephalography (EEG) have shown to be reliable indicator of distraction. However EEG signals are very complex and the technology is intrusive to the drivers, which creates serious doubt for its practical applications. The objective of this study is to investigate if Galvanic Skin Responses (GSR) can be used to detect distraction under naturalistic driving condition using a wrist band wearable. Six driver subjects participated in our realistic driving experiments. Our experimental results demonstrated high accuracies of detection under subject dependents scenarios. We also investigated the possibility of subject independent distraction detection employing non-linear space transformation based on kernel analysis and support vector machines (SVM).