Nothing Special   »   [go: up one dir, main page]
Skip to main content
SpringerLink
Log in

Comparative evaluation of performance measures for human driving skills

  • Original Research
  • Published:
Intelligent Service Robotics Aims and scope Submit manuscript

Abstract

In this paper, we evaluate the adequacy of several performance measures for the evaluation of driving skills between different drivers. This work was motivated by the need for a training system that captures the driving skills of an expert driver and transfers the skills to novice drivers using a haptic-enabled driving simulator. The performance measures examined include traditional task performance measures, e.g., the mean position error, and a stochastic distance between a pair of hidden Markov models (HMMs), each of which is trained for an individual driver. The emphasis of the latter is on the differences between the stochastic somatosensory processes of human driving skills. For the evaluation, we developed a driving simulator and carried out an experiment that collected the driving data of an expert driver whose data were used as a reference for comparison and of many other subjects. The performance measures were computed from the experimental data, and they were compared to each other. We also collected the subjective judgement scores of the driver’s skills made by a highly-experienced external evaluator, and these subjective scores were compared with the objective performance measures. Analysis results showed that the HMM-based distance metric had a moderately high correlation between the subjective scores and it was also consistent with the other task performance measures, indicating the adequacy of the HMM-based metric as an objective performance measure for driving skill learning. The findings of this work can contribute to developing a driving simulator for training with an objective assessment function of driving skills.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Notes

  1. The results of Experiment I was presented in our previous conference paper [14]. The permission to reprint its content including text and figures was granted from IEEE under the license number 3171111368907.

  2. Although these speed limits may seem low, the actual driving speed felt rather fast in the simulation because of the viewport setting used for graphics.

References

  1. Irrlicht engine, a free open source 3d engine. http://irrlicht.sourceforge.net/

  2. Newton game dynamics, open-source physics engine. www.newtondynamics.com/

  3. Berndt H, Emmert J, Dietmayer K (2008) Continuous driver intention recognition with hidden Markov models. In: Proceedings of the International IEEE Conference on Intelligent Transportation Systems, pp 1189–1194. doi:10.1109/ITSC.2008.4732630

  4. Calinon S, Billard A (2005) Recognition and reproduction of gestures using a probabilistic framework combining pca, ica and hmm. In: Proceedings of the 22nd international conference on Machine learning, ICML ’05, pp 105–112. ACM, New York, NY, USA. doi:10.1145/1102351.1102365. http://doi.acm.org/10.1145/1102351.1102365

  5. Cox TF, Cox MAA (1994) Multidimensional Scaling. Chapman & Hall, London

    MATH  Google Scholar 

  6. Fraser AM (2008) Hidden Markov models and dynamical systems. SIAM (Society for Industrial and Applied Mathematics)

  7. Juang BH, Rabiner LR (1985) A probabilistic distance measure for hidden Markov models. AT &T Tech J 64(2):391–408

    Article  MathSciNet  Google Scholar 

  8. Kim S, Park G, Yim S, Choi S, Choi S (2009) Gesture-recognizing hand-held interface with vibrotactile feedback for 3d interaction. IEEE Trans Consumer Electron 55(3):1169–1177. doi:10.1109/TCE.2009.5277972

    Article  Google Scholar 

  9. Kruskal J (1964) Multidimensional scaling by optimizing goodness of fit to a nonmetric hypothesis. Psychometrika 29(1):1–27

    Article  MathSciNet  MATH  Google Scholar 

  10. Lee J, Choi S (2010) Effects of haptic guidance and disturbance on motor learning: potential advantage of haptic disturbance. In: Proceedings of the Haptics Symposium, IEEE, pp 335–342

  11. Meng X, Lee KK, Xu Y (2006) Human driving behavior recognition based on hidden Markov models. In: Proceedings of the IEEE International Conference on Robotics and Biomimetics, pp 274–279. doi:10.1109/ROBIO.2006.340166

  12. Mitrovic D (2005) Reliable method for driving events recognition. IEEE Trans Intel Transp Syst 6(2):198–205. doi:10.1109/TITS.2005.848367

    Article  Google Scholar 

  13. Murphy K (2005) Hidden markov model (hmm) toolbox for matlab. http://www.cs.ubc.ca/murphyk/Software/HMM/hmm.html (1998). Last updated: June 2005

  14. Osgouei RH, Choi S (2012) Evaluation of driving skills using an hmm-based distance measure. In: Proceedings of the IEEE International Symposium on Haptic Audio Visual Environments and Games, pp. 50–55. (\(\copyright \) 2012 IEEE. Reprinted, with permission, from Evaluation of Driving Skills Using an HMM-based Distance Measure by Reza Haghighi Osgouei and Seungmoon Choi. In: Proceedings of the IEEE International Symposium on Haptic Audio Visual Environments and Games)

  15. Porikli F (2004) Trajectory distance metric using hidden Markov model based representation. In: IEEE European Conference on Computer Vision, PETS, Workshop, vol. 3

  16. Rabiner LR (1989) A tutorial on hidden Markov models and selected applications in speech recognition. Proc IEEE 77:257–286

    Article  Google Scholar 

  17. Rosen J, Solazzo M, Hannaford B, Sinanan M (2002) Task decomposition of laparoscopic surgery for objective evaluation of surgical residents’ learning curve using hidden Markov model. Comput Aided Surg 7:49–61

    Article  Google Scholar 

  18. Takano W, Matsushita A, Iwao K, Nakamura Y (2008) Recognition of human driving behaviors based on stochastic symbolization of time series signal. In: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, pp 167–172. doi:10.1109/IROS.2008.4650671

  19. Torkkola K, Venkatesan S, Liu H (2005) Sensor sequence modeling for driving. In: Proceedings of the International Florida Artificial Intelligence Research Society Conference

  20. Yang J, Xu Y, Chen C (1993) Hidden markov model approach to skill learning and its application in telerobotics. In: Proceedings of the IEEE International Conference on Robotics and Automation, vol. 1, pp 396–402. doi:10.1109/ROBOT.1993.292013

  21. Zucchini W (2009) Hidden Markov models for time series: an introduction using R. CRC Press, Boca Raton

    Book  Google Scholar 

Download references

Acknowledgments

This work was supported by NRF through an ERC 2011-0030075, a Pioneer program 2011-0027995, a BRL 2012-0008835, and a grant 013R1A2A2A01016907.

Author information

Authors and Affiliations

  1. Pohang University of Science and Technology (POSTECH), No. 108, Science Building 4, Cheongam-ro, Nam-gu, Pohang, Gyungbuk, 790-784, Republic of Korea

    Reza Haghighi Osgouei

  2. Pohang University of Science and Technology (POSTECH), Pohang, Gyungbuk, Republic of Korea

    Hojin Lee & Seungmoon Choi

Authors
  1. Reza Haghighi Osgouei
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hojin Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Seungmoon Choi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Reza Haghighi Osgouei.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Haghighi Osgouei, R., Lee, H. & Choi, S. Comparative evaluation of performance measures for human driving skills. Intel Serv Robotics 6, 169–180 (2013). https://doi.org/10.1007/s11370-013-0134-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11370-013-0134-6

Keywords

Search

Navigation