Abstract
To facilitate the application of support vector machines (SVMs) in embedded systems, we propose and test a parallel and scalable digital architecture based on the sequential minimal optimization (SMO) algorithm for training SVMs. By taking advantage of the mature and popular SMO algorithm, the numerical instability issues that may exist in traditional numerical algorithms are avoided. The error cache updating task, which dominates the computation time of the algorithm, is mapped into multiple processing units working in parallel. Experiment results show that using the proposed architecture, SVM training problems can be solved effectively with inexpensive fixed-point arithmetic and good scalability can be achieved. This architecture overcomes the drawbacks of the previously proposed SVM hardware that lacks the necessary flexibility for embedded applications, and thus is more suitable for embedded use, where scalability is an important concern.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Anguita, D., Boni, A., Ridella, S., 2003. A digital architecture for support vector machines: theory, algorithm, and FPGA implementation. IEEE Trans. Neur. Networks, 14(5):993–1009. [doi:10.1109/TNN.2003.816033]
Anguita, D., Pischiutta, S., Ridella, S., Sterpi, D., 2006. Feed-forward support vector machine without multipliers. IEEE Trans. Neur. Networks, 17(5):1328–1331. [doi:10.1109/TNN.2006.877537]
Biasi, I., Boni, A., Zorat, A., 2005. A Reconfigurable Parallel Architecture for SVM Classification. Proc. IEEE Int. Joint Conf. on Neural Networks, p.2867–2872. [doi:10. 1109/IJCNN.2005.1556380]
Burges, C.J.C., 1998. A tutorial on support vector machines for pattern recognition. Data Min. Knowl. Discov., 2(2):121–167. [doi:10.1023/A:1009715923555]
Catanzaro, B., Sundaram, N., Keutzer, K., 2008. Fast Support Vector Machine Training and Classification on Graphics Processors. Proc. 25th Int. Conf. on Machine Learning, p.104–111. [doi:10.1145/1390156.1390170]
Chen, S., Gibson, G.J., Cowan, C.F.N., Grant, P.M., 1990. Adaptive equalization of finite nonlinear channels using multilayer perceptrons. EURASIP Signal Process., 20(2): 107–119.
Choi, W.Y., Ahn, D., Pan, S.B., Chung, K.I., Chung, Y.W., Chung, S.H., 2006. SVM-based speaker verification system for match-on-card and its hardware implementation. ETRI J., 28(3):320–328. [doi:10.4218/etrij.06.0105.0022]
Frieß, T.T., Cristianini, N., Campbell, C., 1998. The Kernel-Adatron Algorithm: A Fast and Simple Learning Procedure for Support Vector Machines. Proc. 15th Int. Conf. on Machine Learning, p.188–196.
Graf, H.P., Cadambi, S., Durdanovic, I., Jakkula, V., Sankaradass, M., Cosatto, E., Chakradhar, S.T., 2008. A Massively Parallel Digital Learning Processor. 22nd Annual Conf. on Neural Information Processing Systems, p.529–536.
Keerthi, S.S., Shevade, S.K., Bhattacharyya, C., Murthy, K.R.K., 2001. Improvements to Platt’s SMO algorithm for SVM classifier design. Neur. Comput., 13(3):637–649. [doi:10.1162/089976601300014493]
Manikandan, J., Venkataramani, B., Avanthi, V., 2009. FPGA Implementation of Support Vector Machine Based Isolated Digit Recognition System. Proc. 22nd Int. Conf. on VLSI Design, p.347–352. [doi:10.1109/VLSI.Design.2009.23]
Platt, J.C., 1999. Fast Training of Support Vector Machines Using Sequential Minimal Optimization. In: Schölkopf, B., Burges, C., Smola, A. (Eds.), Advances in Kernel Methods: Support Vector Learning. MIT Press, Cambridge, MA, p.185–208.
Schölkopf, B., Burges, C.J.C., Smola, A.J., 1999. Advances in Kernel Methods: Support Vector Learning. MIT Press, Cambridge, MA, p.1–16.
Sebald, D.J., Bucklew, J.A., 2000. Support vector machine techniques for nonlinear equalization. IEEE Trans. Signal Process., 48(11):3217–3226. [doi:10.1109/78.875477]
Sun, Z., Zhang, L., Tang, E., 2005. An incremental learning method based on SVM for online sketchy shape recognition. LNCS, 3610:655–659. [doi:10.1007/11539087_82]
Vapnik, V.N., 1998. Statistical Learning Theory. Wiley, New York, p.493–520.
Wee, J.W., Lee, C.H., 2004. Concurrent support vector machine processor for disease diagnosis. LNCS, 3316:1129–1134. [doi:10.1007/b103766]
Author information
Authors and Affiliations
Corresponding author
Additional information
Project (No. 60720106003) supported by the National Natural Science Foundation of China
Rights and permissions
About this article
Cite this article
Cao, Kk., Shen, Hb. & Chen, Hf. A parallel and scalable digital architecture for training support vector machines. J. Zhejiang Univ. - Sci. C 11, 620–628 (2010). https://doi.org/10.1631/jzus.C0910500
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1631/jzus.C0910500
Key words
- Support vector machine (SVM)
- Sequential minimal optimization (SMO)
- Field-programmable gate array (FPGA)
- Scalable architecture