Abstract
Emulations of cellular nonlinear networks (CNN) on digital reconfigurable hardware have proved to be adequate for highly-efficient computation of massive data, exceeding the accuracy and flexibility of full-custom designs. Based on a recently-proposed architecture for the emulation of a large-scale CNN universal machine, a new real-time video processing system has been developed. Due to its free programmability and massively-parallel architecture the system is very suitable for high-speed computation of complex algorithms that follow the idea of spatio-temporal computing. Implemented on a state-of-the-art Xilinx Zynq system-on-chip, the proposed setup is capable of processing a \(640\times 480\)p video stream with up to 1,700 fps, depending on the respective algorithm.
Similar content being viewed by others
References
Abbo, A.A., Kleihorst, R.P., Choudhary, V., Sevat, L., Wielage, P., Mouy, S., Vermeulen, B., Heijligers, M.: Xetal-II: a 107 GOPS, 600 mW massively parallel processor for video scene analysis. Solid-State Circ. IEEE J. 43(1), 192–201 (2008)
Analog Devices Inc. 225 MHz, High Performance HDMI Transmitter with ARC: ADV7511 (2010). http://www.analog.de
Arce, G.R.: Nonlinear signal processing: a statistical approach. Wiley, New Jersey (2005)
ARM Ltd. AMBA AXI Protocol Specification. http://www.arm.com/ (2003)
Avnet Inc. DVI I/O FMC Module Hardware Guide. http://www.avnet.com (2010)
Blug, A., Strohm, P., Carl, D., Hofler, H., Blug, B., Kailer, A.: On the potential of current CNN cameras for industrial surface inspection. In: Cellular Nanoscale Networks and Their Applications (CNNA), 2012 13th International Workshop on, IEEE, pp 1–6 (2012)
Braunschweig, R., Müller, J., Müller, J., Tetzlaff, R.: NERO mastering 300k CNN cells. In: Circuit Theory and Design (ECCTD), 2013 European Conference on, pp. 1–4 (2013). doi:10.1109/ECCTD.2013.6662202
Butter, M., Leis, M., Sandtke, M., McLean, M., Lincoln, J., Wilson, A.: The Leverage Effect of Photonics Technologies: The European Perspective. Final Report, March 2011. Study prepared for the European Commission, DG Information Society and Media under reference SMART 2009/0066 (2011)
Cheng, C.C., Lin, C.H., Li, C.T., Chen, L.G.: ivisual: An intelligent visual sensor soc with 2790 fps cmos image sensor and 205 gops/w vision processor. Solid-State Circ., IEEE J. 44(1), 127–135 (2009). doi:10.1109/JSSC.2008.2007158
Chua, L.O., Yang, L.: Cellular neural networks: theory. IEEE Trans. Circ. Syst. 35(10), 1257–1272 (1988)
Chua, L.O., Roska, T., Venetianer, P.L.: The CNN is universal as the Turing machine. IEEE Trans. Circ. Syst. I-Regular Papers 40, 289–291 (1993). doi:10.1109/81.224308
Chua, L.O., Hasler, M., Moschytz, G.S., Neirynck, J.: Autonomous cellular neural networks: A unified paradigm for pattern formation and active wave propagation. Circ. Syst. I: Fundam. Theory Appl., IEEE Trans. 42(10), 559–577 (1995)
Dudek, P.: An asynchronous cellular logic network for trigger-wave image processing on fine-grain massively parallel arrays. IEEE Trans. Circ. Syst. II: Exp. Briefs 53(5), 354–358 (2006)
Dudek, P., Hicks, P.J.: A general-purpose processor-per-pixel analog SIMD vision chip. Circ. Syst. I: Regular Papers, IEEE Trans. 52(1), 13–20 (2005)
Földesy, P., Zarandy, A., Rekeczky, C.: Configurable 3D-integrated focal-plane cellular sensor-processor array architecture. Int. J. Circ. Theory Appl. 36(5–6), 573–588 (2008)
Hoefflinger, B.: Chips 2020. Springer (2012)
Kahle, J.A., Day, M.N., Hofstee, H.P., Johns, C.R., Maeurer, T.R., Shippy, D.J.: Introduction to the Cell multiprocessor. Ibm J. Res. Dev. 49, 589–604 (2005). doi:10.1147/rd.494.0589
Karacs, K, et al.: Software Library for Cellular Wave Computing Engines V. 3.1. Budapest, Hungary: MTA-SZTAKI and Pazmany University (2010)
Kayaer, K., Tavsanoglu, V.: A new approach to emulate CNN on FPGAs for real time video processing. In: Cellular Neural Networks and Their Applications, 2008. CNNA 2008. 11th International Workshop on, IEEE, pp 23–28 (2008)
Kirk, D.: NVIDIA CUDA software and GPU parallel computing architecture. ISMM 7, 103–104 (2007)
Linan, G., Espejo, S., Domínguez-Castro, R., Rodríguez-Vázquez, A.: ACE4k: An analog I/O 64\(\times\) 64 visual microprocessor chip with 7-bit analog accuracy. Int. J. Circ. Theory Appl. 30(2–3), 89–116 (2002)
Lopich, A., Dudek, P. A general-purpose vision processor with 160\(\times\) 80 pixel-parallel SIMD processor array. In: Custom Integrated Circuits Conference (CICC), 2013 IEEE, IEEE, pp 1–4 (2013)
Martínez, J.J., Garrigós, J., Toledo, J., Fernández, E., Manuel Ferrández, J.: Implementation of a CNN-based retinomorphic model on a high performance reconfigurable computer. Neurocomputing 74(8), 1290–1297 (2011)
Müller, J., Müller, J., Tetzlaff, R.: A new cellular nonlinear network emulation on FPGA for EEG signal processing in epilepsy. In: Proceedings of SPIE, vol 8068, p 80680M (2011)
Müller, J., Becker, R., Müller, J., Tetzlaff, R. CESAR: Emulating Cellular Networks on FPGA. In: Cellular Nanoscale Networks and Their Applications (CNNA), 2012 13th International Workshop on, pp 1–5, (2012). doi:10.1109/CNNA.2012.6331409
Nagy, Z., Szolgay, P.: Configurable multilayer CNN-UM emulator on FPGA. IEEE Trans. Circ. Syst. I: Fundam. Theory Appl. 50(6), 774–778 (2003). doi:10.1109/TCSI.2003.812611
Nicolosi, L., Abt, F., Blug, A., Heider, A., Tetzlaff, R., Höfler, H.: A novel spatter detection algorithm based on typical cellular neural network operations for laser beam welding processes. Measur. Sci. Technol. 23(015), 401 (2012)
Nieto, A., Brea, V., Vilarino, D.: SIMD array on FPGA for B/W image processing. In: Cellular Neural Networks and Their Applications. CNNA 2008. 11th International Workshop on, IEEE, pp 202–207 (2008)
Rodriguez-Vazquez, A., Linan-Cembrano, G., Carranza, L., Roca-Moreno, E., Carmona-Galan, R., Jimenez-Garrido, F., Dominguez-Castro, R., Meana, S.E.: ACE16k: the third generation of mixed-signal SIMD-CNN ACE chips toward VSoCs. IEEE Trans. Circ. Syst. I: Regular Papers 51(5), 851–863 (2004). doi:10.1109/TCSI.2004.827621
Rodriguez-Vazquez, A., Dominguez-Castro, R., Jimenez-Garrido, F., Morillas, S., Listan, J., Alba, L., Utrera, C., Espejo, S., Romay, R.: The Eye-RIS CMOS vision system. In: Casier, H., Steyaert, M., van Roermund, A.H.M. (eds.) Analog Circuit Design. Sensors, Actuators and Power Drivers; Integrated Power Amplifiers from Wireline to RF; Very High Frequency Front Ends, Springer, Internetausgabe, pp 15–32 (2008)
Roska, T., Chua, L.O.: The CNN universal machine: an analogic array computer. IEEE Trans. Circ. Syst. II: Analog Digital Signal Process. 40(3), 163–173 (1993)
Steiner, G., Sobottka, S.B., Koch, E., Schackert, G., Kirsch, M.: Intraoperative imaging of cortical cerebral perfusion by time-resolved thermography and multivariate data analysis. J. Biomed. Optics 16(1):016, 001–016,001 (2011)
Venetianer, P.L., Werblin, F., Roska, T., Chua, L.O.: Analogic cnn algorithms for some image compression and restoration tasks. IEEE Trans. Circ. Syst. 1, Fundam. Theory Appl. 42(5), 278–284 (1995)
Vörösházi, Z., Kiss, A., Nagy, Z., Szolgay, P.: Implementation of embedded emulated-digital CNN-UM global analogic programming unit on FPGA and its application. Int. J. Circ. Theory Appl. 36, 589–603 (2008). doi:10.1002/cta.507
Xilinx Inc. LogiCORE IP AXI Video Direct Memory Access v3.01a. http://www.xilinx.com (2011)
Xilinx Inc. LogiCORE IP AXI DMA v6.03a. http://www.xilinx.com (2012)
Xilinx Inc. Zynq-7000 All Programmable SoC Overview. http://www.xilinx.com (2013)
Yildiz, N., Cesur, E., Tavsanoglu, V.: Demonstration of the Second Generation Real-time Cellular Neural Network Processor: RTCNNP-v2. In: Cellular Nanoscale Networks and Their Applications (CNNA), 2012 13th International Workshop on, pp 1–2 (2012). doi:10.1109/CNNA.2012.6331471
Zarándy, Á.: Focal-plane sensor-processor chips. Springer, Berlin (2011)
Zarándy, Á., Rekeczky, C.: 2D operators on topographic and non-topographic architectures—implementation, efficiency analysis, and architecture selection methodology. Int. J. Circ. Theory Appl. 39(10), 983–1005 (2011)
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Müller, J., Müller, J. & Tetzlaff, R. NEROvideo: a general-purpose CNN-UM video processing system. J Real-Time Image Proc 12, 763–774 (2016). https://doi.org/10.1007/s11554-014-0451-9
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11554-014-0451-9