Towards Layer-Wise Optimization of Contextual Neural Networks with Constant Field of Aggregation

1. Mendez, K.M., Broadhurst, D., Reinke S.N., The application of artificial neural networks in metabolomics: a historical perspective. Metabolomics 15(11), 142. Springer (2019). https://doi.org/10.1007/s11306-019-1608-0

2. Tsai, Y.C., et al.: FineNet: a joint convolutional and recurrent neural network model to forecast and recommend anomalous financial items. In: Proceedings of the 13th ACM Conference on Recom-mender Systems RecSys 2019, New York, USA, pp. 536–537. ACM (2019)

   Google Scholar 

3. Chen, S., Zhang, S., Shang, J., Chen, B., Zheng, N.: Brain-inspired cognitive model with attention for self-driving cars. IEEE Trans. Cogn. Dev. Syst. 11(1), 13–25. IEEE (2019)

   Google Scholar 

4. Nasser, I.M., Abu-Naser, S.S.: Lung cancer detection using artificial neural network. Int. J. of Eng. Inf. Syst. (IJEAIS) 3(3), 17–23 (2019)

   Google Scholar 

5. Suleymanova, I., et al.: A deep convolutional neural network approach for astrocyte detection. Sci. Rep. 8(12878), 1–7 (2018)

   Google Scholar 

6. Wang, Z.H., Horng, G.J., Hsu, T.H., Chen, C.C., Jong, G.J.: A novel facial thermal feature extraction method for non-contact healthcare system. IEEE Access 8, 86545–86553. IEEE (2020)

   Google Scholar 

7. Qiming, Z., et al.: Artificial neural networks enabled by nanophotonics. Light: Sci. Appl. 8(1), 14. Nature Publishing Group (2019)

   Google Scholar 

8. Guest, D., Cranmer, K., Whiteson, D.: Deep learning and its application to LHC physics. Annu. Rev. Nucl. Part. Sci. 68, 1–22 (2018)

   Article  Google Scholar 

9. Liu, L., Zheng, Y., Tang, D., Yuan, Y., Fan, C., Zhou, K.: Automatic skin binding for production characters with deep graph networks. ACM Trans. Graph. (SIGGRAPH) 38(4), Article 114, 12 (2019)

   Google Scholar 

10. Gao, D., Li, X., Dong, Y., Peers, P., Xu, K., Tong, X.: Deep inverse rendering for high-resolution SVBRDF estimation from an arbitrary number of images. ACM Trans. Graph. (SIGGRAPH) 38(4), article 134, 15 (2019)

    Google Scholar 

11. Gong, K., et al.: Iterative PET image reconstruction using convolutional neural network representation. IEEE Trans. Med. Imag. 38(3), 675–685. IEEE (2019)

    Google Scholar 

12. Munkhdalai, L., Park, K.-H., Batbaatar, E., Theera-Umpon, N., Ryu, K.H.: Deep learning-based demand forecasting for Korean postal delivery service. IEEE Access 8, 188135–188145 (2020)

    Google Scholar 

13. Batbaatar, E., Li, M., Ryu, K.H.: Semantic-emotion neural network for emotion recognition from text. IEEE Access 7, 111866–111878. IEEE (2019)

    Google Scholar 

14. Higgins, I., et al.: β-VAE: learning basic visual concepts with a constrained variational framework. In: International Conference Learning Representations. ICLR 2017, vol. 2, no. 5, pp. 1–22 (2017)

    Google Scholar 

15. Karras, T., Aila, T., Laine, S., Lehtinen, J.: Progressive growing of GANs for improved quality, stability, and variation. In: International Conference on Learning Representations. ICLR 2018, pp. 1–26 (2018)

    Google Scholar 

16. Huang, X., Tan, H., Lin, G., Tian, Y.: A LSTM-based bidirectional translation model for optimizing rare words and terminologies. In: 2018 IEEE International Conference on Artificial Intelligence and Big Data (ICAIBD), China, pp. 5077–5086. IEEE (2018)

    Google Scholar 

17. Athiwaratkun, B., Stokes, J.W.: Malware classification with LSTM and GRU language models and a character-level CNN. In: Proceedings 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), USA, 2017, pp. 2482–2486. IEEE (2017)

    Google Scholar 

18. Amato, F., et al.: Multilayer perceptron: an intelligent model for classification and intrusion detection. In: 31st International Conference on Advanced Information Networking and Applications Workshops (WAINA), Taipei, Taiwan, pp. 686–691. IEEE (2017)

    Google Scholar 

19. Dozono, H., Niina, G., Araki, S.: Convolutional self organizing map. In: 2016 IEEE International Conference on Computational Science and Computational Intelligence (CSCI), pp. 767–771. IEEE (2016)

    Google Scholar 

20. Gościewska, K., Frejlichowski, D.: A combination of moment descriptors, fourier transform and matching measures for action recognition based on shape. In: Krzhizhanovskaya, V.V., et al. (eds.) ICCS 2020. LNCS, vol. 12138, pp. 372–386. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-50417-5_28

    Chapter  Google Scholar 

21. Frejlichowski, D.: Low-level greyscale image descriptors applied for intelligent and contextual approaches. In: Nguyen, N.T., Gaol, F.L., Hong, T.-P., Trawinski, B. (eds.) ACIIDS 2019. LNCS (LNAI), vol. 11431. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-14799-0

22. Huk, M.: Backpropagation generalized delta rule for the selective attention Sigma-if artificial neural network. Int. J. App. Math. Comp. Sci. 22, 449–459 (2012)

    Google Scholar 

23. Huk, M.: Notes on the generalized backpropagation algorithm for contextual neural networks with conditional aggregation functions. J. Intell. Fuzzy Syst. 32, 1365–1376. IOS Press (2017)

    Google Scholar 

24. Huk, M.: Stochastic optimization of contextual neural networks with RMSprop. In: Nguyen, N.T., Jearanaitanakij, K., Selamat, A., Trawiński, B., Chittayasothorn, S. (eds.) ACIIDS 2020. LNCS (LNAI), vol. 12034, pp. 343–352. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-42058-1_29

    Chapter  Google Scholar 

25. Burnell, E.D., Wołk, K., Waliczek, K., Kern, R.: The impact of constant field of attention on properties of contextual neural networks. In: Nguyen, N.T., Trawinski, B., et al. (eds.) 12th Asian Conference on Intelligent Information and Database Systems, ACIIDS 2020. LNAI, vol. 12034, pp. 364–375, Springer (2020). https://doi.org/10.1007/978-3-030-42058-1_31

26. Huk, M., Non-uniform initialization of inputs groupings in contextual neural networks. In: Nguyen, N., Gaol F., Hong TP., Trawiński B. (eds) Intelligent Information and Database Systems. ACIIDS 2019. LNCS, vol. 11432, pp. 420–428. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-14802-7_36

27. Huk, M.: Training contextual neural networks with rectifier activation functions: role and adoption of sorting methods. J. Intell. Fuzzy Syst. 37(6), 7493–7502. IOS Press (2019)

    Google Scholar 

28. Huk, M.: Weights ordering during training of contextual neural networks with generalized error backpropagation: importance and selection of sorting algorithms. In: Nguyen, N.T., Hoang, D.H., Hong, T.-P., Pham, H., Trawiński, B. (eds.) ACIIDS 2018. LNCS (LNAI), vol. 10752, pp. 200–211. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-75420-8_19

    Chapter  Google Scholar 

29. Szczepanik, M., et al.: Multiple classifier error probability for multi-class problems. Eksploatacja i Niezawodnosc - Maintenance and Reliability 51(3), 12–16 (2011). https://doi.org/10.17531/ein

30. Huk, M.: Measuring computational awareness in contextual neural networks. In: 2016 IEEE International Conference on Systems, Man, and Cybernetics (SMC), Budapest, pp. 002254–002259 (2016). https://doi.org/10.1109/SMC.2016.7844574

31. Huk, M., Measuring the effectiveness of hidden context usage by machine learning methods under conditions of increased entropy of noise. In: 2017 3rd IEEE International Conference on Cybernetics (CYBCONF), Exeter, pp. 1–6 (2017). https://doi.org/10.1109/CYBConf.2017.7985787

32. Huk, M., Pietraszko, J.: Contextual neural-network based spectrum prediction for cognitive radio. In: 4th International Conference on Future Generation Communication Technology (FGCT 2015). IEEE Computer Society, London, UK, pp. 1–5 (2015)

    Google Scholar 

33. Szczepanik, M., Jóźwiak, I.: Data management for fingerprint recognition algorithm based on characteristic points’ groups. In: New Trends in Databases and Information Systems. Foundations of Computing and Decision Sciences, vol. 38, no. 2, pp. 123–130, Springer (2013). https://doi.org/10.1007/978-3-642-32518-2_40

34. Szczepanik, M., Jóźwiak, I.: Fingerprint recognition based on minutes groups using directing attention algorithms. In: Rutkowski, L., Korytkowski, M., Scherer, R., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds.) ICAISC 2012. LNCS (LNAI), vol. 7268, pp. 347–354. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-29350-4_42

    Chapter  Google Scholar 

35. Kwiatkowski, J., et al.: Context-sensitive text mining with fitness leveling genetic algorithm. In: 2015 IEEE 2nd International Conference on Cybernetics (CYBCONF), Gdynia, Poland, 2015, pp. 1–6. Electronic Publication (2015). https://doi.org/10.1109/CYBConf.2015.7175957. ISBN: 978-1-4799-8321-6

36. Huk, M.: Using context-aware environment for elderly abuse prevention. In: Nguyen, N.T., Trawiński, B., Fujita, H., Hong, T.-P. (eds.) ACIIDS 2016. LNCS (LNAI), vol. 9622, pp. 567–574. Springer, Heidelberg (2016). https://doi.org/10.1007/978-3-662-49390-8_55

    Chapter  Google Scholar 

37. Huk, M.: Context-related data processing with artificial neural networks for higher reliability of telerehabilitation systems. In: 17th International Conference on E-health Networking, Application & Services (HealthCom). IEEE Computer Society, Boston, USA, pp. 217–221 (2015)

    Google Scholar 

38. Privitera, C.M., Azzariti, M., Stark, L.W.: Locating regions-of-interest for the Mars Rover expedition. Int. J. Remote Sens. 21, 3327–3347. Taylor and Francis (2000)

    Google Scholar 

39. UCI Machine Learning Repository. https://archive.ics.uci.edu/ml

40. Golub, T.R., et al.: Molecular classification of cancer: class discovery and class prediction by gene expression monitoring. Science 286, 531–537 (1999)

    Article  Google Scholar 

41. Glosser, C., Piermarocchi, C., Shanker, B.: Analysis of dense quantum dot systems using a self-consistent Maxwell-Bloch framework. In: Proceedings of 2016 IEEE International Symposium on Antennas and Propagation (USNC-URSI), Puerto Rico, pp. 1323–1324. IEEE (2016)

    Google Scholar 

42. H2O.ai documentation. https://docs.h2o.ai/h2o/latest-stable/h2o-docs/index.html

43. Rodriguez, J.D., et al.: Sensitivity analysis of k-fold cross validation in prediction error estimation. IEEE Trans. Patt. Anal. Mach. Int. 32(3), 569–575 (2010)

    Google Scholar 

44. Bouckaert, R.R.: Estimating replicability of classifier learning experiments. In: Proceedings of the 21st International Conference on Machine Learning, Banf, Canada (2004)

    Google Scholar 

Download references