Abstract
Online Automatic Assessment System (OAAS) is a vital component in Online Judge systems. It is used to assess the correctness of programs submitted by the users. Developing OAAS is a non-trivial and challenging task due to various functional and non-functional dependencies. Few OAAS architectures were reported in the literature; however, their stability in handling the voluminous data produced by millions of submissions over a long duration is unknown. In this paper, we present the internal architecture of our OAAS, which has assessed five million program codes and has been operating stably for more than 10 years. We will also share some of our real-world experiences related to the operation of this system.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Wasik, S., Antczak, M., Badura, J., Laskowski, A., Sternal, T.: A survey on online judge systems and their applications. ACM Comput. Surv. (CSUR) 51(1), 1–34 (2018)
Watanobe, Y., Intisar, C.M., Cortez, R., Vazhenin, A.: Next-generation programming learning platform: architecture and challenges. In: SHS Web of Conferences, vol. 77, p. 01004 (2020)
Petit, J., et al.: Jutge.org: characteristics and experiences. IEEE Trans. Learn. Technol. 11(3), 321–333 (2018). https://doi.org/10.1109/TLT.2017.2723389
Georgouli, K., Guerreiro, P.: Incorporating an automatic judge into blended learning programming activities. In: Luo, X., Spaniol, M., Wang, L., Li, Q., Nejdl, W., Zhang, W. (eds.) ICWL 2010. LNCS, vol. 6483, pp. 81–90. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-17407-0_9
Rahman, M.M., Watanobe, Y., Nakamura, K.: Source code assessment and classification based on estimated error probability using attentive LSTM language model and its application in programming education. Appl. Sci. 10(8), 2973 (2020)
Watanobe, Y.: Aizu online judge. https://onlinejudge.u-aizu.ac.jp
Revilla, M.A., Manzoor, S., Liu, R.: Competitive learning in informatics: The UVa online judge experience. Olympiads Inf. 2(10), 131–148 (2008)
Kosowski, A., Małafiejski, M., Noiński, T.: Application of an online judge & contester system in academic tuition. In: Leung, H., Li, F., Lau, R., Li, Q. (eds.) ICWL 2007. LNCS, vol. 4823, pp. 343–354. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-78139-4_31
Labs, S.R.: Online compilers and programming challenges APIs - sphere engine. https://sphere-engine.com. Accessed 7 June 2020
Prologin, A.: “camisole,”. https://camisole.prologin.org. Accessed 7 June 2020
Došilović, H.Z., Mekterović, I.: Robust and scalable online code execution system. In: 43rd International Convention on Information, Communication and Electronic Technology (MIPRO). Opatija, Croatia 2020, pp. 1627–1632 (2020)
Mekterović, I., Brkić, L., Milašinović, B., Baranović, M.: Building a comprehensive automated programming assessment system. IEEE Access 8, 81154–81172 (2020). https://doi.org/10.1109/ACCESS.2020.2990980
Acknowledgement
This research was funded by the Japan Society for the Promotion of Science (JSPS) KAKENHI (Grant Number 19K12252).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Watanobe, Y., Rahman, M.M., Rage, U.K., Penugonda, R. (2021). Online Automatic Assessment System for Program Code: Architecture and Experiences. In: Fujita, H., Selamat, A., Lin, J.CW., Ali, M. (eds) Advances and Trends in Artificial Intelligence. From Theory to Practice. IEA/AIE 2021. Lecture Notes in Computer Science(), vol 12799. Springer, Cham. https://doi.org/10.1007/978-3-030-79463-7_23
Download citation
DOI: https://doi.org/10.1007/978-3-030-79463-7_23
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-79462-0
Online ISBN: 978-3-030-79463-7
eBook Packages: Computer ScienceComputer Science (R0)