research-article

Open access

Research on the Construction of Curriculum Instruction Effect Evaluation Based on BP Neural Network Model

Authors: Yingying Liu, Hongyun Zhang, Xiaolan HeAuthors Info & Claims

ICIIP '24: Proceedings of the 2024 9th International Conference on Intelligent Information Processing

Pages 402 - 407

https://doi.org/10.1145/3696952.3697007

Published: 21 November 2024 Publication History

All formats PDF

Abstract

Although there is no distinctive header, this is the abstract. This submission template allows authors to submit their papers for review to an ACM Conference or Journal without any output design specifications incorporated at this point in the process. The ACM manuscript template is a single column document that allows authors to type their content into the pre-existing set of paragraph formatting styles applied to the sample placeholder text here. Throughout the document you will find further instructions on how to format your text. If your conference's review process will be double-blind: The submitted document should not include author information and should not include acknowledgments, citations or discussion of related work that would make the authorship apparent. Submissions containing author identifying information may be subject to rejection without review. Upon acceptance, the author and affiliation information must be added to your paper.

1 Introduction

Objective and accurate measurement of instruction effect is one of the main means to evaluate the degree of achievement of teaching objectives and improve teaching quality. Because the development of information technology, cluster analysis, grey system, fuzzy comprehensive evaluation and other methods have been widely used in the evaluation of instruction quality and effectiveness. Fan Sheng et al. (2022) used K-Means ++ algorithm for cluster analysis, which improved the accuracy of course data analysis, efficiency and objectivity of evaluation. Xu Weiguo and Wu Gang (2010) built a grey correlation assessment model of instruction quality in military colleges according to the actual characteristics of teaching work in military colleges, and the case analysis shows that the assessment model built can more accurately reflect the teaching quality. Cao Qiang & Yu Wenmei (2018) proposed a way to improve the instruction effect of experimental courses by using the fuzzy comprehensive evaluation method under the analytic hierarchy process. However, these methods have the problem that human factors have great influence.

BP neural network refers to a multi-layer feed-through neural network based on error back propagation algorithm. It was proposed by Rumelhart and McCelland in 1986 and is also the most widely used model among dozens of neural network models. BP neural network is an effective machine learning tool, which can effectively solve the nonlinear comprehensive evaluation problem, reduce the influence of human factors on the decision-making result, and help us more accurately measure and evaluate the learning effect of students, and improve the quality of education and instruction. Lapedes and Farber(1987) are the first scholars to apply neural networks to the field of prediction. Bai L &Guo XX (2015) applied BP neural network algorithm to evaluate education quality. Luo Juchuan and Qing Yanmei (2013) used BP neural network to build a computer graphics teaching quality evaluation model and carried out an evaluation of the actual teaching situation. In order to effectively and accurately evaluate instruction quality, Yue Qi & Wen Xin (2018) proposed a hybrid intelligent algorithm based on genetic algorithm (GA) and backpropagation neural network (BPNN) for evaluating instruction quality, and verified that the constructed model can effectively realize the evaluation of teaching quality. By inputting enough samples for learning and training, BP neural network can overcome the problem that instruction effect evaluation is affected by supervisor factors and make the evaluation of teaching effect more objective and accurate. At present, there are abundant researches on the evaluation of instruction quality using BP neural network, while there are insufficient researches on the evaluation of curriculum instruction effect. This study uses BP neural network to construct an evaluation model of curriculum instruction effect, and on the basis of correct and objective evaluation of curriculum instruction effect, finds out existing problems and provides ideas for future teaching reform.

2 Construction of curriculum instruction effect evaluation index system

2.1 Contents of Curriculum Instruction Effect Evaluation

The evaluation of curriculum instruction effect is a very key education management work, which is a comprehensive evaluation of teaching process and effect. Wang Yongmei and Cheng Lei (2020) proposed that the evaluation of curriculum instruction effect should include three aspects: implementation mode, content design and students' concept. Mao Linglin & He Xin (2020) proposed that the evaluation of course teaching effect includes two parts: process evaluation and summative evaluation. Yu Wenyan & Luo Jiayun (2014) proposed that the evaluation of curriculum instruction effect should be based on three aspects: exam results, evaluation system and student research. Curriculum instruction effect evaluation is an important part of teaching quality evaluation, but different from the content of teaching quality evaluation, it is generally believed that teaching effect refers to the achievement level of students. However, based on the domestic requirements for college students' quality education, on the basis of comprehensive analysis of the influencing factors of curriculum instruction effect evaluation and the characteristics of teaching reform, This research defines the content of curriculum instruction effect evaluation as four aspects: students' listening effect, teachers' instruction effect, students' learning performance and students' thinking and innovation ability.

2.2 Curriculum Instruction Effect Evaluation Index System

This research established an index system for comprehensive evaluation of curriculum instruction effect: students' listening effect, teachers' instruction effect, students' learning performance and students' thinking and innovation ability were taken as first-level indicators, and these four first-level indicators were further refined into 20 second-level indicators (Table 1). The second-level indicators were scored using a five-level Likert scale, including very consistent, consistent, general, inconsistent and very inconsistent, which were recorded as 5, 4, 3, 2 and 1 points respectively.

Table 1:

First-level Index	Second-level Index	Index Interpretation
Students' Listening Effect	Learning interest ( x₁)	Students' interest in learning has increased.
	Learning attitude (x₂)	Students have a positive attitude towards learning.
	Knowledge acquisition (x₃)	Students think they have mastered what they have learned.
	Practical operation (x₄)	Students feel that they have mastered the operational skills they have learned.
	Self-evaluation (x₅)	Students' self-evaluation has improved.
	Classroom experience (x₆)	Good class experience.
Teachers' Instruction Effect	Classroom atmosphere (x₇)	The classroom atmosphere is good in the teaching activities.
	Teacher-student interaction (x₈)	There is a good and positive interaction between them.
	Class participation (x₉)	There is a high level of classroom participation in teaching activities.
	Instructional objective (x₁₀)	The teaching objectives are clearly understood.
	Instructional content (x₁₁)	The teaching content is well-detailed and inspiring.
	Instructional design (x₁₂)	Instructional design enhances the student experience.
	Instructional method (x₁₃)	The teaching methods are rich and effective.
	Teaching satisfaction (x₁₄)	Students are highly satisfied with the teaching activities.
Students' Learning Performance	Process assessment score (x₁₅)	Good results have been achieved in the process assessment.
Students' Learning Performance	Summary assessment score (x₁₆)	Good results were achieved in the summative assessment.
Students' Thinking and InnovationAbility	Independent learning ability (x₁₇)	Students' ability to actively acquire knowledge, skills and experience increases.
	Teamwork ability (x₁₈)	Students' ability to participate in teamwork, share ideas, respect others, and solve problems together is enhanced.
	Problem solving skills (x₁₉)	Students' ability to analyze problems from different perspectives, locate key factors, and propose appropriate solutions improves.
	Flexibility (x₂₀)	Students' ability to adapt and change existing ideas and strategies in different situations is improved.

Table 1: Evaluation Index of Curriculum Instruction Effect

2.3 Curriculum Instruction Effect Evaluation Establishment Steps

The steps of establishing the evaluation model of curriculum instruction effect are as follows:

(1) Establish an evaluation index system through analysis and research.

(2) Collect evaluation sample data and pre-process the data.

(3) Determine various parameters of the BP neural network algorithm, including the number of hidden layer neuron nodes, activation function, loss function, learning rate, training frequency, etc.

(4) Through the input of training samples into the evaluation model, continuous iterative training is carried out until the trigger litigation stops.

(5) Input validation samples to test the training effect of BP neural network model. If the result reaches the requirement, stop and proceed to the next step; otherwise, return to step (3) and train the network again.

(6) Input test samples to get evaluation results.

Figure 1:

3 Evaluation model construction

3.1 Prepare the Data

This study collected data through a questionnaire. The questionnaire items were 20 secondary level indicators constructed, and a five-level Likert scale was used for scoring, including very consistent, consistent, average, inconsistent and very inconsistent, which were recorded as 5, 4, 3, 2 and 1 points respectively. A questionnaire survey was conducted on 140 students studying Hotel Operation Management in Wuhan Business School. During the curriculum of study, a total of 10 surveys were conducted, with 1 sample for each questionnaire, and 1400 sample data were obtained. In order to solve the problem of different data dimensions, the collected data is normalized to the interval (0,1]. According to the literature research, the maximum and minimum value method was used to normalize the data in this study, and the formula is as follows:

\begin{equation} {X}_n = \frac{{{x}_n - {x}_{n\min }}}{{{x}_{n\max } - {x}_{n\min }}} \end{equation}

(1)

The processed data is segmented, and 70% (980) of the 1400 data are randomly selected as the training set, 15% (210) as the verification data, and the other 15% (210) as the test set.

After data normalization, this study uses entropy method to determine the weight coefficient of each secondary index. Firstly, the contribution degree of the ith student under the jth index is calculated by formula (2); Secondly, formula (3) is to calculate the entropy value of the jth index; formula (4) is to calculate the difference coefficient; Finally, the weight of the jth index is determined by formula (5).

\begin{equation} {P}_{ij} = \frac{{{Z}_{ij}}}{{\sum\limits_{i = 1}^n {{Z}_{ij}} }} \end{equation}

(2)

\begin{equation} {{\rm{e}}}_j = - \frac{1}{{{\rm{In }}n}}\sum\limits_{i = 1}^n {{P}_{ij}} {\rm{In}}({P}_{ij}) \end{equation}

(3)

\begin{equation} {{\rm{g}}}_j = 1 - {e}_j \end{equation}

(4)

\begin{equation} {W}_j = \frac{{{g}_j}}{{\sum\limits_{j = 1}^m {{g}_j} }} \end{equation}

(5)

The results are shown in Table 2.

Table 2:

Secondary index	Weight coefficient	Secondary index	Weight coefficient	Secondary index	Weight coefficient	Secondary index	Weight coefficient
X₁	0.0414	X₆	0.0391	X₁₁	0.0512	X₁₆	0.0604
X₂	0.0556	X₇	0.0401	X₁₂	0.0497	X₁₇	0.0578
X₃	0.0496	X₈	0.0551	X₁₃	0.0458	X₁₈	0.0496
X₄	0.0476	X₉	0.0457	X₁₄	0.0487	X₁₉	0.0588
X₅	0.0489	X₁₀	0.0414	X₁₅	0.0569	X₂₀	0.0462

Table 2: Weight Coefficient of Secondary Index

3.2 Construction of BP Neural Network Model

BP neural network is composed of three layers: input layer, hidden layer and output layer (Figure 2). The number of neurons and hidden layer should be adjusted according to the actual situation. A reasonable structure can reduce the number of training times and improve the accuracy of curriculum instruction effect evaluation. The basic BP algorithm includes two aspects: Forward propagation of the signal and back propagation of the error, that is, calculating the actual output in the direction from input to output, The correction of weights and thresholds is carried out from the direction of output to input. In the reverse propagation of the error, the output error of each layer of neurons is calculated layer by layer starting from the output layer, and then the weight and threshold of each layer are adjusted according to the error gradient descent method, so that the final output of the modified network can be close to the expected value.

Figure 2:

The formula for each layer is:

\begin{equation} y = T\left( {wx + b} \right) \end{equation}

(6)

T stands for activation function, b stands for threshold, and w stands for connection weight. Output of the jth node of the hidden layer is:

\begin{equation} {h}_j = f(ne{t}_j),j = 1,2,3, \cdots ,z \end{equation}

(7)

\begin{equation} ne{t}_j = \sum\limits_{i = 1}^n {{w}_{ij}{x}_i + {b}_j} ,j = 1,2,3, \cdots ,z \end{equation}

(8)

Namely,

\begin{equation} {h}_j = f\left( {\sum\limits_{i = 1}^n {{w}_{ij}{x}_i + {b}_j} } \right) \end{equation}

(9)

Where j=1,2....z indicates the index of the hidden layer node. xi is the value of the input node i, and wij and bj are the weights and thresholds from the input node i to the hidden node j, respectively.

Output of the kth node of the output layer is:

\begin{equation} {o}_k = f(ne{t}_k),k = 1,2,3, \cdots ,m \end{equation}

(10)

\begin{equation} ne{t}_k = \sum\limits_{j = 1}^z {{w}_{jk}{h}_j + {b}_k} ,k = 1,2,3, \cdots ,m \end{equation}

(11)

Namely,

\begin{equation} {o}_k = f\left( {\sum\limits_{j = 1}^z {{w}_{jk}{h}_j + {b}_k} } \right) \end{equation}

(12)

Where k=1,2....m indicates the index of the node in the output layer. hj is the value of the hidden node j, and wjk and bk are the weights and thresholds from the hidden node j to the output node k, respectively.

The significance of the activation function for neural network models is that it is not limited to fitting linear functions, but can fit nonlinear functions. In the early days, the Sigmoid function was mostly used as the activation function in neural network models, but it has been gradually replaced by the ReLU (Rectigied Linear Units) function and its variants in modern deep learning practice. According to the needs of this study, the ReLU function is selected as the activation function, so that the network can be trained directly, instead of a large number of pre-training to prevent the problem of gradient vanishing.

\begin{equation} f(x) = \max (0,x) \end{equation}

(13)

The graph is shown in Figure 3.

Figure 3:

The loss function is used to evaluate the gap between the predicted and true values of the model. In this study, the mean square error loss function is used (14):

\begin{equation} MSE = \frac{{\sum\limits_{i = 1}^n {{{\left( {f(x) - y} \right)}}^2} }}{n} \end{equation}

(14)

The error is backpropagated by gradient descent method, and the connection weights and thresholds of each layer are corrected. The training is not stopped until the required or target training times are reached.

\begin{equation} {w}_{n^{'}} = {w}_n - \alpha \frac{{\partial MSE}}{{\partial w}} \end{equation}

(15)

\begin{equation} {b}_{n^{'}} = {b}_n - \alpha \frac{{\partial MSE}}{{\partial b}} \end{equation}

(16)

Where, bn is the threshold of each layer, wn is the connection weight of each layer, bn’ is the correction value of the threshold, wn' is the correction value of the connection weight, α is the learning rate.

BP algorithms with different network structures have different problem-solving abilities. The more complex the structure, the stronger the ability to solve nonlinear problems, but the longer the training time, and the too simple structure will make it difficult to converge the network training. Appropriate selection of the number of units can achieve the purpose of training the network well and completing it efficiently. In 1989, Robert Hechi-Nielson proposed Kolmogorov's theorem and gave a proof that any continuous function in a closed interval can be approximated by a BP neural network with a hidden layer, so that a three-layer BP neural network can complete any n-dimensional to m-dimensional mapping. Shih-Chi Huang and Yih-FangHuang (1991) also show that a three-layer neural network with finite hidden layer elements can approximate a continuous function with arbitrary precision In this study, the number of nodes in the input layer is determined by the data of 20 secondary indicators in the index system, and the number of output nodes is only the final result of the effect evaluation. The comprehensive evaluation set = {excellent, good, medium, qualified, unqualified} is divided into 5 levels, and the corresponding output value interval set = {[0.9-1], [0.8-0.9]. [0.7-0.8), [0.6-0.7), [0-0.6)}. In addition, in the curriculum instruction effect evaluation model based on BP neural network, the empirical formula below is used in this study to estimate the number of hidden layer nodes.

\begin{equation} L \le \sqrt {n\left( {m + 3} \right)} + 1 \end{equation}

(17)

Where n and m are the number of nodes in the input layer and the output layer respectively. In this study, n is 20, m is 1, so the number of nodes in the hidden layer is 9, and the 20-9-1 three-layer BP network structure is established. Its common matrix form is:

\begin{equation} f\left( x \right) = {W}^{\left( o \right)}{\mathop{\rm Re}\nolimits} LU\left( {{W}^hx + {b}^{\left( h \right)}} \right) + {b}^{\left( o \right)} \end{equation}

(18)

Here W is the weight matrix, x and b are vectors, and superscripts (o) and (h) represent the output layer and the hidden layer, respectively.

3.3 Model Training and Verification

The purpose of model training is to use the processed data to train BP neural network, so that the network can build a nonlinear mapping relationship between the evaluation index of course teaching effect and its corresponding evaluation level. In the process of model training, the learning rate is set to 0.001, and the training times are 2000 times. And 210 sets of validation data were used to monitor whether the training was heading towards overfitting. By comparing the variation trend of training error and validation error, it is found that no overfitting occurs.

4 Simulation prediction and results

For the trained network, the data of the test group was imported for prediction, and implemented with the help of Neural Net Fitting in Matlab software, the error between the predicted value and the true value was within 0.3, and the evaluation level was consistent. The experimental results were satisfactory. Some of the results of the predicted value and the true value were shown in Table 3. Therefore, the established teaching data analysis model is reasonable and can accurately reflect the teaching effect of the course.

Table 3:

Test sample	True Value	Evaluation Grade	Predicted Value	Simulation Grade	Relative error (%)
1	0.88	Good	0.8785	Good	0.17
2	0.92	Excellent	0.9133	Excellent	0.73
3	0.95	Excellent	0.9425	Excellent	0.79
4	0.92	Excellent	0.9211	Excellent	0.12
5	0.73	Medium	0.7455	Medium	2.12
6	0.86	Good	0.8598	Good	0.02
7	0.77	Medium	0.7459	Medium	1.83
8	0.81	Good	0.8192	Good	1.14
9	0.87	Good	0.8479	Good	2.54
10	0.97	Excellent	0.9583	Excellent	1.21
11	0.68	Qualified	0.6758	Qualified	0.62
12	0.93	Excellent	0.9581	Excellent	2.16
13	0.84	Good	0.8198	Good	2.40
14	0.81	Good	0.8159	Good	0.73
15	0.93	Excellent	0.9526	Excellent	2.43

Table 3: Part Evaluation Result of BP Neural Network Model on Curriculum Instruction Effect

5 Conclusion

To improve the evaluation level of curriculum instruction effect is a link to improve education quality. Aiming at the actual situation of college curriculum instruction and the requirements of higher education development for teaching quality evaluation, this study introduced the theory of artificial neural network into teaching effect evaluation, integrated complex indicators and quantified them, and established a curriculum instruction effect evaluation system from four aspects: students' listening effect, teachers' instruction effect, students' learning performance, students' thinking and innovation ability. It is an application innovation of big data computing in teaching quality management to establish the evaluation model of curriculum instruction effect with BP neural network and use the trained model to evaluate the curriculum instruction effect.

BP neural network has a strong learning ability, and the evaluation model of curriculum instruction effect based on BP neural network can overcome the shortcomings of traditional evaluation methods, and significantly reduce the influence of human factors. From the training process, when the number of samples is small, the test accuracy is not high, and when the number of samples increases to more than 500, the accuracy can reach more than 94%. Therefore, the test results of this study show that the constructed evaluation model has a high accuracy rate and can meet the requirements of course teaching effect evaluation.

Acknowledgments

This paper is the research result of the project "Hotel Operation Management Major Core Curriculum Construction" approved by Wuhan Business University.

References

[1]

Bai L, Guo XX. The model of evaluating teaching quality based on BP Neural Network Algorithm[J]. Appplied Mechaincs & Materials, 2015,719-720:321-323.

Abstract

1 Introduction

2 Construction of curriculum instruction effect evaluation index system

2.1 Contents of Curriculum Instruction Effect Evaluation

2.2 Curriculum Instruction Effect Evaluation Index System

2.3 Curriculum Instruction Effect Evaluation Establishment Steps

3 Evaluation model construction

3.1 Prepare the Data

3.2 Construction of BP Neural Network Model

3.3 Model Training and Verification

4 Simulation prediction and results

5 Conclusion

Acknowledgments

References

Index Terms

Recommendations

BP Neural Network-Based Research of University Science Research Capability Evaluation

Research on sales forecasting based on ARIMA and BP neural network combined model

Research of Population Prediction Based on GA-BP Neural Network

Comments

Information

Published In

Publisher

Publication History

Check for updates

Author Tags

Qualifiers

Conference

Acceptance Rates

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

View options

PDF

eReader

HTML Format

Login options

Full Access

Figures

Other

Share

Share this Publication link

Share on social media

Affiliations