Objective Students' classroom behavior analysis is important for assessing students' classroom participation. A new analysis and processing model of students' classroom behavior data is proposed to comprehensively assess students' participation in classroom. Methods Based on classroom video information,Kinect sensor was used to obtain the correlation between limb bones and facial features and observable student behaviors. Relevant features were extracted to construct deep neural network(DNN) classifiers,and different levels of attention concentration were classified. Using the Kinect sensor's limb bones' point information and audio array for multimodal fusion,the frequency of students' putting up hands and answering questions was counted.
Results It was verified that there was a correlation between the level of attention and the specific behavior of the students (the correlation coefficients between level 1 and level 2 and the behavior of watching blackboard were 0.63 and 0.55,respectively; the correlation coefficient between level 3 and the behavior of looking around was 0.78). Using the DNN to classify attention levels,the accuracy rate was 91.2%,which was 12.3% higher than the support vector machine (SVM). The accuracy of student location recognition using audio arrays was 89.0%. Finally,each student's attention level picture for each lesson,the proportion of each attention level,and the number of times of the students' raising their hands in the classroom as well as the number of standing up to answer questions formed a student class participation analysis table. Conclusion By assessing students' classroom behaviors and combining relevant indicators of classroom participation,they can comprehensively and objectively reflect the classroom performance of different students in the classroom,and can be used as a reference for teachers.