Setting Up Automated Programming Assessment System for Higher Education Database Course

AUTHOR(S):

TITLE

Setting Up Automated Programming Assessment System for Higher Education Database Course

ABSTRACT

The need for decreasing the workload among educators, timely feedback and accuracy and consistency on the grading results are the common reasons that motivate the development of Automated Programming Assessment Systems (APAS). Using our newly deployed APAS on Databases course we have evaluated students’ knowledge using a “little and often” pattern and employed gathered data to predict students’ performance on the course. The course’s pass percentage and students’ grades were predicted using several data mining techniques. Besides internal data gathered during course’s execution, additional external variables (like grade point average) were considered for the data mining models. Our analysis shows that the accuracy of prediction is not highly affected if external variables are unknown. The pass percentage accuracy predictions is sufficiently high, especially after the half of semester (~80%), which allows for proactive approach towards the students we believe will fail the course. Other than that, we came up with few valuable insights into the structure and content of assignments that shall be applied in the next course cycle, in accordance with our intention to use course’s data to improve the teaching and optimize the staff’s man-hours.

KEYWORDS

Educational Data Mining, Academic Performance, Prediction, Automated Assessment

REFERENCES

[1] B. Wächter, M. Kelo , Q. Lam, P. Effertz , C. Jost and S. Kottowski, “University Quality Indicators: A Critical Assessment,” 2015.

[2] A. M. Shahiri, H. Wahidah and A. R. Nur’aini, “A Review on Predicting Student’s Performance using Data Mining Techniques,” in The Third Information Systems International Conference, Surabaya, Indonesia, 2015.

[3] G. W. Dekker, M. Pechenizkiy and J. M. Vleeshouwers, “Predicting students drop out : a case study,” in Proceedings of the 2nd International Conference on Educational Data Mining, EDM, Cordoba, Spain, 2009.

[4] C. Romero, S. Ventura, C. Hervás and P. G. Espejo, “Data Mining Algorithms to Classify Students,” in Proceedings of the 1st International Conference on, Proceedings of the 1st International Conference on, 2008.

[5] B. Minaei-Bidgoli and W. F. Punch, “Using Genetic Algorithms for Data Mining Using Genetic Algorithms for Data Mining,” in Genetic and Evolutionary, 2003.

[6] J. F. Superby, J. P. Vandamme and N. Meskens, “Determination of Factors Influencing the Achievement of the First-year University Students using Data Mining Methods,” in Workshop on Educational Data Mining, 2006.

[7] L. V. Fausett and W. Elwasif, “Predicting Performance from Test Scores using Backpropagation and Counterpropagation,” in IEEE World Congress on Computational Intelligence, Orlando, FL, USA, 1994.

[8] S. B. Kotsiantis and P. E. Pintelas, “Predicting Students Marks in Hellenic Open University,” in Fifth IEEE International Conference on Advanced Learning Technologies (ICALT'05), Kaohsiung, Taiwan, 2005.

[9] M. D. Calvo-Flores, E. Gibaja, M. C. Pegalajar Jiménez and O. Pérez, “Predicting Students’ Marks from from Moodle logs using neural network models,” in Current Developments in Technology-Assisted Education, 2006.

[10] J. English and T. English, “Experiences of Using Automated Assessment in,” Journal of Information Technology Education: Innovations in Practice, vol. 14, pp. 237-254, 2015.

[11] R. S. Pettit, J. Homer, K. Holcomb, S. Nevan and S. Mengel, "Are automated assessment tools helpful in programming courses?," in 122nd ASEE Anual Conference & Exposition, Seattle, Washington, 2015.

[12] M. Hull, D. Powell and E. Klein, “Infandago: Automated Grading for Student Programming,” in Proceedings, Darmstadt, Germany. June, 2011.

[13] V. Pieterse, “Automated Assessment of Programming Assignments,” in Proceedings of the 3rd Computer Science Education Research Conference on Computer Science Education Research, Arnhem, Netherlands — April 04 - 05, 2013 , 2013.

[14] R. Singh, S. Gulwani and A. Solar-Le, “Automated feedback generation for introductory programming assignments,” in Proceedings of the 34th ACM SIGPLAN conference on, 2013.

[15] R. Romli, S. Sulaiman and K. Z. Zamlic, “Improving Automated Programming Assessments User Experience Evaluation Using FaSt-generator,” in The Third Information Systems International Conference, 2015.

[16] W. Siever, "Automated assessment in data structures: a summary of experience," Journal of Computing Sciences in Colleges, vol. 30, no. 2, pp. 17-22, 2014.

[17] M.-J. Laakso, T. Salakoski, A. Korhonen, and L. Malmi, “Automatic Assessment of Exercises for Algorithms and Data Structures – a Case Study with TRAKLA2,” in Proceedings of 4th Finnish/Baltic Sea Conference on Computer Science Education, 2004.

[18] J. C. Prior and R. Lister, "The Backwash Effect on SQL Skills Grading," in Proceedings of the 9th annual SIGCSE conference on Innovation and technology in computer science education, Leeds, United Kngdm — June 28 - 30, 2004 , 2004.

[19] L. Baumstark and E. Rudolph, “Automated online grading for virtual machine-based systems administration courses,” in Proceeding of the 44th ACM technical symposium on Computer science education, Denver, Colorado, USA, 2013.

[20] A. McAllister and M. Yu Feng , "Automated grading of student-designed GUI programs," in World Transactions on Engineering and Technology Education, 2005.

[21] J. English, “Automated assessment of GUI programs using JEWL,” in Proceedings of the 9th Annual SIGCSE Conference on Innovation and Technology in Computer Science Education, Leeds, UK, 2004.

[22] V. Vachharajani, J. Pareek and S. Gulabani, “Effective Label Matching for Automatic Evaluation of Use -- Case Diagrams,” in 2012 IEEE Fourth International Conference on Technology for Education (T4E),, Hyderabad, India, 2012.

[23] P. G. Thomas, "Online Automatic Marking of Diagrams," Systemic Practice and Action Research ·, 2013.

[24] B. Chandra, M. Joseph, B. Radhakrishnan, S. Acharya and S. Sudarshan, “Partial marking for automated grading of SQL queries,” Proceedings of the VLDB Endowment, vol. 9, no. 13, pp. 1541-1544, 2016.

[25] S. Dekeyser, M. De Raadt and T. Yu Lee, “Computer Assisted Assessment of SQL Query Skills,” in Eighteenth Australasian Database Conference (ADC2007), Ballarat, Australia, 2007.

[26] "The R Project for Statistical Computing," [Online]. Available: https://www.r-project.org/. [Accessed 2017].

[27] “rpart: Recursive Partitioning and Regression Trees,” [Online]. Available: https://cran.r-project.org/web/packages/rpart/index.html. [Accessed 2017].

[28] D. Krstajic, . L. J. Buturovic, D. E. Leahy and S. Thomas, "Cross-validation pitfalls when selecting and assessing regression and classification models.," Journal of Cheminformatics, 2014.

Cite this paper

Igor Mekterovic, Ljiljana Brkic. (2017) Setting Up Automated Programming Assessment System for Higher Education Database Course. International Journal of Education and Learning Systems, 2, 287-294