||Applying learning analytics to explore the effects of programming learning process on computational thinking ability
||Department of Engineering Science
programming learning process
Computational thinking (CT) ability is becoming increasingly critical. Training students to integrate computational thinking ability is also a significant issue facing our current education system. The concepts and principles of computational thinking are very abstract and difficult to understand; besides, traditional programming has a high initial learning threshold; many students find it difficult to learn, and it can easily cause fear and disinterest. In addition, current computational thinking practices generally use quantitative methods to evaluate effectiveness and impact such as computational thinking tests, scales, questionnaires, and project evaluations and thus measure students' computational thinking. After our investigation, we found that current research is less focused on qualitative research methods such as the learning process or case studies in applied behavior analysis. Therefore, this study aims to explore the effects of the programming learning process on computational thinking ability. To this end, this study builds a visual programming computational thinking learning platform and designs an 18-week computational thinking programming learning course to train learners' computational thinking ability. In addition, we developed a real-time behavior tracking record module to collect learners' programming learning processes and behaviors in learning activities; meanwhile, we applied different learning analytics to analyze the collected data and thus the learning behavior patterns of diverse learners.
Table of Contents IV
List of Tables VI
List of Figures VII
Chapter 1 Introduction 1
1.1. Background and Motivations 1
1.2. The purpose of this study 6
1.3. Research questions 7
1.4. The Limitations of Research 7
1.5. The structure of doctoral dissertation 8
Chapter 2 Literature Review 9
2.1. Computational thinking 9
2.2. Assessing of computational thinking 13
2.3. Training computational thinking through programming 17
2.4. Learning behavior analytics 19
Chapter 3 Research Methodology 23
3.1. System development 23
3.2. Research instruments and environment 29
3.3. Experiment Design and Procedure 33
Chapter 4 Experiment Results 39
4.1. Results of computational thinking ability 39
4.2. Results of Visual-based Computational Thinking Test (VCTT) 42
4.3. Results of MSLQ, CACQ and FCTQ 44
4.4. The result of correlation analysis 46
4.5. The result of learning analytics 48
Chapter 5 Discussion and Conclusions 52
Abdul-Rahman, S.-S., & Du Boulay, B. (2014). Learning programming via worked-examples: Relation of learning styles to cognitive load. Computers in Human Behavior, 30, 286-298.
Alammary, A. (2019). Blended learning models for introductory programming courses: A systematic review. PloS one, 14(9), e0221765.
Ali, L., Hatala, M., Gašević, D., & Jovanović, J. (2012). A qualitative evaluation of evolution of a learning analytics tool. Computers & Education, 58(1), 470-489.
Atmatzidou, S., & Demetriadis, S. (2016). Advancing students’ computational thinking skills through educational robotics: A study on age and gender relevant differences. Robotics and Autonomous Systems, 75, 661-670.
Barr, D., Harrison, J., & Conery, L. (2011). Computational thinking: A digital age skill for everyone. Learning & Leading with Technology, 38(6), 20-23.
Barr, V., & Stephenson, C. (2011). Bringing computational thinking to K-12: what is Involved and what is the role of the computer science education community? Acm Inroads, 2(1), 48-54.
Bebras International Challenge on Informatics and Computational Thinking. (2015). Retrieved from http://www.bebras.org
Bower, M., Wood, L. N., Lai, J. W., Howe, C., Lister, R., Mason, R., . . . Veal, J. (2017). Improving the computational thinking pedagogical capabilities of school teachers. Australian Journal of Teacher Education, 42(3), 4.
Brennan, K., & Resnick, M. (2012a). New frameworks for studying and assessing the development of computational thinking. Paper presented at the Proceedings of the 2012 annual meeting of the American Educational Research Association, Vancouver, Canada.
Brennan, K., & Resnick, M. (2012b). New frameworks for studying and assessing the development of computational thinking. Paper presented at the Proceedings of the 2012 annual meeting of the American Educational Research Association, Vancouver, Canada.
Buitrago Flórez, F., Casallas, R., Hernández, M., Reyes, A., Restrepo, S., & Danies, G. (2017). Changing a Generation’s Way of Thinking: Teaching Computational Thinking Through Programming. Review of Educational Research, 0034654317710096.
Campbell, J. P., DeBlois, P. B., & Oblinger, D. G. (2007). Academic analytics: A new tool for a new era. EDUCAUSE review, 42(4), 40.
Carlisle, M. C., Wilson, T., Humphries, J., & Hadfield, S. (2009). Raptor: a visual programming environment for teaching object-oriented programming. Journal of Computing Sciences in Colleges, 24(4), 275-281.
Chalmers, C. (2018). Robotics and computational thinking in primary school. International Journal of Child-Computer Interaction, 17, 93-100.
Chang, C.-K. (2014). Effects of using Alice and Scratch in an introductory programming course for corrective instruction. Journal of Educational Computing Research, 51(2), 185-204.
Chen, G., Shen, J., Barth-Cohen, L., Jiang, S., Huang, X., & Eltoukhy, M. (2017). Assessing elementary students’ computational thinking in everyday reasoning and robotics programming. Computers & Education, 109, 162-175.
Cheng, K. H., & Tsai, C. C. (2016). The interaction of child–parent shared reading with an augmented reality (AR) picture book and parents' conceptions of AR learning. British Journal of Educational Technology, 47(1), 203-222.
Clow, D. (2012). The learning analytics cycle: closing the loop effectively.
Council, N. R. (2011). Report of a workshop on the pedagogical aspects of computational thinking: National Academies Press.
Cutumisu, M., Adams, C., & Lu, C. (2019). A Scoping Review of Empirical Research on Recent Computational Thinking Assessments. Journal of Science Education and Technology, 28(6), 651-676.
Denning, P. J. (2009). The profession of IT Beyond computational thinking. Communications of the ACM, 52(6), 28-30.
Doleck, T., Bazelais, P., Lemay, D. J., Saxena, A., & Basnet, R. B. (2017). Algorithmic thinking, cooperativity, creativity, critical thinking, and problem solving: exploring the relationship between computational thinking skills and academic performance. Journal of Computers in Education, 4(4), 355-369.
Esteves, M., Fonseca, B., Morgado, L., & Martins, P. (2011). Improving teaching and learning of computer programming through the use of the Second Life virtual world. British Journal of Educational Technology, 42(4), 624-637.
Filvà, D. A., Forment, M. A., García-Peñalvo, F. J., Escudero, D. F., & Casañ, M. J. (2019). Clickstream for learning analytics to assess students’ behavior with Scratch. Future Generation Computer Systems, 93, 673-686.
García-Peñalvo, F. J. (2018). Editorial computational thinking. IEEE Revista Iberoamericana de Tecnologias del Aprendizaje, 13(1), 17-19.
González, M. R. (2015). Computational thinking test: Design guidelines and content validation. Paper presented at the Proceedings of EDULEARN15 Conference.
Gretter, S., & Yadav, A. (2016). Computational thinking and media & information literacy: An integrated approach to teaching twenty-first century skills. TechTrends, 60(5), 510-516.
Grover, S. (2017). Assessing algorithmic and computational thinking in K-12: Lessons from a middle school classroom. In Emerging research, practice, and policy on computational thinking (pp. 269-288): Springer.
Grover, S., & Pea, R. (2013). Computational thinking in K–12: A review of the state of the field. Educational researcher, 42(1), 38-43.
Grover, S., & Pea, R. (2018). Computational Thinking: A competency whose time has come. Computer science education: Perspectives on teaching and learning in school, 19.
Guzdial, M. (2008). Education Paving the way for computational thinking. Communications of the ACM, 51(8), 25-27.
Guzdial, M., & Morrison, B. (2016). Growing computer science education into a STEM education discipline. Communications of the ACM, 59(11), 31-33.
Hadad, R., Thomas, K., Kachovska, M., & Yin, Y. (2019). Practicing Formative Assessment for Computational Thinking in Making Environments. Journal of Science Education and Technology, 1-12.
Hatley, L., Winston-Proctor, C. E., Paige, G. M., & Clark, K. (2017). CULTURE AND COMPUTATIONAL THINKING. Culture, Learning, and Technology: Research and Practice, 109.
Heintz, F., Mannila, L., & Färnqvist, T. (2016). A review of models for introducing computational thinking, computer science and computing in K-12 education. Paper presented at the 2016 IEEE Frontiers in Education conference (FIE).
Hou, H.-T. (2012). Exploring the behavioral patterns of learners in an educational massively multiple online role-playing game (MMORPG). Computers & Education, 58(4), 1225-1233.
Hsu, T.-C., Chang, S.-C., & Hung, Y.-T. (2018). How to learn and how to teach computational thinking: Suggestions based on a review of the literature. Computers & Education, 126, 296-310.
Hu, C. (2011). Computational thinking: what it might mean and what we might do about it. Paper presented at the Proceedings of the 16th annual joint conference on Innovation and technology in computer science education.
Jaipal-Jamani, K., & Angeli, C. (2017). Effect of robotics on elementary preservice teachers’ self-efficacy, science learning, and computational thinking. Journal of Science Education and Technology, 26(2), 175-192.
Jona, K., Wilensky, U., Trouille, L., Horn, M., Orton, K., Weintrop, D., & Beheshti, E. (2014). Embedding computational thinking in science, technology, engineering, and math (CT-STEM). Paper presented at the future directions in computer science education summit meeting, Orlando, FL.
Kalelioğlu, F. (2015). A new way of teaching programming skills to K-12 students: Code. org. Computers in Human Behavior, 52, 200-210.
Koh, K. H., Basawapatna, A., Bennett, V., & Repenning, A. (2010). Towards the automatic recognition of computational thinking for adaptive visual language learning. Paper presented at the 2010 IEEE Symposium on Visual Languages and Human-Centric Computing.
Koh, K. H., Basawapatna, A., Nickerson, H., & Repenning, A. (2014). Real time assessment of computational thinking. Paper presented at the 2014 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC).
Korkmaz, Ö., Cakir, R., & Özden, M. Y. (2017). A validity and reliability study of the computational thinking scales (CTS). Computers in Human Behavior, 72, 558-569.
Kucuk, S., & Sisman, B. (2017). Behavioral patterns of elementary students and teachers in one-to-one robotics instruction. Computers & Education, 111, 31-43.
Lämsä, J., Hämäläinen, R., Koskinen, P., Viiri, J., & Mannonen, J. (2020). The potential of temporal analysis: Combining log data and lag sequential analysis to investigate temporal differences between scaffolded and non-scaffolded group inquiry-based learning processes. Computers & Education, 143, 103674.
Lau, W. W., & Yuen, A. H. (2009). Exploring the effects of gender and learning styles on computer programming performance: implications for programming pedagogy. British Journal of Educational Technology, 40(4), 696-712.
Levine, T., & Donitsa-Schmidt, S. (1998). Computer use, confidence, attitudes, and knowledge: A causal analysis. Computers in Human Behavior, 14(1), 125-146.
Liu, M.-C., & Huang, Y.-M. (2017). The use of data science for education: The case of social-emotional learning. Smart Learning Environments, 4(1), 1.
Lu, O. H., Huang, J. C., Huang, A. Y., & Yang, S. J. (2017). Applying learning analytics for improving students engagement and learning outcomes in an MOOCs enabled collaborative programming course. Interactive Learning Environments, 25(2), 220-234.
Lye, S. Y., & Koh, J. H. L. (2014). Review on teaching and learning of computational thinking through programming: What is next for K-12? Computers in Human Behavior, 41, 51-61.
Matcha, W., Gasevic, D., & Pardo, A. (2019). A Systematic Review of Empirical Studies on Learning Analytics Dashboards: A Self-Regulated Learning Perspective. IEEE Transactions on Learning Technologies.
Mladenović, M., Boljat, I., & Žanko, Ž. (2018). Comparing loops misconceptions in block-based and text-based programming languages at the K-12 level. Education and Information Technologies, 23(4), 1483-1500.
Moreno-León, J., Robles, G., & Román-González, M. (2015). Dr. Scratch: Automatic analysis of scratch projects to assess and foster computational thinking. RED. Revista de Educación a Distancia(46), 1-23.
Mueller, J., Beckett, D., Hennessey, E., & Shodiev, H. (2017). Assessing computational thinking across the curriculum. In Emerging research, practice, and policy on computational thinking (pp. 251-267): Springer.
Nardelli, E. (2019). Do we really need computational thinking? Communications of the ACM, 62(2), 32-35.
Noone, M., & Mooney, A. (2018). Visual and textual programming languages: a systematic review of the literature. Journal of Computers in Education, 5(2), 149-174.
Pala, F. K., & Mıhçı Türker, P. (2019). The effects of different programming trainings on the computational thinking skills. Interactive Learning Environments, 1-11.
Papadakis, S. (2020). Evaluating a game-development approach to teach introductory programming concepts in secondary education. International Journal of Technology Enhanced Learning, 12(2), 127-145.
Papadakis, S., Kalogiannakis, M., & Zaranis, N. (2016). Developing fundamental programming concepts and computational thinking with ScratchJr in preschool education: a case study. International Journal of Mobile Learning and Organisation, 10(3), 187-202.
Papert, S. (1990). Children, computers and powerful ideas. In: New York: Basic Books.
Park, Y., & Jo, I.-H. (2019). Factors that affect the success of learning analytics dashboards. Educational Technology Research and Development, 67(6), 1547-1571.
Pintrich, P. R. (1991). A manual for the use of the Motivated Strategies for Learning Questionnaire (MSLQ).
Rich, P. J., & Hodges, C. B. (2017). Emerging research, practice, and policy on computational thinking: Springer.
Román-González, M., Moreno-León, J., & Robles, G. (2019). Combining assessment tools for a comprehensive evaluation of computational thinking interventions. In Computational thinking education (pp. 79-98): Springer, Singapore.
Román-González, M., Pérez-González, J.-C., & Jiménez-Fernández, C. (2017). Which cognitive abilities underlie computational thinking? Criterion validity of the Computational Thinking Test. Computers in Human Behavior, 72, 678-691.
Sáez-López, J.-M., Román-González, M., & Vázquez-Cano, E. (2016). Visual programming languages integrated across the curriculum in elementary school: A two year case study using “Scratch” in five schools. Computers & Education, 97, 129-141.
Sands, P., Yadav, A., & Good, J. (2018). Computational thinking in K-12: In-service teacher perceptions of computational thinking. In Computational thinking in the STEM disciplines (pp. 151-164): Springer.
Saurabh, S., & Gautam, S. (2019). Modelling and statistical analysis of YouTube's educational videos: A channel Owner's perspective. Computers & Education, 128(1), 145-158.
Sedrakyan, G., Malmberg, J., Verbert, K., Järvelä, S., & Kirschner, P. A. (2018). Linking learning behavior analytics and learning science concepts: designing a learning analytics dashboard for feedback to support learning regulation. Computers in Human Behavior.
Seehorn, D., Carey, S., Fuschetto, B., Lee, I., Moix, D., O'Grady-Cunniff, D., . . . Verno, A. (2011). CSTA K--12 Computer Science Standards: Revised 2011.
Seiter, L., & Foreman, B. (2013). Modeling the learning progressions of computational thinking of primary grade students. Paper presented at the Proceedings of the ninth annual international ACM conference on International computing education research.
Selby, C., & Woollard, J. (2013). Computational thinking: the developing definition.
Sengupta, P., Dickes, A., & Farris, A. (2018). Toward a phenomenology of computational thinking in STEM education. In Computational thinking in the STEM disciplines (pp. 49-72): Springer.
Sengupta, P., Kinnebrew, J. S., Basu, S., Biswas, G., & Clark, D. (2013). Integrating computational thinking with K-12 science education using agent-based computation: A theoretical framework. Education and Information Technologies, 18(2), 351-380.
Shute, V. J., Sun, C., & Asbell-Clarke, J. (2017). Demystifying computational thinking. Educational Research Review, 22, 142-158.
Slisko, J. (2017). Self-Regulated Learning in A General University Course: Design of Learning Tasks, Their Implementation and Measured Cognitive Effects. Journal of European Education, 7(2), 12-24.
So, H.-J., Jong, M. S.-Y., & Liu, C.-C. (2020). Computational thinking education in the Asian Pacific region. In: Springer.
Tang, X., Yin, Y., Lin, Q., Hadad, R., & Zhai, X. (2020). Assessing computational thinking: A systematic review of empirical studies. Computers & Education, 148, 103798.
Terry, M. (2002). Translating learning style theory into developmental education practice: An article based on Gregorc's cognitive learning styles. Journal of College Reading and Learning, 32(2), 154-176.
Voogt, J., Fisser, P., Good, J., Mishra, P., & Yadav, A. (2015). Computational thinking in compulsory education: Towards an agenda for research and practice. Education and Information Technologies, 20(4), 715-728.
Weintrop, D., Beheshti, E., Horn, M., Orton, K., Jona, K., Trouille, L., & Wilensky, U. (2016). Defining computational thinking for mathematics and science classrooms. Journal of Science Education and Technology, 25(1), 127-147.
Wells, J. G. (2019). STEM education: The potential of technology education. In: Council on Technology and Engineering Teacher Education.
Wing, J. M. (2006). Computational thinking. Communications of the ACM, 49(3), 33-35.
Wing, J. M. (2008a). Computational thinking and thinking about computing. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, 366(1881), 3717-3725.
Wing, J. M. (2008b). Computational thinking and thinking about computing. Philosophical transactions of the royal society of London A: mathematical, physical and engineering sciences, 366(1881), 3717-3725.
Wing, J. M., & Stanzione, D. (2016). Progress in computational thinking, and expanding the HPC community. In: ACM New York, NY, USA.
Wu, S.-Y., & Hou, H.-T. (2015). How cognitive styles affect the learning behaviors of online problem-solving based discussion activity: A lag sequential analysis. Journal of Educational Computing Research, 52(2), 277-298.
Yadav, A., Good, J., Voogt, J., & Fisser, P. (2017). Computational thinking as an emerging competence domain. In Competence-based vocational and professional education (pp. 1051-1067): Springer.
Yadav, A., Zhou, N., Mayfield, C., Hambrusch, S., & Korb, J. T. (2011). Introducing computational thinking in education courses. Paper presented at the Proceedings of the 42nd ACM technical symposium on Computer science education.
Yağcı, M. (2019). A valid and reliable tool for examining computational thinking skills. Education and Information Technologies, 24(1), 929-951.
Yang, T.-Y., Brinton, C. G., Joe-Wong, C., & Chiang, M. (2017). Behavior-based grade prediction for MOOCs via time series neural networks. IEEE Journal of Selected Topics in Signal Processing, 11(5), 716-728.
Yang, X., Cheng, P.-Y., Lin, L., Huang, Y. M., & Ren, Y. (2019). Can an Integrated System of Electroencephalography and Virtual Reality Further the Understanding of Relationships Between Attention, Meditation, Flow State, and Creativity? Journal of Educational Computing Research, 57(4), 846-876.
Yin, Y., Hadad, R., Tang, X., & Lin, Q. (2019). Improving and Assessing Computational Thinking in Maker Activities: the Integration with Physics and Engineering Learning. Journal of Science Education and Technology, 1-26.
Zhang, J.-H., Zhang, Y.-X., Zou, Q., & Huang, S. (2018). What learning analytics tells us: Group behavior analysis and individual learning diagnosis based on long-term and large-scale data. Journal of Educational Technology & Society, 21(2), 245-258.
Zhong, B., Wang, Q., Chen, J., & Li, Y. (2016). An exploration of three-dimensional integrated assessment for computational thinking. Journal of Educational Computing Research, 53(4), 562-590.