||Development of a Fuzzy-Based Prior Knowledge Diagnostic System with Multiple Attribute Evaluation Applied in a Bioinformatics Course
||Department of Engineering Science
Fuzzy multi-attribute decision making
Students learn new instructions well by building on relevant prior knowledge as such knowledge affects how instructors and students interact with the learning materials. Moreover, studies have found that good prior knowledge can enable students to attain better learning motivation, comprehension, and performance. This suggests it is important to assist students in obtaining the relevant prior knowledge, as this can enable them to engage meaningfully with the learning materials. Generally, tests are often used to help instructors assess students’ prior knowledge. Nevertheless, conventional testing approaches usually assign only a score to each student, and this may mean that students are unable to realize their own individual weaknesses clearly. To address this problem, previous work has developed a prior knowledge testing and diagnosis (PKT&D) system to assist instructors and students in diagnosing and strengthening prior knowledge. Although, the PKT&D system has shown its effectiveness in helping students improve their learning performance, past experiences of applying this model also reveal the limits of applying it. One of the major problems of applying the PKT&D system is that it does not consider the necessity of multiple attributes when identifying the learning problems of individual students. Moreover, the importance ratios of these attributes may be varied for different educational contexts or if the instructors have different opinions about them. Therefore, this study applied the Efficient Fuzzy Weighted Average (EFWA) technique to develop a Fuzzy Prior Knowledge Test and Diagnosis (FPKT&D) system with a multi-attribute decision making model to deal with these above weaknesses. To demonstrate the usefulness of the FPKT&D system, a quasi-experiment was conducted to evaluate its efficacy with regard to improving teaching and learning performance. Furthermore, an analysis was also performed to investigate how accurately the system can diagnose students’ prior knowledge and thus provide them with appropriate materials to strengthen this.
List of Tables
Chapter 1 Introduction 1
Chapter 2 Related Works 4
2.1 The importance of prior knowledge in learning 4
2.2 Review of the PKD (Prior knowledge Diagnosis) Model 5
2.3 Fuzzy Sets in Multi-attribute Decision Making 8
2.3.1 Introduction of fuzzy set theory 9
2.3.2 Fuzzy MADM method 11
2.4 Application of fuzzy techniques in e-learning 11
Chapter 3 Methodology 13
3.1 Analysis of decision attributes 14
3.2 Fuzzy Prior Knowledge Diagnosis Model 18
3.3 Illustrative example 24
Chapter 4 System Implementation 32
4.1 System architecture and components 32
4.2 System operation procedures 34
Chapter 5 Experiment 38
5.1 Research instruments, measures, and goals 38
5.2 Experimental design, participants, and procedure 39
Chapter 6 Results 43
6.1 Learning motivation survey 43
6.2 Learning attitude survey 45
6.3 Perceived usefulness survey 47
6.4 Pre-test/Post-test evaluation 49
6.5 Interview investigation 51
6.5.1 Perceptions of the instruction 54
6.5.2 Perceptions of the interaction 55
6.5.3 Perceptions of the technology 55
Chapter 7 Diagnosis evaluation of FPKT&D system 57
7.1 Evaluation design 57
7.2 Correctness rate analysis 57
Chapter 8 Conclusions and Suggestions 59
8.1 Contribution of the FPKT&D system to interdisciplinary learning 59
8.2 Further applications of FPKT&D system for educators 61
8.3 Limitations and future work 61
List of Tables
Table 1 Difficulty degree of each test item 14
Table 2 Association between test items and concepts 15
Table 3 Relationships between concepts 16
Table 4 Relationship between students’ answers and test items 17
Table 5 The definitions of membership functions 20
Table 6 Description of EFWA algorithm (Lee and Park, 1997.) 22
Table 7 Illustrative example of the difficulty of each test item 25
Table 8 Illustrative example of the relationships between test items and concepts 25
Table 9 Illustrative example of the relationships among concepts 25
Table 10 Illustrative example of the relationships between students’ answers and test items 26
Table 11 Illustrative example of the relationships among test items, concepts, and the fourth student’s answers 26
Table 12 The input values related to the fourth student’s answers with regard to the second concept 28
Table 13 Major teaching and learning activities in the bioinformatics course 41
Table 14 ANCOVA results of the learning motivation post-test score among the three groups 44
Table 15 The paired t-test results of learning motivation for the three groups of students 44
Table 16 Student attitudes towards learning bioinformatics 46
Table 17 Experiment group students’ perceptions of using the FPKT&D system 48
Table 18 Pre-test ANOVA on knowledge of bioinformatics of the three groups 50
Table 19 The paired t-test results of the learning improvement of the three groups 51
Table 20 Post-test ANOVA on the bioinformatics knowledge of the three groups 51
Table 21 Example interview comments about the three topics 52
Table 22 Evaluation of correctness rate results 58
List of Figures
Figure 1 Membership function of a fuzzy set 9
Figure 2 Triangular membership function 10
Figure 3 The hierarchical structure of the decision problem. 13
Figure 4 The membership functions of the rating level 19
Figure 5 The membership function of relative importance 19
Figure 6 The membership functions of alternatives 21
Figure 7 The resulting membership function 31
Figure 8 The architecture of the FPKT&D system 34
Figure 9 Screenshot of the test-sheet development interface 35
Figure 10 Screenshot of the relevant test items selection interface 36
Figure 11 Screenshot of the assessment results for instructors 36
Figure 12 Screenshot of the testing interface 37
Figure 13 Screenshot of the diagnostic results interface for students 37
Figure 14 Experimental process 42
 Ausubel, D., Novak, J., & Hanesian, H. (1978). Educational Psychology: A Cognitive View (2nd Ed.), New York: Holt, Rinehart & Winston.
 Bai, S. M., & Chen, S. M. (2008a). Automatically constructing concept maps based on fuzzy rules for adaptive learning systems. Expert Systems with Applications, 35(1), 41-49.
 Bai, S.M., & Chen, S. M. (2008b). Evaluating students’ learning achievement using fuzzy membership functions and fuzzy rules. Expert Systems with Applications, 34, 399-410.
 Bai, S. M., & Chen, S. M. (2008c). Automatically constructing grade membership functions of fuzzy rules for students’ evaluation. Expert Systems with Applications, 35(3), 1408-1414.
 Barab, S. A., & Landa, A. (1997). Designing effective interdisciplinary anchors. Educational Leadership, 54(6), 52–55.
 Baas, S., & Kwakernaak, H. (1977). Rating and raking of multiple-aspect alternatives using fuzzy sets. Automatica, 13(1), 47-58.
 Biswas, N. B. (2007). Knowledge and Pedagogy: An Essential Proposition in Response to Teacher Preparation. US-China Education Review, 4(7), 1-14.
 Biswas, R. (1995). An application of fuzzy sets in students’ evaluation. Fuzzy Sets and Systems, 74, 187-194.
 Chen S. M. & Li T. K. (2011). Evaluating students’ learning achievement based on fuzzy rules with fuzzy reasoning capability. Expert Systems with Applications, 38, 4368-4381
 Chen, C. M. (2008). Intelligent web-based learning system with personalized learning path guidance. Computers & Education, 51, 787-814.
 Chen, C. M., & Chung, C. J. (2008). Personalized mobile English vocabulary learning system based on item response theory and learning memory cycle. Computers & Education, 51(2), 624-645.
 Chen, C. M., Lee, H. M., & Chen, Y. H. (2005). Personalized e-learning system using item response theory. Computers and Education, 44(3), 237-255.
 Chen, S. J., & Hwang, C. L. (1992). Fuzzy multiple attribute decision making - methods and applications. Germany: Springer-Verlag.
 Chen, S. M. (1994). A new method for handling multi criteria fuzzy decision making problems. Cybernetics and Systems, 25, 409-420.
 Chen, S. M., & Lee, C. H. (1999). New methods for students’ evaluation using fuzzy sets. Fuzzy Sets and Systems, 104, 209-218.
 Chen, T. C. (2001). A fuzzy approach to select the location of the distribution center. Fuzzy Sets and Systems, 118, 65-73.
 Chen. S. H. (1985). Ranking fuzzy numbers with maximizing set and minimizing set. Fuzzy Sets and Systems, 17, 113-130.
 Cheng, S. C., Huang, Y. M., Chen, J. N., & Lin, Y. T. (2005). Automatic leveling system for e-learning examination pool using entropy-based decision tree. Lecture Notes in Computer Science, 3583, 273-278.
 Cheng, S. C., Lin, Y. T., & Huang, Y. M. (2009). Dynamic question generation system for web-based testing using particle swarm optimization. Expert Systems with Applications, 36(1), 616–624.
 Cheng, S.-Y., Lin, C.-S., Chen, H.-H., & Heh, J.-S. (2005). Learning and diagnosis of individual and class conceptual perspectives: An intelligent systems approach using clustering techniques. Computers and Education, 44(3), 257-283.
 Clarke, T., Ayres, P., & Sweller, J. (2005). The impact of sequencing and prior knowledge on learning mathematics through spreadsheet applications. Educational Technology Research and Development, 53(3), 15-24.
 Cobos-Moyano, A., Martin-Blas, T., & Oñate-Gómez, C. (2009). Evaluating background and prior knowledge: A case study on engineering graphics learning. Computers & Education, 53(3), 695-700.
 Czerniak, C. M., Weber, W. B., Jr., & Sandmann, A. (1999). A literature review of science and mathematics integration. School Science and Mathematics, 99(8), 421-30.
 Davis, F., Bagozzi, R., & Warshaw, R. (1989). User acceptance of computer technology: A comparison of two theoretical models. Management Science, 35(8), 982-1003.
 Dobois, D., & Prade, H. (1980). Fuzzy sets and systems. New York: Academic Press.
 Dochy, F. J.R.C., Moerkerke, G., & Marten, R. (1996). Integrating assessment, learning and instruction: assessment of domain-specific and domain-transcending prior knowledge and program. Studies in Educational Evaluation, 22(4), 309-339.
 Dochy,FJRC.(1992). Assessment of Prior Knowledge as a Determinant for Future Learning: The use of prior knowledge state tests and knowledge profiles. Utrecht/London: Lemma BV.43-72.
 Dochy,FJRC.,De Ridjt, C., Dyck, W.(2002). Cognitive prerequisites and learning. Active Learning Higer Educ. 3:265-84.
 Doran, R. (1980). Basic measurement and evaluation of science instruction. Washington D.C.: National Science Teachers Association.
 Dombi, J. (1990). Membership function as an evaluation. Fuzzy Sets and Systems, 35,1-21.
 Dong, W. M., & Wong, F. S. (1987). Fuzzy weighted averages and implementation of the extension principle. Fuzzy Sets and Systems, 21(2), 183-199.
 Gardner, L., Sheridan, D., & White, D. (2002). A web-based learning and assessment system to support flexible education. Journal of Computer Assisted Learning, 18(2), 125-136
 Gerber, M., Grund, S., & Grote, G. (2008). Distributed collaboration activities in a blended learning scenario and the effects on learning performance. Journal of Computer Assisted Learning, 24(3), 232-244.
 Hailikari, T., Katajavouri,N., & Lindblom-Ylanne, S. (2008). The relevance of prior knowledge in learning and instructional design. American Journal of Pharmaceutical Education, 1-8.
 Hailikari,T, Nevgi, A., Lindblom-Ylänne, S. (2007). Exploring alternative ways of assessing prior knowledge, its components and their relation to student achievement: a mathematics-based case study. Stud Educ Eval.33:320-37.
 Hsieh, T. C. & Wang, T. I. (2010) A mining-based approach on discovering courses pattern for constructing suitable learning path, Expert Systems with Applications, 37(6), 4156-4167.
 Huang, C. J., Liu, M. C., Chu, S. S., & Cheng, C. L. (2007). An intelligent learning diagnosis system for web-based thematic learning platform. Computers & Education, 48(4), 658-679.
 Huang, M. J., Huang, H. S., & Chen, M. Y. (2007). Constructing a personalized e-learning system based on genetic algorithm and case-based reasoning approach. Expert Systems with Applications, 33(3), 551-564.
 Huang, Y. M., Lin, Y. T., & Cheng, S. C. (2009). An adaptive testing system for supporting versatile educational assessment. Computers & Education, 52(1), 53-67.
 Huang, Y. M., Lin, Y. T., & Cheng, S. C. (2010). Effectiveness of a Mobile Plant Learning System in a science curriculum in Taiwanese elementary education. Computers & Education, 54(1), 47-58.
 Hwang, C. L., & Yoon, K. P. (1981). Multiple Attribute Decision Making: Methods and Applications. New York: Springer.
 Hwang, G. J., Chu, H. C., Yin, P. Y., & Lin, J. Y. (2008). An innovative parallel test-sheet composition approach to meet multiple assessment criteria for national tests. Computers & Education, 51(3), 1058-1072.
 Hwang, G. J., Hsiao, C. L., & Tseng, J. C. R. (2003). A computer-assisted approach to diagnosing student learning problems in science courses. Journal of Information Science and Engineering, 19(2), 229-248.
 Jacobs, H. H. (1989). Interdisciplinary curriculum: Design and implementation. Alexandria, VA: Association for Supervision and Curriculum Development.
 Jong, B., Lin, T., Wu, Y. & Chan, T. (2004). Diagnostic and remedial learning strategy based on conceptual graphs. Journal of Computer Assisted Learning, 20(5), 377-386.
 Jong, T. D. & Joolingen, W. R. V. (1998). Scientific Discovery Learning with Computer Simulations of Conceptual Domains. Review of Educational Research, 68(2), 179-201.
 Juhl, L., Yearsley, K., & Silva, A. J. (1997). Interdisciplinary Project-Based Learning through an Environmental Water Quality Study. Journal of Chemical Education, 74(12), 1431-1433.
 Kong, S. C. & So, W. M. W. (2008). A study of building a resource-based learning environment with the inquiry learning approach: Knowledge of family trees. Computers & Education, 50(1), 37-60.
 Lai, C. Y. & Wu, C. C. (2006). Using handhelds in a jigsaw cooperative learning environment. Journal of Computer Assisted Learning, 22(4), 284-297.
 Lavnitskaya, L., Clark, D., Montgomery, G., & Primean, R. (2002). Interdisciplinary Learning: Process and Outcomes. Innovative Higher Education, 27(2), 95-111.
 Law, C.-K. (1996). Using fuzzy numbers in educational grading system. Fuzzy Sets and Systems, 83, 311-323.
 Lee, C. H., Lee, G. G., & Leu, Y. H. (2009). Application of automatically constructed concept map of learning to conceptual diagnosis of e-learning. Expert Systems with Applications, 36(2), 1675-1684.
 Lee, D. H., & Park, D. (1997). An efficient algorithm for fuzzy weighted average. Fuzzy Sets and Systems, 87(1), 39-45.
 Lin, Y. C., Lin Y. T., Huang, Y. M. (2011). Development of a diagnostic system using a testing-based approach for strengthening student prior knowledge. Computers & Education, 57(2), 1557-1570.
 Liu, H. C. & Andre, T. (2006). How college students’ prior knowledge level affected their use of computer simulations to learn electrochemistry. Paper Presented at the 2006 Annual Meeting sessions for the American Educational Research Association (AERA). San Francisco, CA, USA.
 Liu, I. F., Chen, M. C., Sun, Y. S., Wible, D., & Kuo, C. H. (2010). Extending the TAM model to explore the factors that affect intention to use an online learning community. Computers & Education, 54(2), 600–610.
 Loepp, F. L. (1999). Models of curriculum integration. Journal of Technology Studies, 25(2), 21–25.
 Ma, J., & Zhou, D. (2000). Fuzzy set approach to the assessment of student-centered learning. IEEE Transactions on Education, 43 (2), 237-241.
 Manathunga, C, Lant, P., & Mellick, G. (2006). Imagining an interdisciplinary doctoral pedagogy. Teaching in Higher Education, 11(3), 365-379.
 Mitchell, T. J. F., Chen, S. Y., & Macredie, R. D. (2005). Hypermedia learning and prior knowledge: domain expertise vs. system expertise. Journal of Computer Assisted Learning, 21(1), 53-64.
 Moos, D. C., & Azevedo, R. (2008). Self-regulated learning with hypermedia: The role of prior domain knowledge. Contemporary Educational Psychology, 33(2), 270-298.
 Moreno, R. (2004). Decreasing Cognitive Load for Novice Students: Effects of Explanatory versus Corrective Feedback in Discovery-Based Multimedia. Instructional Science, 32(1-2), 99-113.
 Ozuru, Y., Dempsey, K., & McNamara, D.S. (2009). Prior knowledge, reading skill, and text cohesion in the comprehension of science texts. Learning and Instruction, 19(3), 228-242.
 Panjaburee, P., Hwang, G. J., Triampo, W., & Shih, B. Y. (2010). A Multi-Expert Approach for Developing Testing and Diagnostic Systems Based on the Concept Effect Model. Computers & Education, 55(2), 527-540.
 Pintrich, P. R., & De Groot, E. V. (1990). Motivational and self-regulated learning components of classroom academic performance. Journal of Educational Psychology, 82(1), 33-40.
 Ribeiro, R. A. (1996). Fuzzy multiple attribute decision making: A review and new preference elicitation techniques. Fuzzy Sets and Systems, 78, 155-18.
 Roschelle, J. (1995). Learning in Interactive Environments: Prior knowledge and new experience. American Association of Museums, Washington, D.C..
 Ross, T. J., Sorensen, H. C., Savage, S. J., & Carson, J. M. (1990). DAPS: expert system for structural damage assessment. Journal of Computer and Civil Engineering, 4(4), 327-348.
 Saleh, I., & Kim, S. I. (2009). A fuzzy system for evaluating students' learning achievement. Expert Systems with Applications, 36(3), 6236-6243.
 Santhanam, R. and Kyparisis, G. J. (1995). A Multiple Criteria Decision Model for Information System Project Selection. Computers and Operations Research 22(8), 807-818.
 Seel, N.M. & Dinter, F.R. (1995). Instruction and mental model progression: learner-dependent effects of teaching strategies on knowledge acquisition and analogical transfer. Educational Research and Evaluation I, (1), 4-35.
 Shao, X., Zhang, L., Gao, L., & Chen, R (2006). Fuzzy Multiple Attributive Group Decision-Making for Conflict Resolution in Collaborative Design. In Proceedings of FSKD, 990-999.
 Springer, C. R., & Pear, J. J. (2008). Performance measures in courses using computer-aided personalized system of instruction. Computers & Education, 51(2), 829-835.
 Treagust, D. F. (1988). Development and use of diagnostic tests to evaluate students' misconceptions in science. International Journal of Science Education, 10(2), 159-169.
 Tsaganou, G., Grigoriadou, M., Cavoura, T., & Koutra, D. (2003). Evaluating an intelligent diagnosis system of historical text comprehension. Expert Systems with Applications, 25, 493-502.
 Tsai, C. C., & Chou, C. (2002). Diagnosing students' alternative conceptions in science. Journal of Computer Assisted Learning, 18(2), 157-165.
 Tseng, J. C. R., Chu, H. C., Hwang, G. J., & Tsai, C. C. (2008). Development of an adaptive learning system with two sources of personalization information. Computers & Education, 51(2), 776-786.
 Tseng, S. S., Sue, P. C., Su, J. M., Weng, J. F., & Tsai, W. N. (2007). A new approach for constructing the concept map. Computers & Education, 49(3), 691-707.
 Wang, H. Y., & Chen, S. M. (2008). Evaluating students’ answer scripts using fuzzy numbers associated with degrees of confidence. IEEE Transactions on Fuzzy Systems, 16(2), 403-415.
 Yates, G., & Chandler, M. (1994). Prior Knowledge. SET: Research Information for Teachers, 2, Item 6.
 Zadeh, L. A. (1965). Fuzzy sets. Information and Control, 8, 338-353.
 Zhang, D. S., Zhao, J. L., Zhou, L., & Nunamaker, Jr. J. F. (2004). Can e-learning replace classroom learning?. Communications of the ACM, 47(5), 75-79.
 Zimmermann, H.J., 1987, Fuzzy Sets, Decision Making, and Expert Systems, Kluwer, Boston.