進階搜尋


   電子論文尚未授權公開,紙本請查館藏目錄
(※如查詢不到或館藏狀況顯示「閉架不公開」,表示該本論文不在書庫,無法取用。)
系統識別號 U0026-0107201421471400
論文名稱(中文) 單調性ε支援向量機模式之研究
論文名稱(英文) A Study on a Monotonic ε Support Vector Machine Model
校院名稱 成功大學
系所名稱(中) 資訊管理研究所
系所名稱(英) Institute of Information Management
學年度 102
學期 2
出版年 103
研究生(中文) 高偉哲
研究生(英文) Wei-Che Kao
學號 R76011066
學位類別 碩士
語文別 英文
論文頁數 62頁
口試委員 指導教授-李昇暾
口試委員-林清河
口試委員-耿伯文
口試委員-鄭亦君
中文關鍵字 支援向量機  ε支援向量機  先驗知識  單調性限制式 
英文關鍵字 Support vector machine  ε support vector machine  Prior knowledge  Monotonicity constraint 
學科別分類
中文摘要 模式識別在機器學習上有著緊不可分的關係,並且在現在的學術領域上是一個非常活躍的學科。在模式識別上,分類分析是使用分類器來幫助我們將輸入的資料數據分類至正確的類別。而資料探勘便是一種解決實際應用上所會面臨到的分類問題的一種技術。資料探勘技術在分類分析上有著許多的應用分類器,支援向量機即是其中一種。支援向量機是一種最新銳尖端的類神經網路技術,其基於數學、統計、最佳化學習來做分類,現在被應用在許多領域,像是信用評級、企業財務困境預測、文件分類、手寫辨識,與生物訊息學等等。
在最近的學者研究中,學者試圖利用距離學習演算法來對支援向量機進行效能的提升,於是就推導出了ε支援向量機。ε支援向量機試圖最佳化類別內半徑與類別外半徑的比例誤差,這讓ε支援向量機使用上比傳統支援向量機簡易了許多,不需要特徵選擇、探討權重、或是多核函數學習。
分類器應用在實務分類問題上已相對成熟,但是絕大部分的分類器是資料導向的,這產生了一個實務與學術上的偏誤,為了去解決這個問題,我們考慮加入領域上的先驗知識以促進分類器性能。在現實生活中的許多問題裡我們可以觀察到其實屬性與類別間存在著所謂的單調關係,考慮這些先驗的單調關係我們試圖加入單調性進入ε支援向量機,這些單調性的規範我們可以透過資料或是專家來給定,並以不等式的方式加入ε支援向量機的二次規劃問題中。在本研究的最後,我們會對本研究的新方法與傳統ε支援向量機的結果來進行比較。並且相信在最後的結果中,的確加入單調的領域先驗知識是可以增進支援向量機的分類正確率的。
英文摘要 Pattern recognition is a very active field of research intimately bound to machine learning. Classification is a part of this area that aims to build classifiers that can determine the class of an input pattern. Data mining techniques have been applied to solve classification problems in real world applications. Support vector machines (SVMs) have recently been introduced for classification and quickly became the state-of-the-art. Its excellent ability is the focus of research in machine learning. There are human experts in many fields whose knowledge can significantly influence the effectiveness classification. Now, the incorporation of prior knowledge into SVMs has become the key element in improving the performance of SVM’s in many applications.
In recent studies, researchers attempted to improve the performance of SVMs by using the distance metric learning algorithm, and the result was the εSVM. The εSVM optimizes the radius-margin ratio error, and is thus simpler than traditional SVMs because it does not involve feature selection, weighting, and multiple kernel learning. In this study, we explore the incorporation of prior knowledge, in the form of monotonicity constraints, in an εSVM. Our classification model is implemented by constructing monotonicity constraints into anεSVM and determining the contribution of different information. Experiment results show that the proposed model, which considers the monotonicity of prior knowledge and contribution of different data, performs better than the original εSVM model in solving classification problems.
論文目次 摘要 I
ABSTRACT II
誌謝 III
Chapter 1 Introduction 1
1.1 Background and motivation 1
1.2 Research Objectives 5
1.3 Structure of Research 5
Chapter2 Literature Review 7
2.1 Basic Support vector machine (Basic SVM) 7
2.1.1 Evolution of SVM 7
2.1.2 Derivation of Basic SVM 9
2.2 ε Support vector machine (εSVM) 13
2.2.1 SVM and LMNN concepts 13
2.2.2 SVM from a Metric Learning Perspective 16
2.2.3 Architecture of εSVM 18
2.3 Prior knowledge - monotonicity 20
Chapter 3 Research Methodology 24
3.1 Data preprocessing 25
3.2 Concept of monotonicity 26
3.3 Constructing Monotonicity Constraints 28
3.4 Derivation of the Monotonicity Constrained εSVM Model 29
3.5 MC-εSVM Algorithm 33
Chapter 4 Experiment and Result Analysis 36
4.1 Experiment steps 36
4.2 Performance measures 37
4.3 Experiment and result 39
4.3.1 Toy example 39
4.3.2 Real-data Collection 43
4.3.3 Experiment result 53
Chapter 5 Conclusion and Suggestions 56
5.1 Conclusion 56
5.2 Recommendations for future research 57
Reference 58
參考文獻 Aha, D. W., Kibler, D., & Albert, M. K. (1991). Instance-based learning algorithms. Machine learning, 6(1), 37-66.
Archer, N. P., & Wang, S. (1993). Learning bias in neural networks and an approach to controlling its effect in monotonic classification. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 15(9), 962-966. doi: 10.1109/34.232084
Blitzer, J., Weinberger, K. Q., & Saul, L. K. (2005). Distance metric learning for large margin nearest neighbor classification. Paper presented at the Advances in neural information processing systems.
Boser, B. E., Guyon, I. M., & Vapnik, V. N. (1992). A training algorithm for optimal margin classifiers. Paper presented at the Proceedings of the fifth annual workshop on Computational learning theory.
Burges, C. J. C. (1998). A Tutorial on Support Vector Machines for Pattern Recognition. Data Min. Knowl. Discov., 2(2), 121-167. doi: 10.1023/a:1009715923555
Chapelle, O., Vapnik, V., Bousquet, O., & Mukherjee, S. (2002). Choosing multiple parameters for support vector machines. Machine learning, 46(1-3), 131-159.
Chopra, S., Hadsell, R., & LeCun, Y. (2005). Learning a similarity metric discriminatively, with application to face verification. Paper presented at the Computer Vision and Pattern Recognition, 2005. CVPR 2005. IEEE Computer Society Conference on.
Cortes, C., & Vapnik, V. (1995). Support-Vector Networks. Mach. Learn., 20(3), 273-297. doi: 10.1023/a:1022627411411
Cover, T., & Hart, P. (1967). Nearest neighbor pattern classification. Information Theory, IEEE Transactions on, 13(1), 21-27.
Cristianini, N., & Shawe-Taylor, J. (2000). An introduction to support vector machines and other kernel-based learning methods: Cambridge university press.
Davis, S. M., & Botkin, J. W. (1994). The Monster Under the Bed: How Business Is Mastering the Opportunity of Knowledge for Profit: Simon & Schuster.
Decherchi, S., Ridella, S., Zunino, R., Gastaldo, P., & Anguita, D. (2010). Using Unsupervised Analysis to Constrain Generalization Bounds for Support Vector Classifiers. Neural Networks, IEEE Transactions on, 21(3), 424-438. doi: 10.1109/tnn.2009.2038695
Dembczyński, K., Kotłowski, W., & Słowiński, R. (2008). Ensemble of Decision Rules for Ordinal Classification with Monotonicity Constraints. In G. Wang, T. Li, J. Grzymala-Busse, D. Miao, A. Skowron & Y. Yao (Eds.), Rough Sets and Knowledge Technology (Vol. 5009, pp. 260-267): Springer Berlin Heidelberg.
Do, H., Kalousis, A., & Hilario, M. (2009). Feature weighting using margin and radius based error bound optimization in svms Machine Learning and Knowledge Discovery in Databases (pp. 315-329): Springer.
Do, H., Kalousis, A., Wang, J., & Woznica, A. (2012). A metric learning perspective of SVM: on the relation of LMNN and SVM. Paper presented at the International Conference on Artificial Intelligence and Statistics.
Do, H., Kalousis, A., Woznica, A., & Hilario, M. (2009). Margin and radius based multiple kernel learning Machine Learning and Knowledge Discovery in Databases (pp. 330-343): Springer.
Doumpos, M., & Pasiouras, F. (2005). Developing and testing models for replicating credit ratings: A multicriteria approach. Computational Economics, 25(4), 327-341.
Doumpos, M., & Zopounidis, C. (2009). MONOTONIC SUPPORT VECTOR MACHINES FOR CREDIT RISK RATING. New Mathematics and Natural Computation, 05(03), 557-570. doi: doi:10.1142/S1793005709001520
Doumpos, M., Zopounidis, C., & Golfinopoulou, V. (2007). Additive support vector machines for pattern classification. Systems, Man, and Cybernetics, Part B: Cybernetics, IEEE Transactions on, 37(3), 540-550.
Duivesteijn, W., & Feelders, A. (2008a). Nearest Neighbour Classification with Monotonicity Constraints. Paper presented at the Proceedings of the 2008 European Conference on Machine Learning and Knowledge Discovery in Databases - Part I, Antwerp, Belgium.
Duivesteijn, W., & Feelders, A. (2008b). Nearest neighbour classification with monotonicity constraints Machine Learning and Knowledge Discovery in Databases (pp. 301-316): Springer.
Evgeniou, T., Boussios, C., & Zacharia, G. (2005). Generalized robust conjoint estimation. Marketing Science, 24(3), 415-429.
Falck, T., Suykens, J. A., & De Moor, B. (2009). Robustness analysis for least squares kernel based regression: an optimization approach. Paper presented at the Decision and Control, 2009 held jointly with the 2009 28th Chinese Control Conference. CDC/CCC 2009. Proceedings of the 48th IEEE Conference on.
Gai, K., Chen, G., & Zhang, C.-s. (2010). Learning kernels with radiuses of minimum enclosing balls. Paper presented at the Advances in neural information processing systems.
Gamarnik, D. (1998). Efficient learning of monotone concepts via quadratic optimization. Paper presented at the Proceedings of the eleventh annual conference on Computational learning theory, Madison, Wisconsin, United States.
Goldberger, J., Roweis, S., Hinton, G., & Salakhutdinov, R. (2004). Neighbourhood components analysis.
Greco, S., Matarazzo, B., & Słowiński, R. (1998). A new rough set approach to evaluation of bankruptcy risk. Operational tools in the management of financial risks, 121-136.
Hoffman, R. R. (1987). The problem of extracting the knowledge of experts from the perspective of experimental psychology. AI magazine, 8(2), 53.
Huang, K., Ying, Y., & Campbell, C. (2009). Gsml: A unified framework for sparse metric learning. Paper presented at the Data Mining, 2009. ICDM'09. Ninth IEEE International Conference on.
Kim, H. S., & Sohn, S. Y. (2010). Support vector machines for default prediction of SMEs based on technology credit. European Journal of Operational Research, 201(3), 838-846. doi: http://dx.doi.org/10.1016/j.ejor.2009.03.036
Kramer, K. A., Hall, L. O., Goldgof, D. B., Remsen, A., & Luo, T. (2009). Fast support vector machines for continuous data. Trans. Sys. Man Cyber. Part B, 39(4), 989-1001. doi: 10.1109/tsmcb.2008.2011645
Li, S.-T., Shiue, W., & Huang, M.-H. (2006). The evaluation of consumer loans using support vector machines. Expert Systems with Applications, 30(4), 772-782. doi: http://dx.doi.org/10.1016/j.eswa.2005.07.041
Min, R., & Cheng, H. (2009). Effective image retrieval using dominant color descriptor and fuzzy support vector machine. Pattern Recognition, 42(1), 147-157.
Orrù, G., Pettersson-Yeo, W., Marquand, A. F., Sartori, G., & Mechelli, A. (2012). Using support vector machine to identify imaging biomarkers of neurological and psychiatric disease: a critical review. Neuroscience & Biobehavioral Reviews, 36(4), 1140-1152.
Pazzani, M. J., Mani, S., & Shankle, W. R. (2001). Acceptance of Rules Generated by Machine Learning among Medical Experts. Methods of Information in Medicine(2001 (Vol. 40): Issue 5 2001), 380-385.
Pendharkar, P. C. (2005). A data envelopment analysis-based approach for data preprocessing. Knowledge and Data Engineering, IEEE Transactions on, 17(10), 1379-1388. doi: 10.1109/tkde.2005.155
Pendharkar, P. C., & Rodger, J. A. (2003). Technical efficiency-based selection of learning cases to improve forecasting accuracy of neural networks under monotonicity assumption. Decision Support Systems, 36(1), 117-136. doi: http://dx.doi.org/10.1016/S0167-9236(02)00138-0
Potharst, R., & Feelders, A. J. (2002). Classification trees for problems with monotonicity constraints. SIGKDD Explor. Newsl., 4(1), 1-10. doi: 10.1145/568574.568577
Qinghua, H., Xunjian, C., Lei, Z., Zhang, D., Maozu, G., & Yu, D. (2012). Rank Entropy-Based Decision Trees for Monotonic Classification. Knowledge and Data Engineering, IEEE Transactions on, 24(11), 2052-2064. doi: 10.1109/tkde.2011.149
Rakotomamonjy, A. (2003). Variable selection using svm based criteria. The Journal of Machine Learning Research, 3, 1357-1370.
Schölkopf, B., Burges, C., & Vapnik, V. (1995). Extracting support data for a given task. Paper presented at the KDD.
Schultz, M., & Joachims, T. (2004). Learning a distance metric from relative comparisons. Advances in neural information processing systems, 16, 41.
Shalev-Shwartz, S., Singer, Y., & Ng, A. Y. (2004). Online and batch learning of pseudo-metrics. Paper presented at the Proceedings of the twenty-first international conference on Machine learning.
Shental, N., Hertz, T., Weinshall, D., & Pavel, M. (2006). Adjustment learning and relevant component analysis Computer Vision—ECCV 2002 (pp. 776-790): Springer.
Shilton, A., Palaniswami, M., Ralph, D., & Ah Chung, T. (2005). Incremental training of support vector machines. Neural Networks, IEEE Transactions on, 16(1), 114-131. doi: 10.1109/tnn.2004.836201
Shin, K.-S., Lee, T. S., & Kim, H.-j. (2005). An application of support vector machines in bankruptcy prediction model. Expert Systems with Applications, 28(1), 127-135. doi: http://dx.doi.org/10.1016/j.eswa.2004.08.009
Vapnik, V. N. (1995). The nature of statistical learning theory: Springer-Verlag New York, Inc.
Vapnik, V. N. (1998). Statistical learning theory: Wiley.
Vapnik, V. N., & Chervonenkis, A. J. (1974). Theory of pattern recognition.
Vazirigiannis, M., Halkidi, M., & Gunopulos, D. (2003). Uncertainty handling and quality assessment in data mining. London; New York: Springer.
Wang, S. (1995). The Unpredictability of Standard Back Propagation Neural Networks in Classification Applications. Management Science, 41(3), 555-559. doi: 10.2307/2632981
Wang, S. (2003). Adaptive non-parametric efficiency frontier analysis: a neural-network-based model. Computers & Operations Research, 30(2), 279-295. doi: http://dx.doi.org/10.1016/S0305-0548(01)00095-8
Weinberger, K. Q., & Saul, L. K. (2009). Distance metric learning for large margin nearest neighbor classification. The Journal of Machine Learning Research, 10, 207-244.
Xusheng, G., Jingshun, D., & Wei, C. (2010). Flight accident modeling and predicting based on least squares support vector machine. Paper presented at the Educational and Information Technology (ICEIT), 2010 International Conference on.
Yu, H.-F., Hsieh, C.-J., Chang, K.-W., & Lin, C.-J. (2012). Large linear classification when data cannot fit in memory. ACM Transactions on Knowledge Discovery from Data (TKDD), 5(4), 23.
論文全文使用權限
  • 同意授權校內瀏覽/列印電子全文服務,於2024-12-31起公開。


  • 如您有疑問,請聯絡圖書館
    聯絡電話:(06)2757575#65773
    聯絡E-mail:etds@email.ncku.edu.tw