進階搜尋


   電子論文尚未授權公開,紙本請查館藏目錄
(※如查詢不到或館藏狀況顯示「閉架不公開」,表示該本論文不在書庫,無法取用。)
系統識別號 U0026-1906201423250300
論文名稱(中文) 具單調性限制式支援向量迴歸模型之研究
論文名稱(英文) Toward a Monotonicity Constrained Support Vector Regression Model
校院名稱 成功大學
系所名稱(中) 資訊管理研究所
系所名稱(英) Institute of Information Management
學年度 102
學期 2
出版年 103
研究生(中文) 陳煜棣
研究生(英文) Yu-Di Chen
學號 R76013018
學位類別 碩士
語文別 英文
論文頁數 48頁
口試委員 指導教授-李昇暾
口試委員-林清河
口試委員-耿伯文
口試委員-鄭亦君
中文關鍵字 單調性支援向量迴歸  支援向量迴歸  SVM  單調限制式  資料探勘  先驗知識 
英文關鍵字 MC-SVR  SVR  SVM  monotonicity constraint  data mining  prior knowledge 
學科別分類
中文摘要 機器學習技術被廣泛應用於分析和提取知識。而該領域中的資料探勘工具可以自動或者半自動的從資料庫中發現知識。近年來,一種基於統計學習理論的類神經網路,支援向量機(SVM)因其出色的泛化能力成為了大家研究的焦點。支援向量迴歸(SVR)是SVM中最常見的應用,其主要解決連續輸出值的問題。與傳統迴歸方法最小化經驗誤差不同,SVR通過最小化泛化誤差邊界而獲得了良好的推廣能力。SVR已被廣泛應用與時間序列、財務預測、工程分析以及凸二次規劃等各個領域。但是,在實際應用中,結合先驗知識到SVR中可以有效提高單純資料驅動模型的質量,這也更貼近應用中遇到的實際情況。
單調性指的是在模型預測時輸出值與某些屬性有望大或者望小的關係存在,對於此種問題加入單調先驗知識的技術已經被證明可以有效減少模型錯誤。在這項研究中,我們提出了一個以單調知識為導向的新支援向量機迴歸模型。利用專家的知識來檢索資料集的單調規則並構建單調性約束,再將其加入SVR迴歸模型中。在函數預測和實際資料集上進行的實驗表明,新方法在資料基礎上以領域知識為導向,可以有效糾正在資料收集過程中出現的單調性損失,並且回歸表現比傳統SVR方法更好。
英文摘要 Machine learning techniques are widely used for analysis and extraction of knowledge. Data mining, as a tool of machine learning for knowledge discovery in databases (KDD), can automatically or semi-automatically analyze large quantities of data. In recent years, support vector machine (SVM), a state-of-the-art artificial neural network based on statistical learning, has been the focus of research in machine learning due to its excellent ability. Support vector regression (SVR) is the most common form of application of SVMs when the output is continuous. Instead of minimizing the observed training error, SVR attempts to minimize the generalization error bound, so as to achieve generalized performance. SVR has been applied in various fields – time series and financial (noisy and risky) prediction, approximation of complex engineering analyses, convex quadratic programming and choices of loss functions, etc. However, in many real-world problems, incorporating prior knowledge into SVR can improve the quality of models that are only data-driven, and close the wide gap between academic and business goals. We can observe some monotonic relationships between the output value and attributes, and it has been shown that a technique incorporating monotonicity constraints can reduce errors.
In this study, we propose a knowledge-oriented new support vector regression model with monotonicity constraints, and exploit the knowledge of experts to retrieve monotonic rules from datasets. After which we construct monotonicity constraints to implement the proposed regression model. Experiments conducted on function prediction and real-world data sets show that the proposed method, which is not only data driven, but also domain knowledge oriented, can help correct the loss of monotonicity in data during the collection process, and performs better than traditional methods.
論文目次 Catalog
摘要 I
ABSTRACT II
誌謝 III
Catalog IV
Table Catalog V
Figure Catalog V
Chapter 1 Introduction 1
1.1 Background and motivation 1
1.2 Objectives of Research 5
1.3 Organization of Research 5
Chapter2 Literature Review 7
2.1 Support vector machine (SVM) 7
2.1.1 Introduce SVM 7
2.1.2 Architecture of SVMs 9
2.2 Support Vector Regression (SVR) 12
2.3 Prediction Model with Monotonicity Constraints 17
2.4 Data preprocessing 20
Chapter 3 Research Methodology 22
3.1 Construction of monotonicity constrained 23
3.2 Derivation of the Monotonicity Constrained Support Vector Regression (MC-SVR) Model 25
3.3 The Algorithm for Solving MC-SVR 28
Chapter 4 Experiment and Result analysis 31
4.1 Performance Measures 31
4.2 Experimental Design of Function Prediction 32
4.3 the real-world dataset 33
4.4 Experiment Result 35
Chapter 5 Conclusions and Suggestions 41
5.1 Conclusions 41
5.2 Recommendations for future research 41
References 43
參考文獻 Archer, N. P., & Wang, S. (1993). Learning bias in neural networks and an approach to controlling its effect in monotonic classification. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 15(9), 962-966.
Basak, D., Pal, S., & Patranabis, D. C. (2007). Support vector regression. Neural Information Processing-Letters and Reviews, 11(10), 203-224.
Ben-David, A. (1995). Monotonicity maintenance in information-theoretic machine learning algorithms. Machine Learning, 19(1), 29-43.
Burges, C. J. C. (1998). A Tutorial on Support Vector Machines for Pattern Recognition. Data Min. Knowl. Discov., 2(2), 121-167.
Camps-Valls, G., Soria-Olivas, E., Pérez-Ruixo, J., Pérez-Cruz, F., Figueiras-Vidal, A., & Artés-Rodrıguez, A. (2002). Cyclosporine concentration prediction using clustering and Support Vector Regression methods. Electronics Letters, 38(12), 568-570.
Cherkassky, V., & Ma, Y. (2004). Practical selection of SVM parameters and noise estimation for SVM regression. Neural networks, 17(1), 113-126.
Chu, W., & Keerthi, S. S. (2005). New approaches to support vector ordinal regression. Paper presented at the Proceedings of the 22nd international conference on Machine learning.
Cortes, C., & Vapnik, V. (1995). Support-Vector Networks. Mach. Learn., 20(3), 273-297.
Courant, R., & Hilbert, D. (1970). Methods of Mathematical Physics (Vol. I, II). New York: Wiley Interscience.
Cristianini, N., & Shawe-Taylor, J. (2000). An introduction to support vector machines and other kernel-based learning methods: Cambridge university press.
Decherchi, S., Ridella, S., Zunino, R., Gastaldo, P., & Anguita, D. (2010). Using Unsupervised Analysis to Constrain Generalization Bounds for Support Vector Classifiers. Neural Networks, IEEE Transactions on, 21(3), 424-438.
Dembczyński, K., Kotłowski, W., & Słowiński, R. (2008a). Ensemble of decision rules for ordinal classification with monotonicity constraints Rough Sets and Knowledge Technology (pp. 260-267): Springer.
Dembczyński, K., Kotłowski, W., & Słowiński, R. (2008b). Ensemble of Decision Rules for Ordinal Classification with Monotonicity Constraints. In G. Wang, T. Li, J. Grzymala-Busse, D. Miao, A. Skowron & Y. Yao (Eds.), Rough Sets and Knowledge Technology (Vol. 5009, pp. 260-267): Springer Berlin Heidelberg.
Doumpos, M., & Pasiouras, F. (2005). Developing and Testing Models for Replicating Credit Ratings: A Multicriteria Approach. Computational Economics, 25(4), 327-341.
Doumpos, M., & Zopounidis, C. (2009). Monotonic Support Vector Machines For Credit Risk Rating. New Mathematics and Natural Computation, 5(03), 557-570.
Doumpos, M., Zopounidis, C., & Golfinopoulou, V. (2007). Additive Support Vector Machines for Pattern Classification. Systems, Man, and Cybernetics, Part B: Cybernetics, IEEE Transactions on, 37(3), 540-550.
Drucker, H., Burges, C. J., Kaufman, L., Smola, A., & Vapnik, V. (1997). Support vector regression machines. Advances in neural information processing systems, 9, 155-161.
Duivesteijn, W., & Feelders, A. (2008a). Nearest neighbour classification with monotonicity constraints Machine Learning and Knowledge Discovery in Databases (pp. 301-316): Springer.
Duivesteijn, W., & Feelders, A. (2008b). Nearest Neighbour Classification with Monotonicity Constraints. Paper presented at the Proceedings of the 2008 European Conference on Machine Learning and Knowledge Discovery in Databases - Part I, Antwerp, Belgium.
Falck, T., Pelckmans, K., Suykens, J. A., & De Moor, B. (2009). Identification of wiener-hammerstein systems using LS-SVMs. Paper presented at the Proceedings of the 15th IFAC Symposium on System Identification (SYSID 2009), Saint-Malo, France.
Fayyad, U., Piatetsky-Shapiro, G., & Smyth, P. (1996). From data mining to knowledge discovery in databases. AI magazine, 17(3), 37.
Gamarnik, D. (1998). Efficient learning of monotone concepts via quadratic optimization. Paper presented at the Proceedings of the eleventh annual conference on Computational learning theory, Madison, Wisconsin, United States.
Greco, S., Matarazzo, B., & Słowiński, R. (1998). A new rough set approach to evaluation of bankruptcy risk. Operational tools in the management of financial risks, 121-136.
Gruber, C., Gruber, T., Krinninger, S., & Sick, B. (2010). Online Signature Verification With Support Vector Machines Based on LCSS Kernel Functions. Systems, Man, and Cybernetics, Part B: Cybernetics, IEEE Transactions on, 40(4), 1088-1100.
Guyon, I., Matic, N., & Vapnik, V. (1996). Discovering Informative Patterns and Data Cleaning.
Hsieh, T.-J., Hsiao, H.-F., & Yeh, W.-C. (2012). Mining financial distress trend data using penalty guided support vector machines based on hybrid of particle swarm optimization and artificial bee colony algorithm. Neurocomputing, 82, 196-206.
Hu, Q., Che, X., Zhang, L., Zhang, D., Guo, M., & Yu, D. (2011). Rank entropy based decision trees for monotonic classification.
Hua, X., Ni, Y., Ko, J., & Wong, K. (2007). Modeling of temperature–frequency correlation using combined principal component analysis and support vector regression technique. Journal of Computing in Civil Engineering, 21(2), 122-135.
Hua, Z., Wang, Y., Xu, X., Zhang, B., & Liang, L. (2007). Predicting corporate financial distress based on integration of support vector machine and logistic regression. Expert Systems with Applications, 33(2), 434-440.
Huang, W., Nakamori, Y., & Wang, S.-Y. (2005). Forecasting stock market movement direction with support vector machine. Computers & Operations Research, 32(10), 2513-2522.
Huang, Z., Chen, H., Hsu, C.-J., Chen, W.-H., & Wu, S. (2004). Credit rating analysis with support vector machines and neural networks: a market comparative study. Decis. Support Syst., 37(4), 543-558.
Kazem, A., Sharifi, E., Hussain, F. K., Saberi, M., & Hussain, O. K. (2013). Support vector regression with chaos-based firefly algorithm for stock market price forecasting. Applied Soft Computing, 13(2), 947-958.
Kim, H. S., & Sohn, S. Y. (2010). Support vector machines for default prediction of SMEs based on technology credit. European Journal of Operational Research, 201(3), 838-846.
Kim, K.-j. (2003). Financial time series forecasting using support vector machines. Neurocomputing, 55(1), 307-319.
Kramer, K. A., Hall, L. O., Goldgof, D. B., Remsen, A., & Luo, T. (2009). Fast support vector machines for continuous data. Trans. Sys. Man Cyber. Part B, 39(4), 989-1001.
Kumar, M. A., & Gopal, M. (2010). A comparison study on multiple binary-class SVM methods for unilabel text categorization. Pattern Recognition Letters, 31(11), 1437-1444.
Lauer, F., & Bloch, G. (2008a). Incorporating prior knowledge in support vector machines for classification: A review. Neurocomputing, 71(7), 1578-1594.
Lauer, F., & Bloch, G. (2008b). Incorporating prior knowledge in support vector regression. Machine Learning, 70(1), 89-118.
Lauer, F., Suen, C. Y., & Bloch, G. (2007). A trainable feature extractor for handwritten digit recognition. Pattern Recognition, 40(6), 1816-1824.
Liao, S.-P., Lin, H.-T., & Lin, C.-J. (2002). A note on the decomposition methods for support vector regression. Neural Computation, 14(6), 1267-1281.
Lu, C.-J., Lee, T.-S., & Chiu, C.-C. (2009). Financial time series forecasting using independent component analysis and support vector regression. Decision Support Systems, 47(2), 115-125.
Müller, K.-R., Smola, A. J., Rätsch, G., Schölkopf, B., Kohlmorgen, J., & Vapnik, V. (1997). Predicting time series with support vector machines Artificial Neural Networks—ICANN'97 (pp. 999-1004): Springer.
Ma, J., Theiler, J., & Perkins, S. (2003). Accurate on-line support vector regression. Neural Computation, 15(11), 2683-2703.
Mak, M.-W., & Rao, W. (2011). Utterance partitioning with acoustic vector resampling for GMM–SVM speaker verification. Speech Communication, 53(1), 119-130.
Man Gyun, N., Won Seo, P., & Dong Hyuk, L. (2008). Detection and Diagnostics of Loss of Coolant Accidents Using Support Vector Machines. Nuclear Science, IEEE Transactions on, 55(1), 628-636.
Mariéthoz, J., & Bengio, S. (2007). A kernel trick for sequences applied to text-independent speaker verification systems. Pattern Recognition, 40(8), 2315-2324.
Martín-Guerrero, J. D., Camps-Valls, G., Soria-Olivas, E., Serrano-López, A. J., Pérez-Ruixo, J. J., & Jiménez-Torres, N. V. (2003). Dosage individualization of erythropoietin using a profile-dependent support vector regression. Biomedical Engineering, IEEE Transactions on, 50(10), 1136-1142.
Martens, D., De Backer, M., Haesen, R., Baesens, B., Mues, C., & Vanthienen, J. (2006). Ant-based approach to the knowledge fusion problem Ant Colony Optimization and Swarm Intelligence (pp. 84-95): Springer.
Mattera, D., & Haykin, S. (1999). Support vector machines for dynamic reconstruction of a chaotic system. Paper presented at the Advances in kernel methods.
Mukherjee, S., Osuna, E., & Girosi, F. (1997, 24-26 Sep 1997). Nonlinear prediction of chaotic time series using support vector machines. Paper presented at the Neural Networks for Signal Processing [1997] VII. Proceedings of the 1997 IEEE Workshop.
Pazzani, M. J., Mani, S., & Shankle, W. R. (2001). Acceptance of Rules Generated by Machine Learning among Medical Experts. Methods of Information in Medicine(2001 (Vol. 40): Issue 5 2001), 380-385.
Pearson, R. K. (2005). Mining imperfect data: Dealing with contamination and incomplete records: Siam.
Pelckmans, K., Espinoza, M., De Brabanter, J., Suykens, J. A., & De Moor, B. (2005). Primal-dual monotone kernel regression. Neural Processing Letters, 22(2), 171-182.
Pendharkar, P. C. (2005). A data envelopment analysis-based approach for data preprocessing. Knowledge and Data Engineering, IEEE Transactions on, 17(10), 1379-1388.
Pendharkar, P. C., & Rodger, J. A. (2003). Technical efficiency-based selection of learning cases to improve forecasting accuracy of neural networks under monotonicity assumption. Decision Support Systems, 36(1), 117-136.
Pontil, M., & Verri, A. (1997). Properties of Support Vector Machines: Massachusetts Institute of Technology.
Potharst, R., & Feelders, A. J. (2002). Classification trees for problems with monotonicity constraints. ACM SIGKDD Explorations Newsletter, 4(1), 1-10.
Potharst, R., & Feelders, A. J. (2002). Classification trees for problems with monotonicity constraints. SIGKDD Explor. Newsl., 4(1), 1-10.
Hu, Q.H., Che, X.J., Zhang, L., Guo, M.Z. (2012). Rank Entropy-Based Decision Trees for Monotonic Classification. Knowledge and Data Engineering, IEEE Transactions on, 24(11), 2052-2064.
Ravikumar, B., Thukaram, D., & Khincha, H. P. (2009). An Approach Using Support Vector Machines for Distance Relay Coordination in Transmission System. Power Delivery, IEEE Transactions on, 24(1), 79-88.
Schölkopf, B., Bartlett, P., Smola, A., & Williamson, R. (1998). Support vector regression with automatic accuracy control ICANN 98 (pp. 111-116): Springer.
Schölkopf, B., & Smola, A. J. (2002). Learning with Kernels --Support Vector Machines, Regularization, Optimization and Beyond. Cambridge, Massachusetts: The MIT Press.
Shilton, A., Palaniswami, M., Ralph, D., & Ah Chung, T. (2005). Incremental training of support vector machines. Neural Networks, IEEE Transactions on, 16(1), 114-131.
Shin, K.-S., Lee, T. S., & Kim, H.-j. (2005). An application of support vector machines in bankruptcy prediction model. Expert Systems with Applications, 28(1), 127-135.
Song, M., Breneman, C. M., Bi, J., Sukumar, N., Bennett, K. P., Cramer, S., & Tugcu, N. (2002). Prediction of protein retention times in anion-exchange chromatography systems using support vector regression. Journal of chemical information and computer sciences, 42(6), 1347-1357.
Stitson, M., Gammerman, A., Vapnik, V., Vovk, V., Watkins, C., & Weston, J. (1999). Support vector regression with ANOVA decomposition kernels. Advances in kernel methods—Support vector learning, 285-292.
Sun, B.-Y., Li, J., Wu, D. D., Zhang, X.-M., & Li, W.-B. (2010). Kernel discriminant learning for ordinal regression. Knowledge and Data Engineering, IEEE Transactions on, 22(6), 906-910.
Tay, F. E., & Cao, L. (2001). Application of support vector machines in financial time series forecasting. Omega, 29(4), 309-317.
Trafalis, T. B., & Ince, H. (2000). Support vector machine for regression and applications to financial forecasting. Paper presented at the Neural Networks, IEEE-INNS-ENNS International Joint Conference on.
Van Gestel, T., Baesens, B., Van Dijcke, P., Garcia, J., Suykens, J. A., & Vanthienen, J. (2006). A process model to develop an internal rating system: sovereign credit ratings. Decision Support Systems, 42(2), 1131-1151.
Vapnik. (1995). The nature of statistical learning theory: Springer-Verlag New York. Inc.
Vapnik, Golowich, S., & Smola, A. (1997). Support vector method for function approximation, regression estimation, and signal processing. Advances in neural information processing systems, 281-287.
Vapnik, V. N. (1998). Statistical learning theory: Wiley.
Wang, S. (1995). The Unpredictability of Standard Back Propagation Neural Networks in Classification Applications. Management Science, 41(3), 555-559.
Wang, S. (2003). Adaptive non-parametric efficiency frontier analysis: a neural-network-based model. Computers & Operations Research, 30(2), 279-295.
Xia, Y., Liu, Y., & Chen, Z. (2013). Support Vector Regression for prediction of stock trend. Paper presented at the Information Management, Innovation Management and Industrial Engineering (ICIII), 2013 6th International Conference on.
Xu, Y., Wang, X.-B., Ding, J., Wu, L.-Y., & Deng, N.-Y. (2010). Lysine acetylation sites prediction using an ensemble of support vector machine classifiers. Journal of theoretical biology, 264(1), 130-135.
Yeh, C.-Y., Huang, C.-W., & Lee, S.-J. (2011). A multiple-kernel support vector regression approach for stock market price forecasting. Expert Systems with Applications, 38(3), 2177-2186.
Yu, L., Yao, X., Wang, S., & Lai, K. (2011). Credit risk evaluation using a weighted least squares SVM classifier with design of experiment for parameter selection. Expert Systems with Applications, 38(12), 15392-15399.
Zhang, L., Zhou, W.-D., Chang, P.-C., Yang, J.-W., & Li, F.-Z. (2013). Iterated time series prediction with multiple support vector regression models. Neurocomputing, 99, 411-422.
Zhang, X. (1999). Using class-center vectors to build support vector machines. Paper presented at the Neural Networks for Signal Processing IX, 1999. Proceedings of the 1999 IEEE Signal Processing Society Workshop.
論文全文使用權限
  • 同意授權校內瀏覽/列印電子全文服務,於2024-12-31起公開。


  • 如您有疑問,請聯絡圖書館
    聯絡電話:(06)2757575#65773
    聯絡E-mail:etds@email.ncku.edu.tw