進階搜尋


   電子論文尚未授權公開,紙本請查館藏目錄
(※如查詢不到或館藏狀況顯示「閉架不公開」,表示該本論文不在書庫,無法取用。)
系統識別號 U0026-0607201818044700
論文名稱(中文) 用線性概約法來推導F測度抽樣分配以衡量分類方法在不平衡資料檔上效能之研究
論文名稱(英文) A Linear Approximation Approach of F-Measure for Evaluating the Performance of Classification Algorithms on Imbalanced Data Sets
校院名稱 成功大學
系所名稱(中) 資訊管理研究所
系所名稱(英) Institute of Information Management
學年度 106
學期 2
出版年 107
研究生(中文) 陳玟靜
研究生(英文) Wen-Jing Chen
學號 R76051016
學位類別 碩士
語文別 中文
論文頁數 49頁
口試委員 指導教授-翁慈宗
口試委員-蔡青志
口試委員-胡政宏
中文關鍵字 不平衡資料檔  召回率  精確率  F測度  二維常態分配 
英文關鍵字 imbalanced data sets  recall  precision  F-measure  bivariate normal distribution 
學科別分類
中文摘要 面對龐大的資料量,現今通常使用分類正確率來當作評估分類方法的好壞,因為分類正確率是最方便且直接的指標之一,然而,在不平衡資料檔的情況下,分類方法會傾向於將大部分要預測的資料預測為多數的類別值,因此當數量較少的類別值是所要關注的焦點時,使用分類正確率去評估分類方法是不恰當的,所以一般會使用整合召回率和精確率的F測度來評估分類方法在不平衡資料檔上的效能,但由於召回率和精確率具有相依性,且F測度為此二者的調和平均,所以目前並無適當的有母數統計方法來比較不同分類方法的F測度是否有顯著差異,本研究將以二維常態分配去推導兩者結合後的F測度之抽樣分配,進而應用假設檢定去比較兩個分類方法在單一資料檔或是多個資料檔上得到的F測度差異量的是否具有顯著性差異,檢定兩個分類方法間的分類表現。在實證研究的部分,主要是針對不平衡資料檔使用F測度去當作統計檢定的評估測度,選用四個分類方法:簡易貝氏分類法、多層感知器、k最近鄰分類法、基於規則分類法,對十個不平衡資料檔進行兩兩的效能比較,結果顯示簡易貝氏分類法在不平衡資料檔下的表現較差,且與無母數Wilcoxon符號等級檢定作為評估測度來比較,可以發現本研究有母數的方法較能顯現出分類方法間在不平衡資料檔上的效能差異。
英文摘要 The performance of classification algorithms are generally evaluated by accuracy with huge amounts of data. Accuracy is one of the most convenient and direct indicators. However, classification algorithms will tend to predict most of data as the majority of  the category values on imbalanced data sets, accuracy is no longer an appropriate measure for performance evaluation. F-measure is the harmonic mean of precision and recall, and these two indicators are dependent of each other. So, there is no appropriate parametric method to compare the F-measures of different classification algorithms. This study presents parametric methods for comparing the performance of two classification algorithms on one or multiple imbalance data sets when the evaluation measure is bivariate normal distribution by recall and precision. Then hypothesis testing is used to compare whether there is significant difference between two classification algorithms. The main purpose is to use F-measures as performance evaluation on imbalanced data. There are four classification algorithms considered in this study. The experimental results show that Naive Bayes method performs poorly under imbalanced data sets. After we compare with nonparametric Wilcoxon signed- test, we find that the parametric method proposed in this study can effectively compare the performance of two classification algorithms.
論文目次 第一章 緒論 p.1
1.1研究背景與動機 p.1
1.2研究目的 p.3
1.3研究架構 p.3
第二章 文獻探討 p.4
2.1分類方法評估的作法 p.4
2.2不平衡資料的評估測度 p.6
2.3二維常態分配 p.8
2.4分類方法的比較 p.10
2.4.1兩分類方法在單一資料檔下的比較 p.10
2.4.2兩分類方法在多個資料檔下的比較 p.12
2.5小結 p.13
第三章 研究方法 p.15
3.1召回率與精確率個別服從的抽樣分配 p.16
3.2單一資料檔下兩評估測度的整合 p.20
3.3多個資料下使用F測度評估 p.24
3.4方法評估 p.30
第四章 實證研究 p.32
4.1資料檔之特性 p.32
4.2單一資料檔下的比較 p.34
4.3多個資料下的比較 p.38
4.4小結 p.39
第五章 結論與建議 p.41
5.1結論 p.41
5.2建議與未來發展 p.42
參考文獻 p.43
附錄一 分類方法在十個資料檔內之五等分TP、FP、FN、TN、權重值 p.46
參考文獻 林哲玄(2016)。 不平衡資料檔下比較兩分類演算法效能之統計方法。國立成功大學資訊管理研究所碩士論文。

Arlot, S. & Celisse, A. (2010). A survey of cross-validation procedures for model selection. Statistics Surveys, 4, 40-79.

Airola, A., Pahikkala, T., Waegeman, W., De Baets, B., & Salakoski, T. (2011). An experimental comparison of cross-validation techniques for estimating the area under the ROC curve. Computational Statistics & Data Analysis, 55(4), 1828-1844.

Bouckaert, R. R. (2003). Choosing between two learning algorithms based on calibrated tests. Proceedings of the 20th International Conference on Machine Learning (ICML-03), 51-58.

Dietterich, T. G. (1998). Approximate statistical tests for comparing supervised classification learning algorithms. Neural Computation, 10(7), 1895-1923.

Demšar, J. (2006). Statistical comparisons of classifiers over multiple data sets. Journal of Machine Learning Research, 7, 1-30.

De Weerdt, J., De Backer, M., Vanthienen, J., & Baesens, B. (2011). A robust F-measure for evaluating discovered process models. In Computational Intelligence and Data Mining 2011 IEEE Symposium on. 148-155.
George, W. S. & William, G. C. (1937). Statistical Methods. U.S.A:The Lowa State University Press.

García, S., Fernández, A., Luengo, J., & Herrera, F. (2009). A study of statistical techniques and performance measures for genetics-based machine learning: accuracy and interpretability. Soft Computing, 13(10), 959-977.

Hand, D. J., Mannila, H., & Smyth, P. (2001). Principles of Data Mining. MIT press.

Han, J. and Kamber, M. (2006). Data Mining Concepts and Techniques. Morgan kaufmann.

He, H. & Garcia, E. A. (2009). Learning from imbalanced data. IEEE Transactions on Knowledge and Data Engineering, 21(9), 1263-1284.

López, V., Fernández, A., Moreno-Torres, J. G., & Herrera, F. (2012). Analysis of preprocessing vs. cost-sensitive learning for imbalanced classification. Open problems on intrinsic data characteristics. Expert Systems with Applications, 39(7), 6585-6608.

López, V., Fernández, A., García, S., Palade, V., & Herrera, F. (2013). An insight into classification with imbalanced data: Empirical results and current trends on using data intrinsic characteristics. Information Sciences, 250, 113-141.

Mitchell, T. M. (1997). Machine learning. U.S.A: McGraw Hill.


Maratea, A., Petrosino, A., & Manzo, M. (2014). Adjusted F-measure and kernel scaling for imbalanced data learning. Information Sciences, 257, 331-341.

Rodriguez, J. D., Perez, A., & Lozano, J. A. (2010). Sensitivity analysis of k-fold cross validation in prediction error estimation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32(3), 569-575.

Snedecor, G. W. & Cochran, W. G. (1982). Statistical Methods. Ames, Iowa, U.S.A: The Lowa State University Press.

Van RijsbergenC. J. (1979), Information Retrieval. Butterworth.

Witten, I. H. & Frank, E. (2005). Data Mining: Practical Machine Learning Tools and Techniques: Morgan Kaufmann.

Wong, T. T. (2015). Performance evaluation of classification algorithms by k-fold and leave-one-out cross validation. Pattern Recognition, 48, 2839-2846.

Wong, T. T. (2017). Parametric methods for comparing the performance of two classification algorithms evaluated by k-fold cross validation on multiple data sets. Pattern Recognition, 65, 97-107.
論文全文使用權限
  • 同意授權校內瀏覽/列印電子全文服務,於2023-06-15起公開。
  • 同意授權校外瀏覽/列印電子全文服務,於2023-06-15起公開。


  • 如您有疑問,請聯絡圖書館
    聯絡電話:(06)2757575#65773
    聯絡E-mail:etds@email.ncku.edu.tw