進階搜尋


   電子論文尚未授權公開,紙本請查館藏目錄
(※如查詢不到或館藏狀況顯示「閉架不公開」,表示該本論文不在書庫,無法取用。)
系統識別號 U0026-0212201917454000
論文名稱(中文) 基於可能性分佈理論對含名目屬性的小樣本建立穩健的預測模型
論文名稱(英文) Building Robust Models for Small Data Containing Nominal Inputs and Continuous Outputs Based on Possibility Distributions
校院名稱 成功大學
系所名稱(中) 工業與資訊管理學系
系所名稱(英) Department of Industrial and Information Management
學年度 108
學期 1
出版年 108
研究生(中文) 史其仕
研究生(英文) Qi-Shi Shi
學號 R38063508
學位類別 博士
語文別 英文
論文頁數 51頁
口試委員 指導教授-利德江
召集委員-吳植森
口試委員-王維聰
口試委員-蔡長鈞
口試委員-黃信豪
中文關鍵字 小樣本  虛擬樣本  可能性分佈  名目屬性 
英文關鍵字 Small data  virtual sample  possibility distribution  nominal input 
學科別分類
中文摘要 傳統的機器學習演算法通常很難在小樣本學習上建立穩健的模型,因為小樣本學習存在過擬合的問題。在過去的研究中,基於模糊理論的虛擬樣本增生技術已經被廣泛地驗證其在小樣本學習上的有效性。然而,現有的多數虛擬樣本增生技術都用來處理數值型的資料,面對名目屬性,無法通過資訊擴散原理產生虛擬樣本。因此,本研究針對含名目屬性的小樣本預測問題,提出系統性的虛擬樣本增生技術。首先,本研究根據M5’模式樹的名目屬性編碼原理,發展出萃取名目屬性和數值輸出的模糊關係;另外,根據屬性間趨勢相似度的概念,發展出數值屬性和輸出的模糊關係。然後,利用這些模糊關係,在給定其中一個值的情況下推估另外一個值的可能性分佈。最後,通過隨機產生的虛擬值,計算這些虛擬值的可能性值,來產生虛擬樣本。在驗證階段,實驗使用五筆公開資料集,兩種預測模型以及兩個其它的虛擬樣本增生技術作為對照組。實驗結果顯示,使用本研究所產生的虛擬樣本可以改善小樣本的學習效果,且改善的效果與對照組比較,有統計學上的顯著意義。
英文摘要 Regarding building statistically robust models, it is challenging for standard algorithms to learning from small data. In previous studies, virtual sample generation (VSG) techniques have been verified as effective in terms of meeting this challenge. However, most VSG techniques were developed for numerical datasets and classification problems. Therefore, to address situations where the dataset has nominal inputs and continuous outputs, a systemic VSG procedure is proposed in this study to create new samples based on theories of fuzziness and diffusion. At first, based on the concept of the encoding process in the M5’ model tree, this study reveals a useful procedure by which to extract the fuzzy relations between nominal inputs and continuous outputs. Further, with the idea of nonparametric operations, it employs trend similarity to present the fuzzy relations between inputs and outputs. Then, possibility distributions of the inputs and outputs are built based on these fuzzy relations. Finally, virtual samples are created based on these distributions and their possibility values. In the experiments, it uses five public datasets, two prediction models and two other VSG techniques. The experimental results show that the small data using virtual samples created by the proposed method outperform the comparison experiments with the other VSG techniques.
論文目次 摘要 i
Abstract ii
誌謝 iii
List of Tables vi
List of Figures vii
1. Introduction 1
1.1 Backgrounds 1
1.2 Motivation 2
1.3 Objectives 5
1.4 Organization 5
2. Literature Review 7
2.1 Virtual Sample Generation for Learning Small Data 7
2.2 The Theoretical Bases of the Proposed Method 10
2.2.1 The encoding of nominal inputs in the M5’ model tree 10
2.2.2 The possibility theory 11
2.2.3 The mega-trend-diffusion technique 12
3. The Proposed Method 14
3.1 Notation Definitions 14
3.2 Relation Extraction 15
3.2.1 Nominal input preprocessing 15
3.2.2 Numerical input preprocessing 16
3.3 Virtual Sample Generation 17
3.3.1 Creating output values 18
3.3.2 Creating nominal attribute values 18
3.3.3 Creating numerical attribute values 19
3.4 Sample Creation Procedure 21
4. The Experimental Environment 24
4.1 The Examined Datasets 24
4.2 The Experimental Designs 24
4.2.1 The experimental procedure 24
4.2.2 Modeling software and parameter settings 27
5. Experimental Results and Discoveries 28
5.1 Experimental Results and General Discoveries 28
5.2 Parameter Setting Discussion 38
6. Conclusion 42
References 44
參考文獻 Ali, S. S., Howlader, T., & Rahman, S. M. M. (2018). Pooled shrinkage estimator for quadratic discriminant classifier: an analysis for small sample sizes in face recognition. International Journal of Machine Learning and Cybernetics, 9(3), 507-522.
Breiman, L. (1996). Bagging Predictors. Machine Learning, 24(2), 123-140.
Chawla, N. V., Bowyer, K. W., Hall, L. O., & Kegelmeyer, W. P. (2002). SMOTE: synthetic minority over-sampling technique. Journal of artificial intelligence research, 16, 321-357.
Conroy, B., Eshelman, L., Potes, C., & Xu-Wilson, M. (2016). A dynamic ensemble approach to robust classification in the presence of missing data. Machine Learning, 102(3), 443-463.
Cost, S., & Salzberg, S. (1993). A weighted nearest neighbor algorithm for learning with symbolic features. Machine Learning, 10(1), 57-78.
de Jesús Rubio, J. (2018). Error convergence analysis of the SUFIN and CSUFIN. Applied Soft Computing, 72, 587-585.
Efron, B. (1979). Computers and the theory of statistics: thinking the unthinkable. SIAM review, 21(4), 460-480.
Fard, M. J., Wang, P., Chawla, S., & Reddy, C. K. (2016). A Bayesian Perspective on Early Stage Event Prediction in Longitudinal Data. IEEE Transactions on Knowledge and Data Engineering, 28, 3126-3139.
Fernández, A., Garcia, S., Herrera, F., & Chawla, N. V. (2018). SMOTE for Learning from Imbalanced Data: Progress and Challenges, Marking the 15-year Anniversary. Journal of artificial intelligence research, 61, 863-905.
Gosset, W. S. (1908). The probable error of a mean. Biometrika, 6(1), 1-25. doi:10.1093/biomet/6.1.1
Gui, L., Xu, R. F., Lu, Q., Du, J. C., & Zhou, Y. (2018). Negative transfer detection in transductive transfer learning. International Journal of Machine Learning and Cybernetics, 9(2), 185-197.
Hill, L., Aboud, D., Elliott, J., Magnussen, J., Sterling, M., Steffens, D., & Hancock, M. J. (2018). Do findings identified on magnetic resonance imaging predict future neck pain? A systematic review. The Spine Journal, 18(5), 880-891.
Huang, C. (1997). Principle of information diffusion. Fuzzy Sets and Systems, 91(1), 69-90.
Huang, C., & Moraga, C. (2004). A diffusion-neural-network for learning from small samples. International Journal of Approximate Reasoning, 35(2), 137-161.
Kang, G., Wu, L., Guan, Y., & Peng, Z. (2019). A Virtual Sample Generation Method Based on Differential Evolution Algorithm for Overall Trend of Small Sample Data: Used for Lithium-ion Battery Capacity Degradation Data. Ieee Access, 7, 123255-123267.
Kawakita, M., Oie, Y., & Takeuchi, J. i. (2010). A note on model selection for small sample regression. Paper presented at the 2010 International Symposium On Information Theory & Its Applications, Taichung, Taiwan.
Li, D.-C., Chen, C.-C., Chen, W.-C., & Chang, C.-J. (2012). Employing dependent virtual samples to obtain more manufacturing information in pilot runs. International Journal of Production Research, 50(23), 6886-6903.
Li, D.-C., Chen, L.-S., & Lin, Y.-S. (2003). Using functional virtual population as assistance to learn scheduling knowledge in dynamic manufacturing environments. International Journal of Production Research, 41(17), 4011-4024.
Li, D.-C., Huang, W.-T., Chen, C.-C., & Chang, C.-J. (2013). Employing virtual samples to build early high-dimensional manufacturing models. International Journal of Production Research, 51(11), 3206-3224.
Li, D.-C., Huang, W.-T., Chen, C.-C., & Chang, C.-J. (2014). Employing box plots to build high-dimensional manufacturing models for new products in TFT-LCD plants. Neurocomputing, 142, 73-85.
Li, D.-C., Lin, W.-K., Chen, C.-C., Chen, H.-Y., & Lin, L.-S. (2018). Rebuilding sample distributions for small dataset learning. Decision Support Systems, 105, 66-76.
Li, D.-C., Lin, W.-K., Lin, L.-S., Chen, C.-C., & Huang, W.-T. (2017). The attribute-trend-similarity method to improve learning performance for small datasets. International Journal of Production Research, 55(7), 1898-1913.
Li, D.-C., & Liu, C.-W. (2012). Extending attribute information for small data set classification. IEEE Transactions on Knowledge and Data Engineering, 24(3), 452-464.
Li, D.-C., Shi, Q.-S., & Li, M.-D. (2018). Using an attribute conversion approach for sample generation to learn small data with highly uncertain features. International Journal of Production Research, 56(14), 4954-4967.
Li, D.-C., & Wen, I.-H. (2014). A genetic algorithm-based virtual sample generation technique to improve small data set learning. Neurocomputing, 143, 222-230.
Li, D.-C., Wu, C.-S., Tsai, T.-I., & Chang, F. M. (2006). Using mega-fuzzification and data trend estimation in small data set learning for early FMS scheduling knowledge. Computers & Operations Research, 33(6), 1857-1869.
Li, D.-C., Wu, C.-S., Tsai, T.-I., & Lina, Y.-S. (2007). Using mega-trend-diffusion and artificial samples in small data set learning for early flexible manufacturing system scheduling knowledge. Computers & Operations Research, 34(4), 966-982.
Li, D.-C., Yeh, C.-W., Chen, C.-C., & Shih, H.-T. (2014). Using a diffusion wavelet neural network for short-term time series learning in the wafer level chip scale package process. Journal of Intelligent Manufacturing, 27(6), 1261-1272.
Lin, Y.-S., & Li, D.-C. (2010). The Generalized-Trend-Diffusion modeling algorithm for small data sets in the early stages of manufacturing systems. European Journal of Operational Research, 207(1), 121-130.
Meza, A. G., Cortes, T. H., Lopez, A. V. C., Carranza, L. A. P., Herrera, R. T., Ramirez, I. O. C., & Campana, J. A. M. (2017). Analysis of Fuzzy Observability Property for a Class of TS Fuzzy Models. IEEE Latin America Transactions, 15(4), 595-602.
Pan, S. J., & Yang, Q. A. (2010). A Survey on Transfer Learning. IEEE Transactions on Knowledge and Data Engineering, 22(10), 1345-1359.
Patki, N., Wedge, R., & Veeramachaneni, K. (2016). The synthetic data vault. Paper presented at the 2016 IEEE International Conference on Data Science and Advanced Analytics, Montreal, QC, Canada.
Pavlyshenko, B. M. (2019). Machine-learning models for sales time series forecasting. Data, 4(1), 15.
Piri, S., Delen, D., & Liu, T. (2018). A synthetic informative minority over-sampling (SIMO) algorithm leveraging support vector machine to enhance learning from imbalanced datasets. Decision Support Systems, 106, 15-29.
Robnik-Šikonja, M. (2015). Data generators for learning systems based on RBF networks. IEEE transactions on neural networks and learning systems, 27(5), 926-938.
Scott, G. J., England, M. R., Starms, W. A., Marcum, R. A., & Davis, C. H. (2017). Training deep convolutional neural networks for land–cover classification of high-resolution imagery. IEEE Geoscience and Remote Sensing Letters, 14(4), 549-553.
Sezer, E. A., Nefeslioglu, H. A., & Gokceoglu, C. (2014). An assessment on producing synthetic samples by fuzzy C-means for limited number of data in prediction models. Applied Soft Computing, 24, 126-134.
Sharma, A., & Paliwal, K. (2015). Linear discriminant analysis for the small sample size problem: an overview. International Journal of Machine Learning and Cybernetics, 6(3), 443-454.
Sinn, H.-W. (1980). A rehabilitation of the principle of insufficient reason. The Quarterly Journal of Economics, 94(3), 493-506.
Sáez, J. A., Luengo, J., Stefanowski, J., & Herrera, F. (2015). SMOTE–IPF: Addressing the noisy and borderline examples problem in imbalanced classification by a re-sampling method with filtering. Information Sciences, 291, 184-203.
Song, X., Shao, C., Yang, X., & Wu, X. (2017). Sparse representation-based classification using generalized weighted extended dictionary. Soft Computing, 21(15), 4335-4348.
Van De Schoot, R., Broere, J. J., Perryck, K. H., Zondervan-Zwijnenburg, M., & Van Loey, N. E. (2015). Analyzing small data sets using Bayesian estimation: the case of posttraumatic stress symptoms following mechanical ventilation in burn survivors. European Journal of Psychotraumatology, 6(1), Article: 25216.
Wang, W., Shen, J., & Shao, L. (2017). Video salient object detection via fully convolutional networks. Ieee Transactions on Image Processing, 27(1), 38-49.
Wang, X.-Z., Wang, R., & Xu, C. (2018). Discovering the relationship between generalization and uncertainty by incorporating complexity of classification. IEEE transactions on cybernetics, 48(2), 703-715.
Wang, X.-Z., Xing, H.-J., Li, Y., Hua, Q., Dong, C.-R., & Pedrycz, W. (2015). A Study on Relationship Between Generalization Abilities and Fuzziness of Base Classifiers in Ensemble Learning. IEEE Trans. Fuzzy Systems, 23(5), 1638-1654.
Wang, Y., & Witten, I. H. (1997). Induction of model trees for predicting continuous classes. Paper presented at the the Ninth European Conference on Machine Learning, Hamilton, New Zealand.
Zadeh, L. A. (1965). Fuzzy sets. Information and Control, 8(3), 338-353.
Zadeh, L. A. (1978). Fuzzy sets as a basis for a theory of possibility. Fuzzy Sets and Systems, 1(1), 3-28.
Zhang, H., Li, Y., Jiang, Y., Wang, P., Shen, Q., & Shen, C. (2019). Hyperspectral Classification Based on Lightweight 3-D-CNN With Transfer Learning. IEEE Transactions on Geoscience and Remote Sensing, 57(8), 5813-5828.
論文全文使用權限
  • 同意授權校內瀏覽/列印電子全文服務,於2019-12-06起公開。


  • 如您有疑問,請聯絡圖書館
    聯絡電話:(06)2757575#65773
    聯絡E-mail:etds@email.ncku.edu.tw