進階搜尋


 
系統識別號 U0026-2507201923100300
論文名稱(中文) 使用階層注意力網路改善雙調節矩陣分解法
論文名稱(英文) Improving Dual-Regularized Matrix Factorization Using Hierarchical Attention Networks
校院名稱 成功大學
系所名稱(中) 工業與資訊管理學系碩士在職專班
系所名稱(英) Department of Industrial and Information Management (on the job class)
學年度 107
學期 2
出版年 108
研究生(中文) 楊潓賢
研究生(英文) Hui-Xian Yang
電子信箱 johnodin99johnodin99@gmail.com
學號 R37051134
學位類別 碩士
語文別 中文
論文頁數 54頁
口試委員 指導教授-劉任修
口試委員-胡政宏
口試委員-張裕清
中文關鍵字 深度學習  階層式注意網路  矩陣分解  推薦系統 
英文關鍵字 Deep Learning  Hierarchical Attention Network  Matrix Factorization  Recommendation System 
學科別分類
中文摘要 隨著消費者的需求日新月異,商品多樣化變成了趨勢,但面對琳瑯滿目的 商品,消費者常手足無措,不知如何選擇,而推薦演算法能有效解決此問題, 所以商務網站的推薦系統也越來越顯重要。主要常見的推薦系統可分成四大 類,協同過濾推薦、基於內容推薦、基於知識推薦與整合各別優點的混合型推 薦,而前三者中又以協同過濾的推薦效果最好,所以成為了主流,在一般常用 的協同過濾推薦模型(Model-Based Recommendation)效果雖然優異,但大都是以 評價為主要資訊,未充分使用到其他資訊來幫助推薦的品質,因此本論文提出 了一個同時考量了數值評價與文字評論的商品推薦研究,整合HAN(Hierarchical Attention Networks)模型與PMF(Probabilistic Matrix Factorization)來提高推薦正確 率,利用HAN模型擅長處理序列性資料的特性來擷取文字特徵,並使用注意力 機制讓每個文字賦予不一樣的權重,抽取用戶與商品評論權重較高的單字來 代表特徵,而根據本論文的實驗結果顯示本研究所提出的HAN-PMF模型相較 於Wu et al.(2018)提出的DRMF和Mnih and Salakhutdinov(2008)提出的PMF模型為 佳,證明本模型有一定品質的推薦準確率。
英文摘要 Recently, it is getting tougher and tougher to come to a decision because we get so much information through the internet every hour. Therefore, recom- mender systems (RSs) are becoming more and more important. They can help people choose effectively by suggesting suitable options to people. Most of the hy- brid RSs extensively utilize review text information to improve their performance. However, it is still a challenge for RSs to extract the most informative features from huge numbers of reviews.
In this thesis, a new RS called the HAN-PMF model, which combines prob- abilistic matrix factorization and hierarchical attention networks, is proposed. The aim is to learn which kinds of items users like by extracting ratings and reviews. To get an optimal model and parameters, the system uses the Amazon 7 dataset to train the HAN-PMF model and the root mean square error as the evaluation metric. The difference between the results is influenced by the random seed, so we fix the seed and conduct the experiment five times to ensure experimental reliability. Our result shows a 1.47% improvement compared with DRMF and a 13.05% improve- ment compared with PMF. We then extract the 20 most important words to explain customer insight.
論文目次 摘要 i
EXTENDED ABSTRACT ii
誌謝 vi
目錄 vii
表目錄 ix
圖目錄 x
1 緒論 1
1.1 背景與動機......2
1.2 研究目的.......3
1.3 貢獻......3
1.4 論文架構.......4
2 相關文獻探討 5
2.1 推薦系統.......5
2.1.1 基於協同過濾推薦系統.....6
2.1.1.1 PMF.....9
2.1.2 混合模型推薦系統 ....13
2.2 深度學習.......16
2.2.1 GRU ......16
vii
2.2.2 Attention .....17
2.3 小結......18
3 研究方法 19
3.1 問題描述.......19
3.2 Hierarchical Attention Networks神經網路架構 .....20
3.2.1 WordEncoder .....24
3.2.2 WordAttention.....26
3.2.3 SentenceEncoder.....26
3.2.4 SentenceAttention ....27
3.2.5 小結 ......28
3.3 機率矩陣分解法結合階層式深度學習架構 ...29
4 實驗與分析 35
4.1 資料集介紹與資料前處理....35
4.2 衡量指標.......37
4.3 實驗結果.......37
5 結論與未來發展 47
References 48
參考文獻 Aditya, P., Budi, I., and Munajat, Q. (2016). A comparative analysis of memory-based and model-based collaborative filtering on the implementation of recommender sys- tem for e-commerce in indonesia: A case study pt x. In 2016 International Con- ference on Advanced Computer Science and Information Systems (ICACSIS), pages 303–308. IEEE.
Almahairi, A., Kastner, K., Cho, K., and Courville, A. (2015). Learning distributed representations from reviews for collaborative filtering. In Proceedings of the 9th ACM Conference on Recommender Systems, pages 147–154. ACM.
Babu, M. S. P. and Kumar, B. R. S. (2011). An implementation of the user-based collaborative filtering algorithm. IJCSIT) International Journal of Computer Science and Information Technologies, 2(3):1283–1286.
Bahdanau, D., Cho, K., and Bengio, Y. (2014). Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473.
Bao, Y., Fang, H., and Zhang, J. (2014). Topicmf: Simultaneously exploiting ratings and reviews for recommendation. In Twenty-Eighth AAAI conference on artificial intelligence.
Barbosa, L. N., Gemmell, J., Horvath, M., and Heimfarth, T. (2018). Distributed user- based collaborative filtering on an opportunistic network. In 2018 IEEE 32nd Inter- national Conference on Advanced Information Networking and Applications (AINA), pages 266–273. IEEE.
48
Basaldella, M., Antolli, E., Serra, G., and Tasso, C. (2018). Bidirectional lstm recurrent neural network for keyphrase extraction. In Italian Research Conference on Digital Libraries, pages 180–187. Springer.
Blei, D. M., Ng, A. Y., and Jordan, M. I. (2003). Latent dirichlet allocation. Journal of machine Learning research, 3(Jan):993–1022.
Bouvrie, J. (2006). Notes on convolutional neural networks.
Burke, R. (2000). Knowledge-based recommender systems. Encyclopedia of library
and information systems, 69(Supplement 32):175–186.
Burke, R. (2002). Hybrid recommender systems: Survey and experiments. User mod-
eling and user-adapted interaction, 12(4):331–370.
Chang, T.-M. and Hsiao, W.-F. (2011). Model-based collaborative filtering to han- dle data reliability and ordinal data scale. In 2011 Eighth International Conference on Fuzzy Systems and Knowledge Discovery (FSKD), volume 3, pages 2065–2069. IEEE.
Chung, J., Gulcehre, C., Cho, K., and Bengio, Y. (2014). Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv preprint arXiv:1412.3555.
D’Addio, R. M. and Manzato, M. G. (2015). A sentiment-based item description ap- proach for knn collaborative filtering. In Proceedings of the 30th annual ACM sym- posium on applied computing, pages 1060–1065. ACM.
49

Diao, Q., Qiu, M., Wu, C.-Y., Smola, A. J., Jiang, J., and Wang, C. (2014). Jointly modeling aspects, ratings and sentiments for movie recommendation (jmars). In Pro- ceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 193–202. ACM.
Elman, J. L. (1990). Finding structure in time. Cognitive science, 14(2):179–211. Esmaeilzadeh, S., Peh, G. X., and Xu, A. (2019). Neural abstractive text summarization
and fake news detection. arXiv preprint arXiv:1904.00788.
Fakhri, A. A., Baizal, Z., and Setiawan, E. B. (2019). Restaurant recommender sys- tem using user-based collaborative filtering approach: A case study at bandung raya region. In Journal of Physics: Conference Series, volume 1192, page 012023. IOP Publishing.
Hochreiter, S. and Schmidhuber, J. (1997). Long short-term memory. Neural computa- tion, 9(8):1735–1780.
Hui, S., Pengyu, L., and Kai, Z. (2011). Improving item-based collaborative filtering recommendation system with tag. In 2011 2nd International Conference on Artifi- cial Intelligence, Management Science and Electronic Commerce (AIMSEC), pages 2142–2145. IEEE.
Jiang, S., Qian, X., Shen, J., Fu, Y., and Mei, T. (2015). Author topic model-based collaborative filtering for personalized poi recommendations. IEEE transactions on multimedia, 17(6):907–918.
50

Kim, D., Park, C., Oh, J., Lee, S., and Yu, H. (2016). Convolutional matrix factoriza- tion for document context-aware recommendation. In Proceedings of the 10th ACM Conference on Recommender Systems, pages 233–240. ACM.
Koren, Y. (2008). Factorization meets the neighborhood: a multifaceted collaborative filtering model. In Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 426–434. ACM.
Koren, Y., Bell, R., and Volinsky, C. (2009). Matrix factorization techniques for recom- mender systems. Computer, (8):30–37.
Lee, D. D. and Seung, H. S. (2001). Algorithms for non-negative matrix factorization. In Advances in neural information processing systems, pages 556–562.
Ling, G., Lyu, M. R., and King, I. (2014). Ratings meet reviews, a combined approach to recommend. In Proceedings of the 8th ACM Conference on Recommender systems, pages 105–112. ACM.
Luong, M.-T., Pham, H., and Manning, C. D. (2015). Effective approaches to attention- based neural machine translation. arXiv preprint arXiv:1508.04025.
McAuley, J. and Leskovec, J. (2013). Hidden factors and hidden topics: understanding rating dimensions with review text. In Proceedings of the 7th ACM conference on Recommender systems, pages 165–172. ACM.
Mnih, A. and Salakhutdinov, R. R. (2008). Probabilistic matrix factorization. In Ad- vances in neural information processing systems, pages 1257–1264.
51

Qaiser, S. and Ali, R. (2018). Text mining: Use of tf-idf to examine the relevance of words to documents. International Journal of Computer Applications, 975:8887.
Rendle, S. (2010). Factorization machines. In 2010 IEEE International Conference on Data Mining, pages 995–1000. IEEE.
Sarwar, B. M., Karypis, G., Konstan, J. A., Riedl, J., et al. (2001). Item-based collabo- rative filtering recommendation algorithms. Www, 1:285–295.
Schuster, M. and Paliwal, K. K. (1997). Bidirectional recurrent neural networks. IEEE Transactions on Signal Processing, 45(11):2673–2681.
Su, X. and Khoshgoftaar, T. M. (2009). A survey of collaborative filtering techniques. Advances in artificial intelligence, 2009.
Sun, Z. and Luo, N. (2010). A new user-based collaborative filtering algorithm combin- ing data-distribution. In 2010 International Conference of Information Science and Management Engineering, volume 2, pages 19–23. IEEE.
Tan, Y., Zhang, M., Liu, Y., and Ma, S. (2016). Rating-boosted latent topics: Un- derstanding users and items with ratings and reviews. In IJCAI, volume 16, pages 2640–2646.
Vincent, P., Larochelle, H., Lajoie, I., Bengio, Y., and Manzagol, P.-A. (2010). Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion. Journal of machine learning research, 11(Dec):3371–3408.
52

Wang, C. and Blei, D. M. (2011). Collaborative topic modeling for recommending scientific articles. In Proceedings of the 17th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 448–456. ACM.
Wang, H., Wang, N., and Yeung, D.-Y. (2015). Collaborative deep learning for recom- mender systems. In Proceedings of the 21th ACM SIGKDD international conference on knowledge discovery and data mining, pages 1235–1244. ACM.
Wasilewski, J. and Hurley, N. (2018). Intent-aware item-based collaborative filtering for personalised diversification. In Proceedings of the 26th Conference on User Model- ing, Adaptation and Personalization, pages 81–89. ACM.
Wu, H., Zhang, Z., Yue, K., Zhang, B., He, J., and Sun, L. (2018). Dual-regularized ma- trix factorization with deep neural networks for recommender systems. Knowledge- Based Systems, 145:46–58.
Xue, F., He, X., Wang, X., Xu, J., Liu, K., and Hong, R. (2019). Deep item-based collaborative filtering for top-n recommendation. ACM Transactions on Information Systems (TOIS), 37(3):33.
Yang, Z., Yang, D., Dyer, C., He, X., Smola, A., and Hovy, E. (2016). Hierarchical at- tention networks for document classification. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1480–1489.
Zhang, S., Yao, L., Sun, A., and Tay, Y. (2019). Deep learning based recommender system: A survey and new perspectives. ACM Computing Surveys (CSUR), 52(1):5.
53

Zhang, Y., Jin, R., and Zhou, Z.-H. (2010). Understanding bag-of-words model: a statistical framework. International Journal of Machine Learning and Cybernetics, 1(1-4):43–52.
Zheng, L., Noroozi, V., and Yu, P. S. (2017). Joint deep modeling of users and items using reviews for recommendation. In Proceedings of the Tenth ACM International Conference on Web Search and Data Mining, pages 425–434. ACM.
論文全文使用權限
  • 同意授權校內瀏覽/列印電子全文服務,於2019-07-31起公開。
  • 同意授權校外瀏覽/列印電子全文服務,於2019-07-31起公開。


  • 如您有疑問,請聯絡圖書館
    聯絡電話:(06)2757575#65773
    聯絡E-mail:etds@email.ncku.edu.tw