進階搜尋


   電子論文尚未授權公開,紙本請查館藏目錄
(※如查詢不到或館藏狀況顯示「閉架不公開」,表示該本論文不在書庫,無法取用。)
系統識別號 U0026-2107202016371800
論文名稱(中文) 應用主題Attention與變分自編碼器於評論的個人化摘要
論文名稱(英文) Applying Topic Attention and Variational Autoencoder in Personalized Summarization of Reviews
校院名稱 成功大學
系所名稱(中) 資訊管理研究所
系所名稱(英) Institute of Information Management
學年度 108
學期 2
出版年 109
研究生(中文) 林彥廷
研究生(英文) Yen-Ting Lin
學號 R76074080
學位類別 碩士
語文別 中文
論文頁數 69頁
口試委員 指導教授-王惠嘉
口試委員-高宏宇
口試委員-吳政翰
口試委員-李政德
口試委員-陳偉凡
中文關鍵字 個人化評論摘要  Attention機制  變分自編碼器  主題模型 
英文關鍵字 Personalized summary of reviews  Attention mechanism  Variational autoencoder  Topic model 
學科別分類
中文摘要 隨著Web2.0的出現,帶動電子口碑的快速發展,線上消費者評論漸漸成為不可忽視的存在,越來越多消費者在決定購買前會先觀看評論來了解評價。然而消費者評論的數量日漸增長,大量的評論可以帶給消費者許多資訊,但同時也造成了資訊過載的問題,消費者必須耗費許多時間觀看並且消化。面對上述問題,許多店家已經推出推薦系統,減少消費者選擇購買的時間;然而現成的推薦通常都是依據價格、星等或個人瀏覽紀錄,這些訊息是概括性的總評,不能完全針對消費者感興趣的面向評比。

近年來隨著深度學習的發展,文字摘要也成為熱門的研究領域之一,其可以有效地解決資訊過載的問題;而有別於以往的文字摘要都生成統一的摘要,本研究考量到不同使用者對評論內容重視的面向也不同,加入了個人化在摘要生成中,提出了TA-VAE個人化文字摘要模型。首先利用主題模型從使用者先前對其他店家的評論中萃取使用者在意的面向,而其中考量店家類別生成不同的主題字;再利用生成模型-變分自編碼器(Variational Autoencoder)產生摘要,並且透過Attention機制將使用者特徵納入模型考量;最終生成個人化的摘要,期望能夠幫助每個消費者快速地了解一店家的評價。

根據本研究實驗結果發現,在個人化特徵上,考量店家類別偵測使用者特徵,能夠發現使用者更細微的商品在意面向;而本研究提出的主題Attention能夠有效地增加摘要的品質,並且能生成含有使用者特徵的評論摘要。
英文摘要 With the appearance of the Web 2.0, the electronic word of mouth (eWOM) has developed rapidly. In order to have increased insight regarding product quality, more and more customers watch online reviews before buying. However, the growing number of online reviews also presents an information overload problem. There are too many reviews for customers to consume. Nowadays, many companies utilize a recommendation system to help customers make a decision. Although the system may help customers save their time, it is usually based on the price, stars, or browsing history and does not consider the online reviews.

Text summarization is a good way to solve the information overload problem. Along with the progress of deep learning, text summarization has become an important research area. Contrasting other research, our study considers the customer’s interests in our summary task. We propose a new text summarization method which includes personalization, called TA-VAE. First, we use the topic model to detect customers’ interests from previously written online reviews. In the topic model, we select the company's category and the POS tags to find niche interests. Next, we use a popular generation model, variational autoencoder (VAE), to produce the summary. Last, we integrate the customers’ interests into the model by using the Attention mechanism and generate the personalized summary of reviews.

Based on the results of our research, considering the company's category can benefit the topic model to finding more specific customer interests. Plus, the TA-VAE method not only improves the quality of the summary, but also generates the review's summary included customer's features.
論文目次 第1章 緒論 1
1.1. 研究動機與背景 1
1.2. 研究目的 6
1.3. 研究範圍與限制 6
1.4. 研究流程 7
1.5. 論文大綱 8
第2章 文獻探討 9
2.1. 主題偵測 9
2.1.1 文件分群 9
2.1.2 主題模型 10
2.2. 詞嵌入 11
2.2.1 Word2Vec 11
2.2.2 GloVe 12
2.3. 文字摘要 13
2.3.1 循環神經網路(Recurrent Neural Network, RNN) 14
2.3.2 Sequence to Sequence 17
2.3.3 Variational Autoencoder (VAE) 20
2.3.4 Attention 機制 22
2.3.5 個人化文本摘要 24
2.4. 小結 25
第3章 研究方法 26
3.1. 研究架構 26
3.2. 資料蒐集與前處理模組 28
3.3. 主題偵測模組 30
3.4. 詞嵌入模組 32
3.5. 分類模組 33
3.6. 摘要生成模組 33
3.6.1 VAE – 訓練階段 34
3.6.2 摘要生成階段 39
3.7. 小結 40
第4章 系統建置與驗證 42
4.1. 系統環境建置 42
4.2. 實驗方法 42
4.2.1 資料來源 44
4.2.2 評估指標 45
4.3. 參數設定 46
4.3.1 參數一:主題偵測模組的主題數 46
4.3.2 參數二:評論句子長度 48
4.3.3 參數三:類神經訓練參數 48
4.4. 實驗結果 49
4.4.1 實驗一:在主題偵測中,各因素對於生成主題字的影響 49
4.4.2 實驗二:分類對於生成摘要結果的影響 52
4.4.3 實驗三:Attention對於生成摘要結果的影響 54
4.4.4 實驗四:與其他文本摘要模型的比較 57
4.4.5 實驗五:生成摘要之人工檢測品質標準 58
4.5. 小結 59
第5章 結論與未來方向 61
5.1. 研究成果 61
5.2. 未來發展 62
參考文獻 64
參考文獻 Allan, J., Carbonell, J. G., Doddington, G., Yamron, J., & Yang, Y. (2003). Topic Detection and Tracking Pilot Study Final Report.
Arora, R., & Ravindran, B. (2008). Latent Dirichlet Allocation based Multi-Document Summarization. Paper presented at the Proceedings of the second workshop on Analytics for noisy unstructured text data, Singapore.
Bahdanau, D., Cho, K., & Bengio, Y. (2014). Neural Machine Translation by Jointly Learning to Align and Translate. arXiv preprint arXiv:1409.0473.
Bajt, S. K. (2011). Web 2.0 Technologies: Applications for Community Colleges. New Directions for Community Colleges, 2011(154), 53-62.
Barzilay, R., & Elhadad, M. (1999). Using Lexical Chains for Text Summarization. Advances in automatic text summarization, 111-121.
Beil, F., Ester, M., & Xu, X. (2002). Frequent Term-Based Text Clustering. Paper presented at the Proceedings of the eighth ACM SIGKDD international conference on Knowledge discovery and data mining, Edmonton, Alberta, Canada.
Bengio, Y., Simard, P., & Frasconi, P. (1994). Learning Long-term Dependencies with Gradient Descent is Difficult. IEEE transactions on neural networks, 5(2), 157-166.
Berkovsky, S., Baldwin, T., & Zukerman, I. (2008). Aspect-based Personalized Text Summarization. Paper presented at the International Conference on Adaptive Hypermedia and Adaptive Web-Based Systems, Hannover, Germany.
Bhattacharjee, K., & Petzold, L. (2016). What Drives Consumer Choices? Mining Aspects and Opinions on Large Scale Review Data using Distributed Representation of Words. Paper presented at the 2016 IEEE 16th International Conference on Data Mining Workshops (ICDMW), Barcelona, Spain.
Blei, D. M., Ng, A. Y., & Jordan, M. l. I. (2003). Latent Dirichlet Allocation. Journal of machine Learning research, 3(Jan), 993-1022.
Bowman, S. R., Vilnis, L., Vinyals, O., Dai, A. M., Jozefowicz, R., & Bengio, S. (2015). Generating Sentences from a Continuous Space. arXiv preprint arXiv:1511.06349.
BrightLocal. (2018). Comparison of Local Review Sites: Which Platform is Growing the Fastest?, from https://www.brightlocal.com/research/comparison-of-local-review-sites/
Brody, S., & Elhadad, N. (2010). An Unsupervised Aspect-Sentiment Model for Online Reviews. Paper presented at the Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics, Los Angeles, California.
Cao, Z., Li, W., Li, S., & Wei, F. (2017). Improving Multi-Document Summarization via Text Classification. Paper presented at the Thirty-First AAAI Conference on Artificial Intelligence, California, USA.
Chen, Q., Zhu, X. D., Ling, Z. H., Wei, S., & Jiang, H. (2016). Distraction-Based Neural Networks for Modeling Document. Paper presented at the IJCAI, New York, USA.
Cho, K., Van Merriënboer, B., Bahdanau, D., & Bengio, Y. (2014). On the Properties of Neural Machine Translation: Encoder-Decoder Approaches. arXiv preprint arXiv:1409.1259.
Cho, K., Van Merriënboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014). Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation. arXiv preprint arXiv:1406.1078.
Chopra, S., Auli, M., & Rush, A. M. (2016). Abstractive Sentence Summarization with Attentive Recurrent Neural Networks. Paper presented at the Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, San Diego, California.
Christopher, O. (2015). Understanding LSTM Networks. from https://colah.github.io/posts/2015-08-Understanding-LSTMs/
Chu, E., & Liu, P. (2019). MeanSum: A Neural Model for Unsupervised Multi-Document Abstractive Summarization. Paper presented at the International Conference on Machine Learning, Long Beach, USA.
Chung, J., Gulcehre, C., Cho, K. H., & Bengio, Y. (2014). Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling. arXiv preprint arXiv:1412.3555.
D’Avanzo, E., & Pilato, G. (2015). Mining Social Network Users Opinions’ to Aid Buyers’ Shopping Decisions. Computers in Human Behavior, 51, 1284-1294.
Dang, H. T. (2005). Overview of DUC 2005. Paper presented at the Proceedings of the Document Understanding Conference, Vancouver, British Columbia, Canada.
Daugherty, T., Eastin, M. S., & Bright, L. (2008). Exploring Consumer Motivations for Creating User-Generated Content. Journal of interactive advertising, 8(2), 16-25.
Deerwester, S., Dumais, S. T., Furnas, G. W., Landauer, T. K., & Harshman, R. (1990). Indexing by Latent Semantic Analysis. Journal of the American society for information science, 41(6), 391-407.
Doersch, C. (2016). Tutorial on Variational Autoencoders. arXiv preprint arXiv:1606.05908.
Eslami, S. P., Ghasemaghaei, M., & Hassanein, K. (2018). Which Online Reviews do Consumers Find Most Helpful? A Multi-Method Investigation. Decision Support Systems, 113, 32-42.
Fan, & Fuel. (2016). No online customer reviews means BIG problems in 2017. from https://fanandfuel.com/no-online-customer-reviews-means-big-problems-2017/
Hennig-Thurau, T., Walsh, G., & Walsh, G. (2003). Electronic Word-of-Mouth: Motives for and Consequences of Reading Customer Articulations on the Internet. International journal of electronic commerce, 8(2), 51-74.
Hinton, G. E., & Salakhutdinov, R. R. (2006). Reducing the Dimensionality of Data with Neural Networks. science, 313(5786), 504-507.
Hinton, G. E., & Zemel, R. S. (1994). Autoencoders, Minimum Description Length and Helmholtz Free Energy. Paper presented at the Advances in neural information processing systems, Denver, Colorado.
Hochreiter, S., & Schmidhuber, J. (1997). Long Short-term Memory. Neural computation, 9(8), 1735-1780.
Hofmann, T. (1999). Probabilistic Latent Semantic Analysis. Paper presented at the Proceedings of the Fifteenth conference on Uncertainty in artificial intelligence, Stockholm, Sweden.
Hu, M., & Liu, B. (2004). Mining and Summarizing Customer Reviews. Paper presented at the Proceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining, Seattle, WA, USA.
Hu, Y. H., Chen, Y. L., & Chou, H. L. (2017). Opinion Mining from Online Hotel Reviews – A Text Summarization Approach. Information Processing & Management, 53(2), 436-449.
Jang, M., Seo, S., & Kang, P. (2019). Recurrent Neural Network-based Semantic Variational Autoencoder for Sequence-to-sequence Learning. Information Sciences, 490, 59-73.
Jeong, E., & Jang, S. S. (2011). Restaurant Experiences Triggering Positive Electronic Word-of-Mouth (eWOM) Motivations. International Journal of Hospitality Management, 30(2), 356-366.
Kar, M., Nunes, S., & Ribeiro, C. (2015). Summarization of Changes in Dynamic Text Collections Using Latent Dirichlet Allocation Model. Information Processing & Management, 51(6), 809-833.
Kim, Y., & Srivastava, J. (2007). Impact of Social Influence in E-Commerce Decision Making. Paper presented at the Proceedings of the ninth international conference on Electronic commerce, Minneapolis, MN,USA.
Kingma, D. P., & Welling, M. (2013). Auto-encoding Variational Bayes. arXiv preprint arXiv:1312.6114.
Lebanoff, L., Song, K., & Liu, F. (2018). Adapting the Neural Encoder-Decoder Framework from Single to Multi-Document summarization. arXiv preprint arXiv:1808.06218.
Lee, M., & Youn, S. (2009). Electronic Word of Mouth (eWOM) How eWOM Platforms Influence Consumer Product Judgement. International Journal of Advertising, 28(3), 473-499.
Li, J., Li, H., & Zong, C. (2019). Towards Personalized Review Summarization via User-Aware Sequence Network. Paper presented at the Proceedings of the AAAI Conference on Artificial Intelligence, Honolulu, Hawaii, USA.
Lin, C.-Y. (2004). ROUGE: A Package for Automatic Evaluation of summaries.
Lin, C., & He, Y. (2009). Joint Sentiment/Topic Model for Sentiment Analysis. Paper presented at the Proceedings of the 18th ACM conference on Information and knowledge management, Hong Kong, China.
Lloret, E., Boldrini, E., Vodolazova, T., Martínez-Barco, P., Muñoz, R., & Palomar, M. (2015). A Novel Concept-level Approach for Ultra-concise Opinion Summarization. Expert Systems with Applications, 42(20), 7148-7156.
Luca, M. (2016). Reviews, Reputation, and Revenue: The Case of Yelp. com. Com (March 15, 2016). Harvard Business School NOM Unit Working Paper(12-016).
Móro, R. (2012). Personalized Text Summarization based on Important Terms Identification. Paper presented at the 2012 23rd International Workshop on Database and Expert Systems Applications, Vienna, Austria.
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013). Efficient Estimation of Word Representations in Vector Space. arXiv preprint arXiv:1301.3781.
Mikolov, T., Le, Q. V., & Sutskever, I. (2013). Exploiting Similarities among Languages for Machine Translation. arXiv preprint arXiv:1309.4168.
Mikolov, T., Sutskever, I., Chen, K., Corrado, G. S., & Dean, J. (2013). Distributed Representations of Words and Phrases and Their Compositionality. Paper presented at the Advances in neural information processing systems, Harrahs and Harveys, Lake Tahoe.
Naili, M., Chaibi, A. H., & Ghezala, H. H. B. (2017). Comparative Study of Word Embedding Methods in Topic Segmentation. Procedia Computer Science, 112, 340-349.
Nakayama, M., & Wan, Y. (2019). The Cultural Impact on Social Commerce: A Sentiment Analysis on Yelp Ethnic Restaurant Reviews. Information & Management, 56(2), 271-279.
Nallapati, R., Zhou, B., Gulcehre, C., & Xiang, B. (2016). Abstractive Text Summarization Using Sequence-to-Sequence Rnns and Beyond. arXiv preprint arXiv:1602.06023.
Nayeem, M. T., Fuad, T. A., & Chali, Y. (2018). Abstractive Unsupervised Multi-document Summarization Using Paraphrastic Sentence Fusion. Paper presented at the Proceedings of the 27th International Conference on Computational Linguistics, Santa Fe, New Mexico, USA.
Nematzadeh, A., Meylan, S. C., & Griffiths, T. L. (2017). Evaluating Vector-Space Models of Word Representation, or, The Unreasonable Effectiveness of Counting Words Near Other Words. Paper presented at the CogSci, London, UK.
Pennington, J., Socher, R., & Manning, C. (2014). Glove: Global vectors for word representation. Paper presented at the Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), Doha, Qatar.
Rossiello, G., Basile, P., & Semeraro, G. (2017). Centroid-based Text Summarization through Compositionality of Word Embeddings. Paper presented at the Proceedings of the MultiLing 2017 Workshop on Summarization and Summary Evaluation Across Source Types and Genres.
Rush, A. M., Chopra, S., & Weston, J. (2015). A Neural Attention Model for Abstractive Sentence Summarization. arXiv preprint arXiv:1509.00685.
Sak, H., Senior, A., & Beaufays, F. (2014). Long Short-Term memory Recurrent Neural Network Architectures for Large Scale Acoustic Modeling. Paper presented at the Fifteenth annual conference of the international speech communication association, Singapore.
Salton, G., Wong, A., & Yang, C.-S. (1975). A Vector Space Model for Automatic Indexing. Communications of the ACM, 18(11), 613-620.
Schmidhuber, J. (2015). Deep Learning in Neural Networks: An Overview. Neural networks, 61, 85-117.
Sen, S., & Lerman, D. (2007). Why are you Telling me this? An Examination into Negative Consumer Reviews on the Web. Journal of Interactive Marketing, 21(4), 76-94.
Stauss, B. (1997). Global Word of Mouth: Service Bashing on the Internet is a Thorny Issue. Marketing Management, 6(3), 28.
Steinbach, M., Karypis, G., & Kumar, V. (2000). A Comparison of Document Clustering Techniques. Paper presented at the KDD workshop on text mining, Boston, MA, USA.
Stevens, K., Kegelmeyer, P., Andrzejewski, D., & Buttler, D. (2012). Exploring Topic Coherence over Many Models and Many Topics. Paper presented at the Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, Jeju Island, Korea.
Sutskever, I., Vinyals, O., & Le, Q. V. (2014). Sequence to Sequence Learning with Neural Networks. Paper presented at the Advances in neural information processing systems, Montreal, Canada.
Tay, Y., Luu, A. T., & Hui, S. C. (2018). Multi-Pointer Co-Attention Networks for Recommendation. Paper presented at the Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining.
Titov, I., & McDonald, R. (2008). Modeling Online Reviews with Multi-Grain Topic Models. Paper presented at the Proceedings of the 17th international conference on World Wide Web, New York, NY, USA.
Wang, C., Chen, G., & Wei, Q. (2018). A Temporal Consistency Method for Online Review ranking. Knowledge-Based Systems, 143, 259-270.
Xu, W., Liu, X., & Gong, Y. (2003). Document Clustering Based on Non-Negative Matrix Factorization. Paper presented at the Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval, Toronto, Canada.
Yan, R., Nie, J. Y., & Li, X. (2011). Summarize What You Are Interested In: An Optimization Framework for Interactive Personalized Summarization. Paper presented at the Proceedings of the conference on empirical methods in natural language processing, Edinburgh, United Kingdom.
Yang, B., Liu, Y., Liang, Y., & Tang, M. (2019). Exploiting User Experience from Online Customer Reviews for Product Design. International Journal of Information Management, 46, 173-186.
Yao, J., Wan, X., & Xiao, J. (2017). Recent Advances in Document Summarization. Knowledge and Information systems, 53(2), 297-336.
Yoo, C. W., Sanders, G. L. a., & Moon, J. (2013). Exploring the Effect of e-WOM Participation on e-Loyalty in E-commerce. Decision Support Systems, 55(3), 669-678.
Zhang, D., Ma, J., Niu, X., Gao, S., & Song, L. (2012). Multi-Document Summarization of Product Reviews. Paper presented at the 2012 9th International Conference on Fuzzy Systems and Knowledge Discovery, Sichuan, China.
Zhang, D., Xu, H., Su, Z., & Xu, Y. (2015). Chinese Comments Sentiment Classification based on Word2vec and SVMperf. Expert Systems with Applications, 42(4), 1857-1863.
Zhang, H., Ma, Z. C. W., & Cai, Q. (2003). A Study for Documents Summarization based on Personal Annotation. Paper presented at the Proceedings of the HLT-NAACL 03 on Text summarization workshop-Volume 5, Stroudsburg, PA, USA.
論文全文使用權限
  • 同意授權校內瀏覽/列印電子全文服務,於2025-07-01起公開。
  • 同意授權校外瀏覽/列印電子全文服務,於2025-07-01起公開。


  • 如您有疑問,請聯絡圖書館
    聯絡電話:(06)2757575#65773
    聯絡E-mail:etds@email.ncku.edu.tw