||Identifying and Aggregating Disaster-related Messages from Social Media Streams
||Department of Engineering Science
自然或人為災難皆可能導致嚴重的人員傷亡和經濟損失，因此對相關機構而言最具挑戰性的任務之一，就是迅速且準確地掌握災難事件的最新狀況。而隨著近年來社群媒體 (如Twitter、Facebook、PTT等) 的普及，使得災難現場目擊的民眾可以藉由社群媒體發送即時訊息，以迅速聯繫親朋好友並分享相關資訊與照片、影音等，此一資訊傳播方式往往比傳統媒體中的新聞報導更加快速，而成為新型態的資訊管道。在本研究中，我們提出一套方法流程，去改善我們原有的監測災難事件輔助系統在對於社群媒體中識別與災難相關訊息的效果，並從原先視單一訊息為一獨立單位，轉為以事件為一個單位，將描述相同事件的相關資訊進行統整，幫助相關災難應變機構更全面地了解災難現場情況。
Natural and man-made disasters both cause serious casualties and economic damages. Therefore, one of the most challenging tasks for agencies is to rapidly and accurately identify the latest status of a disaster event. With the recent widespread of social media (e.g., Twitter, Facebook, and PTT,) people may easily disseminate and share what they see and hear when they witness a severe accident. These massive and instant messages containing texts, photos and video may form a new type of channel that usually spreads information faster than traditional media do. In this work, we thus propose a scheme that improves the effectiveness of identifying disaster-related messages from social media streams. Also, we further attempt to aggregate messages describing an identical event. This is crucial in helping disaster management agencies to understand the situation in a more comprehensive way.
Chapter 1 Introduction 1
1.1 Motivation and Overview 1
1.2 Contributions of This Work 2
Chapter 2 Preliminaries 3
2.1 Social Media Analysis 3
2.1.1 Basics of Social Media 3
2.1.2 Usage of Social Media 4
2.2 Disaster Event Detection and Tracking Analysis 5
2.2.1 Disaster Management 5
2.2.2 Detecting and Tracking Disaster Events 7
2.2.3 Practical Usage of Disaster Event Detection Techniques 9
Chapter 3 Proposed Scheme for Online Monitoring of Disaster Events 12
3.1 Data Preprocessing 12
3.1.1 Data Source 12
3.1.2 Data Preprocessing 13
3.2 Proposed Scheme of Disaster Events Detection and Tracking 15
3.2.1 Identifying Suspicious Disaster-related Messages 16
3.2.2 Aggregating Actual Disaster-related Messages 18
3.2.3 Disaster Event Tracking and Visualization 20
3.3 Enhancements of the Proposed Scheme 21
Chapter 4 Empirical Studies 23
4.1 Prototype Implementation 23
4.2 Proposed Improved Disaster Event Detection and Tracking System 24
4.3 Case Study 28
4.4 Experimental Process 31
4.4.1 Datasets 31
4.4.2 Enhancement of Identifying Disaster-related Messages 33
4.4.3 Aggregating the Disaster-related Messages 35
Chapter 5 Conclusions and Future Works 37
 A. Aldhaheri and J. Lee, “Event Detection on Large Social Media Using Temporal Analysis,” Proceedings of the IEEE 7th International Conference on Annual Computing and Communication, January 2017.
 A. Ahmed, “Use of Social Media in Disaster Management,” Proceedings of International Conference on Information Systems, 2011.
 Y.-T. Chen, “An Online Supporting Scheme for Monitoring Disaster Events,” Master’s Thesis, Department of Engineering Science, National Cheng Kung University, July 2017.
 T.-Y. Chang, “Information Filtering and Retrieval of Social Media Messages,” Master’s Thesis, Department of Engineering Science, National Cheng Kung University, July 2017.
 S. Choi, “The Analysis Technique of Social Media for Disaster Management,” International Journal of Design & Nature and Ecodynamics, 11(3):396-405, July 2016.
 M.M.F. Cordeiro, “Event Detection: Monitoring and Tracking the Dynamics of Social Networks Communities,” Proceedings of the IEEE 17th International Conference on Mobile Data Management, pages 18-19, June 2016.
 X. Chen, M. Vorvoreanu and K. Madhavan, “Mining Social Media Data for Understanding Students’ Learning Experiences,” IEEE Transactions on Learning Technologies, 7(3):246-259, July 2014.
 E. D'Andrea, P. Ducange, B. Lazzerini and F. Marcelloni, “Real-Time Detection of Traffic from Twitter Stream Analysis,” IEEE Transactions on Intelligent Transportation Systems, 16(4):2269-2283, August 2015.
 X. Dong, D. Mavroeidis, F. Calabrese and P. Frossard, “Multiscale Event Detection in Social Media,” Data Mining and Knowledge Discovery, 29(5):1374-1405, September 2015.
 P. Dewan, M. Gupta, K. Goyal and P. Kumaraguru, “MultiOSN: Realtime Monitoring of Real World Events on Multiple Online Social Media,” Proceedings of the 5th IBM Collaborative Academia Research Exchange Workshop, October 2013.
 L. Dey, S. M. Haque, A. Khurdiya and G. Shroff, “Acquiring Competitive Intelligence from Social Media,” Proceedings of the Joint Workshop on Multilingual OCR and Analytics for Noisy Unstructured Text Data, September 2011.
 N. Gao, L. Gao, Y. He, H. Wang and Q. Sun, “Topic Detection Based on Group Average Hierarchical Clustering,” Proceedings of the International Conference on Advanced Cloud and Big Data, pages 88-92, December 2013.
 H. Gao, G. Barbier and R. Goolsby, “Harnessing the Crowdsourcing Power of Social Media for Disaster Relief,” IEEE Intelligent Systems, 26(3):10-14, June 2011.
 W. He, Z. Shenghua and L. Ling, “Social Media Competitive Analysis and Text Mining: A Case Study in The Pizza Industry,” International Journal of Information Management, 33(3):464-472, June 2013.
 A.L. Hughes and L. Palen, “Twitter Adoption and Use in Mass Convergence and Emergency Events,” Journal on Emergency Management, 6(3-4):248-260, February 2009.
 M. Imran, C. Castillo, F. Diaz and S. Vieweg, “Processing Social Media Messages in Mass Emergency: A Survey,” Proceedings of Journal on ACM Computing Surveys, July 2015.
 Z. Liu, W. Yu, W. Chen, S. Wang and F. Wu, “Short-text Feature Selection for Micro-blog Mining,” Proceedings of the International Conference on Computational Intelligence and Software Engineering, December 2010.
 K. Lau, K. Lee and Y. Ho, “Text Mining for The Hotel Industry,” Cornell Hotel and Restaurant Administration Quarterly, 46(3):344-362, August 2005.
 S. Mazumdar, V. Lanfranchi, N. Ireson and F. Ciravegna, “Visual Analysis of Real-time Social Media for Emergency Response,” Proceedings of the Extended Semantic Web Conference, May 2014.
 A. Marcus, M.S. Bernstein, O. Badar and D.R. Karger, S. Madden and R.C. Miller, “Twitinfo: Aggregating and Visualizing Microblogs for Event Exploration,” Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, May 2011.
 V.Q. Nguyen, H.-J. Yang, K. Kim and A.-R. Oh, “Real-Time Earthquake Detection Using Convolutional Neural Network and Social Data,” Proceedings of the IEEE Third International Conference on Multimedia Big Data, pages 154-157, April 2017.
 A. Nurwidyantoro and E. Winarko, “Event Detection in Social Media: A Survey,” Proceedings of International Conference on ICT for Smart Society, pages 1-5, June 2013.
 S. Patil and S. Kulkarni, “Mining Social Media Data for Understanding Students’ Learning Experiences Using Memetic Algorithm,” Materials Today: Proceedings 5(1):693-699, January 2018.
 Y. Qi, L. Zhou, H. Si, J. Wan and T. Jin, “An Approach to News Event Detection and Tracking Based on Stream of Online News,” Proceedings of the 9th International Conference on Intelligent Human-Machine Systems and Cybernetics, pages 193-196, August 2017.
 W. Sherchan, S. Pervin, C. J. Butler, J. C. Lai, L. Ghahremanlou and B. Han, “Harnessing Twitter and Instagram for Disaster Management,” IBM Journal of Research and Development, 61(6):8:1-8:12, December 2017.
 T. Spielhofer, R. Greenlaw, D. Markham and A. Hahne, “Data Mining Twitter During the UK Floods: Investigating the Potential Use of Social Media in Emergency Management,” Proceedings of the 3rd International Conference on Information and Communication Technologies for Disaster Management, pages 1-6, December 2016.
 G. Siemens and P. Long, “Penetrating the Fog: Analytics in Learning and Education,” Educause Rev, 46(5):30-32, 2011.
 T. Sakaki, M. Okazaki and Y. Matsuo, “Tweet Analysis for Real-Time Event Detection and Earthquake Reporting System Development,” IEEE Transactions on Knowledge and Data Engineering, 25(4):919-931, April 2013.
 T. Sakaki, M. Okazaki and Y. Matsuo, “Earthquake Shakes Twitter Users: Real-time Event Detection by Social Sensors,” Proceedings of the 19th International Conference on World wide web, pages 851-860, April 2010.
 P. Soucy and G. W. Mineau, “Beyond TFIDF Weighting for Text Categorization in the Vector Space Model,” Proceedings of the 19th International Joint Conference on Artificial Intelligence, pages 1130-1135, July 2005.
 G. Valkanas and D. Gunopulos, “Event Detection from Social Media Data,” IEEE Transactions on Computer Society Technical Committee on Data Engineering, 2013.
 Z. Xu, Y. Liu, N. Yen, L. Mei, X. Luo, X. Wei and C. Hu, “Crowdsourcing Based Description of Urban Emergency Events Using Social Media Big Data,” IEEE Transactions on Cloud Computing, pages 1-11, January 2016.
 J. Yin, A. Lampert, M. Cameron, B. Robinson and R. Power, “Using Social Media to Enhance Emergency Situation Awareness,” IEEE Intelligent Systems, 27(6):52-59, February 2012.
 X. Zhou and L. Chen, “Event Detection over Twitter Social Media Streams,” International Journal of Very Large Data Bases, 23(3):381-400, June 2014.
 L. Zheng, C. Shen, L. Tang, C. Zeng, T. Li, S. Luis, and S.-C. Chen, “Data Mining Meets the Needs of Disaster Information Management,” IEEE Transactions on Human-Machine Systems, 43(5):451-464, September 2013.
 Q. Zhao, B. Chen, P. Mitra and B. Chen, “Temporal and Information Flow Based Event Detection from Social Text Streams,” Proceedings of the 22nd National Conference on Artificial Intelligence, 2:1501-1506, July 2007.
 D. F. Maron, “How Social Media Is Changing Disaster Response,” Scientific American, https://www.scientificamerican.com/article/how-social-media-is-changing-disaster-response, June 2013.
 OpView 社群口碑資料庫，http://www.opview.com.tw/socialDB.html.