進階搜尋


下載電子全文  
系統識別號 U0026-2208201315332600
論文名稱(中文) 基於臉部情緒辨識之即時視訊代理的實現
論文名稱(英文) An Implementation of Live Video Conferencing Agent Base on Face Emotion Recognition
校院名稱 成功大學
系所名稱(中) 工程科學系碩博士班
系所名稱(英) Department of Engineering Science
學年度 101
學期 2
出版年 102
研究生(中文) 許斌聖
研究生(英文) Ping-Sheng Hsu
學號 n96981123
學位類別 碩士
語文別 中文
論文頁數 84頁
口試委員 指導教授-王明習
口試委員-鄭國順
口試委員-陳淵琮
中文關鍵字 臉部情緒辨識  臉部偵測  臉部特徵  矩形特徵  視訊會議系統 
英文關鍵字 Facial expression recognition  Face detection  Facial feature  Rectangle feature  Video conference agent 
學科別分類
中文摘要 網路的視訊會議系統已經成為很重要的訊息溝通工具,對於分散在各地的親朋好友,可藉由網路視訊系統互相的見上一面,但是在這個繁忙的時代,不可能每當開啟網路視訊時,對方一定在電腦前與您對談;因此本研究提出基於臉部情緒辨識的視訊代理系統,讓忙碌的人也能夠在無法親自做視訊對談的情況下,讓本系統藉由對對方之情緒辨識結果來回應視訊,以讓希望利用視訊聯繫的對象能夠得到一些回應。本系統實現一個基本的網路視訊系統,讓使用者可以在此系統上利用基本視訊軟體的功能建立互相溝通,提供的功能包含即時文字訊息、視訊影像及聲音;另外,在此基本架構之下更進一步的建立視訊代理系統,當使用者要求視訊的對象不在線上的時候,本系統會切換為視訊代理系統,由代理系統與使用者進行連線,利用視訊接收到的使用者影像進行臉部情緒辨識,將辨識得到的五種情緒-開心、生氣、難過、驚訝及中性表情,以視訊對象預先錄製好的影片內容回應,回應內容針對使用者情緒說一段安撫的話,讓使用者透過視訊代理系統感覺到視訊對象安慰自己,而讓情緒獲得相應的安撫。本研究提出的情緒辨識系統需要先建立一張使用者的中性表情影像,擷取此中性影像的特徵值,再針對由網路攝影機獲得的即時影像計算得到當前表情的特徵值,藉由當前影像與中性表情的特徵值相互比較,進而計算出目前的情緒結果;本系統在實際狀況中測試得到93.5的辨識率,並且能夠以代理系統給予相對應的安撫影像。
英文摘要 Internet video conferencing system has become a very important message communication tool so far. People located at different places can be easily connected via Internet video conferencing system. In this thesis, a live video conferencing agent system has been proposed. The proposed system provides the basic functions, text messages, video images and audio of a video conferencing system. It also designed as an agent system. While the contact person is not on line, the video conferencing system is switched to act as an agent system. The agent system uses facial emotion recognition system to decide the user’s emotion from the images captured by webcam and response the proper information extracted from the data base according to the decided user’s emotion. Five emotions: happy, angry, sad, surprised, and neutral expressions are provided and some corresponding information for each emotion has been created in the data base. The user’s emotion is determined via the comparing the features of captured on line user image with that of user’s neutral expression. This system is tested in real conditions to get recognition rate of 93.5, and the agent system could response the proper information in the result.
論文目次 摘要 I
Abstract II
致謝 III
目錄 IV
圖目錄 VI
表目錄 IX
第一章 緒論 1
1.1 研究動機與目的 1
1.2 相關研究 3
1.3 論文大綱 5
第二章 背景知識 7
2.1 膚色偵測 7
2.1.1 YCbCr色彩空間 7
2.1.2 利用YCbCr對膚色進行偵測 9
2.2 臉部偵測 10
2.2.1 矩形特徵 11
2.2.2 積分影像 14
2.2.3 Adaboost演算法 17
2.2.4 可變子視窗 20
2.2.5 瀑布模型 22
2.3 形態學 23
2.3.1 侵蝕與膨脹及斷開與閉合 24
2.3.2 連通區塊標記(Connected-Component Labeling) 26
2.4 影像縮放 27
2.4.1 最近鄰內插法(Nearest Neighbor Interpolation) 29
2.4.2 雙線性內插法(Bilinear Interpolation) 30
第三章 視訊代理系統及臉部情緒辨識 33
3.1 視訊代理系統架構 33
3.1.1 視訊代理系統-伺服端(Server) 33
3.1.2 視訊代理系統-用戶端 42
3.2 前處理與臉部正規化 42
3.2.1 影像前處理 43
3.2.2 臉部正規化 47
3.3 臉部情緒辨識 51
3.3.1 臉部特徵定位 52
3.3.2 臉部器官狀態 60
3.3.3 情緒分類 67
第四章 實驗結果與討論 70
4.1 實驗環境與介紹 70
4.2 臉部情緒辨識實驗結果 71
4.2.1 特徵點定位 71
4.2.2 臉部情緒辨識 73
4.3 視訊代理的實現 76
第五章 結論與未來研究方向 81
5.1 結論 81
5.2 未來展望 82
參考文獻 83
參考文獻 [1] R. L. Hsu, M. Abdel-Motteleb, A. K. Jain, “Face detection in color images,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 24, no. 5, pp. 696-706, May 2002.
[2] K. Sobottka, I. Pitas, “Extraction of Facial Regions and Features using Color and Shape Information,” Pattern Recognition,1996 ., Proceedings of the 13th International Conference on, vol. 3, pp. 421-425, August 1996.
[3] C. Garcia, G. Tziritas, “Face Detection Using Quantized Skin Color Regions Merging and Wavelet Packet Analysis,” IEEE Transaction on Multimedia, vol 1,no. 3, 1999.
[4] W. S. Yambor, B. A. Draper, J. R. Beveridge, “Analyzing PCA-based Face Recognition Algorithms: Eigenvector Selection and Distance Measures,” In Empirical Methods in Computer Vision. World Scientific Press, 2002.
[5] I. Kotsia, I. Pitas, “Facial Expression Recognition in Image Sequences Using Geometric Deformation Features and Support Vector Machines,” IEEE Transaction on Image Processing, vol. 16,pp.172-187,January 2007.
[6] S. Bashyal, G. K. Venayagamoorthy, “Recognition of facial expressions using Gabor wavelets and learning vector quantization,” Engineering Applications of Artificial Intelligence, vol. 21, pp. 1056-1064, October 2008.
[7] P. Ekman, W. V. Friesen, ” Facial Action Coding System: A Technique for the Measurement of Facial Movement,” San Francisco: Consulting Psychologists Press, 1978.
[8] 胡冠宇, "基於膚色之裸體影像偵測之研究", 碩士論文, 國立成功大學工程科學系碩士班,2004。
[9] A. D. Bimbo, ” Visual information retrieval,” Morgan Kauffman Publishers, Inc., San Francisco, California, 1999.
[10] CCIR, “Encoding parameters of digital television for studios,” CCIR Recommendation 601-2, Int. Radio Consult. Committee, Geneva, Switzerland, 1990.
[11] D. Chai, A. Bouzerdoum, “A Bayesian approach to skin color classification in YCbCr color space,” Machine Learning and Cybernetics, 2002. Proceedings. 2002 International Conference on, IEEE, vol. 2, pp. 1133-1138, Nov. 2002.
[12] P. Viola, M. Jones, “Robust Real-Time Object Detection,” International Journal of Computer Vision, vol. 57, no. 2, pp. 137-154, May 2004.

[13] P. Viola, M. Jones, “Detecting Pedestrians Using Patterns of Motion and Appearance,” International Journal of Computer Vision, vol. 63, no. 2, pp. 153-161, 2005.
[14] C. Papageorgiou, M. Oren, T. Poggio, “A general framework for Object Detection,” In International Conference on Computer Vision, vol. 1, pp. 555-562, January 1998.
[15] Y. Freund, R.E. Schapire, “A Decision-Theoretic Generalization of On-Line Learning and an Application to Boosting,” Journal of Computer and System Sciences, Vol. 55, pp. 119-139, 1997.
[16] G. Woods 原著, 繆紹綱譯, 數位影像處理 Digital image processing, 3rd Ed. 普林斯頓國際有限公司, May 2009.
[17] M. Ilbeygi, H. Shah-Hosseini,” A novel fuzzy facial expression recognition system based on facial feature extraction from color face images,” Engineering Applications of Artificial Intelligence, vol. 25, pp. 130-146, 2012.
[18] Y. Li, S. Wang, Y. Zhao, Q. Ji, “Simultaneous Facial Feature Tracking and Facial Expression Recognition,” IEEE Trans. Image Processing, vol. 22, no. 7, pp. 2559 - 2573, July 2013.
[19] O. Langner, R. G. Dotsch, G. Bijlstra , D. H. J. Wigboldus, “ Presentation and validation of the Rad boud Faces Database,” Cognition and Emotion, vol. 24, no. 8, pp. 1377–1388, 2010.
論文全文使用權限
  • 同意授權校內瀏覽/列印電子全文服務,於2015-08-28起公開。
  • 同意授權校外瀏覽/列印電子全文服務,於2015-08-28起公開。


  • 如您有疑問,請聯絡圖書館
    聯絡電話:(06)2757575#65773
    聯絡E-mail:etds@email.ncku.edu.tw