||Study on Fusion of Omni-directional and PTZ Cameras for Moving Object Detection and Tracking
||Department of Electrical Engineering
vision surveillance system
In recent years, vision surveillance and monitoring systems are issues of high concern. The research topics include computer vision, communication among multi-cameras and data fusion. A panoramic vision surveillance system consists of an omni-directional and a PTZ cameras is implemented in this thesis. With the full field of view (FOV) property of the omni-directional camera, the path and direction of moving objects in the scene can be both tracked and predicted by multi-cue tracking and a Kalman filter, respectively. For information fusion, the architecture of the control would be divided into master-slave control and cooperative control according to the types of PTZ camera movement. In master-slave control, the movement of PTZ camera depends on the moving object azimuth of an omni-directional camera. In cooperative control, a PTZ camera would obtain its target template by SDG background compensation after receiving the omni-directional camera command. Subsequently, the PTZ camera can automatically track the target once successfully acquiring the features of the target, and continuously lock the target into the center of the screen. From the experimental results, the surveillance system can continuously detect and track in an indoor environment. As a result, a panoramic surveillance system combining an omni-directional and a PTZ cameras is successfully implemented in this thesis.
第一章 緒論 1
1.1 前言 1
1.2 研究動機與目的 4
1.3 文獻回顧 4
1.4 本文架構 6
第二章 攝影機簡介 7
2.1 前言 7
2.2 PTZ攝影機 8
2.2.1 一般攝影機之透視投影模型 8
2.2.2 一般攝影機之內部參數校正 10
2.3 全景攝影機 11
2.3.1 全景攝影機簡介 12
2.3.2 全景攝影機之單一視點（SVP）限制 13
2.3.3 全景攝影機之投影模型與內部參數 15
2.3.4 全景攝影機校正之內部參數 18
2.4 雙攝影機校正 19
2.4.1 雙攝影機之對應幾何 20
第三章 全景攝影機目標物偵測與追蹤 26
3.1 前言 26
3.2 全景攝影機之前景偵測 26
3.2.1 前景偵測 26
3.2.2 適應性背景減去法 29
3.3 全景攝影機之多目標物追蹤 32
3.3.1 全景攝影機影像分析 33
3.3.2 目標物追蹤機制移動物之連通物件分析 34
3.3.3 目標物追蹤預測 42
18.104.22.168卡爾曼濾波器（Kalman Filter） 43
第四章 PTZ攝影機目標物偵測與追蹤 46
4.1 前言 46
4.2 PTZ攝影機之控制架構 47
4.2.1 主被動式控制架構 47
4.2.2 合作式控制架構 47
4.3 PTZ攝影機之目標物偵測 51
4.3.1 背景補償法 51
4.3.2 SDG模型 58
4.4 PTZ攝影機之目標物追蹤 64
4.4.1 雙重目標物特徵比對法 64
第五章 雙攝影機視覺監控系統之實現 71
5.1 實驗硬體設備簡介 71
5.1.1 電腦設備 71
5.1.2 全景攝影機 72
5.1.3 Sony EVI-D70 PTZ攝影機 73
5.1.4 UPMOST UPG305 USB影像擷取卡 75
5.2 系統介面設計與運作流程 77
5.3 實驗結果 78
5.3.1 全景攝影機與PTZ攝影機之對應關係 78
5.3.2 全景攝影機多目標物追蹤 81
5.3.3 全景攝影機與PTZ攝影機之視覺監控系統 89
第六章 結論與建議 97
6.1 結論 97
6.2 未來展望與建議 98
 G. R. Bradski and A. Kaehler, Learning opencv: O'Reilly, 2008.
 I. Haritaoglu, D. Harwood, and L. S. Davis, "W4: Real-time surveillance of people and their activities," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 22, pp. 809-830, 2000.
 B. K. P. Horn and B. G. Schunck, "Determining optical flow," Artificial intelligence, vol. 17, pp. 185-203, 1981.
 A. J. Lipton, H. Fujiyoshi, and R. S. Patil, "Moving target classification and tracking from real-time video," presented at the Proceedings of the IEEE Workshop on Applications of Computer Vision, Princeton, NJ , USA, 1998.
 B. D. Lucas and T. Kanade, "An iterative image registration technique with an application to stereo vision," in Proceedinds of the International Joint Conferenrce on Artificial Intelligence, 1981, pp. .674 - 679
 I. Sobel and G. Feldman, "A 3x3 isotropic gradient operator for image processing," in Stanford Artificial Project, 1968, pp. 271-272.
 W. T. Freeman and M. Roth, "Orientation histograms for hand gesture recognition," in Proceedings of th International Workshop on Automatic Face and Gesture-Recognition, Zurich, 1995, pp. 296-301.
 M. J. Swain and D. H. Ballard, "Color indexing," International journal of computer vision, vol. 7, pp. 11-32, 1991.
 D. Murray and A. Basu, "Motion tracking with an active camera," IEEE Transactions on Pattern Analysis and Machine Intelligence, pp. 449-459, 1994.
 J. Hill and W. T. Park, "Real time control of a robot with a mobile camera," in Proceedings of 9th ISIR, 1979, pp. 233-246.
 R. Collins, A. Lipton, T. Kanade, H. Fujiyoshi, D. Duggins, Y. Tsin, D. Tolliver, N. Enomoto, O. Hasegawa, and P. Burt, "A system for video surveillance and monitoring," in Proceedings of American Nuclear Society Eighth Institute Topical Meeting Robotic and Remote Systems, 1999, pp. 25-29.
 P. Amnuaykanjanasin, S. Aramvith, and T. Chalidabhongse, "Real-time face identification using two cooperative active cameras," in Proceedings of the International Conference on Control, Automation, Robotics and Vision, Singapore, 2006, pp. 1-6.
 R. T. Collins, A. J. Lipton, H. Fujiyoshi, and T. Kanade, "Algorithms for cooperative multisensor surveillance," Proceedings of the IEEE, vol. 89, pp. 1456-1477, 2001.
 H. El-Alfy, D. Jacobs, and L. Davis, "Assigning cameras to subjects in video surveillance systems," in Proceedings of the International Conference on Robotics and Automation, Kobe, 2009, pp. 837-843.
 O. Javed and M. Shah, Automated multi-camera surveillance: algorithms and practice vol. 10: Springer-Verlag New York Inc, 2008.
 S. Araki, T. Matsuoka, N. Yokoya, and H. Takemura, "Real-time tracking of multiple moving object contours in a moving camera image sequence," IEICE transactions on information and systems, vol. 83, pp. 1583-1591, 2000.
 T. E. Boult, X. Gao, R. Micheals, and M. Eckmann, "Omni-directional visual surveillance," Image and Vision Computing, vol. 22, pp. 515-534, 2004.
 X. Chen and J. Yang, "Towards monitoring human activities using an omnidirectional camera," in Proceedings of the IEEE International Conference on Multimodal Interfaces, 2002, pp. 423-428.
 J. W. Davis, A. M. Morison, and D. D. Woods, "An adaptive focus-of-attention model for video surveillance and monitoring," Machine Vision and Applications, vol. 18, pp. 41-64, 2007.
 M. L. Wang and H. Y. Lin, "Object recognition from omnidirectional visual sensing for mobile robot applications," in Proceedinds of the IEEE International Conference on Systems, Man, and Cybernetics, 2009, pp. 1941-1946.
 Y. C. Chung, C. Wang, J. Wang, S. Lin, and S. Chen, "Integration of omnidirectional and movable cameras for indoor surveillance," in Proceedings of the Conference on Computer Vision, Graphics Image Process, 2004, p. 1.
 C. H. Chen, Y. Yao, D. Page, B. Abidi, A. Koschan, and M. Abidi, "Heterogeneous fusion of omnidirectional and PTZ cameras for multiple object tracking," IEEE Transactions on Circuits and Systems for Video Technology, vol. 18, pp. 1052-1063, 2008.
 C. S. Fahn and C. S. Lo, "A high-definition human face tracking system using the fusion of omni-directional and PTZ cameras mounted on a mobile robot," in Proceedings of the International Conference on Industrial Electronics and Applications, 2010, pp. 6-11.
 R. Khoshabeh, T. Gandhi, and M. M. Trivedi, "Multi-camera based traffic flow characterization & classification," in Proceedings of the IEEE Intelligent Transportation Systems Conference, Seattle, 2007, pp. 259-264.
 Y. Lu and S. Payandeh, "Cooperative hybrid multi-camera tracking for people surveillance," Canadian Journal of Electrical and Computer Engineering, vol. 33, pp. 145-152, 2008.
 G. Scotti, L. Marcenaro, C. Coelho, F. Selvaggi, and C. Regazzoni, "Dual camera intelligent sensor for high definition 360 degrees surveillance," in IEE Proceedings Vision Image and Signal Processing, 2005, pp. 250-257.
 M. S. Yu, H. Wu, and H. Y. Lin, "A visual surveillance system for mobile robot using omnidirectional and PTZ cameras," in Proceedings of SICE Annual Conference, 2010, pp. 37-42.
 王一洋, "結合廣角攝影機與旋轉式攝影機之視訊監控系統," 碩士論文, 資訊工程學系, 國立台灣大學, 2004.
 K. Yamazawa, Y. Yagi, and M. Yachida, "Omnidirectional imaging with hyperboloidal projection," in Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, Yokohama , Japan 1993, pp. 1029-1034.
 S. Baker and S. K. Nayar, "A theory of single-viewpoint catadioptric image formation," International journal of computer vision, vol. 35, pp. 175-196, 1999.
 R. Boutteau, X. Savatier, J. Ertaud, and B. Mazari, "An omnidirectional stereoscopic system for mobile robot navigation," in Proceedings of the IEEE International Workshop on Robotic and Sensors Environments, Ottawa, 2008, pp. 138-143.
 J. J. Gonzalez-Barbosa and S. Lacroix, "Fast dense panoramic stereovision," in Proceedings of the IEEE International Conference on Robotics and Automation, 2005, pp. 1210-1215.
 D. Scaramuzza, A. Martinelli, and R. Siegwart, "A flexible technique for accurate omnidirectional camera calibration and structure from motion," in Proceedings of the IEEE International Conference on Computer Vision Systems, 2006, p. 45.
 C. Mei and P. Rives, "Single view point omnidirectional camera calibration from planar grids," in Proceedings of the IEEE International Conference on Robotics and Automation, Roma, 2007, pp. 3945-3950.
 K. Levenberg, "A method for the solution of certain problems in least squares," Quarterly of Applied Mathematics, vol. 2, pp. 164-168, 1944.
 D. W. Marquardt, "An algorithm for least-squares estimation of nonlinear parameters," Journal of the society for Industrial and Applied Mathematics, vol. 11, pp. 431-441, 1963.
 H. Yuen, J. Princen, J. Illingworth, and J. Kittler, "Comparative study of Hough transform methods for circle finding," Image and Vision Computing, vol. 8, pp. 71-77, 1990.
 王俊凱, "基於改良式適應性背景相減法與多重影像特徵比對法之多功能即時視覺追蹤系統之設計與實現," 碩士論文, 電機工程學系, 國立成功大學, 2004.
 C. Stauffer and W. E. L. Grimson, "Adaptive background mixture models for real-time tracking," in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 1999, pp. 2-252.
 Z. Zivkovic, "Improved adaptive Gaussian mixture model for background subtraction," in Proceedings of International Conference on Pattern Recognition, 2004, pp. 28-31.
 A. Elgammal, R. Duraiswami, D. Harwood, and L. S. Davis, "Background and foreground modeling using nonparametric kernel density estimation for visual surveillance," Proceedings of the IEEE, vol. 90, pp. 1151-1163, 2002.
 W. Hu, T. Tan, L. Wang, and S. Maybank, "A survey on visual surveillance of object motion and behaviors," IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews, vol. 34, pp. 334-352, 2004.
 M. Sonka, V. Hlavac, and R. Boyle, "Image processing, analysis, and machine vision second edition," International Thomson, 1999.
 A. Senior, A. Hampapur, Y. L. Tian, L. Brown, S. Pankanti, and R. Bolle, "Appearance models for occlusion handling," Image and Vision Computing, vol. 24, pp. 1233-1243, 2006.
 R. E. Kalman, "A new approach to linear filtering and prediction problems," Journal of basic Engineering, vol. 82, pp. 35-45, 1960.
 M. Isard and A. Blake, "Condensation:conditional density propagation for visual tracking," International journal of computer vision, vol. 29, pp. 5-28, 1998.
 "Sony EVI-D70 Technical Manual."
 Y. Ren, C. S. Chua, and Y. K. Ho, "Motion detection with nonstationary background," Machine Vision and Applications, vol. 13, pp. 332-343, 2003.
 陳俊壬, "即時視覺伺服追蹤系統之運動偵測與估算," 碩士論文, 電機工程學系, 國立成功大學, 2003.