||A Dense Matching Method with Feature Based Descriptors for Non-Parallel Optical Axes Images
||Institute of Civil Aviation
3D scanning has been widely applied in different fields because of its rapidity, coverage and low cost. 3D scanning can be used to scan objects of different sizes and in different fields, including manufacturing, architecture, art/history and healthcare. However, when using 3D scanning for small, sensitive items, it is necessary to operate more carefully, especially in the case of small human organs. Generally, a non-contact scanning method is the best choice for such delicate areas of human body. The non-parallel optical axes system stereo vision system, which is a non-contact scanning method, is a feasible method for small items. The theory of depth measurement depends on the geometrical relationship between the cameras and the disparities obtained from image matching. Instead of placing cameras in parallel as is customary in traditional stereo vision, it acquires more information from an image by rotating the optical axes to close to the object. However, the rotation of optical axes results in changes in the image, and the traditional local dense matching method doesn’t perform well when this occurs. Thus, this thesis provides a modified stereo matching method for the non-parallel optical axes system. To overcome the image transformation caused by the rotation of the optical axes, a feature based descriptor is taken as the matching cost and combined with local dense matching for the purpose of reconstructing the dense point cloud. Then, the accuracy is improved by constraining the search area using the modified epipolar constraint of the non-parallel optical axes system. Also, parallelizing the algorithm makes the proposed method more competitive. This research provides a modified image matching method for non-parallel optical axes to scan small human organs.
LIST OF FIGURES VI
LIST OF TABLES VIII
CHAPTER 1 INTRODUCTION 1
1.1 Motivation 1
1.2 Literature Review 5
1.3 Thesis Outline 9
THE NON-PARALLEL OPTICAL AXES STEREO VISION SYSTEM 10
2.1 Stereo Vision 11
2.2 Non-Parallel Optical Axes System 16
2.3 Epipolar Geometry 21
2.4 Depth Measurement System 24
2.5 Unit Conversion 25
2.6 Camera Calibration 29
CHAPTER 3 STEREO MATCHING FOR THE NON-PARALLEL OPTICAL AXES IMAGE 32
3.1 ORB: Oriented FAST and Rotated BRIEF 34
3.1.1 oFAST: FAST Keypoint Orientation 34
3.1.2 BRIEF 36
3.1.3 sBRIEF: steered BRIEF 39
3.1.4 rBRIEF: Rotation-aware BRIEF 40
3.2 The Color-BRIEF 42
3.3 Local Dense Matching 44
3.4 Modified Epipolar Constraint 49
3.5 Bidirectional Matching 52
3.6 Algorithm Acceleration 53
3.7 Parallelization Theory 55
3.7.1 Amdahl’s Law  55
3.7.2 Gustafson’s Law  57
3.8 Image Segmentation 60
CHAPTER 4 EXPERIMENT 62
4.1 Experimental Hardware 63
4.2 Camera Calibration Experiment 68
4.3 Stereo Matching Results 81
4.4 Discussion 97
CHAPTER 5 CONCLUSION AND FUTURE WORK 99
 J. Virtanen, H. Hyyppä, M. Kurkela, M. Vaaja, P. Alho, J. Hyyppä, "Rapid Prototyping — A Tool for Presenting 3-Dimensional," ISPRS INTERNATIONAL JOURNAL OF GEO-INFORMATION, vol. 3, no. 3, pp. 871-890, 2014.
 A. Kuş, "Implementation of 3D Optical Scanning Technology for," Sensors, pp. 1967-1979, 9 MAR 2009.
 PRNewswire, "Arius Technology Collaborates with British Museum Powerhouse, Tate, for 3D Scanning and Reproduction of Art," 7 SEP 2018. [Online]. Available: https://markets.businessinsider.com/news/stocks/arius-technology-collaborates-with-british-museum-powerhouse-tate-for-3d-scanning-and-reproduction-of-art-1027517259. [Accessed 8 SEP 2018].
 P. Treleaven, J. Wells, "3D Body Scanning and Healthcare Applications," Computer, vol. 40, no. 7, pp. 28-34, 2007.
 M. Javaid, A. Haleem, "Additive Manufacturing Applications in Orthopaedics: A Review," Journal of Clinical Orthopaedics and Trauma, vol. 9, no. 3, pp. 202-206, 2018.
 B.Curless, "From Range Scans to 3D Models," in ACM SIGGRAPH Computer Graphics, New York, Nov. 2000.
 I. Puertas, C. J. Luis Pérez, D. Salcedo, J. León, R. Luri, J. P. Fuertes, "Precision Study of a Coordinate Measuring Machine Using Several Contact Probes," in The Manufacturing Engineering Society International Conference, Zaragoza, Spain, 2013.
 M. Kupiec, "Coordinate Meaurement Systems CMM and CMA," Advances in Science and Technology Research Journal, vol. 6, no. 16, pp. 18-23, 2012.
 M. Hansard, S. Lee, O. Choi, R. Horaud , Time-of-Flight Cameras: Principles, Methods and Applications, SpringerBriefs in Computer Science, 2012 .
 J. Carter, K. Schmid, K. Waters, L. Betzhold, B. Hadley, R. Mataosky, J. Halleran, NOAA Coastal Services Center, Lidar 101: An Introduction to Lidar Technology, Data, and Applications, Charleston: NOAA Coastal Services Center, 2012.
 R. A. Morano, C. Ozturk , R. Conn , S. Dubin, S. Zietz , J. Nissano, "Structured Light Using Pseudorandom Codes," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 20, no. 3, pp. 322-327, Mar. 1998.
 J. Geng, "Structured-Light 3D Surface Imaging: a Tutorial," Advances in Optics and Photonics, vol. 3, no. 2, pp. 128-160, 2011.
 E. Auvinet, J. Meunier, J. Ong, G. Durr, M. Gilca, I. Brunette, "Methodology for the Construction and Comparison of 3D Models of the Human Cornea," in 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society, San Diego, CA, Aug. 2012.
 P. MITAS, J. GÓRSKI, J. WILKOWSKI, P. CZARNIAK,, "Unique Problems of a Structured-Light 3D Scanner During Scanning of Various Wood Species," Annals of Warsaw University of Life Sciences – SGGW. Forestry and Wood Technology, pp. 147-150, 2014.
 K. Konolige, "Small Vision Systems: Hardware and Implementation," in 8th International Symposium on Robotics Research, 1997.
 M. Wand, B. Adams, M. Ovsjanikov, A. Berner, M. Bokeloh, P. Jenke, L. J. Guibas, H. Seidel, A. Schilling, "Efficient Reconstruction of Nonrigid Shape and Motion from Real-Time 3D Scanner Data," ACM Transactions on Graphics, vol. 28, no. 2, pp. 15:1-15:15., Apr. 2009.
 I.-C. Wang, 非平行光軸立體視覺系統物體輪廓偵測, 國立成功大學民航研究所學位論文, 2018.
 F. Remondino, M. G. Spera, E. Nocerino, F. Menna, F. Nex, "Dense Image Matching: Comparisons and Analyses," in Digital Heritage International Congress, Marseille, Nov. 2013.
 D. G. Lowe, "Distinctive Image Features from Scale-Invariant Keypoints," International Journal of Computer Vision, vol. 60, no. 2, pp. 91-110, Nov. 2004.
 E. Rublee, V. Rabaud, K. Konolige, G. Bradski, "ORB: An efficient alternative to SIFT or SURF," in 2011 International Conference on Computer Vision, Barcelona, Nov.2011.
 A. Suga, K. Fukuda, T. Takiguchi, Y. Ariki, "Object recognition and segmentation using SIFT and Graph Cuts," in 19th International Conference on Pattern Recognition, Tampa, FL, USA, 2008.
 T. Park, "Implementation of Real Time Based Multi-Object Recognition Algorithm," Institude of Korean Electrical and Electronics Engineers, vol. 17, no. 1, pp. 51-56, 2013.
 B. Morse, Lecture 13: Edge Detection, Brigham Young University, 2000.
 M. Brown, D. G. Lowe, "Invariant Features from Interest Point Groups," in British Machine Vision Conference,, Cardiff, Wales, 2002.
 E. Deza,M.M. Michel, "Metrics on Normed Structures," in Encyclopedia of Distances, Springer-Verlag Berlin Heidelberg, 2009, p. 94.
 H. Bay, T. Tuytelaars, L. V. Gool, "SURF: Speeded Up Robust Features," in European Conference on Computer Vision, Berlin, Heidelberg, 2006.
 E. Rosten, T. Drummond, "Machine Learning for High-Speed Corner Detection," in European Conference on Computer Vision, Berlin, Heidelberg, 2006.
 M. Calonder, V. Lepetit, C. Strecha, P. Fua, "BRIEF: Binary Robust Independent Elementary Features," in European Conference on Computer Vision, Berlin, Heidelberg, 2010.
 D. Scharstein, R. Szeliski, R. Zabih, "A Taxonomy and Evaluation of Dense Two-Frame," in Proceedings IEEE Workshop on Stereo and Multi-Baseline Vision, Kauai, HI, USA, 2001.
 D. Scharstein, R. Szeliski, "Stereo Matching with Nonlinear Diffusion," International Journal of Computer Vision, vol. 28, no. 2, pp. 155-174, 1998.
 D. Scharstein, "1.2.7 Example: SSD," in View Synthesis Using Stereo Vision, Springer-Verlag Berlin Heidelberg, 1997, pp. 19-21.
 R. Bracewell, "Pentagram Notation for Cross Correlation.," in The Fourier Transform and Its Applications., New York, McGraw-Hill,, 1965, p. 46 and 243.
 Y. R. Rao, N. Prathapani, E. Nagabhooshanam, "Application of Normalized Cross Correlation to Image Registration," International Journal of Research in Engineering and Technology, vol. 3, no. 5, 2014.
 E. D. Asta, R. Roncella, "A Comparsion of Semiglobal and Local Dense Matching Algorithms," in Technical Commission V Symposium, Riva del Garda, Italy, Jun. 2014.
 H. Hirschmüller, "Semi-Global Matching: Motivation, Development and Applications," in 53th Photogrammetric Week, University of Stuttgart, Sep. 2011.
 H. Hirschmüller, "Stereo Processing by Semi-Global Matching and Mutual Information," IEEE Transactions on Pattern Analysis and Machine Intelligence, pp. 328-341, Feb. 2008.
 B. D. Lucas, T. Kanade, "An Iterative Image Registration Technique with an Application in Stereo Vision," in 7th International Joint Conference on Artificial Intelligence, Vancouver, 1981.
 L. Matthies, R. Szeliski, and T. Kanade, "Kalman Filter-Based Algorithms for Estimating Depth from Image Sequences," International Journal of Computer Vision, vol. 3, pp. 209-236, 1989.
 V. Kumar, A. Grama, A. Gupta, G. Karypis, Introduction to Parallel Computing: Design and Analysis of Algorithms, Redwood City, CA, USA: Benjamin-Cummings, 1994.
 T.-S. Hsu, T.-C. Wang, "An Improvement Stereo Vision Images Processing for Object Distance Measurement," International Journal of Automation and Smart Technology, vol. 5, no. 2, pp. 85-90, 2015.
 J. Jedlička, M. Potůčková, "Correction of Radial Distortion in Digital Images," in Technical Computing Prague 2007 : 15th Annual Conference Proceedings, Praha, 2007.
 R. Hartley, A. Zisserman, "Epipolar Geometry and the Fundamental Matrix," in Multiple View Geometry in Computer Vision Second Edition, Cambridge University Press, 2004, pp. 239-261.
 R. I. Hartley , "In Defense of the Eight-Point Algorithm," IEEE Transaction on Pattern Recognition and Machine Intelligence, vol. 19, no. 6, pp. 580-593, Jun. 1997.
 I. M. Bomze, V. F. Demyanov, R. Fletcher, T. Terlaky, "The Sequential Quadratic Programming Method," in Nonlinear Optimization, Springer, Berlin, Heidelberg, 2007, pp. 165-214.
 P. Ricaurte, C. Chilán, C. A. Aguilera-Carrasco,B. X. Vintimilla, A. D. Sappa, "Feature Point Descriptors: Infrared and Visible Spectra," Sensors (Basel), pp. 3690-3701, 21 Feb. 2014.
 R. Mur-Artal, J. M. M. Montiel, J. D. Tardos, "ORB-SLAM: a Versatile and Accurate Monocular SLAM System," IEEE Transactions on Robotics, vol. 31, no. 5, pp. 1147-1163, 2015.
 J. Bresenham, "A Linear Algorithm for Incremental Display of Circular Arcs," Communications of the ACM, vol. 20, no. 2, pp. 100-106, Feb. 1977.
 L. I. Smith, "A Tutorial on Principal Components Analysis," Cornell University, USA, 2002.
 Koen E. A. van de Sande,T. Gevers, Cees G. M. Snoek, "Evaluating Color Descriptors for Object and Scene Recognition," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 32, no. 9, pp. 1582 - 1596, 2010.
 M. Kottman, " The Color-BRIEF Feature Descriptor," in Spring Conference on Computer Graphics, Galbov Mlyn, Apr. 2011.
 Y. Lin, N. Lu, X. Lou, F. Zou, Y. Yao, Z. Du, "Matching Cost Filtering for Dense Stereo Correspondence," Mathematical Problems in Engineering, 27 Aug. 2013.
 W. H. Press, B. P. Flannery, S. A. Teukolsky, W. T. Vetterling,, "Interpolation in Two or More Dimensions," in Numerical Recipes in Pascal (First Edition): The Art of Scientific Computing, Cambridge University Press, 1989, pp. 107-109.
 Wikipedia, "CUDA," 16 Sep. 2018. [Online]. Available: https://en.wikipedia.org/w/index.php?title=CUDA&oldid=859833693. [Accessed 19 Sep. 2018].
 C.-Y. Lin, Hybrid OpenMP/AVX Acceleration of a Split Harten, Lax and van Leer Method for the Euler Equations, Tainan: NCKU Dept. of Mechanical Engineering, Jun. 2014.
 G. M. Amdahl, "Validity of the Single Processor Approach to Achieving Large Scale Computing Capabilities," in AFIPS Spring Joint Computer Conference, 1967.
 J. L. Gustafson, "Reevaluating Amdahl's Law," Communications of the ACM, vol. 31, no. 5, pp. 532-533 , May 1988.
 G. Bradski, "The OpenCV Library," Dr. Dobb’s Journal of Software Tools, 2000.