Automatic Template Detection for Camera Calibration
DOI:
https://doi.org/10.33448/rsd-v11i14.36168Keywords:
Template Detection; Camera Calibration; Deep learning.Abstract
Camera calibration is the process of extract the intrinsic and extrinsic parameters of a camera. Those parameters guide the 3-dimensional localization into relation to the 2-dimensional space from the images acquired by the camera. The 3-dimensional correlation can be generated with an object with known measures, being the most common checkerboard for this purpose. From these checker- boards, the usual approach extracts the position of the inner points, equivalent to the corners of the squares, to generate this correlation. A broad range of algorithms tries to find those points on the image. Still, usually, they require previous knowledge about the dimensions of the image, the pattern distribution, or even the pattern type. In some scenario, maybe is difficult, or impossible, to implement such precise solution, targeting these limitations our work proposes a two-step end-to-end convolutional neural network architecture that processes the corner detection on a unique flow. Our proposal is agnostic to checkerboard size, pattern disposal, and positioning. In our work, first, a segmentation CNN extracts only the checkerboard from the input image (CheckerNet); from the extracted checkerboard, we extract the corner points with a corner detection CNN (Point- Net). The PointNet also works as a segmentation CNN, and the generated points are heatmaps related to points on the checkerboard corners. We performed post-processing with a K-Means-based clustering to convert those heatmaps into single positions (x,y) from the image. We compare our proposed method with the other well-known convolutional neural networks used for corner detection MATE and CCDN. For the evaluation, two datasets were used: GoPro e uEye. Our method provides better results in both datasets, reducing missed corners, double detections, false positives, and competitive results on pixel accuracy.
References
Albawi, S., Mohammed, T. A., & Al-Zawi, S. (2017). Understanding of a convolutional neural network. 2017 International Conference on Engineering and Technology (ICET), pp. 1–6. doi:10.1109/ICEngTechnol.2017.8308186
Butt, T. H., & Taj, M. (2022). Camera Calibration Through Camera Projection Loss. ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), p. 2649–2653. doi:10.1109/ICASSP43922.2022.9746819
Chen, B., Xiong, C., & Zhang, Q. (2018). CCDN: Checkerboard Corner Detection Network for Robust Camera Calibration. Intelligent Robotics and Applications, pp. 324–334. doi: 10.1007/978-3-319-97589-4_27
Dantas, M., Dreyer, P., Bezerra, D., Reis, G., Souza, R., Lins, S., … Sadok, D. (2022). Video object segmentation for automatic image annotation of ethernet connectors with environment mapping and 3D projection. Multimedia Tools and Applications. doi:10.1007/s11042-022-13128-z
Donné, S., De Vylder, J., Goossens, B., & Philips, W. (2016). MATE: Machine Learning for Adaptive Calibration Template Detection. Sensors (Basel, Switzerland), 16(11), 1858. doi:10.3390/s16111858
Duda, A., & Frese, U. (2018). Accurate Detection and Localization of Checkerboard Corners for Calibration. 29th British Machine Vision Conference (BMVC-29), 126. doi: http://bmvc2018.org/contents/papers/0508.pdf
Dutta, A., Kar, A., & Chatterji, B. N. (2008). Corner Detection Algorithms for Digital Images in Last Three Decades. IETE Technical Review, 25(3), 123-133. doi:10.4103/02564602.2008.10876651
Fuersattel, P., Dotenco, S., Placht, S., Balda, M., Maier, A., & Riess, C. (2016). OCPAD - Occluded checkerboard pattern detector. 2016 IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 1–9. doi:10.1109/WACV.2016.7477565
Harris, C. & Stephens, M. (1988). A Combined Corner and Edge Detector. Proceedings of the 4th Alvey Vision Conference, pp. 147-151. doi: dx.doi.org/10.5244/c.2.23
Hartley, R., & Zisserman, A. (2004). Multiple View Geometry in Computer Vision (2nd ed.). doi:10.1017/CBO9780511811685
He, X., Zhang, H., Hur, N., Kim, J., Wu, Q., & Kim, T. (2006). Estimation of Internal and External Parameters for Camera Calibration Using 1D Pattern. 2006 IEEE International Conference on Video and Signal Based Surveillance, pp. 93–93. doi:10.1109/AVSS.2006.48
Huang, G., Liu, Z., & Weinberger, K. Q. (2016). Densely Connected Convolutional Networks. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2261–2269. doi:10.1109/CVPR.2017.243
Li, Z., Liu, F., Yang, W., Peng, S., & Zhou, J. (2021). A Survey of Convolutional Neural Networks: Analysis, Applications, and Prospects. IEEE transactions on neural networks and learning systems, pp. 1-12. doi: /10.1109/TNNLS.2021.3084827
Lucchese, L., & Mitra, S. K. (2002). Using saddle points for subpixel feature detection in camera calibration targets. Asia-Pacific Conference on Circuits and Systems, 2002(2), 191–195. doi:10.1109/APCCAS.2002.1115151
Pak, A., Reichel, S., & Burke, J. (2022). Machine-Learning-Inspired Workflow for Camera Calibration. Sensors (Basel, Switzerland), 22(18), 6804. doi: 10.3390/s22186804
Pedra, A. V. B. M., Mendonça, M., Finocchio, M. A. F., de Arruda, L. V. R., & Castanho, J. E. C. (2013). Camera Calibration Using Detection and Neural Networks. IFAC Proceedings Volumes, 46(7), 245–250. doi:10.3182/20130522-3-BR-4036.00077
Placht, S., Fürsattel, P., Mengue, E. A., Hofmann, H., Schaller, C., Balda, M., & Angelopoulou, E. (2014). ROCHADE: Robust Checkerboard Advanced Detection for Camera Calibration. Computer Vision (ECCV 2014), pp. 766–779. doi: 10.1109/ICVS.2006.3
Qi, W., Li, F., & Zhenzhong, L. (2010). Review on camera calibration. 2010 Chinese Control and Decision Conference, pp. 3354-3358. doi:10.1109/CCDC.2010.5498574
Rosten, E., Porter, R., & Drummond, T. (2010). Faster and Better: A Machine Learning Approach to Corner Detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32(1), 105-119. doi:10.1109/TPAMI.2008.275
Scaramuzza, D., Martinelli, A., & Siegwart, R. (2006). A Flexible Technique for Accurate Omnidirectional Camera Calibration and Structure from Motion. Fourth IEEE International Conference on Computer Vision Systems (ICVS'06), pp. 45-45. doi:10.1109/ICVS.2006.3
Sturm, J., Engelhard, N., Endres, F., Burgard, W., & Cremers, D. (2012). A Benchmark for the Evaluation of RGB-D SLAM Systems. International Conference on Intelligent Robot Systems (IROS), 573-580. 10.1109/IROS.2012.6385773
Zhang, J., Luo, B., Xiang, Z., Zhang, Q., Wang, Y., Su, X., Wang, W. (2021). Deep-learning-based adaptive camera calibration for various defocusing degrees. Opt. Lett., 46(22), 5537–5540. 10.1364/OL.443337
Zhang, Q., & Xiong, C. (2017). A New Chessboard Corner Detection Algorithm with Simple Thresholding. Intelligent Robotics and Applications, 532–542. 10.1007/978-3-319-65292-4_46
Zhang, Z. (2000). A flexible new technique for camera calibration. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(11), 1330–1334. doi:10.1109/34.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2022 Marrone Silvério Melo Dantas; Daniel Bezerra; Assis T. de Oliveira Filho; Gibson Barbosa; Iago Richard Rodrigues; Djamel H. J. Sadok; Judith Kelner; Ricardo Souza
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with this journal agree to the following terms:
1) Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
2) Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
3) Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work.
