DPRSMR: Deep Learning-based Persian Road Surface Marking Recognition

Safavi, S. H.; Sadeghi, M.; Ebadpour, M.

doi:10.22061/jecei.2023.9496.627

Document Type : Original Research Paper

Authors

¹ Faculty of Advanced Technologies, University of Mohaghegh Ardabili, Namin, Iran.

² Faculty of Engineering, University of Mohaghegh Ardabili, Ardabil, Iran.

https://doi.org/10.22061/jecei.2023.9496.627

Abstract

Background and Objectives: Persian Road Surface Markings (PRSMs) recognition is a prerequisite for future intelligent vehicles in Iran. First, the existence of Persian texts on the Road Surface Markings (RSMs) makes it challenging. Second, the RSM could appear on the road with different qualities, such as poor, fair, and excellent quality. Since the type of poor-quality RSM is variable from one province to another (i.e., varying road structure and scene complexity), it is a very essential and challenging task to recognize unforeseen poor-quality RSMs. Third, almost all existed datasets have imbalanced classes that affect the accuracy of the recognition problem.
Methods: To address the first challenge, the proposed Persian Road Surface Recognizer (PRSR) approach hierarchically separates the texts and symbols before recognition. To this end, the Symbol Text Separator Network (STS-Net) is proposed. Consequently, the proposed Text Recognizer Network (TR-Net) and Symbol Recognizer Network (SR-Net) respectively recognize the text and symbol. To investigate the second challenge, we introduce two different scenario. Scenario A: Conventional random splitting training and testing data. Scenario B: Since the PRSM dataset include few images of different distance from each scene of RSM, it is highly probable that at least one of these images appear in the training set, making the recognition process easy. Since in any province of Iran, we may see a new type of poor quality RSM, which is unforeseen before (in training set), we design a realistic and challengeable scenario B in which the network is trained using excellent and fair quality RSMs and tested on poor quality ones. Besides, we propose to use the data augmentation technique to overcome the class imbalanced data challenge.
Results: The proposed approach achieves reliable performance (precision of 73.37% for scenario B) on the PRSM dataset . It significantly improves the recognition accuracy up to 15% in different scenarios.
Conclusion: Since the PRSMs include both Persian texts (with different styles) and symbols, prior to recognition process, separating the text and symbol by a proposed STS-Net could increase the recognition rate. Deploying new powerful networks and investigating new techniques to deal with class imbalanced data in the recognition problem of the PRSM dataset as well as data augmentation would be an interesting future work.

Keywords

20.1001.1.23223952.2023.11.2.16.8

Main Subjects

Computer Vision

Open Access

This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit: http://creativecommons.org/licenses/by/4.0/

Publisher’s Note

JECEI Publisher remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Publisher

Shahid Rajaee Teacher Training University

References

[1] T. Veit, J. P. Tarel, P. Nicolle, and P. Charbonnier, "Evaluation of road marking feature extraction," in Proc. 11th International IEEE Conference on Intelligent Transportation Systems: 174–181, 2008.

[2] T. Wu, A. Ranganathan, "A practical system for road marking detection and recognition," in Proc. IEEE Intelligent Vehicles Symposium: 25–30, 2012.

[3] B. Mathibela, P. Newman, I. Posner, "Reading the road: Road marking classification and interpretation," IEEE Trans. Intell. Transp. Syst., 16(4): 2072–2081, 2015.

[4] X. Liu, Z. Deng, H. Lu, L. Cao, "Benchmark for road marking detection: Dataset specification and performance baseline," in Proc. IEEE 20th International Conference on Intelligent Transportation Systems (ITSC), 2017.

[5] F. Yu et al., "BDD100K: A diverse driving video database with scalable annotation tooling," arXiv preprint arXiv:1805.04687, 2018.

[6] S. H. Safavi et al., "Image dataset for persian road surface markings," in Proc. IEEE 10th Iranian Conference on Machine Vision and Image Processing (MVIP): 258–264, 2017.

[7] A. Geiger, P. Lenz, C. Stiller, R. Urtasun, "Vision meets Robotics: The KITTI Dataset," Int. J. Rob. Res., 32(11): 1231–1237, 2013.

[8] J. L. Blanco, F. A. Moreno, J. Gonzalez-Jimenez, "The málaga urban dataset: High-rate stereo and lidars in a realistic urban scenario," Int. J. Rob. Res., 33(2): 207–214, 2014.

[9] W. Maddern, G. Pascoe, C. Linegar, P. Newman, "1 Year, 1000km: The Oxford RobotCar Dataset," Int. J. Rob. Res. (IJRR), 36(1): 3–15, 2017.

[10] W. Maddern, G. Pascoe, M. Gadd, D. Barnes, B. Yeomans, P. Newman, "Real-time kinematic ground truth for the oxford robotcar dataset", in arXiv preprint arXiv: 2002.10152, 2020.

[11] A. Gupta, A. Choudhary, “A framework for camera-based real-time lane and road surface marking detection and recognition,” IEEE Trans. Intell. Veh., 3(4): 476–485, 2018.

[12] T. Bruls, W. Maddern, A. A. Morye, P. Newman, “Mark yourself: Road marking segmentation via weakly-supervised annotations from multimodal data,” in Proc. IEEE International Conference on Robotics and Automation (ICRA): 1863-1870, 2018.

[13] A. Fallah, A. Soliemani, H. Khosravi, “Real-time lane detection based on image edge feature and hough transform,” J. Electr. Comput. Eng. Innovations (JECEI), 9(2): 193-202, 2021.

[14] N. S. Danishevskiy, I. A. Ershov, D. O. Budanov, “Computer vision system for road surface marking recognition,” in Proc. IEEE International Conference on Electrical Engineering and Photonics (EExPolytech): 130-133, 2022.

[15] B. A. Maxwell et al., “Real-time physics-based removal of shadows and shading from road surfaces”, in Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW): 1277–1285, 2019.

[16] E. S. Dawam, X. Feng, “Smart city lane detection for autonomous vehicle” in Proc. IEEE Intl Conf on Dependable, Autonomic and Secure Computing, Intl Conf on Pervasive Intelligence and Computing, Intl Conf on Cloud and Big Data Computing, Intl Conf on Cyber Science and Technology Congress (DASC/PiCom/CBDCom/CyberSciTech): 334-338, 2020.

[17] M. R. Bachute, J. M. Subhedar, “Autonomous driving architectures: insights of machine learning and deep learning algorithms,” Elsevier, Machine Learning with Applications, 6(100164): 1-25 2021.

[18] D. K. Dewangan, S. P. Sahu, “RCNet: road classification convolutional neural networks for intelligent vehicle system,” Intell. Serv. Rob., 14(2): 199-214, 2021.

[19] R. D. Brehar, M. P. Muresan, T. Mariţa, C. C. Vancea, M. Negru, S. Nedevschi, “Pedestrian street-cross action recognition in monocular far infrared sequences,” IEEE Access, (9): 74302-74324, 2021.

[20] J. Redmon, A. Farhadi, “Yolov3: An incremental improvement”, arXiv preprint arXiv:1804.02767, 2018.

[21] Z. Feng, M. Li, M. Stolz, M. Kunert, W. Wiesbeck, “Lane detection with a high-resolution automotive radar by introducing a new type of road marking,” IEEE Trans. Intell. Trans. Syst., 20(7): 2430-2447, 2018.

[22] S. Chen, Z. Zhang, H. Ma, L. Zhang, R. Zhong, “A content-adaptive hierarchical deep learning model for detecting arbitrary-oriented road surface elements using MLS point clouds,” IEEE Trans. Geosc. Remote Sens., 61(5700516): 1-16, 2023.

[23] A. Krizhevsky, I. Sutskever, G. E. Hinton, “ImageNet Classification with Deep Convolutional Neural Networks,” Advances in neural information processing systems (NIPS 2012): 1097–1105, 2012.

[24] K. Simonyan. A. Zisserman, “Very deep convolutional networks for largescale image recognition,” arXiv preprint arXiv:1409.1556, 2014.

[25] R. Girshick, “Fast r-cnn,” in Proc. IEEE International Conference on Computer Vision and Pattern Recognition (CVPR): 1440-1448, 2015.

[26] S. Ren, K. He, R. Girshick, J. Sun, “Faster r-cnn: Towards real-time object detection with region proposal networks,” Advances in neural information processing systems (NIPS), (28):1-9, 2015.

[27] S. Ren, K. He, R. Girshick, J. Sun, “Faster r-cnn: Towards real-time object detection with region proposal networks,” IEEE Trans. Pattern Anal. Mach. Intell. (TPAMI), 39(6):1137-1149, 2017.

[28] X. Ding, X. Zhang, N. Ma, J. Han, G. Ding, J. Sun, “RepVGG: Making VGG-style ConvNets great again,” in Proc. IEEE/CVF conference on computer vision and pattern recognition (CVPR): 13733-13742, 2021.

[29] L. Chen et. al., “Deep integration: A multi-label architecture for road scene recognition,” IEEE Trans. Image Proc., 28(10): 4883–4898, 2019.

[30] K. Lis et. al., “Detecting the unexpected via image resynthesis” in Proc. IEEE/CVF International Conference on Computer Vision (ICCV): 2152-2161, 2019.

LETTERS TO EDITOR

Journal of Electrical and Computer Engineering Innovations (JECEI) welcomes letters to the editor for the post-publication discussions and corrections which allows debate post publication on its site, through the Letters to Editor. Letters pertaining to manuscript published in JECEI should be sent to the editorial office of JECEI within three months of either online publication or before printed publication, except for critiques of original research. Following points are to be considering before sending the letters (comments) to the editor.

[1] Letters that include statements of statistics, facts, research, or theories should include appropriate references, although more than three are discouraged.

[2] Letters that are personal attacks on an author rather than thoughtful criticism of the author’s ideas will not be considered for publication.

[3] Letters can be no more than 300 words in length.

[4] Letter writers should include a statement at the beginning of the letter stating that it is being submitted either for publication or not.

[5] Anonymous letters will not be considered.

[6] Letter writers must include their city and state of residence or work.

[7] Letters will be edited for clarity and length.

Name *

Email Address *

Affiliation *

Comments *

Security Code *

Journal of Electrical and Computer Engineering Innovations (JECEI)

DPRSMR: Deep Learning-based Persian Road Surface Marking Recognition

References

References

Send comment about this article

Volume 11, Issue 2
July 2023
Pages 409-418

DPRSMR: Deep Learning-based Persian Road Surface Marking Recognition

References

References

Send comment about this article

Volume 11, Issue 2July 2023Pages 409-418

Volume 11, Issue 2
July 2023
Pages 409-418