A HYBRID CLASSICAL–DEEP LEARNING MACHINE VISIONPIPELINE FOR AUTOMATED RECOGNITION OF STAMPED DATEAND PRODUCTION CODES ON ALUMINIUM CAN BOTTOMS

Hafiza Sana Fatima; Sobia Zaheer; Kinza Khurshid; Nida Zainab; Mehwish Sarwar; Hamza Gul Shad

doi:10.71146/kjmr977

Authors

Hafiza Sana Fatima Department of Computer Science, University of Lahore, Sargodha, Pakistan. Author
Sobia Zaheer Department of Computer Science, COMSATS University of Science and Technology, Abbottabad, Pakistan. Author
Kinza Khurshid Department of Computer Science, Abbottabad University of Science and Technology, Pakistan. Author
Nida Zainab Department of Computer Science, Abbottabad University of Science and Technology, Pakistan. Author
Mehwish Sarwar Department of Computer Science, Abbottabad University of Science and Technology, Pakistan. Author
Hamza Gul Shad Department of Computer Science, Abbottabad University of Science and Technology, Pakistan. Author

DOI:

https://doi.org/10.71146/kjmr977

Keywords:

camera calibration, Zhang’s method, intrinsic parameters, lens distortion, reprojection error, checkerboard, OpenCV, computer vision

Abstract

In high-speed beverage production lines, automated visual inspection of date codes or production identifiers stamped on the bottoms of the aluminium can is an important requirement, as manual inspection is impractical, and fault rate is required to be maintained below 1ppm (parts per million). The characters on the stamp are also diﬀicult to recognize because the stamping process might not produce high contrast between the stamp and a reflective metal substrate, and illumination of the stamp may vary depending on the specular reflection, the font size varies among different manufacturers, and the physical process of forming a can distorts the stamp characters. In this paper, an entire machine vision pipeline for automated detection and recognition of stamped characters on aluminium can bottoms is presented. The proposed system consists of five-stage image processing that includes grayscale conversion, circular region detection using the Hough transform, contrast limited adaptive histogram equalisation, bilateral denoising, adaptive thresholding and connected component analysis for text region segmentation, and recognition using a convolutional recurrent neural network (Conv-RNN) with EasyOCR. The full pipeline is tested on a synthetic dataset of twelve images of can bottoms collected at four progressively harder diﬀiculty levels ranging from ideal illumination conditions to high levels of noise and glare, with precisely controllable ground-truth parameters for rigorous accuracy evaluation. The system is correct on 10 out of 12 test images, with a mean processing time of 3.5 seconds per image and proved to be suitable for quality audit applications offline. Full automation of pass, warn and fail decisions can be made by format validation with regular expressions, which enforces the date structure (YYYY MM DD) and the production code structure (NNNNx HH:MM). The contributions, limitations and directions for real-time industrial deployment are discussed in detail.

Downloads

Download data is not yet available.

References

[1] Jeonghun Baek, Geewook Kim, Junyeop Lee, Sungsoo Park, Dongyoon Han, Sangdoo Yun, Seong Joon Oh, and Hwalsuk Lee. What is wrong with scene text recognition model comparisons? Dataset and model analysis. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), pages 4715–4723, 2019.

[2] Youngmin Baek, Bado Lee, Dongyoon Han, Sangdoo Yun, and Hwalsuk Lee. Character region awareness for text detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 9365–9374, 2019.

[3] Dana H. Ballard. Generalizing the Hough transform to detect arbitrary shapes. Pattern Recognition, 13(2):111–122, 1981.

[4] Gary Bradski. The OpenCV library. Dr. Dobb’s Journal of Software Tools, 25(11):120–125, 2000.

[5] Tadhg Brosnan and Da-Wen Sun. Improving quality inspection of food products by computer vision: A review. Journal of Food Engineering, 61(1):3–16, 2004.

[6] Xugong Chen, Lianwen Jin, Yuanzhi Zhu, Canjie Luo, and Tianwei Wang. Text recognition in the wild: A survey. ACM Computing Surveys, 54(2):1–35, 2021.

[7] Roland T. Chin and Charles A. Harlow. Automated visual inspection: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 4(6):557–573, 1982.

[8] Yuning Du, Chenxia Chen, Ruoyu Jia, Xiao Bai, Erwei Xie, Cheng Li, Weifeng Liu, and Yuze Ma. PP-OCR: A practical ultra lightweight OCR system. arXiv preprint arXiv:2009.09941, 2020.

[9] Alex Graves, Santiago Fernández, Faustino Gomez, and Jürgen Schmidhuber. Connectionist temporal classification: Labelling unsegmented sequence data with recurrent neural networks. In Proceedings of the 23rd International Conference on Machine Learning (ICML), pages 369–376, 2006.

[10] Lifeng He, Xiwei Ren, Qihui Gao, Xiao Zhao, Bin Yao, and Yuyan Chao. The connected-component labelling problem: A review of state-of-the-art algorithms. Pattern Recognition, 70:25–43, 2017.

[11] Max Jaderberg, Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. Reading text in the wild with convolutional neural networks. International Journal of Computer Vision, 116(1):1–20, 2016.

[12] Nobuyuki Otsu. A threshold selection method from gray-level histograms. IEEE Transactions on Systems, Man, and Cybernetics, 9(1):62–66, 1979.

[13] Baoguang Shi, Xiang Bai, and Cong Yao. An end-to-end trainable neural network for image-based sequence recognition and its application to scene text recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(11):2298–2304, 2016.

[14] Ray Smith. An overview of the Tesseract OCR engine. In Proceedings of the Ninth International Conference on Document Analysis and Recognition (ICDAR), pages 629–633, 2007.

[15] Richard Szeliski. Computer Vision: Algorithms and Applications. Springer, 2nd edition, 2022.

[16] Carlo Tomasi and Roberto Manduchi. Bilateral filtering for gray and color images. pages 839–846, 1998.

[17] Xiaoming Zhang, Fei Ding, Zhiyong Luo, and Hai Liu. A review of intelligent manufacturing for metal forming: Vision-based quality inspection and defect detection. International Journal of Advanced Manufacturing Technology, 116:1–22, 2021.

[18] Xinyu Zhou, Cong Yao, He Wen, Yuzhi Wang, Shuchang Zhou, Weiran He, and Jiajun Liang. EAST: An eﬀicient and accurate scene text detector. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 5551–5560, 2017.

[19] Karel Zuiderveld. Contrast limited adaptive histogram equalisation. In Paul S. Heckbert, editor, Graphics Gems IV, pages 474–485. Academic Press, 1994.