AIRCC's International Journal of Computer Science and Information Technology

Open Access Open Access  Restricted Access Subscription Access

Bag of Visual Words for Word Spotting in Handwritten Documents Based on Curvature Features


Affiliations
1 Department of Computer Science, Kuvempu University, Shimogga, India
 

In this paper, we present a segmentation-based word spotting method for handwritten documents using Bag of Visual Words (BoVW) framework based on curvature features. The BoVW based word spotting methods extract SIFT or SURF features at each keypoint using fixed sized window. The drawbacks of these techniques are that they are memory intensive; the window size cannot be adapted to the length of the query and requires alignment between the keypoint sets. In order to overcome the drawbacks of SIFT or SURF local features based existing methods, we proposed to extract curvature feature at each keypoint of word image in BoVW framework. The curvature feature is scalar value describes the geometrical shape of the strokes and requires less memory space to store. The proposed method is evaluated using mean Average Precision metric through experimentation conducted on popular datasets such as GW, IAM and Bentham datasets. The yielded performances confirmed that our method outperforms existing word spotting techniques.

Keywords

Corner Keypoints, Curvature Feature, Word Image Segmentation, Bag of Visual Words, Codebook, Similarity Measure, Feature Extraction, Word Spotting.
User
Notifications
Font Size

  • Llados, J., Rusinol, M., Fornes, A., Fernandez, D., &Dutta, A. (2012). On the influence of word representations for handwritten word spotting in historical documents. International Journal of Pattern Recognition and Artificial Intelligence 26(5), 1263,002.1–1263,002.25.

  • Plamondon, R., & Srihari, S. (2000). Online and off-line handwriting recognition: a comprehensive survey. IEEE Transactions on Pattern Analysis and Machine Intelligence 22(1), pp.63–84.

  • Madhvanath, S., & Govindaraju, V.(2001). The role of holistic paradigms in handwritten word recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence 23(2), pp.149-164.

  • Lowe, D. G. (2004). Distinctive image features from scale-invariant keypoints. International journal of computer vision, 60(2), pp.91-110.

  • Bay, H., Ess, A., Tuytelaars, T., & Van Gool, L. (2008). Speeded-up robust features (SURF). Computer vision and image understanding, 110(3), pp.346-359.

  • Sivic, J., & Zisserman, A. (2003). Video google: A text retrieval approach to object matching in videos. In iccv Vol. 2, No. 1470, pp.1470-1477.

  • Wang, J., Yang, J., Yu, K., Lv, F., Huang, T., & Gong, Y. (2010). Locality-constrained linear coding for image classification. In Computer Vision and Pattern Recognition (CVPR), 2010 IEEE Conference, pp. 3360-3367.

  • Schroth, G., Hilsenbeck, S., Huitl, R., Schweiger, F., & Steinbach, E. (2011). Exploiting textrelated features for content-based image retrieval. In Multimedia (ISM), IEEE International Symposium, pp.77-84.

  • Rusinol, M., Aldavert, D., Toledo, R., & Llados, J. (2011). Browsing heterogeneous document collections by a segmentation-free word spotting method. In Document Analysis and Recognition (ICDAR), International Conference, IEEE, pp.63-67.

  • Yalniz, I. Z., & Manmatha, R. (2012). An efficient framework for searching text in noisy document images. In Document Analysis Systems (DAS), 10th IAPR International Workshop, IEEE pp.48-52.

  • Jain, R., & Doermann, D. (2012). Logo retrieval in document images. In Document analysis systems (das), 10th iapr international workshop on IEEE, pp.135-139.

  • Smith, D. J., & Harvey, R. W. (2011). Document Retrieval Using SIFT Image Features. J. UCS, 17(1), pp.3-15.

  • Shekhar, R., & Jawahar, C. V. (2012). Word image retrieval using bag of visual words. In Document Analysis Systems (DAS), 2012 10th IAPR International Workshop, IEEE, pp.297-301.

  • Rothacker, L., Rusinol, M., & Fink, G. A. (2013). Bag-of-features HMMs for segmentation-free word spotting in handwritten documents. In Document Analysis and Recognition (ICDAR), 12th International Conference IEEE, pp.1305-1309.

  • Asada, H., & Brady, M. (1986). The curvature primal sketch. IEEE transactions on pattern analysis and machine intelligence, (1), pp. 2-14.

  • Mokhtarian, F., & Mackworth, A. K. (1992). A theory of multiscale, curvature-based shape representation for planar curves. IEEE Transactions on Pattern Analysis and Machine Intelligence, 14(8), pp. 789-805.

  • Shi, M., Fujisawa, Y., Wakabayashi, T., & Kimura, F. (2002). Handwritten numeral recognition using gradient and curvature of gray scale image. Pattern Recognition, 35(10), pp.2051-2059.

  • Kannan, B., Jomy, J., & Pramod, K. V. (2013). A system for offline recognition of handwritten characters in Malayalam script.

  • Jones, G. J., Foote, J. T., Jones, K. S., & Young, S. J. (1995). Video mail retrieval: The effect of word spotting accuracy on precision. In Acoustics, Speech, and Signal Processing, International Conference on IEEE. Vol. 1, pp.309-312.

  • Rath, T. M., & Manmatha, R. (2003). Word image matching using dynamic time warping. In Computer Vision and Pattern Recognition, Proceedings. IEEE Computer Society Conference on Vol. 2, pp.II-II.

  • Marti, U. V., & Bunke, H. (2001). Using a statistical language model to improve the performance of an HMM-based cursive handwriting recognition system. International journal of Pattern Recognition and Artificial intelligence, 15(01), pp.65-90.

  • Rath, T. M., & Manmatha, R. (2003). Features for word spotting in historical manuscripts. In Document Analysis and Recognition, 2003. Proceedings. Seventh International Conference, IEEE, pp.218-222.

  • Lavrenko, V., Rath, T. M., & Manmatha, R. (2004). Holistic word recognition for handwritten historical documents. In Document Image Analysis for Libraries, 2004. Proceedings. First International Workshop, IEEE. pp. 278-287.

  • Rodriguez, J. A., & Perronnin, F. (2008). Local gradient histogram features for word spotting in unconstrained handwritten documents. Proc. 1st ICFHR, pp.7-12.

  • Zhang, B., Srihari, S. N., & Huang, C. (2004). Word image retrieval using binary features. In Electronic Imaging, International Society for Optics and Photonics. pp. 45-53.

  • Johansson, S., Leech, G., & Goodluck, H. (1978). Manual of Information to Accompany the Lancaster-Olso/Bergen Corpus of British English, for Use with Digital Computers.

  • Rodríguez-Serrano, J. A., Perronnin, F., Sánchez, G., & Lladós, J. (2010). Unsupervised writer adaptation of whole-word HMMs with application to word-spotting. Pattern Recognition Letters, 31(8), pp.742-749.

  • Howe, N. R. (2015). Inkball models for character localization and out-of-vocabulary word spotting. In Document Analysis and Recognition (ICDAR), 13th International Conference on IEEE pp.381-385.

  • Khurshid, K., Faure, C., & Vincent, N. (2012). Word spotting in historical printed documents using shape and sequence comparisons. Pattern Recognition, 45(7), pp. 2598-2609.

  • Rodríguez-Serrano, J. A., & Perronnin, F. (2012). A model-based sequence similarity with application to handwritten word spotting. IEEE Transactions on Pattern Analysis and Machine Intelligence, 34(11), pp.2108-2120.

  • Khayyat, M., Lam, L., & Suen, C. Y. (2014). Learning-based word spotting system for Arabic handwritten documents. Pattern Recognition, 47(3), pp.1021-1030.

  • Gatos, B., & Pratikakis, I. (2009). Segmentation-free word spotting in historical printed documents. In Document Analysis and Recognition, (ICDAR), 10th International Conference, IEEE. pp.271-275.

  • Leydier, Y., Ouji, A., LeBourgeois, F., & Emptoz, H. (2009). Towards an omnilingual word retrieval system for ancient manuscripts. Pattern Recognition, 42(9), pp. 2089-2105.

  • Howe, N. R. (2013). Part-structured inkball models for one-shot handwritten word spotting. In Document Analysis and Recognition (ICDAR), 12th International Conference on IEEE pp.582586.

  • Zhang, X., & Tan, C. L. (2013). Segmentation-free keyword spotting for handwritten documents based on heat kernel signature. In Document Analysis and Recognition (ICDAR),12th International Conference on IEEE, pp. 827-831.

  • Wshah, S., Kumar, G., & Govindaraju, V. (2014). Statistical script independent word spotting in offline handwritten documents. Pattern Recognition, 47(3), pp.1039-1050.

  • Almazán, J., Gordo, A., Fornés, A., & Valveny, E. (2014). Word spotting and recognition with embedded attributes. IEEE Transactions on Pattern Analysis and Machine Intelligence, 36(12), pp. 2552-2566.

  • Ghosh, S. K., & Valveny, E. (2015). Query by string word spotting based on character bi-gram indexing. In Document Analysis and Recognition (ICDAR), 13th International Conference on IEEE pp.881-885.

  • Toselli, A. H., Vidal, E., Romero, V., & Frinken, V. (2016). HMM word graph based keyword spotting in handwritten document images. Information Sciences, 370, pp.497-518.

  • Fei-Fei, L., Fergus, R., & Perona, P. (2007). Learning generative visual models from few training examples: An incremental bayesian approach tested on 101 object categories. Computer vision and Image understanding, 106(1), pp. 59-70.

  • Csurka, G., Dance, C., Fan, L., Willamowski, J., & Bray, C. (2004, May). Visual categorization with bags of keypoints. In Workshop on statistical learning in computer vision, ECCV (Vol. 1, No.1-22, pp. 1-2.

  • Shi, Z., Setlur, S., Govindaraju, V. (2009). A steerable directional local profile technique for extraction of handwritten arabic text lines. In: ICDAR. pp.176–180.

  • Harris, C., & Stephens, M. (1988). A combined corner and edge detector. In Alvey vision conference, Vol. 15, No. 50, pp.10-5244.

  • Long, D. G. (1981). The Manuscripts of Jeremy Bentham a Chronological Index to the Collection in the Library of University College, London: Based on the Catalogue by A. Taylor Milne.

  • Almazán, J., Gordo, A., Fornés, A., & Valveny, E. (2012). Efficient Exemplar Word Spotting. In Bmvc, Chicago, Vol. 1, No. 2, pp. 3.

  • Thontadari, C., & Prabhakar, C. J. (2016). Scale Space Co-Occurrence HOG Features for Word Spotting in Handwritten Document Images. International Journal of Computer Vision and Image Processing (IJCVIP), 6(2), pp.71-86.

  • Marti, U. V., & Bunke, H. (2002). The IAM-database: An English sentence database for offline handwriting recognition. International Journal on Document Analysis and Recognition, 5(1), pp.39–46.

  • Zagoris, K., Pratikakis, I., & Gatos, B. (2014). Segmentation-based historical handwritten word spotting using document-specific local features. In Frontiers in Handwriting Recognition (ICFHR), 14th International Conference IEEE, pp.9-14.

  • Lowe, D. G. (1999). Object recognition from local scale-invariant features. In Computer vision, The proceedings of the seventh IEEE international conference, IEEE, Vol. 2, pp. 1150-1157.


Abstract Views: 380

PDF Views: 194




  • Bag of Visual Words for Word Spotting in Handwritten Documents Based on Curvature Features

Abstract Views: 380  |  PDF Views: 194

Authors

C. Thontadari
Department of Computer Science, Kuvempu University, Shimogga, India
C. J. Prabhakar
Department of Computer Science, Kuvempu University, Shimogga, India

Abstract


In this paper, we present a segmentation-based word spotting method for handwritten documents using Bag of Visual Words (BoVW) framework based on curvature features. The BoVW based word spotting methods extract SIFT or SURF features at each keypoint using fixed sized window. The drawbacks of these techniques are that they are memory intensive; the window size cannot be adapted to the length of the query and requires alignment between the keypoint sets. In order to overcome the drawbacks of SIFT or SURF local features based existing methods, we proposed to extract curvature feature at each keypoint of word image in BoVW framework. The curvature feature is scalar value describes the geometrical shape of the strokes and requires less memory space to store. The proposed method is evaluated using mean Average Precision metric through experimentation conducted on popular datasets such as GW, IAM and Bentham datasets. The yielded performances confirmed that our method outperforms existing word spotting techniques.

Keywords


Corner Keypoints, Curvature Feature, Word Image Segmentation, Bag of Visual Words, Codebook, Similarity Measure, Feature Extraction, Word Spotting.

References