[et_pb_section fb_built=“1″ admin_label=“section“ _builder_version=“3.22″][et_pb_row _builder_version=“4.9.4″][et_pb_column type=“4_4″ _builder_version=“3.25″ custom_padding=“|||“ custom_padding__hover=“|||“][et_pb_text _builder_version=“4.9.4″ header_font=“Open Sans Condensed Light local|300|||||||“ header_text_color=“#666666″ header_font_size=“70px“ header_letter_spacing=“1px“ max_width=“100%“ animation_style=“slide“ animation_direction=“left“ header_font_tablet=““ header_font_phone=“||||||||“ header_font_last_edited=“on|phone“ header_font_size_phone=“50px“ locked=“off“]<\/p>\n
<\/p>\n
[\/et_pb_text][\/et_pb_column][\/et_pb_row][\/et_pb_section][et_pb_section fb_built=“1″ _builder_version=“4.4.2″ background_color=“#919191″ background_enable_image=“off“ parallax=“on“ min_height=“194px“ custom_margin=“||-44px|||“ custom_padding=“15px||0px||false|false“ animation_style=“fade“ animation_direction=“right“ background_last_edited=“off|desktop“ background_enable_color_phone=“off“ background_blend_phone=“normal“ border_color_top=“#1e69ae“ border_color_bottom=“#1e69ae“ locked=“off“][et_pb_row _builder_version=“4.4.1″][et_pb_column type=“4_4″ _builder_version=“3.25″ custom_padding=“|||“ custom_padding__hover=“|||“][et_pb_text _builder_version=“4.9.4″ text_font=“Open Sans Condensed||||||||“ text_text_color=“#ffffff“ text_font_size=“26px“ text_letter_spacing=“1px“ text_line_height=“1.4em“ text_orientation=“center“ max_width=“750px“ module_alignment=“center“ min_height=“196.6px“ custom_margin=“||-73px|||“ custom_padding=“16px||0px|||“ animation_style=“slide“ animation_direction=“right“]<\/p>\n
Transkribus performs best on Cyrillic Handwriting, compared with Tesseract, Calamari and Glyph.<\/p>\n
[\/et_pb_text][\/et_pb_column][\/et_pb_row][\/et_pb_section][et_pb_section fb_built=“1″ _builder_version=“4.9.4″ _module_preset=“default“ min_height=“477.8px“][et_pb_row _builder_version=“4.9.4″ _module_preset=“default“][et_pb_column type=“4_4″ _builder_version=“4.9.4″ _module_preset=“default“][et_pb_text _builder_version=“4.9.4″ _module_preset=“default“]<\/p>\n
In over 20 years of experience in content conversion, CCS has collected many datapoints on performance of various OCR engines. From time to time, we conduct evaluations and provide them to our partners. After integrating HTR into our product docWizz in 2021, we have endeavored to build a set of ground truth for training and evaluating HTR systems. In this whitepaper we share the results of our first evaluation conducted.<\/span><\/p>\n [\/et_pb_text][\/et_pb_column][\/et_pb_row][et_pb_row _builder_version=“4.9.4″ _module_preset=“default“][et_pb_column type=“4_4″ _builder_version=“4.9.4″ _module_preset=“default“][et_pb_text _builder_version=“4.9.4″ _module_preset=“default“ custom_padding=“||2px|||“]<\/p>\n We have chosen four systems to evaluate:<\/span><\/p>\n Tesseract<\/span><\/strong> version 4, a free Open Source (OS) system originally developed by Hewlett-Packard, then for some years sponsored by Google.<\/span><\/p>\n Calamari,<\/span><\/strong> a free OS system derived from OCRopy and Kraken.<\/span><\/p>\n Transkribus<\/span><\/strong>, a commercial offering by READ Corporation, Austria. The system was originally developed in the EU founded projects tranScriptorium and READ (Recognition and Enrichment of Archival Documents)<\/span><\/p>\n Glyph<\/span><\/strong>, a still experimental closed source system developed by a group of researchers at Polytechnic University of Bucharest.<\/span><\/p>\n [\/et_pb_text][\/et_pb_column][\/et_pb_row][et_pb_row _builder_version=“4.9.4″ _module_preset=“default“][et_pb_column type=“4_4″ _builder_version=“4.9.4″ _module_preset=“default“][et_pb_text _builder_version=“4.9.4″ _module_preset=“default“ custom_padding=“||2px|||“]<\/p>\n We aim to evaluate HTR Engines<\/u> as a technology. We do not evaluate HTR Services<\/u>. Therefore, OCR services by Google, Microsoft or Amazon are not included in the list above. They are certainly great services, but the technology is tied with the models and any assessment depends strongly nature of the test data used. In our projects we are often faced with old handwriting in less common languages that require training of specific models. Thus, we are more interested in the technical capabilities of engines to train custom models. <\/span><\/p>\n [\/et_pb_text][\/et_pb_column][\/et_pb_row][et_pb_row _builder_version=“4.9.4″ _module_preset=“default“ min_height=“47.2px“][et_pb_column type=“4_4″ _builder_version=“4.9.4″ _module_preset=“default“][et_pb_text _builder_version=“4.9.4″ _module_preset=“default“ min_height=“53.6px“ custom_margin=“||-58px|||“ custom_padding=“||0px|||“]<\/p>\n [\/et_pb_text][\/et_pb_column][\/et_pb_row][et_pb_row column_structure=“3_4,1_4″ _builder_version=“4.9.4″ _module_preset=“default“][et_pb_column type=“3_4″ _builder_version=“4.9.4″ _module_preset=“default“][et_pb_text _builder_version=“4.9.4″ _module_preset=“default“ custom_padding=“||2px|||“]<\/p>\n Our set of ground truth comprises of 590k words in Ukrainian (Cyrillic) language and was collected in a single project. The original documents are from 18th and 19th century. Training of models was started \u201cfrom scratch\u201d, no pre-trained models were used for re-training. We used 98% of our data for training. A 2% random set of the data was excluded from training and used for evaluation. No post-processing like dictionary checks were used in this evaluation. <\/span><\/p>\n [\/et_pb_text][\/et_pb_column][et_pb_column type=“1_4″ _builder_version=“4.9.4″ _module_preset=“default“][et_pb_image src=“https:\/\/content-conversion.com\/wp-content\/uploads\/2024\/01\/Cyrillic-sample.png“ alt=“Cyrillic Handwriting Sample“ title_text=“Cyrillic Handwriting Sample“ _builder_version=“4.9.4″ _module_preset=“default“][\/et_pb_image][\/et_pb_column][\/et_pb_row][et_pb_row _builder_version=“4.9.4″ _module_preset=“default“][et_pb_column type=“4_4″ _builder_version=“4.9.4″ _module_preset=“default“][et_pb_text _builder_version=“4.9.4″ _module_preset=“default“]<\/p>\n HTR engines usually segment images into lines first. Training and recognition are based on single lines. In our experience, the quality of the line segmentation can have a strong impact on the recognition rates. To understand this effect, we did evaluations twice, once with the internal line segmentation provided by the HTR engine and again with an external line segmentation framework. These evaluations include the training of the models. <\/span><\/p>\n In the external case, the engines only \u201csee\u201d images with one line of text. For external line segmentation, we used the OCR-D framework, an OS framework funded by the \u201cDeutsche Forschungs\u00adgemein\u00adschaft\u201d. With Transkribus, external line segmentation was not used for technical reasons. Calamari does not have an internal line segmentation. Glyph performed slightly better with its internal segmentation. Tesseract improved significantly from external line segmentation.<\/span><\/p>\n [\/et_pb_text][\/et_pb_column][\/et_pb_row][et_pb_row _builder_version=“4.9.4″ _module_preset=“default“ custom_padding=“||0px|||“][et_pb_column type=“4_4″ _builder_version=“4.9.4″ _module_preset=“default“][et_pb_text _builder_version=“4.9.4″ _module_preset=“default“ min_height=“23.6px“ custom_padding=“||0px|||“]<\/p>\n [\/et_pb_text][\/et_pb_column][\/et_pb_row][et_pb_row column_structure=“2_5,3_5″ _builder_version=“4.9.4″ _module_preset=“default“][et_pb_column type=“2_5″ _builder_version=“4.9.4″ _module_preset=“default“][et_pb_text _builder_version=“4.9.4″ _module_preset=“default“ custom_padding=“||0px|||“]<\/p>\n We use the Levenshtein Distance as metric to count errors. The percentage values are calculated based on characters (including blanks).<\/span><\/p>\n [\/et_pb_text][\/et_pb_column][et_pb_column type=“3_5″ _builder_version=“4.9.4″ _module_preset=“default“][et_pb_image src=“https:\/\/content-conversion.com\/wp-content\/uploads\/2024\/01\/HTR-Benchmark-results-graphic.png“ alt=“HTR Benchmark results“ title_text=“HTR Benchmark results graphic“ _builder_version=“4.9.4″ _module_preset=“default“][\/et_pb_image][\/et_pb_column][\/et_pb_row][et_pb_row _builder_version=“4.9.4″ _module_preset=“default“][et_pb_column type=“4_4″ _builder_version=“4.9.4″ _module_preset=“default“][et_pb_text _builder_version=“4.9.4″ _module_preset=“default“]<\/p>\n Overall, we are positively surprised that a less than 10% error rate was achieved by two engines without postprocessing. Transkribus gives the best results. It not only gives the best results but has proven to be stable and robust against changes in style of writing in other contexts. Glyph performs surprisingly well, especially in consideration of its minimal development budget. Calamari as a free Engine performs quite well but falls short of Transkribus. Tesseract disappoints and seems not to currently be an option for handwriting even though it gives very good results for printed material.<\/span><\/p>\n Our data has a focus on names, numbers, and places. Even though we have not seen indicators of it, there may be a risk that this characteristic of the training data impacts the relative performance of the engines. <\/span><\/p>\n [\/et_pb_text][\/et_pb_column][\/et_pb_row][\/et_pb_section]<\/p>\n","protected":false},"excerpt":{"rendered":" HTR Bechmark (Handwriting TRanscription) Transkribus performs best on Cyrillic Handwriting, compared with Tesseract, Calamari and Glyph.Introduction In over 20 years of experience in content conversion, CCS has collected many datapoints on performance of various OCR engines. From time to time, we conduct evaluations and provide them to our partners. After integrating HTR into our product […]<\/p>\n","protected":false},"author":3,"featured_media":6884,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"_et_pb_use_builder":"on","_et_pb_old_content":"","_et_gb_content_width":"","footnotes":""},"class_list":["post-7117","page","type-page","status-publish","has-post-thumbnail","hentry"],"_links":{"self":[{"href":"https:\/\/content-conversion.com\/de\/wp-json\/wp\/v2\/pages\/7117","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/content-conversion.com\/de\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/content-conversion.com\/de\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/content-conversion.com\/de\/wp-json\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/content-conversion.com\/de\/wp-json\/wp\/v2\/comments?post=7117"}],"version-history":[{"count":11,"href":"https:\/\/content-conversion.com\/de\/wp-json\/wp\/v2\/pages\/7117\/revisions"}],"predecessor-version":[{"id":7511,"href":"https:\/\/content-conversion.com\/de\/wp-json\/wp\/v2\/pages\/7117\/revisions\/7511"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/content-conversion.com\/de\/wp-json\/wp\/v2\/media\/6884"}],"wp:attachment":[{"href":"https:\/\/content-conversion.com\/de\/wp-json\/wp\/v2\/media?parent=7117"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}HTR engines evaluated<\/h2>\n
Aim and method of evaluation<\/h2>\n
<\/h2>\n
Ground Truth<\/h2>\n
Line Segmentation<\/h2>\n
Results<\/h2>\n
Conclusion<\/h2>\n