Strategies for Addressing OCR Similar Character Errors

When we recognize or evaluate a person, we often use phrases like “秀外慧中” and “才貌双全”. Such evaluations take both appearance and inner qualities into account. When we recognize characters, we generally also consider sound, shape, and meaning. Because of these multiple factors, many characters that are very similar in form are not actually categorized as similar characters by us. In a specific context, words like “相邀” and “祖逖” are easier to distinguish.

However, in OCR text recognition, each individual Chinese character’s form is compared with the forms of existing characters in the database, and the character with the highest appearance matching degree is selected as the recognition result. This is indeed a superficial approach. Therefore, if it is necessary to use OCR to input manuscripts, it is best to select publications that are clearly printed and have clean paper for recognition.

For example, in publications before 2000, although the vast majority of the original text is “骄傲自满”, many were recognized as “骄做自满”, one important reason being that some publications at that time had poor printing quality.

In contrast, many publications after 2000, if the PDF text layer is “骄做自满”, the image layer is also basically “骄做自满”. The few examples of “骄傲自满” being recognized as “骄做自满” in publications after 2000 were ones that I had to put in a lot of effort to find among the pile of search results in Duxiu, and those publications either had poor printing quality or used overly artistic fonts that did not suit OCR.

Another example is the recognition error from the “Wen Yu Dictionary Publishing House”. Upon closely examining the original PDF, it can be seen that there is something resembling an ink spot above the character “汉”. It is this thing that changed the OCR recognition result, causing the Duxiu search page to show the unheard of “Wen Yu Dictionary Publishing House”.

If the manuscript is organized after OCR recognition, it may contain some very subtle errors. Failing to verify the original text can easily lead to mistakes. For example, in the following passage:

In this, “利人甚多” should be “利入甚多”. This error is difficult to correct without checking the original literature.

For instance, the content of the image below comes from the manuscript of the tenth chapter “The Sacred Darkness” of the book I edited, “Dim Blue Dot: Exploring Humanity’s Space Home”. If one does not have relevant astronomical knowledge and only reads through it, it is difficult to make corrections to the names of various satellites in the table.

Therefore, in the absence of relevant knowledge, the original literature serves as an important reference for editing and proofreading. Although the reference material I found differs from the version indicated by the author, it is sufficient to serve as a basis for questioning.

When aiming to hit a target, two conditions are important: aiming more accurately and having a larger target. In the context of recognizing similar characters in OCR, it is necessary to strictly implement character shape review standards and enlarge the scale of errors — all of which should be achieved with the help of some tools.

For example, distinguishing between the characters “己” and “已” in regular script can be difficult when the font size is small on paper. However, distinguishing these two characters in a PDF is quite easy. I provide a few methods for reference:

(1) Use the search function. Place the cursor in front of the character you want to verify, enter “己” or “已” in the search dialog box, click to find the next one, and see if the “next one” found is the one behind the cursor. This is because the computer determines character shapes based on binary code standards (each character has a unique binary number), avoiding the deviations that might occur with human judgment.

(2) Zoom in on the PDF. For instance, if the small four-point font is enlarged to more than three times, it becomes quite easy to distinguish between “己” and “已”.

The above two methods are also very effective in distinguishing between similar characters that are very close in form. You can scroll down to observe the “出” and “晚” in the image below, and then read the corresponding text above (regular script, small four-point), to experience the effect of zooming in.

When using the above methods, make sure to prepare a PDF that allows for search, copy, and other functions. Before performing paper manuscript proofreading, be sure to request such a PDF from the typesetter, as this type of PDF is important for reviewing similar characters, ensuring content consistency, and content positioning.

(3) Use editing software like Heima, Founder, or Phoenix Intelligent Proofreading for verification. Specific steps are omitted.

Due to differences in disciplines and target audiences, various manuscripts have significantly different character quantity requirements for OCR character libraries. Users can select different libraries based on the publication target. For example, for users engaged in natural science publications and primary and secondary school teaching aids, it is recommended not to use OCR libraries that include variant characters. Reducing the size of the character library can improve software operating efficiency and help increase the accuracy of specific manuscript recognition.

The following passage comes from the original manuscript of a certain secondary school teaching aid.

The presence of variant characters in the character library leads OCR to select variant characters as the final recognition result in case of slight recognition deviations, which only complicates matters! If these two variant characters were absent, the recognized result should have been the standard characters. Teaching aid manuscripts, aside from a small number of variant characters used in language teaching aids, hardly require them in other subjects. Selecting the appropriate character library based on publication content helps improve publication quality.

After understanding the characteristics of similar character errors caused by OCR, one should also remind oneself: some issues may not necessarily be caused by OCR, and it is important to understand other possible causes of similar errors; some problems are indeed caused by OCR, and one cannot subjectively attribute them to other reasons. Below, I will provide an example for each of the above situations.

(1) In the revised draft of a certain book, the author’s cursive caused the typesetter to trip. Please look at the two images below:

I feel quite emotional upon seeing this error. The cursive characters “灭” and “天” are very similar, and even in a standard printed form, I would not doubt that the two could be confused due to OCR recognition issues. However, in reality, this is not the fault of OCR. The typesetter tripped, and the editor learned a lesson!

(2) While proofreading a certain paper manuscript, I discovered that a stroke of the character “别” was missing. Because I had previously encountered situations where unclear printing caused stroke loss, I assumed it was a printing issue. Later, I found it was incorrect; I used the manuscript PDF to search the entire text and found that there were 128 standard characters “别” and 3 variant characters “別”. These 3 variant characters “別” were located on two pages, over 80 pages apart, among the 128 standard characters “别”. If the proofreading process relied solely on assumptions, it would be difficult to correct.