Most online newspaper researchers do not recognize that when searching for a word, a term, a name, etc., that they essentially are searching against a database that has been built based on whether a very small section of a newspaper is black or white. It is the formation of these dots that may look like a letter, and a collection of letters, a word.
Research your ancestors on MyHeritage
Understanding OCR in newspaper researchUnderstanding OCR in newspaper research
It is the shape of the collected dots that may (or may not) look like a letter. In other words, the source material is just a bunch of dots to OCR, which stands for Optical character recognition', the mechanical or electronic conversion of images of handwritten, typed or printed text into machine-readable text.[1] Recognizing this fact and understanding that they are often dealing with very poor-quality source material (i.e., old newsprint) should encourage one to figure out a way to craft a winning search criterion. Other than making sure that you are searching in the right place geographically and using a date range that fits your target search of an event or person, what follows is the best way to find more pertinent articles when searching online historical newspapers. If you do newspaper research online as part of your genealogy pursuits or your search of history, then you have certainly been puzzled by some of the search results (or lack thereof) that you have received.
The creation of newspaper images and the application of the OCR process does not always result in what you might expect in the index which is used to match your search criteria.
Newspaper qualityNewspaper quality
There is a simple explanation for this, and it all has to do with quality:
- Quality of the original material – was the newspaper old and brittle when scanned? Was it yellow? Ripped or torn? Creased? Did it have dirt on it or lots of ink spots?
- Was the scan performed that created the digital image, from the original paper or from a microfilm of the paper, or worse a copy of the microfilm? Every additional copy or scan degrades the resulting image and when the OCR process is applied, the index suffers.
- Quality of the OCR software. - some are more accurate than others.
- Quality of the writing in the original newspaper. Did the author get your ancestor’s name or other pertinent words spelled correctly?
- Quality of the typesetter – did the typesetter get every word from the author set up correctly?
Thus, what you are searching probably won’t be a perfect digital database that represents what was originally written by the author and newspaper publisher.
Newspaper example after OCRNewspaper example after OCR
Before we start solving our problem, let’s look at a typical 130-year-old piece of scanned newsprint, and what the text looks like after the OCR process has been executed.
Here is the text after application of the OCR process:
Sun Fra-cisejrs at L->np Branch.
Long BKAHCB (N. J.) June 29.— Among tlie arrivals to-day were: Mr. and Mrs. Al Dayman. Mrs. 1. Ilayman, .Mrs. Newman, Miss K. Newman auU iftai Baf Newman of fcan Fraucisco.
As you can see, the text that was “determined” after applying the OCR process, is less than optimal. And this is mild compared to some of the results obtained from a very poor quality newspaper page. The moral of this story is that to be successful, you must anticipate that a certain letter might be caught by the OCR process as another letter.
Here is a magnified example of a lower case “b” and a lower case “h”:
b h
Look at them closely. The only difference is pretty much at the bottom, where an inadvertent drop of ink in the middle of the opening at the bottom of the “h” turns it into a “b”
It is also coincidentally true for the capitalized versions of these two letters:
B H
Again, the inclusion or absence of ink dots “changes” the letter. I discovered this many years ago when I was looking for an ancestor in a set of California online newspapers. His surname was “Braunhart”, so I changed the search criteria to Braunbart, substituting a lower-case b for a lower-case h in the middle of the string. Lo and behold, I found a significant percentage of additional results. After playing around with other letter “pairs” or “combinations”, I am confident in telling you that you can increase your results by as much as 20% using this simple technique.
Letter pair exchangesLetter pair exchanges
- rn and m (ar n and em)
- m and n
- h and b
- h and n
- Capital D and O
- i, l, 1, /, !, and I are all often interchanged
- 0 and O
- c and e and o
- G and O
- B and 8
- R and K
- r and n
- [, ] and l (el)
- nl and m (en el and em)
- R and B
- n and ri (en and ar eye)
- v and y
- S and 8
- S and 5
- Z and 2
- G and 6
- Y and V
See alsoSee also
Explore more about editing OCR techniques for newspaper records researchExplore more about editing OCR techniques for newspaper records research
- Introducing OldNews.com, A New Website for Exploring Historical Newspapers on the MyHeritage blog
- Read 'Em or Weep: Promise and Pitfalls in Newspaper OCR (BONUS webinar for subscribers) webinar at Legacy Family Tree Webinars
References
- ↑ What is OCR?. IBM