identifying text chars in an image

classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

identifying text chars in an image

Ralph DiMola via use-livecode
I want to [programmatically] take a picture of a Scrabble board and then
take from that image the characters displayed in it, and maintain the
organization of the text. In other words, I want to convert the text
displayed in the image to text strings.

Any ideas where I should start?

I would love to be able to do it in my app. However I'm not completely
against uploading the image to a service that would do it for me; I am
only mostly against it.

Thanks for any ideas to may have.

--
Phil Davis
503-307-4363


_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|

Re: identifying text chars in an image

Ralph DiMola via use-livecode
Well, that'd involve some sort of OCR . . . So you've either got to wrap
some code up from elsewhere, or roll your own OCR routine in LiveCode.

Tough call.

On Mon, Oct 28, 2019, 9:45 AM Phil Davis via use-livecode <
[hidden email]> wrote:

> I want to [programmatically] take a picture of a Scrabble board and then
> take from that image the characters displayed in it, and maintain the
> organization of the text. In other words, I want to convert the text
> displayed in the image to text strings.
>
> Any ideas where I should start?
>
> I would love to be able to do it in my app. However I'm not completely
> against uploading the image to a service that would do it for me; I am
> only mostly against it.
>
> Thanks for any ideas to may have.
>
> --
> Phil Davis
> 503-307-4363
>
>
> _______________________________________________
> use-livecode mailing list
> [hidden email]
> Please visit this url to subscribe, unsubscribe and manage your
> subscription preferences:
> http://lists.runrev.com/mailman/listinfo/use-livecode
>
_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|

Re: identifying text chars in an image

Ralph DiMola via use-livecode
Is this any help.
https://github.com/tesseract-ocr/tesseract

Antti

ma 28. lokak. 2019 klo 11.38 Richmond Mathewson via use-livecode (
[hidden email]) kirjoitti:

> Well, that'd involve some sort of OCR . . . So you've either got to wrap
> some code up from elsewhere, or roll your own OCR routine in LiveCode.
>
> Tough call.
>
> On Mon, Oct 28, 2019, 9:45 AM Phil Davis via use-livecode <
> [hidden email]> wrote:
>
> > I want to [programmatically] take a picture of a Scrabble board and then
> > take from that image the characters displayed in it, and maintain the
> > organization of the text. In other words, I want to convert the text
> > displayed in the image to text strings.
> >
> > Any ideas where I should start?
> >
> > I would love to be able to do it in my app. However I'm not completely
> > against uploading the image to a service that would do it for me; I am
> > only mostly against it.
> >
> > Thanks for any ideas to may have.
> >
> > --
> > Phil Davis
> > 503-307-4363
> >
> >
> > _______________________________________________
> > use-livecode mailing list
> > [hidden email]
> > Please visit this url to subscribe, unsubscribe and manage your
> > subscription preferences:
> > http://lists.runrev.com/mailman/listinfo/use-livecode
> >
> _______________________________________________
> use-livecode mailing list
> [hidden email]
> Please visit this url to subscribe, unsubscribe and manage your
> subscription preferences:
> http://lists.runrev.com/mailman/listinfo/use-livecode
>
_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|

Re: identifying text chars in an image

Ralph DiMola via use-livecode
In reply to this post by Ralph DiMola via use-livecode
The sample stack "DePerspective/Undistort Images 1.0.1"
http://livecodeshare.runrev.com/stack/824/DePerspective-Undistort-Images-1-0-1

uses for a given image tesseract if installed on the system (mac/win/linux).
_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|

Re: identifying text chars in an image

Ralph DiMola via use-livecode
In reply to this post by Ralph DiMola via use-livecode
There are also emscripten ports to javascript you could use via
a browser widget (and a CDN if online).

https://tesseract.projectnaptha.com
https://antimatter15.com/ocrad.js/demo.html

_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|

Re: identifying text chars in an image

Ralph DiMola via use-livecode
Thanks Richmond and hh - really appreciate these ideas. I'll be looking
into them.

Phil


On 10/28/19 7:26 AM, hh via use-livecode wrote:

> There are also emscripten ports to javascript you could use via
> a browser widget (and a CDN if online).
>
> https://tesseract.projectnaptha.com
> https://antimatter15.com/ocrad.js/demo.html
>
> _______________________________________________
> use-livecode mailing list
> [hidden email]
> Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
> http://lists.runrev.com/mailman/listinfo/use-livecode
>
>

--
Phil Davis
503-307-4363


_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|

Re: identifying text chars in an image

Ralph DiMola via use-livecode
In reply to this post by Ralph DiMola via use-livecode
After a second read I think it will be hard to find a service to do that
for you. And this is not doable using tesseract or similar from LiveCode:

AFAIK, no "ordinary" OCR software is able to detect and read horizontal
*and* vertical text and distinguish clearly between the board's "Score
text" and the "Scrabble letters". You (or your service) would have to
write an own "learning routine" for that.

E.g. the algorithm described here:
http://rasdasd.com/projects/Scrabble_Detector/Scrabble_Paper.pdf

There are also approaches for own specialized boards like here
https://github.com/eladj/ScrabbleOCR
You could try to use the java from there via LC Builder's FFI.

> Phil D. wrote:
> I want to [programmatically] take a picture of a Scrabble board and then
> take from that image the characters displayed in it, and maintain the
> organization of the text. In other words, I want to convert the text
> displayed in the image to text strings.
>
> Any ideas where I should start?
>
> I would love to be able to do it in my app. However I'm not completely
> against uploading the image to a service that would do it for me; I am
> only mostly against it.


_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|

Re: identifying text chars in an image

Ralph DiMola via use-livecode
Thanks for the followup on this. I experimented some with tesseract and
came to the same conclusion.

Since the values assigned to each character are standard wthin Scrabble,
I don't really need to detect those tiny numerals. But the alpha chars
are of course essential.

I believe there must be a way to get there. Thankfully no one is
pressuring me to finish it, so I can explore.

Phil


On 10/28/19 1:57 PM, hh via use-livecode wrote:

> After a second read I think it will be hard to find a service to do that
> for you. And this is not doable using tesseract or similar from LiveCode:
>
> AFAIK, no "ordinary" OCR software is able to detect and read horizontal
> *and* vertical text and distinguish clearly between the board's "Score
> text" and the "Scrabble letters". You (or your service) would have to
> write an own "learning routine" for that.
>
> E.g. the algorithm described here:
> http://rasdasd.com/projects/Scrabble_Detector/Scrabble_Paper.pdf
>
> There are also approaches for own specialized boards like here
> https://github.com/eladj/ScrabbleOCR
> You could try to use the java from there via LC Builder's FFI.
>
>> Phil D. wrote:
>> I want to [programmatically] take a picture of a Scrabble board and then
>> take from that image the characters displayed in it, and maintain the
>> organization of the text. In other words, I want to convert the text
>> displayed in the image to text strings.
>>
>> Any ideas where I should start?
>>
>> I would love to be able to do it in my app. However I'm not completely
>> against uploading the image to a service that would do it for me; I am
>> only mostly against it.
>
> _______________________________________________
> use-livecode mailing list
> [hidden email]
> Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
> http://lists.runrev.com/mailman/listinfo/use-livecode
>
>

--
Phil Davis
503-307-4363


_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|

Re: identifying text chars in an image

Ralph DiMola via use-livecode
One thing I found is when I cropped my test image to include only one
horizontal word, tesseract handled that fine. So... if I could detect
the grid pattern in which all the letters are laid out, maybe I could
feed the letters to an OCR tool one word or char at a time and get the
results I'm looking for. That would also give me clear info about
spatial relations between the letters, so I could group them correctly
into words.

So maybe I should first be asking about detecting the grid established
by the scrabble letters.

(It's actually Bananagrams letters. My nerdy grandkids like to use
Scrabble letters when playing Bananagrams so they can see who gets high
score.)

Thanks -
Phil


On 10/28/19 2:23 PM, Phil Davis via use-livecode wrote:

> Thanks for the followup on this. I experimented some with tesseract
> and came to the same conclusion.
>
> Since the values assigned to each character are standard wthin
> Scrabble, I don't really need to detect those tiny numerals. But the
> alpha chars are of course essential.
>
> I believe there must be a way to get there. Thankfully no one is
> pressuring me to finish it, so I can explore.
>
> Phil
>
>
> On 10/28/19 1:57 PM, hh via use-livecode wrote:
>> After a second read I think it will be hard to find a service to do that
>> for you. And this is not doable using tesseract or similar from
>> LiveCode:
>>
>> AFAIK, no "ordinary" OCR software is able to detect and read horizontal
>> *and* vertical text and distinguish clearly between the board's "Score
>> text" and the "Scrabble letters". You (or your service) would have to
>> write an own "learning routine" for that.
>>
>> E.g. the algorithm described here:
>> http://rasdasd.com/projects/Scrabble_Detector/Scrabble_Paper.pdf
>>
>> There are also approaches for own specialized boards like here
>> https://github.com/eladj/ScrabbleOCR
>> You could try to use the java from there via LC Builder's FFI.
>>
>>> Phil D. wrote:
>>> I want to [programmatically] take a picture of a Scrabble board and
>>> then
>>> take from that image the characters displayed in it, and maintain the
>>> organization of the text. In other words, I want to convert the text
>>> displayed in the image to text strings.
>>>
>>> Any ideas where I should start?
>>>
>>> I would love to be able to do it in my app. However I'm not completely
>>> against uploading the image to a service that would do it for me; I am
>>> only mostly against it.
>>
>> _______________________________________________
>> use-livecode mailing list
>> [hidden email]
>> Please visit this url to subscribe, unsubscribe and manage your
>> subscription preferences:
>> http://lists.runrev.com/mailman/listinfo/use-livecode
>>
>>
>

--
Phil Davis
503-307-4363


_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|

Re: identifying text chars in an image

Ralph DiMola via use-livecode
In reply to this post by Ralph DiMola via use-livecode
> Phil D. wrote:
> So maybe I should first be asking about detecting the grid
> established by the scrabble letters.

You need some criteria to find the topleft of the grid and the
size of one letter: color/transparency/a marker(fixed image part)...
You could also use pattern recognition for that.

Then it's the easy part to get all possible letters by computing the
grid and crop (get from imageData) the grid elements by script.
Then doing the OCR for each single char.

If you need that for a standalone, then 'tesseract' with its huge
language packs is probably not appropriate and 'ocrad' could be
good enough for that.

_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|

Re: identifying text chars in an image

Ralph DiMola via use-livecode
Thanks for your great ideas Hermann. In my next experiment I think I'll
start by looking for any scrabble tile by looking for its [known]
background color, and then work out to its edges. That should give me
both the grid cell size and a starting point for discovering all cells
in the grid.

Unless anyone knows of something simpler?

Thanks -
Phil



On 10/28/19 5:43 PM, hh via use-livecode wrote:

>> Phil D. wrote:
>> So maybe I should first be asking about detecting the grid
>> established by the scrabble letters.
> You need some criteria to find the topleft of the grid and the
> size of one letter: color/transparency/a marker(fixed image part)...
> You could also use pattern recognition for that.
>
> Then it's the easy part to get all possible letters by computing the
> grid and crop (get from imageData) the grid elements by script.
> Then doing the OCR for each single char.
>
> If you need that for a standalone, then 'tesseract' with its huge
> language packs is probably not appropriate and 'ocrad' could be
> good enough for that.
>
> _______________________________________________
> use-livecode mailing list
> [hidden email]
> Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
> http://lists.runrev.com/mailman/listinfo/use-livecode
>
>

--
Phil Davis
503-307-4363


_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode