Translate metadata to field content

classic Classic list List threaded Threaded
37 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Translate metadata to field content

Neville Smythe via use-livecode
I'm still working on htmltext in a field. There are anchors that indicate positions elsewhere
in the file. A browser will automatically find and scroll to the right place in the text. I
need to reproduce that.

Is there a way to translate an anchor like "#12345" to the location in the visible field text
it links to?

I've thought of a couple of alternate ways to do it involving lookup files, but it would be
better to do a direct translation to avoid the extra overhead.

--
Jacqueline Landman Gay         |     [hidden email]
HyperActive Software           |     http://www.hyperactivesw.com

_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|

Re: Translate metadata to field content

Neville Smythe via use-livecode
J. Landman Gay wrote:
> I'm still working on htmltext in a field. There are anchors that indicate positions elsewhere
> in the file. A browser will automatically find and scroll to the right place in the text. I
> need to reproduce that.
>
> Is there a way to translate an anchor like "#12345" to the location in the visible field text
> it links to?
>
> I've thought of a couple of alternate ways to do it involving lookup files, but it would be
> better to do a direct translation to avoid the extra overhead.

The closest I have doesn't account for softwraps, but FWIW:


function fwAnchorTagLine pFldObj, pTag
    local tA
    local tLastLine
    local tNumRuns

    put the styledText of pFldObj into tA
    put item 2 of extents(tA) into tLastLine
    repeat with i = 1 to tLastLine
       put item 2 of extents(tA[i]["runs"]) into tNumRuns
       repeat with j = 1 to tNumRuns
          if pTag is in tA[i]["runs"][j]["style"]["linkText"] \
                AND "link" is not in \
                   tA[i]["runs"][j]["style"]["textStyle"] then
             return i
          end if
       end repeat
    end repeat
end fwAnchorTagLine



--
  Richard Gaskin
  Fourth World Systems
  Software Design and Development for the Desktop, Mobile, and the Web
  ____________________________________________________________________
  [hidden email]                http://www.FourthWorld.com

_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|

Re: Translate metadata to field content

Neville Smythe via use-livecode
Thanks, I'll save that for someday. In this case though, the text is wrapped and there are
anchors scattered around inside paragraphs. Unless someone else has an idea it looks like I'm
going to have to extract lookup tables. That's going to be quite a job.

On 2/17/20 3:59 PM, Richard Gaskin via use-livecode wrote:

> J. Landman Gay wrote:
>> I'm still working on htmltext in a field. There are anchors that indicate positions elsewhere
>> in the file. A browser will automatically find and scroll to the right place in the text. I
>> need to reproduce that.
>>
>> Is there a way to translate an anchor like "#12345" to the location in the visible field text
>> it links to?
>>
>> I've thought of a couple of alternate ways to do it involving lookup files, but it would be
>> better to do a direct translation to avoid the extra overhead.
>
> The closest I have doesn't account for softwraps, but FWIW:
>
>
> function fwAnchorTagLine pFldObj, pTag
>     local tA
>     local tLastLine
>     local tNumRuns
>
>     put the styledText of pFldObj into tA
>     put item 2 of extents(tA) into tLastLine
>     repeat with i = 1 to tLastLine
>        put item 2 of extents(tA[i]["runs"]) into tNumRuns
>        repeat with j = 1 to tNumRuns
>           if pTag is in tA[i]["runs"][j]["style"]["linkText"] \
>                 AND "link" is not in \
>                    tA[i]["runs"][j]["style"]["textStyle"] then
>              return i
>           end if
>        end repeat
>     end repeat
> end fwAnchorTagLine
>
>
>


--
Jacqueline Landman Gay         |     [hidden email]
HyperActive Software           |     http://www.hyperactivesw.com


_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|

Re: Translate metadata to field content

Neville Smythe via use-livecode
There’s the formattedStyledText property which is like the formattedText but in the form of a style array. If you sum the number of chars in each run up until you find your tag, then you can should be able to use the formattedheight of char 1 up to the summed index to get the vscroll you need.

Warmest Regards,

Mark.

Sent from my iPhone

> On 17 Feb 2020, at 22:50, J. Landman Gay via use-livecode <[hidden email]> wrote:
>
> Thanks, I'll save that for someday. In this case though, the text is wrapped and there are anchors scattered around inside paragraphs. Unless someone else has an idea it looks like I'm going to have to extract lookup tables. That's going to be quite a job.
>
>> On 2/17/20 3:59 PM, Richard Gaskin via use-livecode wrote:
>> J. Landman Gay wrote:
>>> I'm still working on htmltext in a field. There are anchors that indicate positions elsewhere in the file. A browser will automatically find and scroll to the right place in the text. I need to reproduce that.
>>>
>>> Is there a way to translate an anchor like "#12345" to the location in the visible field text it links to?
>>>
>>> I've thought of a couple of alternate ways to do it involving lookup files, but it would be better to do a direct translation to avoid the extra overhead.
>> The closest I have doesn't account for softwraps, but FWIW:
>> function fwAnchorTagLine pFldObj, pTag
>>    local tA
>>    local tLastLine
>>    local tNumRuns
>>    put the styledText of pFldObj into tA
>>    put item 2 of extents(tA) into tLastLine
>>    repeat with i = 1 to tLastLine
>>       put item 2 of extents(tA[i]["runs"]) into tNumRuns
>>       repeat with j = 1 to tNumRuns
>>          if pTag is in tA[i]["runs"][j]["style"]["linkText"] \
>>                AND "link" is not in \
>>                   tA[i]["runs"][j]["style"]["textStyle"] then
>>             return i
>>          end if
>>       end repeat
>>    end repeat
>> end fwAnchorTagLine
>
>
> --
> Jacqueline Landman Gay         |     [hidden email]
> HyperActive Software           |     http://www.hyperactivesw.com
>
>
> _______________________________________________
> use-livecode mailing list
> [hidden email]
> Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
> http://lists.runrev.com/mailman/listinfo/use-livecode


_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|

Re: Translate metadata to field content

Neville Smythe via use-livecode
In reply to this post by Neville Smythe via use-livecode
> JLG wrote:
> I'm still working on htmltext in a field. There are anchors
> that indicate positions elsewhere in the file. A browser will
> automatically find and scroll to the right place in the text.
> I need to reproduce that.

Besides using style runs you could try the following simple
method that works with the htmltext of a (locked) field and
works also in a browser.

-- Assuming (as usual)
-- [1] you use the following style for a link target
--     <a name="target10">X</a>
--     where X is any char e.g. a space (else LC removes
--     the tag in its htmltext!)
-- [2] you use the following style for a local page link
--     <a href="#target10">Target10</a>

Then the following in the field's script does what you want.

-- jumps to local page links (as described above) ##[-hh 2020]
on linkClicked pUrl
  if pUrl begins with "#" then
    delete char 1 of pUrl
    put "<a name=" &quote& pUrl &quote& ">" into tTarget
    -- of course there should be no return in tTarget
    put the htmltext of me into tHtml
    if tTarget is in tHtml then
      set itemdelimiter to tTarget
      put numToChar(1) after item 1 of tHtml
      set htmltext of me to tHtml
      set itemdelimiter to numToChar(1)
      put 1+length(item 1 of me) into tL
      delete char tL of me
      -- select char tL to tL-1 of me --> variant
      select char tL of me --> see it in locked field
      -- scrollSelectionIntoView --> optional
    end if
  end if
end linkClicked

-- works for any selection in a field ##[-hh 2020]
on scrollSelectionIntoView
  put the selectedLoc into tSL
  put the vscroll of me into tV
  put item 2 of tSL - the top of me into tDiff
  if tDiff > 0.75*the height of me then
    set vscroll of me to tV + 0.4*the height of me
  else if tDiff < 0.25*the height of me then
    set vscroll of me to tV - 0.4*the height of me
  end if
end scrollSelectionIntoView


_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|

Re: Translate metadata to field content

Neville Smythe via use-livecode
In reply to this post by Neville Smythe via use-livecode
Thanks for the reply guys, both Hermann and Mark, and Bernd who sent me materials offline. I'll
be going over these suggestions to see what works best. The file content can be very long
especially when separated into runs (more than 200,000 runs for an average file) so speed will
matter since this is headed for a mobile app.

I was surprised to see two things:

1. FormattedStyledText appears nowhere in the dictionary. I didn't know it existed.

2. FormattedStyledText treats some punctuation as a new run, even if it has not changed text
style. For example, semi-colons are independent runs. Very odd.

I need to do some experimentation and timing tests, but thanks to all for the responses.

On 2/17/20 6:25 PM, Mark Waddingham via use-livecode wrote:

> There’s the formattedStyledText property which is like the formattedText but in the form of a style array. If you sum the number of chars in each run up until you find your tag, then you can should be able to use the formattedheight of char 1 up to the summed index to get the vscroll you need.
>
> Warmest Regards,
>
> Mark.
>
> Sent from my iPhone
>
>> On 17 Feb 2020, at 22:50, J. Landman Gay via use-livecode <[hidden email]> wrote:
>>
>> Thanks, I'll save that for someday. In this case though, the text is wrapped and there are anchors scattered around inside paragraphs. Unless someone else has an idea it looks like I'm going to have to extract lookup tables. That's going to be quite a job.
>>
>>> On 2/17/20 3:59 PM, Richard Gaskin via use-livecode wrote:
>>> J. Landman Gay wrote:
>>>> I'm still working on htmltext in a field. There are anchors that indicate positions elsewhere in the file. A browser will automatically find and scroll to the right place in the text. I need to reproduce that.
>>>>
>>>> Is there a way to translate an anchor like "#12345" to the location in the visible field text it links to?
>>>>
>>>> I've thought of a couple of alternate ways to do it involving lookup files, but it would be better to do a direct translation to avoid the extra overhead.
>>> The closest I have doesn't account for softwraps, but FWIW:
>>> function fwAnchorTagLine pFldObj, pTag
>>>     local tA
>>>     local tLastLine
>>>     local tNumRuns
>>>     put the styledText of pFldObj into tA
>>>     put item 2 of extents(tA) into tLastLine
>>>     repeat with i = 1 to tLastLine
>>>        put item 2 of extents(tA[i]["runs"]) into tNumRuns
>>>        repeat with j = 1 to tNumRuns
>>>           if pTag is in tA[i]["runs"][j]["style"]["linkText"] \
>>>                 AND "link" is not in \
>>>                    tA[i]["runs"][j]["style"]["textStyle"] then
>>>              return i
>>>           end if
>>>        end repeat
>>>     end repeat
>>> end fwAnchorTagLine
>>
>>
>> --
>> Jacqueline Landman Gay         |     [hidden email]
>> HyperActive Software           |     http://www.hyperactivesw.com
>>
>>
>> _______________________________________________
>> use-livecode mailing list
>> [hidden email]
>> Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
>> http://lists.runrev.com/mailman/listinfo/use-livecode
>
>
> _______________________________________________
> use-livecode mailing list
> [hidden email]
> Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
> http://lists.runrev.com/mailman/listinfo/use-livecode
>


--
Jacqueline Landman Gay         |     [hidden email]
HyperActive Software           |     http://www.hyperactivesw.com


_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|

Re: Translate metadata to field content

Neville Smythe via use-livecode
On 2020-02-18 19:14, J. Landman Gay via use-livecode wrote:

> Thanks for the reply guys, both Hermann and Mark, and Bernd who sent
> me materials offline. I'll be going over these suggestions to see what
> works best. The file content can be very long especially when
> separated into runs (more than 200,000 runs for an average file) so
> speed will matter since this is headed for a mobile app.
>
> I was surprised to see two things:
>
> 1. FormattedStyledText appears nowhere in the dictionary. I didn't
> know it existed.

Heh - I'd actually forgotten about it until a couple of weeks ago when I
had to write some code to independently process the each soft-wrapped
line of styled text in a field.

I half wrote some code to do what I wanted in a different way while
wishing 'if only I had implemented the formatted variant for styledText
wayback when' - then I happily remembered that I had!

If you have a moment to file an report in the QC, we can get it added.

> 2. FormattedStyledText treats some punctuation as a new run, even if
> it has not changed text style. For example, semi-colons are
> independent runs. Very odd.

Hmmm - there's no code to do that explicitly (a semi-colon is no
different from any other char!). Non-merged style runs can occur through
some script operations on a field though (the engine uses 0 length runs
sometimes as markers - they would probably cause that effect). It could
also be due to the structure of the htmlText being imported
(potentially). If you have a simple example, then I can probably say one
way or the other.

The formattedStyledText, like the styledText, is just a array version of
the data-structure the engine holds inside (although it never emits 0
length runs).

In this case that shouldn't matter though as it will be the text that
you want to count, not the runs.

Warmest Regards,

Mark.

--
Mark Waddingham ~ [hidden email] ~ http://www.livecode.com/
LiveCode: Everyone can create apps

_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|

Re: Translate metadata to field content

Neville Smythe via use-livecode
In reply to this post by Neville Smythe via use-livecode
J. Landman Gay wrote:

 > I was surprised to see two things:
 >
 > 1. FormattedStyledText appears nowhere in the dictionary. I didn't
 > know it existed.

Me neither.  Weird, as it's very powerful, the key to solving problems
just like this one, and a great many besides.

I just flagged the omission:
https://quality.livecode.com/show_bug.cgi?id=22579


 > 2. FormattedStyledText treats some punctuation as a new run, even if
 > it has not changed text style. For example, semi-colons are
 > independent runs. Very odd.

That is odd. As I play with it I'll see if anything else suggests the
"why" behind that pattern.  Maybe Mark Waddingham can offer some insight
on that.


 > I need to do some experimentation and timing tests, but thanks to all
 > for the responses.

I spoke Dr. Peter Brett a couple years back about performance of using
styledText vs htmlText for tasks where either would be a suitable option.

He reinforced something Mark Waddingham had once written here (or maybe
it was Trevor?), that the styledText array most closely fits the
internal structures of field contents.

In contrast, htmlText requires extensive parsing to achieve similar
results, and usually with much more overhead as expected with such parsing.

In Peter's view, he would be surprised to find any case where parsing
htmlText would be faster than working with the styledText array.  I have
no reason to doubt him, and have been using styledText as the basis for
most things I used to use htmlText for when writing new code.

--
  Richard Gaskin
  Fourth World Systems
  Software Design and Development for the Desktop, Mobile, and the Web
  ____________________________________________________________________
  [hidden email]                http://www.FourthWorld.com


_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|

RE: Translate metadata to field content

Neville Smythe via use-livecode
In reply to this post by Neville Smythe via use-livecode
What is the difference between styledText and formattedStyledText? A quick test yielded identical results. StyledText is in the dictionary.

Ralph DiMola
IT Director
Evergreen Information Services
[hidden email]


-----Original Message-----
From: use-livecode [mailto:[hidden email]] On Behalf Of J. Landman Gay via use-livecode
Sent: Tuesday, February 18, 2020 2:15 PM
To: How to use LiveCode
Cc: J. Landman Gay
Subject: Re: Translate metadata to field content

Thanks for the reply guys, both Hermann and Mark, and Bernd who sent me materials offline. I'll be going over these suggestions to see what works best. The file content can be very long especially when separated into runs (more than 200,000 runs for an average file) so speed will matter since this is headed for a mobile app.

I was surprised to see two things:

1. FormattedStyledText appears nowhere in the dictionary. I didn't know it existed.

2. FormattedStyledText treats some punctuation as a new run, even if it has not changed text style. For example, semi-colons are independent runs. Very odd.

I need to do some experimentation and timing tests, but thanks to all for the responses.

On 2/17/20 6:25 PM, Mark Waddingham via use-livecode wrote:

> There’s the formattedStyledText property which is like the formattedText but in the form of a style array. If you sum the number of chars in each run up until you find your tag, then you can should be able to use the formattedheight of char 1 up to the summed index to get the vscroll you need.
>
> Warmest Regards,
>
> Mark.
>
> Sent from my iPhone
>
>> On 17 Feb 2020, at 22:50, J. Landman Gay via use-livecode <[hidden email]> wrote:
>>
>> Thanks, I'll save that for someday. In this case though, the text is wrapped and there are anchors scattered around inside paragraphs. Unless someone else has an idea it looks like I'm going to have to extract lookup tables. That's going to be quite a job.
>>
>>> On 2/17/20 3:59 PM, Richard Gaskin via use-livecode wrote:
>>> J. Landman Gay wrote:
>>>> I'm still working on htmltext in a field. There are anchors that indicate positions elsewhere in the file. A browser will automatically find and scroll to the right place in the text. I need to reproduce that.
>>>>
>>>> Is there a way to translate an anchor like "#12345" to the location in the visible field text it links to?
>>>>
>>>> I've thought of a couple of alternate ways to do it involving lookup files, but it would be better to do a direct translation to avoid the extra overhead.
>>> The closest I have doesn't account for softwraps, but FWIW:
>>> function fwAnchorTagLine pFldObj, pTag
>>>     local tA
>>>     local tLastLine
>>>     local tNumRuns
>>>     put the styledText of pFldObj into tA
>>>     put item 2 of extents(tA) into tLastLine
>>>     repeat with i = 1 to tLastLine
>>>        put item 2 of extents(tA[i]["runs"]) into tNumRuns
>>>        repeat with j = 1 to tNumRuns
>>>           if pTag is in tA[i]["runs"][j]["style"]["linkText"] \
>>>                 AND "link" is not in \
>>>                    tA[i]["runs"][j]["style"]["textStyle"] then
>>>              return i
>>>           end if
>>>        end repeat
>>>     end repeat
>>> end fwAnchorTagLine
>>
>>
>> --
>> Jacqueline Landman Gay         |     [hidden email]
>> HyperActive Software           |     http://www.hyperactivesw.com
>>
>>
>> _______________________________________________
>> use-livecode mailing list
>> [hidden email]
>> Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
>> http://lists.runrev.com/mailman/listinfo/use-livecode
>
>
> _______________________________________________
> use-livecode mailing list
> [hidden email]
> Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
> http://lists.runrev.com/mailman/listinfo/use-livecode
>


--
Jacqueline Landman Gay         |     [hidden email]
HyperActive Software           |     http://www.hyperactivesw.com


_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode


_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|

RE: Translate metadata to field content

Neville Smythe via use-livecode
In reply to this post by Neville Smythe via use-livecode

>Ralph DiMola wrote:

>What is the difference between styledText and formattedStyledText? A quick test
>yielded identical results. StyledText is in the dictionary.

From what I tested it behaves the same as formattedText. I.e. if you have wrapped text in a field it will keep the wraps when transferring to another field although the destination field is large enough to hold the text unwrapped. Of course formattedText is on raw text and does not include styling.

Whereas formattedText inserts returns at the wrapping formattedStyledText inserts vertical tabs = ASCII 11.

Thus the number of lines returns the same value when using formattedStyledText whereas formattedText increases the number of lines if there is any wrapping.

Kind regards
Bernd
_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|

RE: Translate metadata to field content

Neville Smythe via use-livecode
In reply to this post by Neville Smythe via use-livecode
My previous post was accidentally marking a part of my response as quoting Ralph's question.



This is my response to Ralph's question about the difference between

styledText versus formattedStyledText



From what I tested it behaves the same as formattedText. I.e. if you have
wrapped text in a field it will keep the wraps when transferring to another
field although the destination field is wide enough to hold the text
unwrapped. Of course formattedText is on raw text and does not include styling.

Whereas formattedText inserts returns at the wrapping formattedStyledText
inserts vertical tabs = ASCII 11.

Thus the number of lines returns the same value when using formattedStyledText
whereas formattedText increases the number of lines if there is any wrapping.

Kind regards
Bernd


_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|

Re: Translate metadata to field content

Neville Smythe via use-livecode
In reply to this post by Neville Smythe via use-livecode
On 2/19/20 2:58 AM, Richard Gaskin via use-livecode wrote:
> In Peter's view, he would be surprised to find any case where parsing htmlText would be faster
> than working with the styledText array.

This seems to pan out. Bernd sent me a nice test stack that uses styledText and does what Mark
W. suggested. It counts words and calculates the scroll. (It's a variation on the handler you
posted, Richard.)

I was impressed with Hermann's handler because it avoided scanning through a long series of
runs and used itemDelimiter, which I thought was pretty clever. But it did take much longer to
execute.

In each timing test I looked for a link that occured near the bottom of the text, which meant
that there would be a large number of runs to scan (several thousand), or a large amount of
text for the engine to work with.

Parse runs: 40-50ms
Use itemDel: 400-500ms

I haven't converted Bernd's script to use formattedStyledText yet but that looks like the way
to go.

--
Jacqueline Landman Gay         |     [hidden email]
HyperActive Software           |     http://www.hyperactivesw.com

_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|

Re: Translate metadata to field content

Neville Smythe via use-livecode
In reply to this post by Neville Smythe via use-livecode
On 2/18/20 1:41 PM, Mark Waddingham via use-livecode wrote:

>
>> 2. FormattedStyledText treats some punctuation as a new run, even if
>> it has not changed text style. For example, semi-colons are
>> independent runs. Very odd.
>
> Hmmm - there's no code to do that explicitly (a semi-colon is no different from any other
> char!). Non-merged style runs can occur through some script operations on a field though (the
> engine uses 0 length runs sometimes as markers - they would probably cause that effect). It
> could also be due to the structure of the htmlText being imported (potentially). If you have a
> simple example, then I can probably say one way or the other.

Well, I tried to find one and couldn't repeat it so it must be something to do with the
formatting of the html. I looked at an instance of a semi-colon and it was treated as normal
text in the run. Since I have about a hundred of these files to work with, I'm not sure where
the original issue was, but it doesn't appear to be in the engine code.

I just love these little surprises.

--
Jacqueline Landman Gay         |     [hidden email]
HyperActive Software           |     http://www.hyperactivesw.com

_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|

RE: Translate metadata to field content

Neville Smythe via use-livecode
In reply to this post by Neville Smythe via use-livecode
Wow! Being able to easily discern the number of lines in a wrapped field.
This question has been asked before "Line numbers for soft-wrapped styled
text?" but I don't remember formattedStyledText being one of the solutions.
Nice...

Bernd,
Thanks for the explanation

Ralph DiMola
IT Director
Evergreen Information Services
[hidden email]


-----Original Message-----
From: use-livecode [mailto:[hidden email]] On Behalf
Of Niggemann, Bernd via use-livecode
Sent: Wednesday, February 19, 2020 12:41 PM
To: [hidden email]
Cc: Niggemann, Bernd
Subject: RE: Translate metadata to field content


>Ralph DiMola wrote:

>What is the difference between styledText and formattedStyledText? A
>quick test yielded identical results. StyledText is in the dictionary.

From what I tested it behaves the same as formattedText. I.e. if you have
wrapped text in a field it will keep the wraps when transferring to another
field although the destination field is large enough to hold the text
unwrapped. Of course formattedText is on raw text and does not include
styling.

Whereas formattedText inserts returns at the wrapping formattedStyledText
inserts vertical tabs = ASCII 11.

Thus the number of lines returns the same value when using
formattedStyledText whereas formattedText increases the number of lines if
there is any wrapping.

Kind regards
Bernd
_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode


_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|

Re: Translate metadata to field content

Neville Smythe via use-livecode
In reply to this post by Neville Smythe via use-livecode


>J. Landman Gay wrote:
>I haven't converted Bernd's script to use formattedStyledText yet but that looks like the way to go.

There is no difference to using styledText for this use case. The number of lines and the number of words are the same between the  two.

You mentioned that you want to apply this to a huge text field.

I tested with the script of RevDataGridLibraryBehaviorsDataGridButtonBehavior which I copied twice into a field. That is about 23,000 lines and about 130,000 runs.
This found a specific metadata of a word towards the end of the text in word 80,000 of roughly 100,000 words in about 250 milliseconds (this excludes loading but includes hiliting of word and setting scroll, each about 25 ms).

However the loading time of styledText a little more than 300 milliseconds (no difference between styledText and formattedStyledText, but htmlText loading of this heavily formatted text is 800 ms).

If you can manage preloading of the styledText into e.g. a script local variable at startUp or openCard or first run it would save more than half of the processing time.

here is Richard's script which I changed to get the number of words of the line with the tagged word, the number of lines are taken from the array.

The tagged word is then: word tNumWords of line (current array key)

---------------------------------------------------------
put item 2 of the extents of tDataA into tExtents
   repeat with i = 1 to tExtents
      put item 2 of the extents of tDataA[i]["runs"] into tCounter
      repeat with j = 1 to tCounter
         if tDataA[i]["runs"][j]["metadata"] is tSearchText then
            repeat with m = 1 to j
               add the number of words of tDataA[i]["runs"][m]["text"] to tNumWords
            end repeat
            put true into tFlagExit
            exit repeat
         end if
      end repeat
      if tFlagExit then exit repeat
   end repeat
---------------------------------------------------------

select word tNumWords of line i of field "x"

Kind regards
Bernd
_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|

Re: Translate metadata to field content

Neville Smythe via use-livecode
On 2/19/20 3:40 PM, Niggemann, Bernd via use-livecode wrote:
> However the loading time of styledText a little more than 300 milliseconds (no difference between styledText and formattedStyledText, but htmlText loading of this heavily formatted text is 800 ms).
>
> If you can manage preloading of the styledText into e.g. a script local variable at startUp or openCard or first run it would save more than half of the processing time.
>

I just tested loading one of my sample files and it takes about 250 ms. I can preload
sometimes, but the user will be jumping around among different files so it won't always be
possible. I imagine on a mobile device the loading will be slower, given the different CPUs on
those.

Thanks for your test stack Bernd, it was very helpful.

--
Jacqueline Landman Gay         |     [hidden email]
HyperActive Software           |     http://www.hyperactivesw.com

_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|

Re: Translate metadata to field content

Neville Smythe via use-livecode
In reply to this post by Neville Smythe via use-livecode
On 2020-02-19 21:40, Niggemann, Bernd via use-livecode wrote:

> here is Richard's script which I changed to get the number of words of
> the line with the tagged word, the number of lines are taken from the
> array.
>
> The tagged word is then: word tNumWords of line (current array key)
>
> ---------------------------------------------------------
> put item 2 of the extents of tDataA into tExtents
>    repeat with i = 1 to tExtents
>       put item 2 of the extents of tDataA[i]["runs"] into tCounter
>       repeat with j = 1 to tCounter
>          if tDataA[i]["runs"][j]["metadata"] is tSearchText then
>             repeat with m = 1 to j
>                add the number of words of tDataA[i]["runs"][m]["text"]
> to tNumWords
>             end repeat
>             put true into tFlagExit
>             exit repeat
>          end if
>       end repeat
>       if tFlagExit then exit repeat
>    end repeat
> ---------------------------------------------------------
>
> select word tNumWords of line i of field "x"

That approach is much better than my suggested one, and is independent
of the soft breaks of the text as well :)

It can be made a little more efficient though...

[ DISCLAIMER: I don't have any test data to run these on - so the
following code snippets have not been tested in any way - or
syntax/error checked :D ]

NON-UNIQUE ANCHORS

If the anchors used are non-unique and you want the first matching
anchor in the page from top to bottom / left to right then...

1. Using 'the number of elements of' rather than 'the extents' saves
some time. As the arrays in question are known to be sequences, the
number of elements of SEQUENCE == item 2 of the extents of SEQUENCE

2. Factoring out the common array lookups will save some time.

With these two changes you'd have:

repeat with i = 1 to the number of elements in tDataA
   local tRunsA
   put tDataA[i]["runs"] into tRunsA
   repeat with j = 1 to the number of elements in tRunsA
     if tRunsA[j]["metadata"] is tSearchText then
       repeat with m = 1 to j
         add the number of words of tRunsA[m]["text"] to tNumWords
         put true into tFlagExit
         exit repeat
       end repeat
     end if
   end repeat
   if tFlagExit then
     exit repeat
   end if
end repeat
select word tNumWords of line i of field "x"

UNIQUE ANCHORS

If the anchors being searched for are unique in a document, then using
repeat for each key in both loops will save some time. Although the
search order in the runs and lines will be arbitrary (hash-order), as
the thing being searched for is unique this doesn't matter.

[ The reason this should be faster is that the engine doesn't need to
process the index vars (i / j) before looking up in the array. ]

repeat for each key i in tDataA
   local tRunsA
   put tDataA[i]["runs"] into tRunsA
   repeat for each key j in tRunsA
     if tRunsA[j]["metadata"] is tSearchText then
       repeat with m = 1 to j
         add the number of words of tRunsA[m]["text"] to tNumWords
         put true into tFlagExit
         exit repeat
       end repeat
     end if
   end repeat
   if tFlagExit then
     exit repeat
   end if
end repeat
select word tNumWords of line i of field "x"

RUN WITH METADATA DEFINES SELECTION - NON-UNIQUE SEARCH

If the area you want to select is defined by the run with metadata being
searched for, then you can avoid words altogether and just count
codeunits. Codeunits are the fastest chunk to count as they don't need
any iteration of the content of the string being queried:

repeat with i = 1 to the number of elements in tDataA
   local tRunsA
   put tDataA[i]["runs"] into tRunsA
   repeat with j = 1 to the number of elements in tRunsA
     local tRunA
     put tRunsA[j] into tRunA
     if tRunA["metadata"] is tSearchText then
       repeat with m = 1 to j - 1
         add the number of codeunits of tRunsA[m]["text"] to
tNumCodeunitsBefore
         put the number of codeunits in tRunA["text"] into tNumCodeunits
         put true into tFlagExit
         exit repeat
       end repeat
     end if
   end repeat
   if tFlagExit then
     exit repeat
   end if
end repeat
select codeunit tNumCodeunitsBefore to tNumCodeunitsBefore +
tNumCodeunits - 1 of line i of field "x"

Mutatis mutandis for the unique case using repeat for each key.

Again - none of these methods require formattedStyledText, just
styledText (indeed formattedStyledText wouldn't work with this approach
as that adds extra codeunits - the VTABs - for the soft-breaks which
aren't actually there!).

Hope this helps!

Warmest Regards,

Mark.

--
Mark Waddingham ~ [hidden email] ~ http://www.livecode.com/
LiveCode: Everyone can create apps

_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|

Re: Translate metadata to field content

Neville Smythe via use-livecode
Of course *all* three of my suggested approaches are wrong - I messed up
the inner loop in each...

On 2020-02-20 07:56, Mark Waddingham via use-livecode wrote:

> NON-UNIQUE ANCHORS
> repeat with i = 1 to the number of elements in tDataA
>   local tRunsA
>   put tDataA[i]["runs"] into tRunsA
>   repeat with j = 1 to the number of elements in tRunsA
>     if tRunsA[j]["metadata"] is tSearchText then
>       repeat with m = 1 to j
>         add the number of words of tRunsA[m]["text"] to tNumWords
>         put true into tFlagExit
>         exit repeat
>       end repeat
>     end if
>   end repeat
>   if tFlagExit then
>     exit repeat
>   end if
> end repeat
> select word tNumWords of line i of field "x"

Should be:

  repeat with i = 1 to the number of elements in tDataA
    local tRunsA
    put tDataA[i]["runs"] into tRunsA
    repeat with j = 1 to the number of elements in tRunsA
      if tRunsA[j]["metadata"] is tSearchText then
        repeat with m = 1 to j
          add the number of words of tRunsA[m]["text"] to tNumWords
        end repeat
        put true into tFlagExit
        exit repeat
      end if
    end repeat
    if tFlagExit then
      exit repeat
    end if
  end repeat
  select word tNumWords of line i of field "x"

> UNIQUE ANCHORS

> repeat for each key i in tDataA
>   local tRunsA
>   put tDataA[i]["runs"] into tRunsA
>   repeat for each key j in tRunsA
>     if tRunsA[j]["metadata"] is tSearchText then
>       repeat with m = 1 to j
>         add the number of words of tRunsA[m]["text"] to tNumWords
>         put true into tFlagExit
>         exit repeat
>       end repeat
>     end if
>   end repeat
>   if tFlagExit then
>     exit repeat
>   end if
> end repeat
> select word tNumWords of line i of field "x"

Should be:

  repeat for each key i in tDataA
    local tRunsA
    put tDataA[i]["runs"] into tRunsA
    repeat for each key j in tRunsA
      if tRunsA[j]["metadata"] is tSearchText then
        repeat with m = 1 to j
          add the number of words of tRunsA[m]["text"] to tNumWords
        end repeat
        put true into tFlagExit
        exit repeat
      end if
    end repeat
    if tFlagExit then
      exit repeat
    end if
  end repeat
  select word tNumWords of line i of field "x"

> RUN WITH METADATA DEFINES SELECTION - NON-UNIQUE SEARCH
>
> repeat with i = 1 to the number of elements in tDataA
>   local tRunsA
>   put tDataA[i]["runs"] into tRunsA
>   repeat with j = 1 to the number of elements in tRunsA
>     local tRunA
>     put tRunsA[j] into tRunA
>     if tRunA["metadata"] is tSearchText then
>       repeat with m = 1 to j - 1
>         add the number of codeunits of tRunsA[m]["text"] to
> tNumCodeunitsBefore
>         put the number of codeunits in tRunA["text"] into tNumCodeunits
>         put true into tFlagExit
>         exit repeat
>       end repeat
>     end if
>   end repeat
>   if tFlagExit then
>     exit repeat
>   end if
> end repeat
> select codeunit tNumCodeunitsBefore to tNumCodeunitsBefore +
> tNumCodeunits - 1 of line i of field "x"

Should be:

  repeat with i = 1 to the number of elements in tDataA
    local tRunsA
    put tDataA[i]["runs"] into tRunsA
    repeat with j = 1 to the number of elements in tRunsA
      local tRunA
      put tRunsA[j] into tRunA
      if tRunA["metadata"] is tSearchText then
        repeat with m = 1 to j - 1
          add the number of codeunits of tRunsA[m]["text"] to
tNumCodeunitsBefore
        end repeat
        put the number of codeunits in tRunA["text"] into tNumCodeunits
        put true into tFlagExit
        exit repeat
      end if
    end repeat
    if tFlagExit then
      exit repeat
    end if
  end repeat
  select codeunit tNumCodeunitsBefore to tNumCodeunitsBefore +
tNumCodeunits - 1 of line i of field "x"

Oops!

Mark.

--
Mark Waddingham ~ [hidden email] ~ http://www.livecode.com/
LiveCode: Everyone can create apps

_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|

Re: Translate metadata to field content

Neville Smythe via use-livecode
In reply to this post by Neville Smythe via use-livecode
In reply to Mark Waddingham's comments


Thank you Mark Waddingham for the improved scripts and the hints as to why they improve speed.

I adapted Mark's version for unique occurrence, changed how the position of the target word is determined in the target line.
It is not safe to assume that the sum of words of the runs is the number of words of the line up to the target word. The reason is that runs are depending on formatting and formatting can create a new run in the middle of a word and thus increase word count.
I did not opt for Mark's use of codeunits because I had the impression it was not faster and makes the code less obvious.

--------------------------------------
local tTextOfRuns
repeat for each key i in tDataA
   local tRunsA
   put tDataA[i]["runs"] into tRunsA
   repeat for each key j in tRunsA
      if tRunsA[j]["metadata"] is tSearchText then
         repeat with m = 1 to j
            put tRunsA[m]["text"] after tTextOfRuns
         end repeat
         put the number of words of tTextOfRuns into tNumWords
         put true into tFlagExit
         exit repeat
      end if
   end repeat
   if tFlagExit then
      exit repeat
   end if
end repeat
--------------------------------------
select word tNumWords of line i of field "x"

text consists of 96881 words and 23161 lines of heavily formatted text
(it is the script of RevDataGridLibraryBehaviorsDataGridButtonBehavior copied twice into a field as described before)

word# old new version, times in ms

96881 240 110
80000 220 100
60000 180  60
30000 120 125
10000  85 125
 1000  50  90
    1  50  60

Timing this is a bit tricky. For "repeat with I = 1 to item 2 of the extents" it is obvious that time increases with increasing the target word number.

For "repeat for each key I in tDataA" it is not sequential but faster. However that also makes for variations in speed depending on the internal state of the array structure.

All timings are estimated averages of 5 to 10 measurements . Variability is typically about +-5 to 10 milliseconds with outliers.

However the overall speed gain is quite impressive and well worth the change.
I learned a lot about handling larger datasets using arrays, than you.

Kind regards
Bernd


_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|

Re: Translate metadata to field content

Neville Smythe via use-livecode
In reply to this post by Neville Smythe via use-livecode
As others try to optimize ("ping") I'll try an improvement too ("pong")
with using another method that requires to change your link targets ONCE:

Instead of unique targets <a name="target10"> write in your field

"<p hidden>"&numTochar(1)&"target10"&numTochar(1)&"</p>""

Handler replaceTargets below does it (slowly) but you probably don't
need it on mobile.

So assuming (unusual)

-- [1] you use the following for a link target
--     <p hidden>&#1;target10&#1;<p>
-- [2] you use the following for a local page link
--     <a href="#target10">Target10</a>
-- [3] you don't use <p hidden> elsewhere. Else add
--     an additional marker to it to differentiate.


Then script your field with the following simple handler:

on linkClicked pUrl
  put the milliseconds into m1
  lock messages; lock screen
  if pUrl begins with "#" then
    put numToChar(1)& (char 2 to -1 of pUrl) &numToChar(1) into tTarget
    put 1+offset(tTarget,me)+length(tTarget) into tOff
    select char tOff to tOff+3 of me -- see it in locked field
    -- select char tOff of me -- variant
    scrollSelectionIntoView -- optional
  end if
  put the millisecs -m1 into fld "timing1"
end linkClicked

---- helpers (optionally needed)

-- Note. LC adds also an additional "<p hidden></p>", we don't mind.
on replaceTargets -- should be optimized if used often
  put the millisecs into m1
  lock messages; lock screen
  put the htmltext of fld 1 into tHTML
  set linedel to "<a name="&quote
  set itemdel to "</a>"
  put line 1 of tHtml into tI2
  put numToChar(1) into b1
  repeat for each line L in (line 2 to -1 of tHtml)
    put "<p hidden>"&b1&char 1 to offset(quote,L)-1 of L into item 1 of L
    put offset("</a>",L) into o1
    put b1&"</p>" into char o1 to o1+3 of L
    put L after tI2
  end repeat
  set htmlText of fld 1 to tI2 -- LC translates numToChar(1) to "&#1;"
  put the millisecs-m1 into fld "timing2"
end mouseUp

on scrollSelectionIntoView
  put the selectedLoc into tSL
  put the vscroll of me into tV
  put item 2 of tSL - the top of me into tDiff
  if tDiff > 0.75*the height of me then
    set vscroll of me to tV + 0.4*the height of me
  else if tDiff < 0.25*the height of me then
    set vscroll of me to tV - 0.4*the height of me
  end if
end scrollSelectionIntoView


_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
12