unicode & umlauts

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

unicode & umlauts

dunbarxx via use-livecode
I am working on a stack using Chinese characters and German umlauts.

On windows, pasting the clipboard into Metacard (yes) works fine; Pasting into Livecode 9 on Linux fails (it may be my own machine's fonts' fault?)
Pasting unicode into livecode 9 on windows also works.

So I guess this is my linux box's fault? or is there a known clipboard-unicode pasting problem on linux?

_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|

Re: unicode & umlauts

dunbarxx via use-livecode
Dunno:

I just pasted THIS from NotePad into a textField in LiveCode 9 on Xubuntu:

Eine Überraschung stellen dagegen Wörter,
wie „doppelgaenger“, „schadenfreude“ oder
„poltergeist, dar, die sich, laut dem
#Oxford #Dictionary, längst im englischen
Wortschatz eingenistet haben.
Anscheinend ist die Eigenart des Deutschen,
lange Wörter wider Willen bilden zu können,
beliebter als sich so manch ein Brite oder
#Amerikaner eingestehen möchte.

Everything was preserved.

I performed the same exercise from LibreOffice with no problems.

Richmond.


On 20.06.2018 23:36, Eric A. Engle via use-livecode wrote:

> I am working on a stack using Chinese characters and German umlauts.
>
> On windows, pasting the clipboard into Metacard (yes) works fine; Pasting into Livecode 9 on Linux fails (it may be my own machine's fonts' fault?)
> Pasting unicode into livecode 9 on windows also works.
>
> So I guess this is my linux box's fault? or is there a known clipboard-unicode pasting problem on linux?
>
> _______________________________________________
> use-livecode mailing list
> [hidden email]
> Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
> http://lists.runrev.com/mailman/listinfo/use-livecode


_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|

Re: unicode & umlauts

dunbarxx via use-livecode
In reply to this post by dunbarxx via use-livecode
Yes, I think my problem with Chinese was I had only simplified and not traditional fonts installed. Anyway I installed some libraries now my paste clipboard ctrl v works.

What doesn't work is calling the Yandex api to translate and then send the translation to a livecode field. Even though I definitely have all the fonts for Chinese and German installed Yandex returns a bunch of question marks to the field. :/  
Why is that? What must be done? Because yandex clearly send HTML readable characters since the apis work on the web. What must I do to the text returned?

on mouseup
  repeat with i = 1 to the number of lines in card field "source"
      put URLEncode(line i of cd fld "source") into theText
      put "https://translate.yandex.net/api/v1.5/tr.json/translate?key=trnsl.1.1.20180527T091305Z.7f33f9fb3f66f0bb.d573f1d9a6336a981504916600c45f49255938b3&text="&theText&"&lang=de-en" into tResult
      put URL tResult into tResult
      set the itemdelimiter to "["
      put the last item of tResult into targetText
      put targetText & return after cd fld "target"
  end repeat
end mouseup

This script works For German except it munges up umlauts. It also works for zh-en but then just returns a bunch of question marks. I definitely have all fonts installed. I have tried this in windows 10 and linux mint in case it is still a linux issue.

How am I to process this text from the yandex api to get unicode chinese characters?

Another question: in hypercard cmd + . = stop executing script. Is there any similar command in livecode?

_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|

Re: unicode & umlauts

dunbarxx via use-livecode
Hi Eric,

> Am 21.06.2018 um 17:42 schrieb Eric A. Engle via use-livecode <[hidden email]>:
>
> ...
> Another question: in hypercard cmd + . = stop executing script. Is there any similar command in livecode?

even better, the command is identical in Livecode. :-)


Best

Klaus

--
Klaus Major
http://www.major-k.de
[hidden email]


_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|

Re: unicode & umlauts

dunbarxx via use-livecode
If only. ;-)

Bob S


> On Jun 21, 2018, at 08:59 , Klaus major-k via use-livecode <[hidden email]> wrote:
>
> Hi Eric,
>
>> Am 21.06.2018 um 17:42 schrieb Eric A. Engle via use-livecode <[hidden email]>:
>>
>> ...
>> Another question: in hypercard cmd + . = stop executing script. Is there any similar command in livecode?
>
> even better, the command is identical in Livecode. :-)
>
>
> Best
>
> Klaus
>
> --
> Klaus Major


_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|

Re: unicode & umlauts

dunbarxx via use-livecode
Hi Bob,

> Am 21.06.2018 um 18:23 schrieb Bob Sneidar via use-livecode <[hidden email]>:
>
> If only. ;-)

Monsieur?
Isn't it? It IS!

> Bob S
>> On Jun 21, 2018, at 08:59 , Klaus major-k via use-livecode <[hidden email]> wrote:
>> Hi Eric,
>>> Am 21.06.2018 um 17:42 schrieb Eric A. Engle via use-livecode <[hidden email]>:
>>> ..
>>> Another question: in hypercard cmd + . = stop executing script. Is there any similar command in livecode?
>> even better, the command is identical in Livecode. :-)

Best

Klaus

--
Klaus Major
http://www.major-k.de
[hidden email]


_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|

Re: unicode & umlauts

dunbarxx via use-livecode
In reply to this post by dunbarxx via use-livecode
Hello Eric,
Just an idea but for working with yandex, what if you textEncode and textDecode for the round trip. Like so:

on mouseup
   repeat with i = 1 to the number of lines in card field "source"
      put URLEncode(textEncode(line i of cd fld "source","UTF8")) into theText
      put "https://translate.yandex.net/api/v1.5/tr.json/translate key=trnsl.1.1.20180527T091305Z.7f33f9fb3f66f0bb.d573f1d9a6336a981504916600c45f49255938b3&text="&theText&"&lang=de-en" into tResult
      put URL tResult into tResult
      set the itemdelimiter to "["
      put the last item of tResult into targetText
      put textDecode(targetText,"UTF8") & return after cd fld "target"
   end repeat
end mouseup


Scott Morrow

Elementary Software
(Now with 20% less chalk dust!)
web       http://elementarysoftware.com/
email     [hidden email]
phone booth:  1-800-615-0867
------------------------------------------------------

> On Jun 21, 2018, at 8:42 AM, Eric A. Engle via use-livecode <[hidden email]> wrote:
>
> Yes, I think my problem with Chinese was I had only simplified and not traditional fonts installed. Anyway I installed some libraries now my paste clipboard ctrl v works.
>
> What doesn't work is calling the Yandex api to translate and then send the translation to a livecode field. Even though I definitely have all the fonts for Chinese and German installed Yandex returns a bunch of question marks to the field. :/  
> Why is that? What must be done? Because yandex clearly send HTML readable characters since the apis work on the web. What must I do to the text returned?
>
> on mouseup
>  repeat with i = 1 to the number of lines in card field "source"
>      put URLEncode(line i of cd fld "source") into theText
>      put "https://translate.yandex.net/api/v1.5/tr.json/translate?key=trnsl.1.1.20180527T091305Z.7f33f9fb3f66f0bb.d573f1d9a6336a981504916600c45f49255938b3&text="&theText&"&lang=de-en" into tResult
>      put URL tResult into tResult
>      set the itemdelimiter to "["
>      put the last item of tResult into targetText
>      put targetText & return after cd fld "target"
>  end repeat
> end mouseup
>
> This script works For German except it munges up umlauts. It also works for zh-en but then just returns a bunch of question marks. I definitely have all fonts installed. I have tried this in windows 10 and linux mint in case it is still a linux issue.
>
> How am I to process this text from the yandex api to get unicode chinese characters?
>
> Another question: in hypercard cmd + . = stop executing script. Is there any similar command in livecode?
>
> _______________________________________________
> use-livecode mailing list
> [hidden email]
> Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
> http://lists.runrev.com/mailman/listinfo/use-livecode









_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|

Re: unicode & umlauts

dunbarxx via use-livecode
In reply to this post by dunbarxx via use-livecode
Hi Eric,

it seems that urlEncode does not give the expected hex values for high ASCII characters. Yandex expects UTF8 hex values.

for example

the german o-umlaut after urlEncode is
%F6
which is ASCII 154 in the extended ASCII table

yandex expects

%C3%B6
that is the utf8-urlEncode form. I don't know if Livecode can do that, I could not get it to work.


however yandex also accepts "html" as format

in my limited testing with your modified code it worked for me with german umlauts, maybe it also works with chinese.


----------------------------------------------------------
on mouseup
   local theText, tResult, targetText
   repeat with i = 1 to the number of lines in card field "source"
      put the htmlText of line i of cd fld "source" into theText
      replace "<p>" with "" in theText
      replace "</p>" with "" in theText
      put URLEncode((theText)) into theText
      put  \
            "https://translate.yandex.net/api/v1.5/tr.json/translate?key=trnsl.1.1.20180527T091305Z.7f33f9fb3f66f0bb.d573f1d9a6336a981504916600c45f49255938b3&text="&theText&"&lang=de-en&format=html" \
            into tResult
      put URL tResult into tResult
      set the itemdelimiter to "["
      put the last item of tResult into targetText
      put targetText & return after cd fld "target"
   end repeat
end mouseup
---------------------------------------------------------


Kind regards
Bernd


> Eric wrote:
> What doesn't work is calling the Yandex api to translate and then send the translation to a livecode field. Even though I definitely have all the fonts for Chinese and German installed Yandex returns a bunch of question marks to the field. :/  
> Why is that? What must be done? Because yandex clearly send HTML readable characters since the apis work on the web. What must I do to the text returned?

_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|

Re: unicode & umlauts

dunbarxx via use-livecode
In reply to this post by dunbarxx via use-livecode
Hi Eric,

I retested Scott's solution and it works like a charm. I must have made a mistake when copying the code initially.

It is a lot easier and more sensible than my solution.

this is Scott's script that works for me

----------------------------------------------
on mouseup
   local theText, tResult, targetText
   repeat with i = 1 to the number of lines in card field "source"
      put URLEncode(textencode(line i of cd fld "source","utf-8")) into theText
      put  \
            "https://translate.yandex.net/api/v1.5/tr.json/translate?key=trnsl.1.1.20180527T091305Z.7f33f9fb3f66f0bb.d573f1d9a6336a981504916600c45f49255938b3&text="&theText&"&lang=de-en&format=html" \
            into tResult
      put URL tResult into tResult
      set the itemdelimiter to "["
      put the last item of tResult into targetText
      put textDecode(targetText,"UTF8") & return after cd fld "target"
   end repeat
end mouseup
----------------------------------------------

Kind regards
Bernd
_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode