Problems with jsonToArray and special chars like: æøå

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Problems with jsonToArray and special chars like: æøå

Niggemann, Bernd via use-livecode
I have run into a problem with jsonToArray and the special Norwegian chars æøå. They do not show up in the keys of the array when the array is made from an external file. The external file is generated by LiveCode and the arrayToJSON function. All text are encoded to “UTF-8” before the array is made and passed to arrayToJSON. The script I use to generate the array on start up is as follows:

global kommuneArray

on preOpenStack

put empty into kommuneArray

put "file:" & specialFolderPath("resources")& "/Kommunedata.txt" into tFile

put textDecode(url tFile,"UTF-8") into tData

put jsonToArray(textDecode(tData,"UTF-8")) into kommuneArray

end preOpenStack


The array generated contains all the right information, but keys that should include æ, ø or å do not show up correctly. Any ideas anyone?

Best regards
Tore Nilsen


_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|

Re: Problems with jsonToArray and special chars like: æøå

Niggemann, Bernd via use-livecode
Hi Tore,

jsonToArray uses the mergJSON external. The mergJSON external expects the
data passed to JSONtoArray() to be utf-8 encoded (this is the case with any
LC external, such revDB etc).

So something like this should work:

global kommuneArray

on preOpenStack
  put empty into kommuneArray
  put "file:" & specialFolderPath("resources")& "/Kommunedata.txt" into
tFile
  put textEncode(tFile,"UTF-8") into tData
  put jsonToArray(tData) into kommuneArray
end preOpenStack

Best,
Panos
--

On Wed, Jun 6, 2018 at 11:16 AM, Tore Nilsen via use-livecode <
[hidden email]> wrote:

> I have run into a problem with jsonToArray and the special Norwegian chars
> æøå. They do not show up in the keys of the array when the array is made
> from an external file. The external file is generated by LiveCode and the
> arrayToJSON function. All text are encoded to “UTF-8” before the array is
> made and passed to arrayToJSON. The script I use to generate the array on
> start up is as follows:
>
> global kommuneArray
>
> on preOpenStack
>
> put empty into kommuneArray
>
> put "file:" & specialFolderPath("resources")& "/Kommunedata.txt" into
> tFile
>
> put textDecode(url tFile,"UTF-8") into tData
>
> put jsonToArray(textDecode(tData,"UTF-8")) into kommuneArray
>
> end preOpenStack
>
>
> The array generated contains all the right information, but keys that
> should include æ, ø or å do not show up correctly. Any ideas anyone?
>
> Best regards
> Tore Nilsen
>
>
> _______________________________________________
> use-livecode mailing list
> [hidden email]
> Please visit this url to subscribe, unsubscribe and manage your
> subscription preferences:
> http://lists.runrev.com/mailman/listinfo/use-livecode
_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|

Re: Problems with jsonToArray and special chars like: æøå

Niggemann, Bernd via use-livecode
Hi Panos.

This does not work, and it is somewhat similar to what I tried initially, although I am certain I should use textDecode when importing text to LiveCode, as per the Dictionary. Also note that tFile only holds the url to the file containing the JSON, and your suggestion only encodes the file path. The strange thing is that the text I try to import was encoded as “UTF-8” when it was initially exported from LiveCode. It shows up all right in all other applications that can read .txt files.

When I do only a single run of text decoding, no array is created. When I do a double decoding, the array is created, but the chars in question does not show up in the keys of the array. Another thing that puzzles me is that som values, like -0.2 or -0.4 will show up with som additional zeros in the JSON-text. When I add a space between the minus operator and the number, it is displayed correctly.

I am still at odd with what may be the cause of this.

Regards Tore  

> 6. jun. 2018 kl. 13:40 skrev panagiotis merakos via use-livecode <[hidden email]>:
>
> on preOpenStack
>  put empty into kommuneArray
>  put "file:" & specialFolderPath("resources")& "/Kommunedata.txt" into
> tFile
>  put textEncode(tFile,"UTF-8") into tData
>  put jsonToArray(tData) into kommuneArray
> end preOpenStack
>
> Best,
> Panos
> --

_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|

Re: Problems with jsonToArray and special chars like: æøå

Niggemann, Bernd via use-livecode
Hi Tore,

Sorry, the line:

put "file:" & specialFolderPath("resources")& "/Kommunedata.txt" into tFile

should be:

put url ("file:" & specialFolderPath("resources")& "/Kommunedata.txt") into
tFile

Does it work now?


>>>>>although I am certain I should use textDecode when importing text to
LiveCode
Yes, this is correct. But I think in that case you want to export text from
LC into the outside world (e.g. to the mergJSON external), so the text has
to be utf8-encoded

If this does not work, could you file a bug with a sample stack and the
json file so as we investigate further

Best,
Panos
--


On Wed, Jun 6, 2018 at 1:06 PM, Tore Nilsen via use-livecode <
[hidden email]> wrote:

> Hi Panos.
>
> This does not work, and it is somewhat similar to what I tried initially,
> although I am certain I should use textDecode when importing text to
> LiveCode, as per the Dictionary. Also note that tFile only holds the url to
> the file containing the JSON, and your suggestion only encodes the file
> path. The strange thing is that the text I try to import was encoded as
> “UTF-8” when it was initially exported from LiveCode. It shows up all right
> in all other applications that can read .txt files.
>
> When I do only a single run of text decoding, no array is created. When I
> do a double decoding, the array is created, but the chars in question does
> not show up in the keys of the array. Another thing that puzzles me is that
> som values, like -0.2 or -0.4 will show up with som additional zeros in the
> JSON-text. When I add a space between the minus operator and the number, it
> is displayed correctly.
>
> I am still at odd with what may be the cause of this.
>
> Regards Tore
>
> > 6. jun. 2018 kl. 13:40 skrev panagiotis merakos via use-livecode <
> [hidden email]>:
> >
> > on preOpenStack
> >  put empty into kommuneArray
> >  put "file:" & specialFolderPath("resources")& "/Kommunedata.txt" into
> > tFile
> >  put textEncode(tFile,"UTF-8") into tData
> >  put jsonToArray(tData) into kommuneArray
> > end preOpenStack
> >
> > Best,
> > Panos
> > --
>
> _______________________________________________
> use-livecode mailing list
> [hidden email]
> Please visit this url to subscribe, unsubscribe and manage your
> subscription preferences:
> http://lists.runrev.com/mailman/listinfo/use-livecode
>
_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode