Quantcast

Windows and OSX 64-bit builds?

classic Classic list List threaded Threaded
30 messages Options
12
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Windows and OSX 64-bit builds?

Mark Wieder via use-livecode
Hi folks,

Does anyone know if there are plans for 64 bit windows and mac builds
anytime soon?.... I'm bumping up against limits that seems all too ancient
to be deallng with in 2017.

Just wanna know if its wishful thinking at this point or if its feasible to
wait for 65 bit builds and save myself the work of optimizng for 32-bit?

Thanks,


*Tom Glod*

CEO @ *MakeShyft R.D.A* - www.makeshyft.com



Developer of *U.M.P* - www.IamUMP.com
_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Windows and OSX 64-bit builds?

Mark Wieder via use-livecode
On 2/9/2017 8:40 PM, Tom Glod via use-livecode wrote:
> Hi folks,
>
> Does anyone know if there are plans for 64 bit windows and mac builds
> anytime soon?.... I'm bumping up against limits that seems all too ancient
> to be deallng with in 2017.
>

Out of curiosity, what specific LC limits are you running into? Do you
need a field to hold more than 4GB of text? A limit on sizes of Arrays?
Something else?

_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Windows and OSX 64-bit builds?

Mark Wieder via use-livecode
In reply to this post by Mark Wieder via use-livecode

> On 10 Feb 2017, at 12:40 pm, Tom Glod via use-livecode <[hidden email]> wrote:
>
> Does anyone know if there are plans for 64 bit windows and mac builds
> anytime soon?.... I'm bumping up against limits that seems all too ancient
> to be deallng with in 2017.
>
> Just wanna know if its wishful thinking at this point or if its feasible to
> wait for 65 bit builds and save myself the work of optimizng for 32-bit?

Hi Tom

Mac 64 bit has been available for quite some time now. Windows 64 bit is in motion.

Cheers

Monte
_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Windows and OSX 64-bit builds?

Mark Wieder via use-livecode
In reply to this post by Mark Wieder via use-livecode
trying to export snapshots of lage groups with many image controls. ....
can't get past 10k

On Thu, Feb 9, 2017 at 8:49 PM, Paul Dupuis via use-livecode <
[hidden email]> wrote:

> On 2/9/2017 8:40 PM, Tom Glod via use-livecode wrote:
> > Hi folks,
> >
> > Does anyone know if there are plans for 64 bit windows and mac builds
> > anytime soon?.... I'm bumping up against limits that seems all too
> ancient
> > to be deallng with in 2017.
> >
>
> Out of curiosity, what specific LC limits are you running into? Do you
> need a field to hold more than 4GB of text? A limit on sizes of Arrays?
> Something else?
>
> _______________________________________________
> use-livecode mailing list
> [hidden email]
> Please visit this url to subscribe, unsubscribe and manage your
> subscription preferences:
> http://lists.runrev.com/mailman/listinfo/use-livecode
>



--
*Tom Glod*

CEO @ *MakeShyft R.D.A* - www.makeshyft.com



Developer of *U.M.P* - www.IamUMP.com
_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Windows and OSX 64-bit builds?

Mark Wieder via use-livecode
thanks monte..... thats what i wanted to hear....sorry i didn't notice the
osx 64 builds..... only ever saw llinux 64...... but i work on
windows....so its music to my ears..... what are we looking at ?...a few
months?

On Thu, Feb 9, 2017 at 9:03 PM, Tom Glod <[hidden email]> wrote:

> trying to export snapshots of lage groups with many image controls. ....
> can't get past 10k
>
> On Thu, Feb 9, 2017 at 8:49 PM, Paul Dupuis via use-livecode <
> [hidden email]> wrote:
>
>> On 2/9/2017 8:40 PM, Tom Glod via use-livecode wrote:
>> > Hi folks,
>> >
>> > Does anyone know if there are plans for 64 bit windows and mac builds
>> > anytime soon?.... I'm bumping up against limits that seems all too
>> ancient
>> > to be deallng with in 2017.
>> >
>>
>> Out of curiosity, what specific LC limits are you running into? Do you
>> need a field to hold more than 4GB of text? A limit on sizes of Arrays?
>> Something else?
>>
>> _______________________________________________
>> use-livecode mailing list
>> [hidden email]
>> Please visit this url to subscribe, unsubscribe and manage your
>> subscription preferences:
>> http://lists.runrev.com/mailman/listinfo/use-livecode
>>
>
>
>
> --
> *Tom Glod*
>
> CEO @ *MakeShyft R.D.A* - www.makeshyft.com
>
>
>
> Developer of *U.M.P* - www.IamUMP.com
>



--
*Tom Glod*

CEO @ *MakeShyft R.D.A* - www.makeshyft.com



Developer of *U.M.P* - www.IamUMP.com
_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Windows and OSX 64-bit builds?

Mark Wieder via use-livecode

> On 10 Feb 2017, at 1:04 pm, Tom Glod via use-livecode <[hidden email]> wrote:
>
> thanks monte..... thats what i wanted to hear....sorry i didn't notice the
> osx 64 builds..... only ever saw llinux 64...... but i work on
> windows....so its music to my ears..... what are we looking at ?...a few
> months?

I’m not really sure when the first versions will make a stable release but the work required to get things building on our build servers is on the go. It requires some drastic changes to things is all I know. Peter, Mark or Fraser would be better to give you more detail than that ;-)

Cheers

Monte
_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Windows and OSX 64-bit builds?

Mark Wieder via use-livecode
sounds good.... i'll wait patiently....thanks alot for confirming its a WIP.

On Thu, Feb 9, 2017 at 9:08 PM, Monte Goulding via use-livecode <
[hidden email]> wrote:

>
> > On 10 Feb 2017, at 1:04 pm, Tom Glod via use-livecode <
> [hidden email]> wrote:
> >
> > thanks monte..... thats what i wanted to hear....sorry i didn't notice
> the
> > osx 64 builds..... only ever saw llinux 64...... but i work on
> > windows....so its music to my ears..... what are we looking at ?...a few
> > months?
>
> I’m not really sure when the first versions will make a stable release but
> the work required to get things building on our build servers is on the go.
> It requires some drastic changes to things is all I know. Peter, Mark or
> Fraser would be better to give you more detail than that ;-)
>
> Cheers
>
> Monte
> _______________________________________________
> use-livecode mailing list
> [hidden email]
> Please visit this url to subscribe, unsubscribe and manage your
> subscription preferences:
> http://lists.runrev.com/mailman/listinfo/use-livecode
>



--
*Tom Glod*

CEO @ *MakeShyft R.D.A* - www.makeshyft.com



Developer of *U.M.P* - www.IamUMP.com
_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Windows and OSX 64-bit builds?

Mark Wieder via use-livecode
In reply to this post by Mark Wieder via use-livecode

> On 10 Feb 2017, at 1:03 pm, Tom Glod via use-livecode <[hidden email]> wrote:
>
> trying to export snapshots of lage groups with many image controls. ....
> can't get past 10k

10k controls or 10k pixels. Control rect origins are 16 bit ints and sizes are 16 bit unsigned ints so if you are trying to position a control at > 32767 then it won’t work. That won’t change when we compile the engine as 64 bit.

Cheers

Monte
_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Windows and OSX 64-bit builds?

Mark Wieder via use-livecode
10 K in resolution....

I'm going to do more tests with the 2 cache settings ..... compositor cache
and image cache..... increasing these definately lets me export snapshots
of larger groups...but I have not succeeded in going past 10 K.... I can
display up to 32k..... but snapshot export just hard crashes Livecode as
soon as it is triggered.  .... 32,000*32,000*4 is 4GB+ ...... I would like
my customers to be able to output their content as large images.  10 k is
large enough for now.
_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Windows and OSX 64-bit builds?

Mark Wieder via use-livecode

> On 10 Feb 2017, at 1:33 pm, Tom Glod via use-livecode <[hidden email]> wrote:
>
> but snapshot export just hard crashes Livecode as
> soon as it is triggered

If you have a recipe for a hard crash please post  bug report

Cheers

Monte
_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Windows and OSX 64-bit builds?

Mark Wieder via use-livecode
I will... if u wanna replicate...put an image on a stack..make it 32k x 32k
.....and try and do a export snapshot of the image,  LC goes POOF... Trevor
said tha last version of 8 (8.13)  had some memory issues solves, so i will
try to test is there too.

On Thu, Feb 9, 2017 at 9:35 PM, Monte Goulding via use-livecode <
[hidden email]> wrote:

>
> > On 10 Feb 2017, at 1:33 pm, Tom Glod via use-livecode <
> [hidden email]> wrote:
> >
> > but snapshot export just hard crashes Livecode as
> > soon as it is triggered
>
> If you have a recipe for a hard crash please post  bug report
>
> Cheers
>
> Monte
> _______________________________________________
> use-livecode mailing list
> [hidden email]
> Please visit this url to subscribe, unsubscribe and manage your
> subscription preferences:
> http://lists.runrev.com/mailman/listinfo/use-livecode
>



--
*Tom Glod*

CEO @ *MakeShyft R.D.A* - www.makeshyft.com



Developer of *U.M.P* - www.IamUMP.com
_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Windows and OSX 64-bit builds?

Mark Wieder via use-livecode
In reply to this post by Mark Wieder via use-livecode
On 2/9/2017 9:19 PM, Monte Goulding via use-livecode wrote:
>> On 10 Feb 2017, at 1:03 pm, Tom Glod via use-livecode <[hidden email]> wrote:
>>
>> trying to export snapshots of lage groups with many image controls. ....
>> can't get past 10k
> 10k controls or 10k pixels. Control rect origins are 16 bit ints and sizes are 16 bit unsigned ints so if you are trying to position a control at > 32767 then it won’t work. That won’t change when we compile the engine as 64 bit.
>
>
This is one reason I asked what specific limits are being run into. Juts
making "64-bit" versions of LC for OSX or Window will not magically fix
all limits in LiveCode that exists. Many are based on a coded 16 bit or
32 bit bounds. When running into limits, it is better to report the
specific limit - as a bug report at http://quality.livecode.com/ -
rather than looking for 64 bit versions of the engine. It *may* be that
64 bit address spaces is needed to expand the limit, but the again, it
may not.


_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Windows and OSX 64-bit builds?

Mark Wieder via use-livecode
In reply to this post by Mark Wieder via use-livecode
Tom Glod wrote:

 > 10 K in resolution....
 >
 > I'm going to do more tests with the 2 cache settings ..... compositor
 > cache and image cache..... increasing these definately lets me export
 > snapshots of larger groups...but I have not succeeded in going past
 > 10 K.... I can display up to 32k..... but snapshot export just hard
 > crashes Livecode as soon as it is triggered.  .... 32,000*32,000*4 is
 > 4GB+ ...... I would like my customers to be able to output their
 > content as large images.  10 k is large enough for now.

Curious about this, I ran a test in LC 9 with a new stack with one
button containing this script:

on mouseUp
    set the rect of me to 0,0,6000,6000
    put specialFolderPath("desktop")&"/TestBigImage.png" into tFile
    export snapshot from me to file tFile as PNG
end mouseUp

On Ubuntu 14.04 w/8GB RAM I get the image file generated within seconds,
but then LC takes another several seconds (almost a full minute!) with
one CPU core maxed until it goes back to a normal idle.

Worse, while writing this email I was switching back and forth between
my email client and LC, and apparently resume also takes nearly a full
minute of maxed CPU before I'm able to work.

I'll run strace with that and file a bug report later to see what could
be done, but back to your app's need:

Is raster output the best option for your users?

Even at 10k px that'll be a pretty big file, unwieldy in many image apps
(and apparently prohibitive to export in the Linux version of LC currently).

It would be tedious but not too difficult to write a CardToSVG function
instead, giving your users a widely-supported vector format whose file
will be only slightly larger than the stack it was generated from
(relatively speaking; being a plain-text format I'd guess the output
size would be a small multiple of the stack file size, but certainly far
less than a raster representation of the same layout).

I know Alejandro had written some SVG importers some time ago - anyone
here have at least the beginnings of an SVG exporter we could build upon
as a community project?

--
  Richard Gaskin
  Fourth World Systems
  Software Design and Development for the Desktop, Mobile, and the Web
  ____________________________________________________________________
  [hidden email]                http://www.FourthWorld.com

_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Windows and OSX 64-bit builds?

Mark Wieder via use-livecode
In reply to this post by Mark Wieder via use-livecode
That is a 28 FOOT image!!! at 8 bits per pixel that's 8.2 terabytes. At 24 bits per pixel it's 24.5 terabytes!!!!! Am I missing something?? My math skills may not be the best:

32000*32000*24

Granted LC should never crash, it whould warn the end user that the data exceeds limits, but if I understand what you are doing, it's unreasonable to expect any application to do this.

Bob S



> On Feb 9, 2017, at 18:38 , Tom Glod via use-livecode <[hidden email]> wrote:
>
> I will... if u wanna replicate...put an image on a stack..make it 32k x 32k
> .....and try and do a export snapshot of the image,  LC goes POOF... Trevor
> said tha last version of 8 (8.13)  had some memory issues solves, so i will
> try to test is there too.


_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Windows and OSX 64-bit builds?

Mark Wieder via use-livecode
On Fri, Feb 10, 2017 at 8:54 AM, Bob Sneidar via use-livecode <
[hidden email]> wrote:

> Granted LC should never crash, it whould warn the end user that the data
> exceeds limits, but if I understand what you are doing, it's unreasonable
> to expect any application to do this.
>
>
An, err, long time ago, I found that I could consistently crash both
FreeBSD and Linux by loading a file larger than virtual memory into a
binary editor . . .  (I had had to make an image of a hard disk to search
for versions of a paper when ~ got deleted.  While an older machine, the
hard drive than even VM on the other machine.  I eventually got the paper
back [maybe I split the file?]   . . . just manipulating it was no small
task.  [maybe I finally filtered the disk through strings or tr to a file
on a zip disk for transport?])


--
Dr. Richard E. Hawkins, Esq.
(702) 508-8462
_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Windows and OSX 64-bit builds?

Mark Wieder via use-livecode
In reply to this post by Mark Wieder via use-livecode
On 2017-02-10 03:38, Tom Glod via use-livecode wrote:
> I will... if u wanna replicate...put an image on a stack..make it 32k x
> 32k
> .....and try and do a export snapshot of the image,  LC goes POOF...
> Trevor
> said tha last version of 8 (8.13)  had some memory issues solves, so i
> will
> try to test is there too.

Currently, the engine implements 'export snapshot' by allocating a
raster (32-bits per pixel) of the size of the snapshot you are making,
rendering into it and then compressing it.

So really the maximum size you could hope to snapshot is 16k x* 16k
pixels as that requires 2Gb - the engine in general uses signed integer
indicies, so the maximum memory buffer it can manipulate is 2Gb bytes. A
32-bit process would probably struggle to do that (due to only having
around 2-3Gb of user address space to use) - as there is overhead in
rasterization and then compression; but a 64-bit process should be fine.

There is a bug here as (at least in this specific case) the engine
should fail gracefully (as we know there is a hard limit in the size of
an image the engine can process).

As you correctly point out 32k x 32k comes in at 1Gb pixels - which at
24-bit RGB comes out at 4Gb of data. No 'normal' 32-bit application
which isn't explicitly designed for manipulating huge images will be
able to deal with something that size. I would expect applications such
as Photoshop to be able to deal with them though since I believe their
native raw storage format for images pages from disk as required (so you
never have the 'whole thing' in memory at once - just the bit you are
looking at / editing).

One important thing to remember is that the amount of memory required to
take a snapshot is (SOMECONSTANT * 4 * the number of pixels) in the rect
of the snapshot (I've not checked but I would estimate 0 < SOMECONSTANT
< 2) which means that you can get LiveCode to generate very large
images, but you have to break the problem down by splitting up the
snapshot into bands and probably use an external (command-line) tool to
compress the image into your format of choice (how big an image such a
tool can process, again, will be dependent on whether it needs to load
the entire image into memory to compress it or not).

Rough idea:

     repeat with i = 0 to pHeight / kBandSize

       import snapshot from rect (0, pWidth, i * kBandSize, kBandSize)

       write the imageData of the last image to file tOutputFile

       delete the last image

    end repeat

After this you will have a very large file with the raw xRGB data in it,
so you need to find a tool which can take raw 32-bit xRGB data (with
specified order of the RGB), the width and height and process it into
jpg or png or whatever format you require (I'm hoping others who know
more about such things might be able to chime in here - ImageMagick has
an arsenal of command-line tools, for example).

Warmest Regards,

Mark.

P.S. There is a hard limit of co-ordinate magnitude in LC and thus the
size of any object - 32767 pixels on any side of anything.

--
Mark Waddingham ~ [hidden email] ~ http://www.livecode.com/
LiveCode: Everyone can create apps


_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Windows and OSX 64-bit builds?

Mark Wieder via use-livecode
In reply to this post by Mark Wieder via use-livecode
Please post this "Split it!"- answer, as it is, in LC's blog.
This is good even for real beginners.

Large files or large data shouldn't be a reason for _incomplete_ 64Bit
implementations that would make once again LC Script slower.

> Mark Waddingham wrote:
> >
> > Tom Glod wrote:
> > I will... if u wanna replicate...put an image on a stack..make it
> > 32k x 32k
> > ..... and try and do a export snapshot of the image,  LC goes POOF...
> > Trevor
> > said tha last version of 8 (8.13)  had some memory issues solves, so i
> > will try to test is there too.
>
> Currently, the engine implements 'export snapshot' by allocating a
> raster (32-bits per pixel) of the size of the snapshot you are making,
> rendering into it and then compressing it.
>
> So really the maximum size you could hope to snapshot is 16k x* 16k
> pixels as that requires 2Gb - the engine in general uses signed integer
> indicies, so the maximum memory buffer it can manipulate is 2Gb bytes. A
> 32-bit process would probably struggle to do that (due to only having
> around 2-3Gb of user address space to use) - as there is overhead in
> rasterization and then compression; but a 64-bit process should be fine.
>
> There is a bug here as (at least in this specific case) the engine
> should fail gracefully (as we know there is a hard limit in the size of
> an image the engine can process).
>
> As you correctly point out 32k x 32k comes in at 1Gb pixels - which at
> 24-bit RGB comes out at 4Gb of data. No 'normal' 32-bit application
> which isn't explicitly designed for manipulating huge images will be
> able to deal with something that size. I would expect applications such
> as Photoshop to be able to deal with them though since I believe their
> native raw storage format for images pages from disk as required (so you
> never have the 'whole thing' in memory at once - just the bit you are
> looking at / editing).
>
> One important thing to remember is that the amount of memory required to
> take a snapshot is (SOMECONSTANT * 4 * the number of pixels) in the rect
> of the snapshot (I've not checked but I would estimate 0 < SOMECONSTANT
> < 2) which means that you can get LiveCode to generate very large
> images, but you have to break the problem down by splitting up the
> snapshot into bands and probably use an external (command-line) tool to
> compress the image into your format of choice (how big an image such a
> tool can process, again, will be dependent on whether it needs to load
> the entire image into memory to compress it or not).
>
> Rough idea:
>
>      repeat with i = 0 to pHeight / kBandSize
>         import snapshot from rect (0, pWidth, i * kBandSize, kBandSize)
>         write the imageData of the last image to file tOutputFile
>         delete the last image
>      end repeat
>
> After this you will have a very large file with the raw xRGB data in it,
> so you need to find a tool which can take raw 32-bit xRGB data (with
> specified order of the RGB), the width and height and process it into
> jpg or png or whatever format you require (I'm hoping others who know
> more about such things might be able to chime in here - ImageMagick has
> an arsenal of command-line tools, for example).
>
> Warmest Regards,
>
> Mark.
>
> P.S. There is a hard limit of co-ordinate magnitude in LC and thus the
> size of any object - 32767 pixels on any side of anything.

_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Windows and OSX 64-bit builds?

Mark Wieder via use-livecode
can't  thank you enough for all the extra info mark....I'm going to read it
over a few times to make sure I got everything I could out of it.   will
re-engage this idea as I go forward ... I still need to lock down my
maximum presets, so I will still have to play with this in a week or so.

Thanks

On Fri, Feb 10, 2017 at 5:29 PM, hh via use-livecode <
[hidden email]> wrote:

> Please post this "Split it!"- answer, as it is, in LC's blog.
> This is good even for real beginners.
>
> Large files or large data shouldn't be a reason for _incomplete_ 64Bit
> implementations that would make once again LC Script slower.
>
> > Mark Waddingham wrote:
> > >
> > > Tom Glod wrote:
> > > I will... if u wanna replicate...put an image on a stack..make it
> > > 32k x 32k
> > > ..... and try and do a export snapshot of the image,  LC goes POOF...
> > > Trevor
> > > said tha last version of 8 (8.13)  had some memory issues solves, so i
> > > will try to test is there too.
> >
> > Currently, the engine implements 'export snapshot' by allocating a
> > raster (32-bits per pixel) of the size of the snapshot you are making,
> > rendering into it and then compressing it.
> >
> > So really the maximum size you could hope to snapshot is 16k x* 16k
> > pixels as that requires 2Gb - the engine in general uses signed integer
> > indicies, so the maximum memory buffer it can manipulate is 2Gb bytes. A
> > 32-bit process would probably struggle to do that (due to only having
> > around 2-3Gb of user address space to use) - as there is overhead in
> > rasterization and then compression; but a 64-bit process should be fine.
> >
> > There is a bug here as (at least in this specific case) the engine
> > should fail gracefully (as we know there is a hard limit in the size of
> > an image the engine can process).
> >
> > As you correctly point out 32k x 32k comes in at 1Gb pixels - which at
> > 24-bit RGB comes out at 4Gb of data. No 'normal' 32-bit application
> > which isn't explicitly designed for manipulating huge images will be
> > able to deal with something that size. I would expect applications such
> > as Photoshop to be able to deal with them though since I believe their
> > native raw storage format for images pages from disk as required (so you
> > never have the 'whole thing' in memory at once - just the bit you are
> > looking at / editing).
> >
> > One important thing to remember is that the amount of memory required to
> > take a snapshot is (SOMECONSTANT * 4 * the number of pixels) in the rect
> > of the snapshot (I've not checked but I would estimate 0 < SOMECONSTANT
> > < 2) which means that you can get LiveCode to generate very large
> > images, but you have to break the problem down by splitting up the
> > snapshot into bands and probably use an external (command-line) tool to
> > compress the image into your format of choice (how big an image such a
> > tool can process, again, will be dependent on whether it needs to load
> > the entire image into memory to compress it or not).
> >
> > Rough idea:
> >
> >      repeat with i = 0 to pHeight / kBandSize
> >         import snapshot from rect (0, pWidth, i * kBandSize, kBandSize)
> >         write the imageData of the last image to file tOutputFile
> >         delete the last image
> >      end repeat
> >
> > After this you will have a very large file with the raw xRGB data in it,
> > so you need to find a tool which can take raw 32-bit xRGB data (with
> > specified order of the RGB), the width and height and process it into
> > jpg or png or whatever format you require (I'm hoping others who know
> > more about such things might be able to chime in here - ImageMagick has
> > an arsenal of command-line tools, for example).
> >
> > Warmest Regards,
> >
> > Mark.
> >
> > P.S. There is a hard limit of co-ordinate magnitude in LC and thus the
> > size of any object - 32767 pixels on any side of anything.
>
> _______________________________________________
> use-livecode mailing list
> [hidden email]
> Please visit this url to subscribe, unsubscribe and manage your
> subscription preferences:
> http://lists.runrev.com/mailman/listinfo/use-livecode
>



--
*Tom Glod*

CEO @ *MakeShyft R.D.A* - www.makeshyft.com



Developer of *U.M.P* - www.IamUMP.com
_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Windows and OSX 64-bit builds?

Mark Wieder via use-livecode
In reply to this post by Mark Wieder via use-livecode
I would second that, with an extension:  it would be most helpful to
have some guidance on memory management overall.

Many see the total addressable space afforded by 64-bit systems as
"Cool, I can use a 16-terabyte array!"

But as we've seen in bits and pieces here and elsewhere, total
addressable memory is not what LC will necessarily use, or can use, for
many tasks.  And it some cases it may not be what we really want to do.

OS APIs impose their own limits, and many other factors can come into
play as well.

It would be useful to have at least some general guidance on what those
limitations are, both in terms of LC as it is today and in terms where
it bumps into system limits and general memory management challenges.

Often these implications aren't obvious.

For example, this morning I noted the difficulty with exporting an image
of a 5000 X 5000 px button on Ubuntu.  Later I tried a set of much
smaller objects but spread out much more broadly (10k px and more), and
LC remained very responsive throughout - even exported the snapshot fine.

So I'm guessing from this that the difficulty with making unusually
large standard controls was specific to how Linux' GDK handles them
internally.  Indeed, making LC-handled objects, like graphics,
apparently have no problem being rendered and buffered at such sizes.

Rather than make this an attempt at an exhaustive study of memory
implications (though I'd gladly read it, it would take too long to
write), it might be nice if Mark could draw from examples he's seen in
the community over the years to outline some of the more commonly
requested scenarios.

--
  Richard Gaskin


hh wrote:

> Please post this "Split it!"- answer, as it is, in LC's blog.
> This is good even for real beginners.
>
> Large files or large data shouldn't be a reason for _incomplete_ 64Bit
> implementations that would make once again LC Script slower.
>
>> Mark Waddingham wrote:
>> >
>> > Tom Glod wrote:
>> > I will... if u wanna replicate...put an image on a stack..make it
>> > 32k x 32k
>> > ..... and try and do a export snapshot of the image,  LC goes POOF...
>> > Trevor
>> > said tha last version of 8 (8.13)  had some memory issues solves, so i
>> > will try to test is there too.
>>
>> Currently, the engine implements 'export snapshot' by allocating a
>> raster (32-bits per pixel) of the size of the snapshot you are making,
>> rendering into it and then compressing it.
>>
>> So really the maximum size you could hope to snapshot is 16k x* 16k
>> pixels as that requires 2Gb - the engine in general uses signed integer
>> indicies, so the maximum memory buffer it can manipulate is 2Gb bytes. A
>> 32-bit process would probably struggle to do that (due to only having
>> around 2-3Gb of user address space to use) - as there is overhead in
>> rasterization and then compression; but a 64-bit process should be fine.
>>
>> There is a bug here as (at least in this specific case) the engine
>> should fail gracefully (as we know there is a hard limit in the size of
>> an image the engine can process).
>>
>> As you correctly point out 32k x 32k comes in at 1Gb pixels - which at
>> 24-bit RGB comes out at 4Gb of data. No 'normal' 32-bit application
>> which isn't explicitly designed for manipulating huge images will be
>> able to deal with something that size. I would expect applications such
>> as Photoshop to be able to deal with them though since I believe their
>> native raw storage format for images pages from disk as required (so you
>> never have the 'whole thing' in memory at once - just the bit you are
>> looking at / editing).
>>
>> One important thing to remember is that the amount of memory required to
>> take a snapshot is (SOMECONSTANT * 4 * the number of pixels) in the rect
>> of the snapshot (I've not checked but I would estimate 0 < SOMECONSTANT
>> < 2) which means that you can get LiveCode to generate very large
>> images, but you have to break the problem down by splitting up the
>> snapshot into bands and probably use an external (command-line) tool to
>> compress the image into your format of choice (how big an image such a
>> tool can process, again, will be dependent on whether it needs to load
>> the entire image into memory to compress it or not).
>>
>> Rough idea:
>>
>>      repeat with i = 0 to pHeight / kBandSize
>>         import snapshot from rect (0, pWidth, i * kBandSize, kBandSize)
>>         write the imageData of the last image to file tOutputFile
>>         delete the last image
>>      end repeat
>>
>> After this you will have a very large file with the raw xRGB data in it,
>> so you need to find a tool which can take raw 32-bit xRGB data (with
>> specified order of the RGB), the width and height and process it into
>> jpg or png or whatever format you require (I'm hoping others who know
>> more about such things might be able to chime in here - ImageMagick has
>> an arsenal of command-line tools, for example).
>>
>> Warmest Regards,
>>
>> Mark.
>>
>> P.S. There is a hard limit of co-ordinate magnitude in LC and thus the
>> size of any object - 32767 pixels on any side of anything.


_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Windows and OSX 64-bit builds?

Mark Wieder via use-livecode
In reply to this post by Mark Wieder via use-livecode
isn't it kBytes not bits? So 32,000 * 32000 Bytes (a pixel takes up one Byte in 8 bit color) which comes to 1,024,000,000 BYTES. That's 1.024 terabytes, unless my faculties have wholly abandoned me. Of course, a black and white image is 1,024,000 BYTES, or 1.023 GIGS, but are we talking about black and white images?

That said, I am a couple strong ales in at this point, sooo...

Bob S


> On Feb 10, 2017, at 10:33 , Mark Waddingham via use-livecode <[hidden email]> wrote:
>
> As you correctly point out 32k x 32k comes in at 1Gb pixels - which at 24-bit RGB comes out at 4Gb of data.


_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
12
Loading...