Documentation on Accelerated Rendering

classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|

Documentation on Accelerated Rendering

Bob Sneidar via use-livecode
Our new modular app was crashing badly and also rendering bizarre screens on Android, HQ determined that if Accelerated rendering was on when you close one stack and opened a second one, it was the problem. But Acceleratedrendering is needed for performance, so it seems to work fine,if, instead using some  global backscript that sets acceleratedrending to True "whenever"…  if you explicitly turn it on/true in e the preopenstack handler and set it to false on the close stack or card *before* attempting to close the stack and open another one.. then things work.

But now I'm having issues with some modules where I turned on AccelerateRendering /true.

I would like "generalize" the use cases for acceleratedRendering and move that call out of the stack themselves into a library that is placed into the back script with "start using" But… it's too vague in my head right now to make an intelligent addition to a global library (or multiple library  handlers if that makes sense) that will serve all contexts without the breakage we had before.

My question is:  where is the documentation for AcceleratedRendering

-- exactly what does it do? (don't worry, if the explanation is very technical.. .let me see it!)
-- when does it help
-- what contexts may there be where it could actually impede performance?

I mean more than just "It helps with scrolling groups" but a robust analysis of what is going on, why and when it serves it's purpose etc.

it's not in the dictionary, and the guide has no search box (the intuitive thing a newbie would do for an "feature" of the language/engine would be to search and read all instances where it appears in the documentation, as her homework for that issue/feature/ bit-of-syntax.

BR



_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|

Re: Documentation on Accelerated Rendering

Bob Sneidar via use-livecode
On 4/29/17 9:56 PM, Sannyasin Brahmanathaswami via use-livecode wrote:
> My question is:  where is the documentation for AcceleratedRendering
>
> -- exactly what does it do? (don't worry, if the explanation is very technical.. .let me see it!)
> -- when does it help
> -- what contexts may there be where it could actually impede performance?

I have been wanting to know that for years. I have the general idea but
I want to know exactly what each command or property does. How does
layerMode interact with rendering and when should I set it? (I used it
incorrectly and submitted a spurious bug report, but I'm still not clear
on why I was wrong.)

Here is what (I think) I do know:

AcceleratedRendering creates an offscreen buffer for objects that have a
non-static layerMode. It then only needs to redraw the particular object
instead of the whole card when you move that object, which is much
faster. If it is a scrolling object (group or field) then the layermode
should be set to "scrolling". If you are going to drag things around or
use the "move" command, then those objects should be set to "dynamic".
If the object doesn't move, or only moves very rarely, its layermode
should stay at the default "static" and no buffer is created.

The compositorTileSize, compositorType, and compositorCacheLimit all
affect the size and behavior of the buffer(s). The defaults provided
have always worked for me so I don't change them. I'm not sure how I
would add up the memory requirements for a set of moving objects so I
stick with the defaults.

I'm guessing it would impede performance if it is creating buffers for
objects that aren't really going to move. You'd have the overhead of the
buffer creation and some decrease in available memory. And from my
incorrect bug report, I gather you shouldn't change layermode on the fly.

What I would like to know is:

- What does layerMode tell the engine, and when is it read? I think it
tells the engine to create a buffer for the object. I had read a long
time ago that it would help to reset the layermode to static if the
object was done moving, but there should have been a caveat: don't reset
if the object will move again later, because in that case you want to
preserve the existing buffer. That seems to imply that resetting
layermode to static will dump the buffer. Is that right?

- When set to true, what does acceleratedRendering actually do? Is it
effectively a command to scan every object on the card and create
buffers? Does it scan more than just the current card?

- Is it okay to turn it on and off as needed? Are there disadvantages to
that, assuming the objects won't be moving any more? Can I turn it on at
the beginning of a handler and turn it off at the end if those objects
won't be moving any more?

- When I turn it off, does it dump all the buffers? I think it does.

- What does it do between card changes? Does it keep old buffers until
the cache limit is hit?

--
Jacqueline Landman Gay         |     [hidden email]
HyperActive Software           |     http://www.hyperactivesw.com

_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|

Re: Documentation on Accelerated Rendering

Bob Sneidar via use-livecode
Jacque

thanks for the detailed query, or rather the detailed outline of the Shadows of the Unknown World of Rendering.

appropo the puzzle app/module we have in our new app, which is pretty much failing on Android completely.

@ the rest of you:
we get a quote from a dbase,
take snapshots of word in the correct position,
assign the position of the tile to a custom property of the tile
Tiles are draggable
and when the tile approaches it's "home" location we turn on an outer glow and it snaps into position, on completion, award message and link to read the page form the book the quote was taken from

background picture changes on every new puzzle.

I been working on this little game for nearly three years off and on. Jacque stepped in and opimized my code and took it to the next level

Works great on iOS, users love it (very addicting), but performance is so bad on Android that it's virtually blocked.

@jacque: ". And from my
    incorrect bug report, I gather you shouldn't change layermode on the fly."

so we can't set the layermode of newly created objects a dynamic GUI context where the UX is letting users create object to drag around?

Seems this would be a common algorithm for lot's of different games and "fun stuff" (I also have it working for images)

OK, I'm off to test now with setting the layer mode of the tiles, but it takes *forever* for the puzzle to generated on my old Nexus5.

BR
 

On 4/30/17, 11:52 AM, "use-livecode on behalf of J. Landman Gay via use-livecode" <[hidden email] on behalf of [hidden email]> wrote:

    I'm guessing it would impede performance if it is creating buffers for
    objects that aren't really going to move. You'd have the overhead of the
    buffer creation and some decrease in available memory. And from my
    incorrect bug report, I gather you shouldn't change layermode on the fly.
   
    What I would like to know is:

_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|

Re: Documentation on Accelerated Rendering

Bob Sneidar via use-livecode
Never mind

Jacque

you already have this in our script

  [snip]   set the borderwidth of the last image to "4"
        set the topcolor of the last image to tRGB
        set the layermode of the last image to "dynamic" -- jg: required for smooth movment on slower devices
    end repeat

and we are setting the acceleratedRending on in preopenCard and false on close card

Still no joy on android.
 

On 4/30/17, 3:39 PM, "use-livecode on behalf of Sannyasin Brahmanathaswami via use-livecode" <[hidden email] on behalf of [hidden email]> wrote:

    OK, I'm off to test now with setting the layer mode of the tiles, but it takes *forever* for the puzzle to generated on my old Nexus5.

_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|

Re: Documentation on Accelerated Rendering

Bob Sneidar via use-livecode
In reply to this post by Bob Sneidar via use-livecode
Did you already try the following?

Layer mode:
Use 8.1.4. Before that and in 9-dp6 exact layering is broken.

The combination:
set the acceleratedRendering to false
set the paintcompression to RLE
set the alwaysbuffer of <new image> to true

_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|

Re: Documentation on Accelerated Rendering

Bob Sneidar via use-livecode
In reply to this post by Bob Sneidar via use-livecode
My mistake was resetting the layermode after a drop. It does have to be set
after object creation but that should be the only time it's messed with.
Dragging was too jerky without acceleratedRendering, so I think that has to
remain.
--
Jacqueline Landman Gay         |     [hidden email]
HyperActive Software           |     http://www.hyperactivesw.com



On April 30, 2017 8:48:42 PM Sannyasin Brahmanathaswami via use-livecode
<[hidden email]> wrote:

> Never mind
>
> Jacque
>
> you already have this in our script
>
>   [snip]   set the borderwidth of the last image to "4"
>         set the topcolor of the last image to tRGB
>         set the layermode of the last image to "dynamic" -- jg: required for smooth
>         movment on slower devices
>     end repeat
>
> and we are setting the acceleratedRending on in preopenCard and false on
> close card
>
> Still no joy on android.
>
>
> On 4/30/17, 3:39 PM, "use-livecode on behalf of Sannyasin Brahmanathaswami
> via use-livecode" <[hidden email] on behalf of
> [hidden email]> wrote:
>
>     OK, I'm off to test now with setting the layer mode of the tiles, but it
>     takes *forever* for the puzzle to generated on my old Nexus5.
>
> _______________________________________________
> use-livecode mailing list
> [hidden email]
> Please visit this url to subscribe, unsubscribe and manage your
> subscription preferences:
> http://lists.runrev.com/mailman/listinfo/use-livecode



_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|

Re: Documentation on Accelerated Rendering

Bob Sneidar via use-livecode
In reply to this post by Bob Sneidar via use-livecode
Layering was definitely broken, that's for sure. But in this case we're
referring to the layermode of a draggable object. I haven't worked much
with his latest version yet, when I was doing it I was using fields instead
of images. But since the images are created on the fly from within LC I
assume they'd be RLE.

Aren't all images buffered by default? I thought so but maybe not.
--
Jacqueline Landman Gay         |     [hidden email]
HyperActive Software           |     http://www.hyperactivesw.com



On May 1, 2017 1:07:52 AM hh via use-livecode
<[hidden email]> wrote:

> Did you already try the following?
>
> Layer mode:
> Use 8.1.4. Before that and in 9-dp6 exact layering is broken.
>
> The combination:
> set the acceleratedRendering to false
> set the paintcompression to RLE
> set the alwaysbuffer of <new image> to true
>
> _______________________________________________
> use-livecode mailing list
> [hidden email]
> Please visit this url to subscribe, unsubscribe and manage your
> subscription preferences:
> http://lists.runrev.com/mailman/listinfo/use-livecode



_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|

Re: Documentation on Accelerated Rendering

Bob Sneidar via use-livecode
In reply to this post by Bob Sneidar via use-livecode
On 2017-04-30 23:52, J. Landman Gay via use-livecode wrote:
> I have been wanting to know that for years. I have the general idea
> but I want to know exactly what each command or property does. How
> does layerMode interact with rendering and when should I set it? (I
> used it incorrectly and submitted a spurious bug report, but I'm still
> not clear on why I was wrong.)

This is perhaps something we should have more clearly documented at the
time,
it has been a while since I last looked at it...

Your description is generally correct, but I'll see if I can elaborate
with
more technical details.

First of all, just a reminder about how the engine handles screen
updates...

The engine checks for the need to update the screen after every command
has executed, or after 'unlock screen' if the screen has been locked and
the unlock causes the lock count to reach zero.

During execution of a single command (when the screen is not locked), or
the execution of all commands if under lock screen, any required visual
changes to the object are accumulated in the stack's 'dirty region'.
This is a collection of rectangles (on the stack) which need updated.

The acceleratedRendering property changes the way the engine renders the
stack when changes are applied (either after each command, or after an
unlock screen which resets the lock count to zero).

When it is false, the engine creates an offscreen buffer big enough to
cover the dirty region, uses the dirty region as clip, then iterates
through all objects on the card back to front, rendering each in turn
into the single offscreen buffer. When that is complete, the engine
copies the offscreen buffer to the screen.

When acceleratedRendering is true, however, this changes. Accelerated
rendering mode views the tree of top-level objects as a flat sequence of
layers, ordered from back to front. Each layer can have one of three
layerModes:
   - static
   - dynamic
   - scrolling

The static mode indicates to the engine that that layer is not going to
move, so once rendered once (in theory) it and any thing it covers won't
need to be re-rendered.

The dynamic mode indicates to the engine that the layer is going to
move, so ideally should be rendered independently of everything else.

The scrolling mode is the same as dynamic *except* that it indicates
that the layer is just a view (clipped rectangle) into a larger area -
e.g. what you can see in a field currently, and what the field would
look like if rendered on an infinite screen with no clipping.

When there are multiple static mode objects next to each other in the
list, they are all elided into a single layer. e.g.

    Graphic - static (bottom object)
    Image - static
    Image - static
    Button - dynamic
    Image - static
    Field - static (top object)

This would generate three separately rendered layers - the bottom one
being the first three controls, the middle one being just the button,
the top on being the last two controls.

The key thing here is that these layers can just be composited - i.e.
they are existing bitmaps, which are blended together into the screen
buffer on demand - this is something GPUs are very very very good at. In
particular, when you have moving (dynamic) layers, moving those
layers/objects doesn't require re-rendering anything *just*
re-compositing to the screen with the layers in a different place. This
is the source of the speed improvement you can see with
acceleratedRendering (e.g. you can have, say, 100 objects moving about
smoothly whereas you'd only manage maybe 10 before).

This is not the whole story, as acceleratedRendering mode makes one
other 'optimization' - it doesn't actually use object-sized buffers for
any layer. Instead, it splits up each (merged) layer into tiles with
size the compositorTileSize. So the compositing step is actually
blending together lots of small bitmaps, rather than a collection of
large bitmaps - this makes no real difference to performance (as it is
number of pixels blended which is the key factor, not number of bitmaps)
but does to updating things when any object changes as only those tiles
which intersect with the dirty region need to be considered...
Furthermore, in acceleratedRendering mode, the 'virtual layers' (which
come about through merging together adjacent statics, but leaving
dynamic and scrolling ones alone) each have their own dirty region. This
means, much less re-rendering needs to take place (in any one case)
*and* much less data transfer from memory -> compositor's tile storage /
format need to take place.

In terms of the compositor properties:

   - the compositorType determines what kind of technology the blending
step uses.
     There is a software mode, which works on all platforms and just uses
the
     engine's not-too-shabby compositing code.
     There is a coregraphics mode, which works on Mac/iOS, which uses
CoreGraphics
     (which gives an advantage because CoreGraphics images can be blitted
direct
      to the window buffer)
     There is OpenGL mode, which works on Android/iOS, this uses
tile-sized
     textures on the GPU, and the GPU to do all the blending.

   - the compositorTileSize determines the size of tile to use (which
must be
     a power-of-two up to 256 pixels).

   - the compositorCacheLimit determines the maximum amount of memory to
use
     for tiles.

The third point here needs a little bit of explanation. As
acceleratedRendering mode splits things up into tiles, it can leave
rendering any tile which is not
visible until it is actually visible. For example, if you have a large
opaque dynamic layer over a stack of static layers, chances are that you
will never see most of the static layers underneath, so the tiles for
the (virtual) static layer will never need to be rendered, or stored -
so it doesn't. This minimizes the number of tiles/textures/images which
are created, and minimizes rendering time for any one update.

Now to answer you direct questions:

> - What does layerMode tell the engine, and when is it read? I think it
> tells the engine to create a buffer for the object. I had read a long
> time ago that it would help to reset the layermode to static if the
> object was done moving, but there should have been a caveat: don't
> reset if the object will move again later, because in that case you
> want to preserve the existing buffer. That seems to imply that
> resetting layermode to static will dump the buffer. Is that right?

When you change the layerMode it will cause a recalculation of layers
around it (potentially) when the current dirty list (collection of
per-layer dirty regions) is processed. If a layer is not going to move
for quite a while, then you it might be reasonable to set it to static,
as long as you can anticipate when it might move so you can set it to
dynamic again (setting to dynamic and then moving immediately is likely
to cause a stutter).

> - When set to true, what does acceleratedRendering actually do? Is it
> effectively a command to scan every object on the card and create
> buffers? Does it scan more than just the current card?

It changes the approach to rendering as described above. The
accelRendering buffers, tiles and such are keyed to the stack, but
flushed whenever there is a card change - so essentially it is card
local (but you only ever have one accelRendering context per stack).

> - Is it okay to turn it on and off as needed? Are there disadvantages
> to that, assuming the objects won't be moving any more? Can I turn it
> on at the beginning of a handler and turn it off at the end if those
> objects won't be moving any more?

This would be ill-advised unless all movement you want is happening
within that handler. Turning acceleratedRendering on will mean the
engine has to re-render all visible tiles in all the virtual layers;
turning it off again deletes the accelRendering context for the stack,
dumps all its tiles and then re-renders the stack using the normal
painter's algorithm into offscreen buffer approach.

> - When I turn it off, does it dump all the buffers? I think it does.

Yes.

> - What does it do between card changes? Does it keep old buffers until
> the cache limit is hit?

No - the buffer is flushed whenever the card changes.

Hope this helps!

Warmest Regards,

Mark.

--
Mark Waddingham ~ [hidden email] ~ http://www.livecode.com/
LiveCode: Everyone can create apps

_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|

Re: Documentation on Accelerated Rendering

Bob Sneidar via use-livecode
In reply to this post by Bob Sneidar via use-livecode
Thanks for that, this was very enlightening (should be, once
again, attached to the Dictionary).

If I understand correctly, it explains why testing for speed
has reliable results only on _physical_ mobile devices.

And, while in physical desktop environments a stack/ an app may
run correctly, virtual desktop systems may extremely slow down if
host and guest have different default compositor properties. Even
if the virtual engine tries to use the host's GPU.


_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|

Re: Documentation on Accelerated Rendering

Bob Sneidar via use-livecode
On 2017-05-03 14:07, hh via use-livecode wrote:

> Thanks for that, this was very enlightening (should be, once
> again, attached to the Dictionary).
>
> If I understand correctly, it explains why testing for speed
> has reliable results only on _physical_ mobile devices.
>
> And, while in physical desktop environments a stack/ an app may
> run correctly, virtual desktop systems may extremely slow down if
> host and guest have different default compositor properties. Even
> if the virtual engine tries to use the host's GPU.
>

To be fair, speed tests are only really reliable on the devices you
are wanting to run your code on. When using any sort of GPU, it will
depend on the GPU (although, it should be noted that if the GPU is
only being used for blending - like accelRendering does - then the
cost to update a frame which has no changes to any objects is pretty
constant - and directly proportional to the number of (partially)
visible tiles). If using software (or CG modes) then it will depend
on the speed of the CPU.

At least when acceleratedRendering was written, the CoreGraphics and
software modes running on a decent Mac desktop at the time were
about the same, if not a bit faster than using OpenGL compositor on
a recent iDevice at the time.

Warmest Regards,

Mark.

--
Mark Waddingham ~ [hidden email] ~ http://www.livecode.com/
LiveCode: Everyone can create apps

_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|

Re: Documentation on Accelerated Rendering

Bob Sneidar via use-livecode
In reply to this post by Bob Sneidar via use-livecode
On 5/3/17 6:00 AM, Mark Waddingham via use-livecode wrote:
> On 2017-04-30 23:52, J. Landman Gay via use-livecode wrote:
>> I have been wanting to know that for years. I have the general idea
>> but I want to know exactly what each command or property does. How
>> does layerMode interact with rendering and when should I set it? (I
>> used it incorrectly and submitted a spurious bug report, but I'm still
>> not clear on why I was wrong.)
>
> This is perhaps something we should have more clearly documented at the
> time

Thanks very much for the detailed explanation. It clarifies a lot for
me, I really appreciate the time you took to write that up.

It sounds like there will be delays in the case of objects that are
created on the fly and assigned a dynamic layermode. I'm thinking of
Swami's puzzle where new snapshots are created every time a new puzzle
is made. These are dragged around the card by the user.

I assume the buffers are destroyed when an object is deleted. When a new
puzzle is started, the game deletes all the old snapshots, then creates
new ones and assigns the layermode. Is there an ideal way to handle
this? On iOS it's fine, on Android there is significant delay while the
puzzle is built.

Would changing the default compositor help? Or are we just at the mercy
of the graphics card?

I had originally used fields instead of snapshots thinking that would be
faster, but since both are buffered as bitmaps in either case, I guess
it doesn't matter.

--
Jacqueline Landman Gay         |     [hidden email]
HyperActive Software           |     http://www.hyperactivesw.com

_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|

Re: Documentation on Accelerated Rendering

Bob Sneidar via use-livecode
I think I want to have Mark's baby.

On Wed, May 3, 2017 at 2:38 PM, J. Landman Gay via use-livecode <
[hidden email]> wrote:

> On 5/3/17 6:00 AM, Mark Waddingham via use-livecode wrote:
>
>> On 2017-04-30 23:52, J. Landman Gay via use-livecode wrote:
>>
>>> I have been wanting to know that for years. I have the general idea
>>> but I want to know exactly what each command or property does. How
>>> does layerMode interact with rendering and when should I set it? (I
>>> used it incorrectly and submitted a spurious bug report, but I'm still
>>> not clear on why I was wrong.)
>>>
>>
>> This is perhaps something we should have more clearly documented at the
>> time
>>
>
> Thanks very much for the detailed explanation. It clarifies a lot for me,
> I really appreciate the time you took to write that up.
>
> It sounds like there will be delays in the case of objects that are
> created on the fly and assigned a dynamic layermode. I'm thinking of
> Swami's puzzle where new snapshots are created every time a new puzzle is
> made. These are dragged around the card by the user.
>
> I assume the buffers are destroyed when an object is deleted. When a new
> puzzle is started, the game deletes all the old snapshots, then creates new
> ones and assigns the layermode. Is there an ideal way to handle this? On
> iOS it's fine, on Android there is significant delay while the puzzle is
> built.
>
> Would changing the default compositor help? Or are we just at the mercy of
> the graphics card?
>
> I had originally used fields instead of snapshots thinking that would be
> faster, but since both are buffered as bitmaps in either case, I guess it
> doesn't matter.
>
> --
> Jacqueline Landman Gay         |     [hidden email]
> HyperActive Software           |     http://www.hyperactivesw.com
>
> _______________________________________________
> use-livecode mailing list
> [hidden email]
> Please visit this url to subscribe, unsubscribe and manage your
> subscription preferences:
> http://lists.runrev.com/mailman/listinfo/use-livecode
>



--
On the first day, God created the heavens and the Earth
On the second day, God created the oceans.
On the third day, God put the animals on hold for a few hours,
   and did a little diving.
And God said, "This is good."
_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode