Does any one know what Player "intervals" actually are?

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Does any one know what Player "intervals" actually are?

Mike Doub via use-livecode
Or even more specifically, does any one know what player object
"intervals" (the values for currentTime, startTime, endTime, timeScale,
and duration properties of a player) actually are? Are they frames?
Something else? And does anyone know why they vary by Media Framework
(i.e Quicktime vs Apple Video Foundation vs DirectShow)

For example, the same MP3 file under Quicktime (macOS or Windows) has a
timeScale of 600, and under AVF a timeScale of 44100 and under
DirectShow a timeScale of 10000000.

What's more if you get the duration and the timeScale and convert to
seconds, the different frameworks give (slightly) different values
(largest difference is 3s on a 2m45s MP3)

For example, look at the lengthy list of data below: This is timeScale
and duration for different file formats by media frameworks. A file of
the same name (i.e. audio.aac) is the SAME file on any media Framework
that can read it. Formats readable by the player differ by Media
Framework. The data is tab delimited, so if the listserv preserves the
tabs, you should be able to cut and paste to a spreadsheet for easier
viewing.

LC6.7.11 with QT for Windows                LC9.6.0dp2 with DirectShow w
LAV Codecs                LC9.5.0 with AVF (OSX)
Interval    Duration    Media File        Interval    Duration Media
File        Interval    Duration    Media File
600    7216    audio.aac        10000000    119930840 audio.aac    
44100    525312    audio.aac
600    7202    audio.adts        10000000    540160000 audio.ac3    
48000    2592768    audio.ac3
600    7188    audio.aif        10000000    119836600 audio.adt    
44100    531456    audio.adts
600    7188    audio.aifc        10000000    119836600 audio.adts    
44100    528318    audio.aif
600    7188    audio.aiff        10000000    119800000 audio.aif    
44100    528318    audio.aifc
600    20280    audio.amr        10000000    119800000 audio.aifc    
44100    528318    audio.aiff
600    7188    audio.au        10000000    119800000 audio.aiff    
8000    271200    audio.amr
600    20926    audio.caf        10000000    339000000 audio.amr    
44100    528318    audio.au
90000    2652480    audio.m1a        10000000    119800000 audio.au   
     44100    1538116    audio.caf
90000    2652480    audio.m2a        10000000    348779140 audio.caf   
     48000    1339776    audio.m1a
600    7188    audio.m4a        10000000    348779140 audio.caff    
48000    1339776    audio.m2a
600    7188    audio.m4b        10000000    293760000 audio.m1a    
600    7188    audio.m4a
600    7188    audio.m4p        10000000    293760000 audio.m2a    
600    7188    audio.m4b
600    7188    audio.m4r        10000000    119800000 audio.m4a    
600    7188    audio.m4p
600    7188    audio.mov        10000000    119800000 audio.m4b    
600    7188    audio.m4r
90000    2652480    audio.mp2        10000000    119800000 audio.m4p   
     600    7188    audio.mov
600    7225    audio.mp3        10000000    119800000 audio.m4r    
48000    1339776    audio.mp1
600    7188    audio.mp4        10000000    119800000 audio.mov    
90000    2650320    audio.mp2
90000    2652480    audio.mpa        10000000    293760000 audio.mp1   
     44100    529920    audio.mp3
600    7188    audio.qt        10000000    293760000    audio.mp2    
600    7188    audio.mp4
600    7188    audio.snd        10000000    120160000 audio.mp3    
48000    1339776    audio.mpa
600    7188    audio.wav        10000000    119800000 audio.mp4    
600    7188    audio.qt
600    7188    audio.wave        10000000    293760000 audio.mpa    
44100    528318    audio.wav
600    17779    video.3g2        10000000    119800000 audio.qt    
600    17779    video.3g2
600    17779    video.3gp        10000000    119800000 audio.snd    
600    17779    video.3gp
600    17779    video.3gp2        10000000    543114970 audio.ulaw   
     600    17779    video.3gp2
600    17779    video.3gpp        10000000    543114970 audio.ulw    
600    17779    video.3gpp
600    17680    video.avi        10000000    119800000 audio.wav    
1000000    29466372    video.avi
2997    88300    video.dif        10000000    119800000 audio.wave   
     30000    882882    video.dv
2997    88300    video.dv        10000000    119810000 audio.wma.(WMA1
codec).wma        90000    2650320 video.m2v.(MPEG-2).m2v
90000    2652480    video.m1v        10000000    119810000
audio.wma.(WMA2 codec).wma        600    17677    video.m4v.(AAC codec).m4v
600    17677    video.m4v.(AAC codec).m4v        10000000 296316670   
video.3g2        600    17677    video.m4v.(h264 codec).m4v
600    17677    video.m4v.(h264 codec).m4v        10000000 296316670   
video.3gp        600    17677    video.mov
600    17677    video.mov        10000000    296316670 video.3gp2    
90000    2656130    video.mp4
90000    2656131    video.mp4        10000000    296316670 video.3gpp   
     90000    2650320    video.mpeg.(MPEG-1).mpeg
600    17677    video.mp4v        10000000    186900000 video.asf    
90000    2650320    video.mpeg.(MPEG-2).mpeg
90000    2652480    video.mpeg.(MPEG-1).mpeg        10000000
294666670    video.avi        90000    2650320 video.mpg.(MPEG-1).mpg
90000    2652480    video.mpg.(MPEG-1).mpg        10000000 294627670   
video.dif        90000    2650320 video.mpg.(MPEG-2).mpg
90000    2652480    video.mpv        10000000    294627670 video.dv   
     90000    2656130    video.mqv
90000    2656131    video.mqv        10000000    51600000 video.flv   
     600    17677    video.qt
600    17677    video.qt        10000000    294627670 video.m1v    
90000    2650320    video.vob.(MPEG-2).vob
                 10000000    294627670    video.m2v.(MPEG-2).m2v
                 10000000    294616670    video.m4v.(AAC codec).m4v
                 10000000    294616670    video.m4v.(h264 codec).m4v
                 10000000    58800000    video.mkv
                 10000000    294616670    video.mov
                 10000000    295125560    video.mp4
                 10000000    294616670    video.mp4v
                 10000000    294627670    video.mpeg.(MPEG-1).mpeg
                 10000000    294627670    video.mpeg.(MPEG-2).mpeg
                 10000000    294627670    video.mpg.(MPEG-1).mpg
                 10000000    294627670    video.mpg.(MPEG-2).mpg
                 10000000    294627670    video.mpv
                 10000000    295125560    video.mqv
                 10000000    305280000    video.ogg
                 10000000    520040140    video.ogv
                 10000000    294616670    video.qt
                 10000000    294627670    video.vob.(MPEG-2).vob
                 10000000    294627670    video.vro.(MPEG-2).vro
                 10000000    515446660    video.webm
                 10000000    294880000    video.wmv

_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|

Re: Does any one know what Player "intervals" actually are?

Mike Doub via use-livecode
Hi Paul,

I have had to deal with differing player intervals or timeScales in my application which I develop on the Mac working with the player in LC. It uses Video files only.  I first realized this was a problem when I would create a project file that annotates a video then later compress the video  and the next time I would open the project file all the time based data was off even though it was the same video.  I found that the compression program used a different timeScale than the original movie depending on the compression codec, frame rate etc.

What I had to do to deal with the varying  timeScales was do a  conversion to a common  timescale that I used in the application and when saving to the project file.  I  used an application/document timescale of 1000/second.  I had considered a number of options i.e. something divisible by 30 for 30 FPS, using another common timeScale like 600 or 90000..  In the end I used 1000 or 1000/second since milliseconds is property I used in the application for tracking user action so I considered that  a time scale that was native to the application.  


When I retrieved any of the Player’s time properties -  currentTime, startTime, endTime, timeScale, and duration etc.  I would convert them with the following function to the 1000/sec common timeScale

function convertPlayerTimeScaleTo1000 pTime, pTimeScale

        if pTimeScale = 0 or pTimeScale is empty then

             # This is to prevent Divide by zero errors or other issues if timeScale is empty
              put 600 into pTimeScale

        end if

        local t1000Scale

        put round(pTime/pTimeScale * 1000) into t1000Scale

        return t1000Scale

end convertPlayerTimeScaleTo1000



Then when I set any of those properties I use this function to convert back from the application timescale 1000/sec to the native timeScale.
function convert1000ToPlayerTimescale pTime, pTimeScale

        local tNativeScale

        put round(pTime/1000 * pTimeScale) into tNativeScale

        return tNativeScale

end convert1000ToPlayerTimescale


I did not notice big differences in values when I used different codecs from the same video but I guess that makes sense since I am only working on the Mac.   My application will be running on Windows as well so that is something I am going to have to test.

Regards,

Martin Koob





> On Feb 18, 2020, at 10:13 AM, Paul Dupuis via use-livecode <[hidden email] <mailto:[hidden email]>> wrote:
>
> Or even more specifically, does any one know what player object "intervals" (the values for currentTime, startTime, endTime, timeScale, and duration properties of a player) actually are? Are they frames? Something else? And does anyone know why they vary by Media Framework (i.e Quicktime vs Apple Video Foundation vs DirectShow)
>
> For example, the same MP3 file under Quicktime (macOS or Windows) has a timeScale of 600, and under AVF a timeScale of 44100 and under DirectShow a timeScale of 10000000.
>
> What's more if you get the duration and the timeScale and convert to seconds, the different frameworks give (slightly) different values (largest difference is 3s on a 2m45s MP3)
>
> For example, look at the lengthy list of data below: This is timeScale and duration for different file formats by media frameworks. A file of the same name (i.e. audio.aac) is the SAME file on any media Framework that can read it. Formats readable by the player differ by Media Framework. The data is tab delimited, so if the listserv preserves the tabs, you should be able to cut and paste to a spreadsheet for easier viewing.

_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|

Re: Does any one know what Player "intervals" actually are?

Mike Doub via use-livecode
Hi Martin,

Thanks for you post/reply. As we are updating our application, all times
are being stored in seconds (real number) and converted to and from
intervals as needed. Just like you in your application.

The old version of our application stored data in user document files.
In these files, some data was stored as only intervals, without a
timeScale to properly translate back to seconds for newer frameworks. I
was hoping there might be some rules I could apply to determine how
these interval (only) values could be converted to times without a
timescale value. In doing research and contacting LiveCode support, the
answer is: They can not (or at least not practically). The only way
would be to open the exact same media file in a helper app that is built
in an old version of LIveCode that uses Quicktime to get the original
Interval for each media file referenced in our user documents that needs
intervals converted. This is impractical as we can not expect the
installation of Quicktime under Windows any more and older LC helper app
will not run under macOS Catalina to name just a few of the obstacles.

The table of timescales in my original post is also distorted as many of
the media files are the exact same clip, just encoded in another format.
I am sure if I sampled a wider variety of clips I would see a much wider
range of timeScales.

Lastly, my inquiry was also looking to actually understand what
"intervals" really are, mostly out of curiosity and interest.  I am
still curious as to what exactly "intervals" are. LC support just says
that they get them from the media frameworks and pass them along.

Thanks again for your reply.

_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|

Re: Does any one know what Player "intervals" actually are?

Mike Doub via use-livecode

In your own case (Paul) luckily there is probably very little cause for
concern - because your files have other context available, allowing us
to either convert/confirm, or else entirely replace the most important
interval values. And I don't consider this difficult to tame
sufficiently for practical purposes, again in your own case.

But as a larger issue of depending on LC intervals - yep, it's huge!
Really a stunning eye-opener to see the different values for the
different files. We should give a quick tip to all LC developers: DON'T
use intervals for permanently storing media time values. :)

As for the original question: Given their arbitrary nature per OS/Media
framework, I actually think "interval" may be the best choice of word!
An arbitrary slice of time. Perfectly named, but utterly problematic if
treated as anything other than arbitrary and temporary.

Best wishes,

Curry Kenworthy

Custom Software Development
"Better Methods, Better Results"
LiveCode Training and Consulting
http://livecodeconsulting.com/

_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|

Re: Does any one know what Player "intervals" actually are?

Mike Doub via use-livecode
Hi Paul

I was curious about your original question so tried to do a search and of course wanted to find search terms to define what an interval is. As Curry says it is likely each media type has a different definition.

I did find a retired QuickTime document for the MovieToolbox that explains the time coordinate system and defines a time base which is its current time and its rate.  

https://developer.apple.com/library/archive/documentation/QuickTime/RM/MovieInternals/MTTimeSpace/B-Chapter/2MovieTimeandSpace.html#//apple_ref/doc/uid/TP40000911-MovieTimeandSpace-WorkingWithMovieTime


An interesting thing on this page about the time coordinate systems is the relation between the time scale and the maximum duration of the movie.  

----------
Time coordinate systems have a finite maximum duration that defines the maximum time value for a time coordinate system (the minimum time value is always 0). Note that as a QuickTime movie is edited, the duration changes.
As the value of the time scale increases (as the time unit for a coordinate system gets smaller in terms of absolute time), the maximum absolute time that can be represented in a time coordinate system decreases. For example, if a time value were represented as an unsigned 16-bit integer, its maximum value would be 65,535. In a time coordinate system with a time scale of 1, the maximum time value would represent 65,535 seconds. However, in a time coordinate system with a time scale of 5, the maximum time value would correspond to 13,107 seconds. Hence, a time coordinate system’s duration is limited by its time scale. QuickTime uses 32-bit and 64-bit quantities to represent time values, so you only need to worry about attaining a maximum absolute time in situations where a time coordinate system’s duration is very long or its time scale is very large.
----------

QuickTime is long gone and AVfoundation has taken its place and I found a description of how it handles intervals which is seems to now call just timescale.

https://developer.apple.com/library/archive/documentation/AudioVideo/Conceptual/AVFoundationPG/Articles/06_MediaRepresentations.html#//apple_ref/doc/uid/TP40010188-CH2-SW3


It has an explanation of a rationale for using 600 timescale.  I considered that too for my application timescale but went with 1000 instead.   Perhaps using this would avoid rounding errors.
---------
CMTime <https://developer.apple.com/documentation/coremedia/cmtime> is a C structure that represents time as a rational number, with a numerator (an int64_t value), and a denominator (an int32_t timescale). Conceptually, the timescale specifies the fraction of a second each unit in the numerator occupies. Thus if the timescale is 4, each unit represents a quarter of a second; if the timescale is 10, each unit represents a tenth of a second, and so on. You frequently use a timescale of 600, because this is a multiple of several commonly used frame rates: 24 fps for film, 30 fps for NTSC (used for TV in North America and Japan), and 25 fps for PAL (used for TV in Europe). Using a timescale of 600, you can exactly represent any number of frames in these systems.
In addition to a simple time value, a CMTime structure can represent nonnumeric values: +infinity, -infinity, and indefinite. It can also indicate whether the time been rounded at some point, and it maintains an epoch number.
--------

I tried looking for references for WMV I wasn’t able to find anything.   I am sure as Curry says each media framework uses different terminology so I was probably not finding the right terms.


Regards,

Martin Koob





> On Feb 18, 2020, at 5:58 PM, Curry Kenworthy via use-livecode <[hidden email]> wrote:
>
>
> In your own case (Paul) luckily there is probably very little cause for concern - because your files have other context available, allowing us to either convert/confirm, or else entirely replace the most important interval values. And I don't consider this difficult to tame sufficiently for practical purposes, again in your own case.
>
> But as a larger issue of depending on LC intervals - yep, it's huge! Really a stunning eye-opener to see the different values for the different files. We should give a quick tip to all LC developers: DON'T use intervals for permanently storing media time values. :)
>
> As for the original question: Given their arbitrary nature per OS/Media framework, I actually think "interval" may be the best choice of word! An arbitrary slice of time. Perfectly named, but utterly problematic if treated as anything other than arbitrary and temporary.
>
> Best wishes,
>
> Curry Kenworthy
>
> Custom Software Development
> "Better Methods, Better Results"
> LiveCode Training and Consulting
> http://livecodeconsulting.com/
>
> _______________________________________________
> use-livecode mailing list
> [hidden email]
> Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
> http://lists.runrev.com/mailman/listinfo/use-livecode

_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|

Re: Does any one know what Player "intervals" actually are?

Mike Doub via use-livecode
In reply to this post by Mike Doub via use-livecode
On 2020-02-18 21:40, Paul Dupuis via use-livecode wrote:
> Lastly, my inquiry was also looking to actually understand what
> "intervals" really are, mostly out of curiosity and interest.  I am
> still curious as to what exactly "intervals" are. LC support just says
> that they get them from the media frameworks and pass them along.

Heh - I guess we could have done a little better than the answer we gave
(mostly my fault, Elanor asked internally, and I hadn't thought about
such things in a long while so couldn't really remember the exact
details / didn't read your question that deeply)...

Martin's post has the exact details - to summarize that - the
'timeScale' is the scaling provided to map units used to reference
points in a media to seconds. Each second is divided up into timeScale
units - so a timeScale of 600 means that there are 600 units in every
second.

The reason this is done, rather than use seconds (or milliseconds,
microseconds...) is to prevent error accumulating and also so that most
arithmetic done internally by the frameworks can be done using integers
- not floating point (which is faster, and much easier to predict and
account for numerical error with).

QT allows multiple tracks to be present in a single MOV - each which
have their own framerate, for example. So by using 600 in many cases, it
means that you could overlay video tracks recorded at 24, 25 or 30fps
precisely.

Audio formats tend not to have an inherent notion of timeScale - raw
formats, for example, have N samples per second; whereas MP3 has
'packets' of compressed audio which expand to cover a given time period.
So frameworks will tend to make their own up by the look of it to suit
whatever seemed reasonable at the time - or use some other metric from
the file (I notice that some audio files listed have a timeScale of
44100 - which probably means that audio file is 44.1kHz - the standard
CD quality audio sample rate).

Given the problem you have at hand, the best thing to do is to perhaps
create a small utility which you ask users of your tool to run which
fetches the timeScale for all their files using QT - then you should be
able to analyse that data (en masse) to see what possible timeScales
exist for different formats. That should mean you can, at least, create
a heuristic for guessing the timeScale of any given (existing) file.

Warmest Regards,

Mark.

--
Mark Waddingham ~ [hidden email] ~ http://www.livecode.com/
LiveCode: Everyone can create apps

_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode