There should be a "unique" option on sort . . .

classic Classic list List threaded Threaded
35 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Re: There should be a "unique" option on sort . . .

Richard Gaskin
Peter Haworth wrote:

> Hi Richard,
> It's not so much the arithmetic operation or putting a value into the array
> element, it's the repeat loop itself which I don't think is needed.  It can
> be replaced with "combine pData with return and return".  That one line
> gets rid of the duplicates.  So the function would look like this:
>
> function UniqueListFromArray2 pData
>    local tKeys
>    combine pData with return and return
>    put the keys of pdata into tKeys
>    sort tKeys
>    return tKeys
> end UniqueListFromArray2
>
> I came up with that because the OP mentioned that an engine-embedded way to
> de-dup would be faster than any user written handler, which seems like a
> reasonable assumption.
>
> Pretty academic really because it seems like all the solutions work
> acceptably fast but nevertheless interesting to compare.

Excellent!

I added a new function to the mix which uses the split command, but at
first I found that it was failing the sanity check - the results were
different.

After looking over the results I discovered an interesting anomaly:  the
split command is inherently case sensitive, and turning the caseSenstive
to false within a handler that uses it has no effect.

Is that a bug?

So the only way to use split is when you want to a case-sensitive
results, and if so you'll need to set the caseSensitive to true for both
functions since it governs the sort results:

function UniqueListFromArray3 pData
    set the caseSensitive to true
    split pData using cr and cr
    put the keys of pData into tKeys
    sort lines of tKeys
    return tKeys
end UniqueListFromArray3

So when case sensitivity will do what you need, the result is pretty
impressive - about half the time required for the other fastest option:

10 iterations on 10000 lines of 5 or fewer chars:
"Add 1" Array: 58 ms (5.8 ms per iteration)
"Split" Array: 24 ms (2.4 ms per iteration)
Results match - Each list has 997 lines

--
  Richard Gaskin
  Fourth World
  LiveCode training and consulting: http://www.fourthworld.com
  Webzine for LiveCode developers: http://www.LiveCodeJournal.com
  Follow me on Twitter:  http://twitter.com/FourthWorldSys

_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|

Re: There should be a "unique" option on sort . . .

Peter Haworth
On Mon, Jan 6, 2014 at 11:41 AM, Richard Gaskin
<[hidden email]>wrote:

> So when case sensitivity will do what you need, the result is pretty
> impressive - about half the time required for the other fastest option:
>
> 10 iterations on 10000 lines of 5 or fewer chars:
> "Add 1" Array: 58 ms (5.8 ms per iteration)
> "Split" Array: 24 ms (2.4 ms per iteration)
> Results match - Each list has 997 lines
>

Thanks Richard and sorry for mixing up split and combine, I always find
those terms not particularly reflective of what they do.

Your tests do seem to bear out that a built in command is faster than a
scripted solution, at least for this case.

Not sure what to say about the case sensitivity issue.  I hadn't thought
about it but now you mention it, I don't think I'd expect split to be case
sensitive.  Would be nice if it observed the caseSensitive setting I guess
but I think I'd regard that as an enhancement rather than a bug.

I think some of the other solutions would need to set caseSensitive to true
if necessary.

Pete
lcSQL Software <http://www.lcsql.com>
_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|

Re: There should be a "unique" option on sort . . .

Richard Gaskin
Peter Haworth wrote:
> Not sure what to say about the case sensitivity issue.  I hadn't thought
> about it but now you mention it, I don't think I'd expect split to be case
> sensitive.  Would be nice if it observed the caseSensitive setting I guess
> but I think I'd regard that as an enhancement rather than a bug.
>
> I think some of the other solutions would need to set caseSensitive to true
> if necessary.

Good idea on characterizing it as a feature request - I just submitted that:

<http://quality.runrev.com/show_bug.cgi?id=11651>

--
  Richard Gaskin
  Fourth World
  LiveCode training and consulting: http://www.fourthworld.com
  Webzine for LiveCode developers: http://www.LiveCodeJournal.com
  Follow me on Twitter:  http://twitter.com/FourthWorldSys

_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|

Re: There should be a "unique" option on sort . . .

pmbrig
In reply to this post by Peter Haworth
On Jan 6, 2014, at 3:36 PM, Peter Haworth wrote:

> Thanks Richard and sorry for mixing up split and combine, I always find
> those terms not particularly reflective of what they do.

I think of it this way. An array is like those collecting boxes with horizontal and vertical dividers that separate the interior of the box into compartments. "Combine" is like lifting out the dividers so the contents are no longer each in their own separate compartment, they're combined into one space. "Split" is like inserting the dividers so the contents are split into their own compartments.

I dunno, it works for me.

-- Peter

Peter M. Brigham
[hidden email]
http://home.comcast.net/~pmbrig


_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|

Re: There should be a "unique" option on sort . . .

J. Landman Gay
On 1/6/14 9:52 PM, Peter M. Brigham wrote:

> I think of it this way. An array is like those collecting boxes with
> horizontal and vertical dividers that separate the interior of the
> box into compartments. "Combine" is like lifting out the dividers so
> the contents are no longer each in their own separate compartment,
> they're combined into one space. "Split" is like inserting the
> dividers so the contents are split into their own compartments.
>
> I dunno, it works for me.

That's my visual image too. I think it's because at one of the teaching
sessions Devin held up an egg carton and said, "this is an array." It
stuck with me, only now arrays make me hungry.

--
Jacqueline Landman Gay         |     [hidden email]
HyperActive Software           |     http://www.hyperactivesw.com

_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|

Re: There should be a "unique" option on sort . . .

Geoff Canyon-4
In reply to this post by Richard Gaskin
I *think* I tested up to 100,000 lines -- spent the day traveling back from
Boston to St. Louis, a little groggy -- *but* my keys were always integers.

On Mon, Jan 6, 2014 at 10:13 AM, Richard Gaskin
<[hidden email]>wrote:

> function UniqueListFromChunks pData
>    sort lines of pData
>    # put line 1 of pData is false into tLastLine -- what does this do?
>    put empty into tLastLine
>    repeat for each line tLine in pData
>       if tLine is tLastLine then next repeat
>

NOTE: I corrected the variable names in the above.

Ha ;-) on any day other than yesterday would have done what you did, or
skipped the initialization altogether. BUT -- if the first line of data in
pData is empty (your version) or tLastLine (mine), the above will
(incorrectly) omit it. So the line I gave, "put line 1 of pData is false
into lastLine" guarantees that the first line will be included, without
having to use a conditional that only matters the first iteration inside
the repeat.

Huh, that's weird. I copy/pasted your code into a button and ran it:

10 iterations on 100 lines of 5 or fewer chars:

Array: 2 ms (0.2 ms per iteration)

Chunks: 1 ms (0.1 ms per iteration)

Results match - Each list has 95 lines


I ran that several times, and the winner flip-flopped several times. So I
switched to the long seconds. With that, the "Chunks" version is almost
always the winner (if only by a few ten-thousandths of a second) Typical
result:


10 iterations on 100 lines of 5 or fewer chars:

Array: 0.001466 seconds (0.000147 seconds per iteration)

Chunks: 0.001218 seconds (0.000122 seconds per iteration)

Results match - Each list has 97 lines


Curiouser and curiouser:


10 iterations on 100 lines of 250 or fewer chars:

Array: 0.002393 seconds (0.000239 seconds per iteration)

Chunks: 0.001738 seconds (0.000174 seconds per iteration)

Results match - Each list has 97 lines



With 1000 lines it favors the array more often, but I still saw outcomes
where chunks won (not this result, obviously -- trying to be
representative):


10 iterations on 1000 lines of 5 or fewer chars:

Array: 0.007609 seconds (0.000761 seconds per iteration)

Chunks: 0.007894 seconds (0.000789 seconds per iteration)

Results match - Each list has 617 lines


And then back to chunks (mostly):


10 iterations on 1000 lines of 250 or fewer chars:

Array: 0.015478 seconds (0.001548 seconds per iteration)

Chunks: 0.015227 seconds (0.001523 seconds per iteration)

Results match - Each list has 740 lines


We start converging at 10,000 and beyond:


10 iterations on 10000 lines of 5 or fewer chars:

Array: 0.029378 seconds (0.002938 seconds per iteration)

Chunks: 0.06806 seconds (0.006806 seconds per iteration)

Results match - Each list has 988 lines


10 iterations on 10000 lines of 250 or fewer chars:

Array: 0.071169 seconds (0.007117 seconds per iteration)

Chunks: 0.148104 seconds (0.01481 seconds per iteration)

Results match - Each list has 1492 lines


10 iterations on 100000 lines of 5 or fewer chars:

Array: 0.229239 seconds (0.022924 seconds per iteration)

Chunks: 0.732289 seconds (0.073229 seconds per iteration)

Results match - Each list has 985 lines


10 iterations on 100000 lines of 250 or fewer chars:

Array: 0.604814 seconds (0.060481 seconds per iteration)

Chunks: 2.04249 seconds (0.204249 seconds per iteration)

Results match - Each list has 1494 lines
_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|

Re: There should be a "unique" option on sort . . .

mwieder
In reply to this post by pmbrig
Peter-

Monday, January 6, 2014, 7:52:46 PM, you wrote:

> I think of it this way. An array is like those collecting boxes
> with horizontal and vertical dividers that separate the interior of
> the box into compartments. "Combine" is like lifting out the
> dividers so the contents are no longer each in their own separate
> compartment, they're combined into one space. "Split" is like
> inserting the dividers so the contents are split into their own
> compartments.

> I dunno, it works for me.

Oooo... I like that analogy. I'm constantly having to rethink those
terms when I want to use them, and having this in mind will help
immensely.

--
-Mark Wieder
 [hidden email]


_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
--
 Mark Wieder
 ahsoftware@gmail.com
Reply | Threaded
Open this post in threaded view
|

Re: There should be a "unique" option on sort . . .

Peter Haworth
In reply to this post by Geoff Canyon-4
On Mon, Jan 6, 2014 at 9:51 PM, Geoff Canyon <[hidden email]> wrote:

> I *think* I tested up to 100,000 lines -- spent the day traveling back from
> Boston to St. Louis, a little groggy -- *but* my keys were always integers.
>

Hey Geoff, which version of the array logic did you use, Richard's original
or mine with the split command?  There's a considerable difference between
the two.

Just for fun, tomorrow I'm going to try this by creating an in-memory
sqlite database and using a SELECT statement with a GROUP BY clause just to
see how it compares.  I'd be surprised if it was faster but you never know.

Pete
lcSQL Software <http://www.lcsql.com>
_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|

Re: There should be a "unique" option on sort . . .

Peter Haworth
In reply to this post by pmbrig
That might stick, I'll see next time I use combine/splt in a few weeks!

I keep thinking some form of the convert command might be clearer (or some
other clever verb rather than convert, like transform).  Maybe "transform
tVar to array" or "transform tArray to text".  Not sure what happens with
split/combine if the variable is not in the correct format but whatever it
is, the same could happen with the convert idea.  Shouldn't think that
would ever happen though :-)

Pete
lcSQL Software <http://www.lcsql.com>


On Mon, Jan 6, 2014 at 7:52 PM, Peter M. Brigham <[hidden email]> wrote:

> On Jan 6, 2014, at 3:36 PM, Peter Haworth wrote:
>
> > Thanks Richard and sorry for mixing up split and combine, I always find
> > those terms not particularly reflective of what they do.
>
> I think of it this way. An array is like those collecting boxes with
> horizontal and vertical dividers that separate the interior of the box into
> compartments. "Combine" is like lifting out the dividers so the contents
> are no longer each in their own separate compartment, they're combined into
> one space. "Split" is like inserting the dividers so the contents are split
> into their own compartments.
>
> I dunno, it works for me.
>
> -- Peter
>
> Peter M. Brigham
> [hidden email]
> http://home.comcast.net/~pmbrig
>
>
> _______________________________________________
> use-livecode mailing list
> [hidden email]
> Please visit this url to subscribe, unsubscribe and manage your
> subscription preferences:
> http://lists.runrev.com/mailman/listinfo/use-livecode
>
_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|

Re: There should be a "unique" option on sort . . .

Richard Gaskin
In reply to this post by Geoff Canyon-4
Geoff, I think you may have stumbled across the greatest challenge of
benchmarking:  the subtle differences over which we have little or no
control.

Many factors can affect performance, including differences in chip
architectures with regard to instruction set features, caching, etc.,
system memory caching, background processes, and much more.  There have
even been times when I've seen changing the order in which testing
functions are called can affect outcomes.  And file I/O tests often
benefit from a cold boot due to system caching - sometimes worked around
running purge, but still tedious at best.

When we see stark differences like the ones between your and my results
running the same code in the same version of LiveCode (I'm using 6.5.1),
I think we've moved outside of LiveCode and are now at the mercy of
hardware and software nuances beyond our control.

If nothing else it reminds us of the value of testing comparative
benchmarks across multiple machines.

But one fairly constant thing in all this is that when we can hand off
processing to the compiled C++ code in the engine, in many cases (Regex
and others notwithstanding) we can expect a boost in performance.

I would be interested to see the results if you swap out the original
array function with this one, from
<http://lists.runrev.com/pipermail/use-livecode/2014-January/197038.html>:

function UniqueListFromArray3 pData
     set the caseSensitive to true
     split pData using cr and cr
     put the keys of pData into tKeys
     sort lines of tKeys
     return tKeys
end UniqueListFromArray3

Under the hood, the split command presumably does a series of steps very
similar to what we'd have to do in script to build an array, but without
the overhead of type coersion and other factors that characterize
dynamic compilation, and using an algo written very specifically for
that task.  That is, it still needs to hash every key and move the
contents into the appropriate bucket, but with far less overhead than
running those steps through the interpreter.

The only downside to using split is that it differs from most other
array-building methods in that it's always case-sensitive (see the
enhancement request to allow split to use the caseSensitive property
here: <http://quality.runrev.com/show_bug.cgi?id=11651>).

So to pass the sanity check, you'll need to add this line to the other
function as well:

    set the caseSensitive to true

I'll wager the beverage of your choice at RevLive in San Diego this
September that the split method will be faster even on your system.

--
  Richard Gaskin
  Fourth World
  LiveCode training and consulting: http://www.fourthworld.com
  Webzine for LiveCode developers: http://www.LiveCodeJournal.com
  Follow me on Twitter:  http://twitter.com/FourthWorldSys



Geoff Canyon wrote:

> I *think* I tested up to 100,000 lines -- spent the day traveling back from
> Boston to St. Louis, a little groggy -- *but* my keys were always integers.
>
> On Mon, Jan 6, 2014 at 10:13 AM, Richard Gaskin
> <ambassador at fourthworld.com>wrote:
>
>> function UniqueListFromChunks pData
>>    sort lines of pData
>>    # put line 1 of pData is false into tLastLine -- what does this do?
>>    put empty into tLastLine
>>    repeat for each line tLine in pData
>>       if tLine is tLastLine then next repeat
>>
>
> NOTE: I corrected the variable names in the above.
>
> Ha ;-) on any day other than yesterday would have done what you did, or
> skipped the initialization altogether. BUT -- if the first line of data in
> pData is empty (your version) or tLastLine (mine), the above will
> (incorrectly) omit it. So the line I gave, "put line 1 of pData is false
> into lastLine" guarantees that the first line will be included, without
> having to use a conditional that only matters the first iteration inside
> the repeat.
>
> Huh, that's weird. I copy/pasted your code into a button and ran it:
>
> 10 iterations on 100 lines of 5 or fewer chars:
>
> Array: 2 ms (0.2 ms per iteration)
>
> Chunks: 1 ms (0.1 ms per iteration)
>
> Results match - Each list has 95 lines
>
>
> I ran that several times, and the winner flip-flopped several times. So I
> switched to the long seconds. With that, the "Chunks" version is almost
> always the winner (if only by a few ten-thousandths of a second) Typical
> result:
>
>
> 10 iterations on 100 lines of 5 or fewer chars:
>
> Array: 0.001466 seconds (0.000147 seconds per iteration)
>
> Chunks: 0.001218 seconds (0.000122 seconds per iteration)
>
> Results match - Each list has 97 lines
>
>
> Curiouser and curiouser:
>
>
> 10 iterations on 100 lines of 250 or fewer chars:
>
> Array: 0.002393 seconds (0.000239 seconds per iteration)
>
> Chunks: 0.001738 seconds (0.000174 seconds per iteration)
>
> Results match - Each list has 97 lines
>
>
>
> With 1000 lines it favors the array more often, but I still saw outcomes
> where chunks won (not this result, obviously -- trying to be
> representative):
>
>
> 10 iterations on 1000 lines of 5 or fewer chars:
>
> Array: 0.007609 seconds (0.000761 seconds per iteration)
>
> Chunks: 0.007894 seconds (0.000789 seconds per iteration)
>
> Results match - Each list has 617 lines
>
>
> And then back to chunks (mostly):
>
>
> 10 iterations on 1000 lines of 250 or fewer chars:
>
> Array: 0.015478 seconds (0.001548 seconds per iteration)
>
> Chunks: 0.015227 seconds (0.001523 seconds per iteration)
>
> Results match - Each list has 740 lines
>
>
> We start converging at 10,000 and beyond:
>
>
> 10 iterations on 10000 lines of 5 or fewer chars:
>
> Array: 0.029378 seconds (0.002938 seconds per iteration)
>
> Chunks: 0.06806 seconds (0.006806 seconds per iteration)
>
> Results match - Each list has 988 lines
>
>
> 10 iterations on 10000 lines of 250 or fewer chars:
>
> Array: 0.071169 seconds (0.007117 seconds per iteration)
>
> Chunks: 0.148104 seconds (0.01481 seconds per iteration)
>
> Results match - Each list has 1492 lines
>
>
> 10 iterations on 100000 lines of 5 or fewer chars:
>
> Array: 0.229239 seconds (0.022924 seconds per iteration)
>
> Chunks: 0.732289 seconds (0.073229 seconds per iteration)
>
> Results match - Each list has 985 lines
>
>
> 10 iterations on 100000 lines of 250 or fewer chars:
>
> Array: 0.604814 seconds (0.060481 seconds per iteration)
>
> Chunks: 2.04249 seconds (0.204249 seconds per iteration)
>
> Results match - Each list has 1494 lines

_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|

Re: There should be a "unique" option on sort . . .

Geoff Canyon-4
On Tue, Jan 7, 2014 at 8:29 AM, Richard Gaskin
<[hidden email]>wrote:

> I'll wager the beverage of your choice at RevLive in San Diego this
> September that the split method will be faster even on your system.
>

I'd hesitate to take that bet, since I would expect split to be faster as
well. I wish I had kept my original code. :-/
_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|

Re: There should be a "unique" option on sort . . .

Jan Schenkel
In reply to this post by mwieder
Hi Mark et al,

The 'sort' command should return the same content, not throw out chunks, that might be better as an extension to the 'filter' command.
But your comment about how sort affects the original container made me wonder.

So I took another look at the source for the 'sort' command in cmds.cpp, and it doesn't look that hard to add an 'into' clause.
Putting the sorted data into another container is straightforward to add to MCSort::exec (done that for the 'filter' command).
The hardest part is untangling the MCSort::parse spaghetti.

If I find the time I might take a stab at it…

Cheers,

Jan Schenkel.

=====
Quartam Reports & PDF Library for LiveCode
www.quartam.com

=====
"As we grow older, we grow both wiser and more foolish at the same time."  (La Rochefoucauld)

--------------------------------------------
On Sat, 1/4/14, Mark Wieder <[hidden email]> wrote:

 Subject: Re: There should be a "unique" option on sort . . .
 To: "How to use LiveCode" <[hidden email]>
 Date: Saturday, January 4, 2014, 5:02 PM
 
 > yes, or to only take the first
 that matches the sort key if sorting by
 > other than the full record.
 
 I can see it being slightly useful in certain cases, but it
 leaves me
 feeling a bit queasy. I think it's unsettling enough that
 the sort
 command sorts in place instead of being a function that
 returns a
 sorted copy, and of course it's way too late to change that
 now. So
 deleting items from a dataset while sorting them seems one
 more step
 down that ladder. I do realize that you'd have to specify
 "unique"
 explicitly, but still... if it didn't mess with the original
 data set
 I'd be all over this.
 
 --
 -Mark Wieder
  [hidden email]
 

_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|

Re: There should be a "unique" option on sort . . .

Peter Haworth
I think that would be useful Jan, but it's pretty easy to put the unsorted
container into a different container then sort it so I'm not sure it's
worth your time and effort.
Pete

Pete
lcSQL Software <http://www.lcsql.com>


On Wed, Jan 8, 2014 at 1:52 PM, Jan Schenkel <[hidden email]> wrote:

> Hi Mark et al,
>
> The 'sort' command should return the same content, not throw out chunks,
> that might be better as an extension to the 'filter' command.
> But your comment about how sort affects the original container made me
> wonder.
>
> So I took another look at the source for the 'sort' command in cmds.cpp,
> and it doesn't look that hard to add an 'into' clause.
> Putting the sorted data into another container is straightforward to add
> to MCSort::exec (done that for the 'filter' command).
> The hardest part is untangling the MCSort::parse spaghetti.
>
> If I find the time I might take a stab at it…
>
> Cheers,
>
> Jan Schenkel.
>
> =====
> Quartam Reports & PDF Library for LiveCode
> www.quartam.com
>
> =====
> "As we grow older, we grow both wiser and more foolish at the same time."
>  (La Rochefoucauld)
>
> --------------------------------------------
> On Sat, 1/4/14, Mark Wieder <[hidden email]> wrote:
>
>  Subject: Re: There should be a "unique" option on sort . . .
>  To: "How to use LiveCode" <[hidden email]>
>  Date: Saturday, January 4, 2014, 5:02 PM
>
>  > yes, or to only take the first
>  that matches the sort key if sorting by
>  > other than the full record.
>
>  I can see it being slightly useful in certain cases, but it
>  leaves me
>  feeling a bit queasy. I think it's unsettling enough that
>  the sort
>  command sorts in place instead of being a function that
>  returns a
>  sorted copy, and of course it's way too late to change that
>  now. So
>  deleting items from a dataset while sorting them seems one
>  more step
>  down that ladder. I do realize that you'd have to specify
>  "unique"
>  explicitly, but still... if it didn't mess with the original
>  data set
>  I'd be all over this.
>
>  --
>  -Mark Wieder
>   [hidden email]
>
>
> _______________________________________________
> use-livecode mailing list
> [hidden email]
> Please visit this url to subscribe, unsubscribe and manage your
> subscription preferences:
> http://lists.runrev.com/mailman/listinfo/use-livecode
_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|

Re: There should be a "unique" option on sort . . .

Jan Schenkel
Addiing only an 'into' clauset would indeed be a bit silly - my plan was to transplant another 'filter' feature: the ability to sort an expression.
This would allow you to write:

sort theFirstList & return & theSecondList into theSortedList

Of course, you could still write that in two lines, with little performance penalty.
But a future version of the engine could optimise the one-liner by parallelising the operations…

Jan Schenkel.

=====
Quartam Reports & PDF Library for LiveCode
www.quartam.com

=====
"As we grow older, we grow both wiser and more foolish at the same time."  (La Rochefoucauld)

--------------------------------------------
On Wed, 1/8/14, Peter Haworth <[hidden email]> wrote:

 Subject: Re: There should be a "unique" option on sort . . .
 To: "How to use LiveCode" <[hidden email]>
 Date: Wednesday, January 8, 2014, 3:24 PM
 
 I think that would be useful Jan, but
 it's pretty easy to put the unsorted
 container into a different container then sort it so I'm not
 sure it's
 worth your time and effort.
 Pete
 
 Pete
 lcSQL Software <http://www.lcsql.com>
 
 
 On Wed, Jan 8, 2014 at 1:52 PM, Jan Schenkel <[hidden email]>
 wrote:
 
 > Hi Mark et al,
 >
 > The 'sort' command should return the same content, not
 throw out chunks,
 > that might be better as an extension to the 'filter'
 command.
 > But your comment about how sort affects the original
 container made me
 > wonder.
 >
 > So I took another look at the source for the 'sort'
 command in cmds.cpp,
 > and it doesn't look that hard to add an 'into' clause.
 > Putting the sorted data into another container is
 straightforward to add
 > to MCSort::exec (done that for the 'filter' command).
 > The hardest part is untangling the MCSort::parse
 spaghetti.
 >
 > If I find the time I might take a stab at it…
 >
 > Cheers,
 >
 > Jan Schenkel.
 >
 > =====
 > Quartam Reports & PDF Library for LiveCode
 > www.quartam.com
 >
 > =====
 > "As we grow older, we grow both wiser and more foolish
 at the same time."
 >  (La Rochefoucauld)
 >
 > --------------------------------------------
 > On Sat, 1/4/14, Mark Wieder <[hidden email]>
 wrote:
 >
 >  Subject: Re: There should be a "unique" option on
 sort . . .
 >  To: "How to use LiveCode" <[hidden email]>
 >  Date: Saturday, January 4, 2014, 5:02 PM
 >
 >  > yes, or to only take the first
 >  that matches the sort key if sorting by
 >  > other than the full record.
 >
 >  I can see it being slightly useful in certain
 cases, but it
 >  leaves me
 >  feeling a bit queasy. I think it's unsettling
 enough that
 >  the sort
 >  command sorts in place instead of being a
 function that
 >  returns a
 >  sorted copy, and of course it's way too late to
 change that
 >  now. So
 >  deleting items from a dataset while sorting them
 seems one
 >  more step
 >  down that ladder. I do realize that you'd have to
 specify
 >  "unique"
 >  explicitly, but still... if it didn't mess with
 the original
 >  data set
 >  I'd be all over this.
 >
 >  --
 >  -Mark Wieder
 >   [hidden email]
 >
 >
 > _______________________________________________
 > use-livecode mailing list
 > [hidden email]
 > Please visit this url to subscribe, unsubscribe and
 manage your
 > subscription preferences:
 > http://lists.runrev.com/mailman/listinfo/use-livecode
 _______________________________________________
 use-livecode mailing list
 [hidden email]
 Please visit this url to subscribe, unsubscribe and manage
 your subscription preferences:
 http://lists.runrev.com/mailman/listinfo/use-livecode
 

_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|

Re: There should be a "unique" option on sort . . .

Peter Haworth
OK,  that makes sense and would be useful.

Pete
lcSQL Software
On Jan 8, 2014 10:11 PM, "Jan Schenkel" <[hidden email]> wrote:

> Addiing only an 'into' clauset would indeed be a bit silly - my plan was
> to transplant another 'filter' feature: the ability to sort an expression.
> This would allow you to write:
>
> sort theFirstList & return & theSecondList into theSortedList
>
> Of course, you could still write that in two lines, with little
> performance penalty.
> But a future version of the engine could optimise the one-liner by
> parallelising the operations…
>
> Jan Schenkel.
>
> =====
> Quartam Reports & PDF Library for LiveCode
> www.quartam.com
>
> =====
> "As we grow older, we grow both wiser and more foolish at the same time."
>  (La Rochefoucauld)
>
> --------------------------------------------
> On Wed, 1/8/14, Peter Haworth <[hidden email]> wrote:
>
>  Subject: Re: There should be a "unique" option on sort . . .
>  To: "How to use LiveCode" <[hidden email]>
>  Date: Wednesday, January 8, 2014, 3:24 PM
>
>  I think that would be useful Jan, but
>  it's pretty easy to put the unsorted
>  container into a different container then sort it so I'm not
>  sure it's
>  worth your time and effort.
>  Pete
>
>  Pete
>  lcSQL Software <http://www.lcsql.com>
>
>
>  On Wed, Jan 8, 2014 at 1:52 PM, Jan Schenkel <[hidden email]>
>  wrote:
>
>  > Hi Mark et al,
>  >
>  > The 'sort' command should return the same content, not
>  throw out chunks,
>  > that might be better as an extension to the 'filter'
>  command.
>  > But your comment about how sort affects the original
>  container made me
>  > wonder.
>  >
>  > So I took another look at the source for the 'sort'
>  command in cmds.cpp,
>  > and it doesn't look that hard to add an 'into' clause.
>  > Putting the sorted data into another container is
>  straightforward to add
>  > to MCSort::exec (done that for the 'filter' command).
>  > The hardest part is untangling the MCSort::parse
>  spaghetti.
>  >
>  > If I find the time I might take a stab at it…
>  >
>  > Cheers,
>  >
>  > Jan Schenkel.
>  >
>  > =====
>  > Quartam Reports & PDF Library for LiveCode
>  > www.quartam.com
>  >
>  > =====
>  > "As we grow older, we grow both wiser and more foolish
>  at the same time."
>  >  (La Rochefoucauld)
>  >
>  > --------------------------------------------
>  > On Sat, 1/4/14, Mark Wieder <[hidden email]>
>  wrote:
>  >
>  >  Subject: Re: There should be a "unique" option on
>  sort . . .
>  >  To: "How to use LiveCode" <[hidden email]>
>  >  Date: Saturday, January 4, 2014, 5:02 PM
>  >
>  >  > yes, or to only take the first
>  >  that matches the sort key if sorting by
>  >  > other than the full record.
>  >
>  >  I can see it being slightly useful in certain
>  cases, but it
>  >  leaves me
>  >  feeling a bit queasy. I think it's unsettling
>  enough that
>  >  the sort
>  >  command sorts in place instead of being a
>  function that
>  >  returns a
>  >  sorted copy, and of course it's way too late to
>  change that
>  >  now. So
>  >  deleting items from a dataset while sorting them
>  seems one
>  >  more step
>  >  down that ladder. I do realize that you'd have to
>  specify
>  >  "unique"
>  >  explicitly, but still... if it didn't mess with
>  the original
>  >  data set
>  >  I'd be all over this.
>  >
>  >  --
>  >  -Mark Wieder
>  >   [hidden email]
>  >
>  >
>  > _______________________________________________
>  > use-livecode mailing list
>  > [hidden email]
>  > Please visit this url to subscribe, unsubscribe and
>  manage your
>  > subscription preferences:
>  > http://lists.runrev.com/mailman/listinfo/use-livecode
>  _______________________________________________
>  use-livecode mailing list
>  [hidden email]
>  Please visit this url to subscribe, unsubscribe and manage
>  your subscription preferences:
>  http://lists.runrev.com/mailman/listinfo/use-livecode
>
>
> _______________________________________________
> use-livecode mailing list
> [hidden email]
> Please visit this url to subscribe, unsubscribe and manage your
> subscription preferences:
> http://lists.runrev.com/mailman/listinfo/use-livecode
_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
12