reading from a file

classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|

reading from a file

John Ridge
I need to read from a text file until I hit EOF - but in the meantime I want
to process each of the chunks I'm reading (separated by delimiters).

It seemed obvious and natural to start off, after opening the file,
   
       repeat until EOF
       read from file xxx until <delimiter>
 ...process next chunk...
       end repeat

But, alas, this runs forever - it doesn't encounter EOF, although I was
careful to "close" the file when I wrote it out. I can't find anything in
the docs that explains why it doesn't work. Should it?

Apologies if the answer is just too obvious!
John
--


_______________________________________________
use-revolution mailing list
[hidden email]
http://lists.runrev.com/mailman/listinfo/use-revolution
Reply | Threaded
Open this post in threaded view
|

Re: reading from a file

Alex Tweedly
John Ridge wrote:

>I need to read from a text file until I hit EOF - but in the meantime I want
>to process each of the chunks I'm reading (separated by delimiters).
>
>It seemed obvious and natural to start off, after opening the file,
>  
>       repeat until EOF
>       read from file xxx until <delimiter>
> ...process next chunk...
>       end repeat
>
>But, alas, this runs forever - it doesn't encounter EOF, although I was
>careful to "close" the file when I wrote it out. I can't find anything in
>the docs that explains why it doesn't work. Should it?
>
>Apologies if the answer is just too obvious!
>John
>  
>
EOF is simply a constant for "an end of file character"

One way you can do what you want  (there are bound to be others :-) is ...

   open file tName for read
    repeat forever
       read from file tName until cr
       if the result = "eof" then exit repeat
       put it & cr after msg    -- to check it is being read
  end repeat
  put "done" & cr after msg    -- to check we exit OK

... note "result" can be set to other values if there is an error -
depending on circumstances, you may want to check for that happening.

--
Alex Tweedly       http://www.tweedly.net



--
No virus found in this outgoing message.
Checked by AVG Anti-Virus.
Version: 7.0.323 / Virus Database: 267.6.8 - Release Date: 11/06/2005

_______________________________________________
use-revolution mailing list
[hidden email]
http://lists.runrev.com/mailman/listinfo/use-revolution
Reply | Threaded
Open this post in threaded view
|

Re: reading from a file

John Ridge
on 12/6/05 1:21 pm,  Alex Tweedly wrote :

John Ridge wrote:

>I need to read from a text file until I hit EOF - but in the meantime I want
>to process each of the chunks I'm reading (separated by delimiters).
>
>It seemed obvious and natural to start off, after opening the file,
>  
>       repeat until EOF
>       read from file xxx until <delimiter>
> ...process next chunk...
>       end repeat
>
>But, alas, this runs forever - it doesn't encounter EOF, although I was
>careful to "close" the file when I wrote it out. I can't find anything in
>the docs that explains why it doesn't work. Should it?
>
>Apologies if the answer is just too obvious!
>John
>  
>
EOF is simply a constant for "an end of file character"

One way you can do what you want  (there are bound to be others :-) is ...

  open file tName for read
   repeat forever
      read from file tName until cr
      if the result = "eof" then exit repeat
      put it & cr after msg    -- to check it is being read
 end repeat
 put "done" & cr after msg    -- to check we exit OK

... note "result" can be set to other values if there is an error -
depending on circumstances, you may want to check for that happening.

***************************
Thanks, Alex - of course it's "the result" that I want - but I still like my
syntax! What language am I half-remembering?

I've tried another method that I thought might be faster - instead of
reading a chunk at a time and popping it into a field on a new card, why not
gulp the whole file into a variable, then get each item as a chunk?

In fact it's slower. Ah well...
--


_______________________________________________
use-revolution mailing list
[hidden email]
http://lists.runrev.com/mailman/listinfo/use-revolution
Reply | Threaded
Open this post in threaded view
|

Re: reading from a file

Jon-3
In reply to this post by Alex Tweedly
Alex recommended "read from file tName until cr".  Is this the prefered
approach, or should it be "until return"?  I'm not sure which is the
most portable approach.

:)

Jon


Alex Tweedly wrote:

> John Ridge wrote:
>
>> I need to read from a text file until I hit EOF - but in the meantime
>> I want
>> to process each of the chunks I'm reading (separated by delimiters).
>>
>> It seemed obvious and natural to start off, after opening the file,
>>         repeat until EOF
>>       read from file xxx until <delimiter>
>> ...process next chunk...
>>       end repeat
>>
>> But, alas, this runs forever - it doesn't encounter EOF, although I was
>> careful to "close" the file when I wrote it out. I can't find
>> anything in
>> the docs that explains why it doesn't work. Should it?
>>
>> Apologies if the answer is just too obvious!
>> John
>>  
>>
> EOF is simply a constant for "an end of file character"
>
> One way you can do what you want  (there are bound to be others :-) is
> ...
>
>   open file tName for read
>    repeat forever
>       read from file tName until cr
>       if the result = "eof" then exit repeat
>       put it & cr after msg    -- to check it is being read
>  end repeat
>  put "done" & cr after msg    -- to check we exit OK
>
> ... note "result" can be set to other values if there is an error -
> depending on circumstances, you may want to check for that happening.
>
_______________________________________________
use-revolution mailing list
[hidden email]
http://lists.runrev.com/mailman/listinfo/use-revolution
Reply | Threaded
Open this post in threaded view
|

Re: reading from a file

Alex Tweedly
Jon wrote:

> Alex recommended "read from file tName until cr".  Is this the
> prefered approach, or should it be "until return"?  I'm not sure which
> is the most portable approach.
>
> :)

Jon,

You need to read the Doc entry for "return" very carefully.
Then have a large cup of espresso, and then read it again.  
(substitute your local equivalent to espresso; here in Scotland it's
hard to find a good cup of coffee, so I usually choose Lagavullin Malt
Whisky instead :-)

'cr' is a constant - equivalent to the constant 'return' and equivalent
to the constant 'lf'
they are all equivalent to   numtochar(10)

i.e. CR is not the ascii carriage return character, commonly called "cr".

If you read (or write) a file having specified "text" mode in the open
statement, the native line ending (LF on Unix, carriage return on OSX,
CRLF on Windows) is translated on the fly into the Rev standard which is
CR (or any of the synonyms as above).

So - assuming I have understood it all properly, and I'm on about my
fifth reading of this part of the docs today :-), what I used in the
earlier email is the most portable way to do it - but even it is still
vulnerable to files which been copied from other systems in binary mode
and hence have preserved the "wrong" line endings.

--
Alex Tweedly       http://www.tweedly.net



--
No virus found in this outgoing message.
Checked by AVG Anti-Virus.
Version: 7.0.323 / Virus Database: 267.6.8 - Release Date: 11/06/2005

_______________________________________________
use-revolution mailing list
[hidden email]
http://lists.runrev.com/mailman/listinfo/use-revolution
Reply | Threaded
Open this post in threaded view
|

Re: reading from a file

Jon-3
Alex:

Good to know that "CR" and "RETURN" are the same thing, and not
necessarily a carriage return <sick grin>.  I guess so long as Rev
manages to substitute the locally correct value, I don't really care.  
Still, wouldn't it have been smarter to use Pascal parlance, and call it
EOL (end of line).

Sigh.   So much convoluted history.

:)

Jon


Alex Tweedly wrote:

> Jon wrote:
>
>> Alex recommended "read from file tName until cr".  Is this the
>> prefered approach, or should it be "until return"?  I'm not sure
>> which is the most portable approach.
>>
>> :)
>
>
> Jon,
>
> You need to read the Doc entry for "return" very carefully.
> Then have a large cup of espresso, and then read it again.  
> (substitute your local equivalent to espresso; here in Scotland it's
> hard to find a good cup of coffee, so I usually choose Lagavullin Malt
> Whisky instead :-)
>
> 'cr' is a constant - equivalent to the constant 'return' and
> equivalent to the constant 'lf'
> they are all equivalent to   numtochar(10)
>
> i.e. CR is not the ascii carriage return character, commonly called "cr".
>
> If you read (or write) a file having specified "text" mode in the open
> statement, the native line ending (LF on Unix, carriage return on OSX,
> CRLF on Windows) is translated on the fly into the Rev standard which
> is CR (or any of the synonyms as above).
>
> So - assuming I have understood it all properly, and I'm on about my
> fifth reading of this part of the docs today :-), what I used in the
> earlier email is the most portable way to do it - but even it is still
> vulnerable to files which been copied from other systems in binary
> mode and hence have preserved the "wrong" line endings.
>
_______________________________________________
use-revolution mailing list
[hidden email]
http://lists.runrev.com/mailman/listinfo/use-revolution
Reply | Threaded
Open this post in threaded view
|

Re: reading from a file

see3d
In reply to this post by John Ridge

On Jun 12, 2005, at 1:06 PM, John Ridge wrote:
>
> I've tried another method that I thought might be faster - instead of
> reading a chunk at a time and popping it into a field on a new  
> card, why not
> gulp the whole file into a variable, then get each item as a chunk?
>
> In fact it's slower. Ah well...
> --

John,

The slowness comes from switching from sequential access of the file  
to "pseudo random access" by using chunk statements on the variable.  
If you read in the whole file into a variable, then split the  
variable by return, you will have an array of lines which can be  
randomly accessed quickly.  You can further put one of those lines  
into another variable and split it by some item delimiter, and have  
an array of items to process that line.

However, the fastest way to sequentially access the file read into a  
variable, is with a repeat for each construct.  It virtually lays  
rubber!

repeat for each line thisLine in wholeFileVariable
   repeat for each item thisItem in thisLine
     doSomethingWith thisItem
   end repeat
end repeat

Dennis
_______________________________________________
use-revolution mailing list
[hidden email]
http://lists.runrev.com/mailman/listinfo/use-revolution
Reply | Threaded
Open this post in threaded view
|

Re: reading from a file

Alex Tweedly
In reply to this post by John Ridge
John Ridge wrote:

>on 12/6/05 1:21 pm,  Alex Tweedly wrote :
>
>  
>
>***************************
>Thanks, Alex - of course it's "the result" that I want - but I still like my
>syntax! What language am I half-remembering?
>
>  
>
I liked your syntax too - though it did require a certain amount of
clairvoyance or mind-reading to determine which file you wanted the
"eof" test to apply to :-)
It certainly feels familiar - but can't remember which it's most likely
to be. Some Pascal variant, maybe ?


>I've tried another method that I thought might be faster - instead of
>reading a chunk at a time and popping it into a field on a new card, why not
>gulp the whole file into a variable, then get each item as a chunk?
>
>In fact it's slower. Ah well...
>  
>
That is kind of surprising.
If it's convenient, could you send the two sets of code so I can see
what the differences are, please ?


--
Alex Tweedly       http://www.tweedly.net



--
No virus found in this outgoing message.
Checked by AVG Anti-Virus.
Version: 7.0.323 / Virus Database: 267.6.8 - Release Date: 11/06/2005

_______________________________________________
use-revolution mailing list
[hidden email]
http://lists.runrev.com/mailman/listinfo/use-revolution
Reply | Threaded
Open this post in threaded view
|

Re: reading from a file

John Ridge
In reply to this post by see3d
on 12/6/05 7:32 pm,  Dennis Brown wrote :

John,

The slowness comes from switching from sequential access of the file
to "pseudo random access" by using chunk statements on the variable.
If you read in the whole file into a variable, then split the
variable by return, you will have an array of lines which can be
randomly accessed quickly.  You can further put one of those lines
into another variable and split it by some item delimiter, and have
an array of items to process that line.

However, the fastest way to sequentially access the file read into a
variable, is with a repeat for each construct.  It virtually lays
rubber!

repeat for each line thisLine in wholeFileVariable
  repeat for each item thisItem in thisLine
    doSomethingWith thisItem
  end repeat
     end repeat

***********************
Thanks, Dennis. That looks very interesting. I'm trying to build a stack "on
the fly", reading in a number of text fields for each card - they may
contain return, eol etc characters, so I must write out my own delimiters
when I export the card contents. What you've suggested is that I need two
delimiters - a "line" delimiter to mark the end of a card, and an "item"
delimiter to separate the fields. Then I can try your double "for each"
approach - I'll let you know.

John

--


_______________________________________________
use-revolution mailing list
[hidden email]
http://lists.runrev.com/mailman/listinfo/use-revolution
Reply | Threaded
Open this post in threaded view
|

Re: reading from a file

John Ridge
In reply to this post by John Ridge
on 13/6/05 1:38 am,  [hidden email] wrote :



>I've tried another method that I thought might be faster - instead of
>reading a chunk at a time and popping it into a field on a new card, why not
>gulp the whole file into a variable, then get each item as a chunk?
>
>In fact it's slower. Ah well...
>  
>
That is kind of surprising.
If it's convenient, could you send the two sets of code so I can see
what the differences are, please ?


--
Alex Tweedly       http://www.tweedly.net

--
Thanks again, Alex - somehow this bit of your message got misplaced by the
digest process. The answer is simple, I think - I only used a single item
delimiter, so I had to count on my fingers to know when the next item was
the first field on the next card. I'm going to take up Dennis Brown's
suggestion and use two delimiters with the "repeat for each" construct and
let the engine do the counting!

This should work - the "line" and "item" delimiters are just two ways of
chopping up text into chunks separated by whatever character you choose. It
might be nice if there were more than two delimiters, but in this case two
is fine - one for the card, one for the fields within the card.

John

_______________________________________________
use-revolution mailing list
[hidden email]
http://lists.runrev.com/mailman/listinfo/use-revolution
Reply | Threaded
Open this post in threaded view
|

Re: reading from a file

see3d
In reply to this post by John Ridge

On Jun 13, 2005, at 2:39 AM, John Ridge wrote:

> Thanks, Dennis. That looks very interesting. I'm trying to build a  
> stack "on
> the fly", reading in a number of text fields for each card - they may
> contain return, eol etc characters, so I must write out my own  
> delimiters
> when I export the card contents. What you've suggested is that I  
> need two
> delimiters - a "line" delimiter to mark the end of a card, and an  
> "item"
> delimiter to separate the fields. Then I can try your double "for  
> each"
> approach - I'll let you know.


John,

You can also do this with a single repeat for each.

repeat for each char thisCharacter in wholeFileVariable
     doSomethingWith thisCharacter
end repeat

See the example stack called read anything in RevOnline to see how it  
is done.  If you can scan your file sequentially, you can do anything  
you want to it in short order.  It just depends on the specific  
details you want to accomplish.

Dennis
_______________________________________________
use-revolution mailing list
[hidden email]
http://lists.runrev.com/mailman/listinfo/use-revolution
Reply | Threaded
Open this post in threaded view
|

Re: reading from a file

John Ridge
on 13/6/05 1:50 pm,  Dennis Brown wrote :


On Jun 13, 2005, at 2:39 AM, John Ridge wrote:

> Thanks, Dennis. That looks very interesting. I'm trying to build a
> stack "on
> the fly", reading in a number of text fields for each card - they may
> contain return, eol etc characters, so I must write out my own
> delimiters
> when I export the card contents. What you've suggested is that I
> need two
> delimiters - a "line" delimiter to mark the end of a card, and an
> "item"
> delimiter to separate the fields. Then I can try your double "for
> each"
> approach - I'll let you know.


John,

You can also do this with a single repeat for each.

repeat for each char thisCharacter in wholeFileVariable
    doSomethingWith thisCharacter
end repeat

See the example stack called read anything in RevOnline to see how it
is done.  If you can scan your file sequentially, you can do anything
you want to it in short order.  It just depends on the specific
details you want to accomplish.

Dennis


*******************

There was an earlier thread in which people remarked on the power (and
speed) of the "for each" structure. But I need to be beaten over the head
with an example relevant to a problem of my own!

The point I was making before is really about the "stack of cards" metaphor
- it is deeply built into xCard and of course Rev - and it makes a stack
essentially  two-dimensional as a data storage device. That is, there are
cards, and on each card there are fields. So you can picture cards as rows
and fields as columns, and read in a stream of data accordingly with "for
each", using a "card" delimiter and a "field" delimiter -   arbitrary
characters separating lines (cards), then nested within that a separation
into items (fields).

Of course it's only a metaphor. But it's a powerful user model. For example,
I'm finding it difficult to re-think in terms of using custom properties to
store data, in a stack with only one card - maybe I have to start with a
library of card simulation handlers (CreateCard, DeleteCard, GoNext etc). Or
has someone already done this? That would be nice :)

John
--


_______________________________________________
use-revolution mailing list
[hidden email]
http://lists.runrev.com/mailman/listinfo/use-revolution