Home     |     .Net Programming    |     cSharp Home    |     Sql Server Home    |     Javascript / Client Side Development     |     Ajax Programming

Ruby on Rails Development     |     Perl Programming     |     C Programming Language     |     C++ Programming     |     IT Jobs

Python Programming Language     |     Laptop Suggestions?    |     TCL Scripting     |     Fortran Programming     |     Scheme Programming Language


 
 
Cervo Technologies
The Right Source to Outsource

MS Dynamics CRM 3.0

Python Programming Language

Reading a file and resuming reading.


Hi,

Simple question. Is it possible in python to write code of the type:

-----------------------------
while not eof  <- really want the EOF and not just an empty line!
    readline by line
end while;
-----------------------------

What I am using now is the implicit for loop after a readlines(). I don't
like this at all as a matter of opinion (my background is C++).

But also, in case for one reason or another the program crashes, I want to
be able to rexecute it and for it to resume reading from the same position
as it left. If a while loop like the one above can be implemented I can do
this simply by counting the lines!

I appreciate any help.

Karim

_________________________________________________________________
Windows Live Hotmail. Now with better security, storage and features.  
www.newhotmail.ca?icid=WLHMENCA149

In <mailman.8170.1180100553.32031.python-l@python.org>, Karim Ali
wrote:

> Simple question. Is it possible in python to write code of the type:

> -----------------------------
> while not eof  <- really want the EOF and not just an empty line!
>     readline by line
> end while;
> -----------------------------

while True:
    line = f.readline()
    if not line:
        break
    # Do something with `line`.

> What I am using now is the implicit for loop after a readlines(). I don't
> like this at all as a matter of opinion (my background is C++).

Both look ugly to Pythonistas too.  Files are iterable:

for line in f:
    # Do something with `line`.

> But also, in case for one reason or another the program crashes, I want to
> be able to rexecute it and for it to resume reading from the same position
> as it left. If a while loop like the one above can be implemented I can do
> this simply by counting the lines!

for line_nr, line in enumerate(f):
    # Do something with `line_nr` and `line`.

Ciao,
        Marc 'BlackJack' Rintsch

What about this:

fin = file('test.txt','r')
for line in fin:
    process_line(line)

By the way, readline returns an empty line only at the end of the file.
In all other cases, it will return a string that ends with a newline.
E.g. you can detect if you are at the end of the file or not. But using
the above snippet is usually better. Text files are iterable, line by line.

Best,

   Laszlo

"Karim Ali" <kak@hotmail.com> writes:
> -----------------------------
> while not eof  <- really want the EOF and not just an empty line!
>     readline by line
> end while;
> -----------------------------

for line in open_file:
  ...

It will stop on EOF, not on empty line.

> But also, in case for one reason or another the program crashes, I
> want to be able to rexecute it and for it to resume reading from the
> same position as it left. If a while loop like the one above can be
> implemented I can do this simply by counting the lines!

If you open the file in binary mode, you can easily keep track of the
position in file:

bytepos = 0
with file(filename) as f:
  for line in f:
    ... process line ...
    bytepos += len(line)

If you need to restart the operation, simply seek to the previously
known position:

# restart with old bytyepos
with file(filename) as f:
  f.seek(bytepos)
  for line in f:
    ... process line ...
    bytepos += len(line)

Thanks! This works perfectly. I did not know you can iterate directly using
the file handle!

Karim

_________________________________________________________________
Windows Live Hotmail, with safety bar colour coding, helps identify
suspicious mail before it takes your daughter out on a date. Upgrade today
for a better look. www.newhotmail.ca?icid=WLHMENCA152

 "Karim Ali" <k,,z@h..l.com> wrote:

> Hi,

> Simple question. Is it possible in python to write code of the type:

> -----------------------------
> while not eof  <- really want the EOF and not just an empty line!

readline() reads to the next newline - an empty line *is* EOF -
a blank line has at least a newline.

>     readline by line
> end while;
> -----------------------------

> What I am using now is the implicit for loop after a readlines(). I don't
> like this at all as a matter of opinion (my background is C++).

use readline() in a while true loop, or iterate over the file.

> But also, in case for one reason or another the program crashes, I want to
> be able to rexecute it and for it to resume reading from the same position
> as it left. If a while loop like the one above can be implemented I can do
> this simply by counting the lines!

Biggest problem is saving the count so that you can get at it again after
the crash

You could write to a file and flush and close after every line, but if there
is a crash, you have no guarantee that you will get the exact place back,
and whatever you are doing with the lines could get a duplicate or two
if you recover automatically after the crash. - it depends on what your OS
does to keep buffers in ram and data on disk congruent.

Overwriting the file like this is dangerous also as the crash could come
after the OS has deleted the directory entry, before making the new
file, so you could end with nothing saved...

You could also keep a more detailed journal, with entries like:

I am going to read line 0
I have successfully read line 0
I have processed line 0
I am going to write the output of line 0
I have written the output of line 0
I am going to read line 1

etcetera...

You do this by opening and closing the journal file in append mode
for every entry in it.

Even with this, it can get tricky, because there is the same uncertainty
referred to earlier - but you could get close.

Have a look at databases and commit - they are made to solve this exact
problem.

Is there not another way to see how far you actually got, by examining
the output?

Is it not easier to simply start again and do the whole job over, after
turfing the output?

Why bother about it at all - if you can use readlines, it means the
file fits into memory now - so it is not humongous, so it is probably not
"that" expensive to just do it all over.

I suppose it depends on what it is you are actually trying to achieve.

HTH

- Hendrik

Add to del.icio.us | Digg this | Stumble it | Powered by Megasolutions Inc