python - How to open a file using the open with statement

223
votes

I'm looking at how to do file input and output in Python. I've written the following code to read a list of names (one per line) from a file into another file while checking a name against the names in the file and appending text to the occurrences in the file. The code works. Could it be done better?

I'd wanted to use the with open(... statement for both input and output files but can't see how they could be in the same block meaning I'd need to store the names in a temporary location. 

def filter(txt, oldfile, newfile):
    '''\
    Read a list of names from a file line by line into an output file.
    If a line begins with a particular name, insert a string of text
    after the name before appending the line to the output file.
    '''

    outfile = open(newfile, 'w')
    with open(oldfile, 'r', encoding='utf-8') as infile:
        for line in infile:
            if line.startswith(txt):
                line = line[0:len(txt)] + ' - Truly a great person!\n'
            outfile.write(line)

    outfile.close()
    return # Do I gain anything by including this?

# input the name you want to check against
text = input('Please enter the name of a great person: ')    
letsgo = filter(text,'Spanish', 'Spanish2')
4
pythonfilepython-3.xfile-ioio
"meaning I'd need to store the names in a temporary location"? Can you explain what you mean by this? – S.Lott
Note that filter() is a built-in function and so you should probably choose a different name for your function. – Tom
@Tom does a function in the namespace override the built-in function? – UpTide
@UpTide: Yes, Python operates in LEGB order -- Local, Enclosing, Global, Built-in (see stackoverflow.com/questions/291978/…). So, if you make a global function (filter()), it will be found before the built-in filter() – Tom

4 Answers

335
votes

Python allows putting multiple open() statements in a single with. You comma-separate them. Your code would then be:

def filter(txt, oldfile, newfile):
    '''\
    Read a list of names from a file line by line into an output file.
    If a line begins with a particular name, insert a string of text
    after the name before appending the line to the output file.
    '''

    with open(newfile, 'w') as outfile, open(oldfile, 'r', encoding='utf-8') as infile:
        for line in infile:
            if line.startswith(txt):
                line = line[0:len(txt)] + ' - Truly a great person!\n'
            outfile.write(line)

# input the name you want to check against
text = input('Please enter the name of a great person: ')    
letsgo = filter(text,'Spanish', 'Spanish2')

And no, you don't gain anything by putting an explicit return at the end of your function. You can use return to exit early, but you had it at the end, and the function will exit without it. (Of course with functions that return a value, you use the return to specify the value to return.)

Using multiple open() items with with was not supported in Python 2.5 when the with statement was introduced, or in Python 2.6, but it is supported in Python 2.7 and Python 3.1 or newer.

http://docs.python.org/reference/compound_stmts.html#the-with-statement http://docs.python.org/release/3.1/reference/compound_stmts.html#the-with-statement

If you are writing code that must run in Python 2.5, 2.6 or 3.0, nest the with statements as the other answers suggested or use contextlib.nested.

33
votes

Use nested blocks like this,

with open(newfile, 'w') as outfile:
    with open(oldfile, 'r', encoding='utf-8') as infile:
        # your logic goes right here
13
votes

You can nest your with blocks. Like this:

with open(newfile, 'w') as outfile:
    with open(oldfile, 'r', encoding='utf-8') as infile:
        for line in infile:
            if line.startswith(txt):
                line = line[0:len(txt)] + ' - Truly a great person!\n'
            outfile.write(line)

This is better than your version because you guarantee that outfile will be closed even if your code encounters exceptions. Obviously you could do that with try/finally, but with is the right way to do this.

Or, as I have just learnt, you can have multiple context managers in a with statement as described by @steveha. That seems to me to be a better option than nesting.

And for your final minor question, the return serves no real purpose. I would remove it.

1
votes

Sometimes, you might want to open a variable amount of files and treat each one the same, you can do this with contextlib

from contextlib import ExitStack
filenames = [file1.txt, file2.txt, file3.txt]

with open('outfile.txt', 'a') as outfile:
    with ExitStack() as stack:
        file_pointers = [stack.enter_context(open(file, 'r')) for file in filenames]                
            for fp in file_pointers:
                outfile.write(fp.read())