Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #59105 > unrolled thread

Creating a function for a directory

Back to article view | Back to comp.lang.python

Contents

  Creating a function for a directory Matt <mattgraves7@gmail.com> - 2013-11-11 14:26 -0800
    Re: Creating a function for a directory Joel Goldstick <joel.goldstick@gmail.com> - 2013-11-11 17:36 -0500
    Re: Creating a function for a directory Chris Angelico <rosuav@gmail.com> - 2013-11-12 09:38 +1100
    Re: Creating a function for a directory bob gailer <bgailer@gmail.com> - 2013-11-11 17:42 -0500
    Re: Creating a function for a directory Joel Goldstick <joel.goldstick@gmail.com> - 2013-11-11 17:44 -0500
    Re: Creating a function for a directory Mark Lawrence <breamoreboy@yahoo.co.uk> - 2013-11-11 22:45 +0000
    Re: Creating a function for a directory Rick Johnson <rantingrickjohnson@gmail.com> - 2013-11-11 14:51 -0800
      Re: Creating a function for a directory Chris Angelico <rosuav@gmail.com> - 2013-11-12 10:11 +1100
        Re: Creating a function for a directory Rick Johnson <rantingrickjohnson@gmail.com> - 2013-11-11 21:42 -0800
          Re: Creating a function for a directory Chris Angelico <rosuav@gmail.com> - 2013-11-12 17:02 +1100
      Re: Creating a function for a directory Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2013-11-12 00:33 +0000
        Re: Creating a function for a directory Neil Cerutti <neilc@norwich.edu> - 2013-11-12 13:42 +0000
    Re: Creating a function for a directory Matt <mattgraves7@gmail.com> - 2013-11-11 15:05 -0800
    Re: Creating a function for a directory Chris Angelico <rosuav@gmail.com> - 2013-11-12 09:51 +1100
    Re: Creating a function for a directory unknown <unknown@unknown.com> - 2013-11-12 10:07 +0000
      Re: Creating a function for a directory Peter Otten <__peter__@web.de> - 2013-11-12 11:24 +0100
        Re: Creating a function for a directory unknown <unknown@unknown.com> - 2013-11-12 11:26 +0000

#59105 — Creating a function for a directory

So I want to take the file, "desktop/test.txt" and write it to "desktop/newfolder/test.txt". I tried the below script, and it gave me: "IOError: [Errno 2] No such file or directory: 'desktop/%s.txt'". Any suggestions would be great.



def firstdev(file):
	in_file = open("desktop/%s.txt") % file
	indata = in_file.read()
	out_file = open("desktop/newfolder/%s.txt", 'w') % file
	out_file.write(indata)
	out_file.close()
	in_file.close()
	
firstdev("test")

[toc] | [next] | [standalone]

#59107

On Mon, Nov 11, 2013 at 5:26 PM, Matt <mattgraves7@gmail.com> wrote:
> So I want to take the file, "desktop/test.txt" and write it to "desktop/newfolder/test.txt". I tried the below script, and it gave me: "IOError: [Errno 2] No such file or directory: 'desktop/%s.txt'". Any suggestions would be great.

Its better in the future to cut and paste the traceback rather than
paraphrase...

That being said, it can't find the in_file.  Maybe you aren't in the
directory above desktop?  Try specifying a complete path, instead of
relative.  Once you solve that, be sure the ouput path is specificed
completely and that it exists.
>
>
>
> def firstdev(file):
>         in_file = open("desktop/%s.txt") % file
>         indata = in_file.read()
>         out_file = open("desktop/newfolder/%s.txt", 'w') % file
>         out_file.write(indata)
>         out_file.close()
>         in_file.close()
>
> firstdev("test")
> --
> https://mail.python.org/mailman/listinfo/python-list



-- 
Joel Goldstick
http://joelgoldstick.com

[toc] | [prev] | [next] | [standalone]

#59108

On Tue, Nov 12, 2013 at 9:26 AM, Matt <mattgraves7@gmail.com> wrote:
> So I want to take the file, "desktop/test.txt" and write it to "desktop/newfolder/test.txt". I tried the below script, and it gave me: "IOError: [Errno 2] No such file or directory: 'desktop/%s.txt'". Any suggestions would be great.
>
>
>
> def firstdev(file):
>         in_file = open("desktop/%s.txt") % file
>         indata = in_file.read()
>         out_file = open("desktop/newfolder/%s.txt", 'w') % file

You're using the % operator, which does your interpolations, at the
wrong point. You want to be adjusting the file name, not adjusting the
opened file:

        in_file = open("desktop/%s.txt" % file)
        out_file = open("desktop/newfolder/%s.txt" % file, 'w')

ChrisA

[toc] | [prev] | [next] | [standalone]

#59109

On 11/11/2013 5:26 PM, Matt wrote:
> So I want to take the file, "desktop/test.txt" and write it to "desktop/newfolder/test.txt". I tried the below script, and it gave me: "IOError: [Errno 2] No such file or directory: 'desktop/%s.txt'". Any suggestions would be great.
>
>
>
> def firstdev(file):
> 	in_file = open("desktop/%s.txt") % file
The problem is the above line. Rewrite is thus:

	in_file = open("desktop/%s.txt" % file)


> 	indata = in_file.read()
> 	out_file = open("desktop/newfolder/%s.txt", 'w') % file
Same here:
>          out_file = open("desktop/newfolder/%s.txt"% file, 'w')
>
>
> 	out_file.write(indata)
> 	out_file.close()
> 	in_file.close()
Also don't get in the habit of reassigning built-in functions e;g; file.
> 	
> firstdev("test")


-- 
Bob Gailer
919-636-4239
Chapel Hill NC

[toc] | [prev] | [next] | [standalone]

#59110

[Multipart message — attachments visible in raw view] — view raw

Sorry for incorect answer. Those guys nailed it
On Nov 11, 2013 5:43 PM, "bob gailer" <bgailer@gmail.com> wrote:

> On 11/11/2013 5:26 PM, Matt wrote:
>
>> So I want to take the file, "desktop/test.txt" and write it to
>> "desktop/newfolder/test.txt". I tried the below script, and it gave me:
>> "IOError: [Errno 2] No such file or directory: 'desktop/%s.txt'". Any
>> suggestions would be great.
>>
>>
>>
>> def firstdev(file):
>>         in_file = open("desktop/%s.txt") % file
>>
> The problem is the above line. Rewrite is thus:
>
>         in_file = open("desktop/%s.txt" % file)
>
>
>          indata = in_file.read()
>>         out_file = open("desktop/newfolder/%s.txt", 'w') % file
>>
> Same here:
>
>>          out_file = open("desktop/newfolder/%s.txt"% file, 'w')
>>
>>
>>         out_file.write(indata)
>>         out_file.close()
>>         in_file.close()
>>
> Also don't get in the habit of reassigning built-in functions e;g; file.
>
>>
>> firstdev("test")
>>
>
>
> --
> Bob Gailer
> 919-636-4239
> Chapel Hill NC
>
> --
> https://mail.python.org/mailman/listinfo/python-list
>

[toc] | [prev] | [next] | [standalone]

#59111

On 11/11/2013 22:26, Matt wrote:
> So I want to take the file, "desktop/test.txt" and write it to "desktop/newfolder/test.txt". I tried the below script, and it gave me: "IOError: [Errno 2] No such file or directory: 'desktop/%s.txt'". Any suggestions would be great.
>
>
>
> def firstdev(file):
> 	in_file = open("desktop/%s.txt") % file

in_file = open("desktop/%s.txt" % file)

> 	indata = in_file.read()
> 	out_file = open("desktop/newfolder/%s.txt", 'w') % file
> 	out_file.write(indata)
> 	out_file.close()
> 	in_file.close()
> 	
> firstdev("test")
>

-- 
Python is the second best programming language in the world.
But the best has yet to be invented.  Christian Tismer

Mark Lawrence

[toc] | [prev] | [next] | [standalone]

#59114

On Monday, November 11, 2013 4:26:46 PM UTC-6, Matt wrote:

> So I want to take the file, "desktop/test.txt" and write
> it to "desktop/newfolder/test.txt". I tried the below
> script, and it gave me: "IOError: [Errno 2] No such file
> or directory: 'desktop/%s.txt'". Any suggestions would be
> great.
>
> def firstdev(file):
> 	in_file = open("desktop/%s.txt") % file
> 	indata = in_file.read()
> 	out_file = open("desktop/newfolder/%s.txt", 'w') % file
> 	out_file.write(indata)
> 	out_file.close()
> 	in_file.close()
> firstdev("test")

1. i believe win32 file paths require a qualifying volume
letter.

2. Never, ever, *EVER* write data to disc before confirming
the paths your passing are pointing to the location you
intended to write the data. Use os.path.exists(path) to test
your paths BEFORE trying to write data.

3. Be sure your variables names are both "self documenting"
and "non clobbering". psst: "file" is a builtin! Using
"filename" would be a far wiser choice for a variable
containing a filename. When i see "file", i think of a "file
object"

4. When dealing with files you must be sure that exceptions
are handled cleanly. You don't want open file objects
floating aimlessly around in memory because your naive code
blew chunks.

5. Remember, you cannot write a file into a directory that
does not exist.

6 For OS compatibility always use os.path.join() to join
path parts into a whole. This method will insert the proper
separator for you depending on the OS.

[toc] | [prev] | [next] | [standalone]

#59116

On Tue, Nov 12, 2013 at 9:51 AM, Rick Johnson
<rantingrickjohnson@gmail.com> wrote:
> On Monday, November 11, 2013 4:26:46 PM UTC-6, Matt wrote:
>
>> So I want to take the file, "desktop/test.txt" and write
>> it to "desktop/newfolder/test.txt". I tried the below
>> script, and it gave me: "IOError: [Errno 2] No such file
>> or directory: 'desktop/%s.txt'". Any suggestions would be
>> great.
>>
>> def firstdev(file):
>>       in_file = open("desktop/%s.txt") % file
>>       indata = in_file.read()
>>       out_file = open("desktop/newfolder/%s.txt", 'w') % file
>>       out_file.write(indata)
>>       out_file.close()
>>       in_file.close()
>> firstdev("test")
>
> 1. i believe win32 file paths require a qualifying volume
> letter.

They do not; omitting the drive letter makes the path relative to the
current drive (and since it doesn't start with a directory specifier,
the current directory).

> 2. Never, ever, *EVER* write data to disc before confirming
> the paths your passing are pointing to the location you
> intended to write the data. Use os.path.exists(path) to test
> your paths BEFORE trying to write data.

Why? Why, oh why? If there's a problem, it'll be signalled with an
exception. Testing that the path exists opens you up to race problems,
so you won't see anything now, but some day your code will be in a
concurrent situation and you'll get unexpected exceptions. Why not
just expect the exception?

> 3. Be sure your variables names are both "self documenting"
> and "non clobbering". psst: "file" is a builtin! Using
> "filename" would be a far wiser choice for a variable
> containing a filename. When i see "file", i think of a "file
> object"

This one's arguable. How often do you use the 'file' builtin? I've
worked with files in Python innumerable times, and I don't remember
the last time I used 'file'. Yes, avoid shadowing important builtins
like 'list' and 'int', but 'id' and 'file' aren't that big a deal.

> 4. When dealing with files you must be sure that exceptions
> are handled cleanly. You don't want open file objects
> floating aimlessly around in memory because your naive code
> blew chunks.

They won't float around forever. The garbage collector will get to
them. If you're advocating use of 'with', that's only going to be an
issue if the code's called in a loop AND if it throws before the
close() calls come through.

> 5. Remember, you cannot write a file into a directory that
> does not exist.

So? Exception thrown, traceback printed to console, process terminated
cleanly. I'm not seeing a problem here.

> 6 For OS compatibility always use os.path.join() to join
> path parts into a whole. This method will insert the proper
> separator for you depending on the OS.

Technically true. However, most modern OSes will accept a slash.
There'll be a few situations where that's not true, but the OP an
happily just use slashes for simplicity. Remember the zen:
Practicality beats purity.

ChrisA

[toc] | [prev] | [next] | [standalone]

#59152

On Monday, November 11, 2013 5:11:52 PM UTC-6, Chris Angelico wrote:
> On Tue, Nov 12, 2013 at 9:51 AM, Rick Johnson
> > 1. i believe win32 file paths require a qualifying volume
> > letter.
> They do not; omitting the drive letter makes the path relative to the
> current drive (and since it doesn't start with a directory specifier,
> the current directory).

Hmm. Let's confirm:

    >>> import os
    >>> os.path.exists("C:/Windows/System32")
    True
    >>> os.path.exists("/Windows/System32")
    True

Yep, it's official. "Implicit File Path Resolution"
I need to author a PyWart on this soon!

> > 2. Never, ever, *EVER* write data to disc before confirming
> > the paths your passing are pointing to the location you
> > intended to write the data. Use os.path.exists(path) to test
> > your paths BEFORE trying to write data.
> Why? Why, oh why? If there's a problem, it'll be signalled
> with an exception.

Except when there's a problem that won't be signaled by an
EXCEPTION, but will be signaled by EMOTIONS; like for
instance removing the wrong directory or truncating the
wrong file because of a typo in your source code.

 OPPS! :-'(

> Testing that the path exists opens you up to race
> problems, so you won't see anything now, but some day your
> code will be in a concurrent situation and you'll get
> unexpected exceptions. Why not just expect the exception?

Because today i'm not facing a concurrent situation, so i'm
not going to bother and protect from it. Just like i'm not
facing a snake bite, so i won't bother to lug around a vial
of antidote. Your attempts to discredit me via hypothetical
scenarios is entertaining however.

> This one's arguable. How often do you use the 'file'
> builtin? I've worked with files in Python innumerable
> times, and I don't remember the last time I used 'file'.
> Yes, avoid shadowing important builtins like 'list' and
> 'int', but 'id' and 'file' aren't that big a deal.

My first concern is with the OP adopting "self documenting"
names. Shadowing file is merely ensuring that the OP knows
"file" is a builtin. Shadowing any builtin can create very
difficult bugs to track. This is the point the OP should
remember.

> They won't float around forever. The garbage collector
> will get to them.

That depends on how they are stored. You make too many
assumptions Chris. I don't have an problems with GC's, but
i'm not about to write sloppy code and "assume" the GC is
going to swoop in and save me like a feline trapped in a 
tree.

> > 5. Remember, you cannot write a file into a directory that
> > does not exist.
> So? Exception thrown, traceback printed to console,
> process terminated cleanly. I'm not seeing a problem here.

Now you're just trolling!

    def firstdev(file):
        in_file = open("desktop/%s.txt") % file
        indata = in_file.read()
        out_file = open("desktop/newfolder/%s.txt", 'w') % file

Just from reading that code NO ONE could know for sure if
"newfolder" even existed BEFORE the OP tried to open the
"out_file". Maybe the OP thinks that missing sub-directories
are auto-created by the open function, but there's no way to
know for sure, hence my comment.

> > 6 For OS compatibility always use os.path.join() to join
> > path parts into a whole. This method will insert the proper
> > separator for you depending on the OS.
> Technically true. However, most modern OSes will accept a
> slash. There'll be a few situations where that's not true,
> but the OP an happily just use slashes for simplicity.

Many a developer have lived to regret those words.

PS: I thought Steven was the official "devils advocate"
around here? Hmm. I guess everyone needs a day off.

[toc] | [prev] | [next] | [standalone]

#59155

On Tue, Nov 12, 2013 at 4:42 PM, Rick Johnson
<rantingrickjohnson@gmail.com> wrote:
> On Monday, November 11, 2013 5:11:52 PM UTC-6, Chris Angelico wrote:
>> On Tue, Nov 12, 2013 at 9:51 AM, Rick Johnson
>> > 1. i believe win32 file paths require a qualifying volume
>> > letter.
>> They do not; omitting the drive letter makes the path relative to the
>> current drive (and since it doesn't start with a directory specifier,
>> the current directory).
>
> Hmm. Let's confirm:
>
>     >>> import os
>     >>> os.path.exists("C:/Windows/System32")
>     True
>     >>> os.path.exists("/Windows/System32")
>     True
>
> Yep, it's official. "Implicit File Path Resolution"
> I need to author a PyWart on this soon!

That's not a Python issue at all. And are you seriously going to decry
relative pathnames?!?

>> > 2. Never, ever, *EVER* write data to disc before confirming
>> > the paths your passing are pointing to the location you
>> > intended to write the data. Use os.path.exists(path) to test
>> > your paths BEFORE trying to write data.
>> Why? Why, oh why? If there's a problem, it'll be signalled
>> with an exception.
>
> Except when there's a problem that won't be signaled by an
> EXCEPTION, but will be signaled by EMOTIONS; like for
> instance removing the wrong directory or truncating the
> wrong file because of a typo in your source code.

Which os.path.exists will not protect you from. So?

>> Testing that the path exists opens you up to race
>> problems, so you won't see anything now, but some day your
>> code will be in a concurrent situation and you'll get
>> unexpected exceptions. Why not just expect the exception?
>
> Because today i'm not facing a concurrent situation, so i'm
> not going to bother and protect from it. Just like i'm not
> facing a snake bite, so i won't bother to lug around a vial
> of antidote. Your attempts to discredit me via hypothetical
> scenarios is entertaining however.
>> > 6 For OS compatibility always use os.path.join() to join
>> > path parts into a whole. This method will insert the proper
>> > separator for you depending on the OS.
>> Technically true. However, most modern OSes will accept a
>> slash. There'll be a few situations where that's not true,
>> but the OP an happily just use slashes for simplicity.
>
> Many a developer have lived to regret those words.

(Reordered to bring together two comments getting one response) Do you
care about what you currently are seeing, or what you might some day
see? Concurrency is a common problem on many platforms; platforms not
supporting slashes between paths are rare, and most of them will have
other porting issues. Why are you ignoring the likely, while trying to
prevent the unlikely?

>> They won't float around forever. The garbage collector
>> will get to them.
>
> That depends on how they are stored. You make too many
> assumptions Chris. I don't have an problems with GC's, but
> i'm not about to write sloppy code and "assume" the GC is
> going to swoop in and save me like a feline trapped in a
> tree.

Ah, actually yes you are. When have you ever explicitly disposed of an
object in Python? You just let the GC deal with it for you.

>> > 5. Remember, you cannot write a file into a directory that
>> > does not exist.
>> So? Exception thrown, traceback printed to console,
>> process terminated cleanly. I'm not seeing a problem here.
>
> Now you're just trolling!

Mercurial signals some errors by dumping an exception to stderr. This
works quite nicely, even in production. If your script can't recover
from being given a non-writeable directory, don't catch the exception.

>     def firstdev(file):
>         in_file = open("desktop/%s.txt") % file
>         indata = in_file.read()
>         out_file = open("desktop/newfolder/%s.txt", 'w') % file
>
> Just from reading that code NO ONE could know for sure if
> "newfolder" even existed BEFORE the OP tried to open the
> "out_file". Maybe the OP thinks that missing sub-directories
> are auto-created by the open function, but there's no way to
> know for sure, hence my comment.

And the OP might have thought that it would automatically publish that
to the world, tweet the URL, and create a Facebook status saying "just
opened a file lolol" and get five people to Like it. Doesn't mean we
need to probe before doing things.

ChrisA

[toc] | [prev] | [next] | [standalone]

#59125

On Mon, 11 Nov 2013 14:51:45 -0800, Rick Johnson wrote:

> 2. Never, ever, *EVER* write data to disc before confirming the paths
> your passing are pointing to the location you intended to write the
> data. Use os.path.exists(path) to test your paths BEFORE trying to write
> data.

This is subject to a race condition, which opens you to a security 
vulnerability: "time of check to time of use" bugs.

If you follow Rick's advice, and write code like this:

if os.path.exists(path):
    write_to(path)  # custom function to open and write to the file
else:
    handle_missing_file()


your code is doubly buggy. First, os.path.exists only tells you if the 
path exists, not whether it is writable. Perhaps it is on read-only 
media, or you don't have permission to open it, or it's a directory.

Secondly, even if the file exists at the moment you call os.path.exists, 
there is no guarantee that it will still exist a moment later when you 
try writing to it. Another process may delete or rename the file, or 
change permissions in the meantime. So you have to write:

if os.path.exists(path):
    try:
        write_to(path)
    except (IOError, OSError):
        handle_missing_or_locked_file()
else:
    handle_missing_or_locked_file()


But now your test doesn't actually accomplish anything. Worse, just 
because the path *doesn't* exist when you check using exists, that 
doesn't mean it won't exist by the time you call write_to!

Using os.path.exists before opening a file is, for the most part, a waste 
of time.



-- 
Steven

[toc] | [prev] | [next] | [standalone]

#59193

On 2013-11-12, Steven D'Aprano
<steve+comp.lang.python@pearwood.info> wrote:
> Using os.path.exists before opening a file is, for the most
> part, a waste of time.

I use it in conjuction with file creation times to check that I
didn't forget to update/create one of the data files a process
might need.

Since I have to create the data file manually, race conditions
seem unlikely.

-- 
Neil Cerutti

[toc] | [prev] | [next] | [standalone]

#59115

Thank you guys so much. Brain fart moment. I appreciate it

[toc] | [prev] | [next] | [standalone]

#59131

On Tue, Nov 12, 2013 at 9:44 AM, Joel Goldstick
<joel.goldstick@gmail.com> wrote:
> Sorry for incorect answer. Those guys nailed it

Your answer wasn't incorrect, because it didn't give any false
information. Bob and I saw the problem itself and gave advice, but you
gave useful general advice on how to find the problem, which is worth
bearing in mind. Matt, read Joel's response for useful tips on how to
figure out what might have gone wrong. :)

ChrisA

[toc] | [prev] | [next] | [standalone]

#59175

On Mon, 11 Nov 2013 14:26:46 -0800, Matt wrote:

> So I want to take the file, "desktop/test.txt" and write it to
> "desktop/newfolder/test.txt". I tried the below script, and it gave me:
> "IOError: [Errno 2] No such file or directory: 'desktop/%s.txt'". Any
> suggestions would be great.
> 
> 
> 
> def firstdev(file):
> 	in_file = open("desktop/%s.txt") % file indata = in_file.read()
> 	out_file = open("desktop/newfolder/%s.txt", 'w') % file
> 	out_file.write(indata)
> 	out_file.close()
> 	in_file.close()
> 	
> firstdev("test")

would it not be more efficient and less error prone to use the os module 
to copy the file rather than manually reading & re-writing it (unless of-
course you intend to expand this in future to process the data first)?

[toc] | [prev] | [next] | [standalone]

#59179

unknown wrote:

> On Mon, 11 Nov 2013 14:26:46 -0800, Matt wrote:
> 
>> So I want to take the file, "desktop/test.txt" and write it to
>> "desktop/newfolder/test.txt". I tried the below script, and it gave me:
>> "IOError: [Errno 2] No such file or directory: 'desktop/%s.txt'". Any
>> suggestions would be great.
>> 
>> 
>> 
>> def firstdev(file):
>> in_file = open("desktop/%s.txt") % file indata = in_file.read()
>> out_file = open("desktop/newfolder/%s.txt", 'w') % file
>> out_file.write(indata)
>> out_file.close()
>> in_file.close()
>> 
>> firstdev("test")
> 
> would it not be more efficient and less error prone to use the os module
> to copy the file rather than manually reading & re-writing it (unless of-
> course you intend to expand this in future to process the data first)?

Hmm, I don't see a suitable function in os -- but there is 

shutil.copy()

[toc] | [prev] | [next] | [standalone]

#59184

On Tue, 12 Nov 2013 11:24:02 +0100, Peter Otten wrote:

> unknown wrote:
> 
>> On Mon, 11 Nov 2013 14:26:46 -0800, Matt wrote:
>> 
>>> So I want to take the file, "desktop/test.txt" and write it to
>>> "desktop/newfolder/test.txt". I tried the below script, and it gave
>>> me:
>>> "IOError: [Errno 2] No such file or directory: 'desktop/%s.txt'". Any
>>> suggestions would be great.
>>> 
>>> 
>>> 
>>> def firstdev(file):
>>> in_file = open("desktop/%s.txt") % file indata = in_file.read()
>>> out_file = open("desktop/newfolder/%s.txt", 'w') % file
>>> out_file.write(indata)
>>> out_file.close()
>>> in_file.close()
>>> 
>>> firstdev("test")
>> 
>> would it not be more efficient and less error prone to use the os
>> module to copy the file rather than manually reading & re-writing it
>> (unless of-
>> course you intend to expand this in future to process the data first)?
> 
> Hmm, I don't see a suitable function in os -- but there is
> 
> shutil.copy()

that's what i get for posting on the fly without checking first but i 
think the principle holds.

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web