Path: csiph.com!v102.xanadu-bbs.net!xanadu-bbs.net!feeder.erje.net!eu.feeder.erje.net!eweka.nl!lightspeed.eweka.nl!194.109.133.81.MISMATCH!newsfeed.xs4all.nl!newsfeed2.news.xs4all.nl!xs4all!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail Return-Path: X-Original-To: python-list@python.org Delivered-To: python-list@mail.python.org X-Spam-Status: OK 0.000 X-Spam-Evidence: '*H*': 1.00; '*S*': 0.00; 'python.': 0.02; 'python,': 0.02; '"this': 0.03; '16,': 0.03; 'syntax': 0.04; 'string.': 0.05; 'subject:Python': 0.06; 'context': 0.07; 'responding': 0.07; 'assuming': 0.09; 'bash': 0.09; 'exist,': 0.09; 'f.close()': 0.09; 'filename': 0.09; 'function:': 0.09; 'oh,': 0.09; 'python:': 0.09; 'sub': 0.09; 'subject:How': 0.10; 'cc:addr:python-list': 0.11; 'python': 0.11; 'assume': 0.14; '"i': 0.16; "'w')": 0.16; 'dd,': 0.16; 'does,': 0.16; "don'": 0.16; 'least.': 0.16; 'nail': 0.16; 'programmer,': 0.16; 'python."': 0.16; 'to:addr:pearwood.info': 0.16; 'to:addr:steve+comp.lang.python': 0.16; "to:name:steven d'aprano": 0.16; 'user-defined': 0.16; '\xa0you': 0.16; 'subject: ?': 0.16; 'language': 0.16; 'wrote:': 0.18; 'module': 0.19; 'file,': 0.19; 'touch.': 0.19; '8bit%:5': 0.22; 'command': 0.22; '>>>': 0.22; 'example': 0.22; 'aug': 0.22; 'shell': 0.22; 'cc:addr:python.org': 0.22; 'creating': 0.23; '>>>': 0.24; 'byte': 0.24; 'bytes': 0.24; 'rapidly': 0.24; 'skip:\xa0 20': 0.24; 'stick': 0.24; '\xa0if': 0.24; 'fine': 0.24; 'question': 0.24; 'cc:2**0': 0.24; 'cc:no real name:2**0': 0.24; 'script': 0.25; '>': 0.26; 'first,': 0.26; 'order.': 0.26; 'shown': 0.26; 'post': 0.26; 'least': 0.26; 'header:In- Reply-To:1': 0.27; "doesn't": 0.30; 'message-id:@mail.gmail.com': 0.30; "i'm": 0.30; 'url:mailman': 0.30; 'comments': 0.31; '"': 0.31; "d'aprano": 0.31; 'equivalent.': 0.31; 'file:': 0.31; 'steven': 0.31; 'file': 0.32; 'probably': 0.32; 'regular': 0.32; 'another': 0.32; 'open': 0.33; 'worked': 0.33; 'url:python': 0.33; 'running': 0.33; 'fri,': 0.33; 'guess': 0.33; 'copying': 0.34; 'problem': 0.35; 'skip:s 30': 0.35; '(2)': 0.35; 'case,': 0.35; 'etc': 0.35; 'usual': 0.35; 'but': 0.35; 'received:google.com': 0.35; 'there': 0.35; 'really': 0.36; 'disk': 0.36; 'done,': 0.36; 'done': 0.36; 'url:listinfo': 0.36; 'doing': 0.36; 'thanks': 0.36; 'possible': 0.36; 'url:org': 0.36; 'should': 0.36; 'skip:o 20': 0.38; 'skip:& 10': 0.38; '8bit%:4': 0.38; 'tasks': 0.38; 'files': 0.38; 'pm,': 0.38; 'that,': 0.38; 'anything': 0.39; 'skip:& 20': 0.39; 'does': 0.39; 'sure': 0.39; 'commands': 0.60; 'solve': 0.60; 'most': 0.60; 'skip:o 30': 0.61; 'new': 0.61; 'range': 0.61; 'simply': 0.61; 'simple': 0.61; "you're": 0.61; 'save': 0.62; 'such': 0.63; 'today': 0.64; 'more': 0.64; 'temporary': 0.65; 'close': 0.67; 'believe': 0.68; 'yes': 0.68; 'reverse': 0.68; 'inline': 0.74; 'touch': 0.74; 'goal': 0.75; '100': 0.79; 'subject:this': 0.83; '4:29': 0.84; 'completely,': 0.84; '\xa0but': 0.84; '\xa0have': 0.84; 'hate': 0.91; 'shell,': 0.91; '2013': 0.98 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=pIMm+avGZuLOXTVE5bwaOYXCrtpTjZ5zsuJ+S6mgQ4I=; b=pU9CVLtvpwvEtj/Np3qkTR4Rf9U9MZWjlvjy7KSRek1yI4ICmQDgm9rVWaxsLV+wFg SrP4cNsEmQ0E60zSrftKED1NplucdXHAZB+lYNXgJOV8jD8UrcK31c84bkdTsy/jrj0W msIjnQgOXpxX6xHYGZY+Djfi63mX3Jn0OWs2/UeF+WUcGvW3GgTYMtU3u4oG+rkZWQSx LYFnb4i0hEaDOv0eFAcVXuExH8czkglFgPN6sJYRKs+AOdR0wzKJaLhHbwPKJAjYi2Gu DVKXqp4h7/VKzBMtDmYyiT1lp1oYjGAp432pB527ZFxw1qNj/DzgzqfZyLGwwI43Ufms STzg== MIME-Version: 1.0 X-Received: by 10.112.136.38 with SMTP id px6mr2492666lbb.29.1376845561645; Sun, 18 Aug 2013 10:06:01 -0700 (PDT) In-Reply-To: <520e060d$0$30000$c3e8da3$5496439d@news.astraweb.com> References: <520e060d$0$30000$c3e8da3$5496439d@news.astraweb.com> Date: Sun, 18 Aug 2013 22:36:01 +0530 Subject: Re: How to I do this in Python ? From: Ganesh Pal To: "Steven D'Aprano" Content-Type: multipart/alternative; boundary=089e0112cc6a9cd07c04e43bd645 X-Mailman-Approved-At: Sun, 18 Aug 2013 21:58:15 +0200 Cc: python-list@python.org X-BeenThere: python-list@python.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: General discussion list for the Python programming language List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Newsgroups: comp.lang.python Message-ID: Lines: 487 NNTP-Posting-Host: 2001:888:2000:d::a6 X-Trace: 1376855896 news.xs4all.nl 15903 [2001:888:2000:d::a6]:42708 X-Complaints-To: abuse@xs4all.nl Xref: csiph.com comp.lang.python:52658 --089e0112cc6a9cd07c04e43bd645 Content-Type: text/plain; charset=ISO-8859-1 Hi Steven , Firstly thanks for responding to the question and also guiding me on how to post the question in the right order ( general to least important order ) Please find the comments >>> inline On Fri, 16 Aug 2013 11:51:32 +0530, Ganesh Pal wrote: > My goal is to create various kinds of files like sparse, regular > ,directories, hard and symlinks etc > what would be the best way to do achieve this ? Use your shell, such as bash or csh or equivalent. For simple tasks like that, it will solve the problem much more simply than Python. There are three good reasons for doing this in Python: - "This is only a small part of a larger Python application." - "I'm doing this to learn how to use Python." - "I really hate my shell." But of you just want to get the job done, and don't care what language you use, use the shell. Now, having said that, I'm going to assume you have a good reason to use Python: >>> You have hit the nail on the head - The main goal there was to learn python and avoid using shell commands as much as possible , I want to avoid shell and use Python even if its costly. Example : I don' want to create sparse files using dd command ( #dd if=/dev/zero of=sparse-file bs=1 count=1 seek=1024k ) If the same can be done fseek . But I guess we might have to d > Case (2) : > > Is there a better way to create the files in Python other than using > sub process module and running dd command as shown below .. > > Example : > > # creating sparse File > sparse_path = os.path.join(path,'sparsefiles') > os.makedirs(sparse_path) > os.chdir(sparse_path) > sparsefiles = "dd if=/dev/zero of=sp1 count=0 bs=1 seek=10G" > process_0 = subprocess.Popen(sparsefiles, stdout=subprocess.PIPE, > stderr=subprocess.PIPE, shell=True) > > # Creating Regular files > Regular_path = os.path.join(path,'regularfiles') > os.makedirs(Regular_path) > os.chdir(Regular_path) > regularfiles = " dd if=/dev/urandom of=file1 count=0 bs=1 seek=10" > process_1 = subprocess.Popen(regularfiles, stdout=subprocess.PIPE, > stderr=subprocess.PIPE, shell=True) What do you mean by "better"? There's probably nothing that will be faster at rapidly copying bytes from one file to another than dd. But not with a blocksize of 1 byte at a time. It's more usual to set bs=512 or bs=1024. Oh, I see you're not actually writing anything to the file (count=0). In that case, instead of using dd, you should use touch. I'm not sure that shell=True is a good idea. In Python, to create a new empty file, setting its contents to empty if it already exists: open("filename", "w").close() That will open the file, creating it if it doesn't exist, emptying it if it does, then close it. To touch a file without emptying it: open("filename", "a").close() To make a sparse file, assuming your file system supports it, I believe you actually have to write at least one byte to the file: f = open('foo', 'w') f.seek(10000) f.write('\0') f.close() >>> Thanks for the suggestion on creating empty files and they worked fine . I was today playing with the temp-file module to create temporary files and directories , Iam yet to explore it completely, but in current context , I had a quick question using temp-file module is its possible to create empty temporary files and save them on disk in the user-defined path ? If " yes " then can this also be an alternative and does this have any drawback ? On Fri, Aug 16, 2013 at 4:29 PM, Steven D'Aprano < steve+comp.lang.python@pearwood.info> wrote: > Hi Ganesh, and welcome! > > Unfortunately, you ask your questions in reverse order. The most general > (and important) question comes last, and the least important first, so > I'm going to slice-and-dice your post and answer from most general to > least. > > > On Fri, 16 Aug 2013 11:51:32 +0530, Ganesh Pal wrote: > > > My goal is to create various kinds of files like sparse, regular > > ,directories, hard and symlinks etc > > what would be the best way to do achieve this ? > > Use your shell, such as bash or csh or equivalent. For simple tasks like > that, it will solve the problem much more simply than Python. > > There are three good reasons for doing this in Python: > > - "This is only a small part of a larger Python application." > > - "I'm doing this to learn how to use Python." > > - "I really hate my shell." > > But of you just want to get the job done, and don't care what language > you use, use the shell. > > > Now, having said that, I'm going to assume you have a good reason to use > Python: > > > > Case (2) : > > > > Is there a better way to create the files in Python other than using > > sub process module and running dd command as shown below .. > > > > Example : > > > > # creating sparse File > > sparse_path = os.path.join(path,'sparsefiles') > > os.makedirs(sparse_path) > > os.chdir(sparse_path) > > sparsefiles = "dd if=/dev/zero of=sp1 count=0 bs=1 seek=10G" > > process_0 = subprocess.Popen(sparsefiles, stdout=subprocess.PIPE, > > stderr=subprocess.PIPE, shell=True) > > > > # Creating Regular files > > Regular_path = os.path.join(path,'regularfiles') > > os.makedirs(Regular_path) > > os.chdir(Regular_path) > > regularfiles = " dd if=/dev/urandom of=file1 count=0 bs=1 seek=10" > > process_1 = subprocess.Popen(regularfiles, stdout=subprocess.PIPE, > > stderr=subprocess.PIPE, shell=True) > > What do you mean by "better"? There's probably nothing that will be > faster at rapidly copying bytes from one file to another than dd. But not > with a blocksize of 1 byte at a time. It's more usual to set bs=512 or > bs=1024. > > Oh, I see you're not actually writing anything to the file (count=0). In > that case, instead of using dd, you should use touch. > > I'm not sure that shell=True is a good idea. > > In Python, to create a new empty file, setting its contents to empty if > it already exists: > > open("filename", "w").close() > > > That will open the file, creating it if it doesn't exist, emptying it if > it does, then close it. > > To touch a file without emptying it: > > open("filename", "a").close() > > > To make a sparse file, assuming your file system supports it, I believe > you actually have to write at least one byte to the file: > > f = open('foo', 'w') > f.seek(10000) > f.write('\0') > f.close() > > > > [...] > > How do I loop my script to create 100 of files like sp1 , sp2 ,sp3,.. > > sp100 .. using the same syntax > > To generate the various file names, you need to loop over a counter from > 1 to 100, and stick the count into a string. Use a for-loop and the range > function: > > for i in range(1, 101): > filename = "sp" + str(i) > open(filename, "a").close() > > > If you are a C programmer, you might prefer this style: > > filename = "sp%d" % i > > Or a more object-oriented style: > > filename = "sp{:d}".format(i) > > > > -- > Steven > -- > http://mail.python.org/mailman/listinfo/python-list > --089e0112cc6a9cd07c04e43bd645 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
Hi Steven ,

Firstly thanks for responding to the question and also guiding me on how t= o post the question in the right order ( general to least important order )= =A0

Please find the commen= ts >>> inline

On Fri, 16 Aug 2013 11:51:32 +0530,= Ganesh Pal wrote:

> My goal is to create various kinds of files = like sparse, regular
> ,directories, hard and symlinks etc
> wh= at would be the best way to do achieve this ?

Use your shell, such as bash = or csh or equivalent. For simple tasks like
that, it will solve the proble= m much more simply than Python.

There= are three good reasons for doing this in Python:

- "This is only a small part of a larger Python application.= "

- &qu= ot;I'm doing this to learn how to use Python."

- "I really hate my shell."

But o= f you just want to get the job done, and don't care what language
you = use, use the shell.


Now, having said that, I'm going to assume= you have a good reason to use
Python:


= =A0 =A0 =A0 =A0 Example : I don' want to create sparse files using =A0d= d =A0 command ( #dd if=3D/dev/zero of=3Dsparse-file b= s=3D1 count=3D1 seek=3D1024k )=A0If =A0the same can be done =A0fseek .

=A0 =A0 =A0 =A0 =A0But I guess we might have to d=A0 =A0 = =A0 =A0=A0
=A0 =A0=A0
=

>= Case (2) :
>
> Is there a better way to create the files in Python other than= using
> sub process module and running dd command as shown below ..<= br>>
> Example :
>
> # creating sparse File
> = =A0sparse_path =3D os.path.join(path,'sparsefiles')
> =A0 =A0 =A0os.makedirs(sparse_path)
> =A0 =A0 =A0os.chdir(sparse= _path)
> =A0 =A0 =A0sparsefiles =3D "dd if=3D/dev/zero of=3Dsp1 = count=3D0 bs=3D1 seek=3D10G"
> =A0 =A0 =A0process_0 =3D subproce= ss.Popen(sparsefiles, stdout=3Dsubprocess.PIPE,
> stderr=3Dsubprocess.PIPE, shell=3DTrue)
>
> =A0 =A0 # Crea= ting Regular files
> =A0 =A0 =A0Regular_path =3D os.path.join(path,&#= 39;regularfiles')
> =A0 =A0 =A0os.makedirs(Regular_path)
> = =A0 =A0 =A0os.chdir(Regular_path)
> =A0 =A0 =A0regularfiles =3D " dd if=3D/dev/urandom of=3Dfile1 cou= nt=3D0 bs=3D1 seek=3D10"
> =A0 =A0 =A0process_1 =3D subprocess.P= open(regularfiles, stdout=3Dsubprocess.PIPE,
> stderr=3Dsubprocess.PI= PE, shell=3DTrue)

What do you mean by "better"? There's probably nothing that w= ill be
faster at rapidly copying bytes from one file to another than dd.= But not
with a blocksize of 1 byte at a time. It's more usual to se= t bs=3D512 or
bs=3D1024.

Oh, I see you're not actually writing anything to the= file (count=3D0). In
that case, instead of using dd, you should use tou= ch.

I'm not sure that shell=3DTrue is a good idea.

In Pyt= hon, to create a new empty file, setting its contents to empty if
it already exists:

open("filename", "w").close()=


That will open the file, creating it if it doesn't exist, e= mptying it if
it does, then close it.

To touch a file without emp= tying it:

open("filename", "a").close()


To make a = sparse file, assuming your file system supports it, I believe
you actual= ly have to write at least one byte to the file:

f =3D open('foo&= #39;, 'w')
f.seek(10000)
f.write('\0')
f.close()

>>> =A0 Thanks for the sugg= estion on creating empty files and =A0they worked fine . I was today playin= g with the temp-file =A0module to create =A0temporary files and directories= ,
=A0 =A0 =A0 =A0 = =A0Iam yet to explore it completely, but in current context , I had a quick= question using temp-file module is its =A0possible to create empty =A0temp= orary files and save them on disk in the user-defined path ?
=A0 =A0 =A0 =A0 = =A0If =A0" yes " =A0then can this also be an alternative and does= this have any drawback =A0?

On Fri, Aug 16, 2013 at 4:29 PM, Steven D'Aprano <= s= teve+comp.lang.python@pearwood.info> wrote:
Hi Ganesh, and welcome!

Unfortunately, you ask your questions in reverse order. The most general (and important) question comes last, and the least important first, so
I'm going to slice-and-dice your post and answer from most general to least.


On Fri, 16 Aug 2013 11:51:32 +0530, Ganesh Pal wrote:

> My goal is to create various kinds of files like sparse, regular
> ,directories, hard and symlinks etc
> what would be the best way to do achieve this ?

Use your shell, such as bash or csh or equivalent. For simple tasks l= ike
that, it will solve the problem much more simply than Python.

There are three good reasons for doing this in Python:

- "This is only a small part of a larger Python application."

- "I'm doing this to learn how to use Python."

- "I really hate my shell."

But of you just want to get the job done, and don't care what language<= br> you use, use the shell.


Now, having said that, I'm going to assume you have a good reason to us= e
Python:


> Case (2) :
>
> Is there a better way to create the files in Python other than using > sub process module and running dd command as shown below ..
>
> Example :
>
> # creating sparse File
> =A0sparse_path =3D os.path.join(path,'sparsefiles')
> =A0 =A0 =A0os.makedirs(sparse_path)
> =A0 =A0 =A0os.chdir(sparse_path)
> =A0 =A0 =A0sparsefiles =3D "dd if=3D/dev/zero of=3Dsp1 count=3D0 = bs=3D1 seek=3D10G"
> =A0 =A0 =A0process_0 =3D subprocess.Popen(sparsefiles, stdout=3Dsubpro= cess.PIPE,
> stderr=3Dsubprocess.PIPE, shell=3DTrue)
>
> =A0 =A0 # Creating Regular files
> =A0 =A0 =A0Regular_path =3D os.path.join(path,'regularfiles')<= br> > =A0 =A0 =A0os.makedirs(Regular_path)
> =A0 =A0 =A0os.chdir(Regular_path)
> =A0 =A0 =A0regularfiles =3D " dd if=3D/dev/urandom of=3Dfile1 cou= nt=3D0 bs=3D1 seek=3D10"
> =A0 =A0 =A0process_1 =3D subprocess.Popen(regularfiles, stdout=3Dsubpr= ocess.PIPE,
> stderr=3Dsubprocess.PIPE, shell=3DTrue)

What do you mean by "better"? There's probably nothing = that will be
faster at rapidly copying bytes from one file to another than dd. But not with a blocksize of 1 byte at a time. It's more usual to set bs=3D512 o= r
bs=3D1024.

Oh, I see you're not actually writing anything to the file (count=3D0).= In
that case, instead of using dd, you should use touch.

I'm not sure that shell=3DTrue is a good idea.

In Python, to create a new empty file, setting its contents to empty if
it already exists:

open("filename", "w").close()


That will open the file, creating it if it doesn't exist, emptying it i= f
it does, then close it.

To touch a file without emptying it:

open("filename", "a").close()


To make a sparse file, assuming your file system supports it, I believe
you actually have to write at least one byte to the file:

f =3D open('foo', 'w')
f.seek(10000)
f.write('\0')
f.close()



[...]
> How do I =A0loop my script to create 100 of files = =A0like sp1 , sp2 ,sp3,..
> sp100 =A0.. using the same syntax

To generate the various file names, you need to loop over a counter f= rom
1 to 100, and stick the count into a string. Use a for-loop and the range function:

for i in range(1, 101):
=A0 =A0 filename =3D "sp" + str(i)
=A0 =A0 open(filename, "a").close()


If you are a C programmer, you might prefer this style:

=A0 =A0 filename =3D "sp%d" % i

Or a more object-oriented style:

=A0 =A0 filename =3D "sp{:d}".format(i)



--
Steven
--
http://mail.python.org/mailman/listinfo/python-list

--089e0112cc6a9cd07c04e43bd645--