Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #47207 > unrolled thread

How to store a variable when a script is executing for next time execution?

Back to article view | Back to comp.lang.python

Contents

  How to store a variable when a script is executing for next time execution? Avnesh Shakya <avnesh.nitk@gmail.com> - 2013-06-06 03:50 -0700
    Re: How to store a variable when a script is executing for next time execution? Cameron Simpson <cs@zip.com.au> - 2013-06-06 21:19 +1000
    Re: How to store a variable when a script is executing for next time execution? Avnesh Shakya <avnesh.nitk@gmail.com> - 2013-06-06 17:36 +0530
    Re: How to store a variable when a script is executing for next time execution? Dave Angel <davea@davea.name> - 2013-06-06 08:14 -0400
    Re: How to store a variable when a script is executing for next time execution? Jussi Piitulainen <jpiitula@ling.helsinki.fi> - 2013-06-06 15:46 +0300
    Re: How to store a variable when a script is executing for next time execution? Chris Angelico <rosuav@gmail.com> - 2013-06-07 01:37 +1000
    Re: How to store a variable when a script is executing for next time execution? MRAB <python@mrabarnett.plus.com> - 2013-06-06 16:54 +0100
    Re: How to store a variable when a script is executing for next time execution? Chris Angelico <rosuav@gmail.com> - 2013-06-07 01:56 +1000

#47207 — How to store a variable when a script is executing for next time execution?

hi,
   I am running a python script and it will create a file name like filename0.0.0 and If I run it again then new file will create one more like filename0.0.1...... my code is-

i = 0
for i in range(1000):
    try:
        with open('filename%d.%d.%d.json'%(0,0,i,)): pass
        continue
    except IOError:
        dataFile = file('filename%d.%d.%d.json'%(0,0,i,), 'a+')
        break
But It will take more time after creating many files, So i want to store value of last var "i" in a variable so that when i run my script again then I can use it. for example-
                 my last created file is filename0.0.27 then it should store 27 in a variable and when i run again then new file should be created 0.0.28 according to last value "27", so that i could save time and it can create file fast..

Please give me suggestion for it.. How is it possible?
Thanks

[toc] | [next] | [standalone]

#47214

On 06Jun2013 03:50, Avnesh Shakya <avnesh.nitk@gmail.com> wrote:
| hi,
|    I am running a python script and it will create a file name like filename0.0.0 and If I run it again then new file will create one more like filename0.0.1...... my code is-
| 
| i = 0
| for i in range(1000):
|     try:
|         with open('filename%d.%d.%d.json'%(0,0,i,)): pass
|         continue
|     except IOError:
|         dataFile = file('filename%d.%d.%d.json'%(0,0,i,), 'a+')
|         break
| But It will take more time after creating many files, So i want to store value of last var "i" in a variable so that when i run my script again then I can use it. for example-
|                  my last created file is filename0.0.27 then it should store 27 in a variable and when i run again then new file should be created 0.0.28 according to last value "27", so that i could save time and it can create file fast..
| 
| Please give me suggestion for it.. How is it possible?

Write it to a file? Read the file next time the script runs?

BTW, trying to open zillions of files is slow.
But using listdir to read the directory you can see all the names.
Pick the next free one (and then test anyway).
-- 
Cameron Simpson <cs@zip.com.au>

The mark must be robust enough to survive MP3 transmission over the Internet,
but remain inaudible when played on the yet to be launched DVD-Audio players.
- the SDMI audio watermarkers literally ask for the impossible, since all
  audio compressors aim to pass _only_ human perceptible data
  http://www.newscientist.com/news/news.jsp?id=ns224836

[toc] | [prev] | [next] | [standalone]

#47221

[Multipart message — attachments visible in raw view] — view raw

Thanks.


On Thu, Jun 6, 2013 at 4:49 PM, Cameron Simpson <cs@zip.com.au> wrote:

> On 06Jun2013 03:50, Avnesh Shakya <avnesh.nitk@gmail.com> wrote:
> | hi,
> |    I am running a python script and it will create a file name like
> filename0.0.0 and If I run it again then new file will create one more like
> filename0.0.1...... my code is-
> |
> | i = 0
> | for i in range(1000):
> |     try:
> |         with open('filename%d.%d.%d.json'%(0,0,i,)): pass
> |         continue
> |     except IOError:
> |         dataFile = file('filename%d.%d.%d.json'%(0,0,i,), 'a+')
> |         break
> | But It will take more time after creating many files, So i want to store
> value of last var "i" in a variable so that when i run my script again then
> I can use it. for example-
> |                  my last created file is filename0.0.27 then it should
> store 27 in a variable and when i run again then new file should be created
> 0.0.28 according to last value "27", so that i could save time and it can
> create file fast..
> |
> | Please give me suggestion for it.. How is it possible?
>
> Write it to a file? Read the file next time the script runs?
>
> BTW, trying to open zillions of files is slow.
> But using listdir to read the directory you can see all the names.
> Pick the next free one (and then test anyway).
> --
> Cameron Simpson <cs@zip.com.au>
>
> The mark must be robust enough to survive MP3 transmission over the
> Internet,
> but remain inaudible when played on the yet to be launched DVD-Audio
> players.
> - the SDMI audio watermarkers literally ask for the impossible, since all
>   audio compressors aim to pass _only_ human perceptible data
>   http://www.newscientist.com/news/news.jsp?id=ns224836
> --
> http://mail.python.org/mailman/listinfo/python-list
>

[toc] | [prev] | [next] | [standalone]

#47222

On 06/06/2013 06:50 AM, Avnesh Shakya wrote:
> hi,
>     I am running a python script and it will create a file name like filename0.0.0 and If I run it again then new file will create one more like filename0.0.1...... my code is-
>
> i = 0

Redundant initialization of i.

> for i in range(1000):
>      try:
>          with open('filename%d.%d.%d.json'%(0,0,i,)): pass
>          continue
>      except IOError:
>          dataFile = file('filename%d.%d.%d.json'%(0,0,i,), 'a+')
>          break
> But It will take more time after creating many files, So i want to store value of last var "i" in a variable

There are no variables once the program ends.  You mean you want to 
store it in the file.  That's known as persistent storage, and in the 
general case you could use pickle or something like that.  But in your 
simple case, the easiest thing would be to simply write the last value 
of i out to a file in the same directory.

Then when your program starts, it opens that extra file and reads in the 
value of i.  And uses that for the starting value in the loop.

  so that when i run my script again then I can use it. for example-
>                   my last created file is filename0.0.27 then it should store 27 in a variable and when i run again then new file should be created 0.0.28 according to last value "27", so that i could save time and it can create file fast..
>
> Please give me suggestion for it.. How is it possible?
> Thanks
>

Incidentally, instead of opening each one, why not check its existence? 
  Should be quicker, and definitely clearer.

Entirely separate suggestion, since I dislike having extra housekeeping 
files that aren't logically necessary, and that might become out of synch :

If you're planning on having the files densely populated (meaning no 
gaps in the numbering), then you could use a binary search to find the 
last one.  Standard algorithm would converge with 10 existence checks if 
you have a limit of 1000 files.

-- 
DaveA

[toc] | [prev] | [next] | [standalone]

#47225

Avnesh Shakya writes:

> I am running a python script and it will create a file name like
> filename0.0.0 and If I run it again then new file will create one
> more like filename0.0.1...... my code is-
> 
> i = 0
> for i in range(1000):
>     try:
>         with open('filename%d.%d.%d.json'%(0,0,i,)): pass
>         continue
>     except IOError:
>         dataFile = file('filename%d.%d.%d.json'%(0,0,i,), 'a+')
>         break
>
> But It will take more time after creating many files, So i want to
> store value of last var "i" in a variable so that when i run my
> script again then I can use it. for example- my last created file is
> filename0.0.27 then it should store 27 in a variable and when i run
> again then new file should be created 0.0.28 according to last value
> "27", so that i could save time and it can create file fast..

You could get a list of all filenames that match the pattern. Extract
the last components as numbers, and add 1 to the maximum.

  i = 1 + max(int(name.split('.')[-1])
              for name in glob.glob('filename.0.0.*))

That assumes that there already is at least one such file and all such
files have a last component that can be parsed as an int. Take an
appropriate amount of care.

Or you could also create a file, say lastname.0.0.31, to track the
name, and when you find it there, create filename.0.0.32 and replace
lastname.0.0.32; panic if there is more than one lastname.0.0.*, or
fewer than one.

Or as above but track with nextname.0.0.31 to create filename.0.0.31
and replace the tracking name with nextname.0.0.32 for the next file.

Or save the number somewhere else.

[toc] | [prev] | [next] | [standalone]

#47233

On Thu, Jun 6, 2013 at 10:14 PM, Dave Angel <davea@davea.name> wrote:
> If you're planning on having the files densely populated (meaning no gaps in
> the numbering), then you could use a binary search to find the last one.
> Standard algorithm would converge with 10 existence checks if you have a
> limit of 1000 files.

Or, if you can dedicate a directory to those files, you could go even simpler:

dataFile = open('filename0.0.%d.json'%len(os.listdir()), 'w')

The number of files currently existing equals the number of the next file.

ChrisA

[toc] | [prev] | [next] | [standalone]

#47237

On 06/06/2013 16:37, Chris Angelico wrote:
> On Thu, Jun 6, 2013 at 10:14 PM, Dave Angel <davea@davea.name> wrote:
>> If you're planning on having the files densely populated (meaning no gaps in
>> the numbering), then you could use a binary search to find the last one.
>> Standard algorithm would converge with 10 existence checks if you have a
>> limit of 1000 files.
>
> Or, if you can dedicate a directory to those files, you could go even simpler:
>
> dataFile = open('filename0.0.%d.json'%len(os.listdir()), 'w')
>
> The number of files currently existing equals the number of the next file.
>
Assuming no gaps.

[toc] | [prev] | [next] | [standalone]

#47238

On Fri, Jun 7, 2013 at 1:54 AM, MRAB <python@mrabarnett.plus.com> wrote:
> On 06/06/2013 16:37, Chris Angelico wrote:
>>
>> On Thu, Jun 6, 2013 at 10:14 PM, Dave Angel <davea@davea.name> wrote:
>>>
>>> If you're planning on having the files densely populated (meaning no gaps
>>> in
>>> the numbering), then you could use a binary search to find the last one.
>>> Standard algorithm would converge with 10 existence checks if you have a
>>> limit of 1000 files.
>>
>>
>> Or, if you can dedicate a directory to those files, you could go even
>> simpler:
>>
>> dataFile = open('filename0.0.%d.json'%len(os.listdir()), 'w')
>>
>> The number of files currently existing equals the number of the next file.
>>
> Assuming no gaps.

Which is also a stated assumption of the binary search. The only
additional assumption of the file-count method is that there be no
other files in that directory.

ChrisA

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web