Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #90965 > unrolled thread

Re: Best approach to create humongous amount of files

Back to article view | Back to comp.lang.python

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Re: Best approach to create humongous amount of files Tim Chase <python.list@tim.thechases.com> - 2015-05-20 11:12 -0500

#90965 — Re: Best approach to create humongous amount of files

On 2015-05-20 17:59, Peter Otten wrote:
> Tim Chase wrote:
> >   wordlist[:] = [ # just lowercase all-alpha words
> >     word
> >     for word in wordlist
> >     if word.isalpha() and word.islower()
> >     ]
> 
> Just a quick reminder: if the data is user-provided you have to
> sanitize it:

Thus my sanitizing to isalpha()+islower() words in my sample.

> I expect that performance will be dominated by I/O; if that's
> correct the extra work of serializing the JSON should not do much
> harm.

I seem to recall that there was a change-over, that an older JSON
library was particularly slow, but that a later replacement sped that
up immensely.  So performance may depend heavily on which version
you're running.

[to the OP] But yes, if you're trusting unsanitized data, Peter's
suggestion would be the way to go.

-tkc

[toc] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web