Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #11264 > unrolled thread

Re: String concatenation - which is the fastest way ?

Back to article view | Back to comp.lang.python

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Re: String concatenation - which is the fastest way ? Stefan Behnel <stefan_ml@behnel.de> - 2011-08-12 11:25 +0200

#11264 — Re: String concatenation - which is the fastest way ?

przemolicc@poczta.fm, 11.08.2011 16:39:
> On Thu, Aug 11, 2011 at 02:48:43PM +0100, Chris Angelico wrote:
>> On Thu, Aug 11, 2011 at 2:46 PM,<przemolicc@poczta.fm>  wrote:
>>> This is the way I am going to use.
>>> But what is the best data type to hold so many rows and then operate on them ?
>>>
>>
>> List of strings. Take it straight from your Oracle interface and work
>> with it directly.
>
> Can I use this list in the following way ?
> subprocess_1 - run on list between 1 and 10000
> subprocess_2 - run on list between 10001 and 20000
> subprocess_3 - run on list between 20001 and 30000
> etc
> ...

Sure. Just read the data as it comes in from the database and fill up a 
chunk, then hand that on to a process. You can also distribute it in 
smaller packets, just check what size gives the best throughput.

Still, I'd give each work parcel a number and then reorder the results 
while collecting them, that allows you to vary the chunk size and the 
process size independently, without having to wait for a process that 
happens to take longer.

Stefan

[toc] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web