Re: String concatenation - which is the fastest way ?

From	Stefan Behnel <stefan_ml@behnel.de>
Subject	Re: String concatenation - which is the fastest way ?
Date	2011-08-11 13:59 +0200
References	(1 earlier) <CAPTjJmrF0GcVs0onfoCHAbMe38b5iLXgFX1R7G_RXcKxjPH5wQ@mail.gmail.com> <20110810133146.GE5045@host.pgf.com.pl> <j1ub3q$65c$1@dough.gmane.org> <20110811064030.GB4990@host.pgf.com.pl> <CAPTjJmqiiKcT7Zg-XkbrQ4r-keU9+1Kj=+sEwa4xp==2EG-5aQ@mail.gmail.com>
Newsgroups	comp.lang.python
Message-ID	<mailman.2174.1313064009.1164.python-list@python.org> (permalink)

Show all headers | View raw

Chris Angelico, 11.08.2011 12:59:
> On Thu, Aug 11, 2011 at 7:40 AM,<przemolicc@poczta.fm>  wrote:
>> I am not a database developer so I don't want to change the whole process
>> of data flow between applications in my company. Another process is
>> reading this XML from particular Oracle table so I have to put the final XML there.
>
> I think you may be looking at a submission to
> http://www.thedailywtf.com/ soon. You seem to be working in a rather
> weird dataflow. :( Under the circumstances, you're probably going to
> want to go with the original ''.join() option.
>
>> This server has 256 GB of RAM so memory is not a problem.
>> Also the select which fetches the data is sorted. That is why I have to
>> carefully divide into subtasks and then merge it in correct order.
>
> There's no guarantee that all of that 256GB is available to you, of course.
>
> What may be the easiest way is to do the select in a single process,
> then partition it and use the Python multiprocessing module to split
> the job into several parts. Then you need only concatenate the handful
> of strings.

Or join them using an n-way merge.

Stefan

Back to comp.lang.python | Previous | Next | Find similar | Unroll thread

Thread

Re: String concatenation - which is the fastest way ? Stefan Behnel <stefan_ml@behnel.de> - 2011-08-11 13:59 +0200

csiph-web