Path: csiph.com!usenet.pasdenom.info!weretis.net!feeder4.news.weretis.net!eternal-september.org!feeder.eternal-september.org!mx05.eternal-september.org!.POSTED!not-for-mail From: aminer Newsgroups: comp.programming Subject: Re: Parallel archiver using my ParallelZlib is here... Date: Tue, 19 Feb 2013 10:59:26 -0800 Organization: A noiseless patient Spider Lines: 116 Message-ID: <5123CB8E.4030307@toto.ca> References: <0d594a6d-6353-4aea-97f3-4f8ac908c778@googlegroups.com> <5122D99A.1080006@toto.ca> <03289761-d848-4582-9209-aa7aadaedc13@googlegroups.com> Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Injection-Info: mx05.eternal-september.org; posting-host="8427fd05edc201bb6b1d125b80ca38e6"; logging-data="21560"; mail-complaints-to="abuse@eternal-september.org"; posting-account="U2FsdGVkX1+aOwngzrYFNvX+eo0MzIa3cRyNYvslkAY=" User-Agent: Mozilla/5.0 (Windows NT 6.0; WOW64; rv:17.0) Gecko/20130107 Thunderbird/17.0.2 In-Reply-To: <03289761-d848-4582-9209-aa7aadaedc13@googlegroups.com> Cancel-Lock: sha1:wgf6lzEw7Q1OpjgziT7chMJRfbw= Xref: csiph.com comp.programming:3044 On 2/19/2013 7:47 AM, bob wrote: > On Monday, February 18, 2013 4:46:32 PM UTC-6, aminer wrote: >> On 2/18/2013 8:56 AM, bob wrote: >> >>> On Friday, February 15, 2013 9:37:43 PM UTC-6, aminer wrote: >> >>>> Hello, >> >>>> >> >>>> >> >>>> >> >>>> I want to add an important thing... >> >>>> >> >>>> >> >>>> >> >>>> I have designed my parallel archiver 1.0 to be fault tolerant to power >> >>>> >> >>>> failures etc. >> >>>> >> >>>> >> >>>> >> >>>> You have to call the LoadIndex() method just after you create >> >>>> >> >>>> you object with the constructor and it will be fault tolerant. >> >>>> >> >>>> >> >>>> >> >>>> >> >>>> >> >>>> >> >>>> >> >>>> Thank you, >> >>>> >> >>>> Amine Moulay Ramdane. >> >>> >> >>> How much faster does it operate when you go from using 1 thread to using 2 threads? >> >> >> >> >> >> My parallel archiver is using Parallel Zlib so it's fast, so since >> >> Parrallel Zlib is fast , the P (parallel part) of the Amdahl equation is >> >> much smaller than with Parallel Bzip, so since it's using a lot the >> >> memory and the harddisk also, it gives 2.5x speed with 4 threads on 4 >> >> cores. And thats not so bad since we are speeding up the things 2.5x >> >> times, ParallelBzip is mush slower, so the P part is much bigger than with >> >> Parallel Zlib , and it gives over 3.4x speed with Parallel Bzip. >> >> >> >> >> >> >> >> Thank's, >> >> Amine Moulay Ramdane. > > How is the work of compressing something divided among the threads? > > Thanks. > Read this: "What you want instead, is to do what pigz does: divide the input into fixed-size blocks (with block size equal to 128 KiB), and have P threads that are each busily compressing a block. That is, pigz starts by reading P blocks and starting a compression thread on each block. It then waits for the first thread to finish, outputs its result, and then can reuse that thread to compress the (P+1)st block." http://www.cs.ucla.edu/classes/fall12/cs131/hw/hw3.html Amine Moulay Ramdane.