Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > comp.programming > #3029 > unrolled thread
| Started by | aminer <aminer@toto.ca> |
|---|---|
| First post | 2013-02-15 22:41 -0800 |
| Last post | 2013-02-20 08:28 -0800 |
| Articles | 6 — 2 participants |
Back to article view | Back to comp.programming
This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by
below is the oldest one visible, not the original post.
Re: Parallel archiver using my ParallelZlib is here... aminer <aminer@toto.ca> - 2013-02-15 22:41 -0800
Re: Parallel archiver using my ParallelZlib is here... bob <bob@coolfone.comze.com> - 2013-02-18 08:56 -0800
Re: Parallel archiver using my ParallelZlib is here... aminer <aminer@toto.ca> - 2013-02-18 17:47 -0800
Re: Parallel archiver using my ParallelZlib is here... bob <bob@coolfone.comze.com> - 2013-02-19 07:47 -0800
Re: Parallel archiver using my ParallelZlib is here... aminer <aminer@toto.ca> - 2013-02-19 10:59 -0800
Re: Parallel archiver using my ParallelZlib is here... bob <bob@coolfone.comze.com> - 2013-02-20 08:28 -0800
| From | aminer <aminer@toto.ca> |
|---|---|
| Date | 2013-02-15 22:41 -0800 |
| Subject | Re: Parallel archiver using my ParallelZlib is here... |
| Message-ID | <kfmusn$akh$2@dont-email.me> |
Hello, I want to add an important thing... I have designed my parallel archiver 1.0 to be fault tolerant to power failures etc. You have to call the LoadIndex() method just after you create you object with the constructor and it will be fault tolerant. Thank you, Amine Moulay Ramdane.
[toc] | [next] | [standalone]
| From | bob <bob@coolfone.comze.com> |
|---|---|
| Date | 2013-02-18 08:56 -0800 |
| Message-ID | <0d594a6d-6353-4aea-97f3-4f8ac908c778@googlegroups.com> |
| In reply to | #3029 |
On Friday, February 15, 2013 9:37:43 PM UTC-6, aminer wrote: > Hello, > > > > I want to add an important thing... > > > > I have designed my parallel archiver 1.0 to be fault tolerant to power > > failures etc. > > > > You have to call the LoadIndex() method just after you create > > you object with the constructor and it will be fault tolerant. > > > > > > > > Thank you, > > Amine Moulay Ramdane. How much faster does it operate when you go from using 1 thread to using 2 threads?
[toc] | [prev] | [next] | [standalone]
| From | aminer <aminer@toto.ca> |
|---|---|
| Date | 2013-02-18 17:47 -0800 |
| Message-ID | <5122D99A.1080006@toto.ca> |
| In reply to | #3037 |
On 2/18/2013 8:56 AM, bob wrote: > On Friday, February 15, 2013 9:37:43 PM UTC-6, aminer wrote: >> Hello, >> >> >> >> I want to add an important thing... >> >> >> >> I have designed my parallel archiver 1.0 to be fault tolerant to power >> >> failures etc. >> >> >> >> You have to call the LoadIndex() method just after you create >> >> you object with the constructor and it will be fault tolerant. >> >> >> >> >> >> >> >> Thank you, >> >> Amine Moulay Ramdane. > > How much faster does it operate when you go from using 1 thread to using 2 threads? My parallel archiver is using Parallel Zlib so it's fast, so since Parrallel Zlib is fast , the P (parallel part) of the Amdahl equation is much smaller than with Parallel Bzip, so since it's using a lot the memory and the harddisk also, it gives 2.5x speed with 4 threads on 4 cores. And thats not so bad since we are speeding up the things 2.5x times, ParallelBzip is mush slower, so the P part is much bigger than with Parallel Zlib , and it gives over 3.4x speed with Parallel Bzip. Thank's, Amine Moulay Ramdane.
[toc] | [prev] | [next] | [standalone]
| From | bob <bob@coolfone.comze.com> |
|---|---|
| Date | 2013-02-19 07:47 -0800 |
| Message-ID | <03289761-d848-4582-9209-aa7aadaedc13@googlegroups.com> |
| In reply to | #3039 |
On Monday, February 18, 2013 4:46:32 PM UTC-6, aminer wrote: > On 2/18/2013 8:56 AM, bob wrote: > > > On Friday, February 15, 2013 9:37:43 PM UTC-6, aminer wrote: > > >> Hello, > > >> > > >> > > >> > > >> I want to add an important thing... > > >> > > >> > > >> > > >> I have designed my parallel archiver 1.0 to be fault tolerant to power > > >> > > >> failures etc. > > >> > > >> > > >> > > >> You have to call the LoadIndex() method just after you create > > >> > > >> you object with the constructor and it will be fault tolerant. > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> Thank you, > > >> > > >> Amine Moulay Ramdane. > > > > > > How much faster does it operate when you go from using 1 thread to using 2 threads? > > > > > > My parallel archiver is using Parallel Zlib so it's fast, so since > > Parrallel Zlib is fast , the P (parallel part) of the Amdahl equation is > > much smaller than with Parallel Bzip, so since it's using a lot the > > memory and the harddisk also, it gives 2.5x speed with 4 threads on 4 > > cores. And thats not so bad since we are speeding up the things 2.5x > > times, ParallelBzip is mush slower, so the P part is much bigger than with > > Parallel Zlib , and it gives over 3.4x speed with Parallel Bzip. > > > > > > > > Thank's, > > Amine Moulay Ramdane. How is the work of compressing something divided among the threads? Thanks.
[toc] | [prev] | [next] | [standalone]
| From | aminer <aminer@toto.ca> |
|---|---|
| Date | 2013-02-19 10:59 -0800 |
| Message-ID | <5123CB8E.4030307@toto.ca> |
| In reply to | #3043 |
On 2/19/2013 7:47 AM, bob wrote: > On Monday, February 18, 2013 4:46:32 PM UTC-6, aminer wrote: >> On 2/18/2013 8:56 AM, bob wrote: >> >>> On Friday, February 15, 2013 9:37:43 PM UTC-6, aminer wrote: >> >>>> Hello, >> >>>> >> >>>> >> >>>> >> >>>> I want to add an important thing... >> >>>> >> >>>> >> >>>> >> >>>> I have designed my parallel archiver 1.0 to be fault tolerant to power >> >>>> >> >>>> failures etc. >> >>>> >> >>>> >> >>>> >> >>>> You have to call the LoadIndex() method just after you create >> >>>> >> >>>> you object with the constructor and it will be fault tolerant. >> >>>> >> >>>> >> >>>> >> >>>> >> >>>> >> >>>> >> >>>> >> >>>> Thank you, >> >>>> >> >>>> Amine Moulay Ramdane. >> >>> >> >>> How much faster does it operate when you go from using 1 thread to using 2 threads? >> >> >> >> >> >> My parallel archiver is using Parallel Zlib so it's fast, so since >> >> Parrallel Zlib is fast , the P (parallel part) of the Amdahl equation is >> >> much smaller than with Parallel Bzip, so since it's using a lot the >> >> memory and the harddisk also, it gives 2.5x speed with 4 threads on 4 >> >> cores. And thats not so bad since we are speeding up the things 2.5x >> >> times, ParallelBzip is mush slower, so the P part is much bigger than with >> >> Parallel Zlib , and it gives over 3.4x speed with Parallel Bzip. >> >> >> >> >> >> >> >> Thank's, >> >> Amine Moulay Ramdane. > > How is the work of compressing something divided among the threads? > > Thanks. > Read this: "What you want instead, is to do what pigz does: divide the input into fixed-size blocks (with block size equal to 128 KiB), and have P threads that are each busily compressing a block. That is, pigz starts by reading P blocks and starting a compression thread on each block. It then waits for the first thread to finish, outputs its result, and then can reuse that thread to compress the (P+1)st block." http://www.cs.ucla.edu/classes/fall12/cs131/hw/hw3.html Amine Moulay Ramdane.
[toc] | [prev] | [next] | [standalone]
| From | bob <bob@coolfone.comze.com> |
|---|---|
| Date | 2013-02-20 08:28 -0800 |
| Message-ID | <c7393f2e-6630-4490-ad9d-2f752fd0817e@googlegroups.com> |
| In reply to | #3044 |
On Tuesday, February 19, 2013 9:59:31 AM UTC-6, aminer wrote: > On 2/19/2013 7:47 AM, bob wrote: > > > On Monday, February 18, 2013 4:46:32 PM UTC-6, aminer wrote: > > >> On 2/18/2013 8:56 AM, bob wrote: > > >> > > >>> On Friday, February 15, 2013 9:37:43 PM UTC-6, aminer wrote: > > >> > > >>>> Hello, > > >> > > >>>> > > >> > > >>>> > > >> > > >>>> > > >> > > >>>> I want to add an important thing... > > >> > > >>>> > > >> > > >>>> > > >> > > >>>> > > >> > > >>>> I have designed my parallel archiver 1.0 to be fault tolerant to power > > >> > > >>>> > > >> > > >>>> failures etc. > > >> > > >>>> > > >> > > >>>> > > >> > > >>>> > > >> > > >>>> You have to call the LoadIndex() method just after you create > > >> > > >>>> > > >> > > >>>> you object with the constructor and it will be fault tolerant. > > >> > > >>>> > > >> > > >>>> > > >> > > >>>> > > >> > > >>>> > > >> > > >>>> > > >> > > >>>> > > >> > > >>>> > > >> > > >>>> Thank you, > > >> > > >>>> > > >> > > >>>> Amine Moulay Ramdane. > > >> > > >>> > > >> > > >>> How much faster does it operate when you go from using 1 thread to using 2 threads? > > >> > > >> > > >> > > >> > > >> > > >> My parallel archiver is using Parallel Zlib so it's fast, so since > > >> > > >> Parrallel Zlib is fast , the P (parallel part) of the Amdahl equation is > > >> > > >> much smaller than with Parallel Bzip, so since it's using a lot the > > >> > > >> memory and the harddisk also, it gives 2.5x speed with 4 threads on 4 > > >> > > >> cores. And thats not so bad since we are speeding up the things 2.5x > > >> > > >> times, ParallelBzip is mush slower, so the P part is much bigger than with > > >> > > >> Parallel Zlib , and it gives over 3.4x speed with Parallel Bzip. > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> Thank's, > > >> > > >> Amine Moulay Ramdane. > > > > > > How is the work of compressing something divided among the threads? > > > > > > Thanks. > > > > > > > > > Read this: > > > > > > "What you want instead, is to do what pigz does: divide the input into > > fixed-size blocks (with block size equal to 128 KiB), and have P threads > > that are each busily compressing a block. That is, pigz starts by > > reading P blocks and starting a compression thread on each block. It > > then waits for the first thread to finish, outputs its result, and then > > can reuse that thread to compress the (P+1)st block." > > > > http://www.cs.ucla.edu/classes/fall12/cs131/hw/hw3.html > > > > > > > > Amine Moulay Ramdane. Are all of those 128 kibibyte blocks compressed independently of one another? For instance, is the 100th block completely independent from the 1st block?
[toc] | [prev] | [standalone]
Back to top | Article view | comp.programming
csiph-web