Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.programming.threads > #2683 > unrolled thread

A more scalable Parallel Varfiler and a more scalable Parallel Hashlist

Back to article view | Back to comp.programming.threads

Contents

  A more scalable Parallel Varfiler and a more scalable Parallel Hashlist Ramine <ramine@1.1> - 2014-11-16 15:30 -0800
    Re: A more scalable Parallel Varfiler and a more scalable Parallel Hashlist Ramine <ramine@1.1> - 2014-11-16 15:46 -0800
      Re: A more scalable Parallel Varfiler and a more scalable Parallel Hashlist Tom <mail2devnull@web.de> - 2014-11-24 15:20 +0100
      Re: A more scalable Parallel Varfiler and a more scalable Parallel Hashlist Tom <mail2devnull@web.de> - 2014-11-24 15:22 +0100

#2683 — A more scalable Parallel Varfiler and a more scalable Parallel Hashlist

Hello,


I have implemented a more scalable Parallel varfiler that uses my 
scalable RWLock in each bucket of the Parallel hashtable, and it can be 
used also as a parallel hashtable and that uses ParallelHashList 
(Parallel Hashtable) with O(1) best case and when there is collisions in 
the hashtable, the average time complexity will be O(log(n)) in each 
bucket were the collisions happened. The parallel Hahstable uses lock 
striping and my scalable RWLock in each bucket , this allows multiple 
threads to write and read concurently. also ParallelHashList maintains 
an independant counter , that counts the number of entries , for each 
segment of the hashtable and uses a lock for each counter, this is also 
for better scalability.

ParallelVarFiler is a Parallel HashTable that can be saved automatically 
or manually to a file or to a stream or to a string and it can be 
restored from  a file or from a stream or from a string back to the 
Hashtable in memory, and it's fault tolerant to power failures etc., 
When you use a file name in the constructor, many readers and one writer 
can proceed concurrently. But when you pass an empty file name to the 
constructor, many writers and many readers can proceed concurrently in 
memory. and when you read a data item it only use the parallel hashtable 
in memory, hardisk is not used.

You can collect different types of variables and save them into one file 
or stream. TParallelVarFiler reads and writes on files, streams and 
Strings. You can collect Strings, Integers and Dates, in fact everything 
that can be stored in a variant. In addition you can collect Streams.

You can use also ParallelVarFiler to send parameters over network. It 
can be used for example to share data by any IPC mechanism.


An important note:


I have used my scalable RWLock in each bucket so that it scales better
on multicores, but to scale better on multicores please use a number
of rwlocks equal to 2000 in the constructor.

My Parallel Varfiler is now compatible with Delphi 7 to 2007 and
with FreePascal and Lazarus and with all the Delphi XE versions,
please download the zipfile called "parallelvarfiler_srw_xe.zip" for the 
Delphi XE versions , and please download the zipfile called 
"parallelvarfiler_srw.zip" for the Delphi7 to 2007 and for the 
FreePascal and Lazarus.


You can download my more scalable Parallel Varfiler from:

https://sites.google.com/site/aminer68/more-scalable-parallel-varfiler


I have also implemented a more scalable Parallel Hashlist that scales
better , please download it from:

https://sites.google.com/site/aminer68/more-scalable-parallel-hashlist



Thank you,
Amine Moulay Ramdane.

[toc] | [next] | [standalone]

#2684

Hello,


I have done some benchmarks on multicores with an x86 Quadcore processor 
and Parallel Hashlist (a parallel hashtable) and my Parallel Varfiler 
scale in-memory to 3.9x (please look at the test2.pas example inside the 
zipfile and test them yourself) , so my Parallel Varfiler and  Parallel 
Hashlist are great tools ! so i hope you will enjoy my inventions !


You can download them from:


https://sites.google.com/site/aminer68/more-scalable-parallel-varfiler

and from:

https://sites.google.com/site/aminer68/more-scalable-parallel-hashlist



Thank you,
Amine Moulay Ramdane.

[toc] | [prev] | [next] | [standalone]

#2707

the only one who responds to or comments on your posts is yourself. Does this tell you anything about the usefulness of your posts? or work? or life?

[toc] | [prev] | [next] | [standalone]

#2708

the only one who responds to or comments on your posts is yourself. Does this tell you anything about the usefulness of your posts? or work? or life?

[toc] | [prev] | [standalone]

Back to top | Article view | comp.programming.threads

csiph-web