Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.databases > #45 > unrolled thread

Re: Need suggestions on transaction performance comparison

Back to article view | Back to comp.databases

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Re: Need suggestions on transaction performance comparison paul c <anonymous@not-for-mail.invalid> - 2011-04-07 07:52 -0700
    Re: Need suggestions on transaction performance comparison Keve Nagy <keve@see.my.sig> - 2011-04-27 11:42 +0200
      Re: Need suggestions on transaction performance comparison paul c <toledobythesea@gmail.com> - 2011-05-10 06:49 -0700

#45 — Re: Need suggestions on transaction performance comparison

On 20/03/2011 6:13 AM, Keve Nagy wrote:
>
> Hello Everyone,
> I would like to compare a couple of free (as in Richard Stallman)
> database engines for a study, evaluating their suitability for one of my
> university projects.
> The subjects are Firebird, SQLite, MySQL and PostgreSQL. All installed
> on the same hardware and OS, but having only one of them active at a time.
> One of the hypothetical scenarios is a Parliamentary Election's Day,
> when millions of votes (think of the total population of a country) are
> submitted during a fairly limited amount of time (e.g. 12 hours). Each
> vote corresponds to a single INSERT statement in its own transaction,
> many of which happens simultaneously.
>
> My plan is to run some stress-tests, simulating such a scenario with a
> random or changing load of concurrent transactions. And I am hoping to
> measure the DB performance, hopefully feed the results to something like
> gnuplot to have charts for illustration.
> I have difficulties figuring out how to do this. Running such test on a
> single database engine already seems quite a challenge, let alone doing
> it on four different engines.
>
> I am looking for suggestions or pointers on ways to perform concurrent
> transactions on the above mentioned engines, and ways to measure DB
> performance during the tests. If you know of a tool that can be used, or
> a method to simulate such load, or sample source code that does such
> thing, or an article discussing this topic, please let me know!
>
> Regards,

Hard to imagine a single system (not to mention the communication 
hardware) for that.  1) Do the math, say 50 million votes over 12 hours, 
that would be over one thousand 'insert transactions' per second on 
average.  The peaks would likely be several or many times that.  If 
there are more elected positions per voter than just a single 
parliamentary seat, multiply some more.  2) Look at all the plausible 
requirements, not just the population.  Eg., factors like how to size 
recovery logs or not wanting to have to arrange a second election if the 
single system fails, not to mention the arguments amongst the second set 
of losers.  Some things aren't naturally centralized, in most western 
countries votes are counted on a local basis, eg. per riding.  That 
might result in an average load of only 100 transactions per second and 
several hundreds peak.  But in most (so-called) democracies the 
candidates will have their own scrutineers and they'll want either paper 
ballots or paper output from the voting systems.  It's hard to imagine a 
computer system that can do a judicial re-count.


I'm not up on current hardware but I'd say that besides doing the paper 
analysis of the stuff above you could reasonably limit your stress-tests 
to a much smaller scale.

[toc] | [next] | [standalone]

#67

> Hard to imagine a single system (not to mention the communication
> hardware) for that. 1) Do the math, say 50 million votes over 12 hours,
> that would be over one thousand 'insert transactions' per second on
> average. The peaks would likely be several or many times that. If there
> are more elected positions per voter than just a single parliamentary
> seat, multiply some more. 2) Look at all the plausible requirements, not
> just the population. Eg., factors like how to size recovery logs or not
> wanting to have to arrange a second election if the single system fails,
> not to mention the arguments amongst the second set of losers. Some
> things aren't naturally centralized, in most western countries votes are
> counted on a local basis, eg. per riding. That might result in an
> average load of only 100 transactions per second and several hundreds
> peak. But in most (so-called) democracies the candidates will have their
> own scrutineers and they'll want either paper ballots or paper output
> from the voting systems. It's hard to imagine a computer system that can
> do a judicial re-count.
>
>
> I'm not up on current hardware but I'd say that besides doing the paper
> analysis of the stuff above you could reasonably limit your stress-tests
> to a much smaller scale.

Thanks for pointing these out, Paul!
Even though I was aware of these numbers, I didn't really think through 
their total effect the way you just highlighted. I have already amended 
the test plan accordingly, considering only a scaled-down version of the 
task. Having the election broken down to regions is a perfect example.

In the meantime I have also made some progress in the implememtation of 
the load test. I concluded that Perl DBI can be my best friend here, 
allowing me to construct a tool that spawns and inserts data-records as 
required. And it can be run on virtually any O/S, and against any of my 
target database types.

Where I still don't really have an answer is the measurement part.
I don't see how to measure each database's performance of handling the 
INSERTs. Can't quite grab what to measure, how I could measure that, and 
how to feed it into something like gnuplot to get comparable graphs.

Thoughts and pointers on such technical details would be more than welcome!


Regards,
-- 
Keve Nagy * Debrecen * Hungary
keve(at)mail(dot)poliod(dot)hu

[toc] | [prev] | [next] | [standalone]

#71

On Apr 27, 2:42 am, Keve Nagy <k...@see.my.sig> wrote:
...
> Even though I was aware of these numbers, I didn't really think through
> their total effect the way you just highlighted. ...

> ... Can't quite grab what to measure ...
> ...

I don't know why you acknowledge Jasen B's suggestions (from more than
a month ago) and then ignore them.

> Thoughts and pointers on such technical details would be more than welcome!

Sounds like the assignment is late.  From what was written, technical
advice seems beside the point.  Ie., you have been given a measurement
problem but after all this time it seems you haven 't decided what to
measure.  That would make tool technique moot.  It would have been
better to try to run your tests without any measurements at all - that
experience might have prompted you to refine them or try Jasen's
metrics without further ado or even consider a more regimented field.

[toc] | [prev] | [standalone]

Back to top | Article view | comp.databases

csiph-web