Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.java.programmer > #9995

Re: JMS scalability question

Show key headers only | View raw

saxo123@gmx.de wrote:

>  let's say there were a situation like this: Some producers send
>  thousands of messages through a JMS system to some thousands
>  consumers. What is the best approach towards scalability?

Your problem description is too generic to say anything about your
performance. I'd claim a few thousands consumers shouldn't be a problem for
a JMS broker worth its salt.

On the other hand if those consumers are on the same jvm you'll now have
thousands of threads fighting over local resourses.

So the big question is what those consumers do with the payload? Are those
thousands of consumers running on the same system, or jvm even?  Then, do
the business logic in the consumers hog network, memory, cpu time, database
resources? Is the system able to cope with that kind of concurrent fight
over system resourses? It might be a problem, it might not, but every system
has its bottlenecks.

What I'm trying to say is that just consuming lots of messages concurrently
by itself isn't hard. It is when you try to do something interesting with
those messages that you're likely to run into performance issuses.

>  1) 1 JMS publish-subscriber channel with one dispatcher at the end
>  that takes every msg from the channel and propagates it to the
>  respective consumer out of those thousand consumers.

Each endpoint consumer can consume directly from the jms. What purpose would
in-between home grown dispatcher have? Consuming messages isn't expensive,
and a dispatcher might turn out to be a bottleneck in the system. I'd call 1
a antipattern.

>  2) 1 JMS publish-subscriber channel where every consumer peeks a newly
>  arrived msg and checks whether it is for him and takes it from the
>  queue if so.

If you choose path 2, you'll want to look into message selectors and
flagging each message with heades so consumers that should not grab the
message doesn't even see the message. You do not want a jms consumer to
reject a dispatched message when the message cannot be redelivered to the
same consumer at a later time. Again a jms antipattern.

I'd caution against message selectors when your consumers can predict
which destination to poll. Using a dedicated destination - queue or topic
name - for each message type is more efficient when the consumers doesn't
have to consume several different message types.

>  3) Many JMS publish-subscriber channels with one dispatcher for
>  everyone. I favour this one.
>
>  4) Thousand point-to-point connections. Sure not, I guess.

I get the impression that your messages are unique and should only be
consumed once by a single consumer? If that's correct you should use the
point-to-point queue based jms-pattern in 4, rather than using pub-sub with
topics in 3. You'll only want to use topics when the same message should be
dispatched to multiple consumers.

The typical example use for pub-sub is stock tickers, where many clients
(consumers) want the same stock update (message) about many different stocks
(topics).  The typical example for point-to-point is a purchase system,
where a new order (message) should be approved (consumed from the approval
queue) once by a single accountant (consumer) before shipping (handed over
to the order shipping queue).

--
Fredrik Jonson

Back to comp.lang.java.programmer | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

JMS scalability question saxo123@gmx.de - 2011-11-15 02:07 -0800
  Re: JMS scalability question Roedy Green <see_website@mindprod.com.invalid> - 2011-11-16 06:21 -0800
    Re: JMS scalability question saxo123@gmx.de - 2011-11-16 09:35 -0800
      Re: JMS scalability question Lew <lewbloch@gmail.com> - 2011-11-16 10:30 -0800
        Re: JMS scalability question saxo123@gmx.de - 2011-11-16 14:25 -0800
          Re: JMS scalability question Lew <lewbloch@gmail.com> - 2011-11-16 17:18 -0800
            Re: JMS scalability question saxo123@gmx.de - 2011-11-16 22:59 -0800
              Re: JMS scalability question jlp <jlp@jlp.com> - 2011-11-17 08:41 +0100
                Re: JMS scalability question saxo123@gmx.de - 2011-11-17 01:00 -0800
      Re: JMS scalability question Roedy Green <see_website@mindprod.com.invalid> - 2011-11-16 16:16 -0800
  Re: JMS scalability question Fredrik Jonson <fredrik@jonson.org> - 2011-11-17 08:11 +0000
    Re: JMS scalability question saxo123@gmx.de - 2011-11-17 01:26 -0800
      Re: JMS scalability question Fredrik Jonson <fredrik@jonson.org> - 2011-11-17 21:01 +0000
        Re: JMS scalability question Saxo <saxo123@gmx.de> - 2011-11-17 23:27 -0800

csiph-web