Path: csiph.com!x330-a1.tempe.blueboxinc.net!usenet.pasdenom.info!news.albasani.net!fu-berlin.de!uni-berlin.de!individual.net!not-for-mail From: Fredrik Jonson Newsgroups: comp.lang.java.programmer Subject: Re: JMS scalability question Date: 17 Nov 2011 08:11:13 GMT Lines: 68 Message-ID: References: X-Trace: individual.net p2oMJ3x13SwzL0rW7UW0ewtc3uF4CZvQUftfq/FRDB/zdM5eo= Cancel-Lock: sha1:BJ2VUs/d22yH3r3ft8WcyZORm3s= User-Agent: slrn/pre1.0.0-18 (Linux) Xref: x330-a1.tempe.blueboxinc.net comp.lang.java.programmer:9995 saxo123@gmx.de wrote: > let's say there were a situation like this: Some producers send > thousands of messages through a JMS system to some thousands > consumers. What is the best approach towards scalability? Your problem description is too generic to say anything about your performance. I'd claim a few thousands consumers shouldn't be a problem for a JMS broker worth its salt. On the other hand if those consumers are on the same jvm you'll now have thousands of threads fighting over local resourses. So the big question is what those consumers do with the payload? Are those thousands of consumers running on the same system, or jvm even? Then, do the business logic in the consumers hog network, memory, cpu time, database resources? Is the system able to cope with that kind of concurrent fight over system resourses? It might be a problem, it might not, but every system has its bottlenecks. What I'm trying to say is that just consuming lots of messages concurrently by itself isn't hard. It is when you try to do something interesting with those messages that you're likely to run into performance issuses. > 1) 1 JMS publish-subscriber channel with one dispatcher at the end > that takes every msg from the channel and propagates it to the > respective consumer out of those thousand consumers. Each endpoint consumer can consume directly from the jms. What purpose would in-between home grown dispatcher have? Consuming messages isn't expensive, and a dispatcher might turn out to be a bottleneck in the system. I'd call 1 a antipattern. > 2) 1 JMS publish-subscriber channel where every consumer peeks a newly > arrived msg and checks whether it is for him and takes it from the > queue if so. If you choose path 2, you'll want to look into message selectors and flagging each message with heades so consumers that should not grab the message doesn't even see the message. You do not want a jms consumer to reject a dispatched message when the message cannot be redelivered to the same consumer at a later time. Again a jms antipattern. I'd caution against message selectors when your consumers can predict which destination to poll. Using a dedicated destination - queue or topic name - for each message type is more efficient when the consumers doesn't have to consume several different message types. > 3) Many JMS publish-subscriber channels with one dispatcher for > everyone. I favour this one. > > 4) Thousand point-to-point connections. Sure not, I guess. I get the impression that your messages are unique and should only be consumed once by a single consumer? If that's correct you should use the point-to-point queue based jms-pattern in 4, rather than using pub-sub with topics in 3. You'll only want to use topics when the same message should be dispatched to multiple consumers. The typical example use for pub-sub is stock tickers, where many clients (consumers) want the same stock update (message) about many different stocks (topics). The typical example for point-to-point is a purchase system, where a new order (message) should be approved (consumed from the approval queue) once by a single accountant (consumer) before shipping (handed over to the order shipping queue). -- Fredrik Jonson