Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.java.programmer > #22295 > unrolled thread

nThreads

Back to article view | Back to comp.lang.java.programmer

Contents

  nThreads bob smith <bob@coolfone.comze.com> - 2013-02-13 15:25 -0800
    Re: nThreads Joerg Meier <joergmmeier@arcor.de> - 2013-02-14 00:46 +0100
      Re: nThreads Roedy Green <see_website@mindprod.com.invalid> - 2013-02-14 03:43 -0800
      Re: nThreads Arne Vajhoej <arne@vajhoej.dk> - 2013-02-15 08:56 -0500
    Re: nThreads markspace <markspace@nospam.nospam> - 2013-02-13 16:06 -0800
      Re: nThreads Joerg Meier <joergmmeier@arcor.de> - 2013-02-14 01:46 +0100
        Re: nThreads Lew <lewbloch@gmail.com> - 2013-02-13 17:27 -0800
        Re: nThreads Robert Klemme <shortcutter@googlemail.com> - 2013-02-14 22:19 +0100
        Re: nThreads Arne Vajhoej <arne@vajhoej.dk> - 2013-02-15 08:59 -0500
    Re: nThreads Roedy Green <see_website@mindprod.com.invalid> - 2013-02-13 18:37 -0800
      Re: nThreads Arne Vajhoej <arne@vajhoej.dk> - 2013-02-15 09:01 -0500
    Re: nThreads Arved Sandstrom <asandstrom2@eastlink.ca> - 2013-02-14 07:07 -0400
    Re: nThreads Daniel Pitts <newsgroup.nospam@virtualinfinity.net> - 2013-02-14 09:55 -0800
    Re: nThreads Arne Vajhoej <arne@vajhoej.dk> - 2013-02-15 08:55 -0500

#22295 — nThreads

I'm looking at this method in the Executors class:

--------------------------------
newFixedThreadPool
public static ExecutorService newFixedThreadPool(int nThreads)

Creates a thread pool that reuses a fixed set of threads operating off a shared unbounded queue. If any thread terminates due to a failure during execution prior to shutdown, a new one will take its place if needed to execute subsequent tasks.

--------------------------------

How much is too much in terms of me telling it how many threads I want?  Is it okay to ask for 100 threads?  What about 256?

Thanks.

[toc] | [next] | [standalone]

#22296

On Wed, 13 Feb 2013 15:25:33 -0800 (PST), bob smith wrote:

> How much is too much in terms of me telling it how many threads I want?  Is it okay to ask for 100 threads?  What about 256?

While modern operating systems support thousands of threads, there are
really not many advantages in using more threads than you have CPUs/cores,
and the additional context switching of having more threads than cores
comes with its own performance cost.

Liebe Gruesse,
		Joerg

-- 
Ich lese meine Emails nicht, replies to Email bleiben also leider
ungelesen.

[toc] | [prev] | [next] | [standalone]

#22304

On Thu, 14 Feb 2013 00:46:50 +0100, Joerg Meier <joergmmeier@arcor.de>
wrote, quoted or indirectly quoted someone who said :

>While modern operating systems support thousands of threads, there are
>really not many advantages in using more threads than you have CPUs/cores,

It pays to have more threads than cores if your threads are probing
websites.  You want to have enough waiting to keep your port busy. You
can also do that in more efficient but complicated way with a single
thread that processes the next response in turn.

Your advice would apply for CPU intensive tasks.
-- 
Roedy Green Canadian Mind Products http://mindprod.com
The first 90% of the code accounts for the first 90% of the development time.
The remaining 10% of the code accounts for the other 90% of the development 
time. 
~ Tom Cargill  Ninety-ninety Law 

[toc] | [prev] | [next] | [standalone]

#22321

On 2/13/2013 6:46 PM, Joerg Meier wrote:
> On Wed, 13 Feb 2013 15:25:33 -0800 (PST), bob smith wrote:
>
>> How much is too much in terms of me telling it how many threads I want?  Is it okay to ask for 100 threads?  What about 256?
>
> While modern operating systems support thousands of threads, there are
> really not many advantages in using more threads than you have CPUs/cores,
> and the additional context switching of having more threads than cores
> comes with its own performance cost.

That is only for CPU intensive tasks.

If tasks are waiting then it is a very different story.

Arne

[toc] | [prev] | [next] | [standalone]

#22298

On 2/13/2013 3:25 PM, bob smith wrote:
> How much is too much in terms of me telling it how many threads I
> want?  Is it okay to ask for 100 threads?  What about 256?

It depends on your system and your application, and you should measure 
your system performance (and other metrics) to determine what is best 
for your particular case.  I'm sorry I don't know much about practical 
ways to go about those measurements.

However, 100 to 300 threads is trivial for most modern system.  Since 
most tasks will be IO-bound, it make sense to have lots of threads so 
some can be running while others wait for data to be returned.  (Sorry 
to contradict Mr. Meier but I'm pretty sure he's off base.)  Given a 
"typical" application I'd probably use 300 threads to start off and 
measure from there.

3000 is probably approaching some sort of upper limit on most "average" 
servers.  Don't neglect to measure actual running performance, but I 
don't think commodity hardware can handle that many practically.

[toc] | [prev] | [next] | [standalone]

#22299

On Wed, 13 Feb 2013 16:06:37 -0800, markspace wrote:

> However, 100 to 300 threads is trivial for most modern system.  Since 
> most tasks will be IO-bound, it make sense to have lots of threads so 
> some can be running while others wait for data to be returned.  (Sorry 
> to contradict Mr. Meier but I'm pretty sure he's off base.)  Given a 

Obviously, you can pretty much have Integer.MAX_VALUE threads if all of
them are waiting without negatively impacting your waiting performance -
that's not really a sensible way to look at that question ;)

I said, and stand by, that context switching is not free and is the cost of
having too many threads. Threads that sleep or otherwise wait, such as for
IO, don't typically cause much context switching.

That being said, I would expect the case of 100s of threads waiting for IO
probably an edge case (for use cases such as web servers), and the majority
of cases would have IO restricted to a small number of the threads. YMMV.

> 3000 is probably approaching some sort of upper limit on most "average" 
> servers.  Don't neglect to measure actual running performance, but I 
> don't think commodity hardware can handle that many practically.

We once ran (by accident) something that triggered 100,000 threads (+ GC
and whatever) on a normal Linux server with a quad core CPU and it still
worked (although much slower than with the intended 100). As far as I know,
neither Linux nor the hardware was anything unusual.

Liebe Gruesse,
		Joerg

-- 
Ich lese meine Emails nicht, replies to Email bleiben also leider
ungelesen.

[toc] | [prev] | [next] | [standalone]

#22300

Joerg Meier wrote:
> markspace wrote:
>> However, 100 to 300 threads is trivial for most modern system.  Since 
>> most tasks will be IO-bound, it make sense to have lots of threads so 
>> some can be running while others wait for data to be returned.  (Sorry 
>> to contradict Mr. Meier but I'm pretty sure he's off base.)  Given a 
> 
> Obviously, you can pretty much have Integer.MAX_VALUE threads if all of
> them are waiting without negatively impacting your waiting performance -
> that's not really a sensible way to look at that question ;)

Perhaps not, but see below.

> I said, and stand by, that context switching is not free and is the cost of
> having too many threads. Threads that sleep or otherwise wait, such as for
> IO, don't typically cause much context switching.

But how "not free" is it? Let's look at your experience in a minute.

> That being said, I would expect the case of 100s of threads waiting for IO
> probably an edge case (for use cases such as web servers), and the majority
> of cases would have IO restricted to a small number of the threads. YMMV.
> 
>> 3000 is probably approaching some sort of upper limit on most "average" 
>> servers.  Don't neglect to measure actual running performance, but I 
>> don't think commodity hardware can handle that many practically.
> 
> We once ran (by accident) something that triggered 100,000 threads (+ GC
> and whatever) on a normal Linux server with a quad core CPU and it still
> worked (although much slower than with the intended 100). As far as I know,
> neither Linux nor the hardware was anything unusual.

The question is how much slower, and whether 1,000 threads would have experienced
a severe slowdown for that setup. Or 10,000.

Quite often resource limits show up as a discontinuity in the scaling graph. Service time 
degrades gracefully up to some n, then catastrophically above that.

In your case there might have been such a knee in that 10^3- 10^4 range. The numbers 
would tell.

And that's the point for the OP. You cannot know ahead of time what the right number is, 
really. You need to measure, under realistic loads. (And be pretty good at simulating "realistic".)

Notice that answers here are in terms of "if you're I/O bound", "if you're waiting", "if this", "if that". 

You need to know which ifs apply to your case.

-- 
Lew

[toc] | [prev] | [next] | [standalone]

#22311

On 14.02.2013 01:46, Joerg Meier wrote:
> On Wed, 13 Feb 2013 16:06:37 -0800, markspace wrote:
>
>> However, 100 to 300 threads is trivial for most modern system.  Since
>> most tasks will be IO-bound, it make sense to have lots of threads so
>> some can be running while others wait for data to be returned.  (Sorry
>> to contradict Mr. Meier but I'm pretty sure he's off base.)  Given a
>
> Obviously, you can pretty much have Integer.MAX_VALUE threads if all of
> them are waiting without negatively impacting your waiting performance -
> that's not really a sensible way to look at that question ;)

Obviously.

> I said, and stand by, that context switching is not free and is the cost of
> having too many threads.

Nobody questioned that.

> Threads that sleep or otherwise wait, such as for
> IO, don't typically cause much context switching.

Yes, but in a pool with a fixed upper limit of threads they consume a 
slot while sitting there doing nothing (and causing no CPU cost).  In 
this case a core will be idle if you limit pool size to number of cores. 
  If the application is doing IO (or anything else which might cause 
intermediate blocking) you are unnecessarily restricting throughput of 
the application if you take number of cores (or threads as they are 
called in some architectures) as limit.  What the best limit is 
obviously depends on the nature of the application.

> We once ran (by accident) something that triggered 100,000 threads (+ GC
> and whatever) on a normal Linux server with a quad core CPU and it still
> worked (although much slower than with the intended 100). As far as I know,
> neither Linux nor the hardware was anything unusual.

There you go.

Cheers

	robert

-- 
remember.guy do |as, often| as.you_can - without end
http://blog.rubybestpractices.com/

[toc] | [prev] | [next] | [standalone]

#22322

On 2/13/2013 7:46 PM, Joerg Meier wrote:
> On Wed, 13 Feb 2013 16:06:37 -0800, markspace wrote:
>> However, 100 to 300 threads is trivial for most modern system.  Since
>> most tasks will be IO-bound, it make sense to have lots of threads so
>> some can be running while others wait for data to be returned.  (Sorry
>> to contradict Mr. Meier but I'm pretty sure he's off base.)  Given a
>
> Obviously, you can pretty much have Integer.MAX_VALUE threads if all of
> them are waiting without negatively impacting your waiting performance -
> that's not really a sensible way to look at that question ;)
>
> I said, and stand by, that context switching is not free and is the cost of
> having too many threads. Threads that sleep or otherwise wait, such as for
> IO, don't typically cause much context switching.
>
> That being said, I would expect the case of 100s of threads waiting for IO
> probably an edge case (for use cases such as web servers), and the majority
> of cases would have IO restricted to a small number of the threads. YMMV.

I would expect >95% of Java developers to work in Java EE space and
<5% of Java developers to work outside Java EE space.

In Java EE space you do have threads that are waiting and thread pools
to manage threads (both for web container and EJB container).

So I would reverse what is edge and what is normal.

Arne

[toc] | [prev] | [next] | [standalone]

#22301

On Wed, 13 Feb 2013 15:25:33 -0800 (PST), bob smith
<bob@coolfone.comze.com> wrote, quoted or indirectly quoted someone
who said :

>How much is too much in terms of me telling it how many threads I want?  Is=
> it okay to ask for 100 threads?  What about 256?

The last time I looked, each thread ate up 1MB.

I think the best thing to do is run some benchmarks.  It will depend
on how much RAM you have, how many CPU's how much time each thread
spends sleeping.  It also depends on how busy the machine is with
other work.

See http://mindprod.com/project/tweakable.html for a proposal to solve
this class of problem.
-- 
Roedy Green Canadian Mind Products http://mindprod.com
The first 90% of the code accounts for the first 90% of the development time.
The remaining 10% of the code accounts for the other 90% of the development 
time. 
~ Tom Cargill  Ninety-ninety Law 

[toc] | [prev] | [next] | [standalone]

#22323

On 2/13/2013 9:37 PM, Roedy Green wrote:
> On Wed, 13 Feb 2013 15:25:33 -0800 (PST), bob smith
> <bob@coolfone.comze.com> wrote, quoted or indirectly quoted someone
> who said :
>
>> How much is too much in terms of me telling it how many threads I want?  Is=
>> it okay to ask for 100 threads?  What about 256?
>
> The last time I looked, each thread ate up 1MB.

That is typical configurable.

And the default depends on OS, Java vendor, versions etc..

Arne

[toc] | [prev] | [next] | [standalone]

#22303

On 02/13/2013 07:25 PM, bob smith wrote:
> I'm looking at this method in the Executors class:
>
> --------------------------------
> newFixedThreadPool
> public static ExecutorService newFixedThreadPool(int nThreads)
>
> Creates a thread pool that reuses a fixed set of threads operating off a shared unbounded queue. If any thread terminates due to a failure during execution prior to shutdown, a new one will take its place if needed to execute subsequent tasks.
>
> --------------------------------
>
> How much is too much in terms of me telling it how many threads I want?  Is it okay to ask for 100 threads?  What about 256?
>
> Thanks.
>
Just a cautionary, and that's not saying you're ignoring this aspect: 
the question "how many" in terms of units of concurrent execution is a 
question that should be answered by looking at logic requirements first, 
API and platform performance capabilities second.

For example, given N units of work, described by tasks (and ultimately 
executed by threads under the control of some Executor, say), the 
business logic for one particular use case might require you to process 
these tasks 1 by 1, in some order, in another use case the business 
logic might dictate that you have to process in groups of M < N, and in 
yet another use case the business logic places no _logical_ bound on how 
many tasks can be executed at the same time.

Also, consider whether you have a concurrent problem or a parallel 
problem. If your application is doing a bunch of things, among them 
handling continual requests (user or external system inputs), that's 
really concurrency. But if you've got one problem that benefits from 
being split up, that's parallelism and you may want to look at ForkJoin 
instead.

Task design also is driven by business logic and comes before choosing 
things like thread pools and sizes thereof. Task granularity in a 
concurrent problem should be whatever logic dictates. An important 
consideration here too is synchronous versus asynchronous, and how you 
plan to get task results (if any). Runnables vs. Callables, Futures, etc.

Addressing number of threads for pools, a bit, a lot of small tasks can 
be worse than a moderate number of medium-sized ones. As others have 
said, only performance testing will tell. But don't artificially chop up 
your logic in a _concurrent_ problem thinking that you can process a 
passel of these things with umpteen threads, because that may backfire.

Overall point being, about the last thing I'd be worrying about is how 
many threads to assign to a fixed size thread pool. *Eventually* I'd 
think about it, sure. After lots of testing.

AHS

[toc] | [prev] | [next] | [standalone]

#22306

On 2/13/13 3:25 PM, bob smith wrote:
> I'm looking at this method in the Executors class:
>
> --------------------------------
> newFixedThreadPool
> public static ExecutorService newFixedThreadPool(int nThreads)
>
> Creates a thread pool that reuses a fixed set of threads operating off a shared unbounded queue. If any thread terminates due to a failure during execution prior to shutdown, a new one will take its place if needed to execute subsequent tasks.
>
> --------------------------------
>
> How much is too much in terms of me telling it how many threads I want?  Is it okay to ask for 100 threads?  What about 256?
>
> Thanks.
>
>
As you have read in the rest of this thread, the answer is "it depends".

You can ask for many many threads (thousands). The real question is "how 
many threads *should* I for task X?"

Which, of course, depends on the task.

A rule of thumb I use for CPU bound tasks (ray-tracing for example) is 
N+1 where N is the number of CPU cores.  The +1 is to have an extra 
thread available to pick up slack if any one of the other threads ends 
up blocked by something, such as synchronization, paging, etc...

For IO bound tasks, I tend not to use a fixed pool, but use a pool that 
automatically adjusts its size to meet demand. You also need to be aware 
of the effect that you might have down-stream.  If your IO is all local 
disk access on a spinning media (eg, non SDD hard disk), too many 
threads *might* cause undue head shifting, slowing down the overall 
operation.

For Memory (or other allocated resource) Intensive tasks, you want to 
have an upper bound of threads that prevents any thread from failing due 
to running out of resources.

For mixed tasks (some tasks which block on IO for a portion of time, 
then process the result in a CPU intensive manor), you'll have to make a 
guess and then fine-tune to each machine.  Provide sensible defaults but 
let the end-user configure it.

As a matter of fact, many applications have configuration options for 
thread-pool size.  Let the end-user (or administrator at least) 
decide/figure out what is best for their particular use-cases, 
workflows, and systems.

HTH,
Daniel.

[toc] | [prev] | [next] | [standalone]

#22320

On 2/13/2013 6:25 PM, bob smith wrote:
> I'm looking at this method in the Executors class:
>
> --------------------------------
> newFixedThreadPool
> public static ExecutorService newFixedThreadPool(int nThreads)
>
> Creates a thread pool that reuses a fixed set of threads operating off a shared unbounded queue. If any thread terminates due to a failure during execution prior to shutdown, a new one will take its place if needed to execute subsequent tasks.
>
> --------------------------------
>
> How much is too much in terms of me telling it how many threads I want?  Is it okay to ask for 100 threads?  What about 256?

It depends on the problem domain.

With CPU intensive tasks you do not want many threads. Somewhere
1-2 x number of thread contexts in CPU (for x86-64 that is number of
cores without HT and twice number of cores with HT).

With tasks that mostly wait, then you may want a lot of threads. But at
some point the overhead of having many threads becomes significant. I
would try and keep number of threads <500 on smaller systems.

Arne

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.java.programmer

csiph-web