Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.c++ > #15460 > unrolled thread

Is there (semi-) portable IO-Completion-Port-like functionality in C++ land?

Back to article view | Back to comp.lang.c++

Contents

  Is there (semi-) portable IO-Completion-Port-like functionality in C++ land? "K. Frank" <kfrank29.c@gmail.com> - 2012-04-18 16:46 -0700
    Re: Is there (semi-) portable IO-Completion-Port-like functionality in C++ land? Christopher Pisz <someone@somewhere.net> - 2012-04-19 17:41 -0500
      Re: Is there (semi-) portable IO-Completion-Port-like functionality in C++ land? "K. Frank" <kfrank29.c@gmail.com> - 2012-04-20 10:43 -0700
        Re: Is there (semi-) portable IO-Completion-Port-like functionality in C++ land? Christopher Pisz <someone@somewhere.net> - 2012-04-20 17:48 -0500
          Re: Is there (semi-) portable IO-Completion-Port-like functionality in C++ land? "K. Frank" <kfrank29.c@gmail.com> - 2012-04-20 19:53 -0700
            Re: Is there (semi-) portable IO-Completion-Port-like functionality in C++ land? Nobody <nobody@nowhere.com> - 2012-04-22 03:17 +0100
            Re: Is there (semi-) portable IO-Completion-Port-like functionality in C++ land? Christof Meerwald <NOSPAM-seeMySig+uzk5+@usenet.cmeerw.org> - 2012-04-22 20:09 +0000

#15460 — Is there (semi-) portable IO-Completion-Port-like functionality in C++ land?

Hello Group!

Now that C++11 offers an atomic / synchronization / threading model,
has
an IO-Completion-Port-like synchronization object been implemented?

IO Completion Ports are discussed in MSDN:

   http://msdn.microsoft.com/en-us/library/aa365198%28VS.85%29.aspx

As a reminder, the Windows IO Completion Port has the following
interesting
feature:  The IO Completion Port (as I understand it) controls a LIFO
queue of worker threads that services a FIFO queue of "things to do"
(the so-called IO Completion Packets).  The interesting point is that
the IO Completion Port has a "concurrency value" and will not release
a worker thread to service the "packet" queue unless the number of
actively running threads (controlled by the IO Completion Port) is
less than the concurrency value.  That is, the IO Completion Port
tries to keep the number of actively running threads equal to the
concurrency value.

The point of this is that you can set the concurrency value equal to
the number of processors on the machine.  Now you can keep each
processor busy running its own active thread, but you don't slow
things down by gratuitously context switching to other threads in
thread pool.

This seems like a very good approach to getting good performance out
of a multi-core machine if you can partition your workload into to
"things to do" that can naturally be serviced by a thread pool.
(I think Microsoft designed IO Completion Ports with asynchronous
I/O in mind, but I view them as offering a more generally useful
approach to thread pools.)

Does C++11 offer the ability to implement something similar in a
semi-portable way?  Are there any implementations out there?

I'm pretty sure that there is nothing like this in std::thread.
I've also looked at Boost.Asio, and didn't see anything along
these lines.  (The Windows implementation of Boost.Asio appears
to use native IO Completion Ports under the hood, but appears
not to expose this functionality.)

So, are there any IO-Completion-Port-like thingies in C++ land,
or anything on the horizon?


Thanks.


K. Frank

[toc] | [next] | [standalone]

#15480

On 4/18/2012 6:46 PM, K. Frank wrote:
> Hello Group!
>
> Now that C++11 offers an atomic / synchronization / threading model,
> has
> an IO-Completion-Port-like synchronization object been implemented?
>
> IO Completion Ports are discussed in MSDN:
>
>     http://msdn.microsoft.com/en-us/library/aa365198%28VS.85%29.aspx
>
> As a reminder, the Windows IO Completion Port has the following
> interesting
> feature:  The IO Completion Port (as I understand it) controls a LIFO
> queue of worker threads that services a FIFO queue of "things to do"
> (the so-called IO Completion Packets).  The interesting point is that
> the IO Completion Port has a "concurrency value" and will not release
> a worker thread to service the "packet" queue unless the number of
> actively running threads (controlled by the IO Completion Port) is
> less than the concurrency value.  That is, the IO Completion Port
> tries to keep the number of actively running threads equal to the
> concurrency value.
>
> The point of this is that you can set the concurrency value equal to
> the number of processors on the machine.  Now you can keep each
> processor busy running its own active thread, but you don't slow
> things down by gratuitously context switching to other threads in
> thread pool.
>
> This seems like a very good approach to getting good performance out
> of a multi-core machine if you can partition your workload into to
> "things to do" that can naturally be serviced by a thread pool.
> (I think Microsoft designed IO Completion Ports with asynchronous
> I/O in mind, but I view them as offering a more generally useful
> approach to thread pools.)
>
> Does C++11 offer the ability to implement something similar in a
> semi-portable way?  Are there any implementations out there?
>
> I'm pretty sure that there is nothing like this in std::thread.
> I've also looked at Boost.Asio, and didn't see anything along
> these lines.  (The Windows implementation of Boost.Asio appears
> to use native IO Completion Ports under the hood, but appears
> not to expose this functionality.)
>
> So, are there any IO-Completion-Port-like thingies in C++ land,
> or anything on the horizon?
>
>
> Thanks.
>
>
> K. Frank


boost::asio::io_service does exactly what windows io completion ports 
do, in fact i uses them to do it. What is it that you think is missing? 
Or maybe you just didn't understand how to do what you want?

If you'd like to run a number of concurrent threads equal to the number 
of processors or cores, than get that number like you normally would, 
then create that number of boost::threads, in each thread call 
io_service::run. Give the io_service a work object so it doesn't close 
when it runs out of work to do in stays in queue.

You can also use a thread_group, but I haven't tried that yet. I just 
create an array of threads.

[toc] | [prev] | [next] | [standalone]

#15497

Hi Christopher!

Thank you for your reply and for pointing me to
boost::asio::io_service.

On Apr 19, 6:41 pm, Christopher Pisz <some...@somewhere.net> wrote:
> On 4/18/2012 6:46 PM, K. Frank wrote:
>
> > Hello Group!
>
> > Now that C++11 offers an atomic / synchronization / threading model,
> > has
> > an IO-Completion-Port-like synchronization object been implemented?
>
> > IO Completion Ports are discussed in MSDN:
>
> >    http://msdn.microsoft.com/en-us/library/aa365198%28VS.85%29.aspx
> ...
> > ...
> > The interesting point is that
> > the IO Completion Port has a "concurrency value" and will not release
> > a worker thread to service the "packet" queue unless the number of
> > actively running threads (controlled by the IO Completion Port) is
> > less than the concurrency value.  That is, the IO Completion Port
> > tries to keep the number of actively running threads equal to the
> > concurrency value.
>
> > The point of this is that you can set the concurrency value equal to
> > the number of processors on the machine.  Now you can keep each
> > processor busy running its own active thread, but you don't slow
> > things down by gratuitously context switching to other threads in
> > thread pool.
>
> > ...
>
> boost::asio::io_service does exactly what windows io completion ports
> do, in fact i uses them to do it. What is it that you think is missing?
> Or maybe you just didn't understand how to do what you want?

Looking at boost::asio::io_service, the IO Completion Port feature
that I think is missing is the "concurrency value."  (More precisely,
I think io_service implements a concurrency-value feature only on
windows, and not generally across platforms.  More on this, below.)

> If you'd like to run a number of concurrent threads equal to the number
> of processors or cores, than get that number like you normally would,
> then create that number of boost::threads, in each thread call
> io_service::run.

This makes sense in practice, but differs in detail from
what one can do with an IO Completion Port.

Specifically, let's say I have four processor cores.  With
an IO Completion Port I can set the concurrency value to
four, and then service the IO Completion Port with a thread
pool of, say, twenty threads.  As long as the threads don't
block, then the IO Completion Port will only let four of the
thread-pool threads run.  Each processor keeps busy, but we
avoid unnecessary context switching.  So far, this is the
same as running an io_service (or IO Completion Port) with
a thread pool of four threads.

But now suppose two of the threads block (for example, by
making a synchronous database call).  If I understand things
correctly, in the io_service case only two threads will be
actively running, leaving two processor cores idle.  In
contrast, the IO Completion Port will release two more threads
(from its thread pool of twenty) to work on the packet queue,
keeping all four cores busy.

To me that's the key issue.  You have the ability to have
more threads in the thread pool than cores so that the cores
keep busy even if some threads block, while (usually) only
having as many actively running threads as cores, thereby
avoiding unnecessary context switching.

This feature of IO Completion Ports has always struck me as
both clever and practical.

> ...
> You can also use a thread_group, but I haven't tried that yet. I just
> create an array of threads.

Just thinking out loud here ...

So (using C++11 std::thread terminology here) if an
IO-Completion-Port-like construct used a condition_variable
to synchronize the thread pool's access to the "packet"
queue, then we would want something like:

   if (
      "packet added to empty queue"  &&
      "active threads  <  concurrency_value"
   )  cv.notify_one();

and

   if (
      "active threads falls below concurrency_value"  &&
      "queue not empty"
   )  cv.notify_one();

Of course, this is just suggestive.  I don't pretend to know
how to implement an IO Completion Port without it already being
built into the operating system.

Coming back to io_service, it looks like it _does_ offer
concurrency_value functionality on windows.  One of the
io_service constructors takes a "concurrency_hint"
argument:

   io_service (std::size_t concurrency_hint);

I am willing to bet that this argument is ignored on
non-windows platforms, and that on windows platforms
io_service is implemented using an IO Completion Ports,
and that concurrency_hint is passed to the IO Completion
Port as its concurrency value.  (I haven't verified it,
but it looks true to me.)

This is altogether reasonable.  I can use io_service across
platforms, and on windows (but only on windows) I get the
benefit of the concurrency-value optimization.

Unfortunately this also suggests that implementing the
concurrency-value functionality on top of "standard"
synchronization primitives (without os support) isn't
so easy (because otherwise the boost guys might have
done it).

So, two questions:

Is it practical to implement the IO Completion Port
concurrency-value feature using "standard" primitives?

Does anyone have any historical color or insight into the
thinking of the Boost.Asio developers on this issue?


Thanks again.


K. Frank

[toc] | [prev] | [next] | [standalone]

#15498

On 4/20/2012 12:43 PM, K. Frank wrote:
SNIP
> But now suppose two of the threads block (for example, by
> making a synchronous database call).  If I understand things
> correctly, in the io_service case only two threads will be
> actively running, leaving two processor cores idle.  In
> contrast, the IO Completion Port will release two more threads
> (from its thread pool of twenty) to work on the packet queue,
> keeping all four cores busy.
>
> To me that's the key issue.  You have the ability to have
> more threads in the thread pool than cores so that the cores
> keep busy even if some threads block, while (usually) only
> having as many actively running threads as cores, thereby
> avoiding unnecessary context switching.
>
> This feature of IO Completion Ports has always struck me as
> both clever and practical.

I see now.
As you say below, I would think the boost io_service would mimic that 
behavior on Windows. I'd have to write a test case. Let me know what 
your results are if you write one. I wouldn't expect that to be the case 
on non-windows platforms.

>> ...
>> You can also use a thread_group, but I haven't tried that yet. I just
>> create an array of threads.
>
> Just thinking out loud here ...
>
> So (using C++11 std::thread terminology here) if an
> IO-Completion-Port-like construct used a condition_variable
> to synchronize the thread pool's access to the "packet"
> queue, then we would want something like:
>
>     if (
>        "packet added to empty queue"&&
>        "active threads<   concurrency_value"
>     )  cv.notify_one();
>
> and
>
>     if (
>        "active threads falls below concurrency_value"&&
>        "queue not empty"
>     )  cv.notify_one();
>
> Of course, this is just suggestive.  I don't pretend to know
> how to implement an IO Completion Port without it already being
> built into the operating system.
>
> Coming back to io_service, it looks like it _does_ offer
> concurrency_value functionality on windows.  One of the
> io_service constructors takes a "concurrency_hint"
> argument:
>
>     io_service (std::size_t concurrency_hint);
>
> I am willing to bet that this argument is ignored on
> non-windows platforms, and that on windows platforms
> io_service is implemented using an IO Completion Ports,
> and that concurrency_hint is passed to the IO Completion
> Port as its concurrency value.  (I haven't verified it,
> but it looks true to me.)
>
> This is altogether reasonable.  I can use io_service across
> platforms, and on windows (but only on windows) I get the
> benefit of the concurrency-value optimization.
>
> Unfortunately this also suggests that implementing the
> concurrency-value functionality on top of "standard"
> synchronization primitives (without os support) isn't
> so easy (because otherwise the boost guys might have
> done it).

I also believe you are correct in that I would not expect that behavior 
from non-Windows platforms, because as far as I know, it is only 
available on Windows. I remember when I was doing a socket library for 
Linux that I couldn't find a thing.

> So, two questions:
>
> Is it practical to implement the IO Completion Port
> concurrency-value feature using "standard" primitives?

I wouldn't dare try to do something similar myself, just because of 
risk/cost/benefit. Maybe for a purely academic just for fun exercise.

> Does anyone have any historical color or insight into the
> thinking of the Boost.Asio developers on this issue?

Good question for the boost mailing list. You can get to it using Gmane. 
They have instructions on the boost site. The response rate isn't great 
though. Most talk is about the C++11 features.

[toc] | [prev] | [next] | [standalone]

#15500

Hello Christopher!

Thanks for the follow-up.

On Apr 20, 6:48 pm, Christopher Pisz <some...@somewhere.net> wrote:
> On 4/20/2012 12:43 PM, K. Frank wrote:
> SNIP
> ...
> > Unfortunately this also suggests that implementing the
> > concurrency-value functionality on top of "standard"
> > synchronization primitives (without os support) isn't
> > so easy (because otherwise the boost guys might have
> > done it).
>
> I also believe you are correct in that I would not expect that behavior
> from non-Windows platforms, because as far as I know, it is only
> available on Windows. I remember when I was doing a socket library for
> Linux that I couldn't find a thing.

Yes, I've been thinking about this issue off and on for
a while.  Whenever I've looked at other os'es -- primarily
linux -- for this IO-Completion-Port concurrency-value
functionality, I've come up empty.

So I figure it must be one of two things:

Either IO Completion Ports aren't really that useful in
practice (but the idea seems very sensible to me), or
they're hard to implement, absent some existing low-level
support.

(Any linux gurus out there?  Why doesn't linux offer the
equivalent of IO Completion Ports?  Are they not really
a good idea, or are they too hard to implement?)

> > So, two questions:
>
> > Is it practical to implement the IO Completion Port
> > concurrency-value feature using "standard" primitives?
>
> I wouldn't dare try to do something similar myself, just because of
> risk/cost/benefit. Maybe for a purely academic just for fun exercise.

I'd certainly be interested in hearing any thoughts on how
one might go about it.

I can't see any way to get started.  It seems to me that any
IO-Completion-Port-like construct would have to know somehow
which of its associated threads are running and which are
blocked.  So I guess that the threads would have to notify
the construct.

Let's say that your threading model supports two kinds of
primitive (i.e., simpler than IO Completion Ports) synchronization
objects: mutexes and condition variables.  It would seem to
me that the mutex and condition-variable implementation
would have to know to notify "interested parties" (i.e., the
IO Completion Port) when threads are blocked and resumed.

So, according to this reasoning, unless support for IO
Completion Ports is already built into your synchronization
primitives, you wont be able to implement IO Completion
Ports.

(Of course, this would not preclude implementing IO
Completion Ports by reimplementing the other synchronization
primitives with the necessary support.  I.e., one could
imagine writing, for example, an "enhanced" pthreads library
that included IO Completion Ports, by making that library's
synchronization primitives "IO-Completion-Port-aware.")

I'm not really convinced that the above analysis is correct,
but, at the moment, that's how it looks to me.

>
> > Does anyone have any historical color or insight into the
> > thinking of the Boost.Asio developers on this issue?
>
> Good question for the boost mailing list. You can get to it using Gmane.
> They have instructions on the boost site. The response rate isn't great
> though. Most talk is about the C++11 features.

Thanks for the pointer to the boost mailing list.  I would
be curious to hear what they have to say.

I appreciate your insights.


K. Frank

[toc] | [prev] | [next] | [standalone]

#15506

On Fri, 20 Apr 2012 19:53:17 -0700, K. Frank wrote:

> Why doesn't linux offer the equivalent of IO Completion Ports?

The pthreads API doesn't include thread pools, which would be a
prerequisite.

There are a number of third-party thread pool libraries. Once you have
that, I would have thought that implementing completion ports would be
straightforward enough.

OTOH, async I/O is often unnecessary if you're using threads (likewise for
non-blocking I/O or select/poll). You can normally just use blocking I/O
and threads, which has the advantage of working with files (accessing
files never "blocks", even if there's a delay) as well as demand-paging of
the process' code and data.

On Linux, the POSIX aio API is implemented by glibc in user-space using
threads.

[toc] | [prev] | [next] | [standalone]

#15522

On Fri, 20 Apr 2012 19:53:17 -0700 (PDT), K. Frank wrote:
[...]
> (Of course, this would not preclude implementing IO
> Completion Ports by reimplementing the other synchronization
> primitives with the necessary support.  I.e., one could
> imagine writing, for example, an "enhanced" pthreads library
> that included IO Completion Ports, by making that library's
> synchronization primitives "IO-Completion-Port-aware.")

That would roughly be my understanding as well - except that most of
the "enhancements" would need to be made in the kernel for pretty much
any blocking syscall (when it enters a wait state). And I suspect that
this might come with some (performance) cost - but as there is no
standard mandating that functionality, it would be hard to justify.


Christof

-- 
http://cmeerw.org                              sip:cmeerw at cmeerw.org
mailto:cmeerw at cmeerw.org                   xmpp:cmeerw at cmeerw.org

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.c++

csiph-web