Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.c++ > #67321

Re: Why can't I understand what coroutines are?

From "Chris M. Thomasson" <invalid_chris_thomasson_invalid@invalid.com>
Newsgroups comp.lang.c++
Subject Re: Why can't I understand what coroutines are?
Date 2019-08-05 14:34 -0700
Organization Aioe.org NNTP Server
Message-ID <qia7cs$92f$1@gioia.aioe.org> (permalink)
References (2 earlier) <cone.1564925032.455203.25830.1004@monster.email-scan.com> <20190804143647.511134ccbc3ef6771a692cf7@cvine--nospam--.freeserve.co.uk> <qi77m2$13kh$1@gioia.aioe.org> <20190804203030.9a065d398363744cae8888b0@cvine--nospam--.freeserve.co.uk> <qi8iq2$2ld$1@dont-email.me>

Show all headers | View raw

On 8/4/2019 11:36 PM, Paavo Helde wrote:
> On 4.08.2019 22:30, Chris Vine wrote:
>> On Sun, 4 Aug 2019 11:20:51 -0700
>> "Chris M. Thomasson" <invalid_chris_thomasson_invalid@invalid.com>
>> wrote:
>>> On 8/4/2019 6:36 AM, Chris Vine wrote:
>>>> On Sun, 04 Aug 2019 09:23:52 -0400
>>>> Sam <sam@email-scan.com> wrote:
>>>>> Chris Vine writes:
>>>>>
>>>>>> Asynchronous programming with callbacks normally works so that when a
>>>>>> i/o operation might block because the resource in question (read or
>>>>>> write) is unavailable, instead of blocking the function returns
>>>>>> immediately and registers a callback with the program's event loop
>>>>>> (usually based on select() or poll()), representing the i/o 
>>>>>> operation's
>>>>>> continuation when the resource becomes available.  Programming 
>>>>>> this way
>>>>>> using callbacks in the style of, say, Node.js is troublesome 
>>>>>> because it
>>>>>> gives rise to what is sometimes called inversion of control, aka
>>>>>> "callback hell".  In particular, control flow ceases to appear as a
>>>>>> sequential series of i/o operations which can easily be reasoned 
>>>>>> about.
>>>>>
>>>>> This problem was solved a long time ago. It is called "threads".
>>>>>
>>>>> The funniest thing about this is that Gnome is most extreme case of 
>>>>> this.
>>>>> The entire Gnome stack is just one event dispatching main loop. 
>>>>> Everything
>>>>> is callbacks. But since Gnome is C code, none of this will help it, 
>>>>> that's
>>>>> the funny part.
>>>>
>>>> Threads are very definitely not the answer.  Threads work well with
>>>> cpu-bound workloads.  They are hopeless with i/o-bound workloads, which
>>>> is why co-operative (possibly single-threaded) multi-tasking using
>>>> coroutines has become popular.
>>>>
>>>> If you use native OS threads for i/o bound workloads, you are doing it
>>>> wrong.
>>>>
>>>
>>> Threads can work fairly well with IO, take a look at IOCP:
>>>
>>> https://docs.microsoft.com/en-us/windows/win32/fileio/i-o-completion-ports 
>>>
>>
>> I don't know microsoft's implementations, but as far as I can make out
>> the i/o is still done asynchronously (the file operations themselves do
>> not block) but when an asynchronous operation completes that completion
>> is handled by the completion port.  Although threads can wait on the
>> completion port for an i/o operation to reach completion, there appears
>> to be some form of multiplexing so that one thread can handle a number
>> of completions, and the documentation suggests that having a number of
>> threads equal to the number of cores (as opposed to the number of
>> connections) as a good starting point, which seems reasonable.
>>
>> But I cannot say the documentation you refer to is that clear: is my
>> summary wrong (and if so, what's the point of completion ports in the
>> first place)?  I find it hard to believe microsoft would have gone for
>> the one thread per connection option favoured by my respondent.
> 
> IOCP is used in Windows to *reduce* the number of needed threads (at 
> least in user space), which is important because in Windows the thread 
> creation is a pretty heavyweight operation. So it's strange that IOCP is 
> mentioned as an example of "Threads can work fairly well with IO", it's 
> rather the opposite, at least on Windows.

Yeah. IOCP is the way to do things on Windows wrt creating scaleable 
servers. I am fond of the following function:

https://docs.microsoft.com/en-us/windows/win32/api/mswsock/nf-mswsock-transmitfile

It works well with IOCP. Also,

https://docs.microsoft.com/en-us/windows/win32/api/mswsock/nc-mswsock-lpfn_transmitpackets

There is a neat way to wait on IOCP that can dequeue several events in 
one shot:

https://docs.microsoft.com/en-us/windows/win32/fileio/getqueuedcompletionstatusex-func

Iirc, certain Windows os's limited the number of TransmitFile functions 
to a max of two concurrent inflight calls at any time. The server 
versions of the OS would allow for as many as the system could handle at 
a time.



> 
> The Linux equivalent of IOCP is AIO, see e.g. "man aio_read". However, 
> some googling suggests the Linux implementation is not yet optimal (I 
> guess this means it is not yet built into the kernel to the same extent 
> as in Windows).
> 
> Also, the popular Boost.Asio library has "asynchronous" already in its 
> name and uses async IO in the background as appropriate. It also has 
> implementations of coroutines, so maybe one can study its documentation 
> and examples in order to get familiar with coroutines.
> 
> BTW, Boost.Asio does not create a "thread per connection". Vice versa, 
> it expects the client program to set up a thread pool (with the number 
> of threads based on the number of cores, typically) which is reused for 
> all requests.
> 
>

Back to comp.lang.c++ | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Why can't I understand what coroutines are? Juha Nieminen <nospam@thanks.invalid> - 2019-08-04 09:37 +0000
  Re: Why can't I understand what coroutines are? "Chris M. Thomasson" <invalid_chris_thomasson_invalid@invalid.com> - 2019-08-04 02:58 -0700
    Re: Why can't I understand what coroutines are? Sam <sam@email-scan.com> - 2019-08-04 09:18 -0400
      Re: Why can't I understand what coroutines are? "Chris M. Thomasson" <invalid_chris_thomasson_invalid@invalid.com> - 2019-08-04 11:15 -0700
  Re: Why can't I understand what coroutines are? Louis Krupp <lkrupp@nospam.pssw.com.invalid> - 2019-08-04 04:58 -0600
    Re: Why can't I understand what coroutines are? Juha Nieminen <nospam@thanks.invalid> - 2019-08-05 08:31 +0000
      Re: Why can't I understand what coroutines are? Martijn van Buul <pino@dohd.org> - 2019-08-05 09:51 +0000
        Re: Why can't I understand what coroutines are? "Chris M. Thomasson" <invalid_chris_thomasson_invalid@invalid.com> - 2019-08-05 14:44 -0700
  Re: Why can't I understand what coroutines are? Chris Vine <chris@cvine--nospam--.freeserve.co.uk> - 2019-08-04 14:08 +0100
    Re: Why can't I understand what coroutines are? Sam <sam@email-scan.com> - 2019-08-04 09:23 -0400
      Re: Why can't I understand what coroutines are? Chris Vine <chris@cvine--nospam--.freeserve.co.uk> - 2019-08-04 14:36 +0100
        Re: Why can't I understand what coroutines are? Sam <sam@email-scan.com> - 2019-08-04 10:31 -0400
          Re: Why can't I understand what coroutines are? Chris Vine <chris@cvine--nospam--.freeserve.co.uk> - 2019-08-04 16:42 +0100
            Re: Why can't I understand what coroutines are? Sam <sam@email-scan.com> - 2019-08-04 16:43 -0400
              Re: Why can't I understand what coroutines are? "Chris M. Thomasson" <invalid_chris_thomasson_invalid@invalid.com> - 2019-08-04 13:58 -0700
                Re: Why can't I understand what coroutines are? Sam <sam@email-scan.com> - 2019-08-04 17:36 -0400
                Re: Why can't I understand what coroutines are? Mr Flibble <flibbleREMOVETHISBIT@i42.co.uk> - 2019-08-04 22:43 +0100
                Re: Why can't I understand what coroutines are? Chris Vine <chris@cvine--nospam--.freeserve.co.uk> - 2019-08-04 22:52 +0100
                Re: Why can't I understand what coroutines are? Mr Flibble <flibbleREMOVETHISBIT@i42.co.uk> - 2019-08-04 23:10 +0100
                Re: Why can't I understand what coroutines are? Sam <sam@email-scan.com> - 2019-08-04 21:32 -0400
                Re: Why can't I understand what coroutines are? Mr Flibble <flibbleREMOVETHISBIT@i42.co.uk> - 2019-08-05 18:32 +0100
                Re: Why can't I understand what coroutines are? scott@slp53.sl.home (Scott Lurndal) - 2019-08-05 17:48 +0000
                Re: Why can't I understand what coroutines are? Sam <sam@email-scan.com> - 2019-08-05 20:14 -0400
                Re: Why can't I understand what coroutines are? leigh.v.johnston@googlemail.com - 2019-08-06 04:13 -0700
                Re: Why can't I understand what coroutines are? leigh.v.johnston@googlemail.com - 2019-08-06 04:23 -0700
                Re: Why can't I understand what coroutines are? Chris Vine <chris@cvine--nospam--.freeserve.co.uk> - 2019-08-06 22:19 +0100
                Re: Why can't I understand what coroutines are? "Chris M. Thomasson" <invalid_chris_thomasson_invalid@invalid.com> - 2019-08-06 15:19 -0700
                Re: Why can't I understand what coroutines are? Chris Vine <chris@cvine--nospam--.freeserve.co.uk> - 2019-08-07 00:01 +0100
                Re: Why can't I understand what coroutines are? scott@slp53.sl.home (Scott Lurndal) - 2019-08-06 23:52 +0000
                Re: Why can't I understand what coroutines are? Öö Tiib <ootiib@hot.ee> - 2019-08-06 16:57 -0700
                Re: Why can't I understand what coroutines are? scott@slp53.sl.home (Scott Lurndal) - 2019-08-07 14:41 +0000
                Re: Why can't I understand what coroutines are? Öö Tiib <ootiib@hot.ee> - 2019-08-08 07:10 -0700
                Re: Why can't I understand what coroutines are? scott@slp53.sl.home (Scott Lurndal) - 2019-08-06 23:49 +0000
                Re: Why can't I understand what coroutines are? Robert Wessel <robertwessel2@yahoo.com> - 2019-08-04 22:04 -0500
                Re: Why can't I understand what coroutines are? "Chris M. Thomasson" <invalid_chris_thomasson_invalid@invalid.com> - 2019-08-04 20:10 -0700
        Re: Why can't I understand what coroutines are? "Chris M. Thomasson" <invalid_chris_thomasson_invalid@invalid.com> - 2019-08-04 11:20 -0700
          Re: Why can't I understand what coroutines are? Chris Vine <chris@cvine--nospam--.freeserve.co.uk> - 2019-08-04 20:30 +0100
            Re: Why can't I understand what coroutines are? "Chris M. Thomasson" <invalid_chris_thomasson_invalid@invalid.com> - 2019-08-04 14:03 -0700
            Re: Why can't I understand what coroutines are? Paavo Helde <myfirstname@osa.pri.ee> - 2019-08-05 09:36 +0300
              Re: Why can't I understand what coroutines are? "Chris M. Thomasson" <invalid_chris_thomasson_invalid@invalid.com> - 2019-08-05 14:34 -0700
        Re: Why can't I understand what coroutines are? scott@slp53.sl.home (Scott Lurndal) - 2019-08-04 19:46 +0000
      Re: Why can't I understand what coroutines are? Juha Nieminen <nospam@thanks.invalid> - 2019-08-05 08:37 +0000
        Re: Why can't I understand what coroutines are? David Brown <david.brown@hesbynett.no> - 2019-08-05 12:06 +0200
          Re: Why can't I understand what coroutines are? Juha Nieminen <nospam@thanks.invalid> - 2019-08-05 12:20 +0000
            Re: Why can't I understand what coroutines are? David Brown <david.brown@hesbynett.no> - 2019-08-06 15:06 +0200
              Re: Why can't I understand what coroutines are? David Brown <david.brown@hesbynett.no> - 2019-08-06 15:18 +0200
          Re: Why can't I understand what coroutines are? Bonita Montero <Bonita.Montero@gmail.com> - 2019-08-06 13:58 +0200
  Re: Why can't I understand what coroutines are? Sam <sam@email-scan.com> - 2019-08-04 09:17 -0400
  Re: Why can't I understand what coroutines are? Robert Wessel <robertwessel2@yahoo.com> - 2019-08-04 10:12 -0500
    Re: Why can't I understand what coroutines are? Juha Nieminen <nospam@thanks.invalid> - 2019-08-05 08:42 +0000
      Re: Why can't I understand what coroutines are? Chris Vine <chris@cvine--nospam--.freeserve.co.uk> - 2019-08-05 13:35 +0100
      Re: Why can't I understand what coroutines are? Bart <bc@freeuk.com> - 2019-08-05 20:19 +0100
      Re: Why can't I understand what coroutines are? Christian Gollwitzer <auriocus@gmx.de> - 2019-08-05 22:26 +0200
        Re: Why can't I understand what coroutines are? "Alf P. Steinbach" <alf.p.steinbach+usenet@gmail.com> - 2019-08-05 23:09 +0200
          Re: Why can't I understand what coroutines are? Tim Rentsch <tr.17687@z991.linuxsc.com> - 2019-08-09 19:49 -0700
            Re: Why can't I understand what coroutines are? "Alf P. Steinbach" <alf.p.steinbach+usenet@gmail.com> - 2019-08-10 05:13 +0200
          Re: Why can't I understand what coroutines are? Christian Gollwitzer <auriocus@gmx.de> - 2019-08-10 08:23 +0200
            Re: Why can't I understand what coroutines are? "Alf P. Steinbach" <alf.p.steinbach+usenet@gmail.com> - 2019-08-10 10:43 +0200
              Re: Why can't I understand what coroutines are? Christian Gollwitzer <auriocus@gmx.de> - 2019-08-11 22:19 +0200
                Re: Why can't I understand what coroutines are? Tim Rentsch <tr.17687@z991.linuxsc.com> - 2019-08-14 08:25 -0700
        Re: Why can't I understand what coroutines are? "Chris M. Thomasson" <invalid_chris_thomasson_invalid@invalid.com> - 2019-08-10 00:19 -0700
        Re: Why can't I understand what coroutines are? Juha Nieminen <nospam@thanks.invalid> - 2019-08-11 21:16 +0000
      Re: Why can't I understand what coroutines are? "Chris M. Thomasson" <invalid_chris_thomasson_invalid@invalid.com> - 2019-08-05 23:43 -0700
      Re: Why can't I understand what coroutines are? Bonita Montero <Bonita.Montero@gmail.com> - 2019-08-10 10:50 +0200
  Re: Why can't I understand what coroutines are? Mr Flibble <flibbleREMOVETHISBIT@i42.co.uk> - 2019-08-04 17:57 +0100
  Re: Why can't I understand what coroutines are? "Alf P. Steinbach" <alf.p.steinbach+usenet@gmail.com> - 2019-08-05 01:11 +0200
    Re: Why can't I understand what coroutines are? Juha Nieminen <nospam@thanks.invalid> - 2019-08-05 08:50 +0000
      Re: Why can't I understand what coroutines are? "Alf P. Steinbach" <alf.p.steinbach+usenet@gmail.com> - 2019-08-05 16:59 +0200
    Re: Why can't I understand what coroutines are? Manfred <noname@add.invalid> - 2019-08-06 16:24 +0200
    Re: Why can't I understand what coroutines are? "Alf P. Steinbach" <alf.p.steinbach+usenet@gmail.com> - 2019-08-07 00:52 +0200
      Re: Why can't I understand what coroutines are? Tim Rentsch <tr.17687@z991.linuxsc.com> - 2019-08-09 18:11 -0700
  Re: Why can't I understand what coroutines are? Martijn van Buul <pino@dohd.org> - 2019-08-05 07:46 +0000

csiph-web