Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #70043

Re: threading

Path csiph.com!v102.xanadu-bbs.net!xanadu-bbs.net!goblin2!goblin.stu.neva.ru!newsfeed.xs4all.nl!newsfeed2a.news.xs4all.nl!xs4all!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail
Return-Path <python-python-list@m.gmane.org>
X-Original-To python-list@python.org
Delivered-To python-list@mail.python.org
X-Spam-Status OK 0.022
X-Spam-Evidence '*H*': 0.96; '*S*': 0.00; 'programmer': 0.03; 'explicit': 0.07; 'socket': 0.07; 'correct,': 0.09; 'modes': 0.09; 'received:80.91': 0.09; 'received:80.91.229': 0.09; 'received:gmane.org': 0.09; 'received:list': 0.09; 'blocking': 0.16; 'buffer.': 0.16; 'combinations': 0.16; 'eof': 0.16; 'frees': 0.16; 'guarded': 0.16; 'operation.': 0.16; 'points:': 0.16; 'readability': 0.16; 'received:80.91.229.3': 0.16; 'received:plane.gmane.org': 0.16; 'resist': 0.16; 'situation.': 0.16; 'sizable': 0.16; 'sockets': 0.16; 'prevent': 0.16; 'pushed': 0.16; 'wrote:': 0.18; 'code.': 0.18; 'finished': 0.19; 'written': 0.21; 'header:User-Agent:1': 0.23; 'specify': 0.24; '(or': 0.24; 'task': 0.26; 'header:X-Complaints-To:1': 0.27; 'idea': 0.28; 'function': 0.29; 'correct': 0.29; 'raise': 0.29; "doesn't": 0.30; 'returned': 0.30; 'getting': 0.31; 'easier': 0.31; 'context.': 0.31; 'facility': 0.31; 'seemingly': 0.31; 'quite': 0.32; "i'd": 0.34; 'could': 0.34; 'connection': 0.35; 'objects': 0.35; 'but': 0.35; 'there': 0.35; 'method': 0.36; 'possible': 0.36; 'similar': 0.36; 'should': 0.36; 'example,': 0.37; 'virtual': 0.37; 'two': 0.37; 'implement': 0.38; 'server': 0.38; 'connections': 0.38; 'machines': 0.38; 'handle': 0.38; 'to:addr:python-list': 0.38; 'sure': 0.39; 'to:addr:python.org': 0.39; 'received:org': 0.40; 'how': 0.40; 'even': 0.60; 'read': 0.60; 'most': 0.60; 'tell': 0.60; 'simple': 0.61; 'such': 0.63; 'choose': 0.64; 'strategy': 0.64; 'more': 0.64; 'different': 0.65; 'account': 0.65; 'close': 0.67; 'tasks.': 0.68; 'fact,': 0.69; 'safe': 0.72; 'gotten': 0.74; 'obvious': 0.74; 'guaranteed': 0.75; 'facility.': 0.84; 'tolerate': 0.84; 'tricky': 0.84; 'wakes': 0.84; 'discipline': 0.91; 'write()': 0.91; 'hand,': 0.93; 'connection,': 0.95; 'race': 0.95
X-Injected-Via-Gmane http://gmane.org/
To python-list@python.org
From Sturla Molden <sturla.molden@gmail.com>
Subject Re: threading
Date Thu, 10 Apr 2014 15:24:46 +0000 (UTC)
References <87wqexmmuc.fsf@elektro.pacujo.net>
Mime-Version 1.0
Content-Type text/plain; charset=UTF-8
Content-Transfer-Encoding 8bit
X-Gmane-NNTP-Posting-Host 39-82-11.connect.netcom.no
User-Agent NewsTap/4.0.1 (iPad)
X-
X-BeenThere python-list@python.org
X-Mailman-Version 2.1.15
Precedence list
List-Id General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe <https://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive <http://mail.python.org/pipermail/python-list/>
List-Post <mailto:python-list@python.org>
List-Help <mailto:python-list-request@python.org?subject=help>
List-Subscribe <https://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
Newsgroups comp.lang.python
Message-ID <mailman.9140.1397143500.18130.python-list@python.org> (permalink)
Lines 73
NNTP-Posting-Host 2001:888:2000:d::a6
X-Trace 1397143500 news.xs4all.nl 2833 [2001:888:2000:d::a6]:40275
X-Complaints-To abuse@xs4all.nl
Xref csiph.com comp.lang.python:70043

Show key headers only | View raw

Marko Rauhamaa <marko@pacujo.net> wrote:

> Other points:
> 
>  * When you wake up from select() (or poll(), epoll()), you should treat
>    it as a hint. The I/O call (accept()) could still raise
>    socket.error(EAGAIN).
> 
>  * The connections returned from accept() have to be individually
>    registered with select() (poll(), epoll()).
> 
>  * When you write() into a connection, you may be able to send only part
>    of the data or get EAGAIN. You need to choose a buffering strategy --
>    you should not block until all data is written out. Also take into
>    account how much you are prepared to buffer.
> 
>  * There are two main modes of multiplexing: level-triggered and
>    edge-triggered. Only epoll() (and kqueue()) support edge-triggered
>    wakeups. Edge-triggered requires more discipline from the programmer
>    but frees you from having to tell the multiplexing facility if you
>    are interested in readability or writability in any given situation.
> 
>    Edge-triggered wakeups are only guaranteed after you have gotten an
>    EAGAIN from an operation. Make sure you keep on reading/writing until
>    you get an EAGAIN. On the other hand, watch out so one connection
>    doesn't hog the process because it always has active I/O to perform.
> 
>  * You should always be ready to read to prevent deadlocks.
> 
>  * Sockets can be half-closed. Your state machines should deal with the
>    different combinations gracefully. For example, you might read an EOF
>    from the client socket before you have pushed the response out. You
>    must not close the socket before the response has finished writing.
>    On the other hand, you should not treat the half-closed socket as
>    readable.
> 
>  * While a single-threaded process will not have proper race conditions,
>    you must watch out for preemption. IOW, you might have Object A call
>    a method of Object B, which calls some other method of Object A.
>    Asyncio has a task queue facility. If you write your own main loop,
>    you should also implement a similar task queue. The queue can then be
>    used to make such tricky function calls in a safe context.
> 
>  * Asyncio provides timers. If you write your own main loop, you should
>    also implement your own timers.
> 
>    Note that modern software has to tolerate suspension (laptop lid,
>    virtual machines). Time is a tricky concept when your server wakes up
>    from a coma.
> 
>  * Specify explicit states. Your connection objects should have a data
>    member named "state" (or similar). Make your state transitions
>    explicit and obvious in the code. In fact, log them. Resist the
>    temptation of deriving the state implicitly from other object
>    information.
> 
>  * Most states should be guarded with a timer. Make sure to document for
>    each state, which timers are running.
> 
>  * In each state, check that you handle all possible events and
>    timeouts. The state/transition matrix will be quite sizable even for
>    seemingly simple tasks.


And exactly how is getting all of this correct any easier than just using
threads and blocking i/o?

I'd like to see the programmer who can get all of this correct, but has no
idea how to use a queue og mutex without deadlocking.


Sturla

Back to comp.lang.python | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Re: threading Ben Finney <ben+python@benfinney.id.au> - 2014-04-07 13:05 +1000
  Re: threading Roy Smith <roy@panix.com> - 2014-04-06 23:48 -0400
    Re: threading Chris Angelico <rosuav@gmail.com> - 2014-04-07 13:56 +1000
      Re: threading Roy Smith <roy@panix.com> - 2014-04-07 08:26 -0400
        Re: threading Chris Angelico <rosuav@gmail.com> - 2014-04-07 22:34 +1000
          Re: threading Roy Smith <roy@panix.com> - 2014-04-07 09:22 -0400
            Re: threading Mark Lawrence <breamoreboy@yahoo.co.uk> - 2014-04-07 14:41 +0100
            Re: threading Marko Rauhamaa <marko@pacujo.net> - 2014-04-07 16:49 +0300
              Re: threading Chris Angelico <rosuav@gmail.com> - 2014-04-08 00:27 +1000
                Re: threading Marko Rauhamaa <marko@pacujo.net> - 2014-04-07 17:51 +0300
                Re: threading Chris Angelico <rosuav@gmail.com> - 2014-04-08 01:12 +1000
            Re: threading Chris Angelico <rosuav@gmail.com> - 2014-04-08 00:24 +1000
      Re: threading Rick Johnson <rantingrickjohnson@gmail.com> - 2014-04-08 18:09 -0700
        Re: threading "Neil D. Cerutti" <neilc@norwich.edu> - 2014-04-09 09:50 -0400
          Re: threading Rick Johnson <rantingrickjohnson@gmail.com> - 2014-04-09 08:51 -0700
            Re: threading MRAB <python@mrabarnett.plus.com> - 2014-04-09 18:47 +0100
              Re: threading Gregory Ewing <greg.ewing@canterbury.ac.nz> - 2014-04-10 11:35 +1200
                Re: threading Roy Smith <roy@panix.com> - 2014-04-09 19:53 -0400
                Re: threading Andrew Berg <robotsondrugs@gmail.com> - 2014-04-09 19:02 -0500
                Re: threading Steven D'Aprano <steve@pearwood.info> - 2014-04-10 02:43 +0000
                Re: threading Chris Angelico <rosuav@gmail.com> - 2014-04-10 13:08 +1000
                Re: threading Mark Lawrence <breamoreboy@yahoo.co.uk> - 2014-04-10 09:23 +0100
                Re: threading Chris Angelico <rosuav@gmail.com> - 2014-04-10 19:11 +1000
            Re: threading Chris Angelico <rosuav@gmail.com> - 2014-04-10 04:00 +1000
            Re: threading Steven D'Aprano <steve@pearwood.info> - 2014-04-10 03:44 +0000
              Re: threading Chris Angelico <rosuav@gmail.com> - 2014-04-10 13:54 +1000
    Re: threading Ben Finney <ben+python@benfinney.id.au> - 2014-04-07 15:22 +1000
    Re: threading Ethan Furman <ethan@stoneleaf.us> - 2014-04-08 11:09 -0700
    Re: threading Sturla Molden <sturla.molden@gmail.com> - 2014-04-08 21:41 +0200
      Re: threading Grant Edwards <invalid@invalid.invalid> - 2014-04-08 20:30 +0000
        Re: threading Sturla Molden <sturla.molden@gmail.com> - 2014-04-09 00:32 +0200
          Re: threading Rustom Mody <rustompmody@gmail.com> - 2014-04-08 19:17 -0700
  Re: threading Marko Rauhamaa <marko@pacujo.net> - 2014-04-07 08:10 +0300
    Re: threading Paul Rubin <no.email@nospam.invalid> - 2014-04-06 22:39 -0700
      Re: threading Marko Rauhamaa <marko@pacujo.net> - 2014-04-07 08:46 +0300
      Re: threading Dennis Lee Bieber <wlfraed@ix.netcom.com> - 2014-04-07 19:47 -0400
        Re: threading Marko Rauhamaa <marko@pacujo.net> - 2014-04-08 08:19 +0300
          Re: threading Sturla Molden <sturla.molden@gmail.com> - 2014-04-08 10:47 +0000
            Re: threading Marko Rauhamaa <marko@pacujo.net> - 2014-04-08 15:10 +0300
              Re: threading Sturla Molden <sturla.molden@gmail.com> - 2014-04-08 16:37 +0000
                Re: threading Marko Rauhamaa <marko@pacujo.net> - 2014-04-08 20:17 +0300
            Re: threading Roy Smith <roy@panix.com> - 2014-04-08 09:19 -0400
              Re: threading Sturla Molden <sturla.molden@gmail.com> - 2014-04-08 15:44 +0000
                Re: threading Paul Rubin <no.email@nospam.invalid> - 2014-04-08 09:38 -0700
                Re: threading Mark Lawrence <breamoreboy@yahoo.co.uk> - 2014-04-09 14:42 +0100
          Re: threading "Frank Millman" <frank@chagford.com> - 2014-04-09 15:23 +0200
            Re: threading Marko Rauhamaa <marko@pacujo.net> - 2014-04-09 16:55 +0300
              Re: threading "Frank Millman" <frank@chagford.com> - 2014-04-09 16:46 +0200
                Re: threading Marko Rauhamaa <marko@pacujo.net> - 2014-04-09 20:31 +0300
                Re: threading Chris Angelico <rosuav@gmail.com> - 2014-04-10 03:52 +1000
                Re: threading Mark H Harris <harrismh777@gmail.com> - 2014-04-10 08:29 -0500
                Re: threading Sturla Molden <sturla.molden@gmail.com> - 2014-04-09 19:20 +0000
          Re: threading Chris Angelico <rosuav@gmail.com> - 2014-04-09 23:47 +1000
            Re: threading Roy Smith <roy@panix.com> - 2014-04-09 10:44 -0400
          Re: threading "Frank Millman" <frank@chagford.com> - 2014-04-09 16:30 +0200
            Re: threading Roy Smith <roy@panix.com> - 2014-04-09 10:52 -0400
              Re: threading Gregory Ewing <greg.ewing@canterbury.ac.nz> - 2014-04-10 11:19 +1200
            Re: threading Marko Rauhamaa <marko@pacujo.net> - 2014-04-09 19:48 +0300
          Re: threading Chris Angelico <rosuav@gmail.com> - 2014-04-10 00:44 +1000
          Re: threading Sturla Molden <sturla.molden@gmail.com> - 2014-04-09 15:29 +0000
          Re: threading Terry Reedy <tjreedy@udel.edu> - 2014-04-09 12:14 -0400
          Re: threading Chris Angelico <rosuav@gmail.com> - 2014-04-10 02:25 +1000
          Re: threading Sturla Molden <sturla.molden@gmail.com> - 2014-04-09 16:32 +0000
          Re: threading Dennis Lee Bieber <wlfraed@ix.netcom.com> - 2014-04-09 19:44 -0400
          Re: threading Chris Angelico <rosuav@gmail.com> - 2014-04-10 11:05 +1000
          Re: threading "Frank Millman" <frank@chagford.com> - 2014-04-10 11:17 +0200
          Re: threading Chris Angelico <rosuav@gmail.com> - 2014-04-10 19:40 +1000
          Re: threading "Frank Millman" <frank@chagford.com> - 2014-04-10 13:10 +0200
            Re: threading Marko Rauhamaa <marko@pacujo.net> - 2014-04-10 14:43 +0300
              Re: threading Roy Smith <roy@panix.com> - 2014-04-10 08:56 -0400
              Re: threading Sturla Molden <sturla.molden@gmail.com> - 2014-04-10 15:24 +0000
                Re: threading Marko Rauhamaa <marko@pacujo.net> - 2014-04-10 19:20 +0300
              Re: threading Chris Angelico <rosuav@gmail.com> - 2014-04-11 01:32 +1000
                Re: threading Marko Rauhamaa <marko@pacujo.net> - 2014-04-10 19:25 +0300
                Re: threading Chris Angelico <rosuav@gmail.com> - 2014-04-11 03:08 +1000
                Re: threading Rustom Mody <rustompmody@gmail.com> - 2014-04-10 11:14 -0700
                Re: threading Marko Rauhamaa <marko@pacujo.net> - 2014-04-10 22:44 +0300
                Re: threading Rustom Mody <rustompmody@gmail.com> - 2014-04-10 13:21 -0700
                Re: threading Marko Rauhamaa <marko@pacujo.net> - 2014-04-10 23:44 +0300
                Re: threading Rustom Mody <rustompmody@gmail.com> - 2014-04-10 22:15 -0700
                Re: threading Rustom Mody <rustompmody@gmail.com> - 2014-04-10 23:50 -0700
                Re: threading Marko Rauhamaa <marko@pacujo.net> - 2014-04-11 18:36 +0300
                Re: threading Chris Angelico <rosuav@gmail.com> - 2014-04-12 01:53 +1000
                Re: threading Mark Lawrence <breamoreboy@yahoo.co.uk> - 2014-04-11 16:58 +0100
                Re: threading Rustom Mody <rustompmody@gmail.com> - 2014-04-11 11:54 -0700
                Re: threading Marko Rauhamaa <marko@pacujo.net> - 2014-04-11 22:27 +0300
                Re: threading Sturla Molden <sturla.molden@gmail.com> - 2014-04-11 01:51 +0200
                Re: threading Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-04-11 05:35 +0000
                Re: threading Sturla Molden <sturla.molden@gmail.com> - 2014-04-11 09:26 +0000
                Re: threading Roy Smith <roy@panix.com> - 2014-04-11 08:36 -0400
                Re: threading Grant Edwards <invalid@invalid.invalid> - 2014-04-11 16:18 +0000
                Re: threading Sturla Molden <sturla.molden@gmail.com> - 2014-04-11 02:21 +0200
                Re: threading Terry Reedy <tjreedy@udel.edu> - 2014-04-10 20:23 -0400
          Re: threading Chris Angelico <rosuav@gmail.com> - 2014-04-10 21:19 +1000
      Re: threading Sturla Molden <sturla.molden@gmail.com> - 2014-04-08 02:06 +0000
        Re: threading alister <alister.nospam.ware@ntlworld.com> - 2014-04-08 11:07 +0000
          Re: threading Roy Smith <roy@panix.com> - 2014-04-08 09:13 -0400
            Re: threading Chris Angelico <rosuav@gmail.com> - 2014-04-08 23:23 +1000
              Re: threading alister <alister.nospam.ware@ntlworld.com> - 2014-04-08 14:15 +0000
                Re: threading Sturla Molden <sturla.molden@gmail.com> - 2014-04-08 16:06 +0000
            Re: threading Sturla Molden <sturla.molden@gmail.com> - 2014-04-08 15:40 +0000
              Re: threading Paul Rubin <no.email@nospam.invalid> - 2014-04-08 09:46 -0700
                Re: threading Chris Angelico <rosuav@gmail.com> - 2014-04-09 02:46 +1000
                Re: threading Sturla Molden <sturla.molden@gmail.com> - 2014-04-08 17:17 +0000
          Re: threading Sturla Molden <sturla.molden@gmail.com> - 2014-04-08 15:19 +0000

csiph-web