Path: csiph.com!newsfeed.hal-mli.net!feeder3.hal-mli.net!newsfeed.hal-mli.net!feeder1.hal-mli.net!newsfeed.xs4all.nl!newsfeed3.news.xs4all.nl!xs4all!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail Return-Path: X-Original-To: python-list@python.org Delivered-To: python-list@mail.python.org X-Spam-Status: OK 0.048 X-Spam-Evidence: '*H*': 0.90; '*S*': 0.00; 'exercise': 0.04; 'argument': 0.05; 'option,': 0.09; 'cc:addr:python-list': 0.11; 'thread': 0.14; 'windows': 0.15; 'be:': 0.16; 'entities.': 0.16; 'from:addr:rosuav': 0.16; 'from:name:chris angelico': 0.16; 'rough': 0.16; 'roy': 0.16; 'threads.': 0.16; '(you': 0.16; 'wrote:': 0.18; 'code,': 0.22; 'cc:addr:python.org': 0.22; 'mon,': 0.24; 'cc:2**0': 0.24; 'sort': 0.25; 'header:In-Reply-To:1': 0.27; 'idea': 0.28; 'point': 0.28; "doesn't": 0.30; 'compared': 0.30; 'message-id:@mail.gmail.com': 0.30; 'controlled': 0.31; 'something': 0.35; 'received:google.com': 0.35; 'there': 0.35; 'consistent': 0.36; 'done': 0.36; 'should': 0.36; 'performance': 0.37; 'represent': 0.38; 'connections': 0.38; 'pm,': 0.38; 'expensive': 0.39; 'though,': 0.39; 'even': 0.60; 'hope': 0.61; "you're": 0.61; 'guarantee': 0.63; 'such': 0.63; 'more': 0.64; 'smith': 0.68; 'foreign': 0.74; 'yourself': 0.78; 'potentially': 0.81; 'alone.': 0.84; 'spawned': 0.84; 'system!': 0.84; 'to:none': 0.92; 'connection,': 0.95 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:cc :content-type; bh=iWuSDXN3WoKTLUMzvJtHesn0FVc63Offvf7IMkzsI7g=; b=tf1sBxL3MlQSc51JvzvCPTpvvA16BdXFrHiyTfNKQ5LwVunZIKSLxAd4kO2JqG3ODw l6M4ePFEWJpxUqLvJJamWdIjDEf0yxaLT4ayLu7sxC+ulZDvIK/zKu6HRi4zOboZ0R+L knMK+4EmLrrG1DGvwblBh5XpAh5XCgQ/GWdGQpI7vu521VCHxu5nsJcOM6QOY3zVORIZ wfkOuEN79EQ8KnWds3hi4VWWTHZ4L/aCdUBILFPbBZ+hLnQz/nGrWtx7qdzdU33riAsI AvhGw4xcdD3maAjrg2ahDhgVPdhpqUTdnj9z0+47Y0qtIxeZaf6qPeD6jJR64NSrlND0 d0IA== MIME-Version: 1.0 X-Received: by 10.66.142.201 with SMTP id ry9mr31162378pab.14.1396880824982; Mon, 07 Apr 2014 07:27:04 -0700 (PDT) In-Reply-To: <87zjjxp7uv.fsf@elektro.pacujo.net> References: <87zjjxp7uv.fsf@elektro.pacujo.net> Date: Tue, 8 Apr 2014 00:27:04 +1000 Subject: Re: threading From: Chris Angelico Cc: "python-list@python.org" Content-Type: text/plain; charset=UTF-8 X-BeenThere: python-list@python.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: General discussion list for the Python programming language List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Newsgroups: comp.lang.python Message-ID: Lines: 30 NNTP-Posting-Host: 2001:888:2000:d::a6 X-Trace: 1396880827 news.xs4all.nl 2919 [2001:888:2000:d::a6]:46801 X-Complaints-To: abuse@xs4all.nl Xref: csiph.com comp.lang.python:69813 On Mon, Apr 7, 2014 at 11:49 PM, Marko Rauhamaa wrote: > Roy Smith : > >> The idea that we should continue to use threading just because Windows >> makes process creation hideously expensive compared to thread creation >> doesn't impress me as an argument in favor of threading. It impresses >> me as an argument in favor of ditching Windows. > > There are many reasons to start processes or threads. One of them is > performance (you have more than one CPU core). When performance is the > objective, my rough guidelines would be: > > * Start your processes during initialization. > > * Start about twice as many processes as there are CPUs. > > IOW, the processes are there to exercise the CPUs and should not > represent individual connections or other dynamic entities. That's potentially brutal on a shared system! I hope it's controlled by an option, or that you do this only in something you're writing for yourself alone. There are other reasons for forking per connection, though, such as state cleanup. You can guarantee that each job is done in a consistent state if it's spawned from the exact same point in the master process every time. That can help enormously if you're allowing any sort of foreign code, even from a trusted programmer. ChrisA