Path: csiph.com!usenet.pasdenom.info!aioe.org!news.stack.nl!newsfeed.xs4all.nl!newsfeed2a.news.xs4all.nl!xs4all!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail Return-Path: X-Original-To: python-list@python.org Delivered-To: python-list@mail.python.org X-Spam-Status: OK 0.003 X-Spam-Evidence: '*H*': 0.99; '*S*': 0.00; 'python,': 0.02; 'interfaces': 0.04; 'cpython': 0.05; '(so': 0.07; 'pypy': 0.07; 'similar,': 0.09; 'stackless': 0.09; 'things,': 0.09; 'cc:addr :python-list': 0.11; 'python': 0.11; 'changes': 0.15; 'cc:name:python list': 0.16; 'concurrency,': 0.16; 'concurrent': 0.16; 'example).': 0.16; 'threads.': 0.16; 'wrote:': 0.18; 'library': 0.18; 'variable': 0.18; 'later': 0.20; 'cc:addr:python.org': 0.22; 'load': 0.23; 'cc:2**0': 0.24; 'options': 0.25; 'switch': 0.26; 'header:In-Reply-To:1': 0.27; 'am,': 0.29; 'related': 0.29; "doesn't": 0.30; 'message- id:@mail.gmail.com': 0.30; 'libraries': 0.31; 'another': 0.32; 'fri,': 0.33; "i'd": 0.34; 'tool': 0.35; 'test': 0.35; 'received:google.com': 0.35; 'there': 0.35; 'really': 0.36; '14,': 0.36; 'hi,': 0.36; 'nov': 0.38; 'how': 0.40; 'most': 0.60; 'new': 0.61; 'show': 0.63; 'skip:n 10': 0.64; 'more': 0.64; 'account': 0.65; 'to:addr:gmail.com': 0.65; 'bothered': 0.68; 'choose.': 0.84; 'fortunately,': 0.84; 'processes,': 0.91; 'imagine': 0.93; 'hundred': 0.95 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=rRUMj/p8niTcE1YpJoMAdwVj1pEBMqy2KSwC41Ee0kw=; b=JbvBZ2a1ay6kAgjz6hhreuJbsmnxGF8IsfJQf9fqrHNoREmueaTFppVUxsJcRX/kDb ykWl+9Lv8PQnYZfx3j1St4PPUP1Cw3mAghy5bPt4ml0d7npIS60NF5RV7iU9y7dHJdzN o+5ihX93cJaSPSp+E2A+UrrA9qyU8q3aMvOEHMoiK+rRnZff5FsPB2/C+3Ig5C2tToCl 9TlL9sjlI4AdzGEVKFjy123QTVP5CgKYXPacLRF5CVfHQLyHu3dZM/b0SGAURLXIWfFe 99iqhxeb3d0Vhz1+kp3yYXbBfA62XG/fNdyhTtLqxoL/gxM69ojqqCaXU0a4/g43fgUc e/ZA== MIME-Version: 1.0 X-Received: by 10.112.247.43 with SMTP id yb11mr17575066lbc.51.1416115100103; Sat, 15 Nov 2014 21:18:20 -0800 (PST) In-Reply-To: References: Date: Sat, 15 Nov 2014 21:18:20 -0800 Subject: Re: Efficient Threading From: Dan Stromberg To: Empty Account Content-Type: text/plain; charset=UTF-8 Cc: Python List X-BeenThere: python-list@python.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: General discussion list for the Python programming language List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Newsgroups: comp.lang.python Message-ID: Lines: 22 NNTP-Posting-Host: 2001:888:2000:d::a6 X-Trace: 1416115109 news.xs4all.nl 2927 [2001:888:2000:d::a6]:52831 X-Complaints-To: abuse@xs4all.nl Xref: csiph.com comp.lang.python:80852 On Fri, Nov 14, 2014 at 10:42 AM, Empty Account wrote: > Hi, > > I am thinking about writing a load test tool in Python, so I am interested > in how I can create the most concurrent threads/processes with the fewest OS > resources. I would imagine that I/O would need to be non-blocking. > > There are a number of options including standard library threading, gevent, > stackless python, cython parallelism etc. Because I am new to Python, I am > unsure on which libraries to choose. I am not really bothered on the tool > chain, just as long as it is Python related (so I'd use PyPy for example). If you need a large amount of concurrency, you might look at Jython with threads. Jython threads well. If you don't intend to do more than a few hundred concurrent things, you might just go with CPython and multiprocessing. Fortunately, the interfaces are similar, so you can try one and switch to another later without huge issues. The main difference is that multiprocessing doesn't show variable changes to other processes, while multithreading (more often) does.