Path: csiph.com!v102.xanadu-bbs.net!xanadu-bbs.net!news.albasani.net!rt.uk.eu.org!newsfeed.xs4all.nl!newsfeed4.news.xs4all.nl!xs4all!post.news.xs4all.nl!not-for-mail Return-Path: X-Original-To: python-list@python.org Delivered-To: python-list@mail.python.org X-Spam-Status: OK 0.008 X-Spam-Evidence: '*H*': 0.98; '*S*': 0.00; 'languages,': 0.04; 'feasible.': 0.09; 'received:80.91': 0.09; 'received:80.91.229': 0.09; 'received:gmane.org': 0.09; 'received:list': 0.09; '\xe2\x80\x94': 0.09; 'python': 0.11; 'distinct': 0.16; 'experiments': 0.16; 'finney': 0.16; 'processes.': 0.16; 'received:80.91.229.3': 0.16; 'received:plane.gmane.org': 0.16; 'reliably': 0.16; 'reproduce': 0.16; 'sign,': 0.16; 'somehow.': 0.16; 'library': 0.18; 'module': 0.19; 'passing': 0.19; 'memory': 0.22; 'separate': 0.22; 'header:User-Agent:1': 0.23; 'module,': 0.24; 'decide': 0.24; "i've": 0.25; 'header:X-Complaints-To:1': 0.27; 'needed.': 0.30; 'code': 0.31; 'easier': 0.31; 'apparently': 0.31; 'writes:': 0.31; 'run': 0.32; 'url:python': 0.33; 'running': 0.33; 'common': 0.35; 'usual': 0.35; 'but': 0.35; 'there': 0.35; 'url:org': 0.36; 'problems': 0.38; 'ben': 0.38; 'url:library': 0.38; 'to:addr:python-list': 0.38; 'that,': 0.38; 'sure': 0.39; 'to:addr:python.org': 0.39; 'received:org': 0.40; 'how': 0.40; 'even': 0.60; '8bit%:29': 0.60; 'most': 0.60; 'url:3': 0.61; "you're": 0.61; 'choose': 0.64; 'more': 0.64; 'different': 0.65; 'between': 0.67; 'determine': 0.67; 'results': 0.69; 'sharing': 0.69; 'execution.': 0.84; 'received:125': 0.84 X-Injected-Via-Gmane: http://gmane.org/ To: python-list@python.org From: Ben Finney Subject: Re: threading Date: Mon, 07 Apr 2014 13:05:03 +1000 References: Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Gmane-NNTP-Posting-Host: jigong.madmonks.org X-Public-Key-ID: 0xBD41714B X-Public-Key-Fingerprint: 9CFE 12B0 791A 4267 887F 520C B7AC 2E51 BD41 714B X-Public-Key-URL: http://www.benfinney.id.au/contact/bfinney-gpg.asc X-Post-From: Ben Finney User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.4 (gnu/linux) Cancel-Lock: sha1:h9m8sDCatomajPMqRTtrvRyPMWs= X-BeenThere: python-list@python.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: General discussion list for the Python programming language List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Newsgroups: comp.lang.python Message-ID: Lines: 37 NNTP-Posting-Host: 2001:888:2000:d::a6 X-Trace: 1396839923 news.xs4all.nl 2938 [2001:888:2000:d::a6]:40115 X-Complaints-To: abuse@xs4all.nl Xref: csiph.com comp.lang.python:69792 Onder Hazaroglu writes: > I've been using threading library to run some experiments parallel. Threading is very difficult to get right, much more so than the usual separation into distinct processes. How did you decide against the normal means of parallel execution and instead choose threading? > There is no message passing between my threads but still it messes up > somehow. The results are different than running it separated. Yes, this is a common result of threading; problems are much harder to reproduce and much more entangled with apparently unrelated factors. This is one of the main reasons to avoid threading if at all feasible. > Is there a way to make sure that there is no memory sharing between > threads? The *whole point* of threading (AFAIK) is to share memory and other process-distinct resources. If you're looking to avoid that, don't use threading! Instead, use separate processes for parallel execution. Parallel processing is achieved much more reliably and deterministically with separate processes. Python even makes this much easier than most languages, with the ‘multiprocessing’ module in the standard library . I would recommend you make easier-to-understand and easier-to-debug code with that module, and only consider threading if you determine it's needed. -- \ “Stop — Drive sideways.” —detour sign, Kyushu, Japan | `\ | _o__) | Ben Finney