Path: csiph.com!v102.xanadu-bbs.net!xanadu-bbs.net!feeder.erje.net!eu.feeder.erje.net!newsfeed.xs4all.nl!newsfeed1.news.xs4all.nl!xs4all!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail Return-Path: X-Original-To: python-list@python.org Delivered-To: python-list@mail.python.org X-Spam-Status: OK 0.039 X-Spam-Evidence: '*H*': 0.92; '*S*': 0.00; 'nasty': 0.07; 'read-only': 0.09; 'run,': 0.09; 'stack,': 0.09; 'runs': 0.10; 'cheers': 0.12; 'thread': 0.14; "wouldn't": 0.14; 'any.': 0.16; 'callables': 0.16; 'fork': 0.16; 'malloc': 0.16; 'manner,': 0.16; 'sockets': 0.16; 'subject:Pypy': 0.16; 'thread,': 0.16; 'wrote:': 0.18; 'code.': 0.18; 'library': 0.18; 'pointed': 0.19; 'stack': 0.19; 'meant': 0.20; 'feb': 0.22; 'code,': 0.22; 'memory': 0.22; 'error': 0.23; '(such': 0.24; "aren't": 0.24; 'convenient': 0.24; 'space.': 0.24; 'paul': 0.24; 'options': 0.25; 'pass': 0.26; 'point': 0.28; 'message-id:@mail.gmail.com': 0.30; 'url:mailman': 0.30; 'code': 0.31; 'directly,': 0.31; 'safely': 0.31; 'them?': 0.31; 'probably': 0.32; 'run': 0.32; 'quite': 0.32; '(including': 0.33; 'url:python': 0.33; 'bugs': 0.33; 'could': 0.34; 'anywhere': 0.35; 'objects': 0.35; 'but': 0.35; 'received:google.com': 0.35; 'there': 0.35; 'module.': 0.36; 'url:listinfo': 0.36; 'subject:?': 0.36; 'url:org': 0.36; 'example,': 0.37; 'implement': 0.38; 'to:addr:python-list': 0.38; 'track': 0.38; 'weight': 0.39; 'to:addr:python.org': 0.39; 'space': 0.40; 'url:mail': 0.40; 'even': 0.60; 'easy': 0.60; 'problems.': 0.60; 'high': 0.63; 'here': 0.66; 'close': 0.67; '8bit%:21': 0.69; 'sharing': 0.69; 'safe': 0.72; 'risk': 0.72; 'touch': 0.74; 'guaranteed': 0.75; 'saw': 0.77; 'vital': 0.78; '2015': 0.84; 'lighter': 0.84; 'quicker': 0.84; 'approach.': 0.91; 'careful': 0.91; 'prone': 0.91; 'state.': 0.95 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:references:from:date:message-id:subject:to :content-type; bh=tRPyNcEoWCnN+DM7uVG3yRdaWnYoSi3oY4LIeiNzQS0=; b=lMXMB5iu20SZc4GbD6ScjLfQazjvrBXrks/uNOmZZ3cZZhHlXJzY8CgDTk/Zljnxww eWrALgnaNe5jC7+a4ot3dLh/I6b+W469GCCPw2MPMELd+vpXm+y8WjFN8lQ8s5POsPFU IWAYsmmzduc/7OtLJAttxMDVr4++PngQbIaXl/jXuVMQSae+GurlkWHrll4MhgZudOXQ EUK9dPjEVYK12AyRu95AbCaPxwKBCF5cZeYewRM8HbwBYN1evrRcnQg+PgoyjoweyJZG EEtIO8s4Ycatb07MfqL3FcKxlpB8h9rQRq6e6Ze81q5TfeuU4xIBkNxaHmNJNnMeGRyY pt0A== X-Received: by 10.66.140.39 with SMTP id rd7mr10795218pab.25.1424676720958; Sun, 22 Feb 2015 23:32:00 -0800 (PST) MIME-Version: 1.0 References: <87fv9xdb22.fsf@jester.gateway.pace.com> <54ea7ff4$0$12983$c3e8da3$5496439d@news.astraweb.com> <87zj85bcyu.fsf@jester.gateway.pace.com> <87lhjpb89i.fsf@jester.gateway.pace.com> <87h9udb1eq.fsf@jester.gateway.pace.com> From: Ryan Stuart Date: Mon, 23 Feb 2015 07:32:00 +0000 Subject: Re: Future of Pypy? To: python-list@python.org Content-Type: multipart/alternative; boundary=001a1133248edf671a050fbc652b X-BeenThere: python-list@python.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: General discussion list for the Python programming language List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Newsgroups: comp.lang.python Message-ID: Lines: 106 NNTP-Posting-Host: 2001:888:2000:d::a6 X-Trace: 1424676730 news.xs4all.nl 2941 [2001:888:2000:d::a6]:35356 X-Complaints-To: abuse@xs4all.nl Xref: csiph.com comp.lang.python:86199 --001a1133248edf671a050fbc652b Content-Type: text/plain; charset=UTF-8 On Mon Feb 23 2015 at 4:15:42 PM Paul Rubin wrote: > > What do you mean about Queues working with processes? I meant > Queue.Queue. There is multiprocessing.Queue but that's much less > capable, and it uses cumbersome IPC like pipes or sockets instead of a > lighter weight lock. Threads can also share read-only data and you can > pass arbitrary objects (such as code callables that you want the other > thread to execute--this is quite useful) through Queue.Queue. I don't > think you can do that with the multiprocessing module. > These things might be convenient but they are error prone for the reasons pointed out. Also, the majority can be achieved via the process approach. For example, using fork to take a copy of the current process (including the heap) you want to use will give you access to any callables on the heap. The vital point here is that fork takes a *copy* of the process and runs in a *separate* memory space. This means there can be no accidents here. If it were to run in the same memory space like a thread then bugs anywhere in the code you run could cause very nasty problems. This includes not only bugs in your code, but also bugs in any other library code. Not only are they nasty, they could be close to invisible. And when I saw any code you run, I literally mean any. Even if you are extra careful to not touch any shared state in your code, you can almost be guaranteed that code higher up the stack, like malloc for example, *will* be using shared state. The risk of unintended and difficult to track issues when using threads is very high because of shared state. Even if you aren't sharing state in your code directly, code higher up the stack will be sharing state. That is the whole point of a thread, that's what they were invented for. Using threads safely might well be impossible much less verifiable. So when there are other options that are just as viable/functional, result in far less risk and are often much quicker to implement correctly, why wouldn't you use them? If it were easy to use threads in a verifiably safe manner, then there probably wouldn't be a GIL. Cheers > -- > https://mail.python.org/mailman/listinfo/python-list > --001a1133248edf671a050fbc652b Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
On Mon Feb 23 2015 at 4:15:42 P= M Paul Rubin <no.email@nospam.invalid> wrote:
What do you mean about Queues working with processes?=C2=A0 I meant
Queue.Queue.=C2=A0 There is multiprocessing.Queue but that's much less<= br> capable, and it uses cumbersome IPC like pipes or sockets instead of a
lighter weight lock.=C2=A0 Threads can also share read-only data and you ca= n
pass arbitrary objects (such as code callables that you want the other
thread to execute--this is quite useful) through Queue.Queue.=C2=A0 I don&#= 39;t
think you can do that with the multiprocessing module.

These things might be convenient but they are error prone f= or the reasons pointed out.=C2=A0Also, the majority can be achieved via the process approach. For example= , using fork to take a copy of the current process (including the heap) you= want to use will give you access to any callables on the heap.=C2=A0

<= div>The vital point here is th= at fork takes a *copy* of the process and runs in a *separate* memory space= . This means there can be no accidents here. If it were to run in the same = memory space like a thread then bugs anywhere in the code you run could cau= se very nasty problems. This includes not only bugs in your code, but also = bugs in any other library code. Not only are they nasty, they could be clos= e to invisible.

And w= hen I saw any code you run, I literally mean any. Even if you are extra car= eful to not touch any shared state in your code, you can almost be=C2=A0guaranteed=C2=A0that code = higher up the stack, like malloc for example, *will* be using shared state.=

<= /div>
The risk of unintend= ed and difficult to track issues when using threads is very high because of= shared state. Even if you aren't sharing state in your code directly, = code higher up the stack will be sharing state. That is the whole point of = a thread, that's what they were invented for. Using threads safely migh= t well be impossible much less verifiable. So when there are other options = that are just as viable/functional, result in far less risk and are often m= uch quicker to implement correctly, why wouldn't you use them? If it we= re easy to use threads in a=C2=A0verifiably=C2=A0safe manner, then there probably=C2=A0wouldn'= t be a GIL.
<= br>
Cheers
=C2=A0
--
https://mail.python.org/mailman/listinfo/python-list
--001a1133248edf671a050fbc652b--