Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > comp.lang.python > #69792 > unrolled thread
| Started by | Ben Finney <ben+python@benfinney.id.au> |
|---|---|
| First post | 2014-04-07 13:05 +1000 |
| Last post | 2014-04-08 15:19 +0000 |
| Articles | 20 on this page of 105 — 22 participants |
Back to article view | Back to comp.lang.python
This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by
below is the oldest one visible, not the original post.
Re: threading Ben Finney <ben+python@benfinney.id.au> - 2014-04-07 13:05 +1000
Re: threading Roy Smith <roy@panix.com> - 2014-04-06 23:48 -0400
Re: threading Chris Angelico <rosuav@gmail.com> - 2014-04-07 13:56 +1000
Re: threading Roy Smith <roy@panix.com> - 2014-04-07 08:26 -0400
Re: threading Chris Angelico <rosuav@gmail.com> - 2014-04-07 22:34 +1000
Re: threading Roy Smith <roy@panix.com> - 2014-04-07 09:22 -0400
Re: threading Mark Lawrence <breamoreboy@yahoo.co.uk> - 2014-04-07 14:41 +0100
Re: threading Marko Rauhamaa <marko@pacujo.net> - 2014-04-07 16:49 +0300
Re: threading Chris Angelico <rosuav@gmail.com> - 2014-04-08 00:27 +1000
Re: threading Marko Rauhamaa <marko@pacujo.net> - 2014-04-07 17:51 +0300
Re: threading Chris Angelico <rosuav@gmail.com> - 2014-04-08 01:12 +1000
Re: threading Chris Angelico <rosuav@gmail.com> - 2014-04-08 00:24 +1000
Re: threading Rick Johnson <rantingrickjohnson@gmail.com> - 2014-04-08 18:09 -0700
Re: threading "Neil D. Cerutti" <neilc@norwich.edu> - 2014-04-09 09:50 -0400
Re: threading Rick Johnson <rantingrickjohnson@gmail.com> - 2014-04-09 08:51 -0700
Re: threading MRAB <python@mrabarnett.plus.com> - 2014-04-09 18:47 +0100
Re: threading Gregory Ewing <greg.ewing@canterbury.ac.nz> - 2014-04-10 11:35 +1200
Re: threading Roy Smith <roy@panix.com> - 2014-04-09 19:53 -0400
Re: threading Andrew Berg <robotsondrugs@gmail.com> - 2014-04-09 19:02 -0500
Re: threading Steven D'Aprano <steve@pearwood.info> - 2014-04-10 02:43 +0000
Re: threading Chris Angelico <rosuav@gmail.com> - 2014-04-10 13:08 +1000
Re: threading Mark Lawrence <breamoreboy@yahoo.co.uk> - 2014-04-10 09:23 +0100
Re: threading Chris Angelico <rosuav@gmail.com> - 2014-04-10 19:11 +1000
Re: threading Chris Angelico <rosuav@gmail.com> - 2014-04-10 04:00 +1000
Re: threading Steven D'Aprano <steve@pearwood.info> - 2014-04-10 03:44 +0000
Re: threading Chris Angelico <rosuav@gmail.com> - 2014-04-10 13:54 +1000
Re: threading Ben Finney <ben+python@benfinney.id.au> - 2014-04-07 15:22 +1000
Re: threading Ethan Furman <ethan@stoneleaf.us> - 2014-04-08 11:09 -0700
Re: threading Sturla Molden <sturla.molden@gmail.com> - 2014-04-08 21:41 +0200
Re: threading Grant Edwards <invalid@invalid.invalid> - 2014-04-08 20:30 +0000
Re: threading Sturla Molden <sturla.molden@gmail.com> - 2014-04-09 00:32 +0200
Re: threading Rustom Mody <rustompmody@gmail.com> - 2014-04-08 19:17 -0700
Re: threading Marko Rauhamaa <marko@pacujo.net> - 2014-04-07 08:10 +0300
Re: threading Paul Rubin <no.email@nospam.invalid> - 2014-04-06 22:39 -0700
Re: threading Marko Rauhamaa <marko@pacujo.net> - 2014-04-07 08:46 +0300
Re: threading Dennis Lee Bieber <wlfraed@ix.netcom.com> - 2014-04-07 19:47 -0400
Re: threading Marko Rauhamaa <marko@pacujo.net> - 2014-04-08 08:19 +0300
Re: threading Sturla Molden <sturla.molden@gmail.com> - 2014-04-08 10:47 +0000
Re: threading Marko Rauhamaa <marko@pacujo.net> - 2014-04-08 15:10 +0300
Re: threading Sturla Molden <sturla.molden@gmail.com> - 2014-04-08 16:37 +0000
Re: threading Marko Rauhamaa <marko@pacujo.net> - 2014-04-08 20:17 +0300
Re: threading Roy Smith <roy@panix.com> - 2014-04-08 09:19 -0400
Re: threading Sturla Molden <sturla.molden@gmail.com> - 2014-04-08 15:44 +0000
Re: threading Paul Rubin <no.email@nospam.invalid> - 2014-04-08 09:38 -0700
Re: threading Mark Lawrence <breamoreboy@yahoo.co.uk> - 2014-04-09 14:42 +0100
Re: threading "Frank Millman" <frank@chagford.com> - 2014-04-09 15:23 +0200
Re: threading Marko Rauhamaa <marko@pacujo.net> - 2014-04-09 16:55 +0300
Re: threading "Frank Millman" <frank@chagford.com> - 2014-04-09 16:46 +0200
Re: threading Marko Rauhamaa <marko@pacujo.net> - 2014-04-09 20:31 +0300
Re: threading Chris Angelico <rosuav@gmail.com> - 2014-04-10 03:52 +1000
Re: threading Mark H Harris <harrismh777@gmail.com> - 2014-04-10 08:29 -0500
Re: threading Sturla Molden <sturla.molden@gmail.com> - 2014-04-09 19:20 +0000
Re: threading Chris Angelico <rosuav@gmail.com> - 2014-04-09 23:47 +1000
Re: threading Roy Smith <roy@panix.com> - 2014-04-09 10:44 -0400
Re: threading "Frank Millman" <frank@chagford.com> - 2014-04-09 16:30 +0200
Re: threading Roy Smith <roy@panix.com> - 2014-04-09 10:52 -0400
Re: threading Gregory Ewing <greg.ewing@canterbury.ac.nz> - 2014-04-10 11:19 +1200
Re: threading Marko Rauhamaa <marko@pacujo.net> - 2014-04-09 19:48 +0300
Re: threading Chris Angelico <rosuav@gmail.com> - 2014-04-10 00:44 +1000
Re: threading Sturla Molden <sturla.molden@gmail.com> - 2014-04-09 15:29 +0000
Re: threading Terry Reedy <tjreedy@udel.edu> - 2014-04-09 12:14 -0400
Re: threading Chris Angelico <rosuav@gmail.com> - 2014-04-10 02:25 +1000
Re: threading Sturla Molden <sturla.molden@gmail.com> - 2014-04-09 16:32 +0000
Re: threading Dennis Lee Bieber <wlfraed@ix.netcom.com> - 2014-04-09 19:44 -0400
Re: threading Chris Angelico <rosuav@gmail.com> - 2014-04-10 11:05 +1000
Re: threading "Frank Millman" <frank@chagford.com> - 2014-04-10 11:17 +0200
Re: threading Chris Angelico <rosuav@gmail.com> - 2014-04-10 19:40 +1000
Re: threading "Frank Millman" <frank@chagford.com> - 2014-04-10 13:10 +0200
Re: threading Marko Rauhamaa <marko@pacujo.net> - 2014-04-10 14:43 +0300
Re: threading Roy Smith <roy@panix.com> - 2014-04-10 08:56 -0400
Re: threading Sturla Molden <sturla.molden@gmail.com> - 2014-04-10 15:24 +0000
Re: threading Marko Rauhamaa <marko@pacujo.net> - 2014-04-10 19:20 +0300
Re: threading Chris Angelico <rosuav@gmail.com> - 2014-04-11 01:32 +1000
Re: threading Marko Rauhamaa <marko@pacujo.net> - 2014-04-10 19:25 +0300
Re: threading Chris Angelico <rosuav@gmail.com> - 2014-04-11 03:08 +1000
Re: threading Rustom Mody <rustompmody@gmail.com> - 2014-04-10 11:14 -0700
Re: threading Marko Rauhamaa <marko@pacujo.net> - 2014-04-10 22:44 +0300
Re: threading Rustom Mody <rustompmody@gmail.com> - 2014-04-10 13:21 -0700
Re: threading Marko Rauhamaa <marko@pacujo.net> - 2014-04-10 23:44 +0300
Re: threading Rustom Mody <rustompmody@gmail.com> - 2014-04-10 22:15 -0700
Re: threading Rustom Mody <rustompmody@gmail.com> - 2014-04-10 23:50 -0700
Re: threading Marko Rauhamaa <marko@pacujo.net> - 2014-04-11 18:36 +0300
Re: threading Chris Angelico <rosuav@gmail.com> - 2014-04-12 01:53 +1000
Re: threading Mark Lawrence <breamoreboy@yahoo.co.uk> - 2014-04-11 16:58 +0100
Re: threading Rustom Mody <rustompmody@gmail.com> - 2014-04-11 11:54 -0700
Re: threading Marko Rauhamaa <marko@pacujo.net> - 2014-04-11 22:27 +0300
Re: threading Sturla Molden <sturla.molden@gmail.com> - 2014-04-11 01:51 +0200
Re: threading Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-04-11 05:35 +0000
Re: threading Sturla Molden <sturla.molden@gmail.com> - 2014-04-11 09:26 +0000
Re: threading Roy Smith <roy@panix.com> - 2014-04-11 08:36 -0400
Re: threading Grant Edwards <invalid@invalid.invalid> - 2014-04-11 16:18 +0000
Re: threading Sturla Molden <sturla.molden@gmail.com> - 2014-04-11 02:21 +0200
Re: threading Terry Reedy <tjreedy@udel.edu> - 2014-04-10 20:23 -0400
Re: threading Chris Angelico <rosuav@gmail.com> - 2014-04-10 21:19 +1000
Re: threading Sturla Molden <sturla.molden@gmail.com> - 2014-04-08 02:06 +0000
Re: threading alister <alister.nospam.ware@ntlworld.com> - 2014-04-08 11:07 +0000
Re: threading Roy Smith <roy@panix.com> - 2014-04-08 09:13 -0400
Re: threading Chris Angelico <rosuav@gmail.com> - 2014-04-08 23:23 +1000
Re: threading alister <alister.nospam.ware@ntlworld.com> - 2014-04-08 14:15 +0000
Re: threading Sturla Molden <sturla.molden@gmail.com> - 2014-04-08 16:06 +0000
Re: threading Sturla Molden <sturla.molden@gmail.com> - 2014-04-08 15:40 +0000
Re: threading Paul Rubin <no.email@nospam.invalid> - 2014-04-08 09:46 -0700
Re: threading Chris Angelico <rosuav@gmail.com> - 2014-04-09 02:46 +1000
Re: threading Sturla Molden <sturla.molden@gmail.com> - 2014-04-08 17:17 +0000
Re: threading Sturla Molden <sturla.molden@gmail.com> - 2014-04-08 15:19 +0000
Page 5 of 6 — ← Prev page 1 2 3 4 [5] 6 Next page →
| From | Rustom Mody <rustompmody@gmail.com> |
|---|---|
| Date | 2014-04-10 23:50 -0700 |
| Message-ID | <19d537ee-dcc3-4265-8fce-bd24201e3dc4@googlegroups.com> |
| In reply to | #70092 |
On Friday, April 11, 2014 10:45:08 AM UTC+5:30, Rustom Mody wrote: > On Friday, April 11, 2014 2:14:42 AM UTC+5:30, Marko Rauhamaa wrote: > > (1) oversimplification which makes it difficult to extend the design > > and handle all of the real-world contingencies > > This I dont... I meant "Dont understand"
[toc] | [prev] | [next] | [standalone]
| From | Marko Rauhamaa <marko@pacujo.net> |
|---|---|
| Date | 2014-04-11 18:36 +0300 |
| Message-ID | <8761mfnai8.fsf@elektro.pacujo.net> |
| In reply to | #70106 |
Rustom Mody <rustompmody@gmail.com>: > On Friday, April 11, 2014 10:45:08 AM UTC+5:30, Rustom Mody wrote: >> On Friday, April 11, 2014 2:14:42 AM UTC+5:30, Marko Rauhamaa wrote: >> > (1) oversimplification which makes it difficult to extend the design >> > and handle all of the real-world contingencies >> >> This I dont... > > I meant "Dont understand" The simplest example: there's no general way to terminate a thread. Hacks exist for some occasions, but they can hardly be considered graceful. Marko
[toc] | [prev] | [next] | [standalone]
| From | Chris Angelico <rosuav@gmail.com> |
|---|---|
| Date | 2014-04-12 01:53 +1000 |
| Message-ID | <mailman.9209.1397231624.18130.python-list@python.org> |
| In reply to | #70147 |
On Sat, Apr 12, 2014 at 1:36 AM, Marko Rauhamaa <marko@pacujo.net> wrote: > The simplest example: there's no general way to terminate a thread. > Hacks exist for some occasions, but they can hardly be considered > graceful. Having followed python-list for some time now, I have to agree... you can't terminate a thread, no matter how many posts it has in it! ChrisA
[toc] | [prev] | [next] | [standalone]
| From | Mark Lawrence <breamoreboy@yahoo.co.uk> |
|---|---|
| Date | 2014-04-11 16:58 +0100 |
| Message-ID | <mailman.9211.1397232006.18130.python-list@python.org> |
| In reply to | #70147 |
On 11/04/2014 16:53, Chris Angelico wrote: > On Sat, Apr 12, 2014 at 1:36 AM, Marko Rauhamaa <marko@pacujo.net> wrote: >> The simplest example: there's no general way to terminate a thread. >> Hacks exist for some occasions, but they can hardly be considered >> graceful. > > Having followed python-list for some time now, I have to agree... you > can't terminate a thread, no matter how many posts it has in it! > > ChrisA > Wro -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence --- This email is free from viruses and malware because avast! Antivirus protection is active. http://www.avast.com
[toc] | [prev] | [next] | [standalone]
| From | Rustom Mody <rustompmody@gmail.com> |
|---|---|
| Date | 2014-04-11 11:54 -0700 |
| Message-ID | <22978792-bb74-4d98-b840-0cde3ee6a9f4@googlegroups.com> |
| In reply to | #70147 |
On Friday, April 11, 2014 9:06:47 PM UTC+5:30, Marko Rauhamaa wrote: > Rustom Mody: > > On Friday, April 11, 2014 10:45:08 AM UTC+5:30, Rustom Mody wrote: > > >> On Friday, April 11, 2014 2:14:42 AM UTC+5:30, Marko Rauhamaa wrote: > > >> > (1) oversimplification which makes it difficult to extend the design > >> > and handle all of the real-world contingencies > >> > >> This I dont... > The simplest example: there's no general way to terminate a thread. > Hacks exist for some occasions, but they can hardly be considered > graceful. I was about to say that this is fairly close to my point, viz: Half-assed support in current languages does not imply any necessary problem in the idea -- just in the mainstream implementations of it. Then looking it up I find Go's goroutines have the same issue. Erlang processes though, are kill-able like quite any unix process http://www.erlang.org/doc/reference_manual/processes.html#id85098 What does that mean?? I am not quite sure... It may mean that Steven's > I think coroutines are awesome, but like all advanced concepts, sometimes > they can be abused, and sometimes they are hard to understand not because > they are hard to understand in and of themselves, but because they are > being used to do something inherently complicated. is right. Personally my view is the other way round: Concurrency (generalizing from coroutines) is not hard. The problem is that imperative programming and concurrency are deeply incompatible because imperative programming is almost by definition sequential programming and concurrency is by definition concurrent, ie non-sequential
[toc] | [prev] | [next] | [standalone]
| From | Marko Rauhamaa <marko@pacujo.net> |
|---|---|
| Date | 2014-04-11 22:27 +0300 |
| Message-ID | <87eh13k6p2.fsf@elektro.pacujo.net> |
| In reply to | #70163 |
Rustom Mody <rustompmody@gmail.com>: > Half-assed support in current languages does not imply any necessary > problem in the idea -- just in the mainstream implementations of it. > > Then looking it up I find Go's goroutines have the same issue. the promise of threads is that you only need to consider a single event out of any given state. Trouble is, in reality, you need ot consider a multitude of events in every state. Threads are not good at presenting that reality. Marko
[toc] | [prev] | [next] | [standalone]
| From | Sturla Molden <sturla.molden@gmail.com> |
|---|---|
| Date | 2014-04-11 01:51 +0200 |
| Message-ID | <mailman.9158.1397173921.18130.python-list@python.org> |
| In reply to | #70058 |
On 10/04/14 21:44, Marko Rauhamaa wrote:
> I'm happy that asyncio is
> happening after all these long years. It would be nice if it supported
> edge-triggered wakeups, but I suppose that isn't supported in all
> operating systems.
I have an issue with the use of coroutines. I think they are to evil.
When a man like David Beazley says this
https://twitter.com/dabeaz/status/440214755764994048
there is something crazy going on.
Sturla
[toc] | [prev] | [next] | [standalone]
| From | Steven D'Aprano <steve+comp.lang.python@pearwood.info> |
|---|---|
| Date | 2014-04-11 05:35 +0000 |
| Message-ID | <53477f29$0$29993$c3e8da3$5496439d@news.astraweb.com> |
| In reply to | #70075 |
On Fri, 11 Apr 2014 01:51:41 +0200, Sturla Molden wrote: > On 10/04/14 21:44, Marko Rauhamaa wrote: >> I'm happy that asyncio is >> happening after all these long years. It would be nice if it supported >> edge-triggered wakeups, but I suppose that isn't supported in all >> operating systems. > > I have an issue with the use of coroutines. I think they are to evil. They are to evil ... as what? To evil as chalk is to cheese? Black is to white? Bees are to honey? I think coroutines are awesome, but like all advanced concepts, sometimes they can be abused, and sometimes they are hard to understand not because they are hard to understand in and of themselves, but because they are being used to do something inherently complicated. > When a man like David Beazley says this > > https://twitter.com/dabeaz/status/440214755764994048 > > there is something crazy going on. Good lord!!! David Beazley has been consumed by the Dark Side and uses Twitter??? There certainly is something crazy going on! -- Steven D'Aprano http://import-that.dreamwidth.org/
[toc] | [prev] | [next] | [standalone]
| From | Sturla Molden <sturla.molden@gmail.com> |
|---|---|
| Date | 2014-04-11 09:26 +0000 |
| Message-ID | <mailman.9190.1397208606.18130.python-list@python.org> |
| In reply to | #70094 |
Steven D'Aprano <steve+comp.lang.python@pearwood.info> wrote: >> I have an issue with the use of coroutines. I think they are to evil. > > They are to evil ... as what? To evil as chalk is to cheese? Black is to > white? Bees are to honey? I think coroutines are one of those things that don't fit the human mind. A class with a couple of queues for input and output is far easier to comprehend. Sturla
[toc] | [prev] | [next] | [standalone]
| From | Roy Smith <roy@panix.com> |
|---|---|
| Date | 2014-04-11 08:36 -0400 |
| Message-ID | <roy-F234BE.08362211042014@news.panix.com> |
| In reply to | #70094 |
In article <53477f29$0$29993$c3e8da3$5496439d@news.astraweb.com>, Steven D'Aprano <steve+comp.lang.python@pearwood.info> wrote: > I think coroutines are awesome, but like all advanced concepts, sometimes > they can be abused, and sometimes they are hard to understand not because > they are hard to understand in and of themselves, but because they are > being used to do something inherently complicated. Advanced, perhaps. But certainly not new. The first "real" computer I worked on (a pdp-11/45) had a hardware instruction which swapped execution contexts. The documentation described it as being designed to support coroutines. That's a machine which was designed in the early 1970s. Heh, Wikipedia's [[Coroutine]] article says, "The term coroutine was coined by Melvin Conway in a 1963 paper". At a high level, threads and coroutines are really very similar. They are both independent execution paths in the same process. I guess the only real difference between them is that thread switching is mediated by the operating system, so it can happen anywhere (i.e. at any instruction boundary). Coroutines scheduling is handled in user code, so you have a lot more control over when context switches happen. This makes it a lot easier to manage operations which must occur atomically. They both operate in the same process memory space, so have the potential to stomp on each others data structures. They also both have the property that there is flow of control happening which is not apparent from a top-down reading of the code (exceptions, to a certain extent, have this same problem). This is fundamentally what makes them difficult to understand.
[toc] | [prev] | [next] | [standalone]
| From | Grant Edwards <invalid@invalid.invalid> |
|---|---|
| Date | 2014-04-11 16:18 +0000 |
| Message-ID | <li94jv$4du$2@reader1.panix.com> |
| In reply to | #70129 |
On 2014-04-11, Roy Smith <roy@panix.com> wrote:
> At a high level, threads and coroutines are really very similar. They
> are both independent execution paths in the same process. I guess the
> only real difference between them is that thread switching is mediated
> by the operating system, so it can happen anywhere (i.e. at any
> instruction boundary).
That's only true if your threading system has pre-emption. Python's
does, but not all do. If your threading system is cooperative rather
than preemptive, then using coroutines is completely idential to
threading with 2 threads.
> Coroutines scheduling is handled in user code,
As is cooperative multithreading.
--
Grant Edwards grant.b.edwards Yow! My BIOLOGICAL ALARM
at CLOCK just went off ... It
gmail.com has noiseless DOZE FUNCTION
and full kitchen!!
[toc] | [prev] | [next] | [standalone]
| From | Sturla Molden <sturla.molden@gmail.com> |
|---|---|
| Date | 2014-04-11 02:21 +0200 |
| Message-ID | <mailman.9159.1397175701.18130.python-list@python.org> |
| In reply to | #70058 |
On 11/04/14 01:51, Sturla Molden wrote: > I have an issue with the use of coroutines. I think they are to evil. > > When a man like David Beazley says this > > https://twitter.com/dabeaz/status/440214755764994048 > > there is something crazy going on. And why did Python get this Tulip beast, instead of a simple libuv wrapper? Node.js has already proven what libuv can do. Sturla
[toc] | [prev] | [next] | [standalone]
| From | Terry Reedy <tjreedy@udel.edu> |
|---|---|
| Date | 2014-04-10 20:23 -0400 |
| Message-ID | <mailman.9160.1397175861.18130.python-list@python.org> |
| In reply to | #70058 |
On 4/10/2014 7:51 PM, Sturla Molden wrote: > On 10/04/14 21:44, Marko Rauhamaa wrote: >> I'm happy that asyncio is >> happening after all these long years. It would be nice if it supported >> edge-triggered wakeups, but I suppose that isn't supported in all >> operating systems. > > I have an issue with the use of coroutines. I think they are to evil. I think 'magical' is the word you should be looking for. > When a man like David Beazley says this > > https://twitter.com/dabeaz/status/440214755764994048 > > there is something crazy going on. There is understanding how to use them, and understanding how they work in this context. I suspect that DB was talking about the second, deeper level. -- Terry Jan Reedy
[toc] | [prev] | [next] | [standalone]
| From | Chris Angelico <rosuav@gmail.com> |
|---|---|
| Date | 2014-04-10 21:19 +1000 |
| Message-ID | <mailman.9132.1397128759.18130.python-list@python.org> |
| In reply to | #69832 |
On Thu, Apr 10, 2014 at 9:10 PM, Frank Millman <frank@chagford.com> wrote: > You seem to be suggesting that I set the socket to 'non-blocking', use > select() to determine when a client is trying to connect, and then call > 'accept' on it to create a new connection. Right! That's about how it goes. Of course, for production work you'll want to wrap all that up for convenience, but fundamentally, that is what happens. ChrisA
[toc] | [prev] | [next] | [standalone]
| From | Sturla Molden <sturla.molden@gmail.com> |
|---|---|
| Date | 2014-04-08 02:06 +0000 |
| Message-ID | <mailman.8989.1396922794.18130.python-list@python.org> |
| In reply to | #69801 |
Dennis Lee Bieber <wlfraed@ix.netcom.com> wrote: > That's been my experience too... Threading works for me... My attempts > at so called asyncio (whatever language) have always led to my having to > worry about losing data if some handler takes too long to return. > > To me, asyncio is closer to a polling interrupt handler, and I still > need a thread to handle the main processing. On a 32 bit system, the virtual address space with limit the scalabiliy for multi-threading in parallel i/o. That is, the stack space for each thread can quickly become a problem. Memory is not a problem on 64-bit servers. Multithreading can be used to solve the C10K problem, contrary to common belief. Before you dissmiss threads, take a look at this: http://www.mailinator.com/tymaPaulMultithreaded.pdf http://stackoverflow.com/questions/17593699/tcp-ip-solving-the-c10k-with-the-thread-per-client-approach My personal feeling is that asynchronous i/o is mostly useful on 32-bit systems, and the problem it actually solves is the limited virtual address space. On a 64 bit system we can just throw more RAM at it and threads be fine. Sturla
[toc] | [prev] | [next] | [standalone]
| From | alister <alister.nospam.ware@ntlworld.com> |
|---|---|
| Date | 2014-04-08 11:07 +0000 |
| Message-ID | <QLQ0v.62026$oI.9712@fx18.am4> |
| In reply to | #69825 |
> My personal feeling is that asynchronous i/o is mostly useful on 32-bit > systems, and the problem it actually solves is the limited virtual > address space. On a 64 bit system we can just throw more RAM at it and > threads be fine. > As my only professional coding experience has been with embedded 8 bit processors with limited resources i naturally abhorrent to the process of "Just throw more RAM (Or any other resource for that matter)at it". It is my personal opinion that the quality of code has diminished steadily as physical limitations on the programmer have been reduced, of course I could be wrong so I would like to here other peoples views on this.
[toc] | [prev] | [next] | [standalone]
| From | Roy Smith <roy@panix.com> |
|---|---|
| Date | 2014-04-08 09:13 -0400 |
| Message-ID | <roy-B9EA87.09132408042014@news.panix.com> |
| In reply to | #69853 |
In article <QLQ0v.62026$oI.9712@fx18.am4>, alister <alister.nospam.ware@ntlworld.com> wrote: > > My personal feeling is that asynchronous i/o is mostly useful on 32-bit > > systems, and the problem it actually solves is the limited virtual > > address space. On a 64 bit system we can just throw more RAM at it and > > threads be fine. > > > As my only professional coding experience has been with embedded 8 bit > processors with limited resources i naturally abhorrent to the process of > "Just throw more RAM (Or any other resource for that matter)at it". > > It is my personal opinion that the quality of code has diminished > steadily as physical limitations on the programmer have been reduced, of > course I could be wrong so I would like to here other peoples views on > this. I can rent a machine with 30 GB (r3.xlarge) of RAM from Amazon for about $3000/year (http://aws.amazon.com/ec2/pricing/). A 60 GB machine (r3.2xlarge) is exactly twice that. You get more CPU and SSD too, but for the moment, I'm thinking about this as just renting memory. Glassdoor.com says the median salary for a "software architect" New York is $115k. By the time I'm done with personnel-related overhead, that person is really going to cost me $150k. Let's say I've got a program which consumes 60 GB of RAM, so I'm renting the 2xlarge instance to run it. My software architect could recode the program to be more efficient, and fit into just 30 GB, saving me $3000/year. How much of his time is it worth to do that? He's costing me about $600/day, so if he can do it in a week, it'll take a year to recoup my investment. Oh, by the way, those prices I quoted are the on-demand prices. If I'm willing to sign a three year contract, I can cut the annual cost almost in half. And the cost of hardware keeps going down, while the cost of good programmers keeps going up.
[toc] | [prev] | [next] | [standalone]
| From | Chris Angelico <rosuav@gmail.com> |
|---|---|
| Date | 2014-04-08 23:23 +1000 |
| Message-ID | <mailman.9011.1396963393.18130.python-list@python.org> |
| In reply to | #69857 |
On Tue, Apr 8, 2014 at 11:13 PM, Roy Smith <roy@panix.com> wrote: > And the cost of hardware keeps going down, while the cost of > good programmers keeps going up. Like everything, it's a matter of trade-offs. Spending a moment thinking about what you're doing before you do it might cut your RAM usage by 20% without materially increasing your programmer time cost. But coding everything in C "for efficiency" is almost certainly a bad trade, for the reason quoted. There can be special boundaries in resource consumption. Fitting something inside an Amazon Micro instance might be important for you, in which case you get just 613 MB of RAM (a weird figure, I know). Or maybe you need to stay within 8GB so you can allocate the other 8GB to the database. Or whatever it is. But any time it's flexible (where you basically just pay twice as much to get twice as much), it's usually worth going as far as you can. ChrisA
[toc] | [prev] | [next] | [standalone]
| From | alister <alister.nospam.ware@ntlworld.com> |
|---|---|
| Date | 2014-04-08 14:15 +0000 |
| Message-ID | <EvT0v.62257$oI.41490@fx18.am4> |
| In reply to | #69860 |
On Tue, 08 Apr 2014 23:23:05 +1000, Chris Angelico wrote: > On Tue, Apr 8, 2014 at 11:13 PM, Roy Smith <roy@panix.com> wrote: >> And the cost of hardware keeps going down, while the cost of good >> programmers keeps going up. > > Like everything, it's a matter of trade-offs. Spending a moment thinking > about what you're doing before you do it might cut your RAM usage by 20% > without materially increasing your programmer time cost. But coding > everything in C "for efficiency" is almost certainly a bad trade, for > the reason quoted. > > There can be special boundaries in resource consumption. Fitting > something inside an Amazon Micro instance might be important for you, in > which case you get just 613 MB of RAM (a weird figure, I know). Or maybe > you need to stay within 8GB so you can allocate the other 8GB to the > database. Or whatever it is. But any time it's flexible (where you > basically just pay twice as much to get twice as much), it's usually > worth going as far as you can. > > ChrisA My main point was that when you have only 8K of ROM & 128 byte of ram you have to think about your algorithms first. Programming is in Assembler because there is no other choice (I would not like to go back to that Python is so much more enjoyable), do you calculate a result or use a look up table (calculating costs ram & there are only a few bytes available for temp storage) but a lookup table costs ROM space. I am sure lack of constraint is what has lead to a number of the abominations on thedailywtf.com. I must admit I can also see the other side of the coin in that it can sometimes lead to "Clever Programming"* which is rarely a good idea. *Programming that makes use of side effects or other trickery that may not be obvious when read later. -- Avoid the Gates of Hell. Use Linux -- unknown source
[toc] | [prev] | [next] | [standalone]
| From | Sturla Molden <sturla.molden@gmail.com> |
|---|---|
| Date | 2014-04-08 16:06 +0000 |
| Message-ID | <mailman.9026.1396973192.18130.python-list@python.org> |
| In reply to | #69867 |
alister <alister.nospam.ware@ntlworld.com> wrote: > My main point was that when you have only 8K of ROM & 128 byte of ram you > have to think about your algorithms first. And that also happens if you have a 32 bit system, with just 2 GB of memory available to user space (the operating system reserves the upper half of the 4 GB address space). Using threads to solve a C10K problem means you must limit yourself to e.g. 150KB of memory (including stack space) per thread. Is that doable? Perhaps not. Can you put 10,000 threads in the kernel and hope for good performance? On Windows, yes, at least since Windows 2000. On Linux? Only recently. That I think was the real argument for asynchronous i/o. Today this is not so relevant, but idea that asynchronous i/o is fundamentally better than threads is still with us. So what we are left with, then, in favor of non-blocking i/o is ability to time-out a non-responsive i/o operation. But that does not mean we need an asynchronous event-driven server. (Actually we can time-out blocking i/o with threads, but it involves using two threads per connection instead of one.) Sturla
[toc] | [prev] | [next] | [standalone]
Page 5 of 6 — ← Prev page 1 2 3 4 [5] 6 Next page →
Back to top | Article view | comp.lang.python
csiph-web