Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #90107

Multi-threaded TCP server class for general use

Show all headers | View raw

Hi,

I needed to develop a highly scalable multi-threaded TCP server in Python and when I started writing it in 2013 I could not find a suitable library that would scale the way I needed but also easy to use.

So I invented one - it's called Pyloom.  If you want to take a look, it's part of my DBIS project at the moment: https://sourceforge.net/p/dbis/code/ci/default/tree/src/pyloom

Question: does anyone see the value of this library?  Would you like me to fork it as a separate project?  Is SF.net good enough or do people prefer Python libraries to be hosted somewhere else?

In a nutshell, Pyloom is a multi-threaded TCP server class which you overload in your own program.  The library provides:

* Connection management.

* 1 or more marshal threads that listen and pick up new connections, passing to a pool of dedicated worker threads, each of which can manage multiple sessions.

* Methods that you override for handling various states: new connection, data received, send data, close connection.

* Can track custom sockets and wake up a session when there is I/O on the socket.

* Has a notification service so that one session can wait for data to be processed/collected by a different session, and can be woken up when the data is ready.

Let me know if anyone is interested in re-using this library.  Tested ok on Linux and Solaris, not tried on Windows yet.

Best regards,
Mark.

Back to comp.lang.python | Previous | Next — Next in thread | Find similar | Unroll thread

Thread

Multi-threaded TCP server class for general use mark.r.bannister@googlemail.com - 2015-05-07 15:20 -0700
  Re: Multi-threaded TCP server class for general use Chris Angelico <rosuav@gmail.com> - 2015-05-08 11:53 +1000

csiph-web