Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #64013 > unrolled thread

Python Scalability TCP Server + Background Game

Back to article view | Back to comp.lang.python

Contents

  Python Scalability TCP Server + Background Game phiwer@gmail.com - 2014-01-15 10:37 -0800
    Re: Python Scalability TCP Server + Background Game Chris Angelico <rosuav@gmail.com> - 2014-01-16 08:08 +1100
      Re: Python Scalability TCP Server + Background Game phiwer@gmail.com - 2014-01-17 23:44 -0800
        Re: Python Scalability TCP Server + Background Game Chris Angelico <rosuav@gmail.com> - 2014-01-18 19:01 +1100
          Re: Python Scalability TCP Server + Background Game phiwer@gmail.com - 2014-01-18 03:54 -0800
    Re: Python Scalability TCP Server + Background Game Asaf Las <roegltd@gmail.com> - 2014-01-18 04:13 -0800
      Re: Python Scalability TCP Server + Background Game phiwer@gmail.com - 2014-01-18 04:40 -0800
        Re: Python Scalability TCP Server + Background Game Mark Lawrence <breamoreboy@yahoo.co.uk> - 2014-01-18 13:19 +0000
          Re: Python Scalability TCP Server + Background Game Philip Werner <phiwer@gmail.com> - 2014-01-19 12:53 -0600
            Re: Python Scalability TCP Server + Background Game Chris Angelico <rosuav@gmail.com> - 2014-01-20 05:58 +1100
              Re: Python Scalability TCP Server + Background Game Philip Werner <phiwer@gmail.com> - 2014-01-21 03:59 -0600

#64013 — Python Scalability TCP Server + Background Game

My problem is as follows: 

I'm developing an online game with the requirement of being able to handle thousands of requests every second.

The frontend consists of web server(s) exposing a rest api. These web servers in turn communicate with a game server over TCP. When a message arrives at the game server, each client handler inserts the client message into a shared message queue, and then waits for the result from the game loop. When the game loop has informed the waiting handler that a result is ready, the handler returns the result to the client.

Things to take note of:

1) The main game loop runs in a separate process, and the intention is to use a Queue from the multiprocess library.

2) The network layer of the game server runs a separate process as well, and my intention was to use gevent or tornado (http://nichol.as/asynchronous-servers-in-python).

3) The game server has a player limit of 50000. My requirement/desire is to be able to serve 50k requests per second (without any caching layer, although the game server will cache data), so people don't get a poor user experience during high peaks.

4) The game is not a real-time based game, but is catered towards the web.

And now to my little problem. All high performance async TCP servers use greenlets (or similar light threads), but does not seem to be compatible with the multiprocess library. From what I've read, Python (largely due to GIL) does not seem suited for this type of task, compared to other languages where threading and IPC is not an issue.

Due to this information, I have developed the initial server using netty in java. I would, however, rather develop the server using python, if possible. But if these limitations truly exist, then I'll go with java for the game server, and then use python for the frontend.

Has anyone developed something similar, and if so, could you point me in the right direction, or perhaps I've missed something along the way?

If one can solve the issue with IPC, it seems the high performance servers tested on this page, http://nichol.as/asynchronous-servers-in-python, only can handle roughly 8k requests per second before performance degrades. Does anyone know how Python high performance TCP servers compare to other language's TCP servers?

Thanks for all replies!

[toc] | [next] | [standalone]

#64017

On Thu, Jan 16, 2014 at 5:37 AM,  <phiwer@gmail.com> wrote:
> 3) The game server has a player limit of 50000. My requirement/desire is to be able to serve 50k requests per second (without any caching layer, although the game server will cache data), so people don't get a poor user experience during high peaks.

Quick smoke test. How big are your requests/responses? You mention
REST, which implies they're going to be based on HTTP. I would expect
you would have some idea of the rough size. Multiply that by 50,000,
and see whether your connection can handle it. For instance, if you
have a 100Mbit/s uplink, supporting 50K requests/sec means your
requests and responses have to fit within about 256 bytes each,
including all overhead. You'll need a gigabit uplink to be able to
handle a 2KB request or response, and that's assuming perfect
throughput. And is 2KB enough for you?

ChrisA

[toc] | [prev] | [next] | [standalone]

#64216

> Quick smoke test. How big are your requests/responses? You mention
> 
> REST, which implies they're going to be based on HTTP. I would expect
> 
> you would have some idea of the rough size. Multiply that by 50,000,
> 
> and see whether your connection can handle it. For instance, if you
> 
> have a 100Mbit/s uplink, supporting 50K requests/sec means your
> 
> requests and responses have to fit within about 256 bytes each,
> 
> including all overhead. You'll need a gigabit uplink to be able to
> 
> handle a 2KB request or response, and that's assuming perfect
> 
> throughput. And is 2KB enough for you?
> 
> 
> 
> ChrisA

My assumption is that there will be mostly reads and some writes; maybe in the order of 80-20%. There is a time element in the game, which forces player's entity to update on-demand. This is part of the reason why I wanted the server to be able to handle so many reques, so that it could handle the read part without having any caching layer.

Let me explain a bit more about the architecture, and possible remedies, to give you an idea:

* On top are the web servers exposing a REST api.

* At the bottom is the game.

* Communication betweeen these layers is handled by a simple text protocol using TCP.

The game has a tick function every now and then, which forwards the game's time. If a player enters the game, a message is sent to the game server (querying for player details), and if the game's tick is greater than the cached version of the player details, then the game updates the player details (and caches it).

This design obviously has its flaws. One being that both reads/writes has to pass through the game server. One way to remedy this is by using a new layer, on top of the game, which would hold the cache. But then another problem arises, that of invalidating the cache when a new tick has been made. I'm leaning towards letting the cache layer check the current tick every now and then, and if new tick is available, update a local variable in the cache (which each new connection would check against). Any thoughts regarding this?

There are some periods in the game, where many people will be online during the same tick, which could potentially cause the game to become slow at times, but maybe this should be accepted for the pleasure of making the game in python... :D

A follow-up question (which is more to the point really): How does other python game development frameworks solve this issue? Do they not use greenlets for the network layer, to be able to use the shared Queue from multiprocess? Do they only use one process for both network and game operations?

On a side note, I'm considering using 0MQ for the message layering between services (web-server <-> cache <-> game) on the back-end. Besides being a great message protocol, it also has built in queues which might be able to remedy the situation when many clients are requesting data. Anyone with experience with regards to this?

(This problem can be boggled down multiple producer, single consumer, and then back to producer again).

Thanks for all the replies.

/Phil

[toc] | [prev] | [next] | [standalone]

#64217

On Sat, Jan 18, 2014 at 6:44 PM,  <phiwer@gmail.com> wrote:
>> Quick smoke test. How big are your requests/responses? You mention
>>
>> REST, which implies they're going to be based on HTTP. I would expect
>>
>> you would have some idea of the rough size. Multiply that by 50,000,
>>
>> and see whether your connection can handle it. For instance, if you
>>
>> have a 100Mbit/s uplink, supporting 50K requests/sec means your
>>
>> requests and responses have to fit within about 256 bytes each,
>>
>> including all overhead. You'll need a gigabit uplink to be able to
>>
>> handle a 2KB request or response, and that's assuming perfect
>>
>> throughput. And is 2KB enough for you?
>>
>>
>>
>> ChrisA
>
> My assumption is that there will be mostly reads and some writes; maybe in the order of 80-20%. There is a time element in the game, which forces player's entity to update on-demand. This is part of the reason why I wanted the server to be able to handle so many reques, so that it could handle the read part without having any caching layer.
>

(You're using Google Groups, which means your replies are
double-spaced and your new text is extremely long lines. Please fix
this, either by the fairly manual job of fixing every post you make,
or the simple method of switching to a better client. Thanks.)

My point was just about the REST API, nothing else. You have to handle
a request and a response for every API call. Whether they're reads or
writes, you need to receive an HTTP request and send an HTTP response
for each one. In order to support the 50k requests per second you hope
for, you would have to handle 50k requests coming in and 50k responses
going out. To do that, you would need - at a very VERY rough estimate
- a maximum request size of 2KB and a gigabit internet connection
(which is expensive). No further calculations are worth pursuing if
you can't handle those requests and responses.

(And small requests tend to be more expensive than large ones. Since
you'll need a minimum of >SYN, <SYN/ACK, >ACK, >DATA, <DATA, >FIN,
<FIN/ACK, >ACK in order to transmit and receive one single packet's
worth of data each way, you're looking at roughly 8 packets minimum
for a one-byte message and one-byte response. But at a very very rough
estimate, 2KB each direction is the most you can sustain.)

ChrisA

[toc] | [prev] | [next] | [standalone]

#64225

> (You're using Google Groups, which means your replies are
> 
> double-spaced and your new text is extremely long lines. Please fix
> 
> this, either by the fairly manual job of fixing every post you make,
> 
> or the simple method of switching to a better client. Thanks.)
> 
> 
> 
> My point was just about the REST API, nothing else. You have to handle
> 
> a request and a response for every API call. Whether they're reads or
> 
> writes, you need to receive an HTTP request and send an HTTP response
> 
> for each one. In order to support the 50k requests per second you hope
> 
> for, you would have to handle 50k requests coming in and 50k responses
> 
> going out. To do that, you would need - at a very VERY rough estimate
> 
> - a maximum request size of 2KB and a gigabit internet connection
> 
> (which is expensive). No further calculations are worth pursuing if
> 
> you can't handle those requests and responses.
> 
> 
> 
> (And small requests tend to be more expensive than large ones. Since
> 
> you'll need a minimum of >SYN, <SYN/ACK, >ACK, >DATA, <DATA, >FIN,
> 
> <FIN/ACK, >ACK in order to transmit and receive one single packet's
> 
> worth of data each way, you're looking at roughly 8 packets minimum
> 
> for a one-byte message and one-byte response. But at a very very rough
> 
> estimate, 2KB each direction is the most you can sustain.)
> 
> 
> 
> ChrisA


Aha, thanks for the info.

But the assumptions you are making does not answer the question.

And the question you raise, although important, is more a financial one, 
not really relevant to the questions I posed.

Regards,
Phil

[toc] | [prev] | [next] | [standalone]

#64228

On Wednesday, January 15, 2014 8:37:25 PM UTC+2, phi...@gmail.com wrote:
> My problem is as follows: 
> ....
> 2) The network layer of the game server runs a separate process as well, 
> and my intention was to use gevent or tornado (http://nichol.as/asynchronous-
>servers-in-python).
> 3) The game server has a player limit of 50000. My requirement/desire is to 
> be able to serve 50k requests per second 
> 4) The game is not a real-time based game, but is catered towards the web.
> Due to this information, I have developed the initial server using netty in 
> java. I would, however, rather develop the server using python,... and then 
> use python for the frontend. ...

do you have references to implementations where python was used as frontend for such traffic load? 

you can have a look to this group for numbers
https://groups.google.com/forum/#!forum/framework-benchmarks

Asaf

[toc] | [prev] | [next] | [standalone]

#64229

Den lördagen den 18:e januari 2014 kl. 13:13:47 UTC+1 skrev Asaf Las:
> On Wednesday, January 15, 2014 8:37:25 PM UTC+2, phi...@gmail.com wrote:
> 
> > My problem is as follows: 
> 
> > ....
> 
> > 2) The network layer of the game server runs a separate process as well, 
> 
> > and my intention was to use gevent or tornado (http://nichol.as/asynchronous-
> 
> >servers-in-python).
> 
> > 3) The game server has a player limit of 50000. My requirement/desire is to 
> 
> > be able to serve 50k requests per second 
> 
> > 4) The game is not a real-time based game, but is catered towards the web.
> 
> > Due to this information, I have developed the initial server using netty in 
> 
> > java. I would, however, rather develop the server using python,... and then 
> 
> > use python for the frontend. ...
> 
> 
> 
> do you have references to implementations where python was used as frontend for such traffic load? 
> 
> 
> 
> you can have a look to this group for numbers
> 
> https://groups.google.com/forum/#!forum/framework-benchmarks
> 
> 
> 
> Asaf

Yes I've looked at those charts, and that was one of the reasons why I decided
to go with netty for the back-end (although I'm probing to see if python 
possibly could match in some way, which I haven't thought of yet...).

With regards to front-end, I haven't fully decided upon which framework to use.
It's easier to scale the front-end, however, given that the problem of
locking game state is not present in this logical part of the architecture.

[toc] | [prev] | [next] | [standalone]

#64230

On 18/01/2014 12:40, phiwer@gmail.com wrote:

[snip the stuff I can't help with]

Here's the link you need to sort the problem with double spacing from 
google groups https://wiki.python.org/moin/GoogleGroupsPython

-- 
My fellow Pythonistas, ask not what our language can do for you, ask 
what you can do for our language.

Mark Lawrence

[toc] | [prev] | [next] | [standalone]

#64310

On Sat, 18 Jan 2014 13:19:24 +0000, Mark Lawrence wrote:

> On 18/01/2014 12:40, phiwer@gmail.com wrote:
> 
> [snip the stuff I can't help with]
> 
> Here's the link you need to sort the problem with double spacing from
> google groups https://wiki.python.org/moin/GoogleGroupsPython

Thanks for the link. I've, hopefully, solved the issue by switching
to Pan instead of using google groups. :)

Regards,
Philip

[toc] | [prev] | [next] | [standalone]

#64311

On Mon, Jan 20, 2014 at 5:53 AM, Philip Werner <phiwer@gmail.com> wrote:
> On Sat, 18 Jan 2014 13:19:24 +0000, Mark Lawrence wrote:
>
>> On 18/01/2014 12:40, phiwer@gmail.com wrote:
>>
>> [snip the stuff I can't help with]
>>
>> Here's the link you need to sort the problem with double spacing from
>> google groups https://wiki.python.org/moin/GoogleGroupsPython
>
> Thanks for the link. I've, hopefully, solved the issue by switching
> to Pan instead of using google groups. :)

Looking a lot more normal and readable now. Thanks!

Note that some people have experienced odd issues with Pan, possibly
relating to having multiple instances running simultaneously. You may
want to take care not to let it open up a duplicate copy of itself.

ChrisA

[toc] | [prev] | [next] | [standalone]

#64393

> Looking a lot more normal and readable now. Thanks!
> 
> Note that some people have experienced odd issues with Pan, possibly
> relating to having multiple instances running simultaneously. You may
> want to take care not to let it open up a duplicate copy of itself.
> 
> ChrisA

Thanks for the heads up.

It is buggy to say the least. Any other program on linux you may suggest?

Regards,
Philip

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web