Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.os.linux.networking > #2584 > unrolled thread

linux multi-port server design in c/c++

Back to article view | Back to comp.os.linux.networking

Contents

  linux multi-port server design in c/c++ noloader@gmail.com - 2013-11-25 22:39 -0800
    Re: linux multi-port server design in c/c++ David Brown <david@westcontrol.removethisbit.com> - 2013-11-26 09:48 +0100
      Re: linux multi-port server design in c/c++ noloader@gmail.com - 2013-11-26 09:47 -0800
        Re: linux multi-port server design in c/c++ David Brown <david@westcontrol.removethisbit.com> - 2013-11-27 10:56 +0100
          Re: linux multi-port server design in c/c++ Aragorn <stryder@telenet.be.invalid> - 2013-11-27 12:14 +0100
            Re: linux multi-port server design in c/c++ David Brown <david@westcontrol.removethisbit.com> - 2013-11-27 13:58 +0100
      Re: linux multi-port server design in c/c++ Jorgen Grahn <grahn+nntp@snipabacken.se> - 2013-12-08 01:04 +0000
    Re: linux multi-port server design in c/c++ Chris Davies <chris-usenet@roaima.co.uk> - 2013-11-26 17:03 +0000
      Re: linux multi-port server design in c/c++ noloader@gmail.com - 2013-11-26 09:33 -0800
    Re: linux multi-port server design in c/c++ Jorgen Grahn <grahn+nntp@snipabacken.se> - 2013-12-08 01:25 +0000
    Re: linux multi-port server design in c/c++ Dave <noone$$@llondel.org> - 2013-12-08 15:12 -0800

#2584 — linux multi-port server design in c/c++

Hi All,

Is anyone aware of a discussion or treatment of a multi-port threaded server on modern Linux boxes written in c/c++? Multi-port would mean, for example, ports 80 and 443.

Contemporary Linux and Unix post-date W. Richard Steven's Advanced Programming in the Unix Environment and Unix Network Programming, so my bibles are not helpful. Googling is returning the Comp Sci 101 examples with a single server/single port.

Specifically, I'm interested in discussions of:

(1) multiple processes servicing a single port and thread the accept()
(2) single process with multiple threads for each port and thread the accept()
(3) the most efficient way to wait and handle connections
(4) how to safely shutdown a multi-port, multi-threaded server

I'm fairly certain fork/exec is no longer recommended, especially since critical libraries like OpenSSL are not fork-safe in a multi-threaded environment.

I also think item (3) can be handled with libevent, but I'm also interested in a proper discussion in this setting so I can avoid the dependency. (I try to minimize or eliminate dependencies).

Finally, my apologies if this is not a programming group. Google's Usenet interface simply sucks. We can't navigate group hierarchies any longer under the new and improved design. And searching for "comp.os.linux groups" returned one result of comp.os.linux.misc.

Thanks in advance.

[toc] | [next] | [standalone]

#2585

On 26/11/13 07:39, noloader@gmail.com wrote:
> Hi All,
> 
> Is anyone aware of a discussion or treatment of a multi-port threaded
> server on modern Linux boxes written in c/c++? Multi-port would mean,
> for example, ports 80 and 443.
> 
> Contemporary Linux and Unix post-date W. Richard Steven's Advanced
> Programming in the Unix Environment and Unix Network Programming, so
> my bibles are not helpful. Googling is returning the Comp Sci 101
> examples with a single server/single port.
> 
> Specifically, I'm interested in discussions of:
> 
> (1) multiple processes servicing a single port and thread the
> accept() (2) single process with multiple threads for each port and
> thread the accept() (3) the most efficient way to wait and handle
> connections (4) how to safely shutdown a multi-port, multi-threaded
> server
> 
> I'm fairly certain fork/exec is no longer recommended, especially
> since critical libraries like OpenSSL are not fork-safe in a
> multi-threaded environment.
> 
> I also think item (3) can be handled with libevent, but I'm also
> interested in a proper discussion in this setting so I can avoid the
> dependency. (I try to minimize or eliminate dependencies).
> 
> Finally, my apologies if this is not a programming group. Google's
> Usenet interface simply sucks. We can't navigate group hierarchies
> any longer under the new and improved design. And searching for
> "comp.os.linux groups" returned one result of comp.os.linux.misc.
> 
> Thanks in advance.
> 

This is not really a programming group - but you might get some ideas
here.  I don't know what the ideal group would be - comp.lang.c and
comp.lang.c++ are more about the language than applications.

There is no such language as "C/C++".  A server like this would be
structured totally differently in C and in C++.  Don't mix these up -
pick one language.

Also, is there a particular reason to pick C or C++ ?  There are lots of
other languages suitable for server design, many with frameworks that
will give you most of what you want here without any effort.  Python
with the "twisted" framework springs to mind (since I've used it myself)
as being very efficient at handling multiple ports with multiple
connections, and it takes care of all the details of resources.  There
are many other frameworks for many other languages which could make the
job a lot easier.

[toc] | [prev] | [next] | [standalone]

#2588

On Tuesday, November 26, 2013 3:48:45 AM UTC-5, David Brown wrote:
> On 26/11/13 07:39, noloader@gmail.com wrote:
> 
> > Hi All,
> 
> > 
> 
> > Is anyone aware of a discussion or treatment of a multi-port threaded
> 
> > server on modern Linux boxes written in c/c++? Multi-port would mean,
> 
> > for example, ports 80 and 443.
> 
> > 
> 
> > Contemporary Linux and Unix post-date W. Richard Steven's Advanced
> 
> > Programming in the Unix Environment and Unix Network Programming, so
> 
> > my bibles are not helpful. Googling is returning the Comp Sci 101
> 
> > examples with a single server/single port.
> 
> > 
> 
> > Specifically, I'm interested in discussions of:
> 
> > 
> 
> > (1) multiple processes servicing a single port and thread the
> 
> > accept() (2) single process with multiple threads for each port and
> 
> > thread the accept() (3) the most efficient way to wait and handle
> 
> > connections (4) how to safely shutdown a multi-port, multi-threaded
> 
> > server
> 
> > 
> 
> > I'm fairly certain fork/exec is no longer recommended, especially
> 
> > since critical libraries like OpenSSL are not fork-safe in a
> 
> > multi-threaded environment.
> 
> > 
> 
> > I also think item (3) can be handled with libevent, but I'm also
> 
> > interested in a proper discussion in this setting so I can avoid the
> 
> > dependency. (I try to minimize or eliminate dependencies).
> 
> > 
> 
> > Finally, my apologies if this is not a programming group. Google's
> 
> > Usenet interface simply sucks. We can't navigate group hierarchies
> 
> > any longer under the new and improved design. And searching for
> 
> > "comp.os.linux groups" returned one result of comp.os.linux.misc.
> 
> > 
> 
> > Thanks in advance.
> 
> > 
> 
> 
> 
> This is not really a programming group - but you might get some ideas
> 
> here.  I don't know what the ideal group would be - comp.lang.c and
> 
> comp.lang.c++ are more about the language than applications.
Oh, sorry about that.

I'll try another group or Stack Overflow.

> Also, is there a particular reason to pick C or C++ ?

Yes. We are looking to maximize performance, and I have over 15 years experience with the languages.

> other languages suitable for server design, many with frameworks that
> 
> will give you most of what you want here without any effort.  Python
> 
> with the "twisted" framework springs to mind (since I've used it myself)
> 
> as being very efficient at handling multiple ports with multiple
> 
> connections, and it takes care of all the details of resources.

Yeah, I thought about Python. There are a few problems with it. (1) I can read Python, but I don't write it very well (this is a big problem in practice ;). (2) Python is interpreted and not native, so I think C/C++ will have somewhat better performance. (3) Python makes some really dumb decisions out of the box, and I don't know where all the devils lie (cf, this 2007 bug: http://bugs.python.org/issue1589).

Jeff

[toc] | [prev] | [next] | [standalone]

#2589

On 26/11/13 18:47, noloader@gmail.com wrote:
> On Tuesday, November 26, 2013 3:48:45 AM UTC-5, David Brown wrote:
<snip>>

>> 
>> This is not really a programming group - but you might get some 
>> ideas
>> 
>> here.  I don't know what the ideal group would be - comp.lang.c 
>> and
>> 
>> comp.lang.c++ are more about the language than applications.
> Oh, sorry about that.

No apology needed - posting here is fine too.  I'm just warning you that
you won't get all the answers!

However, if you are posting to a Usenet group, try to use a proper
newsreader - Google Groups makes a hideous mess out of posts.

> 
> I'll try another group or Stack Overflow.
> 
>> Also, is there a particular reason to pick C or C++ ?
> 
> Yes. We are looking to maximize performance, and I have over 15 years
> experience with the languages.

A lot of people think C and C++ are the answer to "maximum performance".
 Usually they are wrong.

Once the shock of that wild generalisation has blown over, I'll explain
my reasoning.  C and C++ (along with other compiled languages like Ada,
Fortran, ocaml, etc.) give you the smallest and fastest code at the
detail level.  But the speed of a system like this will depend on its
structure, its threading, queuing, synchronisation, its use of blocking
and non-blocking sockets, the algorithms used to track large data
blocks, etc.  These things are not dependent on the quality of a
compiler or interpreter - they are dependent on the quality of the
design.  Higher level languages - such as Python - make it easier to
make good designs than more detail-oriented lower level languages.
Ergo, such code is often faster when written using a higher level language.

Secondly, much of the work in a Python program is handled by the
libraries, rather than being written by the developer.  And much of that
code, especially where there are /real/ performance gains to be made, is
written in C.  And since this code is written by people who have been
willing to spend a lot of time and effort making efficient and
well-tested code, such code is normally faster than you would write
yourself (I don't know how good a programmer you are, but I assume you
have other priorities such as development time and costs that limit your
efforts in making the fastest possible code).

So in theory, C and C++ should give the fastest server code.  In
practice, using the right existing framework greatly overwhelms any
speed benefits from these languages.


And assuming you are getting paid for your work here, you are /not/
looking to maximize performance.  You are looking for /good enough/
performance - anything more is a waste of your employer's or client's
money.  That's worth keeping in mind, especially because employers and
clients often /say/ they want "maximum performance".  You are always
looking for a balance between higher performance, and the costs of
getting there.


Of course, experience with the languages is another matter :-)


> 
>> other languages suitable for server design, many with frameworks 
>> that
>> 
>> will give you most of what you want here without any effort. 
>> Python
>> 
>> with the "twisted" framework springs to mind (since I've used it 
>> myself)
>> 
>> as being very efficient at handling multiple ports with multiple
>> 
>> connections, and it takes care of all the details of resources.
> 
> Yeah, I thought about Python. There are a few problems with it. 

> (1) I  can read Python, but I don't write it very well (this is a big 
> problem in practice ;). 

Python is a relatively easy language to work with (at least for simpler
code), and your aim here is to use an existing framework for most of the
system.

If this is a big enough project, learning Python and writing the system
in Python may be faster than writing it all in C++.

(And Python is just an example - there are many others, such as Ruby on
Rails.)


> (2) Python is interpreted and not native, so I think C/C++ will have
> somewhat better performance.

Python is byte-compiled and runs in a virtual machine.  Performance is
good for some things, poor for others - it is very fast for manipulating
data, handling strings and text, etc., but slow for basic arithmetic.
And for performance you can always use PyPy, which has JIT compilation
to give near-native performance.


> (3) Python makes some really dumb decisions out of the box, and I don't 
> know where all the devils lie (cf, this 2007 bug:
> http://bugs.python.org/issue1589).
> 

That particular case is a "debatable" decision, rather than a "dumb"
decision.  And there will /always/ be such issues with whatever
frameworks or libraries you choose to use - even if you write
/everything/ yourself, you will feel some decisions in the Linux kernel
are "dumb".

But it is a good reason for picking well-known and well-tested languages
and frameworks, rather than picking the latest and greatest trends.
Python /is/ such a well-known and well-tested language, and Twisted is
such a framework - they are massively popular.

Just as a hint, Google is a major backer of Python development, and they
use it for a lot of their servers and other software.



I certainly don't mean to say that you should switch to Python for this
project.  But I do think that at this stage, you should keep your
options open and look at the bigger picture, so that you can make an
informed and justifiable decision.

mvh.,

David



> Jeff
> 

[toc] | [prev] | [next] | [standalone]

#2590

On Wednesday 27 November 2013 10:56, David Brown conveyed the following 
to comp.os.linux.networking...

> On 26/11/13 18:47, noloader@gmail.com wrote:
>
>> On Tuesday, November 26, 2013 3:48:45 AM UTC-5, David Brown wrote:
>>  
>>> This is not really a programming group - but you might get some
>>> ideas here.  I don't know what the ideal group would be -
>>> comp.lang.c and comp.lang.c++ are more about the language than
>>> applications.
>>
>> Oh, sorry about that.
> 
> No apology needed - posting here is fine too.  I'm just warning you
> that you won't get all the answers!
> 
> However, if you are posting to a Usenet group, try to use a proper
> newsreader - Google Groups makes a hideous mess out of posts.

Not to mention that Google Groups is also one of the favorite posting 
media for spammers, which is why most of the regulars filter out 
anything coming in through Google Groups, which reduces a bona fide 
poster's chances of getting a useful reply.

-- 
= Aragorn =

         http://www.linuxcounter.net - registrant #223157

[toc] | [prev] | [next] | [standalone]

#2591

On 27/11/13 12:14, Aragorn wrote:
> On Wednesday 27 November 2013 10:56, David Brown conveyed the following 
> to comp.os.linux.networking...
> 
>> On 26/11/13 18:47, noloader@gmail.com wrote:
>>
>>> On Tuesday, November 26, 2013 3:48:45 AM UTC-5, David Brown wrote:
>>>  
>>>> This is not really a programming group - but you might get some
>>>> ideas here.  I don't know what the ideal group would be -
>>>> comp.lang.c and comp.lang.c++ are more about the language than
>>>> applications.
>>>
>>> Oh, sorry about that.
>>
>> No apology needed - posting here is fine too.  I'm just warning you
>> that you won't get all the answers!
>>
>> However, if you are posting to a Usenet group, try to use a proper
>> newsreader - Google Groups makes a hideous mess out of posts.
> 
> Not to mention that Google Groups is also one of the favorite posting 
> media for spammers, which is why most of the regulars filter out 
> anything coming in through Google Groups, which reduces a bona fide 
> poster's chances of getting a useful reply.
> 

Most of the spam I see on newsgroups is from direct posters - but I
don't see much.  Maybe my newsserver filters spam better than yours.

But as long as there are people who filter out Google Groups posts, your
point is well made.

[toc] | [prev] | [next] | [standalone]

#2631

On Tue, 2013-11-26, David Brown wrote:
> On 26/11/13 07:39, noloader@gmail.com wrote:
>> Hi All,
>> 
>> Is anyone aware of a discussion or treatment of a multi-port threaded
>> server on modern Linux boxes written in c/c++? Multi-port would mean,
>> for example, ports 80 and 443.
>> 
>> Contemporary Linux and Unix post-date W. Richard Steven's Advanced
>> Programming in the Unix Environment and Unix Network Programming, so
>> my bibles are not helpful. Googling is returning the Comp Sci 101
>> examples with a single server/single port.
>> 
>> Specifically, I'm interested in discussions of:
...

>> Finally, my apologies if this is not a programming group.
...
>
> This is not really a programming group - but you might get some ideas
> here.  I don't know what the ideal group would be - comp.lang.c and
> comp.lang.c++ are more about the language than applications.

Yes they are.

Comp.unix.programmer seems like the right place to me, but it has
rather few posters (often with rather strong and unusual opinions).

> There is no such language as "C/C++".  A server like this would be
> structured totally differently in C and in C++.

I don't think so: unless you jump on the C++11 train, what determines
the high-level structure is the Linux APIs you use, not if you use C
or C++. At least in my experience.  Of course C++ will let you do a
lot of things safer and more elegantly, but not this part.

/Jorgen

-- 
  // Jorgen Grahn <grahn@  Oo  o.   .     .
\X/     snipabacken.se>   O  o   .

[toc] | [prev] | [next] | [standalone]

#2586

noloader@gmail.com wrote:
> Is anyone aware of a discussion or treatment of a multi-port threaded
> server on modern Linux boxes written in c/c++? Multi-port would mean,
> for example, ports 80 and 443.

Google offers many seemingly-relevant results for a search of
"multithreaded socket select".


> (1) multiple processes servicing a single port and thread the accept()
> (2) single process with multiple threads for each port and thread the accept()
> (3) the most efficient way to wait and handle connections
> (4) how to safely shutdown a multi-port, multi-threaded server

There may well be discussions on these if you throw the keyword "apache"
in to the mix.

Chris

[toc] | [prev] | [next] | [standalone]

#2587

On Tuesday, November 26, 2013 12:03:26 PM UTC-5, Chris Davies wrote:
> noloader@gmail.com wrote:
> 
> > Is anyone aware of a discussion or treatment of a multi-port threaded
> 
> > server on modern Linux boxes written in c/c++? Multi-port would mean,
> 
> > for example, ports 80 and 443.
> 
> 
> 
> Google offers many seemingly-relevant results for a search of
> 
> "multithreaded socket select".
Thanks Chris. Try adding "multi-port" (or one of its derivatives).

> > (1) multiple processes servicing a single port and thread the accept()
> 
> > (2) single process with multiple threads for each port and thread the accept()
> 
> > (3) the most efficient way to wait and handle connections
> 
> > (4) how to safely shutdown a multi-port, multi-threaded server
> 
> There may well be discussions on these if you throw the keyword "apache"
> 
> in to the mix.
Yeah, I'm trying to avoid studying Apache source code.

Jeff

[toc] | [prev] | [next] | [standalone]

#2632

On Tue, 2013-11-26, noloader@gmail.com wrote:

> Is anyone aware of a discussion or treatment of a multi-port
> threaded server on modern Linux boxes written in c/c++? Multi-port
> would mean, for example, ports 80 and 443.

> Contemporary Linux and Unix post-date W. Richard Steven's Advanced
> Programming in the Unix Environment and Unix Network Programming, so
> my bibles are not helpful.

Sure about that?  There have been updated editions of his books. I
still use the ones from the 1990s, plus the man pages.  epoll(7) is
the only major newer interface I use ... on the other hand I don't do
threads and maybe the state of the art has changed more there.

> Googling is returning the Comp Sci 101
> examples with a single server/single port.

Yes; the quality of such search results is low.  I still see new code
being written by cargo-culting them; it's more primitive than what
you'd get from Stevens' 20 years old edition.

> Specifically, I'm interested in discussions of:

> (1) multiple processes servicing a single port and thread the accept()
> (2) single process with multiple threads for each port and thread the accept()
> (3) the most efficient way to wait and handle connections
> (4) how to safely shutdown a multi-port, multi-threaded server

> I'm fairly certain fork/exec is no longer recommended, especially
> since critical libraries like OpenSSL are not fork-safe in a
> multi-threaded environment.

Depending on what you want to do, fork is still the right choice ...

I'd like to be able to answer your questions (or read the answers) but
sorry, no.  I'm playing with one or two projects of my own, but my
design is based on a single process, epoll and non-blocking sockets.
I expect it to be I/O-bound rather than CPU-bound, and I dislike and
distrust threading ... so I don't think it's useful to you.

/Jorgen

-- 
  // Jorgen Grahn <grahn@  Oo  o.   .     .
\X/     snipabacken.se>   O  o   .

[toc] | [prev] | [next] | [standalone]

#2633

noloader@gmail.com wrote:

> 
> Finally, my apologies if this is not a programming group. Google's Usenet
> interface simply sucks. We can't navigate group hierarchies any longer
> under the new and improved design. And searching for "comp.os.linux
> groups" returned one result of comp.os.linux.misc.
> 
The answer to that is to get a proper threaded newsreader and a subscription 
to an nntp server such as Eternal September, then you can experience proper 
newsgroups.

Google's Usenet interface should be reserved for emergencies only.
-- 
Dave
Too many gadgets, too little time

[toc] | [prev] | [standalone]

Back to top | Article view | comp.os.linux.networking

csiph-web