Path: csiph.com!newsfeed.hal-mli.net!feeder3.hal-mli.net!newsfeed.hal-mli.net!feeder1.hal-mli.net!newsfeed.xs4all.nl!newsfeed4.news.xs4all.nl!xs4all!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail
Return-Path: <rosuav@gmail.com>
X-Original-To: python-list@python.org
Delivered-To: python-list@mail.python.org
X-Spam-Status: OK 0.105
X-Spam-Level: *
X-Spam-Evidence: '*H*': 0.79; '*S*': 0.00; 'subject:Python': 0.06; 'postgresql': 0.07; 'assuming': 0.09; 'elsewhere.': 0.16; 'fails.': 0.16; 'from:addr:rosuav': 0.16; 'from:name:chris angelico': 0.16; 'hardware.': 0.16; 'proxy,': 0.16; 'queuing': 0.16; 'reliably': 0.16; 'stats': 0.16; 'wrote:': 0.18; 'possible,': 0.19; 'written': 0.21; 'aug': 0.22; 'otherwise,': 0.22; 'proxy': 0.24; "shouldn't": 0.24; 'fairly': 0.24; 'server.': 0.24; "i've": 0.25; 'header:In-Reply-To:1': 0.27; 'record': 0.27; 'message-id:@mail.gmail.com': 0.30; 'second,': 0.31; 'probably': 0.32; "we're": 0.32; 'fri,': 0.33; "i'd": 0.34; 'could': 0.34; 'problem': 0.35; 'basic': 0.35; 'connection': 0.35; 'tool': 0.35; 'but': 0.35; 'received:google.com': 0.35; 'ram': 0.36; 'smtp': 0.36; 'should': 0.36; 'handle': 0.38; 'to:addr:python-list': 0.38; 'pm,': 0.38; 'volume': 0.39; 'to:addr:python.org': 0.39; 'how': 0.40; 'logs': 0.60; 'worry': 0.60; 'most': 0.60; "you're": 0.61; "you'll": 0.62; 'making': 0.63; 'act': 0.63; 'more': 0.64; '2000': 0.65; 'talking': 0.65; 'between': 0.67; 'details,': 0.68; 'lose': 0.68; 'hour': 0.70; 'power': 0.76; 'issues;': 0.84; 'usage.': 0.84; 'mails': 0.89; 'notable': 0.91; '2013': 0.98
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=+rnYlhzMzXlcA2nmdUQLdlMN+gvsz6R/CvXEUA0FB78=; b=n6Q5QbUd/PKTc6WkvDizS/+qoUtCrXw+6/L8/hhtvUX+I04Kpdzbm7kNwuuRlCbM9O KvZZnRa4A+8JVhGk7KmzaJMxpLVCsxaP5/8pv/VWvaaW2TDyPKPlBG+QCKXwNRNECAO9 wqehIFAevcpV5/y8JS/ALrAn+D3hjrXOhCvioHHCwo3HlpszfN6JWjCyGfZG3AeIQ82/ dwArYqZ4F8CRj5WINL9k+z6zwSBd4iZu92KUab+u6a6ZconaRN0Y9V3/BygDR+1IaCcG HLNjDwNHUB/OGXG6lJ0rCBkU1qW5n4XWC9wpV5E3vj9VFx/UBNmrkUdaqoC7+urMB+TS tHgg==
MIME-Version: 1.0
X-Received: by 10.58.214.105 with SMTP id nz9mr2011168vec.58.1375451013931; Fri, 02 Aug 2013 06:43:33 -0700 (PDT)
In-Reply-To: <51FBB115.4070608@globe.de>
References: <51FB9F49.9040209@globe.de> <20130802075926.454bd391@bigbox.christie.dr> <51FBB115.4070608@globe.de>
Date: Fri, 2 Aug 2013 14:43:33 +0100
Subject: Re: Python performance
From: Chris Angelico <rosuav@gmail.com>
To: python-list@python.org
Content-Type: text/plain; charset=ISO-8859-1
X-BeenThere: python-list@python.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe: <http://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive: <http://mail.python.org/pipermail/python-list/>
List-Post: <mailto:python-list@python.org>
List-Help: <mailto:python-list-request@python.org?subject=help>
List-Subscribe: <http://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
Newsgroups: comp.lang.python
Message-ID: <mailman.127.1375451016.1251.python-list@python.org>
Lines: 23
NNTP-Posting-Host: 2001:888:2000:d::a6
X-Trace: 1375451016 news.xs4all.nl 15971 [2001:888:2000:d::a6]:38249
X-Complaints-To: abuse@xs4all.nl
Xref: csiph.com comp.lang.python:51810

On Fri, Aug 2, 2013 at 2:16 PM, Schneider <js@globe.de> wrote:
> Queuing the mails for a while is not possible, because the tool should sit
> between the client and smtp-server.
> It should act as proxy, not as server.

I've written an SMTP proxy (primary purpose: check SPF records;
secondary purpose: rate-limit one particular PHP app) that can handle
fairly large throughput; it could probably saturate the network
connection it's working on (10Mbit) without the proxy making a
particularly notable blip on the CPU or RAM usage. Most of the work is
elsewhere.

How bursty will the mail be? If we're talking 2000 all at once and
then an hour of quietness, you may have some issues; but I'd say you
could do 100-200 in a second, on reasonable hardware. Of course, this
is assuming you're writing your logs to a write-cached volume - which
means you'll lose some stats if the power fails. If that's a problem
to you, you'll want to record your stats more reliably (personally I'd
let PostgreSQL worry about the details, ie store into a database),
which will cost throughput. But otherwise, basic stats shouldn't take
more than 5-10ms.

ChrisA