Path: csiph.com!usenet.pasdenom.info!weretis.net!feeder4.news.weretis.net!ecngs!feeder2.ecngs.de!newsfeed.freenet.ag!news2.euro.net!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail Return-Path: X-Original-To: python-list@python.org Delivered-To: python-list@mail.python.org X-Spam-Status: OK 0.001 X-Spam-Evidence: '*H*': 1.00; '*S*': 0.00; 'string.': 0.04; 'binary': 0.05; 'memory.': 0.05; 'things.': 0.05; 'subject:file': 0.07; 'python': 0.09; 'encoding.': 0.09; 'rounding': 0.09; 'subject:using': 0.09; 'stored': 0.10; 'resulting': 0.13; 'dec': 0.15; 'encoding': 0.15; "(it's": 0.16; '3.3,': 0.16; 'ahead!': 0.16; 'consume': 0.16; 'from:addr:rosuav': 0.16; 'from:name:chris angelico': 0.16; 'wed,': 0.16; 'string': 0.17; 'wrote:': 0.17; 'string,': 0.17; 'memory': 0.18; 'written': 0.20; 'all,': 0.21; 'bit': 0.21; 'libraries': 0.22; 'work.': 0.23; 'connected': 0.24; 'header:In-Reply-To:1': 0.25; 'common': 0.26; 'am,': 0.27; 'used,': 0.27; 'converting': 0.27; 'small,': 0.27; 'message- id:@mail.gmail.com': 0.27; "doesn't": 0.28; 'probably': 0.29; 'that.': 0.30; 'attach': 0.30; 'gets': 0.32; 'file': 0.32; 'generally': 0.32; 'structure': 0.32; 'could': 0.32; 'to:addr :python-list': 0.33; 'program,': 0.34; 'received:google.com': 0.34; 'text': 0.34; 'text.': 0.35; 'received:209.85.220': 0.35; "won't": 0.35; 'received:209.85': 0.35; 'but': 0.36; 'possible': 0.37; 'two': 0.37; 'why': 0.37; 'quite': 0.37; 'received:209': 0.37; 'data': 0.37; 'subject:: ': 0.38; 'files': 0.38; 'to:addr:python.org': 0.39; 'where': 0.40; 'header:Received:5': 0.40; 'your': 0.60; 'most': 0.61; 'time,': 0.62; 'more': 0.63; 'email,': 0.66; 'potentially': 0.66; 'emails,': 0.69; 'yourself': 0.77; 'built,': 0.84; 'etc,': 0.84; 'nagy': 0.84; 'subject:sent': 0.84 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=9YZrKTyFT6tmJ6PIxy9IiSh0KT5LpW8mr42zrgPe8A8=; b=gwRnP+aFIyyHm4ATg//XBC5s7Ih8UdxgsFzhz3QBySfg4DJg363+tIlXNCXgcygtRd 2zcHNXFe1RdM/+h9L6TwhNrjqDfkQK/LVPrINPX336dhBebIDSTIVfg701RoYdaaKZNN wWBBqgeB2hgWLNuJyXxj1+Z/arncvKi169AKXuAl0wTcPFM+MxJUpadmTWTKtdGbsvIj Jxwtz0SF3qHZW9Zh4SSnOeVJtG/T3Rng1iMEvFvkNOhrcHU6+ZbiundssA9Q9uz4PKfn yUYROkSRyZN7TYs9eDMrcwQIqUfLt6rvuHaWzQ4Hmi9wx2SkYFBYrju24e9mZOfVLXmE 2/bw== MIME-Version: 1.0 In-Reply-To: <50BE1990.4020009@shopzeus.com> References: <1d8228ff-8f9e-4258-8927-2f964c36d8bf@n5g2000vbk.googlegroups.com> <50BE1990.4020009@shopzeus.com> Date: Wed, 5 Dec 2012 07:54:26 +1100 Subject: Re: using smtp sent large file upto 60MB From: Chris Angelico To: python-list@python.org Content-Type: text/plain; charset=ISO-8859-1 X-BeenThere: python-list@python.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: General discussion list for the Python programming language List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Newsgroups: comp.lang.python Message-ID: Lines: 31 NNTP-Posting-Host: 2001:888:2000:d::a6 X-Trace: 1354654470 news.xs4all.nl 6910 [2001:888:2000:d::a6]:51493 X-Complaints-To: abuse@xs4all.nl Xref: csiph.com comp.lang.python:34253 On Wed, Dec 5, 2012 at 2:41 AM, Laszlo Nagy wrote: > If you still don't want to accept this suggestion, then go ahead! Write a > program, send out 100MB emails, and you will see for yourself that it just > doesn't work. But be aware of a few things. 1) Converting 1MB of binary data into a MIME-packaged email is going to result in about 2MB of text. (It's about 1.5MB for base 64 encoding, which is one of the most common used, plus a bit more for structure around it, and rounding up, call it two meg.) 2) If that 2MB of text is stored as a Python text string, it could potentially consume 4MB or 8MB of memory, unless you're on Python 3.3, in which case it will be only 2MB.. 3) That 2-8MB has to be contiguous. 4) Any manipulation of the resulting string - which will quite probably happen as it's built, as it gets connected to the email, etc, etc, etc - will require even more copies of the string. So all in all, you need a LOT of memory to do your encoding. That's why you're seeing MemoryError - it is simply impossible to attach a huge file to an email without using a fair amount of memory. (It's possible to use that memory a bit at a time, but since emails are generally small, most encoding libraries won't be written to do that. This isn't like movie editing, where it's common to work with files larger than your RAM.) ChrisA