Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #34209 > unrolled thread

using smtp sent large file upto 60MB

Back to article view | Back to comp.lang.python

Contents

  using smtp sent large file upto 60MB moonhkt <moonhkt@gmail.com> - 2012-12-04 00:15 -0800
    Re: using smtp sent large file upto 60MB Chris Angelico <rosuav@gmail.com> - 2012-12-04 21:07 +1100
      Re: using smtp sent large file upto 60MB moonhkt <moonhkt@gmail.com> - 2012-12-04 07:20 -0800
        Re: using smtp sent large file upto 60MB Laszlo Nagy <gandalf@shopzeus.com> - 2012-12-04 16:41 +0100
        Re: using smtp sent large file upto 60MB Chris Angelico <rosuav@gmail.com> - 2012-12-05 07:54 +1100
          Re: using smtp sent large file upto 60MB moonhkt <moonhkt@gmail.com> - 2012-12-04 16:54 -0800
            Re: using smtp sent large file upto 60MB Chris Angelico <rosuav@gmail.com> - 2012-12-05 16:34 +1100
              Re: using smtp sent large file upto 60MB moonhkt <moonhkt@gmail.com> - 2012-12-04 23:40 -0800
            Re: using smtp sent large file upto 60MB Michael Torrie <torriem@gmail.com> - 2012-12-05 08:01 -0700
              Re: using smtp sent large file upto 60MB moonhkt <moonhkt@gmail.com> - 2012-12-10 07:51 -0800

#34209 — using smtp sent large file upto 60MB

Hi All

How to using python send file uptp 60MB ?


s = smtplib.SMTP("localhost")
#~~ s.set_debuglevel(1)
s.sendmail(from_addr, to_addr,m.as_string())
s.quit


For 13MB file have below error
    s.sendmail(from_addr, to_addr,m.as_string())
  File "/opt/freeware/lib/python2.6/email/message.py", line 135, in
as_string
    g.flatten(self, unixfrom=unixfrom)
  File "/opt/freeware/lib/python2.6/email/generator.py", line 84, in
flatten
    self._write(msg)
  File "/opt/freeware/lib/python2.6/email/generator.py", line 119, in
_write
    self._fp.write(sfp.getvalue())
MemoryError: out of memory

[toc] | [next] | [standalone]

#34213

On Tue, Dec 4, 2012 at 7:15 PM, moonhkt <moonhkt@gmail.com> wrote:
> How to using python send file uptp 60MB ?

Step one: Don't. SMTP is almost never the best choice for sending huge
files around.

There are plenty of other ways to share files; send an email with
instructions on how to access the file, rather than attaching the
file. For general consumption, the easiest way is usually to include a
URL for HTTP download. If it's something internal, you might want to
put the file on a shared-access FTP server or file share.

ChrisA

[toc] | [prev] | [next] | [standalone]

#34234

On Dec 4, 6:07 pm, Chris Angelico <ros...@gmail.com> wrote:
> On Tue, Dec 4, 2012 at 7:15 PM, moonhkt <moon...@gmail.com> wrote:
> > How to using python send file uptp 60MB ?
>
> Step one: Don't. SMTP is almost never the best choice for sending huge
> files around.
>
> There are plenty of other ways to share files; send an email with
> instructions on how to access the file, rather than attaching the
> file. For general consumption, the easiest way is usually to include a
> URL for HTTP download. If it's something internal, you might want to
> put the file on a shared-access FTP server or file share.
>
> ChrisA

Thank for suggestion. The next task will be ftp to user folder. But
first tasks is how to using python send huge files.

[toc] | [prev] | [next] | [standalone]

#34238

> Thank for suggestion. The next task will be ftp to user folder. But
> first tasks is how to using python send huge files.
Most SMTP servers are configured not to accept attachments bigger than 
10 or 15MB. In general, you should never send emails with >5MB 
attachments. Not because it is not possible, but because it is 
unreliable, and the solution is never in your hand. The solution depends 
on the SMTP server configuration, and in most cases you don't have 
access to the computers holding the final destination of the emails.

If you still don't want to accept this suggestion, then go ahead! Write 
a program, send out 100MB emails, and you will see for yourself that it 
just doesn't work.

[toc] | [prev] | [next] | [standalone]

#34253

On Wed, Dec 5, 2012 at 2:41 AM, Laszlo Nagy <gandalf@shopzeus.com> wrote:
> If you still don't want to accept this suggestion, then go ahead! Write a
> program, send out 100MB emails, and you will see for yourself that it just
> doesn't work.

But be aware of a few things.

1) Converting 1MB of binary data into a MIME-packaged email is going
to result in about 2MB of text. (It's about 1.5MB for base 64
encoding, which is one of the most common used, plus a bit more for
structure around it, and rounding up, call it two meg.)

2) If that 2MB of text is stored as a Python text string, it could
potentially consume 4MB or 8MB of memory, unless you're on Python 3.3,
in which case it will be only 2MB..

3) That 2-8MB has to be contiguous.

4) Any manipulation of the resulting string - which will quite
probably happen as it's built, as it gets connected to the email, etc,
etc, etc - will require even more copies of the string.

So all in all, you need a LOT of memory to do your encoding. That's
why you're seeing MemoryError - it is simply impossible to attach a
huge file to an email without using a fair amount of memory. (It's
possible to use that memory a bit at a time, but since emails are
generally small, most encoding libraries won't be written to do that.
This isn't like movie editing, where it's common to work with files
larger than your RAM.)

ChrisA

[toc] | [prev] | [next] | [standalone]

#34265

On 12月5日, 上午4時54分, Chris Angelico <ros...@gmail.com> wrote:
> On Wed, Dec 5, 2012 at 2:41 AM, Laszlo Nagy <gand...@shopzeus.com> wrote:
> > If you still don't want to accept this suggestion, then go ahead! Write a
> > program, send out 100MB emails, and you will see for yourself that it just
> > doesn't work.
>
> But be aware of a few things.
>
> 1) Converting 1MB of binary data into a MIME-packaged email is going
> to result in about 2MB of text. (It's about 1.5MB for base 64
> encoding, which is one of the most common used, plus a bit more for
> structure around it, and rounding up, call it two meg.)
>
> 2) If that 2MB of text is stored as a Python text string, it could
> potentially consume 4MB or 8MB of memory, unless you're on Python 3.3,
> in which case it will be only 2MB..
>
> 3) That 2-8MB has to be contiguous.
>
> 4) Any manipulation of the resulting string - which will quite
> probably happen as it's built, as it gets connected to the email, etc,
> etc, etc - will require even more copies of the string.
>
> So all in all, you need a LOT of memory to do your encoding. That's
> why you're seeing MemoryError - it is simply impossible to attach a
> huge file to an email without using a fair amount of memory. (It's
> possible to use that memory a bit at a time, but since emails are
> generally small, most encoding libraries won't be written to do that.
> This isn't like movie editing, where it's common to work with files
> larger than your RAM.)
>
> ChrisA

Thank for your suggestion.

Machine : AIX
Python version : 2.6.2

I am prepare change UNIX script to Python. smtp and ftp are my first
tasks.

But, when using standard unix command mail and uuencode without this
issue.

Our SMTP can send file more than 60MB. But our notes server can
configured 100MB,30MB or 10MB. My notes Mail box can receive 100MB.

In UNIX, by below command send  smtp mail.
uuencode $xfn $xfn | mail -s "$SUBJECT" $NAME

moonhkt

[toc] | [prev] | [next] | [standalone]

#34272

On Wed, Dec 5, 2012 at 11:54 AM, moonhkt <moonhkt@gmail.com> wrote:
> I am prepare change UNIX script to Python. smtp and ftp are my first
> tasks.
>
> But, when using standard unix command mail and uuencode without this
> issue.
>
> Our SMTP can send file more than 60MB. But our notes server can
> configured 100MB,30MB or 10MB. My notes Mail box can receive 100MB.
>
> In UNIX, by below command send  smtp mail.
> uuencode $xfn $xfn | mail -s "$SUBJECT" $NAME

Yes, and it is possible to send that much content via SMTP. It just
isn't something that library authors are going to be overly concerned
about. You may need to jump through a few extra hoops, or maybe just
throw more RAM at the computer (possibly switching to a 64-bit build
of Python if you aren't already using one). However, I would *still*
recommend using a different transport for such large files.

ChrisA

[toc] | [prev] | [next] | [standalone]

#34273

On 12月5日, 下午1時34分, Chris Angelico <ros...@gmail.com> wrote:
> On Wed, Dec 5, 2012 at 11:54 AM, moonhkt <moon...@gmail.com> wrote:
> > I am prepare change UNIX script to Python. smtp and ftp are my first
> > tasks.
>
> > But, when using standard unix command mail and uuencode without this
> > issue.
>
> > Our SMTP can send file more than 60MB. But our notes server can
> > configured 100MB,30MB or 10MB. My notes Mail box can receive 100MB.
>
> > In UNIX, by below command send  smtp mail.
> > uuencode $xfn $xfn | mail -s "$SUBJECT" $NAME
>
> Yes, and it is possible to send that much content via SMTP. It just
> isn't something that library authors are going to be overly concerned
> about. You may need to jump through a few extra hoops, or maybe just
> throw more RAM at the computer (possibly switching to a 64-bit build
> of Python if you aren't already using one). However, I would *still*
> recommend using a different transport for such large files.
>
> ChrisA

Thank a lot. We still using Python version : 2.6.2  on AIX 5.3

[toc] | [prev] | [next] | [standalone]

#34283

On 12/04/2012 05:54 PM, moonhkt wrote:
> Our SMTP can send file more than 60MB. But our notes server can
> configured 100MB,30MB or 10MB. My notes Mail box can receive 100MB.
> 
> In UNIX, by below command send  smtp mail.
> uuencode $xfn $xfn | mail -s "$SUBJECT" $NAME

Just continue to use this set of commands.  You can use the subprocess
module to interact with these programs.

[toc] | [prev] | [next] | [standalone]

#34549

On 12月5日, 下午11時01分, Michael Torrie <torr...@gmail.com> wrote:
> On 12/04/2012 05:54 PM, moonhkt wrote:
>
> > Our SMTP can send file more than 60MB. But our notes server can
> > configured 100MB,30MB or 10MB. My notes Mail box can receive 100MB.
>
> > In UNIX, by below command send  smtp mail.
> > uuencode $xfn $xfn | mail -s "$SUBJECT" $NAME
>
> Just continue to use this set of commands.  You can use the subprocess
> module to interact with these programs.

OK. Will try using subprocess.

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web