Largest possible size for executemany() in PEP-249 (Database API)

Path	csiph.com!v102.xanadu-bbs.net!xanadu-bbs.net!news.glorb.com!news-out.readnews.com!transit3.readnews.com!panix!panix2.panix.com!not-for-mail
From	roy@panix.com (Roy Smith)
Newsgroups	comp.lang.python
Subject	Largest possible size for executemany() in PEP-249 (Database API)
Date	28 Feb 2013 12:05:28 -0500
Organization	PANIX -- Public Access Networks Corp.
Lines	11
Message-ID	<kgo2oo$5tk$1@panix2.panix.com> (permalink)
NNTP-Posting-Host	panix2.panix.com
X-Trace	reader1.panix.com 1362071128 18078 166.84.1.2 (28 Feb 2013 17:05:28 GMT)
X-Complaints-To	abuse@panix.com
NNTP-Posting-Date	Thu, 28 Feb 2013 17:05:28 +0000 (UTC)
Xref	csiph.com comp.lang.python:40139

Show key headers only | View raw

I'm trying to batch up inserts to a database using MySQLdb.  When I
get to something over 10,000 records per call, I get an exception:

_mysql_exceptions.OperationalError: (1153, "Got a packet bigger than
'max_allowed_packet' bytes")

Is there any way (other than trial and error) to know how many records
I can pass in one call before I blow up?  As a practical matter, if I
do batches of 1000 per call, I've probably gotten as much performance
enhancement as I need, but it would be nice to know if there's a
useful way to determine exactly what the maximum is.

Back to comp.lang.python | Previous | Next — Next in thread | Find similar | Unroll thread

Thread

Largest possible size for executemany() in PEP-249 (Database API) roy@panix.com (Roy Smith) - 2013-02-28 12:05 -0500
  Re: Largest possible size for executemany() in PEP-249 (Database API) Neil Hodgson <nhodgson@iinet.net.au> - 2013-03-01 09:26 +1100
  Re: Largest possible size for executemany() in PEP-249 (Database API) Dennis Lee Bieber <wlfraed@ix.netcom.com> - 2013-02-28 20:52 -0500

csiph-web