Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #42289

Re: How to find bad row with db api executemany()?

Path csiph.com!usenet.pasdenom.info!news.etla.org!news.stack.nl!newsfeed.xs4all.nl!newsfeed2.news.xs4all.nl!xs4all!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail
Return-Path <python-python-list@m.gmane.org>
X-Original-To python-list@python.org
Delivered-To python-list@mail.python.org
X-Spam-Status OK 0.001
X-Spam-Evidence '*H*': 1.00; '*S*': 0.00; 'essentially': 0.04; 'argument': 0.05; 'one?': 0.05; 'string': 0.09; 'assuming': 0.09; 'delimited': 0.09; 'exception,': 0.09; 'mysql.': 0.09; 'newline': 0.09; 'raises': 0.09; 'received:80.91': 0.09; 'received:80.91.229': 0.09; 'received:gmane.org': 0.09; 'received:list': 0.09; 'rows': 0.09; 'statements': 0.09; 'violates': 0.09; 'subject:How': 0.10; 'creates': 0.14; 'channel?': 0.16; 'efficiency.': 0.16; 'inserting': 0.16; 'package).': 0.16; 'received:80.91.229.3': 0.16; 'received:plane.gmane.org': 0.16; 'roy': 0.16; 'wrote:': 0.18; '(but': 0.19; 'url:home': 0.24; 'versions': 0.24; '(or': 0.24; 'source': 0.25; 'header:X-Complaints-To:1': 0.27; 'am,': 0.29; "i'm": 0.30; 'gives': 0.31; 'code': 0.31; 'submitting': 0.31; 'compatible': 0.32; 'fri,': 0.33; 'maybe': 0.34; 'could': 0.34; 'subject:with': 0.35; 'problem.': 0.35; 'something': 0.35; 'point.': 0.35; 'but': 0.35; 'there': 0.35; 'charset:us-ascii': 0.36; 'error.': 0.37; 'too': 0.37; 'received:76': 0.38; 'massive': 0.38; 'to:addr:python-list': 0.38; 'short': 0.38; 'skip:. 10': 0.39; 'to:addr:python.org': 0.39; 'received:org': 0.40; 'even': 0.60; 'catch': 0.60; 'dave': 0.60; 'tell': 0.60; 'information': 0.63; 'kind': 0.63; 'direct': 0.67; 'mar': 0.68; 'smith': 0.68; 'caused': 0.69; 'containing': 0.69; 'million': 0.74; 'subject:find': 0.84; 'angel': 0.91; 'rally': 0.91; '2013': 0.98
X-Injected-Via-Gmane http://gmane.org/
To python-list@python.org
From Dennis Lee Bieber <wlfraed@ix.netcom.com>
Subject Re: How to find bad row with db api executemany()?
Date Fri, 29 Mar 2013 18:25:24 -0400
Organization > Bestiaria Support Staff <
References <F40EFF8D-9F4F-4ACB-8671-450F3CD761CA@panix.com> <5155E32A.1000403@davea.name>
Mime-Version 1.0
Content-Type text/plain; charset=us-ascii
Content-Transfer-Encoding 7bit
X-Gmane-NNTP-Posting-Host adsl-76-249-20-75.dsl.klmzmi.sbcglobal.net
X-Newsreader Forte Agent 3.3/32.846
X-No-Archive YES
X-BeenThere python-list@python.org
X-Mailman-Version 2.1.15
Precedence list
List-Id General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe <http://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive <http://mail.python.org/pipermail/python-list/>
List-Post <mailto:python-list@python.org>
List-Help <mailto:python-list-request@python.org?subject=help>
List-Subscribe <http://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
Newsgroups comp.lang.python
Message-ID <mailman.3971.1364595940.2939.python-list@python.org> (permalink)
Lines 32
NNTP-Posting-Host 2001:888:2000:d::a6
X-Trace 1364595940 news.xs4all.nl 6872 [2001:888:2000:d::a6]:33563
X-Complaints-To abuse@xs4all.nl
Xref csiph.com comp.lang.python:42289

Show key headers only | View raw

On Fri, 29 Mar 2013 14:53:30 -0400, Dave Angel <davea@davea.name>
declaimed the following in gmane.comp.python.general:

> On 03/29/2013 10:48 AM, Roy Smith wrote:
> > I'm inserting a gazillion rows into a MySQL database using MySQLdb and cursor.executemany() for efficiency.  Every once in a while, I get a row which violates some kind of database constraint and raises Error.
> >
> > I can catch the exception, but don't see any way to tell which row caused the problem.  Is this information obtainable, short of retrying each row one by one?
> >
> 
> I don't know the direct answer, or even if there is one (way to get 
> MySQL to tell you which one failed), but ...
> 
> Assuming that executeMany is much cheaper than a million calls to 
> executeOne (or whatever).

	If using MySQLdb, there isn't all that much difference... MySQLdb is
still compatible with MySQL v4 (and maybe even v3), and since those
versions don't have "prepared statements", .executemany() essentially
turns into something that creates a newline delimited "list" of
"identical" (but for argument substitution) statements and submits that
to MySQL.

	Just look at the source code (cursors.py in MySQLdb package).

	Hmmm... That gives a rally point. If it IS submitting one massive
string containing all the data copies, could the failure be coming from
submitting something too big for the client/server communication
channel?
-- 
	Wulfraed                 Dennis Lee Bieber         AF6VN
        wlfraed@ix.netcom.com    HTTP://wlfraed.home.netcom.com/

Back to comp.lang.python | Previous | Next — Next in thread | Find similar | Unroll thread

Thread

Re: How to find bad row with db api executemany()? Dennis Lee Bieber <wlfraed@ix.netcom.com> - 2013-03-29 18:25 -0400
  Re: How to find bad row with db api executemany()? Roy Smith <roy@panix.com> - 2013-03-29 20:41 -0400
    Re: How to find bad row with db api executemany()? Chris Angelico <rosuav@gmail.com> - 2013-03-30 11:57 +1100
      Re: How to find bad row with db api executemany()? Roy Smith <roy@panix.com> - 2013-03-29 21:19 -0400
        Re: How to find bad row with db api executemany()? Chris Angelico <rosuav@gmail.com> - 2013-03-30 13:05 +1100
        Re: How to find bad row with db api executemany()? Tim Chase <python.list@tim.thechases.com> - 2013-03-29 22:17 -0500
        Re: How to find bad row with db api executemany()? (PS) Tim Chase <python.list@tim.thechases.com> - 2013-03-29 22:38 -0500
        Re: How to find bad row with db api executemany()? Dennis Lee Bieber <wlfraed@ix.netcom.com> - 2013-03-29 23:38 -0400
    Re: How to find bad row with db api executemany()? Roy Smith <roy@panix.com> - 2013-03-29 22:44 -0400
      Re: How to find bad row with db api executemany()? Chris Angelico <rosuav@gmail.com> - 2013-03-30 13:49 +1100
        Re: How to find bad row with db api executemany()? Roy Smith <roy@panix.com> - 2013-03-29 23:09 -0400
          Re: How to find bad row with db api executemany()? Chris Angelico <rosuav@gmail.com> - 2013-03-30 14:14 +1100
            Re: How to find bad row with db api executemany()? Roy Smith <roy@panix.com> - 2013-03-29 23:36 -0400
              Re: How to find bad row with db api executemany()? Chris Angelico <rosuav@gmail.com> - 2013-03-30 14:57 +1100
                Re: How to find bad row with db api executemany()? Roy Smith <roy@panix.com> - 2013-03-30 00:10 -0400
                Re: How to find bad row with db api executemany()? Chris Angelico <rosuav@gmail.com> - 2013-03-30 15:21 +1100
                Re: How to find bad row with db api executemany()? Roy Smith <roy@panix.com> - 2013-03-30 10:19 -0400
        Re: How to find bad row with db api executemany()? rusi <rustompmody@gmail.com> - 2013-03-29 20:13 -0700
          Re: How to find bad row with db api executemany()? rusi <rustompmody@gmail.com> - 2013-03-29 20:15 -0700
          Re: How to find bad row with db api executemany()? Roy Smith <roy@panix.com> - 2013-03-29 23:40 -0400
      Re: How to find bad row with db api executemany()? Dennis Lee Bieber <wlfraed@ix.netcom.com> - 2013-03-29 23:53 -0400
      Re: How to find bad row with db api executemany()? Dennis Lee Bieber <wlfraed@ix.netcom.com> - 2013-03-30 00:19 -0400
      Re: How to find bad row with db api executemany()? Chris Angelico <rosuav@gmail.com> - 2013-03-30 15:24 +1100
      Re: How to find bad row with db api executemany()? Tim Chase <python.list@tim.thechases.com> - 2013-03-30 06:38 -0500

csiph-web