Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #35203

Re: Python3 + sqlite3: Where's the bug?

Show all headers | View raw

On Fri, Dec 21, 2012 at 2:20 AM, Johannes Bauer <dfnsonfsduifb@gmx.de> wrote:
> Hmm, but this:
>
>                 result = cursor.fetchone()
>                         yield result
>
> Works nicely -- only the fetchmany() makes the example break.

Okay, now it's sounding specific to sqlite. I'll bow out. :)

>
>> Would it spoil your performance improvements to do all the fetchmany
>> calls before yielding anything?
>
> Well this would effectively then be a fetchall() call -- this is
> problematic since the source data is LARGE (spekaing of gigabytes of
> data here).

That would be a "yes", then. Scratch that!

>> Alternatively, can you separate the
>> two by opening a separate database connection for the foo-reading (so
>> it isn't affected by the commit)?
>
> At that point in the code I don't actually have a filename anymore,
> merely the connection. But shouldn't the cursor actually be the
> "correct" solution? I.e. in theory, should the example work at all or am
> I thinking wrong?

You say "db.commit()", not "cur2.commit()", so I don't see that a
cursor would un-break what part-way commits is breaking.

> Because if I'm approaching this from the wrong angle, I'll have no
> choice but to change all that code to open separate connections to the
> same file (something that currently are no provisions for).

Is that an sqlite limitation, or just one of your code?

I poked around at the sqlite3 docs, but didn't find any obvious
"clone" option on the connection, nor a way to retrieve the file name.
That would have been fairly convenient. Oh well.

ChrisA

Back to comp.lang.python | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Python3 + sqlite3: Where's the bug? Johannes Bauer <dfnsonfsduifb@gmx.de> - 2012-12-20 15:52 +0100
  Re: Python3 + sqlite3: Where's the bug? Chris Angelico <rosuav@gmail.com> - 2012-12-21 02:05 +1100
    Re: Python3 + sqlite3: Where's the bug? Johannes Bauer <dfnsonfsduifb@gmx.de> - 2012-12-20 16:20 +0100
      Re: Python3 + sqlite3: Where's the bug? Chris Angelico <rosuav@gmail.com> - 2012-12-21 02:55 +1100
      Re: Python3 + sqlite3: Where's the bug? Hans Mulder <hansmu@xs4all.nl> - 2012-12-20 17:35 +0100
  Re: Python3 + sqlite3: Where's the bug? inq1ltd <inq1ltd@inqvista.com> - 2012-12-20 10:57 -0500

csiph-web