Path: csiph.com!eternal-september.org!feeder.eternal-september.org!border1.nntp.ams1.giganews.com!nntp.giganews.com!bcyclone01.am1.xlned.com!bcyclone01.am1.xlned.com!newsfeed.xs4all.nl!newsfeed8.news.xs4all.nl!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail
Return-Path: <python-python-list@m.gmane.org>
X-Original-To: python-list@python.org
Delivered-To: python-list@mail.python.org
X-Spam-Status: OK 0.002
X-Spam-Evidence: '*H*': 1.00; '*S*': 0.00; 'upgraded': 0.05; 'meantime,': 0.07; 'raises': 0.07; 'subject:sqlite3': 0.07; 'exception,': 0.09; 'received:80.91': 0.09; 'received:80.91.229': 0.09; 'received:gmane.org': 0.09; 'received:list': 0.09; 'sqlite': 0.09; 'throw': 0.09; 'python': 0.10; 'commit': 0.15; 'in-memory': 0.16; 'received:80.91.229.3': 0.16; 'received:plane.gmane.org': 0.16; 'reproduce': 0.16; 'selects': 0.16; 'sqlite3': 0.16; 'later': 0.16; 'obviously': 0.16; 'windows': 0.20; 'implicit': 0.22; 'simpler': 0.22; 'subject:problem': 0.22; 'trying': 0.22; 'module': 0.25; 'header:X-Complaints-To:1': 0.26; 'earlier': 0.27; 'this.': 0.28; "skip:' 10": 0.28; 'implicitly': 0.29; 'connection': 0.30; 'transaction': 0.30; 'connections': 0.30; 'skip:s 30': 0.31; 'post': 0.31; 'anyone': 0.32; 'another': 0.32; 'statement': 0.32; 'useful': 0.33; 'problem': 0.33; 'traceback': 0.33; "skip:' 20": 0.34; 'running': 0.34; 'on,': 0.35; 'knowledge': 0.35; 'quite': 0.35; 'level': 0.35; 'problem.': 0.35; 'according': 0.36; 'but': 0.36; 'success.': 0.36; 'to:addr:python- list': 0.36; 'two': 0.37; 'thanks': 0.37; 'received:org': 0.37; 'thought': 0.37; 'anything': 0.38; 'someone': 0.38; 'does': 0.39; 'to:addr:python.org': 0.40; 'subject:with': 0.40; 'some': 0.40; 'hope': 0.61; 'default': 0.61; 'series': 0.65; 'subject:Data': 0.66; 'here': 0.66; 'frank': 0.72; 'open,': 0.84; 'triggering': 0.84; 'steps.': 0.91; 'transactions': 0.91; 'info,': 0.93
X-Injected-Via-Gmane: http://gmane.org/
To: python-list@python.org
From: "Frank Millman" <frank@chagford.com>
Subject: Data integrity problem with sqlite3
Date: Tue, 11 Aug 2015 13:33:31 +0200
Mime-Version: 1.0
Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=original
Content-Transfer-Encoding: 7bit
X-Gmane-NNTP-Posting-Host: 197.89.67.51
X-MSMail-Priority: Normal
Importance: Normal
X-Newsreader: Microsoft Windows Live Mail 15.4.3502.922
X-MimeOLE: Produced By Microsoft MimeOLE V15.4.3502.922
X-BeenThere: python-list@python.org
X-Mailman-Version: 2.1.20+
Precedence: list
List-Id: General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe: <https://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive: <http://mail.python.org/pipermail/python-list/>
List-Post: <mailto:python-list@python.org>
List-Help: <mailto:python-list-request@python.org?subject=help>
List-Subscribe: <https://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
Newsgroups: comp.lang.python
Message-ID: <mailman.75.1439292832.3627.python-list@python.org>
Lines: 41
NNTP-Posting-Host: 2001:888:2000:d::a6
X-Trace: 1439292832 news.xs4all.nl 2885 [2001:888:2000:d::a6]:56850
X-Complaints-To: abuse@xs4all.nl
X-Received-Bytes: 4873
X-Received-Body-CRC: 2135039041
Xref: csiph.com comp.lang.python:95254

Hi all

I have a 'data integrity' problem with sqlite3 that I have been battling 
with for a while. I have not got to the bottom of it yet but I do have some 
useful info, so I thought I would post it here in the hope that someone with 
some knowledge of the internals of the python sqlite3 module can throw some 
light on it.

I am running python 3.4.3 on Windows 7, and I have upgraded sqlite from the 
original '3.7.something' to '3.8.6'. I do not change the isolation level 
from the default setting.

I have a transaction with a number of steps. One of the later steps raises 
an exception, and I execute a rollback. However, some of the earlier steps 
are committed to the database. Obviously this is a major problem.

I have added 'set_trace_callback' to see exactly what is going on, and in 
the middle of my series of commands I find the following -

    COMMIT
    BEGIN IMMEDIATE

According to the docs, the sqlite3 module commits transactions implicitly 
before a non-DML, non-query statement (i. e. anything other than 
SELECT/INSERT/UPDATE/DELETE/REPLACE).

In my traceback I can only see SELECTs and UPDATEs following the implicit 
commit, so I do not know what is triggering it. I am trying to reproduce the 
problem in a simpler example, but so far without success. My running program 
has two connections to the database open, another connection to an in-memory 
database, and it is all running under asyncio, so it is quite difficult to 
mimic all of this.

I will persevere, but in the meantime, does anyone happen to know under what 
other circumstances sqlite3 might issue an implicit commit?

Thanks

Frank Millman