Path: csiph.com!usenet.pasdenom.info!weretis.net!feeder1.news.weretis.net!feeder.erje.net!eu.feeder.erje.net!bcyclone01.am1.xlned.com!bcyclone01.am1.xlned.com!newsfeed.xs4all.nl!newsfeed3.news.xs4all.nl!xs4all!post.news.xs4all.nl!not-for-mail
Return-Path: <zachary.ware@gmail.com>
X-Original-To: python-list@python.org
Delivered-To: python-list@mail.python.org
X-Spam-Status: OK 0.008
X-Spam-Evidence: '*H*': 0.98; '*S*': 0.00; 'subject:Python': 0.06; 'odd': 0.07; 'inserted': 0.09; 'subject:into': 0.09; '11:32': 0.16; 'bytes).': 0.16; 'cp1252': 0.16; 'csv': 0.16; 'spit': 0.16; 'sqlite': 0.16; 'sucked': 0.16; 'swim': 0.16; 'sender:addr:gmail.com': 0.17; 'fix': 0.17; 'wrote:': 0.18; 'seems': 0.21; 'feb': 0.22; '>>>': 0.22; 'import': 0.22; 'cc:addr:gmail.com': 0.22; 'skip': 0.24; 'mon,': 0.24; 'cc:2**0': 0.24; 'script': 0.25; 'header:In-Reply-To:1': 0.27; 'am,': 0.29; 'message-id:@mail.gmail.com': 0.30; 'file': 0.32; 'agree': 0.35; 'point.': 0.35; 'received:google.com': 0.35; 'there': 0.35; 'skip:f 40': 0.36; 'url:org': 0.36; 'to:addr:python-list': 0.38; 'bad': 0.39; 'to:addr:python.org': 0.39; '8bit%:50': 0.68; 'st.': 0.69; 'day': 0.76; '2015': 0.84; 'fails,': 0.84; 'masters': 0.91; 'url:latest': 0.91
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc:content-type:content-transfer-encoding; bh=t/moqMNUlvXhGgxcaZXTYZSHtK23twM6U+5bIYFRs0I=; b=tqrTgsj1YK1ad/Yo+D+VvqDYwLepfiW3yhJ5r2tuimTvEcaQ9IVX3J6H3ZuyJxTyuS l5FEpAVIfa5+poAdiCnqqkb3ykbIZPQ5zW7TBeFMBMZf8lzP81RqVim/lralmfu/LBPk UvkQ3xg6zAdgRgiC5CIJryjUu1Eo1gHYLtn8HxwB9Inp5jaE6zdPfBz2rulsSAXBK/rD Ys7sSpZGC1ZUA817/CQ1VYoSQbgyDP6IiljgTIyWiKBZV+jw+D+s/kVh+U4XSbJkihCe twec/m7mpHK/1BvO3cdRfjMj3G0iOKRNUmUn7OdZ0cduP8Njtr7VzCPN02ItVt1S2nUm ZrAQ==
X-Received: by 10.202.26.72 with SMTP id a69mr12261834oia.119.1423512363239; Mon, 09 Feb 2015 12:06:03 -0800 (PST)
MIME-Version: 1.0
Sender: zachary.ware@gmail.com
In-Reply-To: <CANc-5UxTFr-jUbDxEpWyBUFF1sznRPJ2ji6aGicCaru9V5rc_Q@mail.gmail.com>
References: <mailman.18555.1423453501.18130.python-list@python.org> <54d83ced$0$12895$c3e8da3$5496439d@news.astraweb.com> <CANc-5UxTFr-jUbDxEpWyBUFF1sznRPJ2ji6aGicCaru9V5rc_Q@mail.gmail.com>
From: Zachary Ware <zachary.ware+pylist@gmail.com>
Date: Mon, 9 Feb 2015 14:05:43 -0600
X-Google-Sender-Auth: hkuKKsFDGzva4kMvcusHsMNlNK0
Subject: Re: Python 3.x stuffing utf-8 into SQLite db
To: Python <python-list@python.org>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
X-BeenThere: python-list@python.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe: <https://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive: <http://mail.python.org/pipermail/python-list/>
List-Post: <mailto:python-list@python.org>
List-Help: <mailto:python-list-request@python.org?subject=help>
List-Subscribe: <https://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
Newsgroups: comp.lang.python
Message-ID: <mailman.18575.1423512371.18130.python-list@python.org>
Lines: 25
NNTP-Posting-Host: 2001:888:2000:d::a6
X-Trace: 1423512371 news.xs4all.nl 2956 [2001:888:2000:d::a6]:35170
X-Complaints-To: abuse@xs4all.nl
X-Received-Bytes: 4314
X-Received-Body-CRC: 1441157663
Xref: csiph.com comp.lang.python:85399

On Mon, Feb 9, 2015 at 11:32 AM, Skip Montanaro
<skip.montanaro@gmail.com> wrote:
> LibreOffice spit out a CSV file
> (with those three odd bytes). My script sucked in the CSV file and
> inserted data into my SQLite db.

If all else fails, you can try ftfy to fix things:
http://ftfy.readthedocs.org/en/latest/

   >>> import ftfy
   >>> ftfy.fix_text('Anderson Barracuda Masters - 2010 St.
Patrick=C3=A2=E2=82=AC=E2=84=A2s Day Swim Meet')
   "Anderson Barracuda Masters - 2010 St. Patrick's Day Swim Meet"

It also seems to agree that there was a bad (en|de)coding with cp1252
at some point.

   >>> ftfy.fixes.fix_encoding_and_explain('Anderson Barracuda Masters
- 2010 St. Patrick=C3=A2=E2=82=AC=E2=84=A2s Day Swim Meet')
   ('Anderson Barracuda Masters - 2010 St. Patrick=E2=80=99s Day Swim Meet'=
,
[('encode', 'sloppy-windows-1252'), ('decode', 'utf-8')])

--=20
Zach