Path: csiph.com!fu-berlin.de!uni-berlin.de!not-for-mail
From: Malcolm Greene <python@bdurham.com>
Newsgroups: comp.lang.python
Subject: Capturing the bad codes that raise UnicodeError exceptions during decoding
Date: Thu, 04 Aug 2016 14:47:06 -0400
Lines: 10
Message-ID: <mailman.170.1470336428.6033.python-list@python.org>
References: <1470336426.893449.686225577.24458777@webmail.messagingengine.com>
Mime-Version: 1.0
Content-Type: text/plain
Content-Transfer-Encoding: 7bit
X-Trace: news.uni-berlin.de oXYzcXnvSdo1cr01o8eAAgZwX53Aq6A03pZdzPc9/WPQ==
Return-Path: <python@bdurham.com>
X-Original-To: python-list@python.org
Delivered-To: python-list@mail.python.org
X-Spam-Status: OK 0.009
X-Spam-Evidence: '*H*': 0.98; '*S*': 0.00; 'csv': 0.09; 'received:internal': 0.09; 'subject: \n ': 0.15; 'decoding': 0.16; 'from:addr:python': 0.16; 'malcolm': 0.16; 'message- id:@webmail.messagingengine.com': 0.16; 'received:10.202': 0.16; 'received:10.202.2': 0.16; 'received:10.202.2.212': 0.16; 'received:66.111': 0.16; 'received:66.111.4': 0.16; 'received:io': 0.16; 'received:messagingengine.com': 0.16; 'received:psf.io': 0.16; 'passes': 0.18; 'pass': 0.22; 'seems': 0.23; 'appreciated.': 0.27; 'figure': 0.27; 'errors.': 0.27; 'subject:that': 0.29; "i'm": 0.30; 'raising': 0.33; 'wrap': 0.33; 'level': 0.35; 'to:addr:python-list': 0.36; 'received:10': 0.37; 'received:66': 0.38; 'thank': 0.38; 'files': 0.38; 'subject:the': 0.39; 'to:addr:python.org': 0.40; 'high': 0.60; 'skip:u 10': 0.61; 'header:Message-Id:1': 0.61; 'repeat': 0.67; 'dirty': 0.93
DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d=bdurham.com; h= content-transfer-encoding:content-type:date:from:message-id :mime-version:subject:to:x-sasl-enc:x-sasl-enc; s=mesmtp; bh=1R0 T9rxEkuEWXePBoa4AGuRffW4=; b=YYJK/DuJzhxdMsVngH/MDkrdrhEPB3EuLTw dFC38H96McP21dbFkAEMItg7seO7UWFhbq9tCwG2awLS97DtLdSbuxl2znSrvymY Yz7ekXaxIkWO4WLto4bomg7Cg4X7pbn7Azyvfd12ID49Wtkou3lp2GRi4OcPX2qC fxxFBd3c=
DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d= messagingengine.com; h=content-transfer-encoding:content-type :date:from:message-id:mime-version:subject:to:x-sasl-enc :x-sasl-enc; s=smtpout; bh=1R0T9rxEkuEWXePBoa4AGuRffW4=; b=ucVZR HMCE2cl5s+PewAoNPqpUTDTi1jfTnvpepzuzDSfXOt9mu/mCZ8qtl9h4N2CPm83R sy0bcJd+AECJg24QGg36FjS78h003d/cfbzDcFc81aAQdkwPBSu1V5Mc0YxGkRsL iPxCHSLL6E/sCZyy4C5chwM9CyzgsP8mfOWq1c=
X-Sasl-Enc: b4bFeLIMrhpMOThPb09FieW/ZuHW6WbAgxDAVn7o3xyP 1470336426
X-Mailer: MessagingEngine.com Webmail Interface - ajax-b9085e99
X-Content-Filtered-By: Mailman/MimeDel 2.1.22
X-BeenThere: python-list@python.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe: <https://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive: <http://mail.python.org/pipermail/python-list/>
List-Post: <mailto:python-list@python.org>
List-Help: <mailto:python-list-request@python.org?subject=help>
List-Subscribe: <https://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
X-Mailman-Original-Message-ID: <1470336426.893449.686225577.24458777@webmail.messagingengine.com>
Xref: csiph.com comp.lang.python:112342

I'm processing a lot of dirty CSV files and would like to track the bad
codes that are raising UnicodeErrors. I'm struggling how to figure out
what the exact codes are so I can track them, them remove them, and then
repeat the decoding process for the current line until the line has been
fully decoded so I can pass this line on to the CSV reader. At a high
level it seems that I need to wrap the decoding of a line until it
passes with out any errors. Any suggestions appreciated.

Thank you,
Malcolm