Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #110683

Re: Processing text data with different encodings

Path csiph.com!fu-berlin.de!uni-berlin.de!not-for-mail
From Chris Angelico <rosuav@gmail.com>
Newsgroups comp.lang.python
Subject Re: Processing text data with different encodings
Date Tue, 28 Jun 2016 21:09:42 +1000
Lines 21
Message-ID <mailman.72.1467112190.2358.python-list@python.org> (permalink)
References <ubl94dxa2e.ln2@news.c0t0d0s0.de> <CAPTjJmqRXOkTjBxwO=vR29gFQfmg-nmBvgeTFucAAjMUWHnq2Q@mail.gmail.com> <mailman.67.1467099972.2358.python-list@python.org> <4gp94dxgr7.ln2@news.c0t0d0s0.de> <5772518a$0$1606$c3e8da3$5496439d@news.astraweb.com> <ik0a4dxatk.ln2@news.c0t0d0s0.de> <CAPTjJmp2iXEpLKpXPGRG+Xoeo9=CGOP9orFeRA5m=Dg2u4CKkA@mail.gmail.com>
Mime-Version 1.0
Content-Type text/plain; charset=UTF-8
X-Trace news.uni-berlin.de 49dBx+WEd30xBLn+c0p2Ugb78DDNk5v38PqxLsSYz4Xg==
Return-Path <rosuav@gmail.com>
X-Original-To python-list@python.org
Delivered-To python-list@mail.python.org
X-Spam-Status OK 0.035
X-Spam-Evidence '*H*': 0.93; '*S*': 0.00; 'subject:text': 0.04; 'badly': 0.07; 'great.': 0.07; 'cc:addr:python-list': 0.09; 'encoding.': 0.09; '2016': 0.16; 'buggy': 0.16; 'encodings': 0.16; 'from:addr:rosuav': 0.16; 'from:name:chris angelico': 0.16; 'received:io': 0.16; 'received:psf.io': 0.16; 'wrote:': 0.16; '>>>': 0.20; 'cc:2**0': 0.20; 'cc:addr:python.org': 0.20; 'header :In-Reply-To:1': 0.24; 'header': 0.24; 'message- id:@mail.gmail.com': 0.27; 'developing': 0.28; "can't": 0.32; 'says': 0.32; 'bill': 0.32; 'michael': 0.33; "d'aprano": 0.33; 'steven': 0.33; 'tue,': 0.34; 'received:google.com': 0.35; 'could': 0.35; 'text': 0.35; 'but': 0.36; 'pm,': 0.36; 'subject:: ': 0.37; 'client': 0.37; 'sources': 0.37; 'experience,': 0.38; 'data': 0.39; 'subject:with': 0.40; 'your': 0.60; 'spam.': 0.61; 'side': 0.62; 'different': 0.63; 'person,': 0.66; 'commercial': 0.73; 'brilliant.': 0.84; 'chrisa': 0.84; 'fragments': 0.84; 'welle': 0.84; 'to:none': 0.91; 'imagine': 0.96
DKIM-Signature v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:cc; bh=Ik7Jlu36kbSK6wuJ4XHbhV5BVHFFJW6UKjh3UuZzcFk=; b=sTmK7sVL+7OGEaW62IdXmddfcuYIhSWqyot0fzaa0pRSdkBK8EIZP4RLOpZmLGiJ63 mGqyuSFFc9y1h39/3EpTuF0vj1muWsKzjHE39N8qoxdD6LwDHniOp1w1N0Yo8DHiahjF o7ifNsD7mzPW90XoUU8AjtSpJ/L2rVvHjNsddPgg0CJ7WpOyFqAgmlwhIdS+j7yIQapz vRKXhGMYoISrCoKCNJH1jLg3MH7ebRry9oKXq3f2To2o37D6eXJOzMJZcoRGz+cg+LSy xm57qyboQ4RGxG2ikYJE4Cumwr1FmEZKNK75ZjCBOVl5sevJgVcrt55dBa7a9SjA2z59 aIRg==
X-Google-DKIM-Signature v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:cc; bh=Ik7Jlu36kbSK6wuJ4XHbhV5BVHFFJW6UKjh3UuZzcFk=; b=QlULisxJFwJCys1MWsTIBSt//0mU5R2sxjwC2PyYxnb9Sc8F1HQLxp0Sl0J3xll/S9 dOKqkzQqi/TMEhO/2AyiDS/XMPVzysU7osFOKbojz2gCCp1/woyVB5BpZQmUGQg1gSVP 1SJzJP39qLM36jvySBDi3RoRSS2pjtYXbZOgXiGjH6HlcRZqEbL9NZS6M90ffZJjRfGh i5RZL4VWBNzBIYL0BAxpM39+zd8fiM32j8XetdWLo/xbaaXoumvML+6xt3NWYcwWdaaO 0Aot2eKBdM6OCfo70230+xIQzPD1ggEA+ICUDkaqIR/3nZuSeCHFhswK0TY3H5h2wQQA 1Wxg==
X-Gm-Message-State ALyK8tKKMpdKKTY4/WryjJoyt8eL5qtFdXF4m25ScCiiUpibD1d19oi5JKKpEUBc5BNhRanxQgPp8kIuBuPO8Q==
X-Received by 10.194.70.41 with SMTP id j9mr2512287wju.30.1467112183572; Tue, 28 Jun 2016 04:09:43 -0700 (PDT)
In-Reply-To <ik0a4dxatk.ln2@news.c0t0d0s0.de>
X-BeenThere python-list@python.org
X-Mailman-Version 2.1.22
Precedence list
List-Id General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe <https://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive <http://mail.python.org/pipermail/python-list/>
List-Post <mailto:python-list@python.org>
List-Help <mailto:python-list-request@python.org?subject=help>
List-Subscribe <https://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
X-Mailman-Original-Message-ID <CAPTjJmp2iXEpLKpXPGRG+Xoeo9=CGOP9orFeRA5m=Dg2u4CKkA@mail.gmail.com>
X-Mailman-Original-References <ubl94dxa2e.ln2@news.c0t0d0s0.de> <CAPTjJmqRXOkTjBxwO=vR29gFQfmg-nmBvgeTFucAAjMUWHnq2Q@mail.gmail.com> <mailman.67.1467099972.2358.python-list@python.org> <4gp94dxgr7.ln2@news.c0t0d0s0.de> <5772518a$0$1606$c3e8da3$5496439d@news.astraweb.com> <ik0a4dxatk.ln2@news.c0t0d0s0.de>
Xref csiph.com comp.lang.python:110683

Show key headers only | View raw

On Tue, Jun 28, 2016 at 8:37 PM, Michael Welle <mwe012008@gmx.net> wrote:
> Steven D'Aprano <steve@pearwood.info> writes:
>
>> On Tue, 28 Jun 2016 06:35 pm, Michael Welle wrote:
>>
>>> my original data is email. The mail header says it's utf-8, but you will
>>> find three or four different encodings in one email. I think at the
>>> sending side they just glue different text fragments from different
>>> sources together without thinking about the encoding.
>>
>> Is this spam? In my experience, the only email that is that badly
>> constructed is spam. I can't imagine how it could be email from a person,
>> coming from a mail client like Thunderbird or Outlook.
> it's mail from an international company. It's not generated by a person
> using an ordinary email client. Other than that your are right ;).

So..... buggy commercial email. Great. Just brilliant.

Can you bill them for your time developing this hack?

ChrisA

Back to comp.lang.python | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Processing text data with different encodings Michael Welle <mwe012008@gmx.net> - 2016-06-28 09:25 +0200
  Re: Processing text data with different encodings Chris Angelico <rosuav@gmail.com> - 2016-06-28 17:46 +1000
    Re: Processing text data with different encodings Michael Welle <mwe012008@gmx.net> - 2016-06-28 10:35 +0200
      Re: Processing text data with different encodings Steven D'Aprano <steve@pearwood.info> - 2016-06-28 20:29 +1000
        Re: Processing text data with different encodings Michael Welle <mwe012008@gmx.net> - 2016-06-28 12:37 +0200
          Re: Processing text data with different encodings Chris Angelico <rosuav@gmail.com> - 2016-06-28 21:09 +1000
  Re: Processing text data with different encodings Peter Otten <__peter__@web.de> - 2016-06-28 10:30 +0200
    Re: Processing text data with different encodings Michael Welle <mwe012008@gmx.net> - 2016-06-28 12:17 +0200
      Re: Processing text data with different encodings Michael Welle <mwe012008@gmx.net> - 2016-06-28 12:44 +0200
      Re: Processing text data with different encodings Steven D'Aprano <steve@pearwood.info> - 2016-06-28 21:26 +1000
        Re: Processing text data with different encodings Michael Welle <mwe012008@gmx.net> - 2016-06-28 14:30 +0200
          Re: Processing text data with different encodings Steven D'Aprano <steve@pearwood.info> - 2016-06-29 00:52 +1000
            Re: Processing text data with different encodings Random832 <random832@fastmail.com> - 2016-06-28 11:01 -0400
            Re: Processing text data with different encodings Michael Welle <mwe012008@gmx.net> - 2016-06-28 17:52 +0200
              Re: Processing text data with different encodings Michael Welle <mwe012008@gmx.net> - 2016-06-29 06:45 +0200
          Re: Processing text data with different encodings Steven D'Aprano <steve@pearwood.info> - 2016-06-29 01:11 +1000
      Re: Processing text data with different encodings Peter Otten <__peter__@web.de> - 2016-06-28 13:31 +0200
        Re: Processing text data with different encodings Michael Welle <mwe012008@gmx.net> - 2016-06-28 15:16 +0200
  Re: Processing text data with different encodings Chris Angelico <rosuav@gmail.com> - 2016-06-28 20:25 +1000
  Re: Processing text data with different encodings Random832 <random832@fastmail.com> - 2016-06-28 11:52 -0400
  Re: Processing text data with different encodings Chris Angelico <rosuav@gmail.com> - 2016-06-29 04:03 +1000

csiph-web