Path: csiph.com!newsfeed.hal-mli.net!feeder3.hal-mli.net!newsfeed.hal-mli.net!feeder1.hal-mli.net!newsfeed.xs4all.nl!newsfeed4.news.xs4all.nl!xs4all!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail
To: python-list@python.org
From: Terry Reedy <tjreedy@udel.edu>
Subject: Re: Why BOM in logging message?
Date: Thu, 10 Jan 2013 18:45:15 -0500
References: <kcksg2$bkp$1@panix2.panix.com> <mailman.344.1357772847.2939.python-list@python.org> <roy-10B053.20174309012013@news.panix.com> <kcmoug$rk4$1@panix2.panix.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:17.0) Gecko/17.0 Thunderbird/17.0
In-Reply-To: <kcmoug$rk4$1@panix2.panix.com>
Precedence: list
Newsgroups: comp.lang.python
Message-ID: <mailman.383.1357861568.2939.python-list@python.org>
Lines: 55
NNTP-Posting-Host: 2001:888:2000:d::a6
Xref: csiph.com comp.lang.python:36595

On 1/10/2013 11:06 AM, Roy Smith wrote:

> Well, this is fascinating.  It turns out that while all of our
> machines report that they're running 2.7.3, they have two different
> versions of /usr/lib/python2.7/logging/handlers.py!
>
> -rw-r--r-- 1 root root 45076 Aug  1 05:39 /usr/lib/python2.7/logging/handlers.py
> -rw-r--r-- 1 root root 45143 Apr 20  2012 /usr/lib/python2.7/logging/handlers.py
>
> The April 24th version has the BOM code in SysLogHander.emit():

The 'BOM' you are discussing here is not a true 2 or 4 byte 
byte-order-mark, but the pseudo-BOM (pseudo because a stream of single 
bytes has no byte order within the single bytes) that Micro$tupid adds 
(sometimes) to utf-8 encoded bytes to mark their encoding as utf-8 
rather than anything else. In otherwords, it is a non-(anti-)standard 
U(nicode)E(ncoding)M(ark). It is actually the utf-8 encoding of the 
2-byte BOM, and hence not a single mark! It is really confusing when 
people use 'BOM' to refer to a UEM sequence instead of a BOM.

> |         # Message is a string. Convert to bytes as required by RFC 5424
> |        if type(msg) is unicode:
> |            msg = msg.encode('utf-8')
> |            if codecs:
> |                msg = codecs.BOM_UTF8 + msg
> |        msg = prio + msg

2.7.3 was released in April.

> and the August 1st version doesn't:
>
> |        # Message is a string. Convert to bytes as required by RFC 5424
> |        if type(msg) is unicode:
> |            msg = msg.encode('utf-8')
> |        msg = prio + msg

The issue referenced in an earlier messaged was to remove the UEM where 
it did not belong.

> Is it possible there's two different 2.7.3 builds that somehow got
> packaged by Ubuntu at different times?

As you discovered, Ubuntu sometimes updates a release with bug patches 
before we release a new official bugfix release. 2.7.4, with many 
bugfixes, is still to see the light of day.

> The pattern of which machines
> have the August code and which have the April code correlates with
> when we rolled out each server instance.

Great detective work ;-).

-- 
Terry Jan Reedy