Path: csiph.com!eternal-september.org!feeder.eternal-september.org!nntp.eternal-september.org!.POSTED!not-for-mail
From: Veek M <veekjunk@foobar.com>
Newsgroups: comp.lang.python
Subject: Re: open: 'ascii', 'backslashreplace' not behaving as expected - why?
Date: Sat, 8 Aug 2026 04:33:15 -0000 (UTC)
Organization: A noiseless patient Spider
Lines: 27
Message-ID: <1156bib$15v7o$3@dont-email.me>
References: <1155tpv$13on7$1@dont-email.me> <1156074$14bmn$1@dont-email.me> <1156av3$15v7o$1@dont-email.me> <1156b47$15v7o$2@dont-email.me>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Injection-Date: Sat, 08 Aug 2026 04:33:16 +0000 (UTC)
Injection-Info: dont-email.me; logging-data="1244408"; mail-complaints-to="abuse@eternal-september.org";	posting-account="U2FsdGVkX1+ZbFIHaAEUoubQqfQgToiY"; posting-host="d75c4908dd89aa0bd2d1c9efc307aee7"
User-Agent: Pan/0.154 (Izium; 517acf4)
Cancel-Lock: sha1:uFx+WEPoDKPls68Gc5dr4YVkPQ8= sha256:EetfWgPSx3i3NI7oyoWaMV7ge7in6NL3P8VEOa50Rt8= sha1:UwAs+odO2z9fyXaPtOOTAvfoTHE= sha256:QWC70MKauQdqqgLXw1GDd2gqvZX9ADACqoSphO9DV2U=
Xref: csiph.com comp.lang.python:197870

On Sat, 8 Aug 2026 04:25:43 -0000 (UTC), Veek M wrote:

> On Sat, 8 Aug 2026 04:22:59 -0000 (UTC), Veek M wrote:
> 
>> On Sat, 8 Aug 2026 01:19:32 -0000 (UTC), Lawrence D’Oliveiro wrote:
>> 
>>> b'\xef\xbf\xbf\n'.decode()
>> 
>> Could you explain how it works and what exactly is going on?
>> 
>> fh.readline() returns a unicode string with the funny chars (bytes 0xff
>> 0xff) encoded as \\xef \\xbf \\xbf - why is it \\? why not just use a
>> single u'\xef\xbf\xbf' - why is he escaping the '\'.
>> 
>> Also - how exactly is he getting ef bf bf and not ff ff?
> 
> oh is 0xff 0xff when encoded to disk in utf-8
> (sys.getsystemdefaultencoding) 0xef 0xbf 0xbf?

yes,
root@laptopveek:/tmp# od -x /tmp/x
0000000 bfef 0abf
0000004

it's the raw utf-8 encoded as bytes but since it is a unicode string why 
doesn't he save it as u'\xef\xbf\xbf' why does he escape the '\' and make 
it '\\x'