Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #71558

Re: Why isn't my re.sub replacing the contents of my MS Word file?

Show all headers | View raw

On Tue, 13 May 2014 23:12:40 -0700, wxjmfauth wrote:

> Le mardi 13 mai 2014 22:26:51 UTC+2, MRAB a écrit :
>> On 2014-05-13 20:01, scottcabit@gmail.com wrote:
>> 
>> > On Tuesday, May 13, 2014 9:49:12 AM UTC-4, Steven D'Aprano wrote:
>> 
>> 
>> >>
>> >> You may have missed my follow up post, where I said I had not
>> >> noticed you
>> 
>> >> were operating on a binary .doc file.
>> 
>> 
>> >>
>> >> If you're not willing or able to use a full-blown doc parser, say by
>> 
>> >> controlling Word or LibreOffice, the other alternative is to do
>> >> something
>> 
>> >> quick and dirty that might work most of the time. Open a doc file,
>> >> or
>> 
>> >> multiple doc files, in a hex editor and *hopefully* you will be able
>> >> to
>> 
>> >> see chunks of human-readable text where you can identify how
>> >> en-dashes
>> 
>> >> and similar are stored.
>> 
>> 
>> >
>> >    I created a .doc file and opened it with UltraEdit in binary (Hex)
>> >    mode. What I see is that there are two characters, one for ndash
>> >    and one for mdash, each a single byte long. 0x96 and 0x97.
>> 
>> >    So I tried this: fStr = re.sub(b'\0x96',b'-',fStr)
>> 
>> 
>> >
>> >    that did nothing in my file. So I tried this: fStr =
>> >    re.sub(b'0x97',b'-',fStr)
>> 
>> 
>> >
>> >    which also did nothing.
>> 
>> >    So, for fun I also tried to just put these wildcards in my
>> >    re.findall so I added |Part \0x96|Part \0x97    to no avail.
>> 
>> 
>> >
>> >    Obviously 0x96 and 0x97 are NOT being interpreted in a re.findall
>> >    or re.sub as hex byte values of 96 and 97 hexadecimal using my
>> >    current syntax.
>> 
>> 
>> >
>> >    So here's my question...if I want to replace all ndash  or mdash
>> >    values with regular '-' symbols using re.sub, what is the proper
>> >    syntax to do so?
>> 
>> 
>> >
>> >    Thanks!
>> 
>> 
>> >
>> 0x96 is a hexadecimal literal for an int. Within a string you need \x96
>> 
>> (it's \x for 2 hex digits, \u for 4 hex digits, \U for 8 hex digits).
> 
> 
> ----------------
> 
>>>> b'0x61' == b'0x61'
> True
>>>> b'0x96' == b'\x96'
> False
> 
> 
> - Python and the coding of characters is an unbelievable mess.
> - Unicode a joke.
> - I can make Python failing with any valid sequence of chars I wish.
> - There is a difference between "look, my code work with my chars" and
> "this code is safely working with any chars".
> 
> jmf


0x96 is not valid ASCII neither is it a valid unicode character in any 
encoding scheme I am familiar with
it is therefore no surprise that python refuses to encode it
it looks like this file is in ANSI - ISO-8859-1

regular expressions are probably overkill fro this issue
loop through the byte array & replace the bytes as needed.

-- 
Under deadline pressure for the next week.  If you want something, it can 
wait.
Unless it's blind screaming paroxysmally hedonistic...

Back to comp.lang.python | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Why isn't my re.sub replacing the contents of my MS Word file? scottcabit@gmail.com - 2014-05-09 12:51 -0700
  Re: Why isn't my re.sub replacing the contents of my MS Word file? MRAB <python@mrabarnett.plus.com> - 2014-05-09 21:03 +0100
    Re: Why isn't my re.sub replacing the contents of my MS Word file? scottcabit@gmail.com - 2014-05-09 13:46 -0700
  Re: Why isn't my re.sub replacing the contents of my MS Word file? Chris Angelico <rosuav@gmail.com> - 2014-05-10 06:08 +1000
  Re: Why isn't my re.sub replacing the contents of my MS Word file? Tim Chase <python.list@tim.thechases.com> - 2014-05-09 15:09 -0500
    Re: Why isn't my re.sub replacing the contents of my MS Word file? scottcabit@gmail.com - 2014-05-09 13:49 -0700
      Re: Why isn't my re.sub replacing the contents of my MS Word file? Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-05-10 00:31 +0000
  Re: Why isn't my re.sub replacing the contents of my MS Word file? Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-05-10 00:12 +0000
    Re: Why isn't my re.sub replacing the contents of my MS Word file? scottcabit@gmail.com - 2014-05-12 10:35 -0700
      Re: Why isn't my re.sub replacing the contents of my MS Word file? Rustom Mody <rustompmody@gmail.com> - 2014-05-12 20:00 -0700
      Re: Why isn't my re.sub replacing the contents of my MS Word file? Dave Angel <davea@davea.name> - 2014-05-12 17:15 -0400
      Re: Why isn't my re.sub replacing the contents of my MS Word file? Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-05-13 13:49 +0000
        Re: Why isn't my re.sub replacing the contents of my MS Word file? Chris Angelico <rosuav@gmail.com> - 2014-05-13 23:55 +1000
        Re: Why isn't my re.sub replacing the contents of my MS Word file? scottcabit@gmail.com - 2014-05-13 12:01 -0700
          Re: Why isn't my re.sub replacing the contents of my MS Word file? MRAB <python@mrabarnett.plus.com> - 2014-05-13 21:26 +0100
            Re: Why isn't my re.sub replacing the contents of my MS Word file? wxjmfauth@gmail.com - 2014-05-13 23:12 -0700
              Re: Why isn't my re.sub replacing the contents of my MS Word file? alister <alister.nospam.ware@ntlworld.com> - 2014-05-14 13:21 +0000
            Re: Why isn't my re.sub replacing the contents of my MS Word file? scottcabit@gmail.com - 2014-05-14 07:40 -0700
  Re: Why isn't my re.sub replacing the contents of my MS Word file? Rustom Mody <rustompmody@gmail.com> - 2014-05-09 21:22 -0700
    Re: Why isn't my re.sub replacing the contents of my MS Word file? wxjmfauth@gmail.com - 2014-05-10 00:11 -0700
      Re: Why isn't my re.sub replacing the contents of my MS Word file? Tim Golden <mail@timgolden.me.uk> - 2014-05-10 09:49 +0100

csiph-web