Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #80437 > unrolled thread

Python script that does batch find and replace in txt files

Back to article view | Back to comp.lang.python

Contents

  Python script that does batch find and replace in txt files Syed Khalid <khalidness@gmail.com> - 2014-11-09 11:58 -0800
    Re: Python script that does batch find and replace in txt files Mark Lawrence <breamoreboy@yahoo.co.uk> - 2014-11-09 20:17 +0000
    Re: Python script that does batch find and replace in txt files Albert-Jan Roskam <fomcl@yahoo.com> - 2014-11-09 20:23 +0000
    Re: Python script that does batch find and replace in txt files Syed Khalid <khalidness@gmail.com> - 2014-11-10 02:22 +0530
      Re: Python script that does batch find and replace in txt files Syed Khalid <khalidness@gmail.com> - 2014-11-09 13:08 -0800
        RE: Python script that does batch find and replace in txt files "Clayton Kirkwood" <crk@godblessthe.us> - 2014-11-09 15:03 -0800
    Re: Python script that does batch find and replace in txt files Syed Khalid <khalidness@gmail.com> - 2014-11-10 02:50 +0530
    Re: Python script that does batch find and replace in txt files MRAB <python@mrabarnett.plus.com> - 2014-11-09 21:44 +0000
    Re: Python script that does batch find and replace in txt files Syed Khalid <khalidness@gmail.com> - 2014-11-10 03:21 +0530
    Re: Python script that does batch find and replace in txt files Denis McMahon <denismfmcmahon@gmail.com> - 2014-11-11 20:04 +0000

#80437 — Python script that does batch find and replace in txt files

Python script that does batch find and replace in txt files Need a python script that opens all .txt files in a folder find replace/delete text and save files.

I have text files and I need to perform below steps for each file. 

Step 1: Put cursor at start of file and Search for "Contact's Name:". Delete all the rows before it.
Step 2: Put cursor at end of file, Search for "Contact's Name:" select option UP.
Step 3: Search for "Photo of the" Replace with blanks
Step 4: Search for "Contact is" Replace with blanks
Step 5: Search for "Contact's Name:" Replace with blanks
Step 6: Search for "Age:" Replace with blanks
Step 7: Search for "Sex:" Replace with blanks
Step 8: Search for "House No:" Replace with blanks
Step 9: Search for "available" Replace with blanks
Step 10: Remove Empty Lines Containing Blank Characters from file
Step 11: Trim Leading Space for each line
Step 12: Trim Trailing Space after each line
Step 13: Search for - (hyphen) Replace with _ (underscore)
Step 14: Save file.

Currently I have recorded a macro in Notepad++.
I open each file, run macro and save file.
As there are many files I was looking for a program to automate the process. 

I posted the same query in Notepad++ forum. I got a reply that it can be done by using Python script.

Kindly do the needful. 

Thank you.
khalidness

[toc] | [next] | [standalone]

#80439

On 09/11/2014 19:58, Syed Khalid wrote:
> Python script that does batch find and replace in txt files Need a python script that opens all .txt files in a folder find replace/delete text and save files.
>
> I have text files and I need to perform below steps for each file.
>
> Step 1: Put cursor at start of file and Search for "Contact's Name:". Delete all the rows before it.
> Step 2: Put cursor at end of file, Search for "Contact's Name:" select option UP.
> Step 3: Search for "Photo of the" Replace with blanks
> Step 4: Search for "Contact is" Replace with blanks
> Step 5: Search for "Contact's Name:" Replace with blanks
> Step 6: Search for "Age:" Replace with blanks
> Step 7: Search for "Sex:" Replace with blanks
> Step 8: Search for "House No:" Replace with blanks
> Step 9: Search for "available" Replace with blanks
> Step 10: Remove Empty Lines Containing Blank Characters from file
> Step 11: Trim Leading Space for each line
> Step 12: Trim Trailing Space after each line
> Step 13: Search for - (hyphen) Replace with _ (underscore)
> Step 14: Save file.
>
> Currently I have recorded a macro in Notepad++.
> I open each file, run macro and save file.
> As there are many files I was looking for a program to automate the process.
>
> I posted the same query in Notepad++ forum. I got a reply that it can be done by using Python script.
>
> Kindly do the needful.
>
> Thank you.
> khalidness
>

No problem once your cheque made payable to the Python Software 
Foundation has been cashed.  I'll pluck a figure of £200 out of the air, 
YMMV.

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence

[toc] | [prev] | [next] | [standalone]

#80440

----- Original Message -----
> From: Syed Khalid <khalidness@gmail.com>
> To: python-list@python.org
> Cc: 
> Sent: Sunday, November 9, 2014 8:58 PM
> Subject: Python script that does batch find and replace in txt files
> 
> Python script that does batch find and replace in txt files Need a python script 
> that opens all .txt files in a folder find replace/delete text and save files.
> 
> I have text files and I need to perform below steps for each file. 
> 
> Step 1: Put cursor at start of file and Search for "Contact's 
> Name:". Delete all the rows before it.
> Step 2: Put cursor at end of file, Search for "Contact's Name:" 
> select option UP.
> Step 3: Search for "Photo of the" Replace with blanks
> Step 4: Search for "Contact is" Replace with blanks
> Step 5: Search for "Contact's Name:" Replace with blanks
> Step 6: Search for "Age:" Replace with blanks
> Step 7: Search for "Sex:" Replace with blanks
> Step 8: Search for "House No:" Replace with blanks
> Step 9: Search for "available" Replace with blanks
> Step 10: Remove Empty Lines Containing Blank Characters from file
> Step 11: Trim Leading Space for each line
> Step 12: Trim Trailing Space after each line
> Step 13: Search for - (hyphen) Replace with _ (underscore)

> Step 14: Save file.

something like (untested)


import glob, codecs, re, os

regex = re.compile(r"Age: |Sex: |House No: ") # etc etc

for txt in glob.glob("/some/path/*.txt"):
    with codecs.open(txt, encoding="utf-8") as f:
        oldlines = f.readlines()
    for i, line in enumerate(oldlines):
        if "Contact's Name: " in line:
            break
    newlines = [regex.sub("", line).strip().replace("-", "_") for line in oldlines[i:]
    with codecs.open(txt + "_out.txt", "wb", encoding="utf-8") as w:
        w.write(os.linesep.join(newlines))

    
    

> 
> Currently I have recorded a macro in Notepad++.
> I open each file, run macro and save file.
> As there are many files I was looking for a program to automate the process. 
> 
> I posted the same query in Notepad++ forum. I got a reply that it can be done by 
> using Python script.
> 
> Kindly do the needful. 
> 
> Thank you.
> khalidness
> 
> -- 
> https://mail.python.org/mailman/listinfo/python-list
> 

[toc] | [prev] | [next] | [standalone]

#80441

[Multipart message — attachments visible in raw view] — view raw

Hi Albert,

Thank you for script.

I am getting the below error :

  File "EamClean.log", line 12
    with codecs.open(txt + "_out.txt", "wb", encoding="utf-8") as w:
       ^
SyntaxError: invalid syntax

Kindly do the needful.

On Mon, Nov 10, 2014 at 1:53 AM, Albert-Jan Roskam <fomcl@yahoo.com> wrote:

>
>
>
>
> ----- Original Message -----
> > From: Syed Khalid <khalidness@gmail.com>
> > To: python-list@python.org
> > Cc:
> > Sent: Sunday, November 9, 2014 8:58 PM
> > Subject: Python script that does batch find and replace in txt files
> >
> > Python script that does batch find and replace in txt files Need a
> python script
> > that opens all .txt files in a folder find replace/delete text and save
> files.
> >
> > I have text files and I need to perform below steps for each file.
> >
> > Step 1: Put cursor at start of file and Search for "Contact's
> > Name:". Delete all the rows before it.
> > Step 2: Put cursor at end of file, Search for "Contact's Name:"
> > select option UP.
> > Step 3: Search for "Photo of the" Replace with blanks
> > Step 4: Search for "Contact is" Replace with blanks
> > Step 5: Search for "Contact's Name:" Replace with blanks
> > Step 6: Search for "Age:" Replace with blanks
> > Step 7: Search for "Sex:" Replace with blanks
> > Step 8: Search for "House No:" Replace with blanks
> > Step 9: Search for "available" Replace with blanks
> > Step 10: Remove Empty Lines Containing Blank Characters from file
> > Step 11: Trim Leading Space for each line
> > Step 12: Trim Trailing Space after each line
> > Step 13: Search for - (hyphen) Replace with _ (underscore)
>
> > Step 14: Save file.
>
> something like (untested)
>
>
> import glob, codecs, re, os
>
> regex = re.compile(r"Age: |Sex: |House No: ") # etc etc
>
> for txt in glob.glob("/some/path/*.txt"):
>     with codecs.open(txt, encoding="utf-8") as f:
>         oldlines = f.readlines()
>     for i, line in enumerate(oldlines):
>         if "Contact's Name: " in line:
>             break
>     newlines = [regex.sub("", line).strip().replace("-", "_") for line in
> oldlines[i:]
>     with codecs.open(txt + "_out.txt", "wb", encoding="utf-8") as w:
>         w.write(os.linesep.join(newlines))
>
>
>
>
> >
> > Currently I have recorded a macro in Notepad++.
> > I open each file, run macro and save file.
> > As there are many files I was looking for a program to automate the
> process.
> >
> > I posted the same query in Notepad++ forum. I got a reply that it can be
> done by
> > using Python script.
> >
> > Kindly do the needful.
> >
> > Thank you.
> > khalidness
> >
> > --
> > https://mail.python.org/mailman/listinfo/python-list
> >
>

[toc] | [prev] | [next] | [standalone]

#80442

My Script, 

I have added 

import glob, codecs, re, os

regex = re.compile(r"Age: |Sex: |House No: ") # etc etc
Script I executed in EditRocket :

for txt in glob.glob("/D:/Python/source/*.txt"):   
    with codecs.open(txt, encoding="utf-8") as f:
        oldlines = f.readlines()
    for i, line in enumerate(oldlines):
        if "Contact's Name:" in line:
            break
    newlines = [regex.sub("", line).strip().replace("-", "_") for line in oldlines[i:]
    with codecs.open(txt + "_out.txt", "wb", encoding="utf-8") as w:
        w.write(os.linesep.join(newlines))


Error Message :

  File "EamClean.log", line 12
    with codecs.open(txt + "_out.txt", "wb", encoding="utf-8") as w:
       ^
SyntaxError: invalid syntax

Kindly do the needful

[toc] | [prev] | [next] | [standalone]

#80507

>-----Original Message-----
>From: Python-list [mailto:python-list-
>bounces+crk=godblessthe.us@python.org] On Behalf Of Syed Khalid
>Sent: Sunday, November 09, 2014 1:08 PM
>To: python-list@python.org
>Subject: Re: Python script that does batch find and replace in txt files
>
>My Script,
>
>I have added
>
>import glob, codecs, re, os
>
>regex = re.compile(r"Age: |Sex: |House No: ") # etc etc Script I
>executed in EditRocket :
>
>for txt in glob.glob("/D:/Python/source/*.txt"):
>    with codecs.open(txt, encoding="utf-8") as f:
>        oldlines = f.readlines()
>    for i, line in enumerate(oldlines):
>        if "Contact's Name:" in line:
>            break
>    newlines = [regex.sub("", line).strip().replace("-", "_") for line
>in oldlines[i:]
>    with codecs.open(txt + "_out.txt", "wb", encoding="utf-8") as w:
>        w.write(os.linesep.join(newlines))
>

To my young eyes, it looks like you are missing a ] at the end of the line
before the with.

>
>Error Message :
>
>  File "EamClean.log", line 12
>    with codecs.open(txt + "_out.txt", "wb", encoding="utf-8") as w:
>       ^
>SyntaxError: invalid syntax
>
>Kindly do the needful
>
>
>
>--
>https://mail.python.org/mailman/listinfo/python-list

[toc] | [prev] | [next] | [standalone]

#80443

[Multipart message — attachments visible in raw view] — view raw

Code after adding path of .txt files :

import glob, codecs, re, os

regex = re.compile(r"Age: |Sex: |House No: ") # etc etc

for txt in glob.glob("D:/Python/source/*.txt"):
    with codecs.open(txt, encoding="utf-8") as f:
        oldlines = f.readlines()
    for i, line in enumerate(oldlines):
        if "Elector's Name:" in line:
            break
    newlines = [regex.sub("", line).strip().replace("-", "_") for line in
oldlines[i:]
    with codecs.open(txt + "_out.txt", "wb", encoding="utf-8") as w:
        w.write(os.linesep.join(newlines))


I executed code in edit rocket.

Error message :


File "EamClean.log", line 12
    with codecs.open(txt + "_out.txt", "wb", encoding="utf-8") as w:
       ^
SyntaxError: invalid syntax





On Mon, Nov 10, 2014 at 2:22 AM, Syed Khalid <khalidness@gmail.com> wrote:

> Hi Albert,
>
> Thank you for script.
>
> I am getting the below error :
>
>   File "EamClean.log", line 12
>     with codecs.open(txt + "_out.txt", "wb", encoding="utf-8") as w:
>        ^
> SyntaxError: invalid syntax
>
> Kindly do the needful.
>
> On Mon, Nov 10, 2014 at 1:53 AM, Albert-Jan Roskam <fomcl@yahoo.com>
> wrote:
>
>>
>>
>>
>>
>> ----- Original Message -----
>> > From: Syed Khalid <khalidness@gmail.com>
>> > To: python-list@python.org
>> > Cc:
>> > Sent: Sunday, November 9, 2014 8:58 PM
>> > Subject: Python script that does batch find and replace in txt files
>> >
>> > Python script that does batch find and replace in txt files Need a
>> python script
>> > that opens all .txt files in a folder find replace/delete text and save
>> files.
>> >
>> > I have text files and I need to perform below steps for each file.
>> >
>> > Step 1: Put cursor at start of file and Search for "Contact's
>> > Name:". Delete all the rows before it.
>> > Step 2: Put cursor at end of file, Search for "Contact's Name:"
>> > select option UP.
>> > Step 3: Search for "Photo of the" Replace with blanks
>> > Step 4: Search for "Contact is" Replace with blanks
>> > Step 5: Search for "Contact's Name:" Replace with blanks
>> > Step 6: Search for "Age:" Replace with blanks
>> > Step 7: Search for "Sex:" Replace with blanks
>> > Step 8: Search for "House No:" Replace with blanks
>> > Step 9: Search for "available" Replace with blanks
>> > Step 10: Remove Empty Lines Containing Blank Characters from file
>> > Step 11: Trim Leading Space for each line
>> > Step 12: Trim Trailing Space after each line
>> > Step 13: Search for - (hyphen) Replace with _ (underscore)
>>
>> > Step 14: Save file.
>>
>> something like (untested)
>>
>>
>> import glob, codecs, re, os
>>
>> regex = re.compile(r"Age: |Sex: |House No: ") # etc etc
>>
>> for txt in glob.glob("/some/path/*.txt"):
>>     with codecs.open(txt, encoding="utf-8") as f:
>>         oldlines = f.readlines()
>>     for i, line in enumerate(oldlines):
>>         if "Contact's Name: " in line:
>>             break
>>     newlines = [regex.sub("", line).strip().replace("-", "_") for line in
>> oldlines[i:]
>>     with codecs.open(txt + "_out.txt", "wb", encoding="utf-8") as w:
>>         w.write(os.linesep.join(newlines))
>>
>>
>>
>>
>> >
>> > Currently I have recorded a macro in Notepad++.
>> > I open each file, run macro and save file.
>> > As there are many files I was looking for a program to automate the
>> process.
>> >
>> > I posted the same query in Notepad++ forum. I got a reply that it can
>> be done by
>> > using Python script.
>> >
>> > Kindly do the needful.
>> >
>> > Thank you.
>> > khalidness
>> >
>> > --
>> > https://mail.python.org/mailman/listinfo/python-list
>> >
>>
>
>

[toc] | [prev] | [next] | [standalone]

#80444

On 2014-11-09 21:20, Syed Khalid wrote:
>
> Code after adding path of .txt files :
>
> import glob, codecs, re, os
>
> regex = re.compile(r"Age: |Sex: |House No: ") # etc etc
>
> for txt in glob.glob("D:/Python/source/*.txt"):
>      with codecs.open(txt, encoding="utf-8") as f:
>          oldlines = f.readlines()
>      for i, line in enumerate(oldlines):
>          if "Elector's Name:" in line:
>              break
>      newlines = [regex.sub("", line).strip().replace("-", "_") for line
> in oldlines[i:]
>      with codecs.open(txt + "_out.txt", "wb", encoding="utf-8") as w:
>          w.write(os.linesep.join(newlines))
>
>
> I executed code in edit rocket.
>
> Error message :
>
>
> File "EamClean.log", line 12
>      with codecs.open(txt + "_out.txt", "wb", encoding="utf-8") as w:
>         ^
> SyntaxError: invalid syntax
>
[snip]

The previous line is missing a ']' on the end. It should be:

     newlines = [regex.sub("", line).strip().replace("-", "_") for line 
in oldlines[i:]]

[toc] | [prev] | [next] | [standalone]

#80445

[Multipart message — attachments visible in raw view] — view raw

Albert,

Thanks a million for script,

It worked fine after I closed the bracket.


import glob, codecs, re, os

regex = re.compile(r"Age: |Sex: |House No:  ") # etc etc

for txt in glob.glob("D:/Python/source/*.txt"):
    with codecs.open(txt, encoding="utf-8") as f:
        oldlines = f.readlines()
    for i, line in enumerate(oldlines):
        if "Elector's Name:" in line:
            break
    newlines = [regex.sub("", line).strip().replace("-", "_") for line in
oldlines[i:]]
    with codecs.open(txt + "_out.txt", "wb", encoding="utf-8") as w:
        w.write(os.linesep.join(newlines))


This program is not deleting the empty lines containing blank characters.
Kindly do the needful.


On Mon, Nov 10, 2014 at 2:50 AM, Syed Khalid <khalidness@gmail.com> wrote:

>
> Code after adding path of .txt files :
>
> import glob, codecs, re, os
>
> regex = re.compile(r"Age: |Sex: |House No: ") # etc etc
>
> for txt in glob.glob("D:/Python/source/*.txt"):
>     with codecs.open(txt, encoding="utf-8") as f:
>         oldlines = f.readlines()
>     for i, line in enumerate(oldlines):
>         if "Elector's Name:" in line:
>             break
>     newlines = [regex.sub("", line).strip().replace("-", "_") for line in
> oldlines[i:]
>     with codecs.open(txt + "_out.txt", "wb", encoding="utf-8") as w:
>         w.write(os.linesep.join(newlines))
>
>
> I executed code in edit rocket.
>
> Error message :
>
>
> File "EamClean.log", line 12
>     with codecs.open(txt + "_out.txt", "wb", encoding="utf-8") as w:
>        ^
> SyntaxError: invalid syntax
>
>
>
>
>
> On Mon, Nov 10, 2014 at 2:22 AM, Syed Khalid <khalidness@gmail.com> wrote:
>
>> Hi Albert,
>>
>> Thank you for script.
>>
>> I am getting the below error :
>>
>>   File "EamClean.log", line 12
>>     with codecs.open(txt + "_out.txt", "wb", encoding="utf-8") as w:
>>        ^
>> SyntaxError: invalid syntax
>>
>> Kindly do the needful.
>>
>> On Mon, Nov 10, 2014 at 1:53 AM, Albert-Jan Roskam <fomcl@yahoo.com>
>> wrote:
>>
>>>
>>>
>>>
>>>
>>> ----- Original Message -----
>>> > From: Syed Khalid <khalidness@gmail.com>
>>> > To: python-list@python.org
>>> > Cc:
>>> > Sent: Sunday, November 9, 2014 8:58 PM
>>> > Subject: Python script that does batch find and replace in txt files
>>> >
>>> > Python script that does batch find and replace in txt files Need a
>>> python script
>>> > that opens all .txt files in a folder find replace/delete text and
>>> save files.
>>> >
>>> > I have text files and I need to perform below steps for each file.
>>> >
>>> > Step 1: Put cursor at start of file and Search for "Contact's
>>> > Name:". Delete all the rows before it.
>>> > Step 2: Put cursor at end of file, Search for "Contact's Name:"
>>> > select option UP.
>>> > Step 3: Search for "Photo of the" Replace with blanks
>>> > Step 4: Search for "Contact is" Replace with blanks
>>> > Step 5: Search for "Contact's Name:" Replace with blanks
>>> > Step 6: Search for "Age:" Replace with blanks
>>> > Step 7: Search for "Sex:" Replace with blanks
>>> > Step 8: Search for "House No:" Replace with blanks
>>> > Step 9: Search for "available" Replace with blanks
>>> > Step 10: Remove Empty Lines Containing Blank Characters from file
>>> > Step 11: Trim Leading Space for each line
>>> > Step 12: Trim Trailing Space after each line
>>> > Step 13: Search for - (hyphen) Replace with _ (underscore)
>>>
>>> > Step 14: Save file.
>>>
>>> something like (untested)
>>>
>>>
>>> import glob, codecs, re, os
>>>
>>> regex = re.compile(r"Age: |Sex: |House No: ") # etc etc
>>>
>>> for txt in glob.glob("/some/path/*.txt"):
>>>     with codecs.open(txt, encoding="utf-8") as f:
>>>         oldlines = f.readlines()
>>>     for i, line in enumerate(oldlines):
>>>         if "Contact's Name: " in line:
>>>             break
>>>     newlines = [regex.sub("", line).strip().replace("-", "_") for line
>>> in oldlines[i:]
>>>     with codecs.open(txt + "_out.txt", "wb", encoding="utf-8") as w:
>>>         w.write(os.linesep.join(newlines))
>>>
>>>
>>>
>>>
>>> >
>>> > Currently I have recorded a macro in Notepad++.
>>> > I open each file, run macro and save file.
>>> > As there are many files I was looking for a program to automate the
>>> process.
>>> >
>>> > I posted the same query in Notepad++ forum. I got a reply that it can
>>> be done by
>>> > using Python script.
>>> >
>>> > Kindly do the needful.
>>> >
>>> > Thank you.
>>> > khalidness
>>> >
>>> > --
>>> > https://mail.python.org/mailman/listinfo/python-list
>>> >
>>>
>>
>>
>

[toc] | [prev] | [next] | [standalone]

#80556

On Sun, 09 Nov 2014 11:58:49 -0800, Syed Khalid wrote:

> Python script that does batch find and replace in txt files Need a
> python script that opens all .txt files in a folder find replace/delete
> text and save files.

Sounds like you need sed, not python.

-- 
Denis McMahon, denismfmcmahon@gmail.com

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web