Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #101701 > unrolled thread

problem

Back to article view | Back to comp.lang.python

Contents

  problem Shivam Gupta <mailtoshivamgupta@gmail.com> - 2016-01-14 21:23 +0530
    Re: problem Irmen de Jong <irmen.NOSPAM@xs4all.nl> - 2016-01-14 20:01 +0100
      ignoring or replacing white lines in a diff "Adriaan Renting" <renting@astron.nl> - 2016-01-14 21:22 +0100
      Re: ignoring or replacing white lines in a diff Zachary Ware <zachary.ware+pylist@gmail.com> - 2016-01-14 14:54 -0600
      Re: ignoring or replacing white lines in a diff Nathan Hilterbrand <nhilterbrand@gmail.com> - 2016-01-14 15:57 -0500
      Re: ignoring or replacing white lines in a diff "Martin A. Brown" <martin@linux-ip.net> - 2016-01-14 12:57 -0800
      Re: ignoring or replacing white lines in a diff Peter Otten <__peter__@web.de> - 2016-01-14 22:05 +0100
      Re: problem Chris Angelico <rosuav@gmail.com> - 2016-01-15 08:10 +1100
      Re: problem eryk sun <eryksun@gmail.com> - 2016-01-14 15:39 -0600
      Re: ignoring or replacing white lines in a diff "Adriaan Renting" <renting@astron.nl> - 2016-01-15 10:44 +0100
      Re: ignoring or replacing white lines in a diff Peter Otten <__peter__@web.de> - 2016-01-15 11:20 +0100

#101701 — problem

Hello,

I am using python 3.5.1 on my windows 8.1. The problem is that whenever i
save any file any after that when i run it the screen just close
immediately after i double click on python file.

Thank you.

[toc] | [next] | [standalone]

#101708

On 14-1-2016 16:53, Shivam Gupta wrote:
> Hello,
> 
> I am using python 3.5.1 on my windows 8.1. The problem is that whenever i
> save any file any after that when i run it the screen just close
> immediately after i double click on python file.
> 
> Thank you.
> 


That's not a python thing, it's the way the windows console behaves. As soon as any
script or console command it executes finishes, it closes.

Either put something like this at the end of your python scripts, to keep the console open:

raw_input("press enter to exit...")


or run them from an existing console prompt instead of clicking on them.



Irmen

[toc] | [prev] | [next] | [standalone]

#101712 — ignoring or replacing white lines in a diff

Maybe someone here has a clue what is going wrong here? Any help is
appreciated.

I'm writing a regression test for a module that generates XML.

I'm using diff to compare the results with a pregenerated one from an
earlier version.

I'm running into two problems:

The diff doesn't seem to behave properly with the -B option. (diff (GNU
diffutils) 2.8.1 on OSX 10.9)

Replacing -B with -I '^[[:space:]]*$' fixes it on the command line,
which should be exactly the same according to:
http://www.gnu.org/software/diffutils/manual/html_node/Blank-Lines.html#Blank-Lines

(for Python problem continue below)

MacRenting 21:00-159> diff -w -B test.xml xml/Ticket_6923.xml
3,5c3,5
<   <version>2.15.0</version>
<   <template version="2.15.0" author="Alwin de Jong,Adriaan Renting"
changedBy="Adriaan Renting">
<   <description>XML Template generator version 2.15.0</description>
---
>           <version>2.6.0</version>
>           <template version="2.6.0" author="Alwin de Jong"
changedBy="Alwin de Jong">
>           <description>XML Template generator version
2.6.0</description>
113d112
<
163d161
<
213d210
<
258d254
<
369d364
<
419d413
<
469d462
<
514d506
<
625d616
<
675d665
<
725d714
<
770d758
<
881d868
<
931d917
<
981d966
<
1026d1010
<
1137d1120
<
1187d1169
<
1237d1218
<
1282d1262
<

/Users/renting/src/CEP4-DevelopClusterModel-Story-Task8432-SAS/XML_generator/test
MacRenting 21:00-160> diff -w -I '^[[:space:]]*$' test.xml
xml/Ticket_6923.xml
3,5c3,5
<   <version>2.15.0</version>
<   <template version="2.15.0" author="Alwin de Jong,Adriaan Renting"
changedBy="Adriaan Renting">
<   <description>XML Template generator version 2.15.0</description>
---
>           <version>2.6.0</version>
>           <template version="2.6.0" author="Alwin de Jong"
changedBy="Alwin de Jong">
>           <description>XML Template generator version
2.6.0</description>


Now I try to use this in Python:

      cmd   = ["diff", "-w", "-I '^[[:space:]]*$'", "./xml/%s.xml" %
name, "test.xml"]
      ## -w ignores differences in whitespace
      ## -I '^[[:space:]]*$' because -B doesn't work for blank lines
(on OSX?)
      p     = subprocess.Popen(cmd, stdin=open('/dev/null'),
stdout=subprocess.PIPE, stderr=subprocess.PIPE)
      logs  = p.communicate()
      diffs = logs[0].splitlines() #stdout
      print "diff reply was %i lines long" % len(diffs)

This doesn't work. I've tried escaping the various bits, like the * and
$, even though with single quotes that should not be needed.

I tried first removing the blank lines from the file:

      import fileinput
      for line in fileinput.FileInput("test.xml",inplace=1):
        if line.rstrip():
          print line

This makes it worse, as it adds and empty line for each line in the
file.

I've tried various other options. The only thing I can think of, is
ditching Python and trying to rewrite the whole script in Bash.
(It's quite complicated, as it loops over various things and does some
pretty output in between and I'm not very fluent in Bash)

Any suggestions?

Thanks for any help provided.

Adriaan Renting.


Adriaan Renting        | Email: renting@astron.nl
Software Engineer Radio Observatory
ASTRON                 | Phone: +31 521 595 100 (797 direct)
P.O. Box 2             | GSM:   +31 6 24 25 17 28
NL-7990 AA Dwingeloo   | FAX:   +31 521 595 101
The Netherlands        | Web: http://www.astron.nl/~renting/

[toc] | [prev] | [next] | [standalone]

#101713 — Re: ignoring or replacing white lines in a diff

On Thu, Jan 14, 2016 at 2:22 PM, Adriaan Renting <renting@astron.nl> wrote:
> Any suggestions?

Instead of trying to make diff behave through subprocess, have a look
at Python's difflib: https://docs.python.org/3/library/difflib.html

In particular, I think `difflib.ndiff(first_list_of_strings,
second_list_of_strings, linejunk=difflib.IS_LINE_JUNK)` might be what
you're looking for (you may not even need to specify linejunk).

-- 
Zach

[toc] | [prev] | [next] | [standalone]

#101714 — Re: ignoring or replacing white lines in a diff

On 01/14/2016 03:22 PM, Adriaan Renting wrote:
> Maybe someone here has a clue what is going wrong here? Any help is
> appreciated.
>
> I'm writing a regression test for a module that generates XML.
>
> I'm using diff to compare the results with a pregenerated one from an
> earlier version.
>
> I'm running into two problems:
>
> The diff doesn't seem to behave properly with the -B option. (diff (GNU
> diffutils) 2.8.1 on OSX 10.9)
>
> Replacing -B with -I '^[[:space:]]*$' fixes it on the command line,
> which should be exactly the same according to:
> http://www.gnu.org/software/diffutils/manual/html_node/Blank-Lines.html#Blank-Lines
>
> (for Python problem continue below)
>
> MacRenting 21:00-159> diff -w -B test.xml xml/Ticket_6923.xml
> 3,5c3,5
> <   <version>2.15.0</version>
> <   <template version="2.15.0" author="Alwin de Jong,Adriaan Renting"
> changedBy="Adriaan Renting">
> <   <description>XML Template generator version 2.15.0</description>
> ---
>>            <version>2.6.0</version>
>>            <template version="2.6.0" author="Alwin de Jong"
> changedBy="Alwin de Jong">
>>            <description>XML Template generator version
> 2.6.0</description>
> 113d112
> <
> 163d161
> <
> 213d210
> <
> 258d254
> <
> 369d364
> <
> 419d413
> <
> 469d462
> <
> 514d506
> <
> 625d616
> <
> 675d665
> <
> 725d714
> <
> 770d758
> <
> 881d868
> <
> 931d917
> <
> 981d966
> <
> 1026d1010
> <
> 1137d1120
> <
> 1187d1169
> <
> 1237d1218
> <
> 1282d1262
> <
>
> /Users/renting/src/CEP4-DevelopClusterModel-Story-Task8432-SAS/XML_generator/test
> MacRenting 21:00-160> diff -w -I '^[[:space:]]*$' test.xml
> xml/Ticket_6923.xml
> 3,5c3,5
> <   <version>2.15.0</version>
> <   <template version="2.15.0" author="Alwin de Jong,Adriaan Renting"
> changedBy="Adriaan Renting">
> <   <description>XML Template generator version 2.15.0</description>
> ---
>>            <version>2.6.0</version>
>>            <template version="2.6.0" author="Alwin de Jong"
> changedBy="Alwin de Jong">
>>            <description>XML Template generator version
> 2.6.0</description>
>
>
> Now I try to use this in Python:
>
>        cmd   = ["diff", "-w", "-I '^[[:space:]]*$'", "./xml/%s.xml" %
> name, "test.xml"]
>        ## -w ignores differences in whitespace
>        ## -I '^[[:space:]]*$' because -B doesn't work for blank lines
> (on OSX?)
>        p     = subprocess.Popen(cmd, stdin=open('/dev/null'),
> stdout=subprocess.PIPE, stderr=subprocess.PIPE)
>        logs  = p.communicate()
>        diffs = logs[0].splitlines() #stdout
>        print "diff reply was %i lines long" % len(diffs)
>
> This doesn't work. I've tried escaping the various bits, like the * and
> $, even though with single quotes that should not be needed.
>
> I tried first removing the blank lines from the file:
>
>        import fileinput
>        for line in fileinput.FileInput("test.xml",inplace=1):
>          if line.rstrip():
>            print line
>
> This makes it worse, as it adds and empty line for each line in the
> file.
>
> I've tried various other options. The only thing I can think of, is
> ditching Python and trying to rewrite the whole script in Bash.
> (It's quite complicated, as it loops over various things and does some
> pretty output in between and I'm not very fluent in Bash)
>
> Any suggestions?
>
> Thanks for any help provided.
>
> Adriaan Renting.
>
>
> Adriaan Renting        | Email: renting@astron.nl
> Software Engineer Radio Observatory
> ASTRON                 | Phone: +31 521 595 100 (797 direct)
> P.O. Box 2             | GSM:   +31 6 24 25 17 28
> NL-7990 AA Dwingeloo   | FAX:   +31 521 595 101
> The Netherlands        | Web: http://www.astron.nl/~renting/
>
>
Without having the files handy to test with, I would suggest that you 
try replacing "-I '^[[:space:]]*$'" with r"-I '^[[:space:]]*$'" as a 
starting point.

Nathan

[toc] | [prev] | [next] | [standalone]

#101715 — Re: ignoring or replacing white lines in a diff

Hello Adriaan,

>Maybe someone here has a clue what is going wrong here? Any help is 
>appreciated.

Have you tried out this tool that does precisely what you need? to 
do yourself?

  https://pypi.python.org/pypi/xmldiff

I can't vouch specifically for it, am simply a user, but I know that 
I have used it happily in the past.  (Other CLI tools, include 
non-Python tools, such as xmllint, which can produce a predictable, 
reproducible XML formatting, too.)

>I'm writing a regression test for a module that generates XML.

Very good.  Good == Testing.

>I'm using diff to compare the results with a pregenerated one from an
>earlier version.

[
Interesting.  I can only speculate randomly about the whitespace 
issue.  Have you examined (with the CLI tools hexdump, od or your 
favorite byte dumper) the two different XML outputs?
]

Back to the lands of Python

>      cmd   = ["diff", "-w", "-I '^[[:space:]]*$'", "./xml/%s.xml" % name, "test.xml"]

It looks like a quoting issue.  I think you are passing the 
following tokens to your OS.  You should be able to run your Python 
program under a system call tracer to see what is actually getting 
exec()d.

I'm accustomed to using strace, but it seems that Macintosh uses 
dtruss.  Anyway, I think your cmd is turning into this (as for as 
your kernel is concerned):

   token 1: diff
   token 2: -w
   token 3: -I '^[[:space:]]*$'
   token 4: ./xml/name.xml
   token 5: test.xml

Try this (untested):

>      cmd = ["diff", "-w", "-I", "^[[:space:]]*$", "./xml/%s.xml" % name, "test.xml"]

But, perhaps the xmldiff module will be what you want.

-Martin

-- 
Martin A. Brown
http://linux-ip.net/

[toc] | [prev] | [next] | [standalone]

#101716 — Re: ignoring or replacing white lines in a diff

Adriaan Renting wrote:

> 
> Maybe someone here has a clue what is going wrong here? Any help is
> appreciated.
> 
> I'm writing a regression test for a module that generates XML.
> 
> I'm using diff to compare the results with a pregenerated one from an
> earlier version.
> 
> I'm running into two problems:
> 
> The diff doesn't seem to behave properly with the -B option. (diff (GNU
> diffutils) 2.8.1 on OSX 10.9)
> 
> Replacing -B with -I '^[[:space:]]*$' fixes it on the command line,
> which should be exactly the same according to:
> http://www.gnu.org/software/diffutils/manual/html_node/Blank-Lines.html#Blank-Lines
> 
> (for Python problem continue below)
> 
> MacRenting 21:00-159> diff -w -B test.xml xml/Ticket_6923.xml
> 3,5c3,5
> <   <version>2.15.0</version>
> <   <template version="2.15.0" author="Alwin de Jong,Adriaan Renting"
> changedBy="Adriaan Renting">
> <   <description>XML Template generator version 2.15.0</description>
> ---
>>           <version>2.6.0</version>
>>           <template version="2.6.0" author="Alwin de Jong"
> changedBy="Alwin de Jong">
>>           <description>XML Template generator version
> 2.6.0</description>
> 113d112
> <
> 163d161
> <
> 213d210
> <
> 258d254
> <
> 369d364
> <
> 419d413
> <
> 469d462
> <
> 514d506
> <
> 625d616
> <
> 675d665
> <
> 725d714
> <
> 770d758
> <
> 881d868
> <
> 931d917
> <
> 981d966
> <
> 1026d1010
> <
> 1137d1120
> <
> 1187d1169
> <
> 1237d1218
> <
> 1282d1262
> <
> 
> /Users/renting/src/CEP4-DevelopClusterModel-Story-Task8432-
SAS/XML_generator/test
> MacRenting 21:00-160> diff -w -I '^[[:space:]]*$' test.xml
> xml/Ticket_6923.xml
> 3,5c3,5
> <   <version>2.15.0</version>
> <   <template version="2.15.0" author="Alwin de Jong,Adriaan Renting"
> changedBy="Adriaan Renting">
> <   <description>XML Template generator version 2.15.0</description>
> ---
>>           <version>2.6.0</version>
>>           <template version="2.6.0" author="Alwin de Jong"
> changedBy="Alwin de Jong">
>>           <description>XML Template generator version
> 2.6.0</description>
> 
> 
> Now I try to use this in Python:
> 
>       cmd   = ["diff", "-w", "-I '^[[:space:]]*$'", "./xml/%s.xml" %
> name, "test.xml"]

Instead of 

..., "-I '^[[:space:]]*$'", ...

try two separate arguments

..., "-I", "^[[:space:]]*$", ...

>       ## -w ignores differences in whitespace
>       ## -I '^[[:space:]]*$' because -B doesn't work for blank lines
> (on OSX?)
>       p     = subprocess.Popen(cmd, stdin=open('/dev/null'),
> stdout=subprocess.PIPE, stderr=subprocess.PIPE)

I don't think you need to specify stdin.

>       logs  = p.communicate()
>       diffs = logs[0].splitlines() #stdout
>       print "diff reply was %i lines long" % len(diffs)
> 
> This doesn't work. I've tried escaping the various bits, like the * and
> $, even though with single quotes that should not be needed.
> 
> I tried first removing the blank lines from the file:
> 
>       import fileinput
>       for line in fileinput.FileInput("test.xml",inplace=1):
>         if line.rstrip():
>           print line
> 
> This makes it worse, as it adds and empty line for each line in the
> file.

Add a trailing comma to suppress the newline:

print line,

> I've tried various other options. The only thing I can think of, is
> ditching Python and trying to rewrite the whole script in Bash.
> (It's quite complicated, as it loops over various things and does some
> pretty output in between and I'm not very fluent in Bash)
> 
> Any suggestions?

Whatever floats your boat ;)

[toc] | [prev] | [next] | [standalone]

#101717

On Fri, Jan 15, 2016 at 6:01 AM, Irmen de Jong <irmen.NOSPAM@xs4all.nl> wrote:
> Either put something like this at the end of your python scripts, to keep the console open:
>
> raw_input("press enter to exit...")
>
>

Small qualification: This is 3.5, so you should use input instead of raw_input.

input("Press enter to exit...")

ChrisA

[toc] | [prev] | [next] | [standalone]

#101718

On Thu, Jan 14, 2016 at 3:10 PM, Chris Angelico <rosuav@gmail.com> wrote:
> On Fri, Jan 15, 2016 at 6:01 AM, Irmen de Jong <irmen.NOSPAM@xs4all.nl> wrote:
>> Either put something like this at the end of your python scripts, to keep the console open:
>>
>> raw_input("press enter to exit...")
>
> Small qualification: This is 3.5, so you should use input instead of raw_input.
>
> input("Press enter to exit...")

You can also make the interpreter enter the REPL by setting
PYTHONINSPECT, which works for the current process since this gets
checked at exit.

    import os
    os.environ['PYTHONINSPECT'] = '1'

Or use subprocess.Popen to start a new cmd shell attached to the
current console:

    import subprocess
    subprocess.Popen('cmd.exe')

[toc] | [prev] | [next] | [standalone]

#101738 — Re: ignoring or replacing white lines in a diff

Thanks for the various people that provided help.

Peter Otten provided me with a working solution:

I had to split the "-I '^[[:space:]]*$'" into two commands.

      cmd   = ["diff", "-w", "-I", r"^[[:space:]]*$", "./xml/%s.xml" %
name, "test.xml"]
      p     = subprocess.Popen(cmd, stdin=open('/dev/null'),
stdout=subprocess.PIPE, stderr=subprocess.PIPE)
      logs  = p.communicate()
      diffs = logs[0].splitlines() #stdout

This also works:

      cmd   = ["diff -w -I '^[[:space:]]*$' ./xml/%s.xml test.xml" %
name]
      p     = subprocess.Popen(cmd, stdin=open('/dev/null'),
stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=True)
      logs  = p.communicate()
      diffs = logs[0].splitlines() #stdout

As to other comments:

- I've found that stdin=open('/dev/null') is essential in
subprocess.Popen to make it work from automated (headless) scripts.
- print line, did remove the extra newlines, but didn't get rid of the
blank lines.
- making it a raw string with r"-I '^[[:space:]]*$'" made no difference
(also tried r"-I ^[[:space:]]*$")
- I didn't investigate difflib further but will keep it in mind for the
future.

Thank you for your help,

Adriaan.


Adriaan Renting        | Email: renting@astron.nl
Software Engineer Radio Observatory
ASTRON                 | Phone: +31 521 595 100 (797 direct)
P.O. Box 2             | GSM:   +31 6 24 25 17 28
NL-7990 AA Dwingeloo   | FAX:   +31 521 595 101
The Netherlands        | Web: http://www.astron.nl/~renting/



>>> On 14-1-2016 at 22:05, Peter Otten <__peter__@web.de> wrote: 
> Adriaan Renting wrote:
> 
>> 
>> Maybe someone here has a clue what is going wrong here? Any help is
>> appreciated.
>> 
>> I'm writing a regression test for a module that generates XML.
>> 
>> I'm using diff to compare the results with a pregenerated one from
an
>> earlier version.
>> 
>> I'm running into two problems:
>> 
>> The diff doesn't seem to behave properly with the -B option. (diff
(GNU
>> diffutils) 2.8.1 on OSX 10.9)
>> 
>> Replacing -B with -I '^[[:space:]]*$' fixes it on the command line,
>> which should be exactly the same according to:
>> 
>
http://www.gnu.org/software/diffutils/manual/html_node/Blank-Lines.html#Blank-L
> ines
>> 
>> (for Python problem continue below)
>> 
>> MacRenting 21:00-159> diff -w -B test.xml xml/Ticket_6923.xml
>> 3,5c3,5
>> <   <version>2.15.0</version>
>> <   <template version="2.15.0" author="Alwin de Jong,Adriaan
Renting"
>> changedBy="Adriaan Renting">
>> <   <description>XML Template generator version
2.15.0</description>
>> ---
>>>           <version>2.6.0</version>
>>>           <template version="2.6.0" author="Alwin de Jong"
>> changedBy="Alwin de Jong">
>>>           <description>XML Template generator version
>> 2.6.0</description>
>> 113d112
>> <
>> 163d161
>> <
>> 213d210
>> <
>> 258d254
>> <
>> 369d364
>> <
>> 419d413
>> <
>> 469d462
>> <
>> 514d506
>> <
>> 625d616
>> <
>> 675d665
>> <
>> 725d714
>> <
>> 770d758
>> <
>> 881d868
>> <
>> 931d917
>> <
>> 981d966
>> <
>> 1026d1010
>> <
>> 1137d1120
>> <
>> 1187d1169
>> <
>> 1237d1218
>> <
>> 1282d1262
>> <
>> 
>> /Users/renting/src/CEP4-DevelopClusterModel-Story-Task8432-
> SAS/XML_generator/test
>> MacRenting 21:00-160> diff -w -I '^[[:space:]]*$' test.xml
>> xml/Ticket_6923.xml
>> 3,5c3,5
>> <   <version>2.15.0</version>
>> <   <template version="2.15.0" author="Alwin de Jong,Adriaan
Renting"
>> changedBy="Adriaan Renting">
>> <   <description>XML Template generator version
2.15.0</description>
>> ---
>>>           <version>2.6.0</version>
>>>           <template version="2.6.0" author="Alwin de Jong"
>> changedBy="Alwin de Jong">
>>>           <description>XML Template generator version
>> 2.6.0</description>
>> 
>> 
>> Now I try to use this in Python:
>> 
>>       cmd   = ["diff", "-w", "-I '^[[:space:]]*$'", "./xml/%s.xml"
%
>> name, "test.xml"]
> 
> Instead of 
> 
> ..., "-I '^[[:space:]]*$'", ...
> 
> try two separate arguments
> 
> ..., "-I", "^[[:space:]]*$", ...
> 
>>       ## -w ignores differences in whitespace
>>       ## -I '^[[:space:]]*$' because -B doesn't work for blank
lines
>> (on OSX?)
>>       p     = subprocess.Popen(cmd, stdin=open('/dev/null'),
>> stdout=subprocess.PIPE, stderr=subprocess.PIPE)
> 
> I don't think you need to specify stdin.
> 
>>       logs  = p.communicate()
>>       diffs = logs[0].splitlines() #stdout
>>       print "diff reply was %i lines long" % len(diffs)
>> 
>> This doesn't work. I've tried escaping the various bits, like the *
and
>> $, even though with single quotes that should not be needed.
>> 
>> I tried first removing the blank lines from the file:
>> 
>>       import fileinput
>>       for line in fileinput.FileInput("test.xml",inplace=1):
>>         if line.rstrip():
>>           print line
>> 
>> This makes it worse, as it adds and empty line for each line in the
>> file.
> 
> Add a trailing comma to suppress the newline:
> 
> print line,
> 
>> I've tried various other options. The only thing I can think of, is
>> ditching Python and trying to rewrite the whole script in Bash.
>> (It's quite complicated, as it loops over various things and does
some
>> pretty output in between and I'm not very fluent in Bash)
>> 
>> Any suggestions?
> 
> Whatever floats your boat ;)

[toc] | [prev] | [next] | [standalone]

#101740 — Re: ignoring or replacing white lines in a diff

Adriaan Renting wrote:

> - print line, did remove the extra newlines, but didn't get rid of the
> blank lines.

You mean you found that

import fileinput
for line in fileinput.FileInput("test.xml", inplace=True):
    if line.strip():
        print line,

does not remove all blank lines? 

If the file contains lone carriage returns try "universal newlines" mode:

for line in fileinput.FileInput("test.xml", inplace=True, mode="U"):
    if line.strip():
        print line,

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web