Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #32668 > unrolled thread

enabling universal newline

Back to article view | Back to comp.lang.python

Contents

  enabling universal newline Peter Kleiweg <pkleiweg@xs4all.nl> - 2012-11-02 23:22 +0100
    Re: enabling universal newline Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2012-11-02 23:09 +0000
      Re: enabling universal newline Peter Otten <__peter__@web.de> - 2012-11-03 09:26 +0100
      Re: enabling universal newline Peter Kleiweg <pkleiweg@xs4all.nl> - 2012-11-03 09:30 +0100

#32668 — enabling universal newline

In Python 3.1 and 3.2

At start-up, the value of sys.stdin.newlines is None, which 
means, universal newline should be enabled. But it isn't.

So I do this:

sys.stdin = io.TextIOWrapper(sys.stdin.detach(), newline=None)

Now, sys.stdin.newlines is still None, but universal newline is 
enabled.

Why is this?



-- 
Peter Kleiweg
http://pkleiweg.home.xs4all.nl/

[toc] | [next] | [standalone]

#32672

On Fri, 02 Nov 2012 23:22:53 +0100, Peter Kleiweg wrote:

> In Python 3.1 and 3.2
> 
> At start-up, the value of sys.stdin.newlines is None, which means,
> universal newline should be enabled. But it isn't.

What makes you think it is not enabled?

sys.stdin.newlines shows you the newlines actually seen. Until you put 
text including newlines through stdin, it will remain None.

http://docs.python.org/2/library/stdtypes.html#file.newlines
http://docs.python.org/3/library/io.html#io.TextIOBase.newlines

For example, I have Python built with universal newlines, but 
stdin.newlines remains None:

py> f = open('test.txt')
py> f.newlines
py> f.readlines()
['a\n', 'b\n', 'c\n', 'd\n']
py> f.newlines
('\r', '\n', '\r\n')
py> sys.stdin.newlines is None
True



-- 
Steven

[toc] | [prev] | [next] | [standalone]

#32684

Steven D'Aprano wrote:

> On Fri, 02 Nov 2012 23:22:53 +0100, Peter Kleiweg wrote:
> 
>> In Python 3.1 and 3.2
>> 
>> At start-up, the value of sys.stdin.newlines is None, which means,
>> universal newline should be enabled. But it isn't.
> 
> What makes you think it is not enabled?

$ python3 -c 'open("tmp.txt", "wb").write(b"a\nb\r\nc\rd")'

This is the output with universal newlines:

$ python3 -c 'print(open("tmp.txt").readlines())'
['a\n', 'b\n', 'c\n', 'd']

But this is what you get from stdin:

$ cat tmp.txt | python3 -c 'import sys; print(sys.stdin.readlines())'
['a\n', 'b\r\n', 'c\rd']

With Peter Kleiweg's fix:

$ cat tmp.txt | python3 -c 'import sys, io; print(io.TextIOWrapper(sys.stdin.detach(), newline=None).readlines())'
['a\n', 'b\n', 'c\n', 'd']

I think it's reasonable to make the latter the default.

[toc] | [prev] | [next] | [standalone]

#32685

Steven D'Aprano schreef op de 2e dag van de slachtmaand van het jaar 2012:

> On Fri, 02 Nov 2012 23:22:53 +0100, Peter Kleiweg wrote:
> 
> > In Python 3.1 and 3.2
> > 
> > At start-up, the value of sys.stdin.newlines is None, which means,
> > universal newline should be enabled. But it isn't.
> 
> What makes you think it is not enabled?

Script 1:

    #!/usr/bin/env python3.1
    import sys
    print(sys.stdin.readlines())

Output:

    ~ test.py < text
    ['a\rbc\rdef\r']    


Script 2:

    #!/usr/bin/env python3.1
    import io, sys
    sys.stdin = io.TextIOWrapper(sys.stdin.detach(), newline=None)
    print(sys.stdin.readlines())

Output:

    ~ test.py < text
    ['a\n', 'bc\n', 'def\n']


-- 
Peter Kleiweg
http://pkleiweg.home.xs4all.nl/

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web