Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #12817

Re: One line command line filter

Followups directed to: comp.lang.python

Show all headers | View raw

Jon Redgrave wrote:

> It seems unreasonably hard to write simple one-line unix command line
> filters in python:
> 
> eg: ls | python -c "<something>  print x.upper()"
> 
> to get at sys.stdin or similar needs an import, which makes a
> subsequent for-loop illegal.
> python -c "import sys; for x in sys.stdin(): print x" <<- SyntaxError
> 
> Am I missing something obvious?
> 
> The best I've come up with is to use sitecustomize.py to add to
> __builtin__
> def stdin():
>   import sys
>   for x in sys.stdin():
>     if not x: return
>     yield x.strip()
> import __builtin__
> __builtin__.stdin = stdin
> 
> This allows
> ls | python -c "for x in stdin(): print x.upper()"
> 
> Is there a better solution - if not is this worth a PEP?

$ touch alpha beta gamma omega
$ ls | python -c 'import sys; sys.stdout.writelines(s.upper() for s in 
sys.stdin if s.startswith(("a", "o")))'
ALPHA
OMEGA

If you are doing this a lot, why don't you write a helper script and invoke 
that? 

$ ls | pyfilter.py -f '"m" in line' -s 'lines = (line + line[::-1] for line 
in map(str.strip, lines))' -s'import re' -p 're.compile(r"(([a-
z])\2)").sub(lambda m: m.group(1).upper(), line)'
alphAAhpla
betAAteb
gaMMAAMMag
omegAAgemo

This relies on the convention that a single line of input is accessible as 
"line" and the complete input is called "lines". Of course the same can be 
done with python -c ..., -- and it is even more readable:

$ ls | python -c 'import re, sys
for line in sys.stdin:
>     line = line.strip()
>     line = line + line[::-1]
>     print re.compile(r"(([a-z])\2)").sub(lambda m: m.group(1).upper(), 
line)
> '
alphAAhpla
betAAteb
gaMMAAMMag
omegAAgemo

> Is there a better solution - if not is this worth a PEP?

The python interpreter could accept multiple -c arguments, but to see how 
this will be received a post on python-ideas should be sufficient.

For the sake of completeness here's the script I used to produce the example 
above:

$ cat pyfilter.py
#!/usr/bin/env python
import sys

def main():
    import argparse
    parser = argparse.ArgumentParser()
    parser.add_argument(
        "-s", "--setup",
        action="append", default=[],
        dest="setups", metavar="SETUP")
    parser.add_argument(
        "-f", "--filter",
        action="append", default=[],
        dest="filters", metavar="FILTER")
    parser.add_argument("-p", "--print", dest="format")

    args = parser.parse_args()
    lines = sys.stdin
    for setup in args.setups:
        exec setup
    for line in lines:
        line = line.rstrip("\n")
        for filter in args.filters:
            if not eval(filter):
                continue
        if args.format:
            line = eval(args.format)
        try:
            print line
        except IOError as e:
            if e.errno == 32: # broken pipe
                break
            raise
if __name__ == "__main__":
    main()

Back to comp.lang.python | Previous | Next — Previous in thread | Find similar | Unroll thread

Thread

One line command line filter Jon Redgrave <jredgrave@capisco.com> - 2011-09-05 13:38 -0700
  Re: One line command line filter Thomas Jollans <t@jollybox.de> - 2011-09-05 22:47 +0200
    Re: One line command line filter Jon Redgrave <jredgrave@capisco.com> - 2011-09-05 14:32 -0700
      Re: One line command line filter Terry Reedy <tjreedy@udel.edu> - 2011-09-05 19:02 -0400
  Re: One line command line filter Terry Reedy <tjreedy@udel.edu> - 2011-09-05 18:21 -0400
    Re: One line command line filter Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2011-09-06 09:18 +1000
      Re: One line command line filter Terry Reedy <tjreedy@udel.edu> - 2011-09-05 21:25 -0400
      Re: One line command line filter Hans Mulder <hansmu@xs4all.nl> - 2011-09-06 13:15 +0200
  Re: One line command line filter Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2011-09-06 09:29 +1000
  Re: One line command line filter John Wiegley <jwiegley@gmail.com> - 2011-09-06 01:51 -0500
  Re: One line command line filter Peter Otten <__peter__@web.de> - 2011-09-06 09:53 +0200

csiph-web