Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #105098 > unrolled thread

sobering observation, python vs. perl

Started by"Charles T. Smith" <cts.private.yahoo@gmail.com>
First post2016-03-17 15:29 +0000
Last post2016-03-18 22:47 +1100
Articles 3 on this page of 43 — 14 participants

Back to article view | Back to comp.lang.python

Contents

  sobering observation, python vs. perl "Charles T. Smith" <cts.private.yahoo@gmail.com> - 2016-03-17 15:29 +0000
    Re: sobering observation, python vs. perl "Charles T. Smith" <cts.private.yahoo@gmail.com> - 2016-03-17 15:40 +0000
      Re: sobering observation, python vs. perl Marko Rauhamaa <marko@pacujo.net> - 2016-03-17 17:48 +0200
        Re: sobering observation, python vs. perl "Charles T. Smith" <cts.private.yahoo@gmail.com> - 2016-03-17 15:59 +0000
          Re: sobering observation, python vs. perl Marko Rauhamaa <marko@pacujo.net> - 2016-03-17 18:07 +0200
            Re: sobering observation, python vs. perl "Charles T. Smith" <cts.private.yahoo@gmail.com> - 2016-03-17 16:15 +0000
    Re: sobering observation, python vs. perl Marko Rauhamaa <marko@pacujo.net> - 2016-03-17 17:47 +0200
      Re: sobering observation, python vs. perl "Charles T. Smith" <cts.private.yahoo@gmail.com> - 2016-03-17 16:06 +0000
        Re: sobering observation, python vs. perl Marko Rauhamaa <marko@pacujo.net> - 2016-03-17 18:30 +0200
          Re: sobering observation, python vs. perl "Charles T. Smith" <cts.private.yahoo@gmail.com> - 2016-03-17 16:32 +0000
    Re: sobering observation, python vs. perl srinivas devaki <mr.eightnoteight@gmail.com> - 2016-03-17 21:18 +0530
      Re: sobering observation, python vs. perl "Charles T. Smith" <cts.private.yahoo@gmail.com> - 2016-03-17 16:15 +0000
    Re: sobering observation, python vs. perl Tim Chase <python.list@tim.thechases.com> - 2016-03-17 10:52 -0500
      Re: sobering observation, python vs. perl "Charles T. Smith" <cts.private.yahoo@gmail.com> - 2016-03-17 16:08 +0000
        Re: sobering observation, python vs. perl Ethan Furman <ethan@stoneleaf.us> - 2016-03-17 09:21 -0700
          Re: sobering observation, python vs. perl "Charles T. Smith" <cts.private.yahoo@gmail.com> - 2016-03-17 16:36 +0000
            Re: sobering observation, python vs. perl Mark Lawrence <breamoreboy@yahoo.co.uk> - 2016-03-17 17:09 +0000
            Re: sobering observation, python vs. perl Ethan Furman <ethan@stoneleaf.us> - 2016-03-17 10:26 -0700
              Re: sobering observation, python vs. perl "Charles T. Smith" <cts.private.yahoo@gmail.com> - 2016-03-17 17:35 +0000
                Re: sobering observation, python vs. perl Ethan Furman <ethan@stoneleaf.us> - 2016-03-17 11:21 -0700
            DSLs in perl and python (Was sobering observation) Rustom Mody <rustompmody@gmail.com> - 2016-03-17 10:47 -0700
              Re: DSLs in perl and python (Was sobering observation) Mark Lawrence <breamoreboy@yahoo.co.uk> - 2016-03-17 22:22 +0000
              Re: DSLs in perl and python (Was sobering observation) MRAB <python@mrabarnett.plus.com> - 2016-03-17 22:43 +0000
                Re: DSLs in perl and python (Was sobering observation) Rustom Mody <rustompmody@gmail.com> - 2016-03-18 05:57 -0700
                  Re: DSLs in perl and python (Was sobering observation) Jussi Piitulainen <jussi.piitulainen@helsinki.fi> - 2016-03-18 15:18 +0200
                  Re: DSLs in perl and python (Was sobering observation) Peter Otten <__peter__@web.de> - 2016-03-18 14:22 +0100
                    Re: DSLs in perl and python (Was sobering observation) Rustom Mody <rustompmody@gmail.com> - 2016-03-18 19:07 -0700
                      DSL design (was DSLs in perl and python) Rustom Mody <rustompmody@gmail.com> - 2016-03-29 06:28 -0700
                        Re: DSL design (was DSLs in perl and python) Chris Angelico <rosuav@gmail.com> - 2016-03-30 00:41 +1100
                        Re: DSL design (was DSLs in perl and python) Jussi Piitulainen <jussi.piitulainen@helsinki.fi> - 2016-03-29 16:45 +0300
                        Finding methods, was Re: DSL design (was DSLs in perl and python) Peter Otten <__peter__@web.de> - 2016-03-29 15:51 +0200
        Re: sobering observation, python vs. perl Marko Rauhamaa <marko@pacujo.net> - 2016-03-17 18:34 +0200
          Re: sobering observation, python vs. perl "Charles T. Smith" <cts.private.yahoo@gmail.com> - 2016-03-17 16:42 +0000
            Re: sobering observation, python vs. perl Marko Rauhamaa <marko@pacujo.net> - 2016-03-17 19:08 +0200
              Re: sobering observation, python vs. perl "Charles T. Smith" <cts.private.yahoo@gmail.com> - 2016-03-17 17:25 +0000
                Re: sobering observation, python vs. perl BartC <bc@freeuk.com> - 2016-03-17 17:53 +0000
                  Re: sobering observation, python vs. perl Rustom Mody <rustompmody@gmail.com> - 2016-03-17 10:59 -0700
                  Re: sobering observation, python vs. perl Marko Rauhamaa <marko@pacujo.net> - 2016-03-17 20:53 +0200
                    Re: sobering observation, python vs. perl BartC <bc@freeuk.com> - 2016-03-17 19:06 +0000
                      Re: sobering observation, python vs. perl Marko Rauhamaa <marko@pacujo.net> - 2016-03-17 21:11 +0200
              Re: sobering observation, python vs. perl Ben Bacarisse <ben.usenet@bsb.me.uk> - 2016-03-17 20:47 +0000
        Re: sobering observation, python vs. perl Peter Otten <__peter__@web.de> - 2016-03-18 10:26 +0100
        Re: sobering observation, python vs. perl Steven D'Aprano <steve@pearwood.info> - 2016-03-18 22:47 +1100

Page 3 of 3 — ← Prev page 1 2 [3]

#105146

FromBen Bacarisse <ben.usenet@bsb.me.uk>
Date2016-03-17 20:47 +0000
Message-ID<87pousx25k.fsf@bsb.me.uk>
In reply to#105119
Marko Rauhamaa <marko@pacujo.net> writes:

> "Charles T. Smith" <cts.private.yahoo@gmail.com>:
>
>> Actually, I saw a study some years ago that concluded that python
>> could be both slower and faster than perl, but that perl had much less
>> deviation than python. I took that and accepted it, but was surprised
>> now that in exactly the field of application that I've traditionally
>> used perl, it really is better, er... faster.
>>
>> Furthermore, the really nice thing about python is its OO, but I've
>> really neglected looking into that with perl's OO capabilities.
>
> I haven't had such log processing needs as you, nor has it come down to
> performance in such a way. Do use the best tool for the job.
>
> (When it comes to freely formatted logs, gleaning information from them
> is somewhat of a lost cause. I've done my best to move to rigorously
> formatted logs that are much more amenable to post processing.)
>
> Perl might be strong on its home turf, but I am a minimalist and
> reductionist -- Perl was intentionally designed to be a maximalist,
> imitating the principles of natural languages. Python has concise,
> crystal-clear semantics that are convenient to work with.
>
> Compare Perl (<URL: http://www.perlmonks.org/?node_id=98357>):
>
>    my $str = "I have a dream";
>    my $find = "have";
>    my $replace = "had";
>    $find = quotemeta $find; # escape regex metachars if present
>    $str =~ s/$find/$replace/g;
>    print $str;
>
> with Python:
>
>    print("I have a dream".replace("have", "had"))

If you know the strings are "have" and "had", you can just write

  print 'I have a dream' =~ s/have/had/r;

but I think your point is to show up the lack of a string (rather than
regex) replace in Perl, so the strings should be considered arbitrarily
"dangerous".  For that purpose it might have been better to give the
example as

  print("I have a dream".replace(find, replace))

for which the closest Perl match is probably

  print 'I have a dream' =~ s/\Q$find/$replace/r;

The closest to the actual line -- where you can just edit two strings
with your only concern being the end quote of the string -- would be
something like

  my $find = 'have'; print 'I have a dream' =~ s{\Q$find}'had'r

I don't want to start a language war!  I'm not saying that this is as
simple and clear as the Python, but a "compare X with Y" should try to
do the best by both X and Y.

-- 
Ben.

[toc] | [prev] | [next] | [standalone]

#105190

FromPeter Otten <__peter__@web.de>
Date2016-03-18 10:26 +0100
Message-ID<mailman.307.1458293216.12893.python-list@python.org>
In reply to#105108
Charles T. Smith wrote:

> On Thu, 17 Mar 2016 10:52:30 -0500, Tim Chase wrote:
> 
>>> Not saying this will make a great deal of difference, but these two
>> items jumped out at me.  I'd even be tempted to just use string
>> manipulations for the isready aspect as well.  Something like
>> (untested)
> 
> well, I don't want to forgo REs in order to have python's numbers be
> better....

As has been said, for simple text processing tasks string methods are the 
preferred approach in Python. I think this is more for clarity than 
performance.

If you need regular expressions a simple way to boost performance may be to 
use the external regex module.

(By the way, if you are looking for a simple way to iterate over multiple 
files use
for line in fileinput.input():
    ...
)

Some numbers:

$ time perl find.pl data/sample*.txt > r1.txt

real    0m0.504s
user    0m0.466s
sys     0m0.036s
$ time python find.py data/sample*.txt > r2.txt

real    0m2.403s
user    0m2.339s
sys     0m0.059s
$ time python find_regex.py data/sample*.txt > r3.txt

real    0m0.693s
user    0m0.631s
sys     0m0.060s
$ time python find_no_re.py data/sample*.txt > r4.txt

real    0m0.319s
user    0m0.267s
sys     0m0.048s

Python 3 slows down things:

$ time python3 find_no_re.py data/sample*.txt > r5.txt

real    0m0.497s
user    0m0.444s
sys     0m0.051s

The scripts:
$ cat find.pl
#!/usr/bin/env perl

while (<>) {
    if (/(.*) is ready/) {
        $tn = $1;
    }
    elsif (/release_req/) {
        print "$tn\n";
    }
}
$ cat find.py
#!/usr/bin/env python
import sys
import re

def main():
    isready = re.compile ("(.*) is ready").match
    relreq = re.compile (".*release_req").match

    tn = ""
    for fn in sys.argv[1:]:
        with open(fn) as fd:
            for line in fd:
                match = isready(line)
                if match:
                    tn = match.group(1)
                elif relreq(line):
                    print(tn)

main()

$ cat find_regex.py
#!/usr/bin/env python
import sys
import regex as re
[rest the same as find.py]

$ cat find_no_re.py
#!/usr/bin/env python
import sys

def main():
    tn = ""
    for fn in sys.argv[1:]:
        with open(fn) as fd:
            for line in fd:
                if " is ready" in line:
                    tn = line.partition(" is ready")[0]
                elif "release_req" in line:
                    print(tn)

main()

The test data was generated with

$ cat make_test_data.py
#!/usr/bin/env python3
import os
import random
import shutil

from itertools import islice


def make_line_factory(words, line_length, isready):
    choice = random.choice

    def make_line():
        while True:
            line = [choice(words)]
            length = len(line[0])
            while length < line_length:
                word = choice(words)
                line.append(word)
                length += len(word) + 1
            if random.randrange(100) < isready:
                pos = random.randrange(len(line))
                line[pos:pos+1] = ["is", "ready"]
            elif random.randrange(100) < isready:
                pos = random.randrange(len(line))
                line[pos:pos] = ["release_req"]
            yield " ".join(line)

    return make_line


def main():
    import argparse
    parser = argparse.ArgumentParser()
    parser.add_argument("--words", default="/usr/share/dict/words")
    parser.add_argument("--line-length", type=int, default=80)
    parser.add_argument("--num-lines", type=eval, default=10**5)
    parser.add_argument("--num-files", type=int, default=4)
    parser.add_argument("--name-template", default="sample{:0{}}.txt")
    parser.add_argument("--data-folder", default="data")
    parser.add_argument("--remove-data-folder", action="store_true")
    parser.add_argument("--first-match-percent", type=int, default=10)
    try:
        import argcomplete
    except ImportError:
        pass
    else:
        argcomplete.autocomplete(parser)

    args = parser.parse_args()

    if args.remove_data_folder:
        shutil.rmtree(args.data_folder)
    os.mkdir(args.data_folder)

    with open(args.words) as f:
        words = [line.strip() for line in f]

    make_line = make_line_factory(
        words, args.line_length, args.first_match_percent)()

    width = len(str(args.num_files))
    for index in range(1, args.num_files+1):
        filename = os.path.join(
            args.data_folder,
            args.name_template.format(index, width))
        print(filename)
        with open(filename, "w") as f:
            for line in islice(make_line, args.num_lines):
                print(line, file=f)


if __name__ == "__main__":
    main()

[toc] | [prev] | [next] | [standalone]

#105201

FromSteven D'Aprano <steve@pearwood.info>
Date2016-03-18 22:47 +1100
Message-ID<56ebeae2$0$1599$c3e8da3$5496439d@news.astraweb.com>
In reply to#105108
On Fri, 18 Mar 2016 03:08 am, Charles T. Smith wrote:

> On Thu, 17 Mar 2016 10:52:30 -0500, Tim Chase wrote:
> 
>>> Not saying this will make a great deal of difference, but these two
>> items jumped out at me.  I'd even be tempted to just use string
>> manipulations for the isready aspect as well.  Something like
>> (untested)
> 
> well, I don't want to forgo REs in order to have python's numbers be
> better....


Even when REs are the wrong tool for the job?

"Yeah, I know I ought to be using a power drill, but all I've got is this
hammer..."




-- 
Steven

[toc] | [prev] | [standalone]

Page 3 of 3 — ← Prev page 1 2 [3]

Back to top | Article view | comp.lang.python

csiph-web