Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #102016

Re: Refactoring in a large code base

Show all headers | View raw

On Sat, Jan 23, 2016 at 12:30 AM, Rustom Mody <rustompmody@gmail.com> wrote:
> You just gave a graphic vivid description...
> of the same thing Marko is describing: ;-) viz.
> A full-size language parser is something that you - an experienced developer -
> make a point of avoiding.

It's worth noting that "experienced developer" covers a huge range of
skills. There are quite a few other areas that I do not tinker with
(crypto, CPU-level optimizations, and such), not because they're
impossible to understand, but because *I* have not the skill to
understand and improve them. This does mean they're complicated
(they're beyond the "one weekend of tinkering" barrier that any
serious geek should be able to invest), but I'm sure there are
language nerds out there who are so familiar with the grammar of
<insert language here> that they'll pick up CPython's grammar and make
a change with confidence that it'll do what they expect.

> So then the question comes down to this: Is this the order of nature?
> Or is it man-made disorder?
> Jury's out on that one for lexers/parsers specifically.

Lexers/parsers are as complicated as the grammars they parse. A lexer
for a simple structured text file can be pretty easy to implement; for
instance, JSON is pretty straight-forward, with only a handful of
cases (insignificant whitespace, three keywords, two recursive
structures that start with specific characters ('{' and '['), strings
(which start with '"'), and numbers (which start with a digit or a
hyphen)), so a parser need only look for those few possibilities and
it knows exactly what else to fetch up. I could probably write a JSON
parser in a fairly short space of time, and wouldn't be scared of
digging into the internals of someone else's. It's when the grammar
adds complexities to deal with the real-world issues of full size
programming languages that it becomes hairier. The CPython grammar is
only ~150 lines of fairly readable directives, but the parser that
implements it is ~3500 lines of C code. Pike merges the two into a
YACC file of nearly 5000 lines of highly optimized code (it has
different grammar paths for things a human would consider the same, in
order to produce distinct code). That's where I'm ubercautious.

> For arbitrary code in general, the problem that it may be arbitrarily and unboundedly
> complex/complicated is the oldest problem in computer science: the halting problem.
>
> IOW anyone who thinks that *arbitrary* complexity can *always* be tamed either
> has a visa to utopia or needs to re-evaluate (or get) a CS degree

Exactly.

ChrisA

Back to comp.lang.python | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

importing: what does "from" do? "Charles T. Smith" <cts.private.yahoo@gmail.com> - 2016-01-21 14:20 +0000
  Re: importing: what does "from" do? Ian Kelly <ian.g.kelly@gmail.com> - 2016-01-21 07:52 -0700
    Re: importing: what does "from" do? "Charles T. Smith" <cts.private.yahoo@gmail.com> - 2016-01-21 15:12 +0000
      Re: importing: what does "from" do? Ian Kelly <ian.g.kelly@gmail.com> - 2016-01-21 08:59 -0700
        Re: importing: what does "from" do? "Charles T. Smith" <cts.private.yahoo@gmail.com> - 2016-01-21 16:30 +0000
          Re: importing: what does "from" do? "Charles T. Smith" <cts.private.yahoo@gmail.com> - 2016-01-21 16:41 +0000
      Re: importing: what does "from" do? Steven D'Aprano <steve@pearwood.info> - 2016-01-22 03:22 +1100
  Re: importing: what does "from" do? John Gordon <gordon@panix.com> - 2016-01-21 15:31 +0000
    Re: importing: what does "from" do? "Charles T. Smith" <cts.private.yahoo@gmail.com> - 2016-01-21 15:37 +0000
      Re: importing: what does "from" do? John Gordon <gordon@panix.com> - 2016-01-21 15:44 +0000
        Re: importing: what does "from" do? "Charles T. Smith" <cts.private.yahoo@gmail.com> - 2016-01-21 15:46 +0000
          Re: importing: what does "from" do? John Gordon <gordon@panix.com> - 2016-01-21 15:59 +0000
        Re: importing: what does "from" do? "Charles T. Smith" <cts.private.yahoo@gmail.com> - 2016-01-21 15:53 +0000
          Re: importing: what does "from" do? Steven D'Aprano <steve@pearwood.info> - 2016-01-22 03:28 +1100
  Re: importing: what does "from" do? Rustom Mody <rustompmody@gmail.com> - 2016-01-21 23:03 -0800
    Refactoring in a large code base (was: importing: what does "from" do?) Ben Finney <ben+python@benfinney.id.au> - 2016-01-22 18:54 +1100
      Re: Refactoring in a large code base Marko Rauhamaa <marko@pacujo.net> - 2016-01-22 10:29 +0200
        Re: Refactoring in a large code base Rustom Mody <rustompmody@gmail.com> - 2016-01-22 01:21 -0800
          Re: Refactoring in a large code base Marko Rauhamaa <marko@pacujo.net> - 2016-01-22 12:19 +0200
            Re: Refactoring in a large code base Chris Angelico <rosuav@gmail.com> - 2016-01-22 22:19 +1100
              Re: Refactoring in a large code base Marko Rauhamaa <marko@pacujo.net> - 2016-01-22 13:54 +0200
                Re: Refactoring in a large code base Chris Angelico <rosuav@gmail.com> - 2016-01-22 23:28 +1100
                Re: Refactoring in a large code base Marko Rauhamaa <marko@pacujo.net> - 2016-01-22 15:00 +0200
                Re: Refactoring in a large code base Chris Angelico <rosuav@gmail.com> - 2016-01-23 00:26 +1100
                Re: Refactoring in a large code base Marko Rauhamaa <marko@pacujo.net> - 2016-01-22 15:48 +0200
                Re: Refactoring in a large code base Chris Angelico <rosuav@gmail.com> - 2016-01-23 01:09 +1100
              Re: Refactoring in a large code base Rustom Mody <rustompmody@gmail.com> - 2016-01-22 04:04 -0800
                Re: Refactoring in a large code base Marko Rauhamaa <marko@pacujo.net> - 2016-01-22 14:21 +0200
                Re: Refactoring in a large code base Thomas Mellman <tmellman@web.de> - 2016-01-22 12:27 +0000
                Re: Refactoring in a large code base Chris Angelico <rosuav@gmail.com> - 2016-01-22 23:34 +1100
                Re: Refactoring in a large code base Rustom Mody <rustompmody@gmail.com> - 2016-01-22 05:30 -0800
                Re: Refactoring in a large code base Marko Rauhamaa <marko@pacujo.net> - 2016-01-22 15:43 +0200
                Re: Refactoring in a large code base Rustom Mody <rustompmody@gmail.com> - 2016-01-22 05:58 -0800
                Re: Refactoring in a large code base Chris Angelico <rosuav@gmail.com> - 2016-01-23 01:08 +1100
            Re: Refactoring in a large code base "Charles T. Smith" <cts.private.yahoo@gmail.com> - 2016-01-22 11:45 +0000

csiph-web