Re: What stage should entities be resolved?

Path	csiph.com!weretis.net!feeder6.news.weretis.net!news.misty.com!news.iecc.com!.POSTED.news.iecc.com!nerds-end
From	Roger L Costello <costello@mitre.org>
Newsgroups	comp.compilers
Subject	Re: What stage should entities be resolved?
Date	Tue, 15 Mar 2022 11:49:15 +0000
Organization	Compilers Central
Lines	62
Sender	news@iecc.com
Approved	comp.compilers@iecc.com
Message-ID	<22-03-032@comp.compilers> (permalink)
References	<22-03-019@comp.compilers> <22-03-025@comp.compilers>
Content-Type	text/plain; charset="utf-8"
Content-Transfer-Encoding	8bit
Injection-Info	gal.iecc.com; posting-host="news.iecc.com:2001:470:1f07:1126:0:676f:7373:6970"; logging-data="73851"; mail-complaints-to="abuse@iecc.com"
Keywords	parse, design
Posted-Date	17 Mar 2022 14:41:44 EDT
X-submission-address	compilers@iecc.com
X-moderator-address	compilers-request@iecc.com
X-FAQ-and-archives	http://compilers.iecc.com
Content-Language	en-US
Xref	csiph.com comp.compilers:2934

Show key headers only | View raw

Thank you DoDi, Chris, and Matt. You have provided truly exceptional information.

One thing that I am still unclear about is this:

	How much knowledge of the language should each stage have?

For instance, as I understand it a C preprocessor goes through a C program and replaces macros. With this:

#define  PI  3.14

the preprocessor will convert this:

area = PI * radius * radius;

to this:

area = 3.14 * radius * radius;

But if PI is inside a quoted string:

"Today is PI day"

then the preprocessor does not replace PI.

So the preprocessor has some knowledge about the language: If a macro is within a quoted string, then don’t replace it.

Similarly, in XML if &amp; is embedded inside a CDATA section:

<![CDATA[&amp;]]>

then a preprocessor must not replace &amp; with &. That is, the preprocessor must have knowledge about the language: If an XML entity is within a CDATA section, then don’t replace it.

So that brings me to my questions:

1. How much knowledge of the language should the preprocessor stage have?

2. How much knowledge of the language should the lexical analysis stage have?

3. How much knowledge of the language should the syntax analysis stage have?

4. How much knowledge of the language should the semantic analysis stage have?

To make the questions concrete, consider this XML:

<foo>…</foo>

Should the lexical analysis stage know that the foo in <foo> is a
start tag (STAG) and the foo in </foo> is an end tag (ETAG)? That
would mean the lexical analysis stage has considerable knowledge of
the XML language. Or should the lexical analysis stage simply identify
the foo in <foo> as a name (NAME) and the foo in </foo> as a name
(NAME)? That would mean the lexical analysis stage has lesser
knowledge of the XML language. How much knowledge of the XML language
should the lexical analysis stage have?

/Roger
[I'd say start and end tags are sufficiently basic that they are different
but your life is made more complicated with tags like <foo opt="42" />
which are both. Since you'll need to parse attributes, lexemes like
< </ > /> make sense.-John]

Back to comp.compilers | Previous | Next — Previous in thread | Next in thread | Find similar

Thread

What stage should entities be resolved? Lexical analysis stage? Syntax analysis stage? Semantic analysis stage? Roger L Costello <costello@mitre.org> - 2022-03-09 17:22 +0000
  Re: What stage should entities be resolved? Lexical analysis stage? Syntax analysis stage? Semantic analysis stage? Hans-Peter Diettrich <DrDiettrich1@netscape.net> - 2022-03-10 09:48 +0100
    Re: What stage should entities be resolved? Christopher F Clark <christopher.f.clark@compiler-resources.com> - 2022-03-12 14:11 +0200
      Re: What stage should entities be resolved? Hans-Peter Diettrich <DrDiettrich1@netscape.net> - 2022-03-14 19:43 +0100
    Re: What stage should entities be resolved? Roger L Costello <costello@mitre.org> - 2022-03-15 11:49 +0000
      Re: What stage should entities be resolved? Hans-Peter Diettrich <DrDiettrich1@netscape.net> - 2022-03-18 00:31 +0100
      Re: What stage should entities be resolved? gah4 <gah4@u.washington.edu> - 2022-03-17 17:06 -0700
      Re: What stage should entities be resolved? Kaz Kylheku <480-992-1380@kylheku.com> - 2022-03-18 17:50 +0000
        Re: What stage should entities be resolved? gah4 <gah4@u.washington.edu> - 2022-03-18 14:08 -0700
        Re: What stage should entities be resolved? Martin Ward <martin@gkc.org.uk> - 2022-03-19 18:17 +0000
      Re: What stage should entities be resolved? "matt.ti...@gmail.com" <matt.timmermans@gmail.com> - 2022-03-20 07:32 -0700
  RE: What stage should entities be resolved? Christopher F Clark <christopher.f.clark@compiler-resources.com> - 2022-03-10 12:54 +0200
  Re: What stage should entities be resolved? Lexical analysis stage? Syntax analysis stage? Semantic analysis stage? matt.timmermans@gmail.com - 2022-03-12 05:12 -0800

csiph-web