Path: csiph.com!weretis.net!feeder6.news.weretis.net!news.misty.com!news.iecc.com!.POSTED.news.iecc.com!nerds-end From: gah4 Newsgroups: comp.compilers Subject: Re: Why does the lexer convert text integer lexemes to binary integers? I thought that lexers should be simple? Date: Thu, 14 Jul 2022 10:03:18 -0700 (PDT) Organization: Compilers Central Lines: 30 Sender: news@iecc.com Approved: comp.compilers@iecc.com Message-ID: <22-07-013@comp.compilers> References: <22-07-011@comp.compilers> Mime-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Injection-Info: gal.iecc.com; posting-host="news.iecc.com:2001:470:1f07:1126:0:676f:7373:6970"; logging-data="65356"; mail-complaints-to="abuse@iecc.com" Keywords: lex, design Posted-Date: 14 Jul 2022 14:13:04 EDT X-submission-address: compilers@iecc.com X-moderator-address: compilers-request@iecc.com X-FAQ-and-archives: http://compilers.iecc.com In-Reply-To: <22-07-011@comp.compilers> Xref: csiph.com comp.compilers:3116 On Thursday, July 14, 2022 at 8:10:56 AM UTC-7, Roger L Costello wrote: > A common example in books on Lex/Flex and Yacc/Bison is evaluating arithmetic > expressions. When the lexer encounters an integer lexeme, it casts the lexeme > to a binary integer and returns the value to the parser. The lexer contains a > rule that looks something like this: > {INTEGER} { yylval.intval = atoi(yytext); return NUMBER; } A common example is an RPN or algebraic calculator. (Which evaluates numerical expressions.) It is a nice simple example, compared to a complete programming language. For algebraic, you get into precedence and such, so real parsing problems, but in a very simple, and easy to follow, case. The other things, is that it isn't so easy to return character strings in C. You would malloc() it and copy the value over, and then hope that somewhere later it is free()d. (I suspect that is done more in Java.) In the STEP processor that I previously wrote about, the only return type is character string, and the processor keeps track of allocation. Macros that do arithmetic convert to numerical type, evaluate an expression, and convert back to characters. The code for the processor itself, as usual for compilers, is written using itself, has a macro to evaluate a constant integer expression. That is useful, as Fortran 66 (and Fortran 77) don't do that.