Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.c > #120483

Re: Lexical Elements

From Keith Thompson <kst-u@mib.org>
Newsgroups comp.lang.c
Subject Re: Lexical Elements
Date 2017-09-28 12:40 -0700
Organization None to speak of
Message-ID <lnfub6wr0l.fsf@kst-u.example.com> (permalink)
References <cc7eadf6-6c89-4139-9050-3606d3c0ab01@googlegroups.com> <4e3d4467-fd5b-42f1-9e6a-335cf1ce88bb@googlegroups.com> <027ced74-afdc-4db1-aa14-7b3fb7a22295@googlegroups.com> <5e681899-3b70-44c5-b919-5dccadf62672@googlegroups.com> <3c9e515f-7fa2-43e0-8b0d-749586b74554@googlegroups.com>

Show all headers | View raw

David Kleinecke <dkleinecke@gmail.com> writes:
[...]
> Everything you say sounds right. But my question is ontological.
> What exactly is THE token?

As far as the language is concerned, a token is a syntactic element of a
C translation unit, a sequence of characters in a source file.  Given:

    int x = 42;

you have 5 tokens, consisting of 3, 1, 1, 2, and 1 characters
respectively.  Tokens are distinguished from non-token character
sequences (like "42;" in the above) syntactically.

A compiler will thpically have some internal representation of a token.
The details of that representation are unspecified, and generally are
whatever is convenient for use in the compiler.

Does that answer your question?

-- 
Keith Thompson (The_Other_Keith) kst-u@mib.org  <http://www.ghoti.net/~kst>
Working, but not speaking, for JetHead Development, Inc.
"We must do something.  This is something.  Therefore, we must do this."
    -- Antony Jay and Jonathan Lynn, "Yes Minister"

Back to comp.lang.c | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Lexical Elements David Kleinecke <dkleinecke@gmail.com> - 2017-09-27 19:03 -0700
  Re: Lexical Elements "Pascal J. Bourguignon" <pjb@informatimago.com> - 2017-09-28 05:33 +0200
    Re: Lexical Elements James Kuyper <jameskuyper@verizon.net> - 2017-09-28 00:19 -0400
      Re: Lexical Elements David Kleinecke <dkleinecke@gmail.com> - 2017-09-27 22:09 -0700
        Re: Lexical Elements Keith Thompson <kst-u@mib.org> - 2017-09-28 08:31 -0700
          Re: Lexical Elements David Kleinecke <dkleinecke@gmail.com> - 2017-09-28 11:53 -0700
            Re: Lexical Elements jameskuyper@verizon.net - 2017-09-28 12:16 -0700
              Re: Lexical Elements David Kleinecke <dkleinecke@gmail.com> - 2017-09-28 15:51 -0700
                Re: Lexical Elements jameskuyper@verizon.net - 2017-09-28 16:42 -0700
            Re: Lexical Elements Keith Thompson <kst-u@mib.org> - 2017-09-28 12:37 -0700
              Re: Lexical Elements David Kleinecke <dkleinecke@gmail.com> - 2017-09-28 16:16 -0700
                Re: Lexical Elements Keith Thompson <kst-u@mib.org> - 2017-09-28 18:39 -0700
                Re: Lexical Elements David Kleinecke <dkleinecke@gmail.com> - 2017-09-28 19:47 -0700
                Re: Lexical Elements jameskuyper@verizon.net - 2017-09-28 20:29 -0700
                Re: Lexical Elements David Kleinecke <dkleinecke@gmail.com> - 2017-09-28 22:36 -0700
                Re: Lexical Elements Keith Thompson <kst-u@mib.org> - 2017-09-29 08:47 -0700
                Re: Lexical Elements David Kleinecke <dkleinecke@gmail.com> - 2017-09-29 11:23 -0700
                Re: Lexical Elements Ben Bacarisse <ben.usenet@bsb.me.uk> - 2017-09-29 18:27 +0100
        Re: Lexical Elements jameskuyper@verizon.net - 2017-09-28 09:13 -0700
    Re: Lexical Elements Richard Damon <Richard@Damon-Family.org> - 2017-09-28 08:15 -0400
  Re: Lexical Elements jameskuyper@verizon.net - 2017-09-27 21:03 -0700
    Re: Lexical Elements David Kleinecke <dkleinecke@gmail.com> - 2017-09-27 22:16 -0700
      Re: Lexical Elements jameskuyper@verizon.net - 2017-09-28 09:45 -0700
        Re: Lexical Elements David Kleinecke <dkleinecke@gmail.com> - 2017-09-28 11:58 -0700
          Re: Lexical Elements jameskuyper@verizon.net - 2017-09-28 12:29 -0700
            Re: Lexical Elements David Kleinecke <dkleinecke@gmail.com> - 2017-09-28 15:52 -0700
              Re: Lexical Elements Joe Pfeiffer <pfeiffer@cs.nmsu.edu> - 2017-09-28 17:40 -0600
              Re: Lexical Elements jameskuyper@verizon.net - 2017-09-28 16:54 -0700
          Re: Lexical Elements Keith Thompson <kst-u@mib.org> - 2017-09-28 12:40 -0700
            Re: Lexical Elements David Kleinecke <dkleinecke@gmail.com> - 2017-09-28 16:12 -0700
          Re: Lexical Elements bartc <bc@freeuk.com> - 2017-09-28 21:04 +0100
            Re: Lexical Elements bartc <bc@freeuk.com> - 2017-09-28 22:12 +0100
            Re: Lexical Elements David Kleinecke <dkleinecke@gmail.com> - 2017-09-28 16:15 -0700
              Re: Lexical Elements bartc <bc@freeuk.com> - 2017-09-29 01:19 +0100

csiph-web