Groups > comp.lang.java.programmer > #20890

Re: question on java lang spec chapter 3.3 (unicode char lexing)

Newsgroups	comp.lang.java.programmer
Date	2013-01-02 17:21 -0800
References	<0f28108e-6d35-43a1-a9df-b6c5636fb0ec@googlegroups.com> <dced751f-cbc8-4040-8c2d-e0ddfdb3134d@googlegroups.com> <50e4d70a$0$288$14726298@news.sunsite.dk>
Message-ID	<f8bac92f-ddce-4dd5-a0a1-1f15888821a0@googlegroups.com> (permalink)
Subject	Re: question on java lang spec chapter 3.3 (unicode char lexing)
From	Lew <lewbloch@gmail.com>

Show all headers | View raw

Arne Vajhøj wrote:
> Lew wrote:
>>Aryeh M. Friedman wrote:
>>> If I am lexer for Java in a 100% unicode [sic] environment (it already uses unicode for all internal
>>> representation of text) and 100% of the code that I will be lexing is from that environment do I need still
>>> deal with unicode escapes (\uXXXX) in real life [vs. theortically complete lexing]... assume that no code
>>> will be imported from non-unicode environments
> 
>> What do you mean "have to deal with"?
>>
>> If you mean to parse Java source, you have to be able to parse Java source. The JLS is the final
>> authority on what that constitutes.
> 
>> Being "in a 100% unicode [sic] environment" (whatever that's supposed to mean) does not excuse
> > any responsibilities.
> 
>> Nor does it obviate the need for the occasional "\uXXXX" in source.
> 
>> However, I don't think the lexer deals with that. Unicode escape sequences are a precompile 
>> phenomenon. Everything is substituted before parsing starts.
> 
> Well - lexing happens before parsing so ...

So does writing source code. What's your point?

My point is that the lexer picks up after the substitution of Unicode sequences.
However, my point is wrong, and yours is right.

http://www.docjar.com/html/api/com/sun/tools/javac/parser/Lexer.java.html

-- 
Lew

Thread

question on java lang spec chapter 3.3 (unicode char lexing) "Aryeh M. Friedman" <Aryeh.Friedman@gmail.com> - 2013-01-02 00:20 -0800
  Re: question on java lang spec chapter 3.3 (unicode char lexing) "Aryeh M. Friedman" <Aryeh.Friedman@gmail.com> - 2013-01-02 00:24 -0800
    Re: question on java lang spec chapter 3.3 (unicode char lexing) Patricia Shanahan <pats@acm.org> - 2013-01-02 12:24 -0800
  Re: question on java lang spec chapter 3.3 (unicode char lexing) Lew <lewbloch@gmail.com> - 2013-01-02 11:16 -0800
    Re: question on java lang spec chapter 3.3 (unicode char lexing) Arne Vajhøj <arne@vajhoej.dk> - 2013-01-02 19:55 -0500
      Re: question on java lang spec chapter 3.3 (unicode char lexing) Lew <lewbloch@gmail.com> - 2013-01-02 17:21 -0800
        Re: question on java lang spec chapter 3.3 (unicode char lexing) Arne Vajhøj <arne@vajhoej.dk> - 2013-01-02 20:40 -0500
  Re: question on java lang spec chapter 3.3 (unicode char lexing) Roedy Green <see_website@mindprod.com.invalid> - 2013-01-02 11:17 -0800
    Re: question on java lang spec chapter 3.3 (unicode char lexing) Arne Vajhøj <arne@vajhoej.dk> - 2013-01-02 19:56 -0500
      Re: question on java lang spec chapter 3.3 (unicode char lexing) "Aryeh M. Friedman" <Aryeh.Friedman@gmail.com> - 2013-01-02 17:27 -0800
        Re: question on java lang spec chapter 3.3 (unicode char lexing) "Aryeh M. Friedman" <Aryeh.Friedman@gmail.com> - 2013-01-02 17:32 -0800
        Re: question on java lang spec chapter 3.3 (unicode char lexing) Lew <lewbloch@gmail.com> - 2013-01-02 17:42 -0800
          Re: question on java lang spec chapter 3.3 (unicode char lexing) "Aryeh M. Friedman" <Aryeh.Friedman@gmail.com> - 2013-01-02 17:55 -0800
            Re: question on java lang spec chapter 3.3 (unicode char lexing) "Aryeh M. Friedman" <Aryeh.Friedman@gmail.com> - 2013-01-02 18:02 -0800
              Re: question on java lang spec chapter 3.3 (unicode char lexing) Arne Vajhøj <arne@vajhoej.dk> - 2013-01-02 21:12 -0500
                Re: question on java lang spec chapter 3.3 (unicode char lexing) "Aryeh M. Friedman" <Aryeh.Friedman@gmail.com> - 2013-01-02 18:16 -0800
                Re: question on java lang spec chapter 3.3 (unicode char lexing) Arne Vajhøj <arne@vajhoej.dk> - 2013-01-02 21:20 -0500
                Re: question on java lang spec chapter 3.3 (unicode char lexing) "Aryeh M. Friedman" <Aryeh.Friedman@gmail.com> - 2013-01-02 18:22 -0800
                Re: question on java lang spec chapter 3.3 (unicode char lexing) Arne Vajhøj <arne@vajhoej.dk> - 2013-01-02 21:26 -0500
                Re: question on java lang spec chapter 3.3 (unicode char lexing) "Aryeh M. Friedman" <Aryeh.Friedman@gmail.com> - 2013-01-02 18:27 -0800
                Re: question on java lang spec chapter 3.3 (unicode char lexing) Arne Vajhøj <arne@vajhoej.dk> - 2013-01-02 21:46 -0500
                Re: question on java lang spec chapter 3.3 (unicode char lexing) "Aryeh M. Friedman" <Aryeh.Friedman@gmail.com> - 2013-01-02 20:41 -0800
                Re: question on java lang spec chapter 3.3 (unicode char lexing) Arne Vajhøj <arne@vajhoej.dk> - 2013-01-06 21:54 -0500
            Re: question on java lang spec chapter 3.3 (unicode char lexing) Arne Vajhøj <arne@vajhoej.dk> - 2013-01-02 21:15 -0500
              Re: question on java lang spec chapter 3.3 (unicode char lexing) "Aryeh M. Friedman" <Aryeh.Friedman@gmail.com> - 2013-01-02 18:20 -0800
        Re: question on java lang spec chapter 3.3 (unicode char lexing) Arne Vajhøj <arne@vajhoej.dk> - 2013-01-02 21:17 -0500
        Re: question on java lang spec chapter 3.3 (unicode char lexing) Patricia Shanahan <pats@acm.org> - 2013-01-02 22:33 -0800
          Re: question on java lang spec chapter 3.3 (unicode char lexing) "Chris Uppal" <chris.uppal@metagnostic.REMOVE-THIS.org> - 2013-01-05 12:58 +0000
            Re: question on java lang spec chapter 3.3 (unicode char lexing) "Aryeh M. Friedman" <Aryeh.Friedman@gmail.com> - 2013-01-05 05:34 -0800
              Re: question on java lang spec chapter 3.3 (unicode char lexing) "Aryeh M. Friedman" <Aryeh.Friedman@gmail.com> - 2013-01-05 05:40 -0800
              Re: question on java lang spec chapter 3.3 (unicode char lexing) Arne Vajhøj <arne@vajhoej.dk> - 2013-01-06 21:56 -0500
      Re: question on java lang spec chapter 3.3 (unicode char lexing) Martin Gregorie <martin@address-in-sig.invalid> - 2013-01-03 21:14 +0000
        Re: question on java lang spec chapter 3.3 (unicode char lexing) Arne Vajhøj <arne@vajhoej.dk> - 2013-01-03 17:51 -0500
          Re: question on java lang spec chapter 3.3 (unicode char lexing) "Aryeh M. Friedman" <Aryeh.Friedman@gmail.com> - 2013-01-03 20:54 -0800
            Re: question on java lang spec chapter 3.3 (unicode char lexing) Martin Gregorie <martin@address-in-sig.invalid> - 2013-01-05 00:15 +0000
            Re: question on java lang spec chapter 3.3 (unicode char lexing) "Chris Uppal" <chris.uppal@metagnostic.REMOVE-THIS.org> - 2013-01-05 13:03 +0000
              Re: question on java lang spec chapter 3.3 (unicode char lexing) "Aryeh M. Friedman" <Aryeh.Friedman@gmail.com> - 2013-01-05 05:25 -0800
                Re: question on java lang spec chapter 3.3 (unicode char lexing) Arne Vajhøj <arne@vajhoej.dk> - 2013-01-06 21:49 -0500
                Re: question on java lang spec chapter 3.3 (unicode char lexing) "Aryeh M. Friedman" <Aryeh.Friedman@gmail.com> - 2013-01-06 23:26 -0800
            Re: question on java lang spec chapter 3.3 (unicode char lexing) Arne Vajhøj <arne@vajhoej.dk> - 2013-01-06 21:44 -0500
  Re: question on java lang spec chapter 3.3 (unicode char lexing) Arne Vajhøj <arne@vajhoej.dk> - 2013-01-02 19:54 -0500

csiph-web