Groups > comp.lang.java.programmer > #20887

Re: question on java lang spec chapter 3.3 (unicode char lexing)

Date	2013-01-02 19:55 -0500
From	Arne Vajhøj <arne@vajhoej.dk>
Newsgroups	comp.lang.java.programmer
Subject	Re: question on java lang spec chapter 3.3 (unicode char lexing)
References	<0f28108e-6d35-43a1-a9df-b6c5636fb0ec@googlegroups.com> <dced751f-cbc8-4040-8c2d-e0ddfdb3134d@googlegroups.com>
Message-ID	<50e4d70a$0$288$14726298@news.sunsite.dk> (permalink)
Organization	SunSITE.dk - Supporting Open source

Show all headers | View raw

On 1/2/2013 2:16 PM, Lew wrote:
> On Wednesday, January 2, 2013 12:20:12 AM UTC-8, Aryeh M. Friedman wrote:
>> If I am lexer for Java in a 100% unicode [sic] environment (it already uses unicode for all internal
>> representation of text) and 100% of the code that I will be lexing is from that environment do I need still
>> deal with unicode escapes (\uXXXX) in real life [vs. theortically complete lexing]... assume that no code
>> will be imported from non-unicode environments
>
> What do you mean "have to deal with"?
>
> If you mean to parse Java source, you have to be able to parse Java source. The JLS is the final
> authority on what that constitutes.
>
> Being "in a 100% unicode [sic] environment" (whatever that's supposed to mean) does not excuse
> any responsibilities.
>
> Nor does it obviate the need for the occasional "\uXXXX" in source.
>
> However, I don't think the lexer deals with that. Unicode escape sequences are a precompile phenomenon. Everything is substituted before parsing starts.

Well - lexing happens before parsing so ...

Arne

Thread

question on java lang spec chapter 3.3 (unicode char lexing) "Aryeh M. Friedman" <Aryeh.Friedman@gmail.com> - 2013-01-02 00:20 -0800
  Re: question on java lang spec chapter 3.3 (unicode char lexing) "Aryeh M. Friedman" <Aryeh.Friedman@gmail.com> - 2013-01-02 00:24 -0800
    Re: question on java lang spec chapter 3.3 (unicode char lexing) Patricia Shanahan <pats@acm.org> - 2013-01-02 12:24 -0800
  Re: question on java lang spec chapter 3.3 (unicode char lexing) Lew <lewbloch@gmail.com> - 2013-01-02 11:16 -0800
    Re: question on java lang spec chapter 3.3 (unicode char lexing) Arne Vajhøj <arne@vajhoej.dk> - 2013-01-02 19:55 -0500
      Re: question on java lang spec chapter 3.3 (unicode char lexing) Lew <lewbloch@gmail.com> - 2013-01-02 17:21 -0800
        Re: question on java lang spec chapter 3.3 (unicode char lexing) Arne Vajhøj <arne@vajhoej.dk> - 2013-01-02 20:40 -0500
  Re: question on java lang spec chapter 3.3 (unicode char lexing) Roedy Green <see_website@mindprod.com.invalid> - 2013-01-02 11:17 -0800
    Re: question on java lang spec chapter 3.3 (unicode char lexing) Arne Vajhøj <arne@vajhoej.dk> - 2013-01-02 19:56 -0500
      Re: question on java lang spec chapter 3.3 (unicode char lexing) "Aryeh M. Friedman" <Aryeh.Friedman@gmail.com> - 2013-01-02 17:27 -0800
        Re: question on java lang spec chapter 3.3 (unicode char lexing) "Aryeh M. Friedman" <Aryeh.Friedman@gmail.com> - 2013-01-02 17:32 -0800
        Re: question on java lang spec chapter 3.3 (unicode char lexing) Lew <lewbloch@gmail.com> - 2013-01-02 17:42 -0800
          Re: question on java lang spec chapter 3.3 (unicode char lexing) "Aryeh M. Friedman" <Aryeh.Friedman@gmail.com> - 2013-01-02 17:55 -0800
            Re: question on java lang spec chapter 3.3 (unicode char lexing) "Aryeh M. Friedman" <Aryeh.Friedman@gmail.com> - 2013-01-02 18:02 -0800
              Re: question on java lang spec chapter 3.3 (unicode char lexing) Arne Vajhøj <arne@vajhoej.dk> - 2013-01-02 21:12 -0500
                Re: question on java lang spec chapter 3.3 (unicode char lexing) "Aryeh M. Friedman" <Aryeh.Friedman@gmail.com> - 2013-01-02 18:16 -0800
                Re: question on java lang spec chapter 3.3 (unicode char lexing) Arne Vajhøj <arne@vajhoej.dk> - 2013-01-02 21:20 -0500
                Re: question on java lang spec chapter 3.3 (unicode char lexing) "Aryeh M. Friedman" <Aryeh.Friedman@gmail.com> - 2013-01-02 18:22 -0800
                Re: question on java lang spec chapter 3.3 (unicode char lexing) Arne Vajhøj <arne@vajhoej.dk> - 2013-01-02 21:26 -0500
                Re: question on java lang spec chapter 3.3 (unicode char lexing) "Aryeh M. Friedman" <Aryeh.Friedman@gmail.com> - 2013-01-02 18:27 -0800
                Re: question on java lang spec chapter 3.3 (unicode char lexing) Arne Vajhøj <arne@vajhoej.dk> - 2013-01-02 21:46 -0500
                Re: question on java lang spec chapter 3.3 (unicode char lexing) "Aryeh M. Friedman" <Aryeh.Friedman@gmail.com> - 2013-01-02 20:41 -0800
                Re: question on java lang spec chapter 3.3 (unicode char lexing) Arne Vajhøj <arne@vajhoej.dk> - 2013-01-06 21:54 -0500
            Re: question on java lang spec chapter 3.3 (unicode char lexing) Arne Vajhøj <arne@vajhoej.dk> - 2013-01-02 21:15 -0500
              Re: question on java lang spec chapter 3.3 (unicode char lexing) "Aryeh M. Friedman" <Aryeh.Friedman@gmail.com> - 2013-01-02 18:20 -0800
        Re: question on java lang spec chapter 3.3 (unicode char lexing) Arne Vajhøj <arne@vajhoej.dk> - 2013-01-02 21:17 -0500
        Re: question on java lang spec chapter 3.3 (unicode char lexing) Patricia Shanahan <pats@acm.org> - 2013-01-02 22:33 -0800
          Re: question on java lang spec chapter 3.3 (unicode char lexing) "Chris Uppal" <chris.uppal@metagnostic.REMOVE-THIS.org> - 2013-01-05 12:58 +0000
            Re: question on java lang spec chapter 3.3 (unicode char lexing) "Aryeh M. Friedman" <Aryeh.Friedman@gmail.com> - 2013-01-05 05:34 -0800
              Re: question on java lang spec chapter 3.3 (unicode char lexing) "Aryeh M. Friedman" <Aryeh.Friedman@gmail.com> - 2013-01-05 05:40 -0800
              Re: question on java lang spec chapter 3.3 (unicode char lexing) Arne Vajhøj <arne@vajhoej.dk> - 2013-01-06 21:56 -0500
      Re: question on java lang spec chapter 3.3 (unicode char lexing) Martin Gregorie <martin@address-in-sig.invalid> - 2013-01-03 21:14 +0000
        Re: question on java lang spec chapter 3.3 (unicode char lexing) Arne Vajhøj <arne@vajhoej.dk> - 2013-01-03 17:51 -0500
          Re: question on java lang spec chapter 3.3 (unicode char lexing) "Aryeh M. Friedman" <Aryeh.Friedman@gmail.com> - 2013-01-03 20:54 -0800
            Re: question on java lang spec chapter 3.3 (unicode char lexing) Martin Gregorie <martin@address-in-sig.invalid> - 2013-01-05 00:15 +0000
            Re: question on java lang spec chapter 3.3 (unicode char lexing) "Chris Uppal" <chris.uppal@metagnostic.REMOVE-THIS.org> - 2013-01-05 13:03 +0000
              Re: question on java lang spec chapter 3.3 (unicode char lexing) "Aryeh M. Friedman" <Aryeh.Friedman@gmail.com> - 2013-01-05 05:25 -0800
                Re: question on java lang spec chapter 3.3 (unicode char lexing) Arne Vajhøj <arne@vajhoej.dk> - 2013-01-06 21:49 -0500
                Re: question on java lang spec chapter 3.3 (unicode char lexing) "Aryeh M. Friedman" <Aryeh.Friedman@gmail.com> - 2013-01-06 23:26 -0800
            Re: question on java lang spec chapter 3.3 (unicode char lexing) Arne Vajhøj <arne@vajhoej.dk> - 2013-01-06 21:44 -0500
  Re: question on java lang spec chapter 3.3 (unicode char lexing) Arne Vajhøj <arne@vajhoej.dk> - 2013-01-02 19:54 -0500

csiph-web