Path: csiph.com!v102.xanadu-bbs.net!xanadu-bbs.net!feeder.erje.net!eu.feeder.erje.net!eternal-september.org!feeder.eternal-september.org!mx05.eternal-september.org!.POSTED!not-for-mail
From: markspace <markspace@nospam.nospam>
Newsgroups: comp.lang.java.programmer
Subject: Re: How to scan Java source texts?
Date: Tue, 11 Jun 2013 12:55:52 -0700
Organization: A noiseless patient Spider
Lines: 19
Message-ID: <kp7v3b$508$1@dont-email.me>
References: <Java-Scanner-20130611180636@ram.dialup.fu-berlin.de> <tokens-20130611205333@ram.dialup.fu-berlin.de>
Mime-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
Injection-Date: Tue, 11 Jun 2013 19:51:07 +0000 (UTC)
Injection-Info: mx05.eternal-september.org; posting-host="fba3415ba68d85d643935af2f52f0b4b"; logging-data="5128"; mail-complaints-to="abuse@eternal-september.org";	posting-account="U2FsdGVkX1/mQGyw1rjGokWbaFCK1DDVQaaFstVcYHg="
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:17.0) Gecko/20130509 Thunderbird/17.0.6
In-Reply-To: <tokens-20130611205333@ram.dialup.fu-berlin.de>
Cancel-Lock: sha1:Cqif6suKvKDTD6FmmcNa2HiroCE=
Xref: csiph.com comp.lang.java.programmer:24363

On 6/11/2013 11:54 AM, Stefan Ram wrote:
> ram@zedat.fu-berlin.de (Stefan Ram) writes:
>> I am able to write a scanner for Java myself, but this would
>> take days. So I would like to shortcut it by using a Java SE
>> (with JDK) call. (I would not like to use a third-party
>
>    It might not be easy to get this right. For example, a


No it's not.  I recommend a third party library.  Antlr has a Java 
syntax already worked out.  There's also other dedicated Java parsers.

Note you're talking about two things here.  Lexing and parsing.  A lexer 
breaks text up into tokens, a parser decides how to interpret the 
result.  Parsers traditionally have a lot more contextual information, 
whereas lexers are just simpler state machines that break up text.