Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.java.programmer > #28213 > unrolled thread

Debugging regexes

Back to article view | Back to comp.lang.java.programmer

Contents

  Debugging regexes Roedy Green <see_website@mindprod.com.invalid> - 2014-04-24 05:39 -0700
    Re: Debugging regexes leif.roar@invalid.com (Leif Roar Moldskred) - 2014-04-24 09:38 -0500
      Re: Debugging regexes Roedy Green <see_website@mindprod.com.invalid> - 2014-04-24 20:21 -0700
        Re: Debugging regexes markspace <markspace@nospam.nospam> - 2014-04-25 00:17 -0700
          Re: Debugging regexes Roedy Green <see_website@mindprod.com.invalid> - 2014-04-25 01:05 -0700
    Re: Debugging regexes Gene Wirchenko <genew@telus.net> - 2014-04-25 10:09 -0700
      Re: Debugging regexes Roedy Green <see_website@mindprod.com.invalid> - 2014-05-05 23:19 -0700

#28213 — Debugging regexes

Regexes are  black box. If they fail, you have no idea where.

It would be nice if Matcher had some debugging methods to tell you the
offset and length of the best/longest match it was able to make, even
if it did not completely match.
-- 
Roedy Green Canadian Mind Products http://mindprod.com
"Don't worry about people stealing an idea; if it's original, you'll
 have to shove it down their throats." 
~ Howard Aiken (born: 1900-03-08 died: 1973-03-14 at age: 73)

[toc] | [next] | [standalone]

#28215

Roedy Green <see_website@mindprod.com.invalid> wrote:
> Regexes are  black box. If they fail, you have no idea where.
> 
> It would be nice if Matcher had some debugging methods to tell you the
> offset and length of the best/longest match it was able to make, even
> if it did not completely match.

If your regex is so complicated that you can't find the error after a
few minutes search with the Mark I eyeball, you should break it up
into multiple passes or rewrite your parsing code to not use a regex.

For long regexes, it can be a good idea to format it across multiple 
lines with comments:

  Pattern exampleRegex = 
    Pattern.compile( "^\\s*"                    // Start of string, optional whitespace
    + "(19[5-9][0-9]|20[0-9][0-9])"             // G1: four digit year, from 1950 to 2099.
    + "-(0[1-9]|1[012])"			// '-' divisor, G2: month, 01 to 12
    + ":((\\w+\\s?)*)"                          // ':' divisor, G3: one or more words, separated by a 
                                                //              single whitespace
    + "(:(.+)?$"                                // Optional: ':' divisor, G6: Remaining contents of line
  );

-- 
Leif Roar Moldskred

[toc] | [prev] | [next] | [standalone]

#28220

On Thu, 24 Apr 2014 09:38:52 -0500, leif.roar@invalid.com (Leif Roar
Moldskred) wrote, quoted or indirectly quoted someone who said :

>Roedy Green <see_website@mindprod.com.invalid> wrote:
>> Regexes are  black box. If they fail, you have no idea where.
>> 
>> It would be nice if Matcher had some debugging methods to tell you the
>> offset and length of the best/longest match it was able to make, even
>> if it did not completely match.
>
>If your regex is so complicated that you can't find the error after a
>few minutes search with the Mark I eyeball, you should break it up
>into multiple passes or rewrite your parsing code to not use a regex.
>
>For long regexes, it can be a good idea to format it across multiple 
>lines with comments:
>
>  Pattern exampleRegex = 
>    Pattern.compile( "^\\s*"                    // Start of string, optional whitespace
>    + "(19[5-9][0-9]|20[0-9][0-9])"             // G1: four digit year, from 1950 to 2099.
>    + "-(0[1-9]|1[012])"			// '-' divisor, G2: month, 01 to 12
>    + ":((\\w+\\s?)*)"                          // ':' divisor, G3: one or more words, separated by a 
>                                                //              single whitespace
>    + "(:(.+)?$"                                // Optional: ':' divisor, G6: Remaining contents of line
>  );
Those are good tips. I often define named constants and build my
regexes out of trusted bits.

My two most common errors are:
forgetting to quote something like - . ( ) $
using \\ when I should have used \ and vice versa.  Sometimes I build
regexes in files, sometimes in Java source.

Maybe I should get on with writing the various proofreading tools I
imagined.

http://mindprod.com/project/regexcomposer.html
http://mindprod.com/project/regexdebugger.html
http://mindprod.com/project/regexproofreader.html
http://mindprod.com/project/regexutility.html
-- 
Roedy Green Canadian Mind Products http://mindprod.com
Computers are like Old Testament gods; lots of rules and no mercy. 
~ Joseph Campbell (born: 1904-03-26 died: 1987-10-31 at age: 83)

[toc] | [prev] | [next] | [standalone]

#28221

On 4/24/2014 8:21 PM, Roedy Green wrote:

> Those are good tips. I often define named constants and build my
> regexes out of trusted bits.
>
> My two most common errors are:
> forgetting to quote something like - . ( ) $
> using \\ when I should have used \ and vice versa.  Sometimes I build
> regexes in files, sometimes in Java source.


A new Java 8 annotation checks regex syntax:

@Regex – Provides compile-time verification that a String intended to be 
used as a regular expression is a properly formatted regular expression


https://blogs.oracle.com/java-platform-group/entry/java_8_s_new_type

[toc] | [prev] | [next] | [standalone]

#28222

On Fri, 25 Apr 2014 00:17:16 -0700, markspace
<markspace@nospam.nospam> wrote, quoted or indirectly quoted someone
who said :

>@Regex – Provides compile-time verification that a String intended to be 
>used as a regular expression is a properly formatted regular expression
>
>
>https://blogs.oracle.com/java-platform-group/entry/java_8_s_new_type

They have also invented @m to ensure units of measure consistency. It
not do unit conversions as you would expect of a full engineering
language, but that at least eliminates most of the errors.
-- 
Roedy Green Canadian Mind Products http://mindprod.com
Computers are like Old Testament gods; lots of rules and no mercy. 
~ Joseph Campbell (born: 1904-03-26 died: 1987-10-31 at age: 83)

[toc] | [prev] | [next] | [standalone]

#28232

On Thu, 24 Apr 2014 05:39:38 -0700, Roedy Green
<see_website@mindprod.com.invalid> wrote:

>Regexes are  black box. If they fail, you have no idea where.

     True.  For that reason, I often use a finite state automaton.  A
regex either succeeds or fails.  I prefer to pick up more information.

     When I use regexes, I keep them simple, because debugging one can
be a real bear.  I will use short regexes with glue code rather than
one monster.

>It would be nice if Matcher had some debugging methods to tell you the
>offset and length of the best/longest match it was able to make, even
>if it did not completely match.

     I wish that there were not so many varieties regex languages.

Sincerely,

Gene Wirchenko

[toc] | [prev] | [next] | [standalone]

#28387

On Fri, 25 Apr 2014 10:09:22 -0700, Gene Wirchenko <genew@telus.net>
wrote, quoted or indirectly quoted someone who said :

>    I wish that there were not so many varieties regex languages

Amen brother.

I use three different schemes every day:
1. Java
2. Funduc search replace
3. Visual Slick edit Unix

It is so hard to remember six different sets of reserved characters,
just to get started.  I wrote a little tool to help
http://mindprod.com/applet/quoter.html

It takes any string and treats it as data and quotes it
by the three different schemes, search  or replace.
and shows it raw, in a Java String, in a CSV file.

I was browsing http://regexbuddy.com

Their products supports many different schemes.  I asked about the
three I use. They did not respond.

I have written a specialized CSV-based regex search/replace.  I like
working with Java regexes best since they are rich in features, and
well documented.

It takes lines of the form

fromString, toString, file1, file2, file3...

At some point I will teach itn wildcards and negatives.

-- 
Roedy Green Canadian Mind Products http://mindprod.com
Culture is your operating system. 
~ Terence McKenna (born: 1946-11-16 died: 2000-04-03 at age: 53)

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.java.programmer

csiph-web