Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.javascript > #16002

Re: regexp help

Followups directed to: comp.lang.javascript

Show key headers only | View raw

Scott Sauyet wrote:

> Thomas 'PointedEars' Lahn wrote:
>> Scott Sauyet wrote:
>>> Thomas 'PointedEars' Lahn wrote:
>>>> [ ... ]
>>>> Which gives me the idea that a Unicode mode for jsx.regexp.RegExp() to
>>>> replace \b, \w and \W in addition to or using the already implemented
>>>> Unicode character properties (\p{Lu} et al.) would come in handy.
>> [ ... ]
>>> I haven't looked at the jsx.regexp implementation. �Would this be
>>> straightforward to add?
>> 
>> It was not straightforward to implement (character property class
>> definitions L, M, N, and Word had to be added, and the character class
>> normalizer had to be generalized � so that `[^a\Wb]' is equivalent to
>> `(?: [^ab]|[\p{Word}])' *before* escape sequence expansion).
>> 
>> But � as my Twitter followers are already aware � I did it today. �
>> Currently, \w and \W only; \b is a bit more tricky.

(You want to do something about that encoding problem of your posting 
agent.)
 
> Yes, I dug around in the implementation a bit this morning, only enough to
> see the broad outlines of what you're doing.  It looks interesting, and it
> looks challenging.  And it looks entirely worth doing.  I have no idea yet
> of how well you've managed to accomplish it.  So maybe I shouldn't be as
> impressed as I am. :-)

The JSdoc for jsx.regexp.RegExp has grown rather descriptive in the meantime 
(good that Resource Builder can automagically remove them for deployment) :)

If you re-run the unit test, you will notice that, based on the 
implementation for \w, it would appear that in the meantime I have mostly 
implemented \b in Unicode mode, and pattern-matching modifiers (?sx-sx) for 
the already implemented `s' (PCRE_DOTALL) and `x' (PCRE_EXTENDED) flags:

<http://PointedEars.de/scripts/test/regexp>

I am working on the rest of \b (`\b' after `('s and `\b' with match(/…/g) 
still need fixing), and other character class escapes, like \d and \s, in 
Unicode mode.  Although Perl and PCRE do not support it, given the current 
implementation of \w, \b, and \p{…}, it might be possible to implement in 
ECMAScript (?u-u) for `u' (would anyone find this useful?).  And then 
there's the greater challenge of implementing `(?im-im)'.
 
>>> Obviously you could make whatever call you choose about
>>> what to include, but is there an obvious call to make?
>> 
>> Yes, Perl already specifies it, so I followed perlrecharclass(1) (but I
>> forgot to include \p{N}; that has been fixed in the meantime).
>> 
>> However, I can see now that it is wrong to call my implementation of \w
>> in Unicode mode *PCRE*-conforming, as pcrepattern(1) specifies it
>> differently: \w is equivalent to [\p{L}\p{N}_] there, while according to
>> perlrecharclass(1) it appears to be [\p{L}\p{M}\p{N}\p{Pc}] in Perl.
>> 
>> I think that Perl's definition is better than PCRE's here [ ... ]
> 
> So yes, there are obvious choices, but which of the obvious choices is
> best is still up for grabs?  :-)
          ^^^^^^^^^^^^^^^^^^
I am not sure I fully understand that idiom.  However, saying that I think 
Perl's definition is better with \w, \W, and \b, already indicated that I 
intended to implement that instead of PCRE's.  So I did.  But AISB, if you 
are not happy with that, you can easily redefine the escape sequence, even 
during runtime (provided that the UCD has been loaded before; I am working 
on that as well):

  var RegExp2 = jsx.regexp.RegExp;

  /* \p{L}\p{M}\p{N}\p{Pc} */
  console.log(RegExp2.propertyClasses["Word"]);

  /* \w, \p{Word} à la PCRE */
  RegExp2.definePropertyClasses({
    Word: "\\p{L}\\p{N}_"
  });

  /* see above */
  var rx = new RegExp2("\\b(?<foo>\\w+\\b)", "u");

  var input = "-trärä-";

  var m = rx.exec(input);

  if (m)
  {
    /* "trärä" "trärä" "trärä"¹ */
    console.log(m[0], m[1], m.groups["foo"]);
  }

  /* trärä */
  console.log((new jsx.regexp.String(input)).match(rx).groups["foo"]);
 
> I mostly have used JSX the way I do several other libraries by regulars
> here, as sources of inspiration, sometimes looking to say, "Ah, so that's
> how Garrett chooses to do it, and that's how Thomas does it, and that's
> how David's works.  I've never incorporated any of these libraries as they
> stand into projects.  I'm not quite sure why except that usually for my
> own projects, I like to roll my own code and for work, I'm almost always
> constrained to use certain libraries. (You can guess which ones!)

ACK.

> But I will try JSX regexp the next time I need to do something interesting
> with regex.

Thanks.
 
> Thank you for your effort with this.

My pleasure.  That ECMAScript supports Unicode natively but TC39 – including 
Brendan Eich, unfortunately – cannot find it within themselves to add a 
Unicode mode for RegExp (let alone PCRE_EXTENDED and PCRE_DOTALL and other 
useful features found in other scripting languages) has bothered me for 
years.  Finally I find the time to do something about it.


PointedEars
___________
¹  No, it's _not_ a premature carnival joke ;-)
-- 
Anyone who slaps a 'this page is best viewed with Browser X' label on
a Web page appears to be yearning for the bad old days, before the Web,
when you had very little chance of reading a document written on another
computer, another word processor, or another network. -- Tim Berners-Lee

Back to comp.lang.javascript | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

regexp help Andrew Poulos <ap_prog@hotmail.com> - 2012-09-07 14:37 +1000
  Re: regexp help Jake Jarvis <pig_in_shoes@yahoo.com> - 2012-09-07 10:55 +0200
    Re: regexp help Thomas 'PointedEars' Lahn <PointedEars@web.de> - 2012-09-07 15:36 +0200
      Re: regexp help Andrew Poulos <ap_prog@hotmail.com> - 2012-09-08 10:27 +1000
        Re: regexp help Thomas 'PointedEars' Lahn <PointedEars@web.de> - 2012-09-08 09:34 +0200
      Re: regexp help "Scott Sauyet" <3103invalid@webnews.fastusenet.org> - 2012-09-11 16:51 +0000
        Re: regexp help Thomas 'PointedEars' Lahn <PointedEars@web.de> - 2012-09-11 22:31 +0200
          Re: regexp help "Scott Sauyet" <3103invalid@webnews.fastusenet.org> - 2012-09-19 11:56 +0000
            Re: regexp help Thomas 'PointedEars' Lahn <PointedEars@web.de> - 2012-09-19 18:51 +0200
              Re: regexp help Scott Sauyet <scott.sauyet@gmail.com> - 2012-09-19 12:52 -0700
                Re: regexp help Thomas 'PointedEars' Lahn <PointedEars@web.de> - 2012-09-20 02:13 +0200
                Re: regexp help Scott Sauyet <scott.sauyet@gmail.com> - 2012-09-20 06:51 -0700
                Re: regexp help Dr J R Stockton <reply1237@merlyn.demon.co.uk.invalid> - 2012-09-20 20:23 +0100
      Re: regexp help "Michael Haufe (TNO)" <tno@thenewobjective.com> - 2012-09-11 14:55 -0700

csiph-web