Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #197871 > unrolled thread

re.sub(r'((?<=\A)|(?<=,))(?=,|\Z)', 'NA', ',1,,,two,3,,,') how does it work?

Back to article view | Back to comp.lang.python

Contents

  re.sub(r'((?<=\A)|(?<=,))(?=,|\Z)', 'NA', ',1,,,two,3,,,') how does it work? Veek M <veekjunk@foobar.com> - 2026-08-09 00:21 +0000
    Re: re.sub(r'((?<=\A)|(?<=,))(?=,|\Z)', 'NA', ',1,,,two,3,,,') how does it work? Jon Ribbens <jon+usenet@unequivocal.eu> - 2026-08-09 01:26 +0000
    Re: re.sub(r'((?<=\A)|(?<=,))(?=,|\Z)', 'NA', ',1,,,two,3,,,') how does it work? ram@zedat.fu-berlin.de (Stefan Ram) - 2026-08-09 11:24 +0000
      Re: re.sub(r'((?<=\A)|(?<=,))(?=,|\Z)', 'NA', ',1,,,two,3,,,') how does it work? Veek M <veekjunk@foobar.com> - 2026-08-11 09:21 +0000

#197871 — re.sub(r'((?<=\A)|(?<=,))(?=,|\Z)', 'NA', ',1,,,two,3,,,') how does it work?

look-ahead look-behind don't consume string so how does it advance through 
the string - could someone clearly explain how it works.

re.sub(r'((?<=\A)|(?<=,))(?=,|\Z)', 'NA', ',1,,,two,3,,,') 
'NA,1,NA,NA,two,3,NA,NA,NA'

re.sub(r'(?<![^,])(?![^,])', 'NA', ',1,,,two,3,,,')
'NA,1,NA,NA,two,3,NA,NA,NA'

My understanding is that there has to be a pattern that consumes the 
string eg: here [^,] consumes two 3 four and 5 but the look-ahead look-
behind eliminate 5

re.findall(r'(?<=,)[^,]+(?=,)', '1,two,3,four,5')
['two', '3', 'four']

If it's matching the empty string '' then why don't we get NA,NA1 etc 
for ,1

[toc] | [next] | [standalone]

#197872

On 2026-08-09, Veek M <veekjunk@foobar.com> wrote:
> look-ahead look-behind don't consume string so how does it advance through 
> the string - could someone clearly explain how it works.
>
> re.sub(r'((?<=\A)|(?<=,))(?=,|\Z)', 'NA', ',1,,,two,3,,,') 
> 'NA,1,NA,NA,two,3,NA,NA,NA'

It's matching the empty string. It'll advance because once it's matched
at a certain position, the regular expression matcher will advance the
start position to the end of that match before checking for another
match.

> re.sub(r'(?<![^,])(?![^,])', 'NA', ',1,,,two,3,,,')
> 'NA,1,NA,NA,two,3,NA,NA,NA'

That's not doing differently to what the above expression doesn't
not do. It's trying to get around the limitation of lookbehind
assertions that every pattern they could match must be of the
same fixed length, by replacing "prior to the match must be the
start of the string, or prior to the match must be a comma" with
"prior to the match must not be a character that is not a comma".

> My understanding is that there has to be a pattern that consumes the 
> string eg: here [^,] consumes two 3 four and 5 but the look-ahead look-
> behind eliminate 5
>
> re.findall(r'(?<=,)[^,]+(?=,)', '1,two,3,four,5')
> ['two', '3', 'four']
>
> If it's matching the empty string '' then why don't we get NA,NA1 etc 
> for ,1

Because the empty string in-between the "," and the "1" fails the
lookahead assertion that after the match must be a comma or the
end of the string (or must not be a character that is not a comma,
in the second version).

[toc] | [prev] | [next] | [standalone]

#197873

Veek M <veekjunk@foobar.com> wrote or quoted:
>If it's matching the empty string '' then why don't we get NA,NA1 etc 
>for ,1

| How Regex Engines Handle Lookarounds and Empty Matches
| 
| In regular expressions, lookarounds - including lookaheads (?= . . . )
| and lookbehinds (?<= . . . ) - are /zero-width assertions/. They act 
| as /conditional checks/. They inspect the string to see if a pattern
| exists, but they do not "consume" (wipe out or move past) any charac-
| ters.
| 
| Understanding how a regex engine processes these assertions requires
| looking at two distinct phases: /evaluation within a single match
| attempt/, and /progression through the string/.
| 
| 1. The Fixed-Position Rule (Chaining Lookarounds)
| 
| When a regex engine tests a pattern at a specific position in a
| string, the internal pointer stays completely still until the entire
| pattern either succeeds or fails.
| 
| If you chain multiple lookarounds together, they all evaluate
| from the exact same spatial slot, one after the other.
| 
| Example: Chaining Lookaheads
| 
| Consider the pattern "(?=x)(?=x)" applied to the string "xy".
| 
| 1.  The engine starts at Position 0 (the empty space right before 
|     "x").
| 
| 2.  First "(?=x)" check: Looks ahead from Position 0, sees "x", and
|     returns "True". The pointer does not move.
| 
| 3.  Second "(?=x)" check: Looks ahead from the same Position 0, sees 
|     "x" again, and returns "True". The pointer does not move.
| 
| Both assertions pass at Position 0. The engine declares a match of 
| length zero at that position. If one's using a substitution function
| like Python's "re.sub(r'(?=x)(?=x)?, 'z', ?xy')", it inserts "z" at
| that empty slot, resulting in "zxy".
| 
| 2. The Forced-Advance Rule (Preventing Infinite Loops)
| 
| If lookarounds don't move the pointer, why doesn't a global "search 
| and replace" loop forever, inserting infinitely many "z" at Position 
| 0?
| 
| Regex engines implement a universal safety mechanism: /The Forced-
| Advance Rule/.
| 
| Standard Match   If a regex matches actual characters (like "[^,]+"),
| the pointer naturally moves past those characters for the next search
| cycle.
| 
| Zero-Width Match   If a regex matches an empty string (zero width),
| the engine applies the safety rule. After completing the match and
| performing any substitution, /it forces the pointer forward by exactly
| one position/ before attempting the next match.
| 
| Edge Cases & Common Logic Traps
| 
| Edge Case A: Contradictory Lookarounds
| 
| What happens if you look ahead for two different characters at the
| same time?
| 
| (?=x)(?=y)
| 
| Result   This will never match anything.
| 
| Why   At any given empty slot in a string, the single character
| immediately following that slot cannot be both "x" and "y" simul-
| taneously.
| 
| Edge Case B: Conflicting Lookahead and Lookbehind
| 
| You can combine lookaheads and lookbehinds to pinpoint exact
| boundaries.
| 
| (?<=x)(?=y)
| 
| Result   This successfully matches the empty space between an "x" and
| a "y" (such as inside the string "xy").
| 
| Why   At Position 1 (between "x" and "y"), the lookbehind looks back-
| ward and sees "x" ("True"), while the lookahead looks forward and sees 
| "y" ("True").
| 
| Edge Case C: Quantifiers on Zero-Width Matches
| 
| What happens if you tell a lookaround to repeat using a quantifier
| like "*" (zero or more times)?
| 
| (?=x)*
| 
| Result   This creates an immediate zero-width match at every single
| position in the string, even where "x" does not exist.
| 
| Why   At a position where "x" is absent, "(?=x)" fails. However, 
| because "*" means "zero or more times," matching it zero times is 
| considered a total success. The engine registers an empty match, 
| forces the pointer forward by one via the safety rule, and repeats
| this at every slot.

  Lines marked with "| " come from my editing, where I start by writing
  prompts for the chatbot and then edit the generated texts and format
  them for USENET.

[toc] | [prev] | [next] | [standalone]

#197877

On 9 Aug 2026 11:24:12 GMT, Stefan Ram wrote:

> Veek M <veekjunk@foobar.com> wrote or quoted:
>>If it's matching the empty string '' then why don't we get NA,NA1 etc
>>for ,1
> 
> | How Regex Engines Handle Lookarounds and Empty Matches |
> | In regular expressions, lookarounds - including lookaheads (?= . . . )
> | and lookbehinds (?<= . . . ) - are /zero-width assertions/. They act |
> as /conditional checks/. They inspect the string to see if a pattern |
> exists, but they do not "consume" (wipe out or move past) any charac- |
> ters.
> |
> | Understanding how a regex engine processes these assertions requires |
> looking at two distinct phases: /evaluation within a single match |
> attempt/, and /progression through the string/.
> |
> | 1. The Fixed-Position Rule (Chaining Lookarounds)
> |
> | When a regex engine tests a pattern at a specific position in a |
> string, the internal pointer stays completely still until the entire |
> pattern either succeeds or fails.
> |
> | If you chain multiple lookarounds together, they all evaluate | from
> the exact same spatial slot, one after the other.
> |
> | Example: Chaining Lookaheads |
> | Consider the pattern "(?=x)(?=x)" applied to the string "xy".
> |
> | 1.  The engine starts at Position 0 (the empty space right before |   
>  "x").
> |
> | 2.  First "(?=x)" check: Looks ahead from Position 0, sees "x", and | 
>    returns "True". The pointer does not move.
> |
> | 3.  Second "(?=x)" check: Looks ahead from the same Position 0, sees |
>     "x" again, and returns "True". The pointer does not move.
> |
> | Both assertions pass at Position 0. The engine declares a match of |
> length zero at that position. If one's using a substitution function |
> like Python's "re.sub(r'(?=x)(?=x)?, 'z', ?xy')", it inserts "z" at |
> that empty slot, resulting in "zxy".
> |
> | 2. The Forced-Advance Rule (Preventing Infinite Loops)
> |
> | If lookarounds don't move the pointer, why doesn't a global "search |
> and replace" loop forever, inserting infinitely many "z" at Position |
> 0?
> |
> | Regex engines implement a universal safety mechanism: /The Forced-
> | Advance Rule/.
> |
> | Standard Match   If a regex matches actual characters (like "[^,]+"),
> | the pointer naturally moves past those characters for the next search
> | cycle.
> |
> | Zero-Width Match   If a regex matches an empty string (zero width),
> | the engine applies the safety rule. After completing the match and |
> performing any substitution, /it forces the pointer forward by exactly |
> one position/ before attempting the next match.
> |
> | Edge Cases & Common Logic Traps |
> | Edge Case A: Contradictory Lookarounds |
> | What happens if you look ahead for two different characters at the |
> same time?
> |
> | (?=x)(?=y)
> |
> | Result   This will never match anything.
> |
> | Why   At any given empty slot in a string, the single character |
> immediately following that slot cannot be both "x" and "y" simul-
> | taneously.
> |
> | Edge Case B: Conflicting Lookahead and Lookbehind |
> | You can combine lookaheads and lookbehinds to pinpoint exact |
> boundaries.
> |
> | (?<=x)(?=y)
> |
> | Result   This successfully matches the empty space between an "x" and
> | a "y" (such as inside the string "xy").
> |
> | Why   At Position 1 (between "x" and "y"), the lookbehind looks back-
> | ward and sees "x" ("True"), while the lookahead looks forward and sees
> | "y" ("True").
> |
> | Edge Case C: Quantifiers on Zero-Width Matches |
> | What happens if you tell a lookaround to repeat using a quantifier |
> like "*" (zero or more times)?
> |
> | (?=x)*
> |
> | Result   This creates an immediate zero-width match at every single |
> position in the string, even where "x" does not exist.
> |
> | Why   At a position where "x" is absent, "(?=x)" fails. However,
> | because "*" means "zero or more times," matching it zero times is |
> considered a total success. The engine registers an empty match,
> | forces the pointer forward by one via the safety rule, and repeats |
> this at every slot.
> 
>   Lines marked with "| " come from my editing, where I start by writing
>   prompts for the chatbot and then edit the generated texts and format
>   them for USENET.

Thank you for writing such a clear and detailed response to my question. I 
am still pondering much of what you said and trying to fit it in with what 
I've read and what others have said. Thanks man!

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web