Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #109952 > unrolled thread

python regex: variable length of positive lookbehind assertion

Back to article view | Back to comp.lang.python

Contents

  python regex: variable length of positive lookbehind assertion Yubin Ruan <ablacktshirt@gmail.com> - 2016-06-14 20:28 -0700
    Re: python regex: variable length of positive lookbehind assertion Lawrence D’Oliveiro <lawrencedo99@gmail.com> - 2016-06-14 21:18 -0700
      Re: python regex: variable length of positive lookbehind assertion Yubin Ruan <ablacktshirt@gmail.com> - 2016-06-14 21:38 -0700
    Re: python regex: variable length of positive lookbehind assertion Jussi Piitulainen <jussi.piitulainen@helsinki.fi> - 2016-06-15 08:10 +0300
    Re: python regex: variable length of positive lookbehind assertion Vlastimil Brom <vlastimil.brom@gmail.com> - 2016-06-15 10:31 +0200
    Re: python regex: variable length of positive lookbehind assertion alister <alister.ware@ntlworld.com> - 2016-06-15 12:27 +0000
      Re: python regex: variable length of positive lookbehind assertion Jussi Piitulainen <jussi.piitulainen@helsinki.fi> - 2016-06-15 15:55 +0300
        Re: python regex: variable length of positive lookbehind assertion Marko Rauhamaa <marko@pacujo.net> - 2016-06-15 17:33 +0300
          Re: python regex: variable length of positive lookbehind assertion Jussi Piitulainen <jussi.piitulainen@helsinki.fi> - 2016-06-15 17:57 +0300
            Re: python regex: variable length of positive lookbehind assertion Michael Torrie <torriem@gmail.com> - 2016-06-15 18:40 -0600
              Re: python regex: variable length of positive lookbehind assertion Jussi Piitulainen <jussi.piitulainen@helsinki.fi> - 2016-06-16 08:39 +0300
                Re: python regex: variable length of positive lookbehind assertion Marko Rauhamaa <marko@pacujo.net> - 2016-06-16 09:03 +0300
        Re: python regex: variable length of positive lookbehind assertion alister <alister.ware@ntlworld.com> - 2016-06-15 15:31 +0000
          Re: python regex: variable length of positive lookbehind assertion Jussi Piitulainen <jussi.piitulainen@helsinki.fi> - 2016-06-15 19:04 +0300

#109952 — python regex: variable length of positive lookbehind assertion

Hi everyone, 
I am struggling writing a right regex that match what I want:

Problem Description:

Given a string like this:

    >>>string = "false_head <a>aaa</a> <a>bbb</a> false_tail \
             true_head some_text_here <a>ccc</a> <a>ddd</a> <a>eee</a> true_tail"

I want to match the all the text surrounded by those "<a> </a>",
but only if those "<a> </a>" locate **in some distance** behind "true_head". That is, I expect to result to be like this:

    >>>import re
    >>>result = re.findall("the_regex",string)
    >>>print result
    ["ccc","ddd","eee"]

How can I write a regex to match that?
I have try to use the **positive lookbehind assertion** in python regex,
but it does not allowed variable length of lookbehind.

Thanks in advance,
Ruan

[toc] | [next] | [standalone]

#109955

On Wednesday, June 15, 2016 at 3:28:37 PM UTC+12, Yubin Ruan wrote:

> I want to match the all the text surrounded by those "<a> </a>",

You are trying to use regex (type 3 grammar) to parse HTML (type 2 grammar) <https://en.wikipedia.org/wiki/Formal_grammar#The_Chomsky_hierarchy>?

No can do <http://stackoverflow.com/questions/1732348/regex-match-open-tags-except-xhtml-self-contained-tags/1732454#1732454>.

[toc] | [prev] | [next] | [standalone]

#109956

On Wednesday, June 15, 2016 at 12:18:31 PM UTC+8, Lawrence D’Oliveiro wrote:
> On Wednesday, June 15, 2016 at 3:28:37 PM UTC+12, Yubin Ruan wrote:
> 
> > I want to match the all the text surrounded by those "<a> </a>",
> 
> You are trying to use regex (type 3 grammar) to parse HTML (type 2 grammar) <https://en.wikipedia.org/wiki/Formal_grammar#The_Chomsky_hierarchy>?
> 
> No can do <http://stackoverflow.com/questions/1732348/regex-match-open-tags-except-xhtml-self-contained-tags/1732454#1732454>.


Yes. I think you are correct. Thanks.

[toc] | [prev] | [next] | [standalone]

#109957

Yubin Ruan writes:

> Hi everyone, 
> I am struggling writing a right regex that match what I want:
>
> Problem Description:
>
> Given a string like this:
>
>     >>>string = "false_head <a>aaa</a> <a>bbb</a> false_tail \
>              true_head some_text_here <a>ccc</a> <a>ddd</a> <a>eee</a> true_tail"
>
> I want to match the all the text surrounded by those "<a> </a>", but
> only if those "<a> </a>" locate **in some distance** behind
> "true_head". That is, I expect to result to be like this:
>
>     >>>import re
>     >>>result = re.findall("the_regex",string)
>     >>>print result
>     ["ccc","ddd","eee"]
>
> How can I write a regex to match that?
> I have try to use the **positive lookbehind assertion** in python regex,
> but it does not allowed variable length of lookbehind.

Don't.

Don't even try to do it all in one regex. Keep your regexen simple and
match in two steps.

For example, capture all such elements together with your marker:

re.findall(r'true_head|<a>[^<]+</a>', string)
==>
['<a>aaa</a>', '<a>bbb</a>',
 'true_head', '<a>ccc</a>', '<a>ddd</a>', '<a>eee</a>']

Then filter the result in the obvious way (not involving any regex any
more, unless needed to recognize the true 'true_head' again). I've kept
the tags at this stage, so a possible '<a>true_head</a>' won't look like
'true_head' yet.

Another way is to find 'true_head' first (if you can recognize it safely
before also recognizing the elements), and then capture the elements in
the latter half only.

[toc] | [prev] | [next] | [standalone]

#109960

2016-06-15 5:28 GMT+02:00 Yubin Ruan <ablacktshirt@gmail.com>:
> Hi everyone,
> I am struggling writing a right regex that match what I want:
>
> Problem Description:
>
> Given a string like this:
>
>     >>>string = "false_head <a>aaa</a> <a>bbb</a> false_tail \
>              true_head some_text_here <a>ccc</a> <a>ddd</a> <a>eee</a> true_tail"
>
> I want to match the all the text surrounded by those "<a> </a>",
> but only if those "<a> </a>" locate **in some distance** behind "true_head". That is, I expect to result to be like this:
>
>     >>>import re
>     >>>result = re.findall("the_regex",string)
>     >>>print result
>     ["ccc","ddd","eee"]
>
> How can I write a regex to match that?
> I have try to use the **positive lookbehind assertion** in python regex,
> but it does not allowed variable length of lookbehind.
>
> Thanks in advance,
> Ruan
> --
> https://mail.python.org/mailman/listinfo/python-list

Hi,
html-like data is generally not very suitable for parsing with regex,
as was explained in the previous answers (especially if comments and
nesting are massively involved).
However, if this suits your data and the usecase, you can use regex
with variable-length lookarounds in a much enhanced "regex" library
for python
https://pypi.python.org/pypi/regex

your pattern might then simply have the form you most likely have
intended, e.g.:
>>> regex.findall(r"(?<=true_head.*)<a>([^<]+)</a>(?=.*true_tail)", "false_head <a>aaa</a> <a>bbb</a> false_tail true_head some_text_here <a>ccc</a> <a>ddd</a> <a>eee</a> true_tail <a>fff</a> another_false_tail")
['ccc', 'ddd', 'eee']
>>>

If you are accustomed to use regular expressions, I'd certainly
recommend this excellent library (besides unlimited lookarounds, there
are repeated and recursive patterns, many unicode-related
enhancements, powerful character set operations, even fuzzy matching
and much more).

hth,
   vbr

[toc] | [prev] | [next] | [standalone]

#109964

On Tue, 14 Jun 2016 20:28:24 -0700, Yubin Ruan wrote:

> Hi everyone,
> I am struggling writing a right regex that match what I want:
> 
> Problem Description:
> 
> Given a string like this:
> 
>     >>>string = "false_head <a>aaa</a> <a>bbb</a> false_tail \
>              true_head some_text_here <a>ccc</a> <a>ddd</a> <a>eee</a>
>              true_tail"
> 
> I want to match the all the text surrounded by those "<a> </a>",
> but only if those "<a> </a>" locate **in some distance** behind
> "true_head". That is, I expect to result to be like this:
> 
>     >>>import re result = re.findall("the_regex",string)
>     >>>print result
>     ["ccc","ddd","eee"]
> 
> How can I write a regex to match that?
> I have try to use the **positive lookbehind assertion** in python regex,
> but it does not allowed variable length of lookbehind.
> 
> Thanks in advance,
> Ruan

don't try to use regex to parse html it wont work reliably
i am surprised no one has mentioned beautifulsoup yet, which is probably 
what you require.





-- 
What we anticipate seldom occurs; what we least expect generally happens.
-- Bengamin Disraeli

[toc] | [prev] | [next] | [standalone]

#109966

alister writes:

> On Tue, 14 Jun 2016 20:28:24 -0700, Yubin Ruan wrote:
>
>> Hi everyone,
>> I am struggling writing a right regex that match what I want:
>> 
>> Problem Description:
>> 
>> Given a string like this:
>> 
>>     >>>string = "false_head <a>aaa</a> <a>bbb</a> false_tail \
>>              true_head some_text_here <a>ccc</a> <a>ddd</a> <a>eee</a>
>>              true_tail"
>> 
>> I want to match the all the text surrounded by those "<a> </a>",
>> but only if those "<a> </a>" locate **in some distance** behind
>> "true_head". That is, I expect to result to be like this:
>> 
>>     >>>import re result = re.findall("the_regex",string)
>>     >>>print result
>>     ["ccc","ddd","eee"]
>> 
>> How can I write a regex to match that?
>> I have try to use the **positive lookbehind assertion** in python regex,
>> but it does not allowed variable length of lookbehind.
>> 
>> Thanks in advance,
>> Ruan
>
> don't try to use regex to parse html it wont work reliably
> i am surprised no one has mentioned beautifulsoup yet, which is probably 
> what you require.

Nothing in the question indicates that the data is HTML.

[toc] | [prev] | [next] | [standalone]

#109974

Jussi Piitulainen <jussi.piitulainen@helsinki.fi>:

> alister writes:
>
>> On Tue, 14 Jun 2016 20:28:24 -0700, Yubin Ruan wrote:
>>> Given a string like this:
>>> 
>>>     >>>string = "false_head <a>aaa</a> <a>bbb</a> false_tail \
>>>              true_head some_text_here <a>ccc</a> <a>ddd</a> <a>eee</a>
>>>              true_tail"
>>>
>>> I want to match the all the text surrounded by those "<a> </a>",
>>> [...]
>>
>> don't try to use regex to parse html it wont work reliably
>> [...]
>
> Nothing in the question indicates that the data is HTML.

And nothing in alister's answer suggests that.


Marko

[toc] | [prev] | [next] | [standalone]

#109975

Marko Rauhamaa writes:

> Jussi Piitulainen writes:
>
>> alister writes:
>>
>>> On Tue, 14 Jun 2016 20:28:24 -0700, Yubin Ruan wrote:
>>>> Given a string like this:
>>>> 
>>>>     >>>string = "false_head <a>aaa</a> <a>bbb</a> false_tail \
>>>>              true_head some_text_here <a>ccc</a> <a>ddd</a> <a>eee</a>
>>>>              true_tail"
>>>>
>>>> I want to match the all the text surrounded by those "<a> </a>",
>>>> [...]
>>>
>>> don't try to use regex to parse html it wont work reliably
>>> [...]
>>
>> Nothing in the question indicates that the data is HTML.
>
> And nothing in alister's answer suggests that.

Now *I'm* surprised.

[toc] | [prev] | [next] | [standalone]

#110000

On 06/15/2016 08:57 AM, Jussi Piitulainen wrote:
> Marko Rauhamaa writes:
>> And nothing in alister's answer suggests that.
> 
> Now *I'm* surprised.

He simply said, here's a regex that can parse the example string the OP
gave us (which maybe looked a bit like HTML, but like you say, may not
be), but don't try to use this method to parse actual HTML because it
won't work reliably.

[toc] | [prev] | [next] | [standalone]

#110013

Michael Torrie writes:

> On 06/15/2016 08:57 AM, Jussi Piitulainen wrote:
>> Marko Rauhamaa writes:
>>> And nothing in alister's answer suggests that.
>> 
>> Now *I'm* surprised.
>
> He simply said, here's a regex that can parse the example string the OP
> gave us (which maybe looked a bit like HTML, but like you say, may not
> be), but don't try to use this method to parse actual HTML because it
> won't work reliably.

Interesting how differently we can read alister's answer. It was only
two sentences, one of which Marko replaced with "[...]" before adding
his own one-liner that is still quoted above. Let me quote alister's
response in full here, the way I see it in Gnus:

# don't try to use regex to parse html it wont work reliably
# i am surprised no one has mentioned beautifulsoup yet, which is probably 
# what you require.

That followed the fully quoted original message, and then there was an
attributed citation from a Bengamin Disraeli, separated as a .sig.

Where in alister's original response do you see a regex that can parse
OP's example? I don't see any regex there. (The text where you seem to
me to say that there is one is still quoted above in the normal way.)

Instead of giving any direct answer to the question, alister expresses
surprise at nobody having suggested an HTML parser. (Marko snipped that,
but I've quoted alister's response in full above, so you can check it
without looking up the original messages.)

A surprise calls for an explanation. Or should I say that I felt that
this particular expression of surprise seemed to me to call for an
explanation, or in the very least that an explanation would not do much
harm and might even be considered mildly interesting. And I saw a fully
adequate explanation: that the question was not about parsing HTML. So I
said so.

[toc] | [prev] | [next] | [standalone]

#110016

Jussi Piitulainen <jussi.piitulainen@helsinki.fi>:

> Michael Torrie writes:
>
>> On 06/15/2016 08:57 AM, Jussi Piitulainen wrote:
>>> Marko Rauhamaa writes:
>>>> And nothing in alister's answer suggests that.
>>> 
>>> Now *I'm* surprised.
>>
>> He simply said, here's a regex that can parse the example string the OP
>> gave us (which maybe looked a bit like HTML, but like you say, may not
>> be), but don't try to use this method to parse actual HTML because it
>> won't work reliably.
>
> Interesting how differently we can read alister's answer. It was only
> two sentences, one of which Marko replaced with "[...]" before adding
> his own one-liner that is still quoted above.
> [...]
>
> That followed the fully quoted original message, and then there was an
> attributed citation from a Bengamin Disraeli, separated as a .sig.
>
> [...]
>
> A surprise calls for an explanation. Or should I say that I felt that
> this particular expression of surprise seemed to me to call for an
> explanation, or in the very least that an explanation would not do much
> harm and might even be considered mildly interesting. And I saw a fully
> adequate explanation: that the question was not about parsing HTML. So I
> said so.

This is so meta.


Marko

[toc] | [prev] | [next] | [standalone]

#109979

On Wed, 15 Jun 2016 15:55:42 +0300, Jussi Piitulainen wrote:

> alister writes:
> 
>> On Tue, 14 Jun 2016 20:28:24 -0700, Yubin Ruan wrote:
>>
>>> Hi everyone,
>>> I am struggling writing a right regex that match what I want:
>>> 
>>> Problem Description:
>>> 
>>> Given a string like this:
>>> 
>>>     >>>string = "false_head <a>aaa</a> <a>bbb</a> false_tail \
>>>              true_head some_text_here <a>ccc</a> <a>ddd</a> <a>eee</a>
>>>              true_tail"
>>> 
>>> I want to match the all the text surrounded by those "<a> </a>",
>>> but only if those "<a> </a>" locate **in some distance** behind
>>> "true_head". That is, I expect to result to be like this:
>>> 
>>>     >>>import re result = re.findall("the_regex",string) print result
>>>     ["ccc","ddd","eee"]
>>> 
>>> How can I write a regex to match that?
>>> I have try to use the **positive lookbehind assertion** in python
>>> regex,
>>> but it does not allowed variable length of lookbehind.
>>> 
>>> Thanks in advance,
>>> Ruan
>>
>> don't try to use regex to parse html it wont work reliably i am
>> surprised no one has mentioned beautifulsoup yet, which is probably
>> what you require.
> 
> Nothing in the question indicates that the data is HTML.

the <a></a> tags are a prety good indicator though
even if it is not HTML the same advise stands for XML (the quote example 
would be invalid if it was XML)

if it is neither for these formats but still using a similar tag 
structure then I would say that Reg ex is still unsuitable & the OP would 
need to write a full parser for the format if one does not already exist



-- 
Farewell we call to hearth and hall!
Though wind may blow and rain may fall,
We must away ere break of day
Far over wood and mountain tall.

	To Rivendell, where Elves yet dwell
	In glades beneath the misty fell,
	Through moor and waste we ride in haste,
	And whither then we cannot tell.

With foes ahead, behind us dread,
Beneath the sky shall be our bed,
Until at last our toil be passed,
Our journey done, our errand sped.

	We must away!  We must away!
	We ride before the break of day!
		-- J. R. R. Tolkien

[toc] | [prev] | [next] | [standalone]

#109981

alister writes:

> On Wed, 15 Jun 2016 15:55:42 +0300, Jussi Piitulainen wrote:
>
>> alister writes:
>> 
>>> On Tue, 14 Jun 2016 20:28:24 -0700, Yubin Ruan wrote:
>>>
>>>> Hi everyone,
>>>> I am struggling writing a right regex that match what I want:
>>>> 
>>>> Problem Description:
>>>> 
>>>> Given a string like this:
>>>> 
>>>>     >>>string = "false_head <a>aaa</a> <a>bbb</a> false_tail \
>>>>              true_head some_text_here <a>ccc</a> <a>ddd</a> <a>eee</a>
>>>>              true_tail"
>>>> 
>>>> I want to match the all the text surrounded by those "<a> </a>",
>>>> but only if those "<a> </a>" locate **in some distance** behind
>>>> "true_head". That is, I expect to result to be like this:
>>>> 
>>>>     >>>import re result = re.findall("the_regex",string) print result
>>>>     ["ccc","ddd","eee"]
>>>> 
>>>> How can I write a regex to match that?
>>>> I have try to use the **positive lookbehind assertion** in python
>>>> regex,
>>>> but it does not allowed variable length of lookbehind.
>>>> 
>>>> Thanks in advance,
>>>> Ruan
>>>
>>> don't try to use regex to parse html it wont work reliably i am
>>> surprised no one has mentioned beautifulsoup yet, which is probably
>>> what you require.
>> 
>> Nothing in the question indicates that the data is HTML.
>
> the <a></a> tags are a prety good indicator though

I can see how they point that way, but to me that alone seemed pretty
weak.

> even if it is not HTML the same advise stands for XML (the quote
> example would be invalid if it was XML)

It's not valid HTML either, for similar reasons. Or is it? I don't even
want to know.

> if it is neither for these formats but still using a similar tag
> structure then I would say that Reg ex is still unsuitable & the OP
> would need to write a full parser for the format if one does not
> already exist

That depends on details that weren't provided.

I work with a data format that mixes element tags with line-oriented
data records, and having a dedicated parser would be more of a hassle. A
couple of very simple regexen are useful in making sure that start tags
have a valid form and extracting attribute-value pairs from them. I'm
not at all experiencing "two problems" here. Some uses of regex are
good. (And now I may be about to experience the third problem. That
makes me sad.)

Anyway, I think you and another person guessed correctly that the OP is
indeed really considering HTML, and then your suggestion is certainly
helpful.

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web