Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.programming > #12280

Re: Advice needed on parsing a youtube page

Show all headers | View raw

Spiros Bousbouras <spibou@gmail.com> writes:

> Otherwise , in your last 2 paragraphs you seem to only restate my question.
> In order to find the right tags , I need to find the JSON strings ; and in
> order to find those I need to locate correctly where each string (whether
> it is part of JSON or part of an HTML tag) starts and ends ; and in order
> to do that I need to know whether a string is part of JSON or part of an
> HTML tag so that I can apply the correct rule to locate the closing double
> quote.

I know it's annoying, but I have to correct some of your terms (or at
least clarify them) so we can all be on the same page.  An HTML tag
looks like this: <p>.  That's an opening tag.  You can also have closing
tags like </p>.  Some tags denote elements (like <hr>) all by
themselves, but often an element is constructed form the opening tag,
the content and the (sometimes implicit) closing tag.

The point of this is just to say the you won't usually find JSON in a
tag (in fact, you won't find it "native" in HTML at all).  You do
sometimes find JSON as an ECMAScript (AKA JavaScript) string.  If you
do, it will be in the content of a <script> element.

The specific string that was causing trouble (content="\Raffaello
D&#39;Andrea\") /is/ inside a tag.  It's the value of the 'content'
attribute in a 'mata' tag.  The slashes are just part of the value of
the attribute.  They don't escape anything.

I could not find any JSON in the document you cited, but I confess I did
not look all that hard.  I did find list of scripting that has parts
that look like JSON, but that's not quite the same thing.

To do what you want in the general case involves parsing HTML.  The
trouble with doing that is that real-world documents often have
ungrammatical constructs, so browsers have parsers that apply a lot of
heuristics to guess what was meant.  Fortunately, you can find HTML
parsers for many programming languages.

If you don't go that route you have to decide how much you are prepared
simply to "wing it".  Can you point out where the parts of the document
are that you want to scrape?  Maybe there is some simple work-around
that does not involve much parsing at all.

-- 
Ben.

Back to comp.programming | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Advice needed on parsing a youtube page Spiros Bousbouras <spibou@gmail.com> - 2021-02-27 14:43 +0000
  Re: Advice needed on parsing a youtube page Spiros Bousbouras <spibou@gmail.com> - 2021-02-27 16:43 +0000
    Re: Advice needed on parsing a youtube page Richard Damon <Richard@Damon-Family.org> - 2021-02-27 14:30 -0500
      Re: Advice needed on parsing a youtube page Spiros Bousbouras <spibou@gmail.com> - 2021-02-27 20:30 +0000
        Re: Advice needed on parsing a youtube page Ben Bacarisse <ben.usenet@bsb.me.uk> - 2021-02-27 21:42 +0000
          Re: Advice needed on parsing a youtube page Spiros Bousbouras <spibou@gmail.com> - 2021-02-28 21:02 +0000
            Re: Advice needed on parsing a youtube page Ben Bacarisse <ben.usenet@bsb.me.uk> - 2021-02-28 23:55 +0000
              Re: Advice needed on parsing a youtube page Spiros Bousbouras <spibou@gmail.com> - 2021-03-02 18:51 +0000
                Re: Advice needed on parsing a youtube page Kristjan Robam <he12457@hotmail.com> - 2021-03-02 17:50 -0800
  Re: Advice needed on parsing a youtube page Richard Damon <Richard@Damon-Family.org> - 2021-02-27 14:09 -0500
    Re: Advice needed on parsing a youtube page Spiros Bousbouras <spibou@gmail.com> - 2021-02-27 20:12 +0000
      Re: Advice needed on parsing a youtube page Richard Damon <Richard@Damon-Family.org> - 2021-02-27 22:18 -0500
        Re: Advice needed on parsing a youtube page Spiros Bousbouras <spibou@gmail.com> - 2021-03-02 17:30 +0000

csiph-web