Advice needed on parsing a youtube page

From	Spiros Bousbouras <spibou@gmail.com>
Newsgroups	comp.programming
Subject	Advice needed on parsing a youtube page
Message-ID	<Rafaello@bongo-ra.co> (permalink)
Organization	virginmedia.com
Date	2021-02-27 14:43 +0000

Show all headers | View raw

I have some code which parses the source of pages for youtube videos and
prints some information. My code has been working fine for months but
recently it choked on  https://www.youtube.com/watch?v=w2itwFJCgFQ .After
some investigations I found the part of the page which causes the problem :
there is a single appearance of

    content="\Raffaello D&#39;Andrea\"

.My code assumed that the closing double quote is escaped so it did not count
it as a closing quote and so parsed incorrectly the rest of the page. So I
have 2 questions :

1. Is the above segment valid according to whatever the relevant specification
is ?

2. My code only needs to parse the JSON content of pages and extract the
information I want. Is there a simple algorithm which strips all the non JSON
content of a page ? Or alternatively , is there an algorithm which parses
correctly constructs like the above , perhaps taking into account surrounding
tags ?


I should warn you that the source of the page has some very long lines , the
longest being 201416 bytes. So examining the content with some text editors or
less  will not work smoothly. Filtering the page through something like
sed 's/,/,\n/g'   makes things a lot more manageable.

-- 
You're on the right path when you understand what Luke Skywalker, Obi-Wan Kenobi,
and Darth Vader did not: It's not the force, it's the field!
  http://www-personal.umich.edu/~williams/emquotes.html
  David N. Williams

Back to comp.programming | Previous | Next — Next in thread | Find similar | Unroll thread

Thread

Advice needed on parsing a youtube page Spiros Bousbouras <spibou@gmail.com> - 2021-02-27 14:43 +0000
  Re: Advice needed on parsing a youtube page Spiros Bousbouras <spibou@gmail.com> - 2021-02-27 16:43 +0000
    Re: Advice needed on parsing a youtube page Richard Damon <Richard@Damon-Family.org> - 2021-02-27 14:30 -0500
      Re: Advice needed on parsing a youtube page Spiros Bousbouras <spibou@gmail.com> - 2021-02-27 20:30 +0000
        Re: Advice needed on parsing a youtube page Ben Bacarisse <ben.usenet@bsb.me.uk> - 2021-02-27 21:42 +0000
          Re: Advice needed on parsing a youtube page Spiros Bousbouras <spibou@gmail.com> - 2021-02-28 21:02 +0000
            Re: Advice needed on parsing a youtube page Ben Bacarisse <ben.usenet@bsb.me.uk> - 2021-02-28 23:55 +0000
              Re: Advice needed on parsing a youtube page Spiros Bousbouras <spibou@gmail.com> - 2021-03-02 18:51 +0000
                Re: Advice needed on parsing a youtube page Kristjan Robam <he12457@hotmail.com> - 2021-03-02 17:50 -0800
  Re: Advice needed on parsing a youtube page Richard Damon <Richard@Damon-Family.org> - 2021-02-27 14:09 -0500
    Re: Advice needed on parsing a youtube page Spiros Bousbouras <spibou@gmail.com> - 2021-02-27 20:12 +0000
      Re: Advice needed on parsing a youtube page Richard Damon <Richard@Damon-Family.org> - 2021-02-27 22:18 -0500
        Re: Advice needed on parsing a youtube page Spiros Bousbouras <spibou@gmail.com> - 2021-03-02 17:30 +0000

csiph-web