Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.os.linux.advocacy > #473725

Re: Need Help: Get Links from Javashit Web Page

From owl <owl@rooftop.invalid>
Newsgroups comp.os.linux.advocacy
Subject Re: Need Help: Get Links from Javashit Web Page
Date 2018-10-26 14:34 +0000
Organization O.W.L.
Message-ID <sz0xbz09bz.w@rooftop.invalid> (permalink)
References <b1fc5d6a-51e4-4b45-bbbb-d4298d3d8c46@googlegroups.com> <11s9z0b893ag.ah8@rooftop.invalid> <c9c8f724-fdb4-4eaa-af14-1b506ec551d7@googlegroups.com>

Show all headers | View raw

fr314159@gmail.com wrote:
> On Friday, October 26, 2018 at 9:17:15 AM UTC-4, owl wrote:
> 
>> 
>> #!/bin/bash
>> lynx -listonly -nonumbers -dump http://web.mit.edu/hml/ncfmf.html \
>> |grep techtv >lout
>>
> 
> That not going to do any good.
> 
> Each link in the parent page (web.mit.edu/hml/ncfmf.html) is to
> a page that contains nothing but a few lines of javascript.  When
> this javascript is executed the result is a link to a YouTube
> video.
> 
> I need to get the YouTube link directly without loading each
> page.  This may not be possible.
> 
 
This one goes to a youtube vid:
https://techtv.mit.edu/collections/ifluids/videos/32602-fluid-dynamics-of-drag-part-3

If you click right click and "copy video URL"
it gives this:

https://youtu.be/cqxT97GDl3w

But the ones that don't go to youtube go URLs like this:

https://d3tsb3m56iwvoq.cloudfront.net/transcoded/techtv/{videokey}/hd.mp4

However, if you take the "videoKey" for the one above that goes to youtube:
 
cd284802f88a4e30b0a34fd9932ad8cc

you can put it in the previous URL format and download it here:

https://d3tsb3m56iwvoq.cloudfront.net/transcoded/techtv/cd284802f88a4e30b0a34fd9932ad8cc/hd.mp4

So I would try code with wget or curl on all the links, extract the
videoKey, and then assemble it into this URL type:

https://d3tsb3m56iwvoq.cloudfront.net/transcoded/techtv/{videokey}/hd.mp4 

Try this (takes about 15 secs to run for me):

----------------------
#!/bin/bash
for i in $(for i in $(lynx -listonly -nonumbers -dump \
http://web.mit.edu/hml/ncfmf.html |grep techtv);do wget -q -O - ${i} \
|grep videoKey |cut -f15 -d:\
|cut -f1 -d,|sed 's/ //g'|sed 's/"//g'; done); do echo \
"https://d3tsb3m56iwvoq.cloudfront.net/transcoded/techtv/${i}/hd.mp4"; done
----------------------

That gives a list of URLs.
They may or may not work.

Back to comp.os.linux.advocacy | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Need Help: Get Links from Javashit Web Page fr314159@gmail.com - 2018-10-26 05:49 -0700
  Re: Need Help: Get Links from Javashit Web Page owl <owl@rooftop.invalid> - 2018-10-26 13:17 +0000
    Re: Need Help: Get Links from Javashit Web Page Steve Carroll <frelwizzen@gmail.com> - 2018-10-26 06:25 -0700
      Re: Need Help: Get Links from Javashit Web Page Steve Carroll <frelwizzen@gmail.com> - 2018-10-26 07:29 -0700
    Re: Need Help: Get Links from Javashit Web Page fr314159@gmail.com - 2018-10-26 06:32 -0700
      Re: Need Help: Get Links from Javashit Web Page Steve Carroll <frelwizzen@gmail.com> - 2018-10-26 07:13 -0700
      Re: Need Help: Get Links from Javashit Web Page owl <owl@rooftop.invalid> - 2018-10-26 14:34 +0000
        Re: Need Help: Get Links from Javashit Web Page fr314159@gmail.com - 2018-10-26 07:44 -0700
          Re: Need Help: Get Links from Javashit Web Page owl <owl@rooftop.invalid> - 2018-10-26 14:50 +0000
            Re: Need Help: Get Links from Javashit Web Page fr314159@gmail.com - 2018-10-26 08:12 -0700
              Re: Need Help: Get Links from Javashit Web Page DFS <nospam@nospam.com> - 2018-10-26 11:24 -0400
                Re: Need Help: Get Links from Javashit Web Page fr314159@gmail.com - 2018-10-26 08:32 -0700
                Re: Need Help: Get Links from Javashit Web Page DFS <nospam@nospam.com> - 2018-10-26 11:46 -0400
                Re: Need Help: Get Links from Javashit Web Page fr314159@gmail.com - 2018-10-26 09:17 -0700
                Re: Need Help: Get Links from Javashit Web Page DFS <nospam@nospam.com> - 2018-10-26 13:08 -0400
                Re: Need Help: Get Links from Javashit Web Page Steve Carroll <frelwizzen@gmail.com> - 2018-10-26 10:35 -0700
                Re: Need Help: Get Links from Javashit Web Page fr314159@gmail.com - 2018-10-26 12:35 -0700
                Re: Need Help: Get Links from Javashit Web Page Steve Carroll <frelwizzen@gmail.com> - 2018-10-26 12:54 -0700
                Re: Need Help: Get Links from Javashit Web Page Snit <usenet@gallopinginsanity.com> - 2018-10-26 13:12 -0700
                Re: Need Help: Get Links from Javashit Web Page DFS <nospam@nospam.com> - 2018-10-26 20:11 -0400
                Re: Need Help: Get Links from Javashit Web Page "F. Russell" <fr@random.info> - 2018-10-27 00:36 +0000
                Re: Need Help: Get Links from Javashit Web Page Steve Carroll <frelwizzen@gmail.com> - 2018-10-26 18:09 -0700
                Re: Need Help: Get Links from Javashit Web Page SilverSlimer <silver@slim.er> - 2018-10-27 11:52 -0400
                Re: Need Help: Get Links from Javashit Web Page Steve Carroll <frelwizzen@gmail.com> - 2018-10-27 09:12 -0700
                Re: Need Help: Get Links from Javashit Web Page owl <owl@rooftop.invalid> - 2018-10-27 18:35 +0000
                Re: Need Help: Get Links from Javashit Web Page Steve Carroll <frelwizzen@gmail.com> - 2018-10-27 12:04 -0700
                Re: Need Help: Get Links from Javashit Web Page Steve Carroll <frelwizzen@gmail.com> - 2018-10-26 17:38 -0700
                Re: Need Help: Get Links from Javashit Web Page SilverSlimer <silver@slim.er> - 2018-10-27 11:43 -0400
                Re: Need Help: Get Links from Javashit Web Page "F. Russell" <fr@random.info> - 2018-10-27 16:24 +0000
                Re: Need Help: Get Links from Javashit Web Page Melzzzzz <Melzzzzz@zzzzz.com> - 2018-10-27 17:12 +0000
                Re: Need Help: Get Links from Javashit Web Page Steve Carroll <frelwizzen@gmail.com> - 2018-10-27 10:31 -0700
                Re: Need Help: Get Links from Javashit Web Page Steve Carroll <frelwizzen@gmail.com> - 2018-10-27 10:15 -0700
                Re: Need Help: Get Links from Javashit Web Page Snit <usenet@gallopinginsanity.com> - 2018-10-27 11:52 -0700
                Re: Need Help: Get Links from Javashit Web Page Steve Carroll <frelwizzen@gmail.com> - 2018-10-27 12:20 -0700
                Windows is cheating. Jeff-Relf.Me  @. - 2018-10-30 07:53 -0700
                Re: Windows is cheating. DFS <nospam@nospam.com> - 2018-10-30 12:51 -0400
                Re: Windows is cheating. Steve Carroll <frelwizzen@gmail.com> - 2018-10-30 10:23 -0700
              Re: Need Help: Get Links from Javashit Web Page Steve Carroll <fretwizzer@gmail.com> - 2018-10-26 10:24 -0700
                Re: Need Help: Get Links from Javashit Web Page Steve Carroll <frelwizzen@gmail.com> - 2018-10-26 10:51 -0700
              fr31 is both delusional and psychotic! Simon bar Sinister <john.h.gohde@gmail.com> - 2018-10-27 06:20 -0700
            Re: Need Help: Get Links from Javashit Web Page "F. Russell" <fr@random.info> - 2018-10-27 00:24 +0000
              Re: Need Help: Get Links from Javashit Web Page owl <owl@rooftop.invalid> - 2018-10-27 01:45 +0000
                Re: Need Help: Get Links from Javashit Web Page Steve Carroll <frelwizzen@gmail.com> - 2018-10-26 18:56 -0700
                Re: Need Help: Get Links from Javashit Web Page Steve Carroll <frelwizzen@gmail.com> - 2018-10-26 19:12 -0700
                Re: Need Help: Get Links from Javashit Web Page Steve Carroll <frelwizzen@gmail.com> - 2018-10-26 20:45 -0700
              Re: Need Help: Get Links from Javashit Web Page DFS <nospam@nospam.com> - 2018-10-26 22:55 -0400
                Re: Need Help: Get Links from Javashit Web Page Steve Carroll <frelwizzen@gmail.com> - 2018-10-26 20:14 -0700
                Re: Need Help: Get Links from Javashit Web Page "F. Russell" <fr@random.info> - 2018-10-27 13:57 +0000
                Re: Need Help: Get Links from Javashit Web Page DFS <nospam@nospam.com> - 2018-10-27 12:03 -0400
              Re: Need Help: Get Links from Javashit Web Page DFS <nospam@nospam.com> - 2018-10-27 12:20 -0400
      Clearly, fr31 is a Lazy Turd. Simon bar Sinister <john.h.gohde@gmail.com> - 2018-10-27 06:19 -0700
        Re: Clearly, fr31 is a Lazy Turd. Steve Carroll <frelwizzen@gmail.com> - 2018-10-27 06:37 -0700
          Re: Clearly, fr31 is a Lazy Turd. Steve Carroll <frelwizzen@gmail.com> - 2018-10-27 08:10 -0700
    Re: Need Help: Get Links from Javashit Web Page Steve Carroll <fretwizzer@gmail.com> - 2018-10-26 10:54 -0700
      Re: Need Help: Get Links from Javashit Web Page owl <owl@rooftop.invalid> - 2018-10-26 20:06 +0000
        Re: Need Help: Get Links from Javashit Web Page Steve Carroll <frelwizzen@gmail.com> - 2018-10-26 13:25 -0700
        Re: Need Help: Get Links from Javashit Web Page Steve Carroll <fretwizzer@gmail.com> - 2018-10-26 13:27 -0700
          Re: Need Help: Get Links from Javashit Web Page Steve Carroll <frelwizzen@gmail.com> - 2018-10-26 13:41 -0700
          Re: Need Help: Get Links from Javashit Web Page Steve Carroll <frelwizzen@gmail.com> - 2018-10-26 13:57 -0700
  Re: Need Help: Get Links from Javashit Web Page DFS <nospam@nospam.com> - 2018-10-26 09:28 -0400
    Re: Need Help: Get Links from Javashit Web Page fr314159@gmail.com - 2018-10-26 06:42 -0700
      Re: Need Help: Get Links from Javashit Web Page DFS <nospam@nospam.com> - 2018-10-26 10:12 -0400
        Re: Need Help: Get Links from Javashit Web Page fr314159@gmail.com - 2018-10-26 07:20 -0700
          Re: Need Help: Get Links from Javashit Web Page DFS <nospam@nospam.com> - 2018-10-26 10:27 -0400
            Re: Need Help: Get Links from Javashit Web Page fr314159@gmail.com - 2018-10-26 07:41 -0700
            Re: Need Help: Get Links from Javashit Web Page SilverSlimer <silver@slim.er> - 2018-10-26 12:09 -0400
        Re: Need Help: Get Links from Javashit Web Page Snit <usenet@gallopinginsanity.com> - 2018-10-26 07:21 -0700
        Re: Need Help: Get Links from Javashit Web Page Snit <usenet@gallopinginsanity.com> - 2018-10-26 07:54 -0700
      Re: Need Help: Get Links from Javashit Web Page Hadron Quark <hadronquark@gmail.com> - 2018-10-28 03:28 +0100
        Re: Need Help: Get Links from Javashit Web Page Steve Carroll <frelwizzen@gmail.com> - 2018-10-27 20:56 -0700
        Re: Need Help: Get Links from Javashit Web Page Steve Carroll <fretwizzer@gmail.com> - 2018-10-29 09:46 -0700
          Re: Need Help: Get Links from Javashit Web Page Steve Carroll <frelwizzen@gmail.com> - 2018-10-29 10:15 -0700
          Re: Need Help: Get Links from Javashit Web Page fr314159@gmail.com - 2018-10-29 10:37 -0700
            Re: Need Help: Get Links from Javashit Web Page Steve Carroll <fretwizzer@gmail.com> - 2018-10-30 08:09 -0700
          Re: Need Help: Get Links from Javashit Web Page Snit <usenet@gallopinginsanity.com> - 2018-10-29 15:25 -0700
            Re: Need Help: Get Links from Javashit Web Page Steve Carroll <frelwizzen@gmail.com> - 2018-10-29 15:39 -0700
  Re: Need Help: Get Links from Javashit Web Page Steve Carroll <frelwizzen@gmail.com> - 2018-10-26 06:41 -0700
  Re: Need Help: Get Links from Javashit Web Page Steve Carroll <frelwizzen@gmail.com> - 2018-10-26 07:44 -0700
  Re: Need Help: Get Links from Javashit Web Page Steve Carroll <frelwizzen@gmail.com> - 2018-10-26 08:00 -0700
  Re: Need Help: Get Links from Javashit Web Page Steve Carroll <frelwizzen@gmail.com> - 2018-10-26 08:15 -0700
  Re: Need Help: Get Links from Javashit Web Page Steve Carroll <frelwizzen@gmail.com> - 2018-10-26 08:31 -0700

csiph-web