Re: python-based downloader (youtube-dl) missing critical feature ...

Path	csiph.com!x330-a1.tempe.blueboxinc.net!usenet.pasdenom.info!gegeweb.org!de-l.enfer-du-nord.net!feeder1.enfer-du-nord.net!feeder.news-service.com!news2.euro.net!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail
Return-Path	<phihag@phihag.de>
X-Original-To	python-list@python.org
Delivered-To	python-list@mail.python.org
X-Spam-Status	OK 0.005
X-Spam-Evidence	'H': 0.99; 'S': 0.00; 'subject:based': 0.07; 'subject:missing': 0.07; 'python': 0.08; 'pgp': 0.09; 'python- based': 0.09; 'url:github': 0.09; 'subject:python': 0.11; 'message-----': 0.12; 'developer': 0.12; 'cc:addr:python-list': 0.15; '(gnu/linux)': 0.16; '(well,': 0.16; '-----begin': 0.16; ' -----end': 0.16; 'developer)': 0.16; 'do)': 0.16; 'email addr:mail.python.org': 0.16; 'gnupg': 0.16; 'hash:': 0.16; 'none);': 0.16; 'url:issue': 0.16; 'wrote:': 0.18; 'reason,': 0.18; 'sfxlen:2': 0.19; 'cc:no real name:20': 0.21; "doesn't": 0.23; 'wonder': 0.23; 'restrict': 0.23; "shouldn't": 0.23; 'header :In-Reply-To:1': 0.23; 'cc:20': 0.25; 'code': 0.25; 'statement': 0.25; '(in': 0.26; 'code.': 0.26; 'urls': 0.28; 'cc:addr:python.org': 0.29; 'print': 0.29; 'differently': 0.30; 'feature,': 0.30; 'developers': 0.31; 'signed': 0.31; 'implementing': 0.32; 'actual': 0.32; 'implement': 0.32; "can't": 0.32; "isn't": 0.32; 'actually': 0.33; 'there': 0.33; 'header :User-Agent:1': 0.33; 'too': 0.33; 'flag': 0.34; 'maintained': 0.34; 'things': 0.35; 'file': 0.36; '...': 0.36; 'missing': 0.36; 'but': 0.37; 'patch': 0.38; 'some': 0.38; 'received:192': 0.38; 'feed': 0.38; 'version:': 0.38; 'should': 0.39; 'header': 0.39; 'progress': 0.39; 'received:192.168.2': 0.39; 'why': 0.39; 'subject: (': 0.39; 'subject:: ': 0.39; 'under': 0.39; 'might': 0.39; 'data': 0.40; 'providing': 0.60; 'your': 0.61; 'order': 0.62; 'maximum': 0.63; 'piece': 0.65; 'plus': 0.66; 'due': 0.66; 'here.': 0.66; 'sites': 0.66; 'note:': 0.68; 'place?': 0.84; 'sorely': 0.84; 'download,': 0.91; 'to:addr:mail.python.org': 0.91
Date	Fri, 04 Nov 2011 13:45:23 +0100
From	Philipp Hagemeister <phihag@phihag.de>
User-Agent	Mozilla-Thunderbird 2.0.0.24 (X11/20100329)
MIME-Version	1.0
To	lbrt@mail.python.org
Subject	Re: python-based downloader (youtube-dl) missing critical feature ...
References	<1320407282.243739@nntp.aceinnovative.com>
In-Reply-To	<1320407282.243739@nntp.aceinnovative.com>
X-Enigmail-Version	0.95.0
OpenPGP	id=FAFB085C
Content-Type	text/plain; charset=UTF-8
Content-Transfer-Encoding	7bit
Cc	python-list@python.org
X-BeenThere	python-list@python.org
X-Mailman-Version	2.1.12
Precedence	list
List-Id	General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe	<http://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive	<http://mail.python.org/pipermail/python-list>
List-Post	<mailto:python-list@python.org>
List-Help	<mailto:python-list-request@python.org?subject=help>
List-Subscribe	<http://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
Newsgroups	comp.lang.python
Message-ID	<mailman.2432.1320412148.27778.python-list@python.org> (permalink)
Lines	59
NNTP-Posting-Host	2001:888:2000:d::a6
X-Trace	1320412148 news.xs4all.nl 6860 [2001:888:2000:d::a6]:51222
X-Complaints-To	abuse@xs4all.nl
Xref	x330-a1.tempe.blueboxinc.net comp.lang.python:15343

Show key headers only | View raw

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

As already said, you should file your request at
https://github.com/rg3/youtube-dl/issue , not here.

A few things to note:

* Not all sites necessarily send the Content-Length header.
* RTMP URLs would have to be treated differently
* Sending a Range header might allow for a better implementation.

Why do you want to restrict the filesize of the download in the first
place? I can't see a use case, but that doesn't mean there isn't one.

Due to me not having lots of time at the moment, your best chance to get
any youtube-dl feature implemented is providing a patch (in form of a
github pull-request, if possible).

- -- Philipp (youtube-dl developer)

lbrt@mail.python.org wrote:
>  python-based youtube-dl
> ~ 
>  http://rg3.github.com/youtube-dl/
> ~ 
>  is sorely missing a flag in order to indicate the maximum file length of the data feed it would download (well, unless, for some reason, it is considered a "feature").
> ~ 
>  I wonder what developers were thinking about when they came up this nice piece of code. If you actually look in the code
> ~ 
> ...
>  data = urllib2.urlopen(basic_request)
>  content_length = data.info()['Content-Length']
> ...
> ~ 
>  you will see they get the content length of the actual data feed and they also print the progress status based on the content length
> ~ 
>  Implementing an if statement a la:
> ~ 
>  max_indicated_content_length = self.params.get('max_content_length', None);
> ~ 
>  if( content_length > max_indicated_content_length ){ [do not download, just report "data feed too large"] }
>  else{ [do] }
> ~ 
>  shouldn't be hard at all
> ~ 
>  youtube-dl is under the Creative Commons License copyrighted by 2006-2011 Ricardo Garcia Gonzalez and maintained by him and a group of python developers
> ~ 
>  They are the ones keeping a mental map of that project. It would be a plus if they implement this feature, but anyother python developer can implemented (please, let me know if/when you do)
> ~ 
>  lbrtchx

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)

iEYEAREKAAYFAk6z3mMACgkQ9eq1gvr7CFyr1wCgpqf8xuORDC4LBVY8WFmtAufG
k+AAoIX+mXa7SGLULP2M67IQ34sBgk1o
=duyH
-----END PGP SIGNATURE-----

Back to comp.lang.python | Previous | Next — Previous in thread | Find similar | Unroll thread

Thread

python-based downloader (youtube-dl) missing critical feature ... lbrt chx _ gemale - 2011-11-04 11:48 +0000
  Re: python-based downloader (youtube-dl) missing critical feature ... Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2011-11-04 12:32 +0000
    Re: python-based downloader (youtube-dl) missing critical feature ... Jean-Michel Pichavant <jeanmichel@sequans.com> - 2011-11-04 14:31 +0100
    RE: python-based downloader (youtube-dl) missing critical feature ... "Prasad, Ramit" <ramit.prasad@jpmorgan.com> - 2011-11-07 16:34 -0500
  Re: python-based downloader (youtube-dl) missing critical feature ... Philipp Hagemeister <phihag@phihag.de> - 2011-11-04 13:45 +0100

csiph-web