X-Received: by 10.236.19.7 with SMTP id m7mr12287yhm.35.1403119474553; Wed, 18 Jun 2014 12:24:34 -0700 (PDT)
X-Received: by 10.182.108.165 with SMTP id hl5mr757obb.27.1403119474411; Wed, 18 Jun 2014 12:24:34 -0700 (PDT)
Path: csiph.com!v102.xanadu-bbs.net!xanadu-bbs.net!news.glorb.com!peer02.iad.highwinds-media.com!news.highwinds-media.com!feed-me.highwinds-media.com!i13no5732015qae.1!news-out.google.com!q9ni11542qaj.0!nntp.google.com!w8no5986310qac.0!postnews.google.com!glegroupsg2000goo.googlegroups.com!not-for-mail
Newsgroups: comp.lang.python
Date: Wed, 18 Jun 2014 12:24:34 -0700 (PDT)
In-Reply-To: <f3b8fccb-aecd-4ae0-8be6-ce4a8e4317a1@googlegroups.com>
Complaints-To: groups-abuse@google.com
Injection-Info: glegroupsg2000goo.googlegroups.com; posting-host=82.23.60.95; posting-account=4-e4QgoAAAAWHVCQEEV9f2rh1zdmF7RH
NNTP-Posting-Host: 82.23.60.95
References: <f3b8fccb-aecd-4ae0-8be6-ce4a8e4317a1@googlegroups.com>
User-Agent: G2/1.0
MIME-Version: 1.0
Message-ID: <3e4dadd0-79d5-4c19-9a8e-04004f62fa63@googlegroups.com>
Subject: Re: pypi - package metdata
From: Maciej Dziardziel <fiedzia@gmail.com>
Injection-Date: Wed, 18 Jun 2014 19:24:34 +0000
Content-Type: text/plain; charset=ISO-8859-1
X-Received-Bytes: 2025
X-Received-Body-CRC: 2934060234
Xref: csiph.com comp.lang.python:73375

On Wednesday, June 18, 2014 8:21:12 PM UTC+1, Maciej Dziardziel wrote:
> I wasn't quiet happy with the way search on pypi works,
> 
> so I've got an idea of getting all package metadata from pypi
> 
> and do search locally. The only problem is that I can't figure out
> 
> where to get the data from. I tried to use bandersnatch to 
> 
> set up mirror, but all I've got was a mixture of egg, whl and exe files.
> 
> Egg and whl are useful maybe, but I guess its not exactly what I want.
> 
> Then I've found api:
> 
> https://pypi.python.org/simple/ provides list of packages,
> 
> and I can use json api to get metadata for each of them.


(Click post to fast, so I am continuing.)

So its doable, but I am really hoping for a better way.
I assume pypi would ban me quickly for making thousands of requests,
and I'd love to be able to update index daily or weekly.

Is there something I am missing?