Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #39054 > unrolled thread

pypi changelog api

Back to article view | Back to comp.lang.python

Contents

  pypi changelog api Gregg Caines <cainus@gmail.com> - 2013-02-17 17:16 -0800
    Re: pypi changelog api Philipp Hagemeister <phihag@phihag.de> - 2013-02-21 14:33 +0100
    Re: pypi changelog api Michael Herman <hermanmu@gmail.com> - 2013-02-21 05:52 -0800
    Re: pypi changelog api Michael Herman <hermanmu@gmail.com> - 2013-02-21 05:58 -0800
    Re: pypi changelog api Philipp Hagemeister <phihag@phihag.de> - 2013-02-21 15:04 +0100
    Re: pypi changelog api Philipp Hagemeister <phihag@phihag.de> - 2013-02-21 15:07 +0100
    Re: pypi changelog api Michael Herman <hermanmu@gmail.com> - 2013-02-21 06:15 -0800
    Re: pypi changelog api Michael Herman <hermanmu@gmail.com> - 2013-02-21 06:19 -0800

#39054 — pypi changelog api

Hey all,

I'm trying to write a script that tracks changes on pypi, and I've come across the xmlrpc interface, specifically the 'changelog' api.  It's definitely what I'm looking for, but I get an absolutely massive xml response from it at once and I was hoping there might be either some way to "page" through it with mulitple requests, or a different-but-similar API.

Thanks in advance,
Gregg

[toc] | [next] | [standalone]

#39423

[Multipart message — attachments visible in raw view] — view raw

Hi Gregg,

to get a smaller response, you can simply pass in a timestamp, like this:

>>> client = xmlrpclib.ServerProxy('http://pypi.python.org/pypi')
>>> import time
>>> client.changelog(int(time.time() - 600))
[['vs.bootstrap.plonetheme', '1.0.1', 1361451748, 'update description,
classifiers'], ['numericalunits', '1.11', 1361451759, 'new release'],
['numericalunits', '1.11', 1361451760, 'add source file
numericalunits-1.11.tar.gz'], ['autobahn_rce', '0.6.0', 1361451798,
'remove'], ['vs.bootstrap.plonetheme', '1.0.1', 1361451816, 'update
description, classifiers'], ['vs.bootstrap.plonetheme', '1.0.1',
1361451882, 'update description, classifiers'], ['autobahn_rce',
'0.5.9', 1361451956, 'new release'], ['autobahn_rce', '0.5.9',
1361451971, 'add source file autobahn_rce-0.5.9.tar.gz']]

I don't think there's way to get older chunks of the changelog though.
What would you need those for?

If you need the entire changelog, just download it once (it's not that
large, barely 40MB). Here it is, up until 1361452402 (now):

http://phihag.de/2013/pypi-changelog-2013-02-20.json.bz2

What I'd like is a real-time push service of changelog entries, but I'm
not certain that would be scalable.

Cheers,

Philipp


On 02/18/2013 02:16 AM, Gregg Caines wrote:
> Hey all,
> 
> I'm trying to write a script that tracks changes on pypi, and I've come across the xmlrpc interface, specifically the 'changelog' api.  It's definitely what I'm looking for, but I get an absolutely massive xml response from it at once and I was hoping there might be either some way to "page" through it with mulitple requests, or a different-but-similar API.
> 
> Thanks in advance,
> Gregg

[toc] | [prev] | [next] | [standalone]

#39424

[Multipart message — attachments visible in raw view] — view raw

I'd love to see https://crate.io/ set up an API or at the very least an RSS
feed for tracking changes. I've emailed the author about this. I think if
enough people do, an RSS feed would be easy to setup.


On Thu, Feb 21, 2013 at 5:33 AM, Philipp Hagemeister <phihag@phihag.de>wrote:

> Hi Gregg,
>
> to get a smaller response, you can simply pass in a timestamp, like this:
>
> >>> client = xmlrpclib.ServerProxy('http://pypi.python.org/pypi')
> >>> import time
> >>> client.changelog(int(time.time() - 600))
> [['vs.bootstrap.plonetheme', '1.0.1', 1361451748, 'update description,
> classifiers'], ['numericalunits', '1.11', 1361451759, 'new release'],
> ['numericalunits', '1.11', 1361451760, 'add source file
> numericalunits-1.11.tar.gz'], ['autobahn_rce', '0.6.0', 1361451798,
> 'remove'], ['vs.bootstrap.plonetheme', '1.0.1', 1361451816, 'update
> description, classifiers'], ['vs.bootstrap.plonetheme', '1.0.1',
> 1361451882, 'update description, classifiers'], ['autobahn_rce',
> '0.5.9', 1361451956, 'new release'], ['autobahn_rce', '0.5.9',
> 1361451971, 'add source file autobahn_rce-0.5.9.tar.gz']]
>
> I don't think there's way to get older chunks of the changelog though.
> What would you need those for?
>
> If you need the entire changelog, just download it once (it's not that
> large, barely 40MB). Here it is, up until 1361452402 (now):
>
> http://phihag.de/2013/pypi-changelog-2013-02-20.json.bz2
>
> What I'd like is a real-time push service of changelog entries, but I'm
> not certain that would be scalable.
>
> Cheers,
>
> Philipp
>
>
> On 02/18/2013 02:16 AM, Gregg Caines wrote:
> > Hey all,
> >
> > I'm trying to write a script that tracks changes on pypi, and I've come
> across the xmlrpc interface, specifically the 'changelog' api.  It's
> definitely what I'm looking for, but I get an absolutely massive xml
> response from it at once and I was hoping there might be either some way to
> "page" through it with mulitple requests, or a different-but-similar API.
> >
> > Thanks in advance,
> > Gregg
>
>
>
> --
> http://mail.python.org/mailman/listinfo/python-list
>
>

[toc] | [prev] | [next] | [standalone]

#39425

[Multipart message — attachments visible in raw view] — view raw

Oh - and I haven't tried this site, but you may be able to set something up
on there to email when the changelog is updated.

http://www.changedetection.com/

On Thu, Feb 21, 2013 at 5:52 AM, Michael Herman <hermanmu@gmail.com> wrote:

> I'd love to see https://crate.io/ set up an API or at the very least an
> RSS feed for tracking changes. I've emailed the author about this. I think
> if enough people do, an RSS feed would be easy to setup.
>
>
> On Thu, Feb 21, 2013 at 5:33 AM, Philipp Hagemeister <phihag@phihag.de>wrote:
>
>> Hi Gregg,
>>
>> to get a smaller response, you can simply pass in a timestamp, like this:
>>
>> >>> client = xmlrpclib.ServerProxy('http://pypi.python.org/pypi')
>> >>> import time
>> >>> client.changelog(int(time.time() - 600))
>> [['vs.bootstrap.plonetheme', '1.0.1', 1361451748, 'update description,
>> classifiers'], ['numericalunits', '1.11', 1361451759, 'new release'],
>> ['numericalunits', '1.11', 1361451760, 'add source file
>> numericalunits-1.11.tar.gz'], ['autobahn_rce', '0.6.0', 1361451798,
>> 'remove'], ['vs.bootstrap.plonetheme', '1.0.1', 1361451816, 'update
>> description, classifiers'], ['vs.bootstrap.plonetheme', '1.0.1',
>> 1361451882, 'update description, classifiers'], ['autobahn_rce',
>> '0.5.9', 1361451956, 'new release'], ['autobahn_rce', '0.5.9',
>> 1361451971, 'add source file autobahn_rce-0.5.9.tar.gz']]
>>
>> I don't think there's way to get older chunks of the changelog though.
>> What would you need those for?
>>
>> If you need the entire changelog, just download it once (it's not that
>> large, barely 40MB). Here it is, up until 1361452402 (now):
>>
>> http://phihag.de/2013/pypi-changelog-2013-02-20.json.bz2
>>
>> What I'd like is a real-time push service of changelog entries, but I'm
>> not certain that would be scalable.
>>
>> Cheers,
>>
>> Philipp
>>
>>
>> On 02/18/2013 02:16 AM, Gregg Caines wrote:
>> > Hey all,
>> >
>> > I'm trying to write a script that tracks changes on pypi, and I've come
>> across the xmlrpc interface, specifically the 'changelog' api.  It's
>> definitely what I'm looking for, but I get an absolutely massive xml
>> response from it at once and I was hoping there might be either some way to
>> "page" through it with mulitple requests, or a different-but-similar API.
>> >
>> > Thanks in advance,
>> > Gregg
>>
>>
>>
>> --
>> http://mail.python.org/mailman/listinfo/python-list
>>
>>
>

[toc] | [prev] | [next] | [standalone]

#39426

[Multipart message — attachments visible in raw view] — view raw

Just FYI, PyPi already has an RSS feed:
https://pypi.python.org/pypi?:action=rss

And instead of mailing the author, you should probably open an issue (or
better yet, a pull request) at https://github.com/crateio/crate.web/issues

For that matter - how does crate.io update their PyPi mirror?

Cheers,

Philipp


On 02/21/2013 02:52 PM, Michael Herman wrote:
> I'd love to see https://crate.io/ set up an API or at the very least an RSS
> feed for tracking changes. I've emailed the author about this. I think if
> enough people do, an RSS feed would be easy to setup.
> 
> 
> On Thu, Feb 21, 2013 at 5:33 AM, Philipp Hagemeister <phihag@phihag.de>wrote:
> 
>> Hi Gregg,
>>
>> to get a smaller response, you can simply pass in a timestamp, like this:
>>
>>>>> client = xmlrpclib.ServerProxy('http://pypi.python.org/pypi')
>>>>> import time
>>>>> client.changelog(int(time.time() - 600))
>> [['vs.bootstrap.plonetheme', '1.0.1', 1361451748, 'update description,
>> classifiers'], ['numericalunits', '1.11', 1361451759, 'new release'],
>> ['numericalunits', '1.11', 1361451760, 'add source file
>> numericalunits-1.11.tar.gz'], ['autobahn_rce', '0.6.0', 1361451798,
>> 'remove'], ['vs.bootstrap.plonetheme', '1.0.1', 1361451816, 'update
>> description, classifiers'], ['vs.bootstrap.plonetheme', '1.0.1',
>> 1361451882, 'update description, classifiers'], ['autobahn_rce',
>> '0.5.9', 1361451956, 'new release'], ['autobahn_rce', '0.5.9',
>> 1361451971, 'add source file autobahn_rce-0.5.9.tar.gz']]
>>
>> I don't think there's way to get older chunks of the changelog though.
>> What would you need those for?
>>
>> If you need the entire changelog, just download it once (it's not that
>> large, barely 40MB). Here it is, up until 1361452402 (now):
>>
>> http://phihag.de/2013/pypi-changelog-2013-02-20.json.bz2
>>
>> What I'd like is a real-time push service of changelog entries, but I'm
>> not certain that would be scalable.
>>
>> Cheers,
>>
>> Philipp
>>
>>
>> On 02/18/2013 02:16 AM, Gregg Caines wrote:
>>> Hey all,
>>>
>>> I'm trying to write a script that tracks changes on pypi, and I've come
>> across the xmlrpc interface, specifically the 'changelog' api.  It's
>> definitely what I'm looking for, but I get an absolutely massive xml
>> response from it at once and I was hoping there might be either some way to
>> "page" through it with mulitple requests, or a different-but-similar API.
>>>
>>> Thanks in advance,
>>> Gregg
>>
>>
>>
>> --
>> http://mail.python.org/mailman/listinfo/python-list
>>
>>
> 

[toc] | [prev] | [next] | [standalone]

#39427

[Multipart message — attachments visible in raw view] — view raw

On 02/21/2013 02:58 PM, Michael Herman wrote:
> Oh - and I haven't tried this site, but you may be able to set something up
> on there to email when the changelog is updated.
> 
> http://www.changedetection.com/
They just query the whole page - I could do that myself, easily. But the
problem is that that would mean lots of load on PyPi (if you query every
minute or so) and outdated packages (if you query less often than that).
Keeping a connection that PyPi would push to seems to be much cleaner.

- Philipp

[toc] | [prev] | [next] | [standalone]

#39429

[Multipart message — attachments visible in raw view] — view raw

totally agree. if you just had a listener setup you could just have a
simple trigger-action setup.

On Thu, Feb 21, 2013 at 6:07 AM, Philipp Hagemeister <phihag@phihag.de>wrote:

> On 02/21/2013 02:58 PM, Michael Herman wrote:
> > Oh - and I haven't tried this site, but you may be able to set something
> up
> > on there to email when the changelog is updated.
> >
> > http://www.changedetection.com/
> They just query the whole page - I could do that myself, easily. But the
> problem is that that would mean lots of load on PyPi (if you query every
> minute or so) and outdated packages (if you query less often than that).
> Keeping a connection that PyPi would push to seems to be much cleaner.
>
> - Philipp
>
>

[toc] | [prev] | [next] | [standalone]

#39430

[Multipart message — attachments visible in raw view] — view raw

yeah, i just checked github, i thought a buddy of mine had submitted an
issue

not sure how they get their updates.

this is an interesting project - https://github.com/kencochrane/pypi-mirrors

On Thu, Feb 21, 2013 at 6:04 AM, Philipp Hagemeister <phihag@phihag.de>wrote:

> Just FYI, PyPi already has an RSS feed:
> https://pypi.python.org/pypi?:action=rss
>
> And instead of mailing the author, you should probably open an issue (or
> better yet, a pull request) at https://github.com/crateio/crate.web/issues
>
> For that matter - how does crate.io update their PyPi mirror?
>
> Cheers,
>
> Philipp
>
>
> On 02/21/2013 02:52 PM, Michael Herman wrote:
> > I'd love to see https://crate.io/ set up an API or at the very least an
> RSS
> > feed for tracking changes. I've emailed the author about this. I think if
> > enough people do, an RSS feed would be easy to setup.
> >
> >
> > On Thu, Feb 21, 2013 at 5:33 AM, Philipp Hagemeister <phihag@phihag.de
> >wrote:
> >
> >> Hi Gregg,
> >>
> >> to get a smaller response, you can simply pass in a timestamp, like
> this:
> >>
> >>>>> client = xmlrpclib.ServerProxy('http://pypi.python.org/pypi')
> >>>>> import time
> >>>>> client.changelog(int(time.time() - 600))
> >> [['vs.bootstrap.plonetheme', '1.0.1', 1361451748, 'update description,
> >> classifiers'], ['numericalunits', '1.11', 1361451759, 'new release'],
> >> ['numericalunits', '1.11', 1361451760, 'add source file
> >> numericalunits-1.11.tar.gz'], ['autobahn_rce', '0.6.0', 1361451798,
> >> 'remove'], ['vs.bootstrap.plonetheme', '1.0.1', 1361451816, 'update
> >> description, classifiers'], ['vs.bootstrap.plonetheme', '1.0.1',
> >> 1361451882, 'update description, classifiers'], ['autobahn_rce',
> >> '0.5.9', 1361451956, 'new release'], ['autobahn_rce', '0.5.9',
> >> 1361451971, 'add source file autobahn_rce-0.5.9.tar.gz']]
> >>
> >> I don't think there's way to get older chunks of the changelog though.
> >> What would you need those for?
> >>
> >> If you need the entire changelog, just download it once (it's not that
> >> large, barely 40MB). Here it is, up until 1361452402 (now):
> >>
> >> http://phihag.de/2013/pypi-changelog-2013-02-20.json.bz2
> >>
> >> What I'd like is a real-time push service of changelog entries, but I'm
> >> not certain that would be scalable.
> >>
> >> Cheers,
> >>
> >> Philipp
> >>
> >>
> >> On 02/18/2013 02:16 AM, Gregg Caines wrote:
> >>> Hey all,
> >>>
> >>> I'm trying to write a script that tracks changes on pypi, and I've come
> >> across the xmlrpc interface, specifically the 'changelog' api.  It's
> >> definitely what I'm looking for, but I get an absolutely massive xml
> >> response from it at once and I was hoping there might be either some
> way to
> >> "page" through it with mulitple requests, or a different-but-similar
> API.
> >>>
> >>> Thanks in advance,
> >>> Gregg
> >>
> >>
> >>
> >> --
> >> http://mail.python.org/mailman/listinfo/python-list
> >>
> >>
> >
>
>
>

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web