Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #111175 > unrolled thread

Packaging multiple wheels in the same package

Back to article view | Back to comp.lang.python

Contents

  Packaging multiple wheels in the same package Nir Cohen <nir36g@gmail.com> - 2016-07-06 11:43 -0700
    Re: Packaging multiple wheels in the same package Ethan Furman <ethan@stoneleaf.us> - 2016-07-06 12:08 -0700
      Re: Packaging multiple wheels in the same package Nir Cohen <nir36g@gmail.com> - 2016-07-06 21:47 -0700
        Re: Packaging multiple wheels in the same package Nir Cohen <nir36g@gmail.com> - 2016-07-13 05:54 -0700
          Re: Packaging multiple wheels in the same package Chris Angelico <rosuav@gmail.com> - 2016-07-13 23:05 +1000
          Re: Packaging multiple wheels in the same package Ethan Furman <ethan@stoneleaf.us> - 2016-07-13 08:01 -0700
            Re: Packaging multiple wheels in the same package Nir Cohen <nir36g@gmail.com> - 2016-07-14 12:40 -0700

#111175 — Packaging multiple wheels in the same package

Hey all,

As part of working on Cloudify (http://github.com/cloudify-cosmo) we've had to provide a way for customers to install our plugins in an environment where PyPI isn't accessible. These plugins are sets of Python packages which necessarily depend on one another (i.e. a regular python package with dependencies).

We decided that we want to package sets of wheels together created or downloaded by `pip wheel`, add relevant metadata, package them together into a single archive (tar.gz or zip) and use the same tool which packs them up to install them later on, on the destination hosts.

We came up with a tool (http://github.com/cloudify-cosmo/wagon) to do just that and that's what we currently use to create and install our plugins.

While wheel solves the problem of generating wheels, there is no single, standard method for taking an entire set of dependencies packaged in a single location and installing them in a different location.

We thought it would be a good idea to propose a PEP for that and wanted to get your feedback before we start writing the proposal.

The logic behind the PEP is that there's a standard for creating a single python package and we would like to propose a layer above it to package multiple python packages either for offline environments or any other requirement a user may have.

We're currently working on a new rearchitected version of that tool to make it more pythonic and clean but the base stays the same.


We would greatly appreciate your feedback on this.

Thanks!


P.S. We encourage you to take a look at the repo (see the `rearchitecture` branch) and read the README.

[toc] | [next] | [standalone]

#111176

On 07/06/2016 11:43 AM, Nir Cohen wrote:

> We decided that we want to package sets of wheels together created or downloaded
 > by `pip wheel`, add relevant metadata, package them together into a 
single archive
 > (tar.gz or zip) and use the same tool which packs them up to install 
them later on,
 > on the destination hosts.
>
> We came up with a tool (http://github.com/cloudify-cosmo/wagon) to do just that
 > and that's what we currently use to create and install our plugins.

Sounds like a great idea!

Once you have your feed-back from here you'll want to take your PEP over 
to the dist-utils sig:

    https://mail.python.org/mailman/listinfo/distutils-sig

--
~Ethan~

[toc] | [prev] | [next] | [standalone]

#111182

On Wednesday, July 6, 2016 at 10:09:01 PM UTC+3, Ethan Furman wrote:
> On 07/06/2016 11:43 AM, Nir Cohen wrote:
> 
> > We decided that we want to package sets of wheels together created or downloaded
>  > by `pip wheel`, add relevant metadata, package them together into a 
> single archive
>  > (tar.gz or zip) and use the same tool which packs them up to install 
> them later on,
>  > on the destination hosts.
> >
> > We came up with a tool (http://github.com/cloudify-cosmo/wagon) to do just that
>  > and that's what we currently use to create and install our plugins.
> 
> Sounds like a great idea!
> 
> Once you have your feed-back from here you'll want to take your PEP over 
> to the dist-utils sig:
> 
>     https://mail.python.org/mailman/listinfo/distutils-sig
> 
> --
> ~Ethan~

Happy to hear :)

btw, pull requests are certainly welcome to make whichever changes required to make Wagon PEP-worthy.

[toc] | [prev] | [next] | [standalone]

#111391

On Thursday, July 7, 2016 at 7:47:22 AM UTC+3, Nir Cohen wrote:
> On Wednesday, July 6, 2016 at 10:09:01 PM UTC+3, Ethan Furman wrote:
> > On 07/06/2016 11:43 AM, Nir Cohen wrote:
> > 
> > > We decided that we want to package sets of wheels together created or downloaded
> >  > by `pip wheel`, add relevant metadata, package them together into a 
> > single archive
> >  > (tar.gz or zip) and use the same tool which packs them up to install 
> > them later on,
> >  > on the destination hosts.
> > >
> > > We came up with a tool (http://github.com/cloudify-cosmo/wagon) to do just that
> >  > and that's what we currently use to create and install our plugins.
> > 
> > Sounds like a great idea!
> > 
> > Once you have your feed-back from here you'll want to take your PEP over 
> > to the dist-utils sig:
> > 
> >     https://mail.python.org/mailman/listinfo/distutils-sig
> > 
> > --
> > ~Ethan~
> 
> Happy to hear :)
> 
> btw, pull requests are certainly welcome to make whichever changes required to make Wagon PEP-worthy.


Sorry to bump this up.. but I'm not sure exactly how I should go about getting as much feedback as possible on this. Any suggestions?

[toc] | [prev] | [next] | [standalone]

#111396

On Wed, Jul 13, 2016 at 10:54 PM, Nir Cohen <nir36g@gmail.com> wrote:
> On Thursday, July 7, 2016 at 7:47:22 AM UTC+3, Nir Cohen wrote:
>> On Wednesday, July 6, 2016 at 10:09:01 PM UTC+3, Ethan Furman wrote:
>> > Sounds like a great idea!
>> >
>> > Once you have your feed-back from here you'll want to take your PEP over
>> > to the dist-utils sig:
>> >
>> >     https://mail.python.org/mailman/listinfo/distutils-sig
>> >
>> > --
>> > ~Ethan~
>>
>> Happy to hear :)
>>
>> btw, pull requests are certainly welcome to make whichever changes required to make Wagon PEP-worthy.
>
> Sorry to bump this up.. but I'm not sure exactly how I should go about getting as much feedback as possible on this. Any suggestions?

Well.... a lot of the people here might *use* wheels, but a relatively
small proportion *create* them. (I'm not big on creating packages -
the only PyPI-published module of mine is a tiny pure-Python one, so
it's about as simple as it gets. And it's not even actively maintained
any more.) Ethan suggested hopping over to distutils-sig - have you
done that? That would be the next step, if you haven't.

ChrisA

[toc] | [prev] | [next] | [standalone]

#111403

On 07/13/2016 05:54 AM, Nir Cohen wrote:
> On Thursday, July 7, 2016 at 7:47:22 AM UTC+3, Nir Cohen wrote:
>> On Wednesday, July 6, 2016 at 10:09:01 PM UTC+3, Ethan Furman wrote:
>>> On 07/06/2016 11:43 AM, Nir Cohen wrote:

>>>> We decided that we want to package sets of wheels together created or downloaded
>>>   > by `pip wheel`, add relevant metadata, package them together into a
>>> single archive
>>>   > (tar.gz or zip) and use the same tool which packs them up to install
>>> them later on,
>>>   > on the destination hosts.
>>>>
>>>> We came up with a tool (http://github.com/cloudify-cosmo/wagon) to do just that
>>>   > and that's what we currently use to create and install our plugins.

> Sorry to bump this up.. but I'm not sure exactly how I should go about getting as much feedback as possible on this. Any suggestions?

You could try Python-Ideas -- plenty of debate happens on that list. ;)

--
~Ethan~

[toc] | [prev] | [next] | [standalone]

#111451

Appreciate it! Will do!

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web