Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #97422 > unrolled thread

packaging code with compiled libraries

Back to article view | Back to comp.lang.python

Contents

  packaging code with compiled libraries Tim <jtim.arnold@gmail.com> - 2015-10-05 12:43 -0700
    Re: packaging code with compiled libraries Rustom Mody <rustompmody@gmail.com> - 2015-10-05 19:42 -0700
    Re: packaging code with compiled libraries Tim Golden <mail@timgolden.me.uk> - 2015-10-06 08:16 +0100
    Re: packaging code with compiled libraries Oscar Benjamin <oscar.j.benjamin@gmail.com> - 2015-10-06 10:50 +0100
      Re: packaging code with compiled libraries Tim <jtim.arnold@gmail.com> - 2015-10-06 08:40 -0700

#97422 — packaging code with compiled libraries

I have a package I want to share but have a question about packaging.

Mostly the package is pure python code, but it also requires some binary libraries (*.so, *.dll, *.dylib).  I want to bundle these libs so users don't have to compile. The package will run on *nix/windows/mac platforms.

Currently I handle this in setup.py. In the 'build' phase, I copy the platform-specific libs to a subdirectory called 'libs'.  

    class MyBuilder(build_py):
        def run(self):
            conditional logic for copying 
            appropriate library files to 'libs'
            etc etc.
            build_py.run()

And that seems to work, but after reading more from the Python Packaging Authority, I wonder if that is the right way.  Should I be using wheels instead? 
I think my brain fried a little bit while going through the doc.

thanks,
--Tim

[toc] | [next] | [standalone]

#97428

On Tuesday, October 6, 2015 at 1:14:05 AM UTC+5:30, Tim wrote:
<attempt snipped>
> And that seems to work, but after reading more from the Python Packaging 
> Authority, I wonder if that is the right way.  Should I be using wheels instead? 
> I think my brain fried a little bit while going through the doc.

You are not alone
https://mail.python.org/pipermail/python-list/2015-July/694818.html
[last para]

[toc] | [prev] | [next] | [standalone]

#97430

On 05/10/2015 20:43, Tim wrote:
> I have a package I want to share but have a question about packaging.
> 
> Mostly the package is pure python code, but it also requires some binary libraries (*.so, *.dll, *.dylib).  I want to bundle these libs so users don't have to compile. The package will run on *nix/windows/mac platforms.
> 
> Currently I handle this in setup.py. In the 'build' phase, I copy the platform-specific libs to a subdirectory called 'libs'.  
> 
>     class MyBuilder(build_py):
>         def run(self):
>             conditional logic for copying 
>             appropriate library files to 'libs'
>             etc etc.
>             build_py.run()
> 
> And that seems to work, but after reading more from the Python Packaging Authority, I wonder if that is the right way.  Should I be using wheels instead? 
> I think my brain fried a little bit while going through the doc.

I suggest you ask on distutils-sig which is where the PPA guys tend to
hang out.

  https://mail.python.org/mailman/listinfo/distutils-sig

TJG

[toc] | [prev] | [next] | [standalone]

#97437

On 5 October 2015 at 20:43, Tim <jtim.arnold@gmail.com> wrote:
>
> I have a package I want to share but have a question about packaging.
>
> Mostly the package is pure python code, but it also requires some binary libraries (*.so, *.dll, *.dylib).  I want to bundle these libs so users don't have to compile. The package will run on *nix/windows/mac platforms.
>
> Currently I handle this in setup.py. In the 'build' phase, I copy the platform-specific libs to a subdirectory called 'libs'.
>
>     class MyBuilder(build_py):
>         def run(self):
>             conditional logic for copying
>             appropriate library files to 'libs'
>             etc etc.
>             build_py.run()
>
> And that seems to work, but after reading more from the Python Packaging Authority, I wonder if that is the right way.  Should I be using wheels instead?
> I think my brain fried a little bit while going through the doc.

The idea of a wheel is that you want to distribute your code fully
precompiled to end users who will be able to install it without
needing any C compilers etc. Of course this requires you to supply
wheels for each platform of interest. If this is what you want to do
then yes absolutely use wheels. Note that if you have installed
setuptools and wheel and you use setuptools in your setup.py then
building a wheel is as simple as running "python setup.py bdist_wheel"
(once your setup.py is complete).

If the binary libraries in question are extension modules then you
should just declare them as such in your setup.py and
distutils/setuptools/wheel will take care of bundling them into the
wheel.

If the binary libraries are not extension modules and you are building
them separately (not using distutils) then you can declare them as
"datafiles" [1] so that they will be bundled into your wheel and
installed alongside your python code.

[1] https://packaging.python.org/en/latest/distributing/#package-data

--
Oscar

[toc] | [prev] | [next] | [standalone]

#97452

On Tuesday, October 6, 2015 at 5:51:48 AM UTC-4, Oscar Benjamin wrote:
> On 5 October 2015 at 20:43, Tim <jtimDOTarnoldATgmail.com> wrote:
> >
> > I have a package I want to share but have a question about packaging.
> >
> > Mostly the package is pure python code, but it also requires some binary libraries (*.so, *.dll, *.dylib).  I want to bundle these libs so users don't have to compile. The package will run on *nix/windows/mac platforms.

> The idea of a wheel is that you want to distribute your code fully
> precompiled to end users who will be able to install it without
> needing any C compilers etc. Of course this requires you to supply
> wheels for each platform of interest. If this is what you want to do
> then yes absolutely use wheels. Note that if you have installed
> setuptools and wheel and you use setuptools in your setup.py then
> building a wheel is as simple as running "python setup.py bdist_wheel"
> (once your setup.py is complete).
> 
> If the binary libraries in question are extension modules then you
> should just declare them as such in your setup.py and
> distutils/setuptools/wheel will take care of bundling them into the
> wheel.
> 
> If the binary libraries are not extension modules and you are building
> them separately (not using distutils) then you can declare them as
> "datafiles" [1] so that they will be bundled into your wheel and
> installed alongside your python code.
> 
> [1] https://packaging.python.org/en/latest/distributing/#package-data
> 
> --
> Oscar

Thanks for that description. I read through the recent archives on distutils-sig and it's pretty much over my head. These libs are built separately (not extension modules) so I will use wheels with the 'datafiles' approach.
--Tim

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web