Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.maint.python > #15326 > unrolled thread

Recommended way of installing system-wide python application and libraries

Back to article view | Back to linux.debian.maint.python

Contents

  Recommended way of installing system-wide python application and libraries Ivan Perez <ivanperezdominguez@gmail.com> - 2023-12-05 11:20 +0100
    Re: Recommended way of installing system-wide python application and  libraries Andrey Rakhmatullin <wrar@wrar.name> - 2023-12-05 11:30 +0100
    Re: Recommended way of installing system-wide python application  and libraries c.buhtz@posteo.jp - 2023-12-05 13:30 +0100
      Re: Recommended way of installing system-wide python application and  libraries Salvo Tomaselli <tiposchi@tiscali.it> - 2023-12-05 17:50 +0100
    Re: Recommended way of installing system-wide python application  and libraries Scott Talbert <swt@techie.net> - 2023-12-08 01:30 +0100
      Re: Recommended way of installing system-wide python application and libraries Ivan Perez <ivanperezdominguez@gmail.com> - 2023-12-18 00:30 +0100

#15326 — Recommended way of installing system-wide python application and libraries

[Multipart message — attachments visible in raw view] — view raw

Hi everyone!

I'm currently trying to bring a tool we have at NASA Ames up to speed:
https://github.com/NASA-SW-VnV/ikos

IKOS is a static analyzer for C. I'm really hoping that IKOS can be
included in Debian in the near future.

IKOS is implemented as a  C++ library, and a number of python
tools/wrappers. The tools call mains in modules defined in a python library
`ikos`.

As of right now, our CMakeFiles attempt to install everything (by default)
under /opt/ikos/.

I'm having lots of issues getting the python portions installed
system-wide. I initially upgraded distutils to setuptools, but a recent
update is now asking that I use a venv. More details and a link to a
dockerfile can be found here:
https://github.com/NASA-SW-VnV/ikos/discussions/241.

While I can hack a solution that "works" (either by making a venv under the
target dir or by means of break-system-packages), I'd prefer to use
recommended practices, and also conform to the way that things are done in
Debian/Ubuntu.

What would be the recommended way of installing IKOS system-wide in Debian?

Thanks,

Ivan

PS. Apologies if this is considered off-topic for this list.

[toc] | [next] | [standalone]

#15327 — Re: Recommended way of installing system-wide python application and libraries

On Tue, Dec 05, 2023 at 02:10:01AM -0800, Ivan Perez wrote:
> Hi everyone!
> 
> I'm currently trying to bring a tool we have at NASA Ames up to speed:
> https://github.com/NASA-SW-VnV/ikos
> 
> IKOS is a static analyzer for C. I'm really hoping that IKOS can be
> included in Debian in the near future.
> 
> IKOS is implemented as a  C++ library, and a number of python
> tools/wrappers. The tools call mains in modules defined in a python library
> `ikos`.
> 
> As of right now, our CMakeFiles attempt to install everything (by default)
> under /opt/ikos/.
> 
> I'm having lots of issues getting the python portions installed
> system-wide. I initially upgraded distutils to setuptools, but a recent
> update is now asking that I use a venv. More details and a link to a
> dockerfile can be found here:
> https://github.com/NASA-SW-VnV/ikos/discussions/241.
> 
> While I can hack a solution that "works" (either by making a venv under the
> target dir or by means of break-system-packages), I'd prefer to use
> recommended practices, and also conform to the way that things are done in
> Debian/Ubuntu.
> 
> What would be the recommended way of installing IKOS system-wide in Debian?
This answer may not be useful for you but the only recommended way of
installing Python modules system-wide is by making a proper Debian package
for them using proper Debian Python packaging helpers, and the only other
recommended way of installing Python modules is venvs.
If I needed to install this I would use either of these two, depending on
whether I actually need to have it system-wide and whether I want an
official package uploaded to Debian.
Alternatively, as you mentioned /opt, if the software supports installing
*everything* in /opt (maybe it adds that to sys.path when it runs), then
doing that is also fine (as it doesn't touch things outside /opt and
cannot break those).

[toc] | [prev] | [next] | [standalone]

#15328 — Re: Recommended way of installing system-wide python application and libraries

Hello Ivan,

I am not an expert in Python Packaging and also not in Debian-specific 
Packaging of Python stuff. So please take my advice with care. But I do 
have some experience and did a lot of experiments.

When it comes to pure Python Packaging I would suggest my demo 
repository. It is not finished yet but it explains some packaging issues 
and use cases with examples.

<https://codeberg.org/buhtz/tech-demo-python-packaging>

When this repo is "finished" and well reviewed by the Python community I 
will start a similar repo with demos about Debian Python Packaing. That 
is the plan.

The background of all that is that I have to migrate a quit old project 
("Back In Time") from a make-based build-system to modern state of the 
art Python build-system.

> update is now asking that I use a venv. More details and a link to a

That is a a "new thing" with Debian 12 following PEP 668 
(<https://peps.python.org/pep-0668/>).
Currently myself I do struggle with this PEP in my development process.
Im the end I do use "--break-system-packages" or "export 
PIP_BREAK_SYSTEM_PACKAGES=1".
It is not a solution but a workaround until I find a better way to 
integrate PEP 668 into my workflow.

I am not sure about it but to my understanding the recommended way to 
install packages via "pip" (e.g. from PyPi) is to use "pipx" instead of 
"pip". "pipx" will handle virtuel environments in the back. But this 
will cause some other issues I haven't solved yet.

Kind
Christian

[toc] | [prev] | [next] | [standalone]

#15329 — Re: Recommended way of installing system-wide python application and libraries

[Multipart message — attachments visible in raw view] — view raw

Hello,

I guess you plan on maintaining it yourself.

You should do an "Intention to package" bug, you do that running:

reportbug wnpp

There is a tool called "lintian" that you will probably find useful to check 
your packaging, as you get it into shape.

I'd recommend the debian-mentors IRC channel on oftc and the same named 
mailing list.

Good luck

-- 
Salvo Tomaselli

"Io non mi sento obbligato a credere che lo stesso Dio che ci ha dotato di
senso, ragione ed intelletto intendesse che noi ne facessimo a meno."
                -- Galileo Galilei

https://ltworf.codeberg.page/

[toc] | [prev] | [next] | [standalone]

#15332 — Re: Recommended way of installing system-wide python application and libraries

[Multipart message — attachments visible in raw view] — view raw

On Tue, 5 Dec 2023, Ivan Perez wrote:

> Hi everyone!
> 
> I'm currently trying to bring a tool we have at NASA Ames up to speed:
> https://github.com/NASA-SW-VnV/ikos
> 
> IKOS is a static analyzer for C. I'm really hoping that IKOS can be included
> in Debian in the near future.
> 
> IKOS is implemented as a  C++ library, and a number of python
> tools/wrappers. The tools call mains in modules defined in a python library
> `ikos`.
> 
> As of right now, our CMakeFiles attempt to install everything (by default)
> under /opt/ikos/.
> 
> I'm having lots of issues getting the python portions installed system-wide.
> I initially upgraded distutils to setuptools, but a recent update is now
> asking that I use a venv. More details and a link to a dockerfile can be
> found here: https://github.com/NASA-SW-VnV/ikos/discussions/241.
> 
> While I can hack a solution that "works" (either by making a venv under the
> target dir or by means of break-system-packages), I'd prefer to use
> recommended practices, and also conform to the way that things are done in
> Debian/Ubuntu.
> 
> What would be the recommended way of installing IKOS system-wide in Debian?

Hello again Ivan (different neighborhood :)),

The "modern" way of building Python libraries/applications seems to be to 
use a pyproject.toml.  The pyproject.toml specifies a backend build system 
that knows how to build the library/application and produce a wheel.

As far as how to do this within an existing cmake project, unfortunately, 
there doesn't seem to be a clear/easy way.  The only cmake example I can 
think off of the top of my head is cvc5.  It still uses setup.py though, 
so not a great future-looking example (and I had to patch it to build the 
Python bindings in Debian): 
https://github.com/cvc5/cvc5/blob/main/src/api/python/CMakeLists.txt

If you're open to other options for building python libraries, meson might 
be something to look into.  I haven't used it extensively, though.  It 
does seem to have better support for building Python though.

Regards,
Scott

[toc] | [prev] | [next] | [standalone]

#15367

[Multipart message — attachments visible in raw view] — view raw

Thanks!! I'll answer to the last comment but I appreciate everyone's input!

Hello again Ivan (different neighborhood :)),

:)))

The "modern" way of building Python libraries/applications seems to be to
> use a pyproject.toml.  The pyproject.toml specifies a backend build system
> that knows how to build the library/application and produce a wheel.


Thanks for this pointer. I didn't know about this.

As far as how to do this within an existing cmake project, unfortunately,
> there doesn't seem to be a clear/easy way.


For now, we ended up switching to venvs. I just published a release
candidate.

https://github.com/NASA-SW-VnV/ikos/discussions/253

Following @salvo's comment, I'm going to try and begin the process of
packaging this for debian. I suspect it may be a bit complicated because
ikos is two things (a library in C++ and a set of tools in Python) under a
trench coat.

This is by far not my domain. If anyone is willing to lend a hand, I could
definitely use any help that I can get, but I understand that folks are
busy and everybody wants help.

Cheers,

Ivan



On Thu, 7 Dec 2023 at 16:12, Scott Talbert <swt@techie.net> wrote:

> On Tue, 5 Dec 2023, Ivan Perez wrote:
>
> > Hi everyone!
> >
> > I'm currently trying to bring a tool we have at NASA Ames up to speed:
> > https://github.com/NASA-SW-VnV/ikos
> >
> > IKOS is a static analyzer for C. I'm really hoping that IKOS can be
> included
> > in Debian in the near future.
> >
> > IKOS is implemented as a  C++ library, and a number of python
> > tools/wrappers. The tools call mains in modules defined in a python
> library
> > `ikos`.
> >
> > As of right now, our CMakeFiles attempt to install everything (by
> default)
> > under /opt/ikos/.
> >
> > I'm having lots of issues getting the python portions installed
> system-wide.
> > I initially upgraded distutils to setuptools, but a recent update is now
> > asking that I use a venv. More details and a link to a dockerfile can be
> > found here: https://github.com/NASA-SW-VnV/ikos/discussions/241.
> >
> > While I can hack a solution that "works" (either by making a venv under
> the
> > target dir or by means of break-system-packages), I'd prefer to use
> > recommended practices, and also conform to the way that things are done
> in
> > Debian/Ubuntu.
> >
> > What would be the recommended way of installing IKOS system-wide in
> Debian?
>
> Hello again Ivan (different neighborhood :)),
>
> The "modern" way of building Python libraries/applications seems to be to
> use a pyproject.toml.  The pyproject.toml specifies a backend build system
> that knows how to build the library/application and produce a wheel.
>
> As far as how to do this within an existing cmake project, unfortunately,
> there doesn't seem to be a clear/easy way.  The only cmake example I can
> think off of the top of my head is cvc5.  It still uses setup.py though,
> so not a great future-looking example (and I had to patch it to build the
> Python bindings in Debian):
> https://github.com/cvc5/cvc5/blob/main/src/api/python/CMakeLists.txt
>
> If you're open to other options for building python libraries, meson might
> be something to look into.  I haven't used it extensively, though.  It
> does seem to have better support for building Python though.
>
> Regards,
> Scott

[toc] | [prev] | [standalone]

Back to top | Article view | linux.debian.maint.python

csiph-web