Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.devel > #101547 > unrolled thread

Wine MinGW system libraries

Back to article view | Back to linux.debian.devel

Contents

  Wine MinGW system libraries Zebediah Figura <zfigura@codeweavers.com> - 2021-09-05 03:40 +0200
    Re: Wine MinGW system libraries Bastien ROUCARIES <roucaries.bastien@gmail.com> - 2021-09-05 09:20 +0200
      Re: Wine MinGW system libraries Stephen Kitt <skitt@debian.org> - 2021-09-05 18:30 +0200
      Re: Wine MinGW system libraries Adrian Bunk <bunk@debian.org> - 2021-09-12 10:00 +0200
        Re: Wine MinGW system libraries Bastien ROUCARIES <roucaries.bastien@gmail.com> - 2021-09-12 15:40 +0200
          Re: Wine MinGW system libraries Adrian Bunk <bunk@debian.org> - 2021-09-12 16:10 +0200
            Re: Wine MinGW system libraries Bastien ROUCARIES <roucaries.bastien@gmail.com> - 2021-09-12 16:20 +0200
              Re: Wine MinGW system libraries Adrian Bunk <bunk@debian.org> - 2021-09-12 16:40 +0200
                Re: Wine MinGW system libraries Bastien ROUCARIES <roucaries.bastien@gmail.com> - 2021-09-12 17:20 +0200
                  Re: Wine MinGW system libraries Adrian Bunk <bunk@debian.org> - 2021-09-12 17:30 +0200
              Re: Wine MinGW system libraries Helmut Grohne <helmut@subdivi.de> - 2021-09-13 17:50 +0200
                Re: Wine MinGW system libraries Stephen Kitt <skitt@debian.org> - 2021-09-13 20:20 +0200
                  Re: Wine MinGW system libraries Helmut Grohne <helmut@subdivi.de> - 2021-09-13 22:10 +0200
                  Re: Wine MinGW system libraries Adrian Bunk <bunk@debian.org> - 2021-09-13 22:30 +0200
                    Re: Wine MinGW system libraries Bastien ROUCARIES <roucaries.bastien@gmail.com> - 2021-09-13 23:00 +0200
                      Re: Wine MinGW system libraries Adrian Bunk <bunk@debian.org> - 2021-09-13 23:40 +0200
        Re: Wine MinGW system libraries Stephen Kitt <skitt@debian.org> - 2021-09-12 17:40 +0200
          Re: Wine MinGW system libraries Adrian Bunk <bunk@debian.org> - 2021-09-12 18:40 +0200
            Re: Wine MinGW system libraries Stephen Kitt <skitt@debian.org> - 2021-09-12 19:10 +0200
              Re: Wine MinGW system libraries Adrian Bunk <bunk@debian.org> - 2021-09-12 20:20 +0200
                Re: Wine MinGW system libraries Stephen Kitt <skitt@debian.org> - 2021-09-13 20:30 +0200
    Re: Wine MinGW system libraries Stephen Kitt <skitt@debian.org> - 2021-09-05 18:20 +0200
      Re: Wine MinGW system libraries Zebediah Figura <zfigura@codeweavers.com> - 2021-09-05 19:20 +0200
        Re: Wine MinGW system libraries Stephen Kitt <skitt@debian.org> - 2021-09-06 09:10 +0200
          Re: Wine MinGW system libraries Bastien ROUCARIES <roucaries.bastien@gmail.com> - 2021-09-06 10:20 +0200
          Re: Wine MinGW system libraries Zebediah Figura <zfigura@codeweavers.com> - 2021-09-06 18:40 +0200
            Re: Wine MinGW system libraries Bastien ROUCARIES <roucaries.bastien@gmail.com> - 2021-09-06 21:00 +0200
              Re: Wine MinGW system libraries Zebediah Figura <zfigura@codeweavers.com> - 2021-09-07 00:00 +0200
                Re: Wine MinGW system libraries Paul Wise <pabs@debian.org> - 2021-09-07 02:50 +0200
                  Re: Wine MinGW system libraries "Bastien Roucariès" <roucaries.bastien@gmail.com> - 2021-09-07 12:40 +0200
                    Re: Wine MinGW system libraries Zebediah Figura <zfigura@codeweavers.com> - 2021-09-07 17:50 +0200
                      Re: Wine MinGW system libraries Bastien ROUCARIES <roucaries.bastien@gmail.com> - 2021-09-07 19:30 +0200
                        Re: Wine MinGW system libraries Bastien ROUCARIES <roucaries.bastien@gmail.com> - 2021-09-07 20:20 +0200
                        Re: Wine MinGW system libraries Zebediah Figura <zfigura@codeweavers.com> - 2021-09-07 23:20 +0200
                          Re: Wine MinGW system libraries Bastien ROUCARIES <roucaries.bastien@gmail.com> - 2021-09-08 01:50 +0200
                            Re: Wine MinGW system libraries Bastien ROUCARIES <roucaries.bastien@gmail.com> - 2021-09-08 09:50 +0200
                              Re: Wine MinGW system libraries Simon McVittie <smcv@debian.org> - 2021-09-08 10:20 +0200
                                Re: Wine MinGW system libraries Zebediah Figura <zfigura@codeweavers.com> - 2021-09-08 17:50 +0200
                                  Re: Wine MinGW system libraries Paul Wise <pabs@debian.org> - 2021-09-09 03:20 +0200
                                    Re: Wine MinGW system libraries Zebediah Figura <zfigura@codeweavers.com> - 2021-09-09 06:50 +0200
                                      Re: Wine MinGW system libraries Paul Wise <pabs@debian.org> - 2021-09-09 07:20 +0200
                                        Re: Wine MinGW system libraries Zebediah Figura <zfigura@codeweavers.com> - 2021-09-09 07:40 +0200
                                          Re: Wine MinGW system libraries Paul Wise <pabs@debian.org> - 2021-09-09 07:50 +0200
                                            Re: Wine MinGW system libraries Zebediah Figura <zfigura@codeweavers.com> - 2021-09-09 08:10 +0200
                                              Re: Wine MinGW system libraries Paul Wise <pabs@debian.org> - 2021-09-09 09:40 +0200
                                                Re: Wine MinGW system libraries Bastien ROUCARIES <roucaries.bastien@gmail.com> - 2021-09-10 11:50 +0200
                                                  Re: Wine MinGW system libraries Paul Wise <pabs@debian.org> - 2021-09-11 08:10 +0200
                                                    Re: Wine MinGW system libraries Bastien ROUCARIES <roucaries.bastien@gmail.com> - 2021-09-12 17:20 +0200
                      Re: Wine MinGW system libraries Paul Wise <pabs@debian.org> - 2021-09-08 01:10 +0200
                        Re: Wine MinGW system libraries Bastien ROUCARIES <roucaries.bastien@gmail.com> - 2021-09-08 01:30 +0200
    Re: Wine MinGW system libraries Adrian Bunk <bunk@debian.org> - 2021-09-11 17:40 +0200
      Re: Wine MinGW system libraries Bastien ROUCARIES <roucaries.bastien@gmail.com> - 2021-09-11 20:10 +0200
    Re: Wine MinGW system libraries Zebediah Figura <zfigura@codeweavers.com> - 2022-04-17 02:30 +0200

Page 1 of 3  [1] 2 3  Next page →

#101547 — Wine MinGW system libraries

Hello all,

I'm a contributor to the Wine project. To summarize the following mail, 
Wine needs special versions of some of its normal dependencies, such as 
libfreetype and libgnutls, built using the MinGW cross-compiler, and I'm 
sending out a mail to major distributions in order to get some feedback 
from our packagers on how these should be built and packaged.

For a long time Wine has built all of its Win32 libraries (DLLs and 
EXEs) as ELF binaries. For various reasons related to application 
compatibility, we have started building our binaries as PE instead, 
using the MinGW cross-compiler. It is our intent to expand this to some 
of our dependencies as well. The list of dependencies that we intend to 
build using MinGW is not quite fixed yet, but we expect it to include 
and be mostly limited to the following:

* libvkd3d
* libFAudio
* libgnutls
* zlib (currently included via manual source import)
* libmpg123
* libgsm
* libpng
* libjpeg-turbo
* libtiff
* libfreetype
* liblcms2
* jxrlib

and dependencies of the above packages (not including CRT dependencies, 
which Wine provides).

There is currently some internal discussion about how these dependencies 
should be built and linked. There are essentially three questions I see 
that need to be resolved, and while these resolutions have a significant 
impact on the Wine building and development process, they also have an 
impact on distributions, and accordingly I'd like to get input from our 
packagers to ensure that their considerations are accurately taken into 
account.

(1) Should we build via source import, or link statically, or dynamically?

Source imports are dispreferred by Debian [1], on the grounds that they 
cause duplication of libraries on disk and in memory, and make it harder 
to handle security updates. They also make building and bisecting 
harder. Static libraries don't seem to be expressly discouraged, but 
share most of the same downsides (see also [2]).

Note however that if they are linked dynamically, we need to make sure 
that we load our packages instead of MinGW builds of open-source 
libraries with applications ship with. There's some internal discussion 
about whether this is possible while using "stock" builds of MinGW 
libraries, but, due to the way the Win32 loader works, we will probably 
need to compile each library, and its dependencies, with a separate, 
wine-specific name, e.g. "libwinefreetype-6.dll" and 
"libwineharfbuzz.dll". For a detailed explantion see footnote [3]. Note 
that all we actually need to change is the name; we don't need to patch 
the source.

Accordingly, although static linking and source imports are generally 
disprefered, it may quite likely be preferable in our case. We don't get 
the benefits of on-disk deduplication, since Wine is essentially the 
only piece of software which needs these libraries.

(2) If we use dynamic libraries, should dependencies be included in the 
main wine package, or packaged separately?

This is mostly a question for packagers, although it also relates to (3).

I expect that Debian will want to answer "packaged separately" here, on 
the grounds that this lets them update (say) Wine's libgnutls 
separately, and in sync with ELF libgnutls, if some security fix is 
needed. There is a snag, though: we need libraries to be copied into the 
prefix (there's some internal effort to allow using something like 
symlinks instead, but this hard and not done yet). Normally we perform 
this copy every time Wine is updated, but if Wine and its dependencies 
aren't updated on the same schedule, we may end up loading an old 
version of a dependency in the prefix.

(3) If dependencies are packaged separately, should Wine build them as 
part of its build tree (e.g. using submodules), or find and link 
(statically or dynamically) to existing binaries?

Linking to existing binaries is generally preferable: it avoids 
duplication on disk; it reduces compile times when compiling a single 
package from source (especially the first time). However, we aren't 
going to benefit from on-disk duplication. And, most importantly, unlike 
with ELF dependencies, there is no standardized way to locate MinGW 
libraries—especially if it comes to Wine-specific libraries. We would 
need a way for Wine's configure script to find these packages—and 
ideally find them automatically, or else fall back to a submodule-based 
approach.

If we rely on distributions to provide our dependencies, the best idea I 
have here would be something like a x86_64-w64-mingw32-pkg-config 
(Fedora actually already ships this, but I think Fedora is the only 
one). And if we use shared libraries rather than static, things get 
worse: we need to know the exact path of each library and its 
dependencies at runtime so that we can copy (or symlink) them into a 
user's WINEPREFIX. For ELF this is the job of ld.so. For MinGW there is 
no standardized install location across distributions.


For what it's worth, the current proposed solution (which has the 
support of the Wine maintainer) involves source imports and submodules. 
There's probably room for changing our approach even after things are 
committed, but I'd still like to get early feedback from distributions, 
and make sure that their interests are accurately represented, before we 
commit.

ἔρρωσθε,
Zebediah
(she/her)

[1] 
https://www.debian.org/doc/debian-policy/ch-source.html#embedded-code-copies

[2] https://wiki.debian.org/StaticLinking

[3] The basic problem is that applications can and often do ship with PE 
builds of cross-platform libraries. These libraries can be ahead of 
Wine's system libraries, behind them, or even built with custom patches. 
Accordingly we really don't want to load "our" freetype in place of 
"their" freetype, or "theirs" in place of "ours". But because of the way 
the Win32 loader works, you can generally only have one DLL of a given 
name loaded in a process, and further attempts to dlopen() [as it were] 
"libfreetype-6.dll" will return the handle to the already loaded 
library, potentially breaking either Wine or the application. There 
*may* be ways we can hack around this internally, but it's not clear 
that it's feasible yet, and so it's probably best to assume that we'll 
need special builds of dynamic libraries.

[toc] | [next] | [standalone]

#101549

[Multipart message — attachments visible in raw view] — view raw

Le dim. 5 sept. 2021 à 03:34, Zebediah Figura <zfigura@codeweavers.com> a
écrit :

> Hello all,
>
> I'm a contributor to the Wine project. To summarize the following mail,
> Wine needs special versions of some of its normal dependencies, such as
> libfreetype and libgnutls, built using the MinGW cross-compiler, and I'm
> sending out a mail to major distributions in order to get some feedback
> from our packagers on how these should be built and packaged.
>
> For a long time Wine has built all of its Win32 libraries (DLLs and
> EXEs) as ELF binaries. For various reasons related to application
> compatibility, we have started building our binaries as PE instead,
> using the MinGW cross-compiler. It is our intent to expand this to some
> of our dependencies as well. The list of dependencies that we intend to
> build using MinGW is not quite fixed yet, but we expect it to include
> and be mostly limited to the following:
>
> * libvkd3d
> * libFAudio
> * libgnutls
> * zlib (currently included via manual source import)
> * libmpg123
> * libgsm
> * libpng
> * libjpeg-turbo
> * libtiff
> * libfreetype
> * liblcms2
> * jxrlib
>
> and dependencies of the above packages (not including CRT dependencies,
> which Wine provides).
>
> There is currently some internal discussion about how these dependencies
> should be built and linked. There are essentially three questions I see
> that need to be resolved, and while these resolutions have a significant
> impact on the Wine building and development process, they also have an
> impact on distributions, and accordingly I'd like to get input from our
> packagers to ensure that their considerations are accurately taken into
> account.
>
> (1) Should we build via source import, or link statically, or dynamically?
>
> Source imports are dispreferred by Debian [1], on the grounds that they
> cause duplication of libraries on disk and in memory, and make it harder
> to handle security updates. They also make building and bisecting
> harder. Static libraries don't seem to be expressly discouraged, but
> share most of the same downsides (see also [2]).
>
> Note however that if they are linked dynamically, we need to make sure
> that we load our packages instead of MinGW builds of open-source
> libraries with applications ship with. There's some internal discussion
> about whether this is possible while using "stock" builds of MinGW
> libraries, but, due to the way the Win32 loader works, we will probably
> need to compile each library, and its dependencies, with a separate,
> wine-specific name, e.g. "libwinefreetype-6.dll" and
> "libwineharfbuzz.dll". For a detailed explantion see footnote [3]. Note
> that all we actually need to change is the name; we don't need to patch
> the source.
>
> Accordingly, although static linking and source imports are generally
> disprefered, it may quite likely be preferable in our case. We don't get
> the benefits of on-disk deduplication, since Wine is essentially the
> only piece of software which needs these libraries.
>
> (2) If we use dynamic libraries, should dependencies be included in the
> main wine package, or packaged separately?
>


Improve dpkg to support partial arch. I volonteer to implement none arch
but i am waiting from guillem here.



> This is mostly a question for packagers, although it also relates to (3).
>
> I expect that Debian will want to answer "packaged separately" here, on
> the grounds that this lets them update (say) Wine's libgnutls
> separately, and in sync with ELF libgnutls, if some security fix is
> needed. There is a snag, though: we need libraries to be copied into the
> prefix (there's some internal effort to allow using something like
> symlinks instead, but this hard and not done yet). Normally we perform
> this copy every time Wine is updated, but if Wine and its dependencies
> aren't updated on the same schedule, we may end up loading an old
> version of a dependency in the prefix.
>
> (3) If dependencies are packaged separately, should Wine build them as
> part of its build tree (e.g. using submodules), or find and link
> (statically or dynamically) to existing binaries?
>
> Linking to existing binaries is generally preferable: it avoids
> duplication on disk; it reduces compile times when compiling a single
> package from source (especially the first time). However, we aren't
> going to benefit from on-disk duplication. And, most importantly, unlike
> with ELF dependencies, there is no standardized way to locate MinGW
> libraries—especially if it comes to Wine-specific libraries. We would
> need a way for Wine's configure script to find these packages—and
> ideally find them automatically, or else fall back to a submodule-based
> approach.
>
> If we rely on distributions to provide our dependencies, the best idea I
> have here would be something like a x86_64-w64-mingw32-pkg-config
> (Fedora actually already ships this, but I think Fedora is the only
> one). And if we use shared libraries rather than static, things get
> worse: we need to know the exact path of each library and its
> dependencies at runtime so that we can copy (or symlink) them into a
> user's WINEPREFIX. For ELF this is the job of ld.so. For MinGW there is
> no standardized install location across distributions
>

Partial arch is the solution hère.


I can help also here

>
> For what it's worth, the current proposed solution (which has the
> support of the Wine maintainer) involves source imports and submodules.
> There's probably room for changing our approach even after things are
> committed, but I'd still like to get early feedback from distributions,
> and make sure that their interests are accurately represented, before we
> commit.
>
> ἔρρωσθε,
> Zebediah
> (she/her)
>
> [1]
>
> https://www.debian.org/doc/debian-policy/ch-source.html#embedded-code-copies
>
> [2] https://wiki.debian.org/StaticLinking
>
> [3] The basic problem is that applications can and often do ship with PE
> builds of cross-platform libraries. These libraries can be ahead of
> Wine's system libraries, behind them, or even built with custom patches.
> Accordingly we really don't want to load "our" freetype in place of
> "their" freetype, or "theirs" in place of "ours". But because of the way
> the Win32 loader works, you can generally only have one DLL of a given
> name loaded in a process, and further attempts to dlopen() [as it were]
> "libfreetype-6.dll" will return the handle to the already loaded
> library, potentially breaking either Wine or the application. There
> *may* be ways we can hack around this internally, but it's not clear
> that it's feasible yet, and so it's probably best to assume that we'll
> need special builds of dynamic libraries.
>
>

[toc] | [prev] | [next] | [standalone]

#101553

[Multipart message — attachments visible in raw view] — view raw

Hi Bastien,

On Sun, 5 Sep 2021 08:53:49 +0200, Bastien ROUCARIES
<roucaries.bastien@gmail.com> wrote:
> Le dim. 5 sept. 2021 à 03:34, Zebediah Figura <zfigura@codeweavers.com> a
> écrit :
> > I'm a contributor to the Wine project. To summarize the following mail,
> > Wine needs special versions of some of its normal dependencies, such as
> > libfreetype and libgnutls, built using the MinGW cross-compiler, and I'm
> > sending out a mail to major distributions in order to get some feedback
> > from our packagers on how these should be built and packaged.
> >
> > For a long time Wine has built all of its Win32 libraries (DLLs and
> > EXEs) as ELF binaries. For various reasons related to application
> > compatibility, we have started building our binaries as PE instead,
> > using the MinGW cross-compiler. It is our intent to expand this to some
> > of our dependencies as well. The list of dependencies that we intend to
> > build using MinGW is not quite fixed yet, but we expect it to include
> > and be mostly limited to the following:
> >
> > * libvkd3d
> > * libFAudio
> > * libgnutls
> > * zlib (currently included via manual source import)
> > * libmpg123
> > * libgsm
> > * libpng
> > * libjpeg-turbo
> > * libtiff
> > * libfreetype
> > * liblcms2
> > * jxrlib
> >
> > and dependencies of the above packages (not including CRT dependencies,
> > which Wine provides).
> >
> > There is currently some internal discussion about how these dependencies
> > should be built and linked. There are essentially three questions I see
> > that need to be resolved, and while these resolutions have a significant
> > impact on the Wine building and development process, they also have an
> > impact on distributions, and accordingly I'd like to get input from our
> > packagers to ensure that their considerations are accurately taken into
> > account.
> >
> > (1) Should we build via source import, or link statically, or dynamically?
> >
> > Source imports are dispreferred by Debian [1], on the grounds that they
> > cause duplication of libraries on disk and in memory, and make it harder
> > to handle security updates. They also make building and bisecting
> > harder. Static libraries don't seem to be expressly discouraged, but
> > share most of the same downsides (see also [2]).
> >
> > Note however that if they are linked dynamically, we need to make sure
> > that we load our packages instead of MinGW builds of open-source
> > libraries with applications ship with. There's some internal discussion
> > about whether this is possible while using "stock" builds of MinGW
> > libraries, but, due to the way the Win32 loader works, we will probably
> > need to compile each library, and its dependencies, with a separate,
> > wine-specific name, e.g. "libwinefreetype-6.dll" and
> > "libwineharfbuzz.dll". For a detailed explantion see footnote [3]. Note
> > that all we actually need to change is the name; we don't need to patch
> > the source.
> >
> > Accordingly, although static linking and source imports are generally
> > disprefered, it may quite likely be preferable in our case. We don't get
> > the benefits of on-disk deduplication, since Wine is essentially the
> > only piece of software which needs these libraries.
> >
> > (2) If we use dynamic libraries, should dependencies be included in the
> > main wine package, or packaged separately?
> >  
> 
> 
> Improve dpkg to support partial arch. I volonteer to implement none arch
> but i am waiting from guillem here.

Thanks for stepping up!

For MinGW-w64 dependencies, an additional requirement is figuring out a
solution for https://bugs.debian.org/606825;
https://wiki.debian.org/Teams/Dpkg/FAQ#Q._Can_we_add_support_for_new_dpkg_architectures.3F
gives additional context.

Regards,

Stephen

[toc] | [prev] | [next] | [standalone]

#101665

On Sun, Sep 05, 2021 at 08:53:49AM +0200, Bastien ROUCARIES wrote:
>...
> Improve dpkg to support partial arch. I volonteer to implement none arch
> but i am waiting from guillem here.
>...

There is also plenty of infrastructure on the buildd, archive and 
release team sides that would likely need changes for supporting 
anything like that.

A multilib based approach might be more realistic for bookworm.

What benefits would a "none arch" even bring compared to building
binary-all packages?
The ability to binNMU is the only one that comes into my mind.

cu
Adrian

[toc] | [prev] | [next] | [standalone]

#101666

Le dim. 12 sept. 2021 à 07:38, Adrian Bunk <bunk@debian.org> a écrit :
>
> On Sun, Sep 05, 2021 at 08:53:49AM +0200, Bastien ROUCARIES wrote:
> >...
> > Improve dpkg to support partial arch. I volonteer to implement none arch
> > but i am waiting from guillem here.
> >...
>
> There is also plenty of infrastructure on the buildd, archive and
> release team sides that would likely need changes for supporting
> anything like that.
>
> A multilib based approach might be more realistic for bookworm.
>
> What benefits would a "none arch" even bring compared to building
> binary-all packages?
> The ability to binNMU is the only one that comes into my mind.
we have at least 3 architectures that need uefi none arch...

So we build three arch-all package with paterning name.


>
> cu
> Adrian

[toc] | [prev] | [next] | [standalone]

#101667

On Sun, Sep 12, 2021 at 01:18:11PM +0000, Bastien ROUCARIES wrote:
> Le dim. 12 sept. 2021 à 07:38, Adrian Bunk <bunk@debian.org> a écrit :
> >
> > On Sun, Sep 05, 2021 at 08:53:49AM +0200, Bastien ROUCARIES wrote:
> > >...
> > > Improve dpkg to support partial arch. I volonteer to implement none arch
> > > but i am waiting from guillem here.
> > >...
> >
> > There is also plenty of infrastructure on the buildd, archive and
> > release team sides that would likely need changes for supporting
> > anything like that.
> >
> > A multilib based approach might be more realistic for bookworm.
> >
> > What benefits would a "none arch" even bring compared to building
> > binary-all packages?
> > The ability to binNMU is the only one that comes into my mind.
> we have at least 3 architectures that need uefi none arch...
> 
> So we build three arch-all package with paterning name.

I might be misunderstanding what you are trying to do.

What release architecture would "none" packages be in the Debian 
archive, and on what buildds would they be built?

Debian buildds are not cross-compiling packages for a different Debian 
architecture, a partial architecture would still require DSA-maintained 
buildds doing native building on this partial architecture.

cu
Adrian

[toc] | [prev] | [next] | [standalone]

#101668

Le dim. 12 sept. 2021 à 13:44, Adrian Bunk <bunk@debian.org> a écrit :
>
> On Sun, Sep 12, 2021 at 01:18:11PM +0000, Bastien ROUCARIES wrote:
> > Le dim. 12 sept. 2021 à 07:38, Adrian Bunk <bunk@debian.org> a écrit :
> > >
> > > On Sun, Sep 05, 2021 at 08:53:49AM +0200, Bastien ROUCARIES wrote:
> > > >...
> > > > Improve dpkg to support partial arch. I volonteer to implement none arch
> > > > but i am waiting from guillem here.
> > > >...
> > >
> > > There is also plenty of infrastructure on the buildd, archive and
> > > release team sides that would likely need changes for supporting
> > > anything like that.
> > >
> > > A multilib based approach might be more realistic for bookworm.
> > >
> > > What benefits would a "none arch" even bring compared to building
> > > binary-all packages?
> > > The ability to binNMU is the only one that comes into my mind.
> > we have at least 3 architectures that need uefi none arch...
> >
> > So we build three arch-all package with paterning name.
>
> I might be misunderstanding what you are trying to do.
>
> What release architecture would "none" packages be in the Debian
> archive, and on what buildds would they be built?
>
> Debian buildds are not cross-compiling packages for a different Debian
> architecture, a partial architecture would still require DSA-maintained
> buildds doing native building on this partial architecture.

I think you misunderstand:
https://wiki.debian.org/Teams/Dpkg/Spec/FreestandingArches

They are a full color gradiant between:
- freestanding arches pure cross compile without any depends except arch:all
- partial cross built arch
- partial arch
- full arch

I believe the first step to get partial cross built arch is to begin
by freestanding arch.

Bastien

> cu
> Adrian

[toc] | [prev] | [next] | [standalone]

#101669

On Sun, Sep 12, 2021 at 01:57:44PM +0000, Bastien ROUCARIES wrote:
> 
> I think you misunderstand:
> https://wiki.debian.org/Teams/Dpkg/Spec/FreestandingArches
> 
> They are a full color gradiant between:
> - freestanding arches pure cross compile without any depends except arch:all
> - partial cross built arch
> - partial arch
> - full arch
> 
> I believe the first step to get partial cross built arch is to begin
> by freestanding arch.

I believe this is mostly irrelevant for the discussion about Wine.

Making Wine in Debian depend on such lofty ideas that might be available 
very far in the future (if ever) would imply no Wine in the next Debian
releases.

Debian 12 will be released mid-2023, which is only 2 years away.
Debian 13 will be released mid-2025, which is only 4 years away.

If your partial cross built arch is ever fully implemented and supported 
by Debian it would be a good idea to migrate Wine to using that.

But a realistic solution for Wine in Debian 12 has to be multilib,
not multiarch.

> Bastien

cu
Adrian

[toc] | [prev] | [next] | [standalone]

#101670

Le dim. 12 sept. 2021 à 14:16, Adrian Bunk <bunk@debian.org> a écrit :
>
> On Sun, Sep 12, 2021 at 01:57:44PM +0000, Bastien ROUCARIES wrote:
> >
> > I think you misunderstand:
> > https://wiki.debian.org/Teams/Dpkg/Spec/FreestandingArches
> >
> > They are a full color gradiant between:
> > - freestanding arches pure cross compile without any depends except arch:all
> > - partial cross built arch
> > - partial arch
> > - full arch
> >
> > I believe the first step to get partial cross built arch is to begin
> > by freestanding arch.
>
> I believe this is mostly irrelevant for the discussion about Wine.
>
> Making Wine in Debian depend on such lofty ideas that might be available
> very far in the future (if ever) would imply no Wine in the next Debian
> releases.
>
> Debian 12 will be released mid-2023, which is only 2 years away.
> Debian 13 will be released mid-2025, which is only 4 years away.
>
> If your partial cross built arch is ever fully implemented and supported
> by Debian it would be a good idea to migrate Wine to using that.
>
> But a realistic solution for Wine in Debian 12 has to be multilib,
> not multiarch.
Yes but current solution used by wine should be compatible with
multiarch. And It was since 2012 that multiarch exists. Maybe time to
speed up

bastien
>
> > Bastien
>
> cu
> Adrian

[toc] | [prev] | [next] | [standalone]

#101672

On Sun, Sep 12, 2021 at 02:57:20PM +0000, Bastien ROUCARIES wrote:
> Le dim. 12 sept. 2021 à 14:16, Adrian Bunk <bunk@debian.org> a écrit :
> >
> > On Sun, Sep 12, 2021 at 01:57:44PM +0000, Bastien ROUCARIES wrote:
> > >
> > > I think you misunderstand:
> > > https://wiki.debian.org/Teams/Dpkg/Spec/FreestandingArches
> > >
> > > They are a full color gradiant between:
> > > - freestanding arches pure cross compile without any depends except arch:all
> > > - partial cross built arch
> > > - partial arch
> > > - full arch
> > >
> > > I believe the first step to get partial cross built arch is to begin
> > > by freestanding arch.
> >
> > I believe this is mostly irrelevant for the discussion about Wine.
> >
> > Making Wine in Debian depend on such lofty ideas that might be available
> > very far in the future (if ever) would imply no Wine in the next Debian
> > releases.
> >
> > Debian 12 will be released mid-2023, which is only 2 years away.
> > Debian 13 will be released mid-2025, which is only 4 years away.
> >
> > If your partial cross built arch is ever fully implemented and supported
> > by Debian it would be a good idea to migrate Wine to using that.
> >
> > But a realistic solution for Wine in Debian 12 has to be multilib,
> > not multiarch.
> Yes but current solution used by wine should be compatible with
> multiarch. And It was since 2012 that multiarch exists. Maybe time to
> speed up

I do consider it quite evil how you are trying to steer upstream and
Debian maintainers of Wine in a direction where Wine would be taken
hostage for some lofty idea you have.

> bastien

cu
Adrian

[toc] | [prev] | [next] | [standalone]

#101688

I believe I can speak with my "main cross building porter" hat on.

On Sun, Sep 12, 2021 at 01:57:44PM +0000, Bastien ROUCARIES wrote:
> They are a full color gradiant between:
> - freestanding arches pure cross compile without any depends except arch:all
> - partial cross built arch
> - partial arch
> - full arch
> 
> I believe the first step to get partial cross built arch is to begin
> by freestanding arch.

Certainly not. As much as I would love to see this happen, it is very
unrealistic. wine can quite simply use -$arch-cross:all packages like
very other bare metal target and switch to our pipe dream once it is
ready (if ever).

cross building is very far off becoming the thing you want it to be. It
presently has a bus factor of around 1.5. Around 59% of source packages
are cross-satisfiable. Around 78% of cross-satisfiable packages are
cross buildable. In other words: less than 50% of packages are cross
buildable. Beyond that, the cross porters cannot keep up with the
current rate of regressions. Fractions are falling.

To make cross building a viable thing we'd need at least three more
porters determined to make it happen. Talk doesn't make it happen.
Patches do. Let me know if you want to help. I'll get you started.

Helmut

[toc] | [prev] | [next] | [standalone]

#101690

[Multipart message — attachments visible in raw view] — view raw

On Mon, 13 Sep 2021 17:39:25 +0200, Helmut Grohne <helmut@subdivi.de> wrote:
> I believe I can speak with my "main cross building porter" hat on.
> 
> On Sun, Sep 12, 2021 at 01:57:44PM +0000, Bastien ROUCARIES wrote:
> > They are a full color gradiant between:
> > - freestanding arches pure cross compile without any depends except
> > arch:all
> > - partial cross built arch
> > - partial arch
> > - full arch
> > 
> > I believe the first step to get partial cross built arch is to begin
> > by freestanding arch.  
> 
> Certainly not. As much as I would love to see this happen, it is very
> unrealistic. wine can quite simply use -$arch-cross:all packages like
> very other bare metal target and switch to our pipe dream once it is
> ready (if ever).

And even that assumes that we can get agreement on what $arch should be for
Windows targets :-(.

> cross building is very far off becoming the thing you want it to be. It
> presently has a bus factor of around 1.5. Around 59% of source packages
> are cross-satisfiable. Around 78% of cross-satisfiable packages are
> cross buildable. In other words: less than 50% of packages are cross
> buildable. Beyond that, the cross porters cannot keep up with the
> current rate of regressions. Fractions are falling.

The bus factor is indeed a problem. For Wine (and even a wider MinGW-w64
ecosystem) we don’t need all that many source packages to be
cross-satisfiable for the whole endeavour to be useful...

The regressions are significant though: if packages can’t stay
cross-satisfiable for Debian cross-targets, there’s little hope they can stay
cross-satisfiable for Windows! This means that separate source packages are
probably the only viable option.

> To make cross building a viable thing we'd need at least three more
> porters determined to make it happen. Talk doesn't make it happen.
> Patches do. Let me know if you want to help. I'll get you started.

Is the process documented anywhere? Or does one simply pick a failure from
http://crossqa.debian.net and figure out what’s going wrong (and hope that
pulling the threads doesn’t reveal a monster...)?

Regards,

Stephen

[toc] | [prev] | [next] | [standalone]

#101693

On Mon, Sep 13, 2021 at 08:19:31PM +0200, Stephen Kitt wrote:
> Is the process documented anywhere? Or does one simply pick a failure from
> http://crossqa.debian.net and figure out what’s going wrong (and hope that
> pulling the threads doesn’t reveal a monster...)?

That describes the process quite well. In particular, crossqa.d.n links
to existing bug reports to avoid duplication of work. A major lack
(beyond people doing the work) is domain-specific knowledge of the
various archive sections. So once you locate a monster, don't stop
there, but contact debian-cross@lists.debian.org or
#debian-bootstrap@oftc and let us combine your domain-specific knowledge
with some cross compilation experience to create an original solution.
So yeah, start with those packages that you know well.

Check http://crossqa.debian.net/src/<sourcepackagename> for your pet
packages or follow the "cross" link in the right sidebar at
tracker.d.o. Also (thanks to IOhannes zmölnig) enable cross building in
your salsa-ci pipeline.

In any case though, don't let wine wait for cross building to become a
thing.

Helmut

[toc] | [prev] | [next] | [standalone]

#101695

On Mon, Sep 13, 2021 at 08:19:31PM +0200, Stephen Kitt wrote:
>...
> For Wine (and even a wider MinGW-w64
> ecosystem) we don’t need all that many source packages to be
> cross-satisfiable for the whole endeavour to be useful...

But you would still need to create and maintain the whole 
infrastructure.

The larger picture is that there are at least 3 big topics:
- cross-architecture dependencies
- partial release architectures
- cross-building a release architecture

Every single of these topics would be a huge amount of work.

If you need all 3 as basis for a solution for Wine,
then that's simply not realistic in the foreseeable future.

> The regressions are significant though: if packages can’t stay
> cross-satisfiable for Debian cross-targets, there’s little hope they can stay
> cross-satisfiable for Windows! This means that separate source packages are
> probably the only viable option.
>...

Separate source packages don't bring any real benefits compared to 
just bundling and building everything in src:wine.

Either would likely imply that Wine in Debian comes without security 
support, and the Release Notes stating that Wine should only be used
for trusted local contents.

> Regards,
> 
> Stephen

cu
Adrian

[toc] | [prev] | [next] | [standalone]

#101696

Le lun. 13 sept. 2021 à 20:24, Adrian Bunk <bunk@debian.org> a écrit :
>
> On Mon, Sep 13, 2021 at 08:19:31PM +0200, Stephen Kitt wrote:
> >...
> > For Wine (and even a wider MinGW-w64
> > ecosystem) we don’t need all that many source packages to be
> > cross-satisfiable for the whole endeavour to be useful...
>
> But you would still need to create and maintain the whole
> infrastructure.
>
> The larger picture is that there are at least 3 big topics:
> - cross-architecture dependencies
> - partial release architectures
> - cross-building a release architecture
>
> Every single of these topics would be a huge amount of work.
>
> If you need all 3 as basis for a solution for Wine,
> then that's simply not realistic in the foreseeable future.
>
> > The regressions are significant though: if packages can’t stay
> > cross-satisfiable for Debian cross-targets, there’s little hope they can stay
> > cross-satisfiable for Windows! This means that separate source packages are
> > probably the only viable option.
> >...
>
> Separate source packages don't bring any real benefits compared to
> just bundling and building everything in src:wine.

No please do not do that. Vendoring is bad and I fight on the
javascript side since four years, against it.
>
> Either would likely imply that Wine in Debian comes without security
> support, and the Release Notes stating that Wine should only be used
> for trusted local contents.
>
> > Regards,
> >
> > Stephen
>
> cu
> Adrian
>

[toc] | [prev] | [next] | [standalone]

#101697

On Mon, Sep 13, 2021 at 08:38:52PM +0000, Bastien ROUCARIES wrote:
> Le lun. 13 sept. 2021 à 20:24, Adrian Bunk <bunk@debian.org> a écrit :
> > On Mon, Sep 13, 2021 at 08:19:31PM +0200, Stephen Kitt wrote:
>...
> > > The regressions are significant though: if packages can’t stay
> > > cross-satisfiable for Debian cross-targets, there’s little hope they can stay
> > > cross-satisfiable for Windows! This means that separate source packages are
> > > probably the only viable option.
> > >...
> >
> > Separate source packages don't bring any real benefits compared to
> > just bundling and building everything in src:wine.
> 
> No please do not do that. Vendoring is bad and I fight on the
> javascript side since four years, against it.
>...

Separate source packages that duplicate existing sources are as much 
vendoring as doing the same inside src:wine.

There are only two realistic options for Wine:
1. building multilib packages from the regular sources
   of the libraries in the archive, or
2. vendoring

Noone seems to be willing to do 1., which leaves 2. as only option.

cu
Adrian

[toc] | [prev] | [next] | [standalone]

#101673

[Multipart message — attachments visible in raw view] — view raw

On Sun, 12 Sep 2021 10:38:52 +0300, Adrian Bunk <bunk@debian.org> wrote:

> On Sun, Sep 05, 2021 at 08:53:49AM +0200, Bastien ROUCARIES wrote:
> >...
> > Improve dpkg to support partial arch. I volonteer to implement none arch
> > but i am waiting from guillem here.
> >...  
> 
> There is also plenty of infrastructure on the buildd, archive and 
> release team sides that would likely need changes for supporting 
> anything like that.
> 
> A multilib based approach might be more realistic for bookworm.
> 
> What benefits would a "none arch" even bring compared to building
> binary-all packages?
> The ability to binNMU is the only one that comes into my mind.

IMO the main benefit of using multiarch would be that packages could be built
for the new architectures without changes (ideally). libz is currently built
for MinGW-w64 in an “Architecture: all” package, but it’s a separate source
package; various other libraries are built with specific support in their
source packages. While one could imagine adding support to all the appropriate
source packages to build similar “Architecture: all” packages, that would
require convincing all the relevant maintainers, and it would end up tying
the testing migrations to MinGW-w64...

In this scenario the solution wouldn’t be a “none” arch, but a Windows arch,
if we can ever resolve https://bugs.debian.org/606825

The buildd situation isn’t necessarily that much of an obstacle: it seems to
me we could have “Windows” buildds which are really Linux amd64 systems, that
cross-build for Windows.

Regards,

Stephen

[toc] | [prev] | [next] | [standalone]

#101674

On Sun, Sep 12, 2021 at 05:31:41PM +0200, Stephen Kitt wrote:
>...
> While one could imagine adding support to all the appropriate
> source packages to build similar “Architecture: all” packages, that would
> require convincing all the relevant maintainers,

Adding a new release architecture (partial or not) requires convincing 
the ftp team and the release team.

It would also require many software changes.

How would for example the dependencies of wine:amd64 be fulfilled?

Just supporting that package dependencies might only be fulfilled by 
also using packages from a different architecture would require changes 
in many places, like for example the testing migration scripts that 
ensure installability after migration.

> and it would end up tying the testing migrations to MinGW-w64...

If this partial arch (and Wine) should be part of Debian releases,
then testing migrations would have to be tied to it in any case.

>...
> The buildd situation isn’t necessarily that much of an obstacle: it seems to
> me we could have “Windows” buildds which are really Linux amd64 systems, that
> cross-build for Windows.

The first obstacle is that if you want to be the first who does cross 
building packages on the buildds, there is likely a lot of work and 
bugfixing ahead for you for getting that working.

> Regards,
> 
> Stephen

cu
Adrian

[toc] | [prev] | [next] | [standalone]

#101675

[Multipart message — attachments visible in raw view] — view raw

On Sun, 12 Sep 2021 19:20:16 +0300, Adrian Bunk <bunk@debian.org> wrote:
> On Sun, Sep 12, 2021 at 05:31:41PM +0200, Stephen Kitt wrote:
> >...
> > While one could imagine adding support to all the appropriate
> > source packages to build similar “Architecture: all” packages, that would
> > require convincing all the relevant maintainers,  
> 
> Adding a new release architecture (partial or not) requires convincing 
> the ftp team and the release team.
> 
> It would also require many software changes.

Yes, it would. However, it requires convincing a well-defined set of people
*once*. If we handle Windows dependencies using “Architecture: all” packages,
we would either end up with something like Fedora’s MinGW-w64 SIG (with a
complete set of independent packages), or we would end up having to convince
a huge set of package maintainers over and over again.

> How would for example the dependencies of wine:amd64 be fulfilled?
> 
> Just supporting that package dependencies might only be fulfilled by 
> also using packages from a different architecture would require changes 
> in many places, like for example the testing migration scripts that 
> ensure installability after migration.

In the same way as the none-arch packages would be. Yes, it’s a lot of work,
but it’s useful for more than just Wine, and it can be done once for all the
interested architectures.

> > and it would end up tying the testing migrations to MinGW-w64...  
> 
> If this partial arch (and Wine) should be part of Debian releases,
> then testing migrations would have to be tied to it in any case.

True, I’d missed that. (In my mind I was comparing this with having separate
source packages.)

> >...
> > The buildd situation isn’t necessarily that much of an obstacle: it seems
> > to me we could have “Windows” buildds which are really Linux amd64
> > systems, that cross-build for Windows.  
> 
> The first obstacle is that if you want to be the first who does cross 
> building packages on the buildds, there is likely a lot of work and 
> bugfixing ahead for you for getting that working.

A lot of that work has already been done for http://crossqa.debian.net/

Fixing this would reduce the burden on all the maintainers who currently
handle cross-built packages in the archive, so overall I think it would be a
net win.

Regards,

Stephen

[toc] | [prev] | [next] | [standalone]

#101677

On Sun, Sep 12, 2021 at 07:03:54PM +0200, Stephen Kitt wrote:
> On Sun, 12 Sep 2021 19:20:16 +0300, Adrian Bunk <bunk@debian.org> wrote:
> > On Sun, Sep 12, 2021 at 05:31:41PM +0200, Stephen Kitt wrote:
> > >...
> > > While one could imagine adding support to all the appropriate
> > > source packages to build similar “Architecture: all” packages, that would
> > > require convincing all the relevant maintainers,  
> > 
> > Adding a new release architecture (partial or not) requires convincing 
> > the ftp team and the release team.
> > 
> > It would also require many software changes.
> 
> Yes, it would. However, it requires convincing a well-defined set of people
> *once*. If we handle Windows dependencies using “Architecture: all” packages,
> we would either end up with something like Fedora’s MinGW-w64 SIG (with a
> complete set of independent packages), or we would end up having to convince
> a huge set of package maintainers over and over again.

Based on the package list provided in the initial email in this thread
(and guessing the amount of transitive dependencies) I would have guessed
that there are perhaps ~ 30 packages that would have to be touched once.

And after that things will just continue working, just like with udebs
which are in some ways similar to what is being discussed here.

> > How would for example the dependencies of wine:amd64 be fulfilled?
> > 
> > Just supporting that package dependencies might only be fulfilled by 
> > also using packages from a different architecture would require changes 
> > in many places, like for example the testing migration scripts that 
> > ensure installability after migration.
> 
> In the same way as the none-arch packages would be. Yes, it’s a lot of work,
> but it’s useful for more than just Wine, and it can be done once for all the
> interested architectures.

It is also the kind of change where it might be required that it is 
fully supported in one release before it can be used by packages in the 
next release, if for example any of dpkg/apt/aptitude/... in the 
previous stable gives an error when seeing dependencies to another 
architecture.

>...
> Regards,
> 
> Stephen

cu
Adrian

[toc] | [prev] | [next] | [standalone]

Page 1 of 3  [1] 2 3  Next page →

Back to top | Article view | linux.debian.devel

csiph-web