Wine MinGW system libraries

From	Zebediah Figura <zfigura@codeweavers.com>
Newsgroups	linux.debian.devel
Subject	Wine MinGW system libraries
Date	2021-09-05 03:40 +0200
Message-ID	<CTPiy-2xn-3@gated-at.bofh.it> (permalink)
Organization	linux.* mail to news gateway

Show all headers | View raw

Hello all,

I'm a contributor to the Wine project. To summarize the following mail, 
Wine needs special versions of some of its normal dependencies, such as 
libfreetype and libgnutls, built using the MinGW cross-compiler, and I'm 
sending out a mail to major distributions in order to get some feedback 
from our packagers on how these should be built and packaged.

For a long time Wine has built all of its Win32 libraries (DLLs and 
EXEs) as ELF binaries. For various reasons related to application 
compatibility, we have started building our binaries as PE instead, 
using the MinGW cross-compiler. It is our intent to expand this to some 
of our dependencies as well. The list of dependencies that we intend to 
build using MinGW is not quite fixed yet, but we expect it to include 
and be mostly limited to the following:

* libvkd3d
* libFAudio
* libgnutls
* zlib (currently included via manual source import)
* libmpg123
* libgsm
* libpng
* libjpeg-turbo
* libtiff
* libfreetype
* liblcms2
* jxrlib

and dependencies of the above packages (not including CRT dependencies, 
which Wine provides).

There is currently some internal discussion about how these dependencies 
should be built and linked. There are essentially three questions I see 
that need to be resolved, and while these resolutions have a significant 
impact on the Wine building and development process, they also have an 
impact on distributions, and accordingly I'd like to get input from our 
packagers to ensure that their considerations are accurately taken into 
account.

(1) Should we build via source import, or link statically, or dynamically?

Source imports are dispreferred by Debian [1], on the grounds that they 
cause duplication of libraries on disk and in memory, and make it harder 
to handle security updates. They also make building and bisecting 
harder. Static libraries don't seem to be expressly discouraged, but 
share most of the same downsides (see also [2]).

Note however that if they are linked dynamically, we need to make sure 
that we load our packages instead of MinGW builds of open-source 
libraries with applications ship with. There's some internal discussion 
about whether this is possible while using "stock" builds of MinGW 
libraries, but, due to the way the Win32 loader works, we will probably 
need to compile each library, and its dependencies, with a separate, 
wine-specific name, e.g. "libwinefreetype-6.dll" and 
"libwineharfbuzz.dll". For a detailed explantion see footnote [3]. Note 
that all we actually need to change is the name; we don't need to patch 
the source.

Accordingly, although static linking and source imports are generally 
disprefered, it may quite likely be preferable in our case. We don't get 
the benefits of on-disk deduplication, since Wine is essentially the 
only piece of software which needs these libraries.

(2) If we use dynamic libraries, should dependencies be included in the 
main wine package, or packaged separately?

This is mostly a question for packagers, although it also relates to (3).

I expect that Debian will want to answer "packaged separately" here, on 
the grounds that this lets them update (say) Wine's libgnutls 
separately, and in sync with ELF libgnutls, if some security fix is 
needed. There is a snag, though: we need libraries to be copied into the 
prefix (there's some internal effort to allow using something like 
symlinks instead, but this hard and not done yet). Normally we perform 
this copy every time Wine is updated, but if Wine and its dependencies 
aren't updated on the same schedule, we may end up loading an old 
version of a dependency in the prefix.

(3) If dependencies are packaged separately, should Wine build them as 
part of its build tree (e.g. using submodules), or find and link 
(statically or dynamically) to existing binaries?

Linking to existing binaries is generally preferable: it avoids 
duplication on disk; it reduces compile times when compiling a single 
package from source (especially the first time). However, we aren't 
going to benefit from on-disk duplication. And, most importantly, unlike 
with ELF dependencies, there is no standardized way to locate MinGW 
libraries—especially if it comes to Wine-specific libraries. We would 
need a way for Wine's configure script to find these packages—and 
ideally find them automatically, or else fall back to a submodule-based 
approach.

If we rely on distributions to provide our dependencies, the best idea I 
have here would be something like a x86_64-w64-mingw32-pkg-config 
(Fedora actually already ships this, but I think Fedora is the only 
one). And if we use shared libraries rather than static, things get 
worse: we need to know the exact path of each library and its 
dependencies at runtime so that we can copy (or symlink) them into a 
user's WINEPREFIX. For ELF this is the job of ld.so. For MinGW there is 
no standardized install location across distributions.


For what it's worth, the current proposed solution (which has the 
support of the Wine maintainer) involves source imports and submodules. 
There's probably room for changing our approach even after things are 
committed, but I'd still like to get early feedback from distributions, 
and make sure that their interests are accurately represented, before we 
commit.

ἔρρωσθε,
Zebediah
(she/her)

[1] 
https://www.debian.org/doc/debian-policy/ch-source.html#embedded-code-copies

[2] https://wiki.debian.org/StaticLinking

[3] The basic problem is that applications can and often do ship with PE 
builds of cross-platform libraries. These libraries can be ahead of 
Wine's system libraries, behind them, or even built with custom patches. 
Accordingly we really don't want to load "our" freetype in place of 
"their" freetype, or "theirs" in place of "ours". But because of the way 
the Win32 loader works, you can generally only have one DLL of a given 
name loaded in a process, and further attempts to dlopen() [as it were] 
"libfreetype-6.dll" will return the handle to the already loaded 
library, potentially breaking either Wine or the application. There 
*may* be ways we can hack around this internally, but it's not clear 
that it's feasible yet, and so it's probably best to assume that we'll 
need special builds of dynamic libraries.

Back to linux.debian.devel | Previous | Next — Next in thread | Find similar | Unroll thread

Thread

Wine MinGW system libraries Zebediah Figura <zfigura@codeweavers.com> - 2021-09-05 03:40 +0200
  Re: Wine MinGW system libraries Bastien ROUCARIES <roucaries.bastien@gmail.com> - 2021-09-05 09:20 +0200
    Re: Wine MinGW system libraries Stephen Kitt <skitt@debian.org> - 2021-09-05 18:30 +0200
    Re: Wine MinGW system libraries Adrian Bunk <bunk@debian.org> - 2021-09-12 10:00 +0200
      Re: Wine MinGW system libraries Bastien ROUCARIES <roucaries.bastien@gmail.com> - 2021-09-12 15:40 +0200
        Re: Wine MinGW system libraries Adrian Bunk <bunk@debian.org> - 2021-09-12 16:10 +0200
          Re: Wine MinGW system libraries Bastien ROUCARIES <roucaries.bastien@gmail.com> - 2021-09-12 16:20 +0200
            Re: Wine MinGW system libraries Adrian Bunk <bunk@debian.org> - 2021-09-12 16:40 +0200
              Re: Wine MinGW system libraries Bastien ROUCARIES <roucaries.bastien@gmail.com> - 2021-09-12 17:20 +0200
                Re: Wine MinGW system libraries Adrian Bunk <bunk@debian.org> - 2021-09-12 17:30 +0200
            Re: Wine MinGW system libraries Helmut Grohne <helmut@subdivi.de> - 2021-09-13 17:50 +0200
              Re: Wine MinGW system libraries Stephen Kitt <skitt@debian.org> - 2021-09-13 20:20 +0200
                Re: Wine MinGW system libraries Helmut Grohne <helmut@subdivi.de> - 2021-09-13 22:10 +0200
                Re: Wine MinGW system libraries Adrian Bunk <bunk@debian.org> - 2021-09-13 22:30 +0200
                Re: Wine MinGW system libraries Bastien ROUCARIES <roucaries.bastien@gmail.com> - 2021-09-13 23:00 +0200
                Re: Wine MinGW system libraries Adrian Bunk <bunk@debian.org> - 2021-09-13 23:40 +0200
      Re: Wine MinGW system libraries Stephen Kitt <skitt@debian.org> - 2021-09-12 17:40 +0200
        Re: Wine MinGW system libraries Adrian Bunk <bunk@debian.org> - 2021-09-12 18:40 +0200
          Re: Wine MinGW system libraries Stephen Kitt <skitt@debian.org> - 2021-09-12 19:10 +0200
            Re: Wine MinGW system libraries Adrian Bunk <bunk@debian.org> - 2021-09-12 20:20 +0200
              Re: Wine MinGW system libraries Stephen Kitt <skitt@debian.org> - 2021-09-13 20:30 +0200
  Re: Wine MinGW system libraries Stephen Kitt <skitt@debian.org> - 2021-09-05 18:20 +0200
    Re: Wine MinGW system libraries Zebediah Figura <zfigura@codeweavers.com> - 2021-09-05 19:20 +0200
      Re: Wine MinGW system libraries Stephen Kitt <skitt@debian.org> - 2021-09-06 09:10 +0200
        Re: Wine MinGW system libraries Bastien ROUCARIES <roucaries.bastien@gmail.com> - 2021-09-06 10:20 +0200
        Re: Wine MinGW system libraries Zebediah Figura <zfigura@codeweavers.com> - 2021-09-06 18:40 +0200
          Re: Wine MinGW system libraries Bastien ROUCARIES <roucaries.bastien@gmail.com> - 2021-09-06 21:00 +0200
            Re: Wine MinGW system libraries Zebediah Figura <zfigura@codeweavers.com> - 2021-09-07 00:00 +0200
              Re: Wine MinGW system libraries Paul Wise <pabs@debian.org> - 2021-09-07 02:50 +0200
                Re: Wine MinGW system libraries "Bastien Roucariès" <roucaries.bastien@gmail.com> - 2021-09-07 12:40 +0200
                Re: Wine MinGW system libraries Zebediah Figura <zfigura@codeweavers.com> - 2021-09-07 17:50 +0200
                Re: Wine MinGW system libraries Bastien ROUCARIES <roucaries.bastien@gmail.com> - 2021-09-07 19:30 +0200
                Re: Wine MinGW system libraries Bastien ROUCARIES <roucaries.bastien@gmail.com> - 2021-09-07 20:20 +0200
                Re: Wine MinGW system libraries Zebediah Figura <zfigura@codeweavers.com> - 2021-09-07 23:20 +0200
                Re: Wine MinGW system libraries Bastien ROUCARIES <roucaries.bastien@gmail.com> - 2021-09-08 01:50 +0200
                Re: Wine MinGW system libraries Bastien ROUCARIES <roucaries.bastien@gmail.com> - 2021-09-08 09:50 +0200
                Re: Wine MinGW system libraries Simon McVittie <smcv@debian.org> - 2021-09-08 10:20 +0200
                Re: Wine MinGW system libraries Zebediah Figura <zfigura@codeweavers.com> - 2021-09-08 17:50 +0200
                Re: Wine MinGW system libraries Paul Wise <pabs@debian.org> - 2021-09-09 03:20 +0200
                Re: Wine MinGW system libraries Zebediah Figura <zfigura@codeweavers.com> - 2021-09-09 06:50 +0200
                Re: Wine MinGW system libraries Paul Wise <pabs@debian.org> - 2021-09-09 07:20 +0200
                Re: Wine MinGW system libraries Zebediah Figura <zfigura@codeweavers.com> - 2021-09-09 07:40 +0200
                Re: Wine MinGW system libraries Paul Wise <pabs@debian.org> - 2021-09-09 07:50 +0200
                Re: Wine MinGW system libraries Zebediah Figura <zfigura@codeweavers.com> - 2021-09-09 08:10 +0200
                Re: Wine MinGW system libraries Paul Wise <pabs@debian.org> - 2021-09-09 09:40 +0200
                Re: Wine MinGW system libraries Bastien ROUCARIES <roucaries.bastien@gmail.com> - 2021-09-10 11:50 +0200
                Re: Wine MinGW system libraries Paul Wise <pabs@debian.org> - 2021-09-11 08:10 +0200
                Re: Wine MinGW system libraries Bastien ROUCARIES <roucaries.bastien@gmail.com> - 2021-09-12 17:20 +0200
                Re: Wine MinGW system libraries Paul Wise <pabs@debian.org> - 2021-09-08 01:10 +0200
                Re: Wine MinGW system libraries Bastien ROUCARIES <roucaries.bastien@gmail.com> - 2021-09-08 01:30 +0200
  Re: Wine MinGW system libraries Adrian Bunk <bunk@debian.org> - 2021-09-11 17:40 +0200
    Re: Wine MinGW system libraries Bastien ROUCARIES <roucaries.bastien@gmail.com> - 2021-09-11 20:10 +0200
  Re: Wine MinGW system libraries Zebediah Figura <zfigura@codeweavers.com> - 2022-04-17 02:30 +0200

csiph-web