Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.devel > #101561

Re: Wine MinGW system libraries

Show all headers | View raw

On 9/6/21 1:34 PM, Bastien ROUCARIES wrote:
> Le lun. 6 sept. 2021 à 18:36, Zebediah Figura <zfigura@codeweavers.com> a
> écrit :
> 
>> On 9/6/21 1:57 AM, Stephen Kitt wrote:
>>> On Sun, 5 Sep 2021 12:14:47 -0500, Zebediah Figura <
>> zfigura@codeweavers.com>
>>> wrote:
>>>> On 9/5/21 11:19 AM, Stephen Kitt wrote:
>>>>> On Sat, 4 Sep 2021 20:17:53 -0500, Zebediah Figura
>>>>> <zfigura@codeweavers.com> wrote:
>>>>>> I'm a contributor to the Wine project. To summarize the following
>> mail,
>>>>>> Wine needs special versions of some of its normal dependencies, such
>> as
>>>>>> libfreetype and libgnutls, built using the MinGW cross-compiler, and
>> I'm
>>>>>> sending out a mail to major distributions in order to get some
>> feedback
>>>>>> from our packagers on how these should be built and packaged.
>>>>>>
>>>>>> For a long time Wine has built all of its Win32 libraries (DLLs and
>>>>>> EXEs) as ELF binaries. For various reasons related to application
>>>>>> compatibility, we have started building our binaries as PE instead,
>>>>>> using the MinGW cross-compiler. It is our intent to expand this to
>> some
>>>>>> of our dependencies as well. The list of dependencies that we intend
>> to
>>>>>> build using MinGW is not quite fixed yet, but we expect it to include
>>>>>> and be mostly limited to the following:
>>>>>>
>>>>>> * libvkd3d
>>>>>> * libFAudio
>>>>>> * libgnutls
>>>>>> * zlib (currently included via manual source import)
>>>>>> * libmpg123
>>>>>> * libgsm
>>>>>> * libpng
>>>>>> * libjpeg-turbo
>>>>>> * libtiff
>>>>>> * libfreetype
>>>>>> * liblcms2
>>>>>> * jxrlib
>>>>>>
>>>>>> and dependencies of the above packages (not including CRT
>> dependencies,
>>>>>> which Wine provides).
>>>>>>
>>>>>> There is currently some internal discussion about how these
>> dependencies
>>>>>> should be built and linked. There are essentially three questions I
>> see
>>>>>> that need to be resolved, and while these resolutions have a
>> significant
>>>>>> impact on the Wine building and development process, they also have an
>>>>>> impact on distributions, and accordingly I'd like to get input from
>> our
>>>>>> packagers to ensure that their considerations are accurately taken
>> into
>>>>>> account.
>>>>>>
>>>>>> (1) Should we build via source import, or link statically, or
>>>>>> dynamically?
>>>>>>
>>>>>> Source imports are dispreferred by Debian [1], on the grounds that
>> they
>>>>>> cause duplication of libraries on disk and in memory, and make it
>> harder
>>>>>> to handle security updates. They also make building and bisecting
>>>>>> harder. Static libraries don't seem to be expressly discouraged, but
>>>>>> share most of the same downsides (see also [2]).
>>>>>>
>>>>>> Note however that if they are linked dynamically, we need to make sure
>>>>>> that we load our packages instead of MinGW builds of open-source
>>>>>> libraries with applications ship with. There's some internal
>> discussion
>>>>>> about whether this is possible while using "stock" builds of MinGW
>>>>>> libraries, but, due to the way the Win32 loader works, we will
>> probably
>>>>>> need to compile each library, and its dependencies, with a separate,
>>>>>> wine-specific name, e.g. "libwinefreetype-6.dll" and
>>>>>> "libwineharfbuzz.dll". For a detailed explantion see footnote [3].
>> Note
>>>>>> that all we actually need to change is the name; we don't need to
>> patch
>>>>>> the source.
>>>>>
>>>>> Assuming Debian provides the dependencies (which is currently true only
>>>>> for zlib), this could be handled in packaging by providing symlinks,
>>>>> couldn’t it? Not in the Wine prefixes, but elsewhere.
>>>>
>>>> Almost :-/
>>>>
>>>> Copying/symlinking libfreetype-1.dll to libwinefreetype-1.dll is easy.
>>>> The problem is that libwinefreetype-1.dll is still going to link to
>>>> libharfbuzz-1.dll, but we need it to link to libwineharfbuzz-1.dll.
>>>
>>> Ah yes, I hadn’t thought it through. So really Wine needs its own
>> parallel
>>> ecosystem of DLLs in any case, which effectively means building them
>> along
>>> with Wine (from Wine’s perspective, ignoring distribution constraints and
>>> preferences).
>>>
>>>>> This also works in Debian:
>>>>>
>>>>> $ sudo apt install libz-mingw-w64-dev mingw-w64-tools
>>>>> $ x86_64-w64-mingw32-pkg-config --libs zlib
>>>>> -L/usr/x86_64-w64-mingw32/lib -lz
>>>>
>>>> Ah, cool, I looked for that and somehow missed it.
>>>>
>>>> Note that's the wrong path for dynamic libraries, though; those go in
>>>> /usr/x86_64-w64-mingw32/bin/. We'll need a way to find those. Maybe
>>>> hardcoding a list won't be too painful, though...
>>>
>>> In Debian they go in /usr/x86_64-w64-mingw32/lib currently, which is why
>>> the .pc file points there. But as you point out, that doesn’t help at
>>> runtime.
>>>
>>> Also, having pkg-config support doesn’t really help with a parallel set
>> of
>>> DLLs, does it?
>>
>> I mean... eh. In theory you could say "here's a library called
>> libwinefreetype, and to find it you do `pkg-config --cflags --libs
>> libwinefreetype`", but that does strike me as more than a little janky.
>>
>>>
>>>>>> For what it's worth, the current proposed solution (which has the
>>>>>> support of the Wine maintainer) involves source imports and
>> submodules.
>>>>>> There's probably room for changing our approach even after things are
>>>>>> committed, but I'd still like to get early feedback from
>> distributions,
>>>>>> and make sure that their interests are accurately represented, before
>> we
>>>>>> commit.
>>>>>
>>>>> Realistically, I think this is the best approach for now. As Debian
>> adds
>>>>> support for PE libraries, we can replace the source imports in the Wine
>>>>> source code; this is done in many other Debian packages for projects
>> which
>>>>> vendor dependencies.
>>>
>>> I still think this is true. With requirement for a Wine-specific set of
>> DLLs,
>>> improving the situation in Debian would involve supporting source
>>> build-dependencies, i.e. being able to say that a given package wants
>> access
>>> to the source code of another package. That’s something that’s been
>> brought
>>> up previously, and is worked around by providing binary packages
>> containing
>>> package source code (e.g. binutils-source, gcc-11-source etc.), but isn’t
>>> really upstream’s concern, in that it’s a Debian implementation detail.
>>>
>>> Going back to your original three questions, I think that the best
>> approach
>>> for you as upstream is to focus on providing a complete set of source
>> code
>>> (including dependencies) which works, and to make it friendlier to
>>> distributions, make the build process capable of handling alternative
>>> locations for the dependencies’ source code or even build artifacts.
>> (This
>>> has a number of knock-on effects — in particular, you should ensure that
>> the
>>> upstream source code for all your dependencies works with Wine, i.e. that
>>> Wine doesn’t require Wine-specific patches to any of its dependencies.)
>>>
>>> Given how varied MinGW-w64 handling is in different distributions,
>> pushing
>>> things further risks making it easier for one distribution and harder
>> for the
>>> others...
>>
>> Thanks for the detailed response!
>>
>> It's probably worth pointing out that:
>>
>> (1) if we use static linking, we should be able to use distribution
>> libraries unmodified. Of course, static linking comes with its own
>> downsides...
>>
>> (2) Like I mentioned or hinted at in my original email, it *may* be
>> possible to use distribution dynamic libraries unmodified. It's not
>> clear that it can be done without breaking compatibility, but if it can,
>> would that change anything?
>>
> 
> Yes i really prefer (2) it means we could use multiarch easilly....
> 
> Please try to document thé blocking point.

So this is mostly documented in the footnote to my original mail, but 
I'll copy it here for reference.

The basic problem is that applications can and often do ship with PE
builds of cross-platform libraries. These libraries can be ahead of
Wine's system libraries, behind them, or even built with custom patches.
Accordingly we really don't want to load "our" freetype in place of
"their" freetype, or "theirs" in place of "ours". But because of the way
the Win32 loader works, you can generally only have one DLL of a given
name loaded in a process, and further attempts to dlopen() [as it were]
"libfreetype-6.dll" will return the handle to the already loaded
library, potentially breaking either Wine or the application.

The easiest way to avoid ever doing that is by giving each library a 
different name (or by not using dynamic linking at all.)

Failing that, we need a way to determine whether a library is a system 
dependency—and thus can *only* be loaded from the system DLL path—or an 
application dependency—and thus should *never* be loaded from the system 
DLL path. I think this can be done, but it gets tricky, and it's not 
clear that there aren't going to be subtle problems from having two 
different DLLs with the same name loaded in the same process. Especially 
keeping in mind that Windows programs do absolutely insane things when 
it comes to depending on OS internals.

Back to linux.debian.devel | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Wine MinGW system libraries Zebediah Figura <zfigura@codeweavers.com> - 2021-09-05 03:40 +0200
  Re: Wine MinGW system libraries Bastien ROUCARIES <roucaries.bastien@gmail.com> - 2021-09-05 09:20 +0200
    Re: Wine MinGW system libraries Stephen Kitt <skitt@debian.org> - 2021-09-05 18:30 +0200
    Re: Wine MinGW system libraries Adrian Bunk <bunk@debian.org> - 2021-09-12 10:00 +0200
      Re: Wine MinGW system libraries Bastien ROUCARIES <roucaries.bastien@gmail.com> - 2021-09-12 15:40 +0200
        Re: Wine MinGW system libraries Adrian Bunk <bunk@debian.org> - 2021-09-12 16:10 +0200
          Re: Wine MinGW system libraries Bastien ROUCARIES <roucaries.bastien@gmail.com> - 2021-09-12 16:20 +0200
            Re: Wine MinGW system libraries Adrian Bunk <bunk@debian.org> - 2021-09-12 16:40 +0200
              Re: Wine MinGW system libraries Bastien ROUCARIES <roucaries.bastien@gmail.com> - 2021-09-12 17:20 +0200
                Re: Wine MinGW system libraries Adrian Bunk <bunk@debian.org> - 2021-09-12 17:30 +0200
            Re: Wine MinGW system libraries Helmut Grohne <helmut@subdivi.de> - 2021-09-13 17:50 +0200
              Re: Wine MinGW system libraries Stephen Kitt <skitt@debian.org> - 2021-09-13 20:20 +0200
                Re: Wine MinGW system libraries Helmut Grohne <helmut@subdivi.de> - 2021-09-13 22:10 +0200
                Re: Wine MinGW system libraries Adrian Bunk <bunk@debian.org> - 2021-09-13 22:30 +0200
                Re: Wine MinGW system libraries Bastien ROUCARIES <roucaries.bastien@gmail.com> - 2021-09-13 23:00 +0200
                Re: Wine MinGW system libraries Adrian Bunk <bunk@debian.org> - 2021-09-13 23:40 +0200
      Re: Wine MinGW system libraries Stephen Kitt <skitt@debian.org> - 2021-09-12 17:40 +0200
        Re: Wine MinGW system libraries Adrian Bunk <bunk@debian.org> - 2021-09-12 18:40 +0200
          Re: Wine MinGW system libraries Stephen Kitt <skitt@debian.org> - 2021-09-12 19:10 +0200
            Re: Wine MinGW system libraries Adrian Bunk <bunk@debian.org> - 2021-09-12 20:20 +0200
              Re: Wine MinGW system libraries Stephen Kitt <skitt@debian.org> - 2021-09-13 20:30 +0200
  Re: Wine MinGW system libraries Stephen Kitt <skitt@debian.org> - 2021-09-05 18:20 +0200
    Re: Wine MinGW system libraries Zebediah Figura <zfigura@codeweavers.com> - 2021-09-05 19:20 +0200
      Re: Wine MinGW system libraries Stephen Kitt <skitt@debian.org> - 2021-09-06 09:10 +0200
        Re: Wine MinGW system libraries Bastien ROUCARIES <roucaries.bastien@gmail.com> - 2021-09-06 10:20 +0200
        Re: Wine MinGW system libraries Zebediah Figura <zfigura@codeweavers.com> - 2021-09-06 18:40 +0200
          Re: Wine MinGW system libraries Bastien ROUCARIES <roucaries.bastien@gmail.com> - 2021-09-06 21:00 +0200
            Re: Wine MinGW system libraries Zebediah Figura <zfigura@codeweavers.com> - 2021-09-07 00:00 +0200
              Re: Wine MinGW system libraries Paul Wise <pabs@debian.org> - 2021-09-07 02:50 +0200
                Re: Wine MinGW system libraries "Bastien Roucariès" <roucaries.bastien@gmail.com> - 2021-09-07 12:40 +0200
                Re: Wine MinGW system libraries Zebediah Figura <zfigura@codeweavers.com> - 2021-09-07 17:50 +0200
                Re: Wine MinGW system libraries Bastien ROUCARIES <roucaries.bastien@gmail.com> - 2021-09-07 19:30 +0200
                Re: Wine MinGW system libraries Bastien ROUCARIES <roucaries.bastien@gmail.com> - 2021-09-07 20:20 +0200
                Re: Wine MinGW system libraries Zebediah Figura <zfigura@codeweavers.com> - 2021-09-07 23:20 +0200
                Re: Wine MinGW system libraries Bastien ROUCARIES <roucaries.bastien@gmail.com> - 2021-09-08 01:50 +0200
                Re: Wine MinGW system libraries Bastien ROUCARIES <roucaries.bastien@gmail.com> - 2021-09-08 09:50 +0200
                Re: Wine MinGW system libraries Simon McVittie <smcv@debian.org> - 2021-09-08 10:20 +0200
                Re: Wine MinGW system libraries Zebediah Figura <zfigura@codeweavers.com> - 2021-09-08 17:50 +0200
                Re: Wine MinGW system libraries Paul Wise <pabs@debian.org> - 2021-09-09 03:20 +0200
                Re: Wine MinGW system libraries Zebediah Figura <zfigura@codeweavers.com> - 2021-09-09 06:50 +0200
                Re: Wine MinGW system libraries Paul Wise <pabs@debian.org> - 2021-09-09 07:20 +0200
                Re: Wine MinGW system libraries Zebediah Figura <zfigura@codeweavers.com> - 2021-09-09 07:40 +0200
                Re: Wine MinGW system libraries Paul Wise <pabs@debian.org> - 2021-09-09 07:50 +0200
                Re: Wine MinGW system libraries Zebediah Figura <zfigura@codeweavers.com> - 2021-09-09 08:10 +0200
                Re: Wine MinGW system libraries Paul Wise <pabs@debian.org> - 2021-09-09 09:40 +0200
                Re: Wine MinGW system libraries Bastien ROUCARIES <roucaries.bastien@gmail.com> - 2021-09-10 11:50 +0200
                Re: Wine MinGW system libraries Paul Wise <pabs@debian.org> - 2021-09-11 08:10 +0200
                Re: Wine MinGW system libraries Bastien ROUCARIES <roucaries.bastien@gmail.com> - 2021-09-12 17:20 +0200
                Re: Wine MinGW system libraries Paul Wise <pabs@debian.org> - 2021-09-08 01:10 +0200
                Re: Wine MinGW system libraries Bastien ROUCARIES <roucaries.bastien@gmail.com> - 2021-09-08 01:30 +0200
  Re: Wine MinGW system libraries Adrian Bunk <bunk@debian.org> - 2021-09-11 17:40 +0200
    Re: Wine MinGW system libraries Bastien ROUCARIES <roucaries.bastien@gmail.com> - 2021-09-11 20:10 +0200
  Re: Wine MinGW system libraries Zebediah Figura <zfigura@codeweavers.com> - 2022-04-17 02:30 +0200

csiph-web