Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.kernel > #64189 > unrolled thread

ZFS in Buster

Back to article view | Back to linux.debian.kernel

Contents

  ZFS in Buster Dan <ganchya@gmail.com> - 2019-05-28 18:50 +0200
    Re: ZFS in Buster "Enrico Weigelt, metux IT consult" <lkml@metux.net> - 2019-05-29 13:10 +0200
    Re: ZFS in Buster Dan <ganchya@gmail.com> - 2019-05-29 13:50 +0200
      Re: ZFS in Buster Ben Hutchings <ben@decadent.org.uk> - 2019-05-29 16:00 +0200
        Re: ZFS in Buster Bastian Blank <waldi@debian.org> - 2019-06-06 16:30 +0200
    Re: ZFS in Buster "Theodore Ts'o" <tytso@mit.edu> - 2019-06-02 04:20 +0200
    Re: ZFS in Buster Dan <ganchya@gmail.com> - 2019-06-03 20:40 +0200
      Re: ZFS in Buster Sam Hartman <hartmans@debian.org> - 2019-06-03 20:50 +0200

#64189 — ZFS in Buster

Dear Debian developers,

ZFS 0.8 has been released with lots of improvements, notably encryption.

Sadly the Linux Kernel has introduced a commit in kernel 4.19 and 5.0
that prevents ZFS from using SIMD. The result is that ZFS won't be
usable in Buster. See the following issue
https://github.com/zfsonlinux/zfs/issues/8793

NixOS reverted that particular commit:
https://www.phoronix.com/scan.php?page=news_item&px=NixOS-Linux-5.0-ZFS-FPU-Drop

Debian is the "Universal Operating System" and gives the user the
option to choose. It provides "vim and emacs", "Gnome and KDE",
"Linux, Hurd, KfreeBSD, etc.". I think that Debian should also provide
the option to use ZFS with encryption.

Would it be possible to provide an alternative patched linux kernel
that works with ZFS?

The ZFS developers proposed the Linux developers to rewrite the whole
ZFS code and use GPL, but surprisingly the linux developers didn't
accept. See below:
https://github.com/zfsonlinux/zfs/issues/8314

Best,
Daniel

[toc] | [next] | [standalone]

#64191

On 28.05.19 18:43, Dan wrote:

> Sadly the Linux Kernel has introduced a commit in kernel 4.19 and 5.0> that prevents ZFS from using SIMD. The result is that ZFS won't be>
usable in Buster. See the following issue>
https://github.com/zfsonlinux/zfs/issues/8793
We recently had this discussion on lkml - yet another case of 3rdparty
folks that just don't follow the license rules.

It's not the kernel who broke zfs, it's zfs that broke itself. The
kernel is GPL, and they just have to follow the rules or go away.

OOT modules are conceptionally messy in the first place. It's sometimes
okay as an temporary workaround, until things get mainlined. But
intentionally keeping things oot for long time is just silly and creates
lots of more problems than it creates.

And they're even using now *deeply* arch-internal functions directly.

> NixOS reverted that particular commit:>
https://www.phoronix.com/scan.php?page=news_item&px=NixOS-Linux-5.0-ZFS-FPU-Drop
Intentional license violation. Not funny.

> Debian is the "Universal Operating System" and gives the user the> option to choose. It provides "vim and emacs", "Gnome and KDE",
If you wanna have something new included, you'll have to sit down and do
the actual work. In the end of the day, it's that simple.

> Would it be possible to provide an alternative patched linux kernel
> that works with ZFS?

You mean patching against the license ?

> The ZFS developers proposed the Linux developers to rewrite the whole
> ZFS code and use GPL, but surprisingly the linux developers didn't
> accept. See below:
> https://github.com/zfsonlinux/zfs/issues/8314

Wait, no. It's not that we refused anything (actually, I don't even
recall any decent discussion on that @lkml). There even wasn't anything
to accept or refuse - except the existing code, that is nowhere near
a quality where any maintainer likes to even have a closer look at.

The major problem is that ZoL always has been oot on purpose, which is
the wrong approach to begin with. That also leads to bad code quality
(eg. lots of useless wrappers, horrible maintenance, ...)

What ZoL folks could do is step by step rewrite it to use mainline
functionality where ever technically feasible and work closely with
upstream to introduce missing functionality. Obviously, their current
proprietary userland interface can't be accepted for mainline - it
has to be reworked to be conformant w/ standard uapi (eg. we already
have it for things like snapshots, deduplication, quotas, ...)

But it's up to ZoL developers to do the actual work and post patches
to lkml. There won't be anybody else doing that.


--mtx

-- 
Enrico Weigelt, metux IT consult
Free software and Linux embedded engineering
info@metux.net -- +49-151-27565287

[toc] | [prev] | [next] | [standalone]

#64192

Hi Jonathan,

On Tue, May 28, 2019 at 8:50 PM Jonathan Carter <jcc@debian.org> wrote:
> On 2019/05/28 18:43, Dan wrote:
> > ZFS 0.8 has been released with lots of improvements, notably encryption.
>
> Yep, it's an exciting feature.
>
> > Sadly the Linux Kernel has introduced a commit in kernel 4.19 and 5.0
> > that prevents ZFS from using SIMD. The result is that ZFS won't be
> > usable in Buster. See the following issue
> > https://github.com/zfsonlinux/zfs/issues/8793
>
> Buster ships zfs-dkms version 0.7.12-2 though, which works just fine on
> buster.

My message was not accurate. I think that the commit has been
introduced in in 4.19.38 (released 2019-05-02). I think that Debian
Buster uses 4.19.37

https://cdn.kernel.org/pub/linux/kernel/v4.x/ChangeLog-4.19.38

The commit also affects ZFS 0.7 because SIMD is used for checksum operations.

There might be a performance penalty in ZFS only if Debian Buster
upgrades to 4.19.38.

Best,
Daniel

[toc] | [prev] | [next] | [standalone]

#64193

[Multipart message — attachments visible in raw view] — view raw

On Wed, 2019-05-29 at 13:43 +0200, Dan wrote:
> Hi Jonathan,
> 
> On Tue, May 28, 2019 at 8:50 PM Jonathan Carter <jcc@debian.org> wrote:
> > On 2019/05/28 18:43, Dan wrote:
> > > ZFS 0.8 has been released with lots of improvements, notably encryption.
> > 
> > Yep, it's an exciting feature.
> > 
> > > Sadly the Linux Kernel has introduced a commit in kernel 4.19 and 5.0
> > > that prevents ZFS from using SIMD. The result is that ZFS won't be
> > > usable in Buster. See the following issue
> > > https://github.com/zfsonlinux/zfs/issues/8793
> > 
> > Buster ships zfs-dkms version 0.7.12-2 though, which works just fine on
> > buster.
> 
> My message was not accurate. I think that the commit has been
> introduced in in 4.19.38 (released 2019-05-02). I think that Debian
> Buster uses 4.19.37
> 
> https://cdn.kernel.org/pub/linux/kernel/v4.x/ChangeLog-4.19.38
> 
> The commit also affects ZFS 0.7 because SIMD is used for checksum operations.
> 
> There might be a performance penalty in ZFS only if Debian Buster
> upgrades to 4.19.38.

Which we will, some time soon.

Ben.

-- 
Ben Hutchings
Horngren's Observation:
              Among economists, the real world is often a special case.

[toc] | [prev] | [next] | [standalone]

#64226

Hi Zigo

On Thu, Jun 06, 2019 at 02:43:16PM +0200, Thomas Goirand wrote:
> In such case, would you consider maintaining this tiny patch?
> https://github.com/NixOS/nixpkgs/pull/61076/commits/7b77c27caa8617c82df5c5af6b9ce6ae010d7f9a

Please read https://bugs.debian.org/929557.

Thanks for following all security precautions.

Bastian

-- 
I've already got a female to worry about.  Her name is the Enterprise.
		-- Kirk, "The Corbomite Maneuver", stardate 1514.0

[toc] | [prev] | [next] | [standalone]

#64202

On Tue, May 28, 2019 at 06:43:55PM +0200, Dan wrote:
> 
> The ZFS developers proposed the Linux developers to rewrite the whole
> ZFS code and use GPL, but surprisingly the linux developers didn't
> accept. See below:
> https://github.com/zfsonlinux/zfs/issues/8314

I've read the thread, and there are a lot of misunderstandings of many
of the key people involved.  There also seems to be a lot of
misunderstanding of what the cause of the "hostility" is coming from
--- it is *not* about "sticking it to Oracle because they chose the
CDDL".

Also, it's not accurate that "linux developers didn't accept".  Ryan
sent a query to Linus, and Linus didn't respond.  I don't know if he
sent a single message, or whether he retried a couple of times.  A
failure to respond is not the same as a rejection.  There are plenty
of reasons why Linus might not have responded.

That being said, I don't propose to relitigate that whole thread here.
If people really care, feel free to contact me privately.  Or it could
be the case that since Ryan has closed, the ZoL community has already
moved on.  Which is also a fine outcome: from most of the upstream
Linux developers that I've talked to; not because they hold any
particular animus against ZoL.  It's just that no one feels
particularly interested in giving ZoL any kind of special treatment
--- the hostility around bypassing the requirements of GPL is about
exactly that; not the identity of the company or project trying to do
those particular things.

As Sam has noted, even in the most permissive interpretation, which is
that the Kernel has chosen to draw the lines around GPL compliance in
a different place as the FSF, does not mean that there are *no* lines.
Indeed, there are lines, and when they are violated, there will be
hostility and a refusal to cooperate, and ZoL is getting no better
*or* no worse treatment in that regard.

Bringing this back to Debian, my perception is that while there is not
unanimity about how the moral and legal requirements of the GPL
should be understood within Debian (just as there is also not
unanimity in the kernel community), the center of gravity within
Debian tends to be weighted towards the less permissive
interpretations of the GPL compared to the Linux Kernel community as
whole.  Which is to say, if you can't get the Linux Kernel community
folks to agree towards a certain flexibility towards evading the
CDDL/GPL license compatibility problems using techniques like "GPL
condoms", it is even less likely that the Debian community is going to
be willing to be so accomodating.

I also agree with Sam that the only way to know for sure is to have a
GR.  So you don't have to take our word for it; but please do
understand it's going to take a lot of community resources to make
that determination.  And there might be better uses of that time and
energy.

Regards,

					- Ted

[toc] | [prev] | [next] | [standalone]

#64211

Hi Mo and Theodore,

On Sun, Jun 2, 2019 at 4:04 AM Theodore Ts'o <tytso@mit.edu> wrote:

> Also, it's not accurate that "linux developers didn't accept".  Ryan
> sent a query to Linus, and Linus didn't respond.  I don't know if he
> sent a single message, or whether he retried a couple of times.  A
> failure to respond is not the same as a rejection.  There are plenty
> of reasons why Linus might not have responded.

Without a long-term agreement between the Linux Kernel and the OpenZFS
project the whole community will suffer. Opensource is about finding
paths to collaborate.

Now lustre is based on ZFS, this kernel regression will impact the HPC
community.

I'm not a lawyer, but I think that OracleZFS benefits from the
improvements made by OpenZFS. If there is an agreement between the
Linux Kernel and the OpenZFS project to convert OpenZFS to GPL, then
Oracle won't benefit any more from OpenZFS and they will be forced to
release the original ZFS code as GPL.

Below you can find an excerpt from the GitHub discussion.

On Jan 19 , 2019 Ryan wrote:

> Recent events WRT Linux 5.0 have made me reconsider user requests
> to pursue mainline inclusion. Linus Torvalds told me in person in 2014
> that he requires signed off from Oracle to merge the code.
> That is not happening, but it occurs to me that it should be possible to
> replace all Oracle copyrighted kernel code with new code over a long
> period of time (several years). This would bypass the need for Oracle’s
> signed off. It would also give us the ability to switch the kernel module
> to a dual CDDL/GPL.

> I suspect that the Illumos community would find the GPL
> (or even the LGPL) more acceptable than a BSD license. When
> Illumos was started, they stated that they do not want Oracle to be
> able to use their code without Oracle giving their changes back.

I understand that ZoL would like to convert OpenZFS to GPL, but they
will need the help and support of the Linux Kernel developers.


On Mon, Jun 3, 2019 at 4:47 PM Mo Zhou <lumin@debian.org> wrote:
>
> 0.7.12-2 works with 4.19.37, but it will be badly broken when the first
> point release of Buster is out. Foreseeable stable RC is grave enough:
>   https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=929929
>

This Linux kernel regression will break lot of computers and people
might loose data. Most of the users will have no clue why their
computer is not working and will blame Debian and the opensource
software.

Best,
Daniel

[toc] | [prev] | [next] | [standalone]

#64212

Hi.

Thanks for bringing up this issue originally.
I think it has started some good discussion with the Debian zfs
maintainers.

However, I think this particular subthread about zfs has served its
purpose.

I cannot find anything in your message that is on topic for the
debian-devel mailing list.

Rehashing github discussions  between two non-debian parties doesn't
really seem of central import to the Debian developer community.

Discussing what the open zfs and kernel community should do and how they
should work together is certainly off-topic for debian-devel.

There are certain zfs discussions that might be on-topic for
debian-devel, but this particular subthread does not appear to be one of
them.
Thanks,

--Sam

[toc] | [prev] | [standalone]

Back to top | Article view | linux.debian.kernel

csiph-web