Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.bugs.dist > #1023115 > unrolled thread

Bug#969223: Can't rm directory on overlayfs in userns

Back to article view | Back to linux.debian.bugs.dist

Contents

  Bug#969223: Can't rm directory on overlayfs in userns Shengjing Zhu <zhsj@debian.org> - 2020-08-29 16:20 +0200
    Bug#969223: Can't rm directory on overlayfs in userns Shengjing Zhu <zhsj@debian.org> - 2020-09-02 06:00 +0200
      Bug#969223: Can't rm directory on overlayfs in userns Nicolas Schier <nicolas@fjasle.eu> - 2020-09-16 10:10 +0200
        Bug#969223: Can't rm directory on overlayfs in userns Shengjing Zhu <zhsj@debian.org> - 2020-09-16 19:00 +0200
          Bug#969223: Can't rm directory on overlayfs in userns Nicolas Schier <nicolas@fjasle.eu> - 2020-09-16 21:00 +0200
            Bug#969223: Can't rm directory on overlayfs in userns Shengjing Zhu <zhsj@debian.org> - 2020-09-16 21:10 +0200

#1023115 — Bug#969223: Can't rm directory on overlayfs in userns

Source: linux
Version: 5.7.10-1
Severity: normal

Hi,

After enabling overlayfs for userns, I find it doesn't work as expected.

$ cat /sys/module/overlay/parameters/permit_mounts_in_userns 
Y

zsj@debian:~/test$ pwd
/home/zsj/test
zsj@debian:~/test$ tree
.
├── lower
│   └── a
│       └── a
├── merged
├── upper
└── work

zsj@debian:~/test$ unshare -m -U -r
root@debian:~/test# mount -t overlay -o rw,lowerdir=/home/zsj/test/lower,upperdir=/home/zsj/test/upper,workdir=/home/zsj/test/work overlay /home/zsj/test/merged
root@debian:~/test# rm -rf merged/a
rm: cannot remove 'merged/a': Input/output error

-- System Information:
Debian Release: bullseye/sid
  APT prefers testing
  APT policy: (500, 'testing')
Architecture: amd64 (x86_64)

Kernel: Linux 5.7.0-2-amd64 (SMP w/4 CPU threads)
Kernel taint flags: TAINT_USER, TAINT_FIRMWARE_WORKAROUND
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8), LANGUAGE not set
Shell: /bin/sh linked to /usr/bin/dash
Init: systemd (via /run/systemd/system)

[toc] | [next] | [standalone]

#1023489

On Sat, Aug 29, 2020 at 10:13 PM Shengjing Zhu <zhsj@debian.org> wrote:
>
> Source: linux
> Version: 5.7.10-1
> Severity: normal
>
> Hi,
>
> After enabling overlayfs for userns, I find it doesn't work as expected.
>
> $ cat /sys/module/overlay/parameters/permit_mounts_in_userns
> Y
>
> zsj@debian:~/test$ pwd
> /home/zsj/test
> zsj@debian:~/test$ tree
> .
> ├── lower
> │   └── a
> │       └── a
> ├── merged
> ├── upper
> └── work
>
> zsj@debian:~/test$ unshare -m -U -r
> root@debian:~/test# mount -t overlay -o rw,lowerdir=/home/zsj/test/lower,upperdir=/home/zsj/test/upper,workdir=/home/zsj/test/work overlay /home/zsj/test/merged
> root@debian:~/test# rm -rf merged/a
> rm: cannot remove 'merged/a': Input/output error
>
> -- System Information:
> Debian Release: bullseye/sid
>   APT prefers testing
>   APT policy: (500, 'testing')
> Architecture: amd64 (x86_64)
>
> Kernel: Linux 5.7.0-2-amd64 (SMP w/4 CPU threads)
> Kernel taint flags: TAINT_USER, TAINT_FIRMWARE_WORKAROUND
> Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8), LANGUAGE not set
> Shell: /bin/sh linked to /usr/bin/dash
> Init: systemd (via /run/systemd/system)

If I upgrade a debian10 VM to testing, it seems to work.
However if I boot a new debian testing VM, it seems not to work.
Both VMs are downloaded from http://cdimage.debian.org/cdimage/cloud/
What can be the difference here? I'm lost on debugging this..

-- 
Shengjing Zhu

[toc] | [prev] | [next] | [standalone]

#1025278

On Wed, Sep 02, 2020 at 11:52:41AM +0800, Shengjing Zhu wrote:
> On Sat, Aug 29, 2020 at 10:13 PM Shengjing Zhu <zhsj@debian.org> wrote:
> >
> > Source: linux
> > Version: 5.7.10-1
> > Severity: normal
> >
> > Hi,
> >
> > After enabling overlayfs for userns, I find it doesn't work as expected.
> >
> > $ cat /sys/module/overlay/parameters/permit_mounts_in_userns
> > Y
> >
> > zsj@debian:~/test$ pwd
> > /home/zsj/test
> > zsj@debian:~/test$ tree
> > .
> > ├── lower
> > │   └── a
> > │       └── a
> > ├── merged
> > ├── upper
> > └── work
> >
> > zsj@debian:~/test$ unshare -m -U -r
> > root@debian:~/test# mount -t overlay -o rw,lowerdir=/home/zsj/test/lower,upperdir=/home/zsj/test/upper,workdir=/home/zsj/test/work overlay /home/zsj/test/merged
> > root@debian:~/test# rm -rf merged/a
> > rm: cannot remove 'merged/a': Input/output error
> >

Hi,

overlayfs uses filesystem xattrs to mark "whiteouts" and redirects of
directories, which are only accessable for root (CAP_SYS_ADMIN), thus,
not when overlay is mounted in a user namespace, cp. e.g. [1,2].

Ubuntu kernel "solves" this by skipping the "trusted."-xattr check, thus
allowing setting and removal of 'trusted.overlay.*' xattrs from within
user namespaces; but those are still visible in all other namespaces.  A
following overlayfs mount done by the real root user will use these
modified xattrs.

To me it would seem to be more adequate if overlayfs would use
'overlay.*' instead of 'trusted.overlay.*', if it is mounted in an
unpriviledged user namespace.  But this would make overlay mounts done
by root incompatible with those done in a user namespace.

Maybe you find #836211 to be related to this.


[1]: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/tree/fs/xattr.c?h=linux-5.7.y#n113
[2]: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/tree/fs/xattr.c?h=linux-5.7.y#n1049
[3]: https://kernel.ubuntu.com/git/ubuntu/ubuntu-focal.git/commit/?id=111cd1a9840ce187e28b49fe4e77b9b5e84386b1

> If I upgrade a debian10 VM to testing, it seems to work.
> However if I boot a new debian testing VM, it seems not to work.
> Both VMs are downloaded from http://cdimage.debian.org/cdimage/cloud/
> What can be the difference here? I'm lost on debugging this..

This confuses me.  Are you sure, you used the same kernel version on
both VMs when mounting overlayfs in userns?

Kind regards,
Nicolas

-- 
epost: nicolas@fjasle.eu               irc://oftc.net/nsc
↳ gpg: 18ed 52db e34f 860e e9fb  c82b 7d97 0932 55a0 ce7f
     -- frykten for herren er opphav til kunnskap --

[toc] | [prev] | [next] | [standalone]

#1025347

On Wed, Sep 16, 2020 at 3:58 PM Nicolas Schier <nicolas@fjasle.eu> wrote:
> > If I upgrade a debian10 VM to testing, it seems to work.
> > However if I boot a new debian testing VM, it seems not to work.
> > Both VMs are downloaded from http://cdimage.debian.org/cdimage/cloud/
> > What can be the difference here? I'm lost on debugging this..
>
> This confuses me.  Are you sure, you used the same kernel version on
> both VMs when mounting overlayfs in userns?
>

I think I just mess up when debugging. It seems it never works.

Maybe we should revert permit_mounts_in_userns? as it doesn't seem to
work. Buster is also affected.

-- 
Shengjing Zhu

[toc] | [prev] | [next] | [standalone]

#1025363

> I think I just mess up when debugging. It seems it never works.
> 
> Maybe we should revert permit_mounts_in_userns? as it doesn't seem to
> work. Buster is also affected.

Please, don't be too fast when thinking about a revert.  Several of my
colleagues (Debian users) cling to the feature since they need it for
using the company's LXC containers; if permit_mounts_in_userns is
removed again, they might be forced to switch to non-Debian kernels or
to live-patch the kernel with fragile stuff like [1], cp. #913880.


[1]: https://rocketgit.com/user/nicolas/overlay-userns-dkms

-- 
epost: nicolas@fjasle.eu               irc://oftc.net/nsc
↳ gpg: 18ed 52db e34f 860e e9fb  c82b 7d97 0932 55a0 ce7f
     -- frykten for herren er opphav til kunnskap --

[toc] | [prev] | [next] | [standalone]

#1025364

On Thu, Sep 17, 2020 at 2:52 AM Nicolas Schier <nicolas@fjasle.eu> wrote:
>
> > I think I just mess up when debugging. It seems it never works.
> >
> > Maybe we should revert permit_mounts_in_userns? as it doesn't seem to
> > work. Buster is also affected.
>
> Please, don't be too fast when thinking about a revert.  Several of my
> colleagues (Debian users) cling to the feature since they need it for
> using the company's LXC containers; if permit_mounts_in_userns is
> removed again, they might be forced to switch to non-Debian kernels or
> to live-patch the kernel with fragile stuff like [1], cp. #913880.

I mean if you can't even remove a directory with files, it's too broken to use.
So your colleagues find the userns overlay works?
Or you mean we should take Ubuntu's patch to fix the issue?

-- 
Shengjing Zhu

[toc] | [prev] | [standalone]

Back to top | Article view | linux.debian.bugs.dist

csiph-web