Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]


Groups > linux.debian.kernel > #56016 > unrolled thread

Bug#847204: nfs-kernel-server: `systemctl status` incorrectly reports server "active" even if not started

Started byRiccardo Murri <riccardo.murri@gmail.com>
First post2016-12-06 15:20 +0100
Last post2016-12-19 11:00 +0100
Articles 3 — 3 participants

Back to article view | Back to linux.debian.kernel


Contents

  Bug#847204: nfs-kernel-server: `systemctl status` incorrectly reports server "active" even if not started Riccardo Murri <riccardo.murri@gmail.com> - 2016-12-06 15:20 +0100
    Bug#847204: nfs-kernel-server: `systemctl status` incorrectly reports server "active" even if not started Michael Biebl <biebl@debian.org> - 2016-12-18 23:50 +0100
      Bug#847204: nfs-kernel-server: `systemctl status` incorrectly reports server "active" even if not started Christian Seiler <christian@iwakd.de> - 2016-12-19 11:00 +0100

#56016 — Bug#847204: nfs-kernel-server: `systemctl status` incorrectly reports server "active" even if not started

FromRiccardo Murri <riccardo.murri@gmail.com>
Date2016-12-06 15:20 +0100
SubjectBug#847204: nfs-kernel-server: `systemctl status` incorrectly reports server "active" even if not started
Message-ID<sLoL7-39C-13@gated-at.bofh.it>
Package: nfs-kernel-server
Version: 1:1.2.8-9

The systemd service unit file for `nfs-kernel-server` incorrectly
reports the service status as "active" even when the NFS server is *not*
running (e.g., empty exports file).

This causes scripts (e.g., Ansible) that use `systemctl status` to check
for the NFS server to fail in a way that is difficult to debug because
the actual reported error might be in a loosely-related task (e.g., when
doing `exportfs -r` or when mounting fs from a client).

Steps to reproduce on a Debian "jessie" VM:

* starting from a basic "jessie" install, install the `nfs-kernel-server`::

    debian@debian-nfs-bug:~$ sudo apt install nfs-kernel-server
    [...]
    The following extra packages will be installed:
      libevent-2.0-5 libldap-2.4-2 libnfsidmap2 libsasl2-2 libsasl2-modules libsasl2-modules-db libtirpc1 nfs-common rpcbind
    Suggested packages:
      libsasl2-modules-otp libsasl2-modules-ldap libsasl2-modules-sql libsasl2-modules-gssapi-mit libsasl2-modules-gssapi-heimdal open-iscsi watchdog
    The following NEW packages will be installed:
      libevent-2.0-5 libldap-2.4-2 libnfsidmap2 libsasl2-2 libsasl2-modules libsasl2-modules-db libtirpc1 nfs-common nfs-kernel-server rpcbind
    0 upgraded, 10 newly installed, 0 to remove and 0 not upgraded.
    Need to get 1123 kB of archives.
    After this operation, 3018 kB of additional disk space will be used.
    Do you want to continue? [Y/n]
    Get:1 http://http.debian.net/debian/ jessie/main libevent-2.0-5 amd64 2.0.21-stable-2 [152 kB]
    [...]
    Setting up libnfsidmap2:amd64 (0.25-5) ...
    Setting up libtirpc1:amd64 (0.2.5-1) ...
    Setting up rpcbind (0.2.1-6+deb8u1) ...
    Setting up nfs-common (1:1.2.8-9) ...

    Creating config file /etc/idmapd.conf with new version
    Adding system user `statd' (UID 106) ...
    Adding new user `statd' (UID 106) with group `nogroup' ...
    Not creating home directory `/var/lib/nfs'.
    Setting up libsasl2-modules:amd64 (2.1.26.dfsg1-13+deb8u1) ...
    Setting up nfs-kernel-server (1:1.2.8-9) ...

    Creating config file /etc/exports with new version

    Creating config file /etc/default/nfs-kernel-server with new version
    Processing triggers for libc-bin (2.19-18+deb8u6) ...
    Processing triggers for systemd (215-17+deb8u5) ...

* note that the NFS server has *not* been started yet, as the rpcbind
  daemon does not know about it::

    debian@debian-nfs-bug:~$ sudo rpcinfo -p
       program vers proto   port  service
        100000    4   tcp    111  portmapper
        100000    3   tcp    111  portmapper
        100000    2   tcp    111  portmapper
        100000    4   udp    111  portmapper
        100000    3   udp    111  portmapper
        100000    2   udp    111  portmapper
        100024    1   udp  56360  status
        100024    1   tcp  39499  status

  Still, `systemctl status` reports it already as loaded/active::

    debian@debian-nfs-bug:~$ sudo systemctl status nfs-kernel-server.service
    * nfs-kernel-server.service - LSB: Kernel NFS server support
       Loaded: loaded (/etc/init.d/nfs-kernel-server)
       Active: active (exited) since Tue 2016-12-06 12:12:58 UTC; 1min 13s ago

    Dec 06 12:12:58 debian-nfs-bug nfs-kernel-server[1544]: Not starting NFS kernel daemon: no exports. ... (warning).
    Dec 06 12:12:58 debian-nfs-bug systemd[1]: Started LSB: Kernel NFS server support.

* now add an export and try to start it again::

    debian@debian-nfs-bug:~$ echo "/home localhost(rw,async.no_subtree_check)" | sudo tee -a /etc/exports
    /home localhost(rw,async.no_subtree_check)

    debian@debian-nfs-bug:~$ sudo systemctl start nfs-kernel-server

  (last command produces no output)

* `systemctl status` reports the server as loaded/active (green on
  terminal, so "ok")::

    debian@debian-nfs-bug:~$ sudo systemctl status nfs-kernel-server.service
    * nfs-kernel-server.service - LSB: Kernel NFS server support
       Loaded: loaded (/etc/init.d/nfs-kernel-server)
       Active: active (exited) since Tue 2016-12-06 12:12:58 UTC; 1min 36s ago

    Dec 06 12:12:58 debian-nfs-bug nfs-kernel-server[1544]: Not starting NFS kernel daemon: no exports. ... (warning).
    Dec 06 12:12:58 debian-nfs-bug systemd[1]: Started LSB: Kernel NFS server support.
    Dec 06 12:14:33 debian-nfs-bug systemd[1]: Started LSB: Kernel NFS server support.

* still `exportfs -r` fails with a strange error message (due to
  `/proc/fs/nfsd` not being mounted) and the NFS server is not running nor
  registered with rpcbind/portmapper::

    debian@debian-nfs-bug:~$ sudo exportfs -r
    exportfs: localhost:/home: Function not implemented

    debian@debian-nfs-bug:~$ sudo showmount -e localhost
    clnt_create: RPC: Program not registered
    debian@debian-nfs-bug:~$ sudo rpcinfo -p
       program vers proto   port  service
        100000    4   tcp    111  portmapper
        100000    3   tcp    111  portmapper
        100000    2   tcp    111  portmapper
        100000    4   udp    111  portmapper
        100000    3   udp    111  portmapper
        100000    2   udp    111  portmapper
        100024    1   udp  56360  status
        100024    1   tcp  39499  status

On the other hand, `systemctl restart nfs-kernel-server` (or a reboot)
does actually start the NFS server.

System information:

    $ lsb_release -a
    No LSB modules are available.
    Distributor ID:	Debian
    Description:	Debian GNU/Linux 8.6 (jessie)
    Release:	8.6
    Codename:	jessie

    $ dpkg -l nfs-kernel-server nfs-common linux-image-3.16.0-4-amd64
    Desired=Unknown/Install/Remove/Purge/Hold
    | Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
    |/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
    ||/ Name                                      Version                   Architecture              Description
    +++-=========================================-=========================-=========================-=======================================================================================
    ii  linux-image-3.16.0-4-amd64                3.16.36-1+deb8u1          amd64                     Linux 3.16 for 64-bit PCs
    ii  nfs-common                                1:1.2.8-9                 amd64                     NFS support files common to client and server
    ii  nfs-kernel-server                         1:1.2.8-9                 amd64                     support for NFS kernel server

    $ uname -a
    Linux debian-nfs-bug 3.16.0-4-amd64 #1 SMP Debian 3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux


Thanks,
Riccardo

[toc] | [next] | [standalone]


#56286

FromMichael Biebl <biebl@debian.org>
Date2016-12-18 23:50 +0100
Message-ID<sPSrf-7RA-5@gated-at.bofh.it>
In reply to#56016

[Multipart message — attachments visible in raw view] — view raw

On Tue, 6 Dec 2016 15:15:12 +0100 Riccardo Murri
<riccardo.murri@gmail.com> wrote:
> 
> Package: nfs-kernel-server
> Version: 1:1.2.8-9
> 
> The systemd service unit file for `nfs-kernel-server` incorrectly
> reports the service status as "active" even when the NFS server is *not*
> running (e.g., empty exports file).

..

>     debian@debian-nfs-bug:~$ sudo systemctl status nfs-kernel-server.service
>     * nfs-kernel-server.service - LSB: Kernel NFS server support
>        Loaded: loaded (/etc/init.d/nfs-kernel-server)
>        Active: active (exited) since Tue 2016-12-06 12:12:58 UTC; 1min 36s ago
> 
>     Dec 06 12:12:58 debian-nfs-bug nfs-kernel-server[1544]: Not starting NFS kernel daemon: no exports. ... (warning).
>     Dec 06 12:12:58 debian-nfs-bug systemd[1]: Started LSB: Kernel NFS server support.
>     Dec 06 12:14:33 debian-nfs-bug systemd[1]: Started LSB: Kernel NFS server support.

The problem is, that nfs-kernel-server in jessie is a sysv init script.
For those, systemd creates a wrapper unit which uses RemainAfterExit=yes
as systemd can not know, if the init script starts a long running
process or not.

(that's why it shows "active (exited)" instead of "active (running)")

The obvious solution is to use a native .service file, where you can set
the proper Type=. This is the case for stretch.

For jessie, that change would most likely be too invasive though,


Michael

-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?

[toc] | [prev] | [next] | [standalone]


#56291

FromChristian Seiler <christian@iwakd.de>
Date2016-12-19 11:00 +0100
Message-ID<sQ2TE-63S-9@gated-at.bofh.it>
In reply to#56286
On 12/18/2016 11:41 PM, Michael Biebl wrote:
> On Tue, 6 Dec 2016 15:15:12 +0100 Riccardo Murri
> <riccardo.murri@gmail.com> wrote:
>>
>> Package: nfs-kernel-server
>> Version: 1:1.2.8-9
>>
>> The systemd service unit file for `nfs-kernel-server` incorrectly
>> reports the service status as "active" even when the NFS server is *not*
>> running (e.g., empty exports file).
> 
> ..
> 
>>     debian@debian-nfs-bug:~$ sudo systemctl status nfs-kernel-server.service
>>     * nfs-kernel-server.service - LSB: Kernel NFS server support
>>        Loaded: loaded (/etc/init.d/nfs-kernel-server)
>>        Active: active (exited) since Tue 2016-12-06 12:12:58 UTC; 1min 36s ago
>>
>>     Dec 06 12:12:58 debian-nfs-bug nfs-kernel-server[1544]: Not starting NFS kernel daemon: no exports. ... (warning).
>>     Dec 06 12:12:58 debian-nfs-bug systemd[1]: Started LSB: Kernel NFS server support.
>>     Dec 06 12:14:33 debian-nfs-bug systemd[1]: Started LSB: Kernel NFS server support.
> 
> The problem is, that nfs-kernel-server in jessie is a sysv init script.
> For those, systemd creates a wrapper unit which uses RemainAfterExit=yes
> as systemd can not know, if the init script starts a long running
> process or not.
> 
> (that's why it shows "active (exited)" instead of "active (running)")
> 
> The obvious solution is to use a native .service file, where you can set
> the proper Type=. This is the case for stretch.

On Stretch this doesn't help though, because of the way the NFS kernel
server works.

On Jessie (and before) the nfs-kernel-server init script would start
all sorts of daemons (such as rpc.svcgssd) that are auxiliary to the
NFS server, but at least for NFSv4 the NFS server itself is a kernel
thread.

It is started via rpc.nfsd $N, which tells the kernel to start $N
kernel threads, and then exits immediately. It stops via rpc.nfsd 0.

The problem here is that there's no userland process that is kept
running here. So that means that the systemd service also has to
have RemainAfterExit=yes to make this work.

And indeed, the upstream systemd unit (which will be used by the
Debian package in Stretch) indeed has that setting set:

https://sources.debian.net/src/nfs-utils/1:1.3.4-2/systemd/nfs-server.service/

I haven't tested the Stretch package yet, so I don't know how that
reacts to an empty /etc/exports, so maybe this specific bug doesn't
occur there any more.

However, if something else manually calls rpc.nfsd 0 during operation,
the service will be stopped, but systemd won't recognize it.

That all said, for the empty /etc/exports case, for Jessie alone:

> For jessie, that change would most likely be too invasive though,

A workaround for Jessie could be to create a file

/etc/systemd/system/nfs-kernel-server.d/stop-if-empty-exports.conf

with the following contents:

[Service]
ExecStartPost=/usr/local/sbin/stop_nfs_systemd_service_if_no_exports.sh

And the contents of that script being something like:

#!/bin/sh

if ... same check as in the init script that /etc/exports is empty ... ; then
  # Enqueue a job for systemd to stop the service
  systemctl stop --no-block nfs-kernel-server.service
fi

(For the Debian package one would obviously use different directories
for these files.)

This won't fix the problem entirely, but at least after boot the
nfs server will be considered stopped.

Regards,
Christian

[toc] | [prev] | [standalone]


Back to top | Article view | linux.debian.kernel


csiph-web