Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.kernel > #84939 > unrolled thread

Bug#1091439:

Back to article view | Back to linux.debian.kernel

Contents

  Bug#1091439:  A CHAUMEAUX <alfrchx@gmail.com> - 2024-12-26 11:10 +0100
    Bug#1091439: installation of nfs-kernel-server hangs Salvatore Bonaccorso <carnil@debian.org> - 2024-12-26 16:00 +0100
    Processed: Re: Bug#1091439: installation of nfs-kernel-server hangs "Debian Bug Tracking System" <owner@bugs.debian.org> - 2024-12-26 16:00 +0100
    Bug#1091439: kernel BUG at fs/nfsd/nfs4recover.c:534 Oops: invalid opcode: 0000 Salvatore Bonaccorso <carnil@debian.org> - 2024-12-27 07:40 +0100

#84939 — Bug#1091439:

[Multipart message — attachments visible in raw view] — view raw

Package: nfs-kernel-server
Version: 1:2.8.2-1

Install of nfs-kernel-server hangs before the end. Last messages :
nfs-mountd.service is a disable or a static unit, not starting it
lfsdcld.service is a disable or statis init, not starting it.

 Check was done  on a new-installed system, just after the first boot. The
problem appears after the changes on iptables;
nfs-kerne-server works whitout any problem on the same machine with others
systems installed before the changes of the iptables.
System: Debian trixie installed on 12-26

[toc] | [next] | [standalone]

#84942 — Bug#1091439: installation of nfs-kernel-server hangs

Control: tags -1 + moreinfo

Hi,

On Thu, Dec 26, 2024 at 11:04:48AM +0100, A CHAUMEAUX wrote:
> Package: nfs-kernel-server
> Version: 1:2.8.2-1
> 
> Install of nfs-kernel-server hangs before the end. Last messages :
> nfs-mountd.service is a disable or a static unit, not starting it
> lfsdcld.service is a disable or statis init, not starting it.
> 
>  Check was done  on a new-installed system, just after the first boot. The
> problem appears after the changes on iptables;
> nfs-kerne-server works whitout any problem on the same machine with others
> systems installed before the changes of the iptables.
> System: Debian trixie installed on 12-26

I believe I can reproduce a problem, and the hang of the installation
is not the root cause, can you confirm you see a kernel splat
something like:

[   35.650181] NFSD: Using /var/lib/nfs/v4recovery as the NFSv4 state recovery directory
[   35.650213] NFSD: Using legacy client tracking operations.
[   35.650217] NFSD: Using /var/lib/nfs/v4recovery as the NFSv4 state recovery directory
[   35.650239] ------------[ cut here ]------------
[   35.650241] kernel BUG at fs/nfsd/nfs4recover.c:534!
[   35.651311] Oops: invalid opcode: 0000 [#1] PREEMPT SMP NOPTI
[   35.652455] CPU: 0 UID: 0 PID: 3555 Comm: nfsdctl Not tainted 6.12.6-amd64 #1  Debian 6.12.6-1
[   35.654108] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.16.2-debian-1.16.2-1 04/01/2014
[   35.655872] RIP: 0010:nfsd4_legacy_tracking_init+0x17d/0x1b0 [nfsd]
[   35.657153] Code: 19 48 89 de 48 c7 c7 50 89 da c0 e8 6d fb ff ff 89 c5 85 c0 0f 85 10 61 00 00 48 c7 c7 90 e3 e1 c
0 31 ed e8 85 36 ef c8 eb 07 <0f> 0b bd f4 ff ff ff 48 8b 44 24 08 65 48 2b 04 25 28 00 00 00 75
[   35.660568] RSP: 0018:ffffaee0804e3890 EFLAGS: 00010282
[   35.661619] RAX: 0000000000000049 RBX: ffff9c93c261a000 RCX: 0000000000000027
[   35.663006] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9c943bc21780
[   35.664383] RBP: ffffffff8b8a3600 R08: 0000000000000000 R09: ffffaee0804e3720
[   35.665805] R10: ffffffff8acb4348 R11: 0000000000000003 R12: ffff9c93c261a000
[   35.667316] R13: ffff9c93c261a000 R14: ffff9c93c18f9b40 R15: ffff9c93c261a000
[   35.668813] FS:  00007fcdd816c680(0000) GS:ffff9c943bc00000(0000) knlGS:0000000000000000
[   35.670491] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[   35.671728] CR2: 00007f191721c280 CR3: 00000001029e8000 CR4: 0000000000350ef0
[   35.673220] Call Trace:
[   35.673826]  <TASK>
[   35.674376]  ? __die_body.cold+0x19/0x27
[   35.675295]  ? die+0x2e/0x50
[   35.675984]  ? do_trap+0xca/0x110
[   35.676761]  ? do_error_trap+0x6a/0x90
[   35.677629]  ? nfsd4_legacy_tracking_init+0x17d/0x1b0 [nfsd]
[   35.678915]  ? exc_invalid_op+0x50/0x70
[   35.679800]  ? nfsd4_legacy_tracking_init+0x17d/0x1b0 [nfsd]
[   35.681049]  ? asm_exc_invalid_op+0x1a/0x20
[   35.682024]  ? nfsd4_legacy_tracking_init+0x17d/0x1b0 [nfsd]
[   35.683290]  nfsd4_client_tracking_init+0x57/0x1b0 [nfsd]
[   35.684493]  nfs4_state_start_net+0x2f9/0x3a0 [nfsd]
[   35.685614]  nfsd_svc+0x1ac/0x310 [nfsd]
[   35.686565]  nfsd_nl_threads_set_doit+0x1a1/0x2f0 [nfsd]
[   35.687775]  genl_family_rcv_msg_doit+0xf2/0x150
[   35.688813]  genl_rcv_msg+0x1b7/0x2c0
[   35.689646]  ? __pfx_nfsd_nl_threads_set_doit+0x10/0x10 [nfsd]
[   35.690966]  ? __pfx_genl_rcv_msg+0x10/0x10
[   35.691903]  netlink_rcv_skb+0x53/0x100
[   35.692771]  genl_rcv+0x28/0x40
[   35.693528]  netlink_unicast+0x245/0x390
[   35.694424]  netlink_sendmsg+0x21b/0x470
[   35.695336]  ? aa_sk_perm+0x46/0x210
[   35.696169]  ____sys_sendmsg+0x3a0/0x3d0
[   35.697055]  ? srso_return_thunk+0x5/0x5f
[   35.697979]  ___sys_sendmsg+0x9a/0xe0
[   35.698832]  __sys_sendmsg+0x7a/0xd0
[   35.699489]  do_syscall_64+0x82/0x190
[   35.700148]  ? srso_return_thunk+0x5/0x5f
[   35.700856]  ? do_syscall_64+0x8e/0x190
[   35.701534]  ? srso_return_thunk+0x5/0x5f
[   35.702244]  ? srso_return_thunk+0x5/0x5f
[   35.702964]  entry_SYSCALL_64_after_hwframe+0x76/0x7e
[   35.703826] RIP: 0033:0x7fcdd82b4970
[   35.704480] Code: 00 f7 d8 64 89 02 b8 ff ff ff ff eb bb 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 00 80 3d f9 c6 0d 00 0
0 74 17 b8 2e 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 58 c3 0f 1f 80 00 00 00 00 48 83 ec 28 89 54
[   35.707394] RSP: 002b:00007ffc3d528d68 EFLAGS: 00000202 ORIG_RAX: 000000000000002e
[   35.708638] RAX: ffffffffffffffda RBX: 000055f42fa93390 RCX: 00007fcdd82b4970
[   35.709792] RDX: 0000000000000000 RSI: 00007ffc3d528da0 RDI: 0000000000000003
[   35.710940] RBP: 000055f42fa952d0 R08: 0000000000000000 R09: 000000000000000d
[   35.711677] R10: 000055f42fa93010 R11: 0000000000000202 R12: 000055f42fa932a0
[   35.712415] R13: 00007ffc3d528da0 R14: 000055f42fa953a4 R15: 0000000000000000
[   35.713160]  </TASK>
[   35.713433] Modules linked in: nfsd auth_rpcgss nfs_acl lockd grace sunrpc intel_rapl_msr intel_rapl_common kvm_amd
 ccp kvm crct10dif_pclmul ghash_clmulni_intel sha512_ssse3 sha256_ssse3 sha1_ssse3 aesni_intel iTCO_wdt intel_pmc_bxt
iTCO_vendor_support gf128mul watchdog crypto_simd cryptd pcspkr virtio_balloon button virtio_console evdev joydev seri
o_raw drm efi_pstore configfs nfnetlink vsock_loopback vmw_vsock_virtio_transport_common vmw_vsock_vmci_transport vsoc
k vmw_vmci qemu_fw_cfg virtio_rng ip_tables x_tables autofs4 ext4 crc16 mbcache jbd2 crc32c_generic dm_mod ahci libahc
i libata xhci_pci xhci_hcd scsi_mod psmouse virtio_blk virtio_net usbcore crc32_pclmul i2c_i801 crc32c_intel lpc_ich i
2c_smbus scsi_common net_failover failover usb_common
[   35.719907] ---[ end trace 0000000000000000 ]---
[   35.720418] RIP: 0010:nfsd4_legacy_tracking_init+0x17d/0x1b0 [nfsd]
[   35.721115] Code: 19 48 89 de 48 c7 c7 50 89 da c0 e8 6d fb ff ff 89 c5 85 c0 0f 85 10 61 00 00 48 c7 c7 90 e3 e1 c
0 31 ed e8 85 36 ef c8 eb 07 <0f> 0b bd f4 ff ff ff 48 8b 44 24 08 65 48 2b 04 25 28 00 00 00 75
[   35.722990] RSP: 0018:ffffaee0804e3890 EFLAGS: 00010282
[   35.723557] RAX: 0000000000000049 RBX: ffff9c93c261a000 RCX: 0000000000000027
[   35.724564] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9c943bc21780
[   35.725314] RBP: ffffffff8b8a3600 R08: 0000000000000000 R09: ffffaee0804e3720
[   35.726083] R10: ffffffff8acb4348 R11: 0000000000000003 R12: ffff9c93c261a000
[   35.726847] R13: ffff9c93c261a000 R14: ffff9c93c18f9b40 R15: ffff9c93c261a000
[   35.727601] FS:  00007fcdd816c680(0000) GS:ffff9c943bc00000(0000) knlGS:0000000000000000
[   35.728459] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[   35.729082] CR2: 00007f191721c280 CR3: 00000001029e8000 CR4: 0000000000350ef0

Can you confirm that?

Regards,
Salvatore

[toc] | [prev] | [next] | [standalone]

#84943 — Processed: Re: Bug#1091439: installation of nfs-kernel-server hangs

Processing control commands:

> tags -1 + moreinfo
Bug #1091439 [nfs-kernel-server] installation of nfs-kernel-server hangs
Added tag(s) moreinfo.

-- 
1091439: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1091439
Debian Bug Tracking System
Contact owner@bugs.debian.org with problems

[toc] | [prev] | [next] | [standalone]

#84953 — Bug#1091439: kernel BUG at fs/nfsd/nfs4recover.c:534 Oops: invalid opcode: 0000

Hi,

On Thu, Dec 26, 2024 at 08:17:45PM +0100, Salvatore Bonaccorso wrote:
> Hi Chuck, hi all,
> 
> On Thu, Dec 26, 2024 at 11:33:01AM -0500, Chuck Lever wrote:
> > On 12/26/24 11:24 AM, Salvatore Bonaccorso wrote:
> > > Hi Jur,
> > > 
> > > On Mon, Dec 09, 2024 at 04:50:05PM +0000, Jur van der Burg via Bugspray Bot wrote:
> > > > Jur van der Burg writes via Kernel.org Bugzilla:
> > > > 
> > > > I tried kernel 6.10.1 and that one is ok. In the mean time I
> > > > upgraded nfs-utils from 2.5.1 to 2.8.1 which seems to fix the issue.
> > > > Sorry for the noise, case closed.
> > > > 
> > > > View: https://bugzilla.kernel.org/show_bug.cgi?id=219580#c2
> > > > You can reply to this message to join the discussion.
> > > 
> > > Are you sure this is solved? I got hit by this today after trying to
> > > check the report from another Debian user:
> > > 
> > > https://bugs.debian.org/1091439
> > > the earlier report was
> > > https://bugs.debian.org/1087900
> > > 
> > > Surprisingly I managed to hit this, after:
> > > 
> > > Doing a fresh Debian installation with Debian unstable, rebooting
> > > after installation. The running kernel is 6.12.6-1 (but now believe it
> > > might be hit in any sufficient earlier version):
> > > 
> > > Notably, in kernel-log I see as well
> > > 
> > > [   50.295209] RPC: Registered tcp NFSv4.1 backchannel transport module.
> > > [   52.158301] NFSD: Using /var/lib/nfs/v4recovery as the NFSv4 state recovery directory
> > > [   52.158333] NFSD: Using legacy client tracking operations.
> > 
> > Hi Salvatore,
> > 
> > If you no longer provision nfsdcltrack in user space, then you want to
> > set CONFIG_NFSD_LEGACY_CLIENT_TRACKING to 'N' in your kernel config.
> 
> Right, while this might not be possible right now in the distribution,
> to confirm, setting CONFIG_NFSD_LEGACY_CLIENT_TRACKING would resolve
> the problem. In the distribution I think we would not yet be able to
> do a hard cut for planned next stable release.
> 
> Remember, that in Debian we only with the current stable release got
> again somehow on "track" with nfs-utils code.
> 
> > Otherwise, Scott Mayhew is the area expert (cc'd).
> 
> Thanks!
> 
> I will try to get more narrow down to the versions to see where the
> problem might be introduced, but if you already have a clue, and know
> what we might try (e.g. commit revert on top, or patch) I'm happy to
> test this as well (since now reliably able to trigger it).

Okay so this was maybe obvious for you already but bisecting leads to
the first bad commit beeing:

74fd48739d04 ("nfsd: new Kconfig option for legacy client tracking")

The Problem is not present in v6.7 and it is triggerable with
74fd48739d04 ("nfsd: new Kconfig option for legacy client tracking")

Most importantly as the switch to defaulting to y was only in later
versions, explicitly setting CONFIG_NFSD_LEGACY_CLIENT_TRACKING=y.

Regards,
Salvatore

[toc] | [prev] | [standalone]

Back to top | Article view | linux.debian.kernel

csiph-web