Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.user > #245570

Re: [Sid] InfiniBand (please fix)

Show all headers | View raw

On 2/16/22 00:11, Grzesiek wrote:
> Hi there,
> 
> I plan to use NFS over RDMA. I decided to buy two QLE7340 InfniBand 
> controllers and DAC cable (no switch, two nodes only). Is QLE7340 
> supported by Sid? Is required software included in Debian repositories?

It seems that the required software is present in Sid unfortunately 
there is a problem with kernels 5.15 and 5.16. When trying to load 
ib_qib you get:

[    4.481908] ib_qib 0000:02:00.0: qib0: Reserving QPNs from 0x656b78 
to 0x656b78 for non-verbs use
[    4.482005] sysfs: cannot create duplicate filename 
'/devices/pci0000:00/0000:00:01.1/0000:02:00.0/infiniband/qib0/ports/1/
linkcontrol'
[    4.482008] CPU: 4 PID: 471 Comm: systemd-udevd Not tainted 
5.16.0-1-amd64 #1  Debian 5.16.7-2
[    4.482011] Hardware name: To Be Filled By O.E.M. To Be Filled By 
O.E.M./Z97 Extreme4, BIOS P2.60 03/06/2018
[    4.482013] Call Trace:
[    4.482014]  <TASK>
[    4.482016]  dump_stack_lvl+0x48/0x5e
[    4.482021]  sysfs_warn_dup.cold+0x17/0x24
[    4.482024]  internal_create_group+0x365/0x380
[    4.482027]  internal_create_groups.part.0+0x3d/0xa0
[    4.482029]  setup_port+0x370/0x680 [ib_core]
[    4.482044]  ? kobject_add+0x7e/0xb0
[    4.482048]  ib_setup_port_attrs+0x98/0x240 [ib_core]
[    4.482058]  ib_register_device+0x57f/0x660 [ib_core]
[    4.482067]  ? vmalloc_node+0x47/0x50
[    4.482071]  rvt_register_device+0x10c/0x270 [rdmavt]
[    4.482076]  qib_register_ib_device+0x608/0x7d0 [ib_qib]
[    4.482089]  qib_init_one+0x17f/0x470 [ib_qib]
[    4.482099]  local_pci_probe+0x45/0x80
[    4.482102]  ? pci_match_device+0xd7/0x130
[    4.482104]  pci_device_probe+0xd2/0x1c0
[    4.482106]  really_probe+0x1f5/0x3f0
[    4.482109]  __driver_probe_device+0xfe/0x180
[    4.482111]  driver_probe_device+0x1e/0x90
[    4.482113]  __driver_attach+0xc0/0x1c0
[    4.482115]  ? __device_attach_driver+0xe0/0xe0
[    4.482117]  ? __device_attach_driver+0xe0/0xe0
[    4.482119]  bus_for_each_dev+0x78/0xc0
[    4.482122]  bus_add_driver+0x149/0x1e0
[    4.482124]  driver_register+0x8f/0xe0
[    4.482126]  ? qib_init_qibfs+0x11/0x11 [ib_qib]
[    4.482135]  qib_ib_init+0x3e/0xf62 [ib_qib]
[    4.482144]  do_one_initcall+0x44/0x200
[    4.482147]  ? kmem_cache_alloc_trace+0x175/0x3d0
[    4.482150]  do_init_module+0x5c/0x280
[    4.482153]  __do_sys_finit_module+0xae/0x110
[    4.482156]  do_syscall_64+0x3b/0xc0
[    4.482159]  entry_SYSCALL_64_after_hwframe+0x44/0xae
[    4.482162] RIP: 0033:0x7fe1bbc36f79
[    4.482164] Code: 48 8d 3d 9a a8 0d 00 0f 05 eb a5 66 0f 1f 44 00 00 
48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8
4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d c7 5e 0d 00 
f7 d8 64 89 01 48
[    4.482167] RSP: 002b:00007fff9e28e408 EFLAGS: 00000246 ORIG_RAX: 
0000000000000139
[    4.482170] RAX: ffffffffffffffda RBX: 0000556a935aedb0 RCX: 
00007fe1bbc36f79
[    4.482171] RDX: 0000000000000000 RSI: 00007fe1bbde6eed RDI: 
000000000000000f
[    4.482173] RBP: 0000000000020000 R08: 0000000000000000 R09: 
0000556a93570540
[    4.482174] R10: 000000000000000f R11: 0000000000000246 R12: 
00007fe1bbde6eed
[    4.482175] R13: 0000000000000000 R14: 0000556a935af730 R15: 
0000556a935aedb0
[    4.482178]  </TASK>
[    4.482195] infiniband qib0: Couldn't register device with driver model
[    4.482209] ib_qib 0000:02:00.0: qib0: Failed to register driver with 
ib core.
[    4.482229] ib_qib 0000:02:00.0: qib0: cannot register verbs: 17!
[    4.482762] ib_qib 0000:02:00.0: Disabling notifier on HCA 0 irq 49
[    4.482818] ib_qib 0000:02:00.0: Disabling notifier on HCA 0 irq 50
[    4.482841] ib_qib 0000:02:00.0: Disabling notifier on HCA 0 irq 51
[    4.482911] ib_qib 0000:02:00.0: Disabling notifier on HCA 0 irq 53
[    4.483485] ib_qib: probe of 0000:02:00.0 failed with error -17

After installing linux-image-5.10.0-10-amd64 from Bullseye ib_qib loads 
without problem. Please fix.

The problem seems to be related to newer kernels.

btw: Is the openibd needed to acquire direct connect between two 
adapters? If so, how to install it on Sid?

Thanks in advance for any help

Greg

Back to linux.debian.user | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

[Sid] InfiniBand Grzesiek <pld@sojka.co> - 2022-02-16 00:20 +0100
  Re: [Sid] InfiniBand (please fix) Grzesiek <pld@sojka.co> - 2022-02-20 01:20 +0100
    Re: [Sid] InfiniBand (please apply the following pach) Grzesiek <pld@sojka.co> - 2022-02-20 13:30 +0100

csiph-web