Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1440134 > unrolled thread

Re: More parallel atomic_open/d_splice_alias fun with NFS and possibly more FSes.

Back to article view | Back to linux.kernel

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Re: More parallel atomic_open/d_splice_alias fun with NFS and possibly more FSes. Oleg Drokin <green@linuxhacker.ru> - 2016-07-10 19:10 +0200
    Re: More parallel atomic_open/d_splice_alias fun with NFS and possibly  more FSes. James Simmons <jsimmons@infradead.org> - 2016-07-10 20:20 +0200
      Re: More parallel atomic_open/d_splice_alias fun with NFS and  possibly more FSes. Al Viro <viro@ZenIV.linux.org.uk> - 2016-07-11 03:10 +0200
        Re: lustre sendmsg stuff Oleg Drokin <green@linuxhacker.ru> - 2016-07-12 01:00 +0200
      Re: More parallel atomic_open/d_splice_alias fun with NFS and  possibly more FSes. Al Viro <viro@ZenIV.linux.org.uk> - 2016-07-11 03:10 +0200
        Re: More parallel atomic_open/d_splice_alias fun with NFS and possibly  more FSes. James Simmons <jsimmons@infradead.org> - 2016-07-11 19:20 +0200

#1440134 — Re: More parallel atomic_open/d_splice_alias fun with NFS and possibly more FSes.

On Jul 4, 2016, at 10:25 PM, Al Viro wrote:

> BTW, could you take a look at
> git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs.git#sendmsg.lustre?
> It's a bunch of simplifications that became possible once sendmsg()/recvmsg()
> switched to iov_iter, stopped mangling the iovecs and went for predictable
> behaviour re advancing the iterator.

Thanks, this looks good to me and passes my testing (on tcp).

+typedef struct bio_vec lnet_kiov_t;

This I guess we'll need to just get rid of all lnet_kiov_t usage, but that's
something we can do ourselves, I guess.

Anyway, your patchset is based on old tree that no longer applies cleanly,
I rebased it to current staging tree to save you time in case
you want to go forward with it.
It's at git@github.com:verygreen/linux.git branch lustre-next-sendmsg

James, can you please give it a try on IB?

Bye,
    Oleg

[toc] | [next] | [standalone]

#1440146 — Re: More parallel atomic_open/d_splice_alias fun with NFS and possibly more FSes.

> On Jul 4, 2016, at 10:25 PM, Al Viro wrote:
> 
> > BTW, could you take a look at
> > git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs.git#sendmsg.lustre?
> > It's a bunch of simplifications that became possible once sendmsg()/recvmsg()
> > switched to iov_iter, stopped mangling the iovecs and went for predictable
> > behaviour re advancing the iterator.
> 
> Thanks, this looks good to me and passes my testing (on tcp).
> 
> +typedef struct bio_vec lnet_kiov_t;
> 
> This I guess we'll need to just get rid of all lnet_kiov_t usage, but that's
> something we can do ourselves, I guess.
> 
> Anyway, your patchset is based on old tree that no longer applies cleanly,
> I rebased it to current staging tree to save you time in case
> you want to go forward with it.
> It's at git@github.com:verygreen/linux.git branch lustre-next-sendmsg
> 
> James, can you please give it a try on IB?

Its broke for the ko2iblnd driver.

[  110.840583] LNet: Using FMR for registration
[  110.991747] LNet: Added LNI 10.37.248.137@o2ib1 [63/2560/0/180]
[  110.998211] ------------[ cut here ]------------
[  111.003012] kernel BUG at lib/iov_iter.c:513!
[  111.007545] invalid opcode: 0000 [#1] SMP
[  111.011731] Modules linked in: ko2iblnd(C) ptlrpc(C+) obdclass(C) 
ksocklnd(C) lnet(C) sha512_generic sha256_generic md5 crc32_generic crc3
2_pclmul libcfs(C) autofs4 ipmi_devintf auth_rpcgss nfsv4 dns_resolver 
8021q iptable_filter ip_tables x_tables ib_ipoib rdma_ucm ib_ucm ib_uv
erbs ib_umad rdma_cm configfs ib_cm iw_cm mlx4_ib ib_core dm_mirror 
dm_region_hash dm_log dm_multipath sg sd_mod joydev pcspkr dm_mod mpt3sas
 raid_class acpi_cpufreq ipmi_ssif ipmi_si ipmi_msghandler isci libsas 
scsi_transport_sas wmi tpm_tis tpm i2c_i801 ahci libahci libata scsi_m
od ehci_pci ehci_hcd button tcp_cubic nfsv3(E) nfs_acl(E) ipv6(E) nfs(E) 
lockd(E) sunrpc(E) grace(E) mlx4_en(E) mlx4_core(E) igb(E) i2c_algo_
bit(E) i2c_core(E) ptp(E) pps_core(E) hwmon(E)
[  111.086669] CPU: 6 PID: 11899 Comm: router_checker Tainted: G         C  
E   4.7.0-rc6+ #1
[  111.095248] Hardware name: Supermicro X9DRT/X9DRT, BIOS 3.0a 02/19/2014
[  111.102040] task: ffff880826d32d80 ti: ffff880811d24000 task.ti: 
ffff880811d24000
[  111.109818] RIP: 0010:[<ffffffff8128daf2>]  [<ffffffff8128daf2>] 
iov_iter_kvec+0x22/0x30
[  111.118302] RSP: 0018:ffff880811d27b28  EFLAGS: 00010246
[  111.123806] RAX: 0000000000000000 RBX: ffff88105e037c00 RCX: 
0000000000000000
[  111.131111] RDX: 0000000000000000 RSI: 0000000000000005 RDI: 
ffff880811d27b78
[  111.138426] RBP: ffff880811d27b28 R08: 0000000000000000 R09: 
0000000000000000
[  111.145751] R10: 0000000000000000 R11: 00000000fffd19f7 R12: 
0000000000000000
[  111.153083] R13: 0000000000000000 R14: 000500010a25ca3b R15: 
ffff880811d27b78
[  111.160407] FS:  0000000000000000(0000) GS:ffff88107fd00000(0000) 
knlGS:0000000000000000
[  111.168797] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[  111.174717] CR2: 00007fd075c48945 CR3: 000000105a9ec000 CR4: 
00000000000406e0
[  111.182031] Stack:
[  111.184224]  ffff880811d27be8 ffffffffa04012bd 0000000000000000 
ffff880811d27b40
[  111.192206]  0000000000000000 0000000000000000 0000000000000000 
0000000000000000
[  111.192206]  0000000000000000 0000000000000000 0000000000000000 
0000000000000000
[  111.200196]  0000880800000002 ffff88105e394000 0000000000000000 
0000000000000000
[  111.208179] Call Trace:
[  111.210818]  [<ffffffffa04012bd>] kiblnd_send+0x51d/0x9e0 [ko2iblnd]
[  111.217370]  [<ffffffffa06ec6bd>] lnet_ni_send+0x3d/0xe0 [lnet]
[  111.223487]  [<ffffffffa06ee223>] lnet_send+0x6b3/0xc80 [lnet]
[  111.229501]  [<ffffffffa06eeb58>] LNetGet+0x368/0x650 [lnet]
[  111.235346]  [<ffffffffa0692a50>] ? cfs_percpt_lock+0x50/0x110 [libcfs]
[  111.242139]  [<ffffffffa06f4d8f>] lnet_ping_router_locked+0x20f/0x840 
[lnet]
[  111.249384]  [<ffffffffa06f5909>] lnet_router_checker+0xd9/0x490 [lnet]
[  111.256192]  [<ffffffff8108347d>] ? default_wake_function+0xd/0x10
[  111.262549]  [<ffffffff810923f1>] ? __wake_up_common+0x51/0x80
[  111.268562]  [<ffffffffa06f5830>] ? lnet_prune_rc_data+0x470/0x470 
[lnet]
[  111.275544]  [<ffffffff81508f9b>] ? schedule+0x3b/0xa0
[  111.280871]  [<ffffffffa06f5830>] ? lnet_prune_rc_data+0x470/0x470 
[lnet]
[  111.287849]  [<ffffffff810779d7>] kthread+0xc7/0xe0
[  111.292904]  [<ffffffff8150c3cf>] ret_from_fork+0x1f/0x40
[  111.298475]  [<ffffffff81077910>] ? 
kthread_freezable_should_stop+0x70/0x70
[  111.305631] Code: 2e 0f 1f 84 00 00 00 00 00 55 40 f6 c6 02 48 89 e5 74 
18 89 37 48 89 57 18 48 89 4f 20 48 c7 47 08 00 00 00 00 4c 89 47 10 c9 c3 
<0f> 0b eb fe 66 2e 0f 1f 84 00 00 00 00 00 48 8b 47 10 55 48 89
[  111.329528] RIP  [<ffffffff8128daf2>] iov_iter_kvec+0x22/0x30
[  111.335533]  RSP <ffff880811d27b28>
[  111.339360] ---[ end trace 1ea9288f558e2c8d ]---

[toc] | [prev] | [next] | [standalone]

#1440192 — Re: More parallel atomic_open/d_splice_alias fun with NFS and possibly more FSes.

On Mon, Jul 11, 2016 at 02:01:13AM +0100, Al Viro wrote:
> On Sun, Jul 10, 2016 at 07:14:18PM +0100, James Simmons wrote:
> 
> > [  111.210818]  [<ffffffffa04012bd>] kiblnd_send+0x51d/0x9e0 [ko2iblnd]
> 
> Mea culpa - in kiblnd_send() this
>         if (payload_kiov)
>                 iov_iter_bvec(&from, ITER_BVEC | WRITE,
>                                 payload_kiov, payload_niov, payload_nob);
>         else
>                 iov_iter_kvec(&from, ITER_BVEC | WRITE,
>                                 payload_iov, payload_niov, payload_nob);
> should have s/BVEC/KVEC/ in the iov_iter_kvec() arguments.  Cut'n'paste
> braindamage...

PS: That was introduced in the last commit in that pile - "lustre: introduce
lnet_copy_{k,}iov2iter(), kill lnet_copy_{k,}iov2{k,}iov()".

[toc] | [prev] | [next] | [standalone]

#1440958 — Re: lustre sendmsg stuff

On Jul 10, 2016, at 9:03 PM, Al Viro wrote:

> On Mon, Jul 11, 2016 at 02:01:13AM +0100, Al Viro wrote:
>> On Sun, Jul 10, 2016 at 07:14:18PM +0100, James Simmons wrote:
>> 
>>> [  111.210818]  [<ffffffffa04012bd>] kiblnd_send+0x51d/0x9e0 [ko2iblnd]
>> 
>> Mea culpa - in kiblnd_send() this
>>        if (payload_kiov)
>>                iov_iter_bvec(&from, ITER_BVEC | WRITE,
>>                                payload_kiov, payload_niov, payload_nob);
>>        else
>>                iov_iter_kvec(&from, ITER_BVEC | WRITE,
>>                                payload_iov, payload_niov, payload_nob);
>> should have s/BVEC/KVEC/ in the iov_iter_kvec() arguments.  Cut'n'paste
>> braindamage...
> 
> PS: That was introduced in the last commit in that pile - "lustre: introduce
> lnet_copy_{k,}iov2iter(), kill lnet_copy_{k,}iov2{k,}iov()".

Is this something you plan to submit to Linus or should I just submit this to
Greg along with other changes?

[toc] | [prev] | [next] | [standalone]

#1440193 — Re: More parallel atomic_open/d_splice_alias fun with NFS and possibly more FSes.

On Sun, Jul 10, 2016 at 07:14:18PM +0100, James Simmons wrote:

> [  111.210818]  [<ffffffffa04012bd>] kiblnd_send+0x51d/0x9e0 [ko2iblnd]

Mea culpa - in kiblnd_send() this
        if (payload_kiov)
                iov_iter_bvec(&from, ITER_BVEC | WRITE,
                                payload_kiov, payload_niov, payload_nob);
        else
                iov_iter_kvec(&from, ITER_BVEC | WRITE,
                                payload_iov, payload_niov, payload_nob);
should have s/BVEC/KVEC/ in the iov_iter_kvec() arguments.  Cut'n'paste
braindamage...

[toc] | [prev] | [next] | [standalone]

#1440783 — Re: More parallel atomic_open/d_splice_alias fun with NFS and possibly more FSes.

> On Sun, Jul 10, 2016 at 07:14:18PM +0100, James Simmons wrote:
> 
> > [  111.210818]  [<ffffffffa04012bd>] kiblnd_send+0x51d/0x9e0 [ko2iblnd]
> 
> Mea culpa - in kiblnd_send() this
>         if (payload_kiov)
>                 iov_iter_bvec(&from, ITER_BVEC | WRITE,
>                                 payload_kiov, payload_niov, payload_nob);
>         else
>                 iov_iter_kvec(&from, ITER_BVEC | WRITE,
>                                 payload_iov, payload_niov, payload_nob);
> should have s/BVEC/KVEC/ in the iov_iter_kvec() arguments.  Cut'n'paste
> braindamage...

That is the fix. Also I believe payload_nob should be payload_nob + 
payload_offset instead. I will send a patch that against Oleg's tree
that address these issues.

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web