Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.user > #191215 > unrolled thread

NFS4 file transfers fail or are very slow

Back to article view | Back to linux.debian.user

Contents

  NFS4 file transfers fail or are very slow Michael <pikim@web.de> - 2018-01-17 00:00 +0100
    Re: NFS4 file transfers fail or are very slow Dan Ritter <dsr@randomstring.org> - 2018-01-17 16:40 +0100
      Re: NFS4 file transfers fail or are very slow Michael <pikim@web.de> - 2018-01-17 23:00 +0100
        Re: NFS4 file transfers fail or are very slow Michael Stone <mstone@debian.org> - 2018-01-17 23:30 +0100
          Re: NFS4 file transfers fail or are very slow Michael <pikim@web.de> - 2018-01-18 07:20 +0100

#191215 — NFS4 file transfers fail or are very slow

Hello guys,

I have recently installed Debian Stretch 9.3 on a new PC and I'd just 
like to provide some NFS shares for other Linux machines in the LAN 
(1GBit). With small files everything works fine, but if I try to copy, 
e.g., a 7GB file with rsync it starts with a high transmission rate 
above 160MB/s and then goes down to around 70-90MB/s. After a random but 
short time there's no more progress going on for about 10-15 seconds. 
When the process then continues, the speed goes down to around 20MB/s 
and then up to about 200MB/s. It seems to continue transferring data, 
but rsync gives me an error message in the end.

On another client I experience the same, but there the last rate lies 
around 85MB/s instead of the 200MB/s from above. I guess this comes 
because the first PC has a SSD and the latter only a HDD. On the first 
client runs Arch Linux, on the latter ubuntu mate 16.04.
Exactly the same share is able to write more than 105MB/s via SMB/CIFS, 
so I would expect an NFS speed in a similar range.

My configurations files look as follows.

/etc/fstab on server:
/dev/disk/by-label/vol1  /vol1  ext4 
defaults,nofail,user_xattr,noexec,usrjquota=aquota.usr,grpjquota=aquota.grp,jqfmt=vfsv0,acl 
0 2
/vol1/temp  /export/temp  none  bind,nofail  0 0

/etc/exports on server:
/export/temp  192.168.192.0/24(fsid=1,rw,subtree_check,secure)
# NFSv4 pseudo filesystem root
/export 192.168.192.0/24(ro,fsid=0,root_squash,no_subtree_check,hide)

/etc/default/nfs-kernel-server on server:
RPCNFSDCOUNT=64
RPCNFSDPRIORITY=0
RPCMOUNTDOPTS="--manage-gids"
NEED_SVCGSSD=
RPCSVCGSSDOPTS=

/etc/fstab on clients:
192.168.192.10:/temp  /mnt/temp  nfs 
rw,bg,intr,soft,users,noauto,_netdev,proto=tcp,retry=3,timeo=10  0 0

I tried different export and mount options, but nothing helped to get 
stable transfer. Sometimes the transmission finishes successfully, but 
with a low rate (<60MB/s).

What can cause this problem?

Best regards,
Michael

[toc] | [next] | [standalone]

#191229

On Tue, Jan 16, 2018 at 11:41:05PM +0100, Michael wrote:
> Hello guys,
> 
> I have recently installed Debian Stretch 9.3 on a new PC and I'd just like
> to provide some NFS shares for other Linux machines in the LAN (1GBit). With
> small files everything works fine, but if I try to copy, e.g., a 7GB file
> with rsync it starts with a high transmission rate above 160MB/s and then
> goes down to around 70-90MB/s. After a random but short time there's no more
> progress going on for about 10-15 seconds. When the process then continues,
> the speed goes down to around 20MB/s and then up to about 200MB/s. It seems
> to continue transferring data, but rsync gives me an error message in the
> end.

Max throughput on 1Gb/s ethernet is slightly less than 125 MB
per second. Bits versus bytes, you know.

Anytime you see a rate above that, you're actually seeing
compression ratios rather than on-the-line bytes. Rsync is doing
that for you,

90MB/s is a normalish large-file transfer rate for
not-very-compressible data being read off of a spinning disk.

rsync also makes an awful lot of disk seeks in the attempt to
not transfer data that it doesn't need to transfer. 

> Exactly the same share is able to write more than 105MB/s via SMB/CIFS, so I
> would expect an NFS speed in a similar range.

Protocol overhead. Have you tried running a network benchmark?

-dsr-

[toc] | [prev] | [next] | [standalone]

#191243

Am 17.01.2018 um 16:37 schrieb Dan Ritter:
 > On Tue, Jan 16, 2018 at 11:41:05PM +0100, Michael wrote:
 >> Hello guys,
 >>
 >> I have recently installed Debian Stretch 9.3 on a new PC and I'd 
just like
 >> to provide some NFS shares for other Linux machines in the LAN 
(1GBit). With
 >> small files everything works fine, but if I try to copy, e.g., a 7GB 
file
 >> with rsync it starts with a high transmission rate above 160MB/s and 
then
 >> goes down to around 70-90MB/s. After a random but short time there's 
no more
 >> progress going on for about 10-15 seconds. When the process then 
continues,
 >> the speed goes down to around 20MB/s and then up to about 200MB/s. 
It seems
 >> to continue transferring data, but rsync gives me an error message 
in the
 >> end.
 >
 > Max throughput on 1Gb/s ethernet is slightly less than 125 MB
 > per second. Bits versus bytes, you know.
 >
 > Anytime you see a rate above that, you're actually seeing
 > compression ratios rather than on-the-line bytes. Rsync is doing
 > that for you,
 >
 > 90MB/s is a normalish large-file transfer rate for
 > not-very-compressible data being read off of a spinning disk.

Just for clarification: my given rates were read from SSD and written to a
spinning disk (WD Red).

 > rsync also makes an awful lot of disk seeks in the attempt to
 > not transfer data that it doesn't need to transfer.

But it does that via NFS as well as via SMB, doesn't it?

 >> Exactly the same share is able to write more than 105MB/s via 
SMB/CIFS, so I
 >> would expect an NFS speed in a similar range.
 >
 > Protocol overhead. Have you tried running a network benchmark?

I installed and ran iperf, but I don't know whether that provides 
reasonable
results:
------------------------------------------------------------
Server listening on TCP port 5001
TCP window size: 85.3 KByte (default)
------------------------------------------------------------
[  4] local 192.168.192.10 port 5001 connected with 192.168.192.20 port 
44282
[ ID] Interval       Transfer     Bandwidth
[  4]  0.0-10.0 sec  1.09 GBytes   936 Mbits/sec

When I copy the same file 3 times onto the same share I get an average of
108 MB/s with SMB/CIFS
  84 MB/s with NFS (async,no_subtree_check)

But in many/most comparisons you can find in the internet, NFS is having a
higher performance. So isn't that the case in real worldapplications?

 > -dsr-
Michael

[toc] | [prev] | [next] | [standalone]

#191245

On Wed, Jan 17, 2018 at 10:55:21PM +0100, Michael wrote:
>When I copy the same file 3 times onto the same share I get an average of
>108 MB/s with SMB/CIFS
> 84 MB/s with NFS (async,no_subtree_check)

What's the server? Also try 
grep MOUNTPOINT /proc/mounts
(where MOUNTPOINT is /whatever your NFS share is)
you should see either nfs2, nfs3, or nfs4 in the second field, then 
rsize= and wsize= and further down proto=tcp or proto=udp. It would be 
helpful to know what those are on your system.

Mike Stone

[toc] | [prev] | [next] | [standalone]

#191254

Am 17.01.2018 um 23:22 schrieb Michael Stone:
> On Wed, Jan 17, 2018 at 10:55:21PM +0100, Michael wrote:
>> When I copy the same file 3 times onto the same share I get an 
>> average of
>> 108 MB/s with SMB/CIFS
>>  84 MB/s with NFS (async,no_subtree_check)
>
> What's the server? Also try grep MOUNTPOINT /proc/mounts
> (where MOUNTPOINT is /whatever your NFS share is)
> you should see either nfs2, nfs3, or nfs4 in the second field, then 
> rsize= and wsize= and further down proto=tcp or proto=udp. It would be 
> helpful to know what those are on your system.
>
> Mike Stone

The server is a self-built machine with an ASRock J4205-ITX mainboard, 
8GB RAM, a Samsung 470 SSD as system disk and 2x WD Reds as data disks. 
It's running the Debian backport kernel 4.13 (4.14 since yesterday).

The result of 'mount | grep /mnt/temp' on the client is

rw,relatime,vers=4.2,rsize=1048576,wsize=1048576,namlen=255,soft,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.192.20,local_lock=none,addr=192.168.192.10

where I just set 'soft,proto=tcp', the others were set as default.

Michael

[toc] | [prev] | [standalone]

Back to top | Article view | linux.debian.user

csiph-web