Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.os.linux.networking > #4719

Re: What networked filesystem to use

Show all headers | View raw

Harold Johanssen <noemail@please.net> writes:

> On Wed, 25 Nov 2020 18:00:36 +0000, Harold Johanssen wrote:
>
>> I have a setup in which my system has a limited amount of disk space,
>> but I am in a network with access to a server with all but infinite
>> amounts of disk space available. I need to use my system to carry out
>> lengthy software compilations, the kind that takes lots of disk space. I
>> therefore have to access disk space from the server above over the 100
>> Mb LAN.
>> 
>> 	My question would be, What filesystem is likely to deliver the
>> best performance under the circumstances described?
>> 
>> 	I have tried with NFS, and it is not great. I will try next SSHFS
>> and WebDAV, but if anybody has experience with these, and other
>> networked filesystems, your feedback would be much appreciated.
>
> 	Thanks to all who replied. I have tried to do some fine tuning in 
> my NSF settings, as described in a number of places in the net. As for my 
> network - well, I'd love to be able to have a 1 Gb network, but that is 
> not going to happen in the short term, unfortunately.
>
> 	My intent was, when posing this question, to get feedback on the 
> performance of different network filesystems with the networking 
> limitations that I described, from anybody who has had experience with 
> comparable setups. This is an all-Linux network.

I no longer have a 100Mb network, but my memory is that I got similar
proportional throughput that I now get from my 1Gb network.

I used the command:

dd if=<bigNetworkFile> of=/dev/null bs=4k

and the dd summary line was:

7655663616 bytes (7.7 GB) copied, 65.8552 s, 116 MB/s

If you take the file size and divide by the time and multiply by 8
bits/byte, you get a read throughput of almost exactly 930Mb, or 93% of
the unattainable maximum.  I haven't done anything fancy like trying to
use jumbo frames.  This is using nfs version 4.

It's possible that with some work, you might do (a little) better with
sshfs.

https://www.admin-magazine.com/HPC/Articles/Sharing-Data-with-SSHFS

Scott
-- 
Scott Hemphill	hemphill@alumni.caltech.edu
"This isn't flying.  This is falling, with style."  -- Buzz Lightyear

Back to comp.os.linux.networking | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

What networked filesystem to use Harold Johanssen <noemail@please.net> - 2020-11-25 18:00 +0000
  Re: What networked filesystem to use Tauno Voipio <tauno.voipio@notused.fi.invalid> - 2020-11-25 20:17 +0200
  Re: What networked filesystem to use Scott Hemphill <hemphill@hemphills.net> - 2020-11-25 13:45 -0500
  Re: What networked filesystem to use Grant Taylor <gtaylor@tnetconsulting.net> - 2020-11-25 12:20 -0700
  Re: What networked filesystem to use Grant Taylor <gtaylor@tnetconsulting.net> - 2020-11-25 12:21 -0700
  Re: What networked filesystem to use David Brown <david.brown@hesbynett.no> - 2020-11-25 20:32 +0100
    Re: What networked filesystem to use William Unruh <unruh@invalid.ca> - 2020-11-25 21:19 +0000
  Re: What networked filesystem to use Harold Johanssen <noemail@please.net> - 2020-11-25 20:26 +0000
    Re: What networked filesystem to use Scott Hemphill <hemphill@hemphills.net> - 2020-11-25 17:21 -0500
    Re: What networked filesystem to use "Carlos E.R." <robin_listas@es.invalid> - 2020-11-26 00:04 +0100
      Re: What networked filesystem to use Jorgen Grahn <grahn+nntp@snipabacken.se> - 2020-11-26 07:27 +0000
        Re: What networked filesystem to use William Unruh <unruh@invalid.ca> - 2020-11-27 16:21 +0000
          Re: What networked filesystem to use "Carlos E.R." <robin_listas@es.invalid> - 2020-11-27 20:22 +0100
      Re: What networked filesystem to use David Brown <david.brown@hesbynett.no> - 2020-11-26 09:22 +0100
        Re: What networked filesystem to use Johann Beretta <beretta@nun-ya-bizness.com> - 2020-11-27 22:37 -0800
          Re: What networked filesystem to use David Brown <david.brown@hesbynett.no> - 2020-11-28 16:22 +0100
            Re: What networked filesystem to use Johann Beretta <beretta@nun-ya-bizness.com> - 2020-11-28 12:37 -0800
              Re: What networked filesystem to use David Brown <david.brown@hesbynett.no> - 2020-11-29 13:26 +0100
                Re: What networked filesystem to use Johann Beretta <beretta@nun-ya-bizness.com> - 2020-11-29 10:44 -0800
                Re: What networked filesystem to use William Unruh <unruh@invalid.ca> - 2020-11-29 19:33 +0000
                Re: What networked filesystem to use Johann Beretta <beretta@nun-ya-bizness.com> - 2020-11-29 13:59 -0800
                Re: What networked filesystem to use David Brown <david.brown@hesbynett.no> - 2020-11-29 22:02 +0100
                Re: What networked filesystem to use Johann Beretta <beretta@nun-ya-bizness.com> - 2020-11-29 14:01 -0800
                Re: What networked filesystem to use David Brown <david.brown@hesbynett.no> - 2020-11-29 23:39 +0100
  Re: What networked filesystem to use "Tristan B. Kildaire" <deavmi@redxen.eu> - 2021-01-18 14:04 +0200
    Re: What networked filesystem to use Jim Jackson <jj@franjam.org.uk> - 2021-01-18 17:57 +0000
  Re: What networked filesystem to use William Unruh <unruh@invalid.ca> - 2021-01-18 18:21 +0000

csiph-web