Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.user > #172889 > unrolled thread

Maximal volume size for the client with a NFS v3 mounting

Back to article view | Back to linux.debian.user

Contents

  Maximal volume size for the client with a NFS v3 mounting Jean-Paul Bouchet <jean-paul.bouchet@inra.fr> - 2016-09-28 19:20 +0200
    Re: Maximal volume size for the client with a NFS v3 mounting Christian Seiler <christian@iwakd.de> - 2016-09-28 20:10 +0200
      Re: Maximal volume size for the client with a NFS v3 mounting "Andrew M.A. Cater" <amacater@galactic.demon.co.uk> - 2016-09-28 21:00 +0200
        Re: Maximal volume size for the client with a NFS v3 mounting Jean-Paul Bouchet <jean-paul.bouchet@inra.fr> - 2016-09-29 18:30 +0200

#172889 — Maximal volume size for the client with a NFS v3 mounting

Hi,

On a Jessie 8.5 system I mount a partition on a NAS server with NFSv3 
protocol using options  "nfs  rw,soft" in /etc/fstab.

The size of the volume on the NAS server side has been extended to 20 
To, but for my Debian system, this extension appears to be limited to 16 
To :
- The df command gives 16106127360 blocks of 1K
- processes trying to write beyond this limit end with error "no space 
left on device"

Is the volume size mounted with NFSv3 limited to 16 To on the client side ?

Could the version 4 of NFS solve this problem ?

Thanks

Jean-Paul

[toc] | [next] | [standalone]

#172891

On 09/28/2016 07:18 PM, Jean-Paul Bouchet wrote:
> On a Jessie 8.5 system I mount a partition on a NAS server with NFSv3
> protocol using options "nfs rw,soft" in /etc/fstab.
> 
> The size of the volume on the NAS server side has been extended to 20
> To, but for my Debian system, this extension appears to be limited to
> 16 To :
> - The df command gives 16106127360 blocks of 1K
> - processes trying to write beyond this limit end with error "no
>   space left on device"
> 
> Is the volume size mounted with NFSv3 limited to 16 To on the client
> side ?

RFC 1813 (the NFSv3 standard) defines the field for the free
space in the response of a NFSv3 server to be a 64bit (probably
unsigned) integer, so that should easily hold more than 16 TiB.

In your case, the 16 TiB limit you experience seems to be if
the size is somehow measured in a 32bit field in units of 4 KiB
blocks. From reading the source code, Linux 3.16 (that comes
with Jessie) only uses 64bit fields in both the NFS client code
and the general kernel structures, so I believe you're running
into a limitation of the NFS server your NAS provides.

I am not sure though - and I don't have any storage with more
than 16 TiB lying around to actually test it - so take my
response as an educated guess based on my read of the kernel
source code.

> Could the version 4 of NFS solve this problem ?

Possibly; it could be that the NFS4 implementation of your NAS
is not limited in that way - it could also be that it is. It
could also be that the NFS server in your NAS doesn't support
volumes > 16 TiB at all.

Another possibility could be that your NAS's NFS server supports
more than 16 TiB just fine, but the underlying filesystem used
on the NAS only supports up to 16 TiB. (For example, if the NAS
were to use the old Linux filesystem ReiserFS, that only supports
volumes up to 16 TiB.) Does the NAS actually show that there's
that much space free on the filesystem it exports? Or does it
only see the 20 TiB on the partitioning level? What NAS are you
using and what software are you running there?

Regards,
Christian

[toc] | [prev] | [next] | [standalone]

#172894

On Wed, Sep 28, 2016 at 08:06:53PM +0200, Christian Seiler wrote:
> On 09/28/2016 07:18 PM, Jean-Paul Bouchet wrote:
> > On a Jessie 8.5 system I mount a partition on a NAS server with NFSv3
> > protocol using options "nfs rw,soft" in /etc/fstab.
> > 
> > The size of the volume on the NAS server side has been extended to 20
> > To, but for my Debian system, this extension appears to be limited to
> > 16 To :
> > - The df command gives 16106127360 blocks of 1K
> > - processes trying to write beyond this limit end with error "no
> >   space left on device"
> > 
> > Is the volume size mounted with NFSv3 limited to 16 To on the client
> > side ?
> 
> RFC 1813 (the NFSv3 standard) defines the field for the free
> space in the response of a NFSv3 server to be a 64bit (probably
> unsigned) integer, so that should easily hold more than 16 TiB.
> 
> In your case, the 16 TiB limit you experience seems to be if
> the size is somehow measured in a 32bit field in units of 4 KiB
> blocks. From reading the source code, Linux 3.16 (that comes
> with Jessie) only uses 64bit fields in both the NFS client code
> and the general kernel structures, so I believe you're running
> into a limitation of the NFS server your NAS provides.
> 
> I am not sure though - and I don't have any storage with more
> than 16 TiB lying around to actually test it - so take my
> response as an educated guess based on my read of the kernel
> source code.
> 

It may depend on what filesystem is on your storage - and also whether
it's been resized to fit the underlying storage.

I did come across a filesystem bug a while where e2fsck didn't handle
> 16TB - but that was fixed.

Some other Linux distros might be limited to 16TB per single volume?

Ext4 is potentially Exabyte capable

What set up your NFS volume in the first place?

All best

Andy C



> > Could the version 4 of NFS solve this problem ?
> 
> Possibly; it could be that the NFS4 implementation of your NAS
> is not limited in that way - it could also be that it is. It
> could also be that the NFS server in your NAS doesn't support
> volumes > 16 TiB at all.
> 
> Another possibility could be that your NAS's NFS server supports
> more than 16 TiB just fine, but the underlying filesystem used
> on the NAS only supports up to 16 TiB. (For example, if the NAS
> were to use the old Linux filesystem ReiserFS, that only supports
> volumes up to 16 TiB.) Does the NAS actually show that there's
> that much space free on the filesystem it exports? Or does it
> only see the 20 TiB on the partitioning level? What NAS are you
> using and what software are you running there?
> 
> Regards,
> Christian

[toc] | [prev] | [next] | [standalone]

#172941

On 28/09/2016 20:59, Andrew M.A. Cater wrote:

> On Wed, Sep 28, 2016 at 08:06:53PM +0200, Christian Seiler wrote:
>> On 09/28/2016 07:18 PM, Jean-Paul Bouchet wrote:
>>> On a Jessie 8.5 system I mount a partition on a NAS server with NFSv3
>>> protocol using options "nfs rw,soft" in /etc/fstab.
>>>
>>> The size of the volume on the NAS server side has been extended to 20
>>> To, but for my Debian system, this extension appears to be limited to
>>> 16 To :
>>> - The df command gives 16106127360 blocks of 1K
>>> - processes trying to write beyond this limit end with error "no
>>>    space left on device"
>>>
>>> Is the volume size mounted with NFSv3 limited to 16 To on the client
>>> side ?
>> RFC 1813 (the NFSv3 standard) defines the field for the free
>> space in the response of a NFSv3 server to be a 64bit (probably
>> unsigned) integer, so that should easily hold more than 16 TiB.
>>
>> In your case, the 16 TiB limit you experience seems to be if
>> the size is somehow measured in a 32bit field in units of 4 KiB
>> blocks. From reading the source code, Linux 3.16 (that comes
>> with Jessie) only uses 64bit fields in both the NFS client code
>> and the general kernel structures, so I believe you're running
>> into a limitation of the NFS server your NAS provides.
>>
>> I am not sure though - and I don't have any storage with more
>> than 16 TiB lying around to actually test it - so take my
>> response as an educated guess based on my read of the kernel
>> source code.
>>
> It may depend on what filesystem is on your storage - and also whether
> it's been resized to fit the underlying storage.
>
> I did come across a filesystem bug a while where e2fsck didn't handle
>> 16TB - but that was fixed.
> Some other Linux distros might be limited to 16TB per single volume?
>
> Ext4 is potentially Exabyte capable
>
> What set up your NFS volume in the first place?
>
> All best
>
> Andy C
>
>
>
>>> Could the version 4 of NFS solve this problem ?
>> Possibly; it could be that the NFS4 implementation of your NAS
>> is not limited in that way - it could also be that it is. It
>> could also be that the NFS server in your NAS doesn't support
>> volumes > 16 TiB at all.
>>
>> Another possibility could be that your NAS's NFS server supports
>> more than 16 TiB just fine, but the underlying filesystem used
>> on the NAS only supports up to 16 TiB. (For example, if the NAS
>> were to use the old Linux filesystem ReiserFS, that only supports
>> volumes up to 16 TiB.) Does the NAS actually show that there's
>> that much space free on the filesystem it exports? Or does it
>> only see the 20 TiB on the partitioning level? What NAS are you
>> using and what software are you running there?
>>
>> Regards,
>> Christian

Thanks to Andy and Christian. Their answers let the administrators of 
the NAS server be convinced that the problem was not on the NFS Debian 
client side, but on the NAS server one's.
The volume was extended to 20 To, but with quota still fixed to 16 To, 
which so was the size seen by the Debian server. Once identified, the 
problem was easy to solve...

Jean-Paul

[toc] | [prev] | [standalone]

Back to top | Article view | linux.debian.user

csiph-web