Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1524445 > unrolled thread

[RFC][PATCH 0/4] Enhanced file stat system call

Back to article view | Back to linux.kernel

Contents

  [RFC][PATCH 0/4] Enhanced file stat system call David Howells <dhowells@redhat.com> - 2016-11-17 15:00 +0100
    Re: [RFC][PATCH 0/4] Enhanced file stat system call David Howells <dhowells@redhat.com> - 2016-11-17 18:10 +0100
    Re: [RFC][PATCH 0/4] Enhanced file stat system call David Howells <dhowells@redhat.com> - 2016-11-17 18:10 +0100
      Re: [RFC][PATCH 0/4] Enhanced file stat system call bfields@fieldses.org (J. Bruce Fields) - 2016-11-17 21:10 +0100
        Re: [RFC][PATCH 0/4] Enhanced file stat system call Andreas Dilger <adilger@dilger.ca> - 2016-11-18 03:50 +0100
          Re: [RFC][PATCH 0/4] Enhanced file stat system call NeilBrown <neilb@suse.com> - 2016-11-18 05:40 +0100
      Re: [RFC][PATCH 0/4] Enhanced file stat system call One Thousand Gnomes <gnomes@lxorguk.ukuu.org.uk> - 2016-11-18 14:50 +0100
        Re: [RFC][PATCH 0/4] Enhanced file stat system call David Howells <dhowells@redhat.com> - 2016-11-18 14:50 +0100
    Re: [RFC][PATCH 0/4] Enhanced file stat system call Michael Kerrisk <mtk.manpages@gmail.com> - 2016-11-17 18:20 +0100
    [PATCH 3/4] statx: NFS: Return enhanced file attributes David Howells <dhowells@redhat.com> - 2016-11-17 18:20 +0100
    [PATCH 2/4] statx: Ext4: Return enhanced file attributes David Howells <dhowells@redhat.com> - 2016-11-17 18:20 +0100
      Re: [PATCH 2/4] statx: Ext4: Return enhanced file attributes Andreas Dilger <adilger@dilger.ca> - 2016-11-18 04:40 +0100
    [PATCH 4/4] statx: AFS: Return enhanced file attributes David Howells <dhowells@redhat.com> - 2016-11-17 18:20 +0100
      Re: [PATCH 4/4] statx: AFS: Return enhanced file attributes Andreas Dilger <adilger@dilger.ca> - 2016-11-18 09:30 +0100
        Re: [PATCH 4/4] statx: AFS: Return enhanced file attributes David Howells <dhowells@redhat.com> - 2016-11-18 09:50 +0100
    Re: [RFC][PATCH 0/4] Enhanced file stat system call One Thousand Gnomes <gnomes@lxorguk.ukuu.org.uk> - 2016-11-17 19:00 +0100
    Re: [PATCH 1/4] statx: Add a system call to make enhanced file info  available Dave Chinner <david@fromorbit.com> - 2016-11-18 00:50 +0100
      Re: [PATCH 1/4] statx: Add a system call to make enhanced file info available Andreas Dilger <adilger@dilger.ca> - 2016-11-18 04:30 +0100
        Re: [PATCH 1/4] statx: Add a system call to make enhanced file info available David Howells <dhowells@redhat.com> - 2016-11-18 11:00 +0100
        Re: [PATCH 1/4] statx: Add a system call to make enhanced file info  available Dave Chinner <david@fromorbit.com> - 2016-11-18 23:10 +0100
          Re: [PATCH 1/4] statx: Add a system call to make enhanced file info available David Howells <dhowells@redhat.com> - 2016-11-19 00:00 +0100
            Re: [PATCH 1/4] statx: Add a system call to make enhanced file info  available Dave Chinner <david@fromorbit.com> - 2016-11-19 23:50 +0100
              Re: [PATCH 1/4] statx: Add a system call to make enhanced file info available David Howells <dhowells@redhat.com> - 2016-11-22 11:40 +0100
                Re: [PATCH 1/4] statx: Add a system call to make enhanced file info  available Jeff Layton <jlayton@redhat.com> - 2016-11-22 15:00 +0100
                Re: [PATCH 1/4] statx: Add a system call to make enhanced file info  available Dave Chinner <david@fromorbit.com> - 2016-11-22 22:00 +0100
            Re: [PATCH 1/4] statx: Add a system call to make enhanced file info  available One Thousand Gnomes <gnomes@lxorguk.ukuu.org.uk> - 2016-11-21 15:40 +0100
              Re: [PATCH 1/4] statx: Add a system call to make enhanced file info  available Dave Chinner <david@fromorbit.com> - 2016-11-21 21:50 +0100
      Re: [PATCH 1/4] statx: Add a system call to make enhanced file info available David Howells <dhowells@redhat.com> - 2016-11-18 10:40 +0100
        Re: [PATCH 1/4] statx: Add a system call to make enhanced file info  available Jeff Layton <jlayton@poochiereds.net> - 2016-11-18 18:20 +0100
          Re: [PATCH 1/4] statx: Add a system call to make enhanced file info available David Howells <dhowells@redhat.com> - 2016-11-18 19:10 +0100
            Re: [PATCH 1/4] statx: Add a system call to make enhanced file info  available Jeff Layton <jlayton@redhat.com> - 2016-11-18 20:00 +0100
              Re: [PATCH 1/4] statx: Add a system call to make enhanced file info available David Howells <dhowells@redhat.com> - 2016-11-18 20:10 +0100
      Re: [PATCH 1/4] statx: Add a system call to make enhanced file info available David Howells <dhowells@redhat.com> - 2016-11-18 10:50 +0100
        Re: [PATCH 1/4] statx: Add a system call to make enhanced file info  available Dave Chinner <david@fromorbit.com> - 2016-11-18 22:50 +0100
          Re: [PATCH 1/4] statx: Add a system call to make enhanced file info available David Howells <dhowells@redhat.com> - 2016-11-18 23:30 +0100
      Re: [PATCH 1/4] statx: Add a system call to make enhanced file info available David Howells <dhowells@redhat.com> - 2016-11-18 11:30 +0100
        Re: [PATCH 1/4] statx: Add a system call to make enhanced file info  available Dave Chinner <david@fromorbit.com> - 2016-11-18 22:30 +0100
          Re: [PATCH 1/4] statx: Add a system call to make enhanced file info available David Howells <dhowells@redhat.com> - 2016-11-18 22:50 +0100
            Re: [PATCH 1/4] statx: Add a system call to make enhanced file info  available Dave Chinner <david@fromorbit.com> - 2016-11-18 23:20 +0100
              Re: [PATCH 1/4] statx: Add a system call to make enhanced file info available "Michael Kerrisk (man-pages)" <mtk.manpages@gmail.com> - 2016-11-19 11:30 +0100
    Re: [PATCH 1/4] statx: Add a system call to make enhanced file info available David Howells <dhowells@redhat.com> - 2016-11-18 09:50 +0100
      Re: [PATCH 1/4] statx: Add a system call to make enhanced file info  available Jeff Layton <jlayton@redhat.com> - 2016-11-18 13:10 +0100
    Re: [PATCH 1/4] statx: Add a system call to make enhanced file info available David Howells <dhowells@redhat.com> - 2016-11-18 10:00 +0100
      Re: [PATCH 1/4] statx: Add a system call to make enhanced file info available Andreas Dilger <adilger@dilger.ca> - 2016-11-18 10:30 +0100

Page 1 of 3  [1] 2 3  Next page →

#1524445 — [RFC][PATCH 0/4] Enhanced file stat system call

Implement a new system call to provide enhanced file stats.  The patches can
be found here:

	http://git.kernel.org/cgit/linux/kernel/git/dhowells/linux-fs.git/log/?h=xstat


===========
DESCRIPTION
===========

The first patch provides this new system call:

	long ret = statx(int dfd,
			 const char *filename,
			 unsigned atflag,
			 unsigned mask,
			 struct statx *buffer);

This is an enhanced file stat function that provides a number of useful
features, in summary:

 (1) More information: creation time, data version number and attributes.
     A subset of these is available through a number of filesystems (such
     as CIFS, NFS, AFS, Ext4 and BTRFS).

 (2) Lightweight stat (AT_STATX_DONT_SYNC): Ask for just those details of
     interest, and allow a network fs to approximate anything not of
     interest, without going to the server.

 (3) Heavyweight stat (AT_STATX_FORCE_SYNC): Force a network fs to flush
     buffers and go to the server, even if it thinks its cached attributes
     are up to date.

 (4) Allow the filesystem to indicate what it can/cannot provide: A
     filesystem can now say it doesn't support a standard stat feature if
     that isn't available.

 (5) Make the fields a consistent size on all arches, and make them large.

 (6) Can be extended by using more request flags and using up the padding
     space in the statx struct.

Note that no lstat() equivalent is required as that can be implemented
through statx() with atflag == 0.  There is also no fstat() equivalent as
that can be implemented through statx() with filename == NULL and the
relevant fd passed as dfd.


=======
TESTING
=======

A test program is added into samples/statx/ by the first patch.

David
---
David Howells (4):
      statx: Add a system call to make enhanced file info available
      statx: Ext4: Return enhanced file attributes
      statx: NFS: Return enhanced file attributes
      statx: AFS: Return enhanced file attributes


 arch/x86/entry/syscalls/syscall_32.tbl |    1 
 arch/x86/entry/syscalls/syscall_64.tbl |    1 
 fs/afs/inode.c                         |   21 ++
 fs/exportfs/expfs.c                    |    4 
 fs/ext4/ext4.h                         |    2 
 fs/ext4/file.c                         |    2 
 fs/ext4/inode.c                        |   38 ++++
 fs/ext4/namei.c                        |    2 
 fs/ext4/symlink.c                      |    2 
 fs/nfs/inode.c                         |   41 ++++
 fs/stat.c                              |  294 +++++++++++++++++++++++++++++---
 include/linux/fs.h                     |    5 -
 include/linux/stat.h                   |   19 +-
 include/linux/syscalls.h               |    3 
 include/uapi/linux/fcntl.h             |    2 
 include/uapi/linux/stat.h              |  124 +++++++++++++
 samples/Kconfig                        |    5 +
 samples/Makefile                       |    3 
 samples/statx/Makefile                 |   10 +
 samples/statx/test-statx.c             |  248 +++++++++++++++++++++++++++
 20 files changed, 771 insertions(+), 56 deletions(-)
 create mode 100644 samples/statx/Makefile
 create mode 100644 samples/statx/test-statx.c

[toc] | [next] | [standalone]

#1524486

Michael Kerrisk <mtk.manpages@gmail.com> wrote:

> Can you please CC linux-api@ on all future iterations of this patch!

Sorry, yes - I remembered right after posting it, of course.

David

[toc] | [prev] | [next] | [standalone]

#1524500

One Thousand Gnomes <gnomes@lxorguk.ukuu.org.uk> wrote:

> >  (2) Lightweight stat (AT_STATX_DONT_SYNC): Ask for just those details of
> >      interest, and allow a network fs to approximate anything not of
> >      interest, without going to the server.
> > 
> >  (3) Heavyweight stat (AT_STATX_FORCE_SYNC): Force a network fs to flush
> >      buffers and go to the server, even if it thinks its cached attributes
> >      are up to date.
> 
> That seems an odd way to do it. Wouldn't it be cleaner and more flexible
> to give a timestamp of the oldest time you consider acceptable (and
> obviously passing 0 indicates whatever you have)

Perhaps, though adding 6-argument syscalls is apparently frowned upon.

> > Note that no lstat() equivalent is required as that can be implemented
> > through statx() with atflag == 0.  There is also no fstat() equivalent as
> > that can be implemented through statx() with filename == NULL and the
> > relevant fd passed as dfd.
> 
> and dfd + a name gives you fstatat() ?

Yes.

> The cover note could be clearer on this.

Fixed.

> Should the fields really be split the way they are for times rather than
> a struct for each one so you can write code generically to handle one of
> those rather than having to have a 4 way switch statement all the time.

It depends.  Doing so leaves 16 bytes of hole in the structure.  I could
ameliorate the wastage by using a union to overlay useful fields in the gaps,
but that's pretty icky and might be compiler dependent.

> Another attribute that would be nice (but migt need some trivial device
> layer tweaking) would be STATX_ATTR_VOLATILE for filesystems that will
> probably evaporate on a reboot. That's useful information for tools like
> installers and also for sanity checking things like backup paths.

There's a FILE_ATTRIBUTE_TEMPORARY that I could map for windows filesystems
that could be used with this.

> Remote needs to have clear semantics: is ext4fs over nbd 'remote' for
> example ?

Hmmm... Interesting question.  Probably should.  But you could be insane and
RAID an nbd and a local disk.  Further, does NFS over a loopback device to
nfsd on the same machine qualify as root?  What if that's exposing a local fs
on NBD?  Perhaps I should drop 'REMOTE' for now.  It sounds like something
that a GUI filemanager might find interesting, though.

David

[toc] | [prev] | [next] | [standalone]

#1524792

On Thu, Nov 17, 2016 at 04:45:45PM +0000, David Howells wrote:
> One Thousand Gnomes <gnomes@lxorguk.ukuu.org.uk> wrote:
> 
> > >  (2) Lightweight stat (AT_STATX_DONT_SYNC): Ask for just those details of
> > >      interest, and allow a network fs to approximate anything not of
> > >      interest, without going to the server.
> > > 
> > >  (3) Heavyweight stat (AT_STATX_FORCE_SYNC): Force a network fs to flush
> > >      buffers and go to the server, even if it thinks its cached attributes
> > >      are up to date.
> > 
> > That seems an odd way to do it. Wouldn't it be cleaner and more flexible
> > to give a timestamp of the oldest time you consider acceptable (and
> > obviously passing 0 indicates whatever you have)
> 
> Perhaps, though adding 6-argument syscalls is apparently frowned upon.
> 
> > > Note that no lstat() equivalent is required as that can be implemented
> > > through statx() with atflag == 0.  There is also no fstat() equivalent as
> > > that can be implemented through statx() with filename == NULL and the
> > > relevant fd passed as dfd.
> > 
> > and dfd + a name gives you fstatat() ?
> 
> Yes.
> 
> > The cover note could be clearer on this.
> 
> Fixed.
> 
> > Should the fields really be split the way they are for times rather than
> > a struct for each one so you can write code generically to handle one of
> > those rather than having to have a 4 way switch statement all the time.
> 
> It depends.  Doing so leaves 16 bytes of hole in the structure.  I could
> ameliorate the wastage by using a union to overlay useful fields in the gaps,
> but that's pretty icky and might be compiler dependent.
> 
> > Another attribute that would be nice (but migt need some trivial device
> > layer tweaking) would be STATX_ATTR_VOLATILE for filesystems that will
> > probably evaporate on a reboot. That's useful information for tools like
> > installers and also for sanity checking things like backup paths.
> 
> There's a FILE_ATTRIBUTE_TEMPORARY that I could map for windows filesystems
> that could be used with this.
> 
> > Remote needs to have clear semantics: is ext4fs over nbd 'remote' for
> > example ?
> 
> Hmmm... Interesting question.  Probably should.  But you could be insane and
> RAID an nbd and a local disk.  Further, does NFS over a loopback device to
> nfsd on the same machine qualify as root?  What if that's exposing a local fs
> on NBD?  Perhaps I should drop 'REMOTE' for now.  It sounds like something
> that a GUI filemanager might find interesting, though.

Sorry, I haven't been paying attention, just popping up for this, but:
"shared" might be a more useful term than "remote".

A filesystem that may be mounted from more than one system is "shared".
Caching performance and semantics of such a filesystem are more
complicated since the filesystem may change out from under us.  This is
what makes e.g. the lightweight/heavyweight stat difference more
interesting in the shared case.

The filesystem should be able to make that shared/unshared distinction
without knowledge of the storage it's sitting on top of.

Answering your questions by that criterion:

	- ext4/nbd: not shared
	- nfs/lo: shared

But, it's fine with me to drop any features for now as long as we can
always add them later.

--b.

[toc] | [prev] | [next] | [standalone]

#1524979

[Multipart message — attachments visible in raw view] — view raw

> On Nov 17, 2016, at 1:00 PM, J. Bruce Fields <bfields@fieldses.org> wrote:
> 
> On Thu, Nov 17, 2016 at 04:45:45PM +0000, David Howells wrote:
>> One Thousand Gnomes <gnomes@lxorguk.ukuu.org.uk> wrote:
>> 
>>>> (2) Lightweight stat (AT_STATX_DONT_SYNC): Ask for just those details of
>>>>     interest, and allow a network fs to approximate anything not of
>>>>     interest, without going to the server.
>>>> 
>>>> (3) Heavyweight stat (AT_STATX_FORCE_SYNC): Force a network fs to flush
>>>>     buffers and go to the server, even if it thinks its cached attributes
>>>>     are up to date.
>>> 
>>> That seems an odd way to do it. Wouldn't it be cleaner and more flexible
>>> to give a timestamp of the oldest time you consider acceptable (and
>>> obviously passing 0 indicates whatever you have)
>> 
>> Perhaps, though adding 6-argument syscalls is apparently frowned upon.
>> 
>>>> Note that no lstat() equivalent is required as that can be implemented
>>>> through statx() with atflag == 0.  There is also no fstat() equivalent as
>>>> that can be implemented through statx() with filename == NULL and the
>>>> relevant fd passed as dfd.
>>> 
>>> and dfd + a name gives you fstatat() ?
>> 
>> Yes.
>> 
>>> The cover note could be clearer on this.
>> 
>> Fixed.
>> 
>>> Should the fields really be split the way they are for times rather than
>>> a struct for each one so you can write code generically to handle one of
>>> those rather than having to have a 4 way switch statement all the time.
>> 
>> It depends.  Doing so leaves 16 bytes of hole in the structure.  I could
>> ameliorate the wastage by using a union to overlay useful fields in the gaps,
>> but that's pretty icky and might be compiler dependent.
>> 
>>> Another attribute that would be nice (but migt need some trivial device
>>> layer tweaking) would be STATX_ATTR_VOLATILE for filesystems that will
>>> probably evaporate on a reboot. That's useful information for tools like
>>> installers and also for sanity checking things like backup paths.
>> 
>> There's a FILE_ATTRIBUTE_TEMPORARY that I could map for windows filesystems
>> that could be used with this.
>> 
>>> Remote needs to have clear semantics: is ext4fs over nbd 'remote' for
>>> example ?
>> 
>> Hmmm... Interesting question.  Probably should.  But you could be insane and
>> RAID an nbd and a local disk.  Further, does NFS over a loopback device to
>> nfsd on the same machine qualify as root?  What if that's exposing a local fs
>> on NBD?  Perhaps I should drop 'REMOTE' for now.  It sounds like something
>> that a GUI filemanager might find interesting, though.
> 
> Sorry, I haven't been paying attention, just popping up for this, but:
> "shared" might be a more useful term than "remote".
> 
> A filesystem that may be mounted from more than one system is "shared".
> Caching performance and semantics of such a filesystem are more
> complicated since the filesystem may change out from under us.  This is
> what makes e.g. the lightweight/heavyweight stat difference more
> interesting in the shared case.
> 
> The filesystem should be able to make that shared/unshared distinction
> without knowledge of the storage it's sitting on top of.
> 
> Answering your questions by that criterion:
> 
> 	- ext4/nbd: not shared
> 	- nfs/lo: shared
> 
> But, it's fine with me to drop any features for now as long as we can
> always add them later.

Please, please, please, let's get the syscall and basic functionality
landed first, and then nit-pick about extensions later.  This has been
dragging on for _years_ and bike shedded to death.

Cheers, Andreas

[toc] | [prev] | [next] | [standalone]

#1525000

[Multipart message — attachments visible in raw view] — view raw

On Fri, Nov 18 2016, Andreas Dilger wrote:

> [ Unknown signature status ]
>
>> On Nov 17, 2016, at 1:00 PM, J. Bruce Fields <bfields@fieldses.org> wrote:
>> 
>> On Thu, Nov 17, 2016 at 04:45:45PM +0000, David Howells wrote:
>>> One Thousand Gnomes <gnomes@lxorguk.ukuu.org.uk> wrote:
>>> 
>>>>> (2) Lightweight stat (AT_STATX_DONT_SYNC): Ask for just those details of
>>>>>     interest, and allow a network fs to approximate anything not of
>>>>>     interest, without going to the server.
>>>>> 
>>>>> (3) Heavyweight stat (AT_STATX_FORCE_SYNC): Force a network fs to flush
>>>>>     buffers and go to the server, even if it thinks its cached attributes
>>>>>     are up to date.
>>>> 
>>>> That seems an odd way to do it. Wouldn't it be cleaner and more flexible
>>>> to give a timestamp of the oldest time you consider acceptable (and
>>>> obviously passing 0 indicates whatever you have)
>>> 
>>> Perhaps, though adding 6-argument syscalls is apparently frowned upon.
>>> 
>>>>> Note that no lstat() equivalent is required as that can be implemented
>>>>> through statx() with atflag == 0.  There is also no fstat() equivalent as
>>>>> that can be implemented through statx() with filename == NULL and the
>>>>> relevant fd passed as dfd.
>>>> 
>>>> and dfd + a name gives you fstatat() ?
>>> 
>>> Yes.
>>> 
>>>> The cover note could be clearer on this.
>>> 
>>> Fixed.
>>> 
>>>> Should the fields really be split the way they are for times rather than
>>>> a struct for each one so you can write code generically to handle one of
>>>> those rather than having to have a 4 way switch statement all the time.
>>> 
>>> It depends.  Doing so leaves 16 bytes of hole in the structure.  I could
>>> ameliorate the wastage by using a union to overlay useful fields in the gaps,
>>> but that's pretty icky and might be compiler dependent.
>>> 
>>>> Another attribute that would be nice (but migt need some trivial device
>>>> layer tweaking) would be STATX_ATTR_VOLATILE for filesystems that will
>>>> probably evaporate on a reboot. That's useful information for tools like
>>>> installers and also for sanity checking things like backup paths.
>>> 
>>> There's a FILE_ATTRIBUTE_TEMPORARY that I could map for windows filesystems
>>> that could be used with this.
>>> 
>>>> Remote needs to have clear semantics: is ext4fs over nbd 'remote' for
>>>> example ?
>>> 
>>> Hmmm... Interesting question.  Probably should.  But you could be insane and
>>> RAID an nbd and a local disk.  Further, does NFS over a loopback device to
>>> nfsd on the same machine qualify as root?  What if that's exposing a local fs
>>> on NBD?  Perhaps I should drop 'REMOTE' for now.  It sounds like something
>>> that a GUI filemanager might find interesting, though.
>> 
>> Sorry, I haven't been paying attention, just popping up for this, but:
>> "shared" might be a more useful term than "remote".
>> 
>> A filesystem that may be mounted from more than one system is "shared".
>> Caching performance and semantics of such a filesystem are more
>> complicated since the filesystem may change out from under us.  This is
>> what makes e.g. the lightweight/heavyweight stat difference more
>> interesting in the shared case.
>> 
>> The filesystem should be able to make that shared/unshared distinction
>> without knowledge of the storage it's sitting on top of.
>> 
>> Answering your questions by that criterion:
>> 
>> 	- ext4/nbd: not shared
>> 	- nfs/lo: shared
>> 
>> But, it's fine with me to drop any features for now as long as we can
>> always add them later.
>
> Please, please, please, let's get the syscall and basic functionality
> landed first, and then nit-pick about extensions later.  This has been
> dragging on for _years_ and bike shedded to death.

I very much agree with this, but I think it will require dropping (not
replacing yet) things that do not have a well defined meaning, including

> 	STATX_ATTR_KERNEL_API		File is kernel API (eg: procfs/sysfs)
> 	STATX_ATTR_REMOTE		File is remote and needs network
> 	STATX_ATTR_FABRICATED		File was made up by fs

Without clear guidance on how the filesystem should choose to set these,
and how a program should interpret them, they are worse than noise.
I imagine each could possibly be useful, but without clear unambiguous
documentation, they aren't.
So just remove them for now, and consider adding them once the core
syscall has landed.

NeilBrown

[toc] | [prev] | [next] | [standalone]

#1525343

> Hmmm... Interesting question.  Probably should.  But you could be insane and
> RAID an nbd and a local disk.  Further, does NFS over a loopback device to
> nfsd on the same machine qualify as root?  What if that's exposing a local fs
> on NBD?  Perhaps I should drop 'REMOTE' for now.  It sounds like something
> that a GUI filemanager might find interesting, though.

GUI file managers already try and guess some of this in order to decide
whether to display icons off remote file systems.

You could be insane but it's always going to be a hint and nothing more
and if it can't be perfect then that just goes in the notes in the manual
page.

Alan

[toc] | [prev] | [next] | [standalone]

#1525346

One Thousand Gnomes <gnomes@lxorguk.ukuu.org.uk> wrote:

> > Hmmm... Interesting question.  Probably should.  But you could be insane and
> > RAID an nbd and a local disk.  Further, does NFS over a loopback device to
> > nfsd on the same machine qualify as root?  What if that's exposing a local fs
> > on NBD?  Perhaps I should drop 'REMOTE' for now.  It sounds like something
> > that a GUI filemanager might find interesting, though.
> 
> GUI file managers already try and guess some of this in order to decide
> whether to display icons off remote file systems.
> 
> You could be insane but it's always going to be a hint and nothing more
> and if it can't be perfect then that just goes in the notes in the manual
> page.

I've dropped REMOTE, FABRICATED, KERNEL_API, NONUNIX_OWNERSHIP, HAS_ACL and
UNLISTED_DENTS for now.  I'm keeping AUTOMOUNT as that seems fairly
straightforward - though I'm sure we can find someone to disagree! ;-)

David

[toc] | [prev] | [next] | [standalone]

#1524516

Hi David,

Can you please CC linux-api@ on all future iterations of this patch!

Thanks,

Michael


On Thu, Nov 17, 2016 at 2:34 PM, David Howells <dhowells@redhat.com> wrote:
>
> Implement a new system call to provide enhanced file stats.  The patches can
> be found here:
>
>         http://git.kernel.org/cgit/linux/kernel/git/dhowells/linux-fs.git/log/?h=xstat
>
>
> ===========
> DESCRIPTION
> ===========
>
> The first patch provides this new system call:
>
>         long ret = statx(int dfd,
>                          const char *filename,
>                          unsigned atflag,
>                          unsigned mask,
>                          struct statx *buffer);
>
> This is an enhanced file stat function that provides a number of useful
> features, in summary:
>
>  (1) More information: creation time, data version number and attributes.
>      A subset of these is available through a number of filesystems (such
>      as CIFS, NFS, AFS, Ext4 and BTRFS).
>
>  (2) Lightweight stat (AT_STATX_DONT_SYNC): Ask for just those details of
>      interest, and allow a network fs to approximate anything not of
>      interest, without going to the server.
>
>  (3) Heavyweight stat (AT_STATX_FORCE_SYNC): Force a network fs to flush
>      buffers and go to the server, even if it thinks its cached attributes
>      are up to date.
>
>  (4) Allow the filesystem to indicate what it can/cannot provide: A
>      filesystem can now say it doesn't support a standard stat feature if
>      that isn't available.
>
>  (5) Make the fields a consistent size on all arches, and make them large.
>
>  (6) Can be extended by using more request flags and using up the padding
>      space in the statx struct.
>
> Note that no lstat() equivalent is required as that can be implemented
> through statx() with atflag == 0.  There is also no fstat() equivalent as
> that can be implemented through statx() with filename == NULL and the
> relevant fd passed as dfd.
>
>
> =======
> TESTING
> =======
>
> A test program is added into samples/statx/ by the first patch.
>
> David
> ---
> David Howells (4):
>       statx: Add a system call to make enhanced file info available
>       statx: Ext4: Return enhanced file attributes
>       statx: NFS: Return enhanced file attributes
>       statx: AFS: Return enhanced file attributes
>
>
>  arch/x86/entry/syscalls/syscall_32.tbl |    1
>  arch/x86/entry/syscalls/syscall_64.tbl |    1
>  fs/afs/inode.c                         |   21 ++
>  fs/exportfs/expfs.c                    |    4
>  fs/ext4/ext4.h                         |    2
>  fs/ext4/file.c                         |    2
>  fs/ext4/inode.c                        |   38 ++++
>  fs/ext4/namei.c                        |    2
>  fs/ext4/symlink.c                      |    2
>  fs/nfs/inode.c                         |   41 ++++
>  fs/stat.c                              |  294 +++++++++++++++++++++++++++++---
>  include/linux/fs.h                     |    5 -
>  include/linux/stat.h                   |   19 +-
>  include/linux/syscalls.h               |    3
>  include/uapi/linux/fcntl.h             |    2
>  include/uapi/linux/stat.h              |  124 +++++++++++++
>  samples/Kconfig                        |    5 +
>  samples/Makefile                       |    3
>  samples/statx/Makefile                 |   10 +
>  samples/statx/test-statx.c             |  248 +++++++++++++++++++++++++++
>  20 files changed, 771 insertions(+), 56 deletions(-)
>  create mode 100644 samples/statx/Makefile
>  create mode 100644 samples/statx/test-statx.c
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html



-- 
Michael Kerrisk Linux man-pages maintainer;
http://www.kernel.org/doc/man-pages/
Author of "The Linux Programming Interface", http://blog.man7.org/

[toc] | [prev] | [next] | [standalone]

#1524518 — [PATCH 3/4] statx: NFS: Return enhanced file attributes

Return enhanced file atrributes from the NFS filesystem.  This includes the
following:

 (1) The change attribute as stx_version if NFSv4.

 (2) STATX_ATTR_AUTOMOUNT and STATX_ATTR_FABRICATED are set on referral or
     submount directories that are automounted upon.  NFS shows one
     directory with a different FSID, but the local VFS has two: the
     mountpoint directory (fabricated) and the root of the filesystem
     mounted upon it.

 (3) STATX_ATTR_REMOTE is set on files acquired over NFS.

Furthermore, what nfs_getattr() does can be controlled as follows:

 (1) If AT_STATX_DONT_SYNC is indicated then this will suppress the
     flushing of outstanding writes and the rereading of the inode's
     attributes with the server as detailed below.

 (2) Otherwise:

     (a) If AT_STATX_FORCE_SYNC is indicated, or mtime, ctime or
     	 data_version (NFSv4 only) are requested then the outstanding
     	 writes will be written to the server first.

     (b) The inode's attributes will be reread from the server:

     	 (i) if AT_STATX_FORCE_SYNC is indicated;

        (ii) if atime is requested (and atime updating is not suppressed by
     	     a mount flag); or

       (iii) if the cached attributes have expired;

If the inode isn't synchronised, then the cached attributes will be used -
even if expired - without reference to the server.

Example output:

	[root@andromeda ~]# ./samples/statx/test-statx /warthog/
	statx(/warthog/) = 0
	results=17ff
	  Size: 4096            Blocks: 8          IO Block: 1048576  directory
	Device: 00:26           Inode: 2           Links: 21
	Access: (0555/dr-xr-xr-x)  Uid:     0   Gid:     0
	Access: 2016-11-14 11:49:14.582749262+0000
	Modify: 2016-09-08 20:39:46.785788707+0100
	Change: 2016-09-08 20:39:46.785788707+0100
	Data version: 57d1be822ed62f23h
	Attributes: 0000000000002000 (-------- -------- -------- -------- -------- -------- --r----- --------)
	IO-blocksize: blksize=1048576

Note that the NFS4 protocol potentially provides a creation time that could
be passed through this interface and system, hidden and archive values that
could be passed as attributes.  There is also a backup time that could be
exposed.

Signed-off-by: David Howells <dhowells@redhat.com>
---

 fs/nfs/inode.c |   41 ++++++++++++++++++++++++++++++++++-------
 1 file changed, 34 insertions(+), 7 deletions(-)

diff --git a/fs/nfs/inode.c b/fs/nfs/inode.c
index bf4ec5ecc97e..73c4502d4abb 100644
--- a/fs/nfs/inode.c
+++ b/fs/nfs/inode.c
@@ -656,12 +656,23 @@ static bool nfs_need_revalidate_inode(struct inode *inode)
 int nfs_getattr(struct vfsmount *mnt, struct dentry *dentry, struct kstat *stat)
 {
 	struct inode *inode = d_inode(dentry);
-	int need_atime = NFS_I(inode)->cache_validity & NFS_INO_INVALID_ATIME;
+	bool force_sync = stat->query_flags & AT_STATX_FORCE_SYNC;
+	bool suppress_sync = stat->query_flags & AT_STATX_DONT_SYNC;
+	bool need_atime = NFS_I(inode)->cache_validity & NFS_INO_INVALID_ATIME;
 	int err = 0;
 
 	trace_nfs_getattr_enter(inode);
-	/* Flush out writes to the server in order to update c/mtime.  */
-	if (S_ISREG(inode->i_mode)) {
+
+	if (NFS_SERVER(inode)->nfs_client->rpc_ops->version < 4)
+		stat->request_mask &= ~STATX_VERSION;
+
+	/* Flush out writes to the server in order to update c/mtime or data
+	 * version if the user wants them.
+	 */
+	if (S_ISREG(inode->i_mode) && !suppress_sync &&
+	    (force_sync || (stat->request_mask &
+			    (STATX_MTIME | STATX_CTIME | STATX_VERSION)))
+	    ) {
 		err = filemap_write_and_wait(inode->i_mapping);
 		if (err)
 			goto out;
@@ -676,11 +687,13 @@ int nfs_getattr(struct vfsmount *mnt, struct dentry *dentry, struct kstat *stat)
 	 *  - NFS never sets MS_NOATIME or MS_NODIRATIME so there is
 	 *    no point in checking those.
 	 */
- 	if ((mnt->mnt_flags & MNT_NOATIME) ||
- 	    ((mnt->mnt_flags & MNT_NODIRATIME) && S_ISDIR(inode->i_mode)))
-		need_atime = 0;
+	if (!(stat->request_mask & STATX_ATIME) ||
+	    (mnt->mnt_flags & MNT_NOATIME) ||
+	    ((mnt->mnt_flags & MNT_NODIRATIME) && S_ISDIR(inode->i_mode)))
+		need_atime = false;
 
-	if (need_atime || nfs_need_revalidate_inode(inode)) {
+	if (!suppress_sync &&
+	    (force_sync || need_atime || nfs_need_revalidate_inode(inode))) {
 		struct nfs_server *server = NFS_SERVER(inode);
 
 		if (server->caps & NFS_CAP_READDIRPLUS)
@@ -693,6 +706,20 @@ int nfs_getattr(struct vfsmount *mnt, struct dentry *dentry, struct kstat *stat)
 		if (S_ISDIR(inode->i_mode))
 			stat->blksize = NFS_SERVER(inode)->dtsize;
 	}
+
+	generic_fillattr(inode, stat);
+	stat->ino = nfs_compat_user_ino64(NFS_FILEID(inode));
+
+	if (stat->request_mask & STATX_VERSION) {
+		stat->version = inode->i_version;
+		stat->result_mask |= STATX_VERSION;
+	}
+
+	if (IS_AUTOMOUNT(inode))
+		stat->attributes |= STATX_ATTR_FABRICATED;
+
+	stat->attributes |= STATX_ATTR_REMOTE;
+
 out:
 	trace_nfs_getattr_exit(inode, err);
 	return err;

[toc] | [prev] | [next] | [standalone]

#1524537 — [PATCH 2/4] statx: Ext4: Return enhanced file attributes

Return enhanced file attributes from the Ext4 filesystem.  This includes
the following:

 (1) The inode creation time (i_crtime) as stx_btime, setting STATX_BTIME.

 (2) The inode i_version as stx_version if a file with I_VERSION set or a
     directory, setting STATX_VERSION.

 (3) Certain FS_xxx_FL flags are copied to stx_attribute flags.


This requires that all ext4 inodes have a getattr call, not just some of
them, so to this end, split the ext4_getattr() function and only call part
of it where appropriate.

Example output:

	[root@andromeda ~]# touch foo
	[root@andromeda ~]# chattr +ai foo
	[root@andromeda ~]# /tmp/test-statx foo
	statx(foo) = 0
	results=fff
	  Size: 0               Blocks: 0          IO Block: 4096    regular file
	Device: 08:12           Inode: 2101950     Links: 1
	Access: (0644/-rw-r--r--)  Uid:     0   Gid:     0
	Access: 2016-02-11 17:08:29.031795451+0000
	Modify: 2016-02-11 17:08:29.031795451+0000
	Change: 2016-02-11 17:11:11.987790114+0000
	 Birth: 2016-02-11 17:08:29.031795451+0000
	Attributes: 0000000000000030 (-------- -------- -------- -------- -------- -------- -------- --ai----)
	IO-blocksize: blksize=4096

Signed-off-by: David Howells <dhowells@redhat.com>
---

 fs/ext4/ext4.h    |    2 ++
 fs/ext4/file.c    |    2 +-
 fs/ext4/inode.c   |   38 +++++++++++++++++++++++++++++++++++---
 fs/ext4/namei.c   |    2 ++
 fs/ext4/symlink.c |    2 ++
 5 files changed, 42 insertions(+), 4 deletions(-)

diff --git a/fs/ext4/ext4.h b/fs/ext4/ext4.h
index 282a51b07c57..f65e4a560c4c 100644
--- a/fs/ext4/ext4.h
+++ b/fs/ext4/ext4.h
@@ -2485,6 +2485,8 @@ extern int  ext4_getattr(struct vfsmount *mnt, struct dentry *dentry,
 				struct kstat *stat);
 extern void ext4_evict_inode(struct inode *);
 extern void ext4_clear_inode(struct inode *);
+extern int  ext4_file_getattr(struct vfsmount *mnt, struct dentry *dentry,
+			      struct kstat *stat);
 extern int  ext4_sync_inode(handle_t *, struct inode *);
 extern void ext4_dirty_inode(struct inode *, int);
 extern int ext4_change_inode_journal_flag(struct inode *, int);
diff --git a/fs/ext4/file.c b/fs/ext4/file.c
index 2a822d30e73f..20bab4b0d6fc 100644
--- a/fs/ext4/file.c
+++ b/fs/ext4/file.c
@@ -705,7 +705,7 @@ const struct file_operations ext4_file_operations = {
 
 const struct inode_operations ext4_file_inode_operations = {
 	.setattr	= ext4_setattr,
-	.getattr	= ext4_getattr,
+	.getattr	= ext4_file_getattr,
 	.listxattr	= ext4_listxattr,
 	.get_acl	= ext4_get_acl,
 	.set_acl	= ext4_set_acl,
diff --git a/fs/ext4/inode.c b/fs/ext4/inode.c
index 9c064727ed62..91a49cbc4c63 100644
--- a/fs/ext4/inode.c
+++ b/fs/ext4/inode.c
@@ -5229,11 +5229,43 @@ int ext4_setattr(struct dentry *dentry, struct iattr *attr)
 int ext4_getattr(struct vfsmount *mnt, struct dentry *dentry,
 		 struct kstat *stat)
 {
-	struct inode *inode;
-	unsigned long long delalloc_blocks;
+	struct inode *inode = d_inode(dentry);
+	struct ext4_inode *raw_inode;
+	struct ext4_inode_info *ei = EXT4_I(inode);
+	unsigned int flags;
+
+	if (EXT4_FITS_IN_INODE(raw_inode, ei, i_crtime)) {
+		stat->result_mask |= STATX_BTIME;
+		stat->btime.tv_sec = ei->i_crtime.tv_sec;
+		stat->btime.tv_nsec = ei->i_crtime.tv_nsec;
+	}
+
+	if (S_ISDIR(inode->i_mode) || IS_I_VERSION(inode)) {
+		stat->result_mask |= STATX_VERSION;
+		stat->version = inode->i_version;
+	}
+
+	ext4_get_inode_flags(ei);
+	flags = ei->i_flags & EXT4_FL_USER_VISIBLE;
+	stat->attributes |= flags & (EXT4_IMMUTABLE_FL | EXT4_NODUMP_FL);
+	if (flags & EXT4_APPEND_FL)
+		stat->attributes |= STATX_ATTR_APPEND;
+	if (flags & EXT4_COMPR_FL)
+		stat->attributes |= STATX_ATTR_COMPRESSED;
+	if (flags & EXT4_ENCRYPT_FL)
+		stat->attributes |= STATX_ATTR_ENCRYPTED;
 
-	inode = d_inode(dentry);
 	generic_fillattr(inode, stat);
+	return 0;
+}
+
+int ext4_file_getattr(struct vfsmount *mnt, struct dentry *dentry,
+		      struct kstat *stat)
+{
+	struct inode *inode = dentry->d_inode;
+	u64 delalloc_blocks;
+
+	ext4_getattr(mnt, dentry, stat);
 
 	/*
 	 * If there is inline data in the inode, the inode will normally not
diff --git a/fs/ext4/namei.c b/fs/ext4/namei.c
index 104f8bfba718..e115281fb8c5 100644
--- a/fs/ext4/namei.c
+++ b/fs/ext4/namei.c
@@ -3882,6 +3882,7 @@ const struct inode_operations ext4_dir_inode_operations = {
 	.tmpfile	= ext4_tmpfile,
 	.rename		= ext4_rename2,
 	.setattr	= ext4_setattr,
+	.getattr	= ext4_getattr,
 	.listxattr	= ext4_listxattr,
 	.get_acl	= ext4_get_acl,
 	.set_acl	= ext4_set_acl,
@@ -3890,6 +3891,7 @@ const struct inode_operations ext4_dir_inode_operations = {
 
 const struct inode_operations ext4_special_inode_operations = {
 	.setattr	= ext4_setattr,
+	.getattr	= ext4_getattr,
 	.listxattr	= ext4_listxattr,
 	.get_acl	= ext4_get_acl,
 	.set_acl	= ext4_set_acl,
diff --git a/fs/ext4/symlink.c b/fs/ext4/symlink.c
index 557b3b0d668c..209b833633e2 100644
--- a/fs/ext4/symlink.c
+++ b/fs/ext4/symlink.c
@@ -93,6 +93,7 @@ const struct inode_operations ext4_symlink_inode_operations = {
 	.readlink	= generic_readlink,
 	.get_link	= page_get_link,
 	.setattr	= ext4_setattr,
+	.getattr	= ext4_getattr,
 	.listxattr	= ext4_listxattr,
 };
 
@@ -100,5 +101,6 @@ const struct inode_operations ext4_fast_symlink_inode_operations = {
 	.readlink	= generic_readlink,
 	.get_link	= simple_get_link,
 	.setattr	= ext4_setattr,
+	.getattr	= ext4_getattr,
 	.listxattr	= ext4_listxattr,
 };

[toc] | [prev] | [next] | [standalone]

#1524995 — Re: [PATCH 2/4] statx: Ext4: Return enhanced file attributes

[Multipart message — attachments visible in raw view] — view raw

On Nov 17, 2016, at 6:35 AM, David Howells <dhowells@redhat.com> wrote:
> 
> Return enhanced file attributes from the Ext4 filesystem.  This includes
> the following:
> 
> (1) The inode creation time (i_crtime) as stx_btime, setting STATX_BTIME.
> 
> (2) The inode i_version as stx_version if a file with I_VERSION set or a
>     directory, setting STATX_VERSION.
> 
> (3) Certain FS_xxx_FL flags are copied to stx_attribute flags.
> 
> 
> This requires that all ext4 inodes have a getattr call, not just some of
> them, so to this end, split the ext4_getattr() function and only call part
> of it where appropriate.
> 
> Example output:
> 
> 	[root@andromeda ~]# touch foo
> 	[root@andromeda ~]# chattr +ai foo
> 	[root@andromeda ~]# /tmp/test-statx foo
> 	statx(foo) = 0
> 	results=fff
> 	  Size: 0               Blocks: 0          IO Block: 4096    regular file
> 	Device: 08:12           Inode: 2101950     Links: 1
> 	Access: (0644/-rw-r--r--)  Uid:     0   Gid:     0
> 	Access: 2016-02-11 17:08:29.031795451+0000
> 	Modify: 2016-02-11 17:08:29.031795451+0000
> 	Change: 2016-02-11 17:11:11.987790114+0000
> 	 Birth: 2016-02-11 17:08:29.031795451+0000
> 	Attributes: 0000000000000030 (-------- -------- -------- -------- -------- -------- -------- --ai----)
> 	IO-blocksize: blksize=4096
> 
> Signed-off-by: David Howells <dhowells@redhat.com>

Reviewed-by: Andreas Dilger <adilger@dilger.ca>

> ---
> 
> fs/ext4/ext4.h    |    2 ++
> fs/ext4/file.c    |    2 +-
> fs/ext4/inode.c   |   38 +++++++++++++++++++++++++++++++++++---
> fs/ext4/namei.c   |    2 ++
> fs/ext4/symlink.c |    2 ++
> 5 files changed, 42 insertions(+), 4 deletions(-)
> 
> diff --git a/fs/ext4/ext4.h b/fs/ext4/ext4.h
> index 282a51b07c57..f65e4a560c4c 100644
> --- a/fs/ext4/ext4.h
> +++ b/fs/ext4/ext4.h
> @@ -2485,6 +2485,8 @@ extern int  ext4_getattr(struct vfsmount *mnt, struct dentry *dentry,
> 				struct kstat *stat);
> extern void ext4_evict_inode(struct inode *);
> extern void ext4_clear_inode(struct inode *);
> +extern int  ext4_file_getattr(struct vfsmount *mnt, struct dentry *dentry,
> +			      struct kstat *stat);
> extern int  ext4_sync_inode(handle_t *, struct inode *);
> extern void ext4_dirty_inode(struct inode *, int);
> extern int ext4_change_inode_journal_flag(struct inode *, int);
> diff --git a/fs/ext4/file.c b/fs/ext4/file.c
> index 2a822d30e73f..20bab4b0d6fc 100644
> --- a/fs/ext4/file.c
> +++ b/fs/ext4/file.c
> @@ -705,7 +705,7 @@ const struct file_operations ext4_file_operations = {
> 
> const struct inode_operations ext4_file_inode_operations = {
> 	.setattr	= ext4_setattr,
> -	.getattr	= ext4_getattr,
> +	.getattr	= ext4_file_getattr,
> 	.listxattr	= ext4_listxattr,
> 	.get_acl	= ext4_get_acl,
> 	.set_acl	= ext4_set_acl,
> diff --git a/fs/ext4/inode.c b/fs/ext4/inode.c
> index 9c064727ed62..91a49cbc4c63 100644
> --- a/fs/ext4/inode.c
> +++ b/fs/ext4/inode.c
> @@ -5229,11 +5229,43 @@ int ext4_setattr(struct dentry *dentry, struct iattr *attr)
> int ext4_getattr(struct vfsmount *mnt, struct dentry *dentry,
> 		 struct kstat *stat)
> {
> -	struct inode *inode;
> -	unsigned long long delalloc_blocks;
> +	struct inode *inode = d_inode(dentry);
> +	struct ext4_inode *raw_inode;
> +	struct ext4_inode_info *ei = EXT4_I(inode);
> +	unsigned int flags;
> +
> +	if (EXT4_FITS_IN_INODE(raw_inode, ei, i_crtime)) {
> +		stat->result_mask |= STATX_BTIME;
> +		stat->btime.tv_sec = ei->i_crtime.tv_sec;
> +		stat->btime.tv_nsec = ei->i_crtime.tv_nsec;
> +	}
> +
> +	if (S_ISDIR(inode->i_mode) || IS_I_VERSION(inode)) {
> +		stat->result_mask |= STATX_VERSION;
> +		stat->version = inode->i_version;
> +	}
> +
> +	ext4_get_inode_flags(ei);
> +	flags = ei->i_flags & EXT4_FL_USER_VISIBLE;
> +	stat->attributes |= flags & (EXT4_IMMUTABLE_FL | EXT4_NODUMP_FL);
> +	if (flags & EXT4_APPEND_FL)
> +		stat->attributes |= STATX_ATTR_APPEND;
> +	if (flags & EXT4_COMPR_FL)
> +		stat->attributes |= STATX_ATTR_COMPRESSED;
> +	if (flags & EXT4_ENCRYPT_FL)
> +		stat->attributes |= STATX_ATTR_ENCRYPTED;
> 
> -	inode = d_inode(dentry);
> 	generic_fillattr(inode, stat);
> +	return 0;
> +}
> +
> +int ext4_file_getattr(struct vfsmount *mnt, struct dentry *dentry,
> +		      struct kstat *stat)
> +{
> +	struct inode *inode = dentry->d_inode;
> +	u64 delalloc_blocks;
> +
> +	ext4_getattr(mnt, dentry, stat);
> 
> 	/*
> 	 * If there is inline data in the inode, the inode will normally not
> diff --git a/fs/ext4/namei.c b/fs/ext4/namei.c
> index 104f8bfba718..e115281fb8c5 100644
> --- a/fs/ext4/namei.c
> +++ b/fs/ext4/namei.c
> @@ -3882,6 +3882,7 @@ const struct inode_operations ext4_dir_inode_operations = {
> 	.tmpfile	= ext4_tmpfile,
> 	.rename		= ext4_rename2,
> 	.setattr	= ext4_setattr,
> +	.getattr	= ext4_getattr,
> 	.listxattr	= ext4_listxattr,
> 	.get_acl	= ext4_get_acl,
> 	.set_acl	= ext4_set_acl,
> @@ -3890,6 +3891,7 @@ const struct inode_operations ext4_dir_inode_operations = {
> 
> const struct inode_operations ext4_special_inode_operations = {
> 	.setattr	= ext4_setattr,
> +	.getattr	= ext4_getattr,
> 	.listxattr	= ext4_listxattr,
> 	.get_acl	= ext4_get_acl,
> 	.set_acl	= ext4_set_acl,
> diff --git a/fs/ext4/symlink.c b/fs/ext4/symlink.c
> index 557b3b0d668c..209b833633e2 100644
> --- a/fs/ext4/symlink.c
> +++ b/fs/ext4/symlink.c
> @@ -93,6 +93,7 @@ const struct inode_operations ext4_symlink_inode_operations = {
> 	.readlink	= generic_readlink,
> 	.get_link	= page_get_link,
> 	.setattr	= ext4_setattr,
> +	.getattr	= ext4_getattr,
> 	.listxattr	= ext4_listxattr,
> };
> 
> @@ -100,5 +101,6 @@ const struct inode_operations ext4_fast_symlink_inode_operations = {
> 	.readlink	= generic_readlink,
> 	.get_link	= simple_get_link,
> 	.setattr	= ext4_setattr,
> +	.getattr	= ext4_getattr,
> 	.listxattr	= ext4_listxattr,
> };
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html


Cheers, Andreas

[toc] | [prev] | [next] | [standalone]

#1524541 — [PATCH 4/4] statx: AFS: Return enhanced file attributes

Return enhanced file attributes from the AFS filesystem.  This includes the
following:

 (1) The data version number as st_version, setting STATX_VERSION.

 (2) STATX_ATTR_AUTOMOUNT will be set on automount directories by virtue of
     S_AUTOMOUNT being set on the inode.  These are referrals to other
     volumes or other cells.

 (3) STATX_ATTR_UNLISTED_DENTS on a directory that does cell lookup for
     non-existent names and mounts them (typically mounted on /afs with -o
     autocell).  The resulting directories are marked STATX_ATTR_FABRICATED
     as they do not actually exist in the mounted AFS directory.

 (4) Files, directories and symlinks accessed over AFS are marked
     STATX_ATTR_REMOTE.

STATX_ATIME, STATX_CTIME and STATX_BLOCKS are cleared as AFS does not
support them.

Example output:

	[root@andromeda ~]# ./samples/statx/test-statx /afs
	statx(/afs) = 0
	results=7ef
	  Size: 2048            Blocks: 0          IO Block: 4096    directory
	Device: 00:25           Inode: 1           Links: 2
	Access: (0777/drwxrwxrwx)  Uid:     0   Gid:     0
	Access: 2006-05-07 00:21:15.000000000+0100
	Modify: 2006-05-07 00:21:15.000000000+0100
	Change: 2006-05-07 00:21:15.000000000+0100
	IO-blocksize: blksize=4096

Signed-off-by: David Howells <dhowells@redhat.com>
---

 fs/afs/inode.c |   21 ++++++++++++++++-----
 1 file changed, 16 insertions(+), 5 deletions(-)

diff --git a/fs/afs/inode.c b/fs/afs/inode.c
index 86cc7264c21c..b08c405a7e1b 100644
--- a/fs/afs/inode.c
+++ b/fs/afs/inode.c
@@ -72,9 +72,9 @@ static int afs_inode_map_status(struct afs_vnode *vnode, struct key *key)
 	inode->i_uid		= vnode->status.owner;
 	inode->i_gid		= GLOBAL_ROOT_GID;
 	inode->i_size		= vnode->status.size;
-	inode->i_ctime.tv_sec	= vnode->status.mtime_server;
-	inode->i_ctime.tv_nsec	= 0;
-	inode->i_atime		= inode->i_mtime = inode->i_ctime;
+	inode->i_mtime.tv_sec	= vnode->status.mtime_server;
+	inode->i_mtime.tv_nsec	= 0;
+	inode->i_atime		= inode->i_ctime = inode->i_mtime;
 	inode->i_blocks		= 0;
 	inode->i_generation	= vnode->fid.unique;
 	inode->i_version	= vnode->status.data_version;
@@ -375,8 +375,7 @@ int afs_validate(struct afs_vnode *vnode, struct key *key)
 /*
  * read the attributes of an inode
  */
-int afs_getattr(struct vfsmount *mnt, struct dentry *dentry,
-		      struct kstat *stat)
+int afs_getattr(struct vfsmount *mnt, struct dentry *dentry, struct kstat *stat)
 {
 	struct inode *inode;
 
@@ -385,6 +384,18 @@ int afs_getattr(struct vfsmount *mnt, struct dentry *dentry,
 	_enter("{ ino=%lu v=%u }", inode->i_ino, inode->i_generation);
 
 	generic_fillattr(inode, stat);
+
+	stat->result_mask &= ~(STATX_ATIME | STATX_CTIME | STATX_BLOCKS);
+	stat->result_mask |= STATX_VERSION;
+	stat->version = inode->i_version;
+
+	if (test_bit(AFS_VNODE_AUTOCELL, &AFS_FS_I(inode)->flags))
+		stat->attributes |= STATX_ATTR_UNLISTED_DENTS;
+
+	if (test_bit(AFS_VNODE_PSEUDODIR, &AFS_FS_I(inode)->flags))
+		stat->attributes |= STATX_ATTR_FABRICATED;
+	else
+		stat->attributes |= STATX_ATTR_REMOTE;
 	return 0;
 }
 

[toc] | [prev] | [next] | [standalone]

#1525075 — Re: [PATCH 4/4] statx: AFS: Return enhanced file attributes

[Multipart message — attachments visible in raw view] — view raw

On Nov 17, 2016, at 6:35 AM, David Howells <dhowells@redhat.com> wrote:
> 
> Return enhanced file attributes from the AFS filesystem.  This includes the
> following:
> 
> (1) The data version number as st_version, setting STATX_VERSION.
> 
> (2) STATX_ATTR_AUTOMOUNT will be set on automount directories by virtue of
>     S_AUTOMOUNT being set on the inode.  These are referrals to other
>     volumes or other cells.
> 
> (3) STATX_ATTR_UNLISTED_DENTS on a directory that does cell lookup for
>     non-existent names and mounts them (typically mounted on /afs with -o
>     autocell).  The resulting directories are marked STATX_ATTR_FABRICATED
>     as they do not actually exist in the mounted AFS directory.
> 
> (4) Files, directories and symlinks accessed over AFS are marked
>     STATX_ATTR_REMOTE.
> 
> STATX_ATIME, STATX_CTIME and STATX_BLOCKS are cleared as AFS does not
> support them.

Rather than clearing specific flags, wouldn't it be better to explicitly
set the flags that are actually being returned?  Otherwise, this would
have the problem that Dave pointed out on the 0/4 patch, that there may
be flags still set from userspace that do not mean anything to AFS.

Cheers, Andreas

> 
> Example output:
> 
> 	[root@andromeda ~]# ./samples/statx/test-statx /afs
> 	statx(/afs) = 0
> 	results=7ef
> 	  Size: 2048            Blocks: 0          IO Block: 4096    directory
> 	Device: 00:25           Inode: 1           Links: 2
> 	Access: (0777/drwxrwxrwx)  Uid:     0   Gid:     0
> 	Access: 2006-05-07 00:21:15.000000000+0100
> 	Modify: 2006-05-07 00:21:15.000000000+0100
> 	Change: 2006-05-07 00:21:15.000000000+0100
> 	IO-blocksize: blksize=4096
> 
> Signed-off-by: David Howells <dhowells@redhat.com>
> ---
> 
> fs/afs/inode.c |   21 ++++++++++++++++-----
> 1 file changed, 16 insertions(+), 5 deletions(-)
> 
> diff --git a/fs/afs/inode.c b/fs/afs/inode.c
> index 86cc7264c21c..b08c405a7e1b 100644
> --- a/fs/afs/inode.c
> +++ b/fs/afs/inode.c
> @@ -72,9 +72,9 @@ static int afs_inode_map_status(struct afs_vnode *vnode, struct key *key)
> 	inode->i_uid		= vnode->status.owner;
> 	inode->i_gid		= GLOBAL_ROOT_GID;
> 	inode->i_size		= vnode->status.size;
> -	inode->i_ctime.tv_sec	= vnode->status.mtime_server;
> -	inode->i_ctime.tv_nsec	= 0;
> -	inode->i_atime		= inode->i_mtime = inode->i_ctime;
> +	inode->i_mtime.tv_sec	= vnode->status.mtime_server;
> +	inode->i_mtime.tv_nsec	= 0;
> +	inode->i_atime		= inode->i_ctime = inode->i_mtime;
> 	inode->i_blocks		= 0;
> 	inode->i_generation	= vnode->fid.unique;
> 	inode->i_version	= vnode->status.data_version;
> @@ -375,8 +375,7 @@ int afs_validate(struct afs_vnode *vnode, struct key *key)
> /*
>  * read the attributes of an inode
>  */
> -int afs_getattr(struct vfsmount *mnt, struct dentry *dentry,
> -		      struct kstat *stat)
> +int afs_getattr(struct vfsmount *mnt, struct dentry *dentry, struct kstat *stat)
> {
> 	struct inode *inode;
> 
> @@ -385,6 +384,18 @@ int afs_getattr(struct vfsmount *mnt, struct dentry *dentry,
> 	_enter("{ ino=%lu v=%u }", inode->i_ino, inode->i_generation);
> 
> 	generic_fillattr(inode, stat);
> +
> +	stat->result_mask &= ~(STATX_ATIME | STATX_CTIME | STATX_BLOCKS);
> +	stat->result_mask |= STATX_VERSION;
> +	stat->version = inode->i_version;
> +
> +	if (test_bit(AFS_VNODE_AUTOCELL, &AFS_FS_I(inode)->flags))
> +		stat->attributes |= STATX_ATTR_UNLISTED_DENTS;
> +
> +	if (test_bit(AFS_VNODE_PSEUDODIR, &AFS_FS_I(inode)->flags))
> +		stat->attributes |= STATX_ATTR_FABRICATED;
> +	else
> +		stat->attributes |= STATX_ATTR_REMOTE;
> 	return 0;
> }
> 
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html


Cheers, Andreas

[toc] | [prev] | [next] | [standalone]

#1525094 — Re: [PATCH 4/4] statx: AFS: Return enhanced file attributes

Andreas Dilger <adilger@dilger.ca> wrote:

> > STATX_ATIME, STATX_CTIME and STATX_BLOCKS are cleared as AFS does not
> > support them.
> 
> Rather than clearing specific flags, wouldn't it be better to explicitly
> set the flags that are actually being returned?  Otherwise, this would
> have the problem that Dave pointed out on the 0/4 patch, that there may
> be flags still set from userspace that do not mean anything to AFS.

I'm not sure it make a difference.  generic_fillattr() has to initialise
result_mask to STATX_BASIC_STATS for the support of any unmodified filesystem.
We can then either clear the bits we don't want or just overwrite the mask
entirely with the bits we do want.

Bits in request_mask that are beyond STATX_BASIC_STATS are not automatically
propagated to result_mask.

Possibly vfs_xgetattr_nosec() should preset the result_mask rather than doing
this in generic_fillattr().

David

[toc] | [prev] | [next] | [standalone]

#1524618

>  (2) Lightweight stat (AT_STATX_DONT_SYNC): Ask for just those details of
>      interest, and allow a network fs to approximate anything not of
>      interest, without going to the server.
> 
>  (3) Heavyweight stat (AT_STATX_FORCE_SYNC): Force a network fs to flush
>      buffers and go to the server, even if it thinks its cached attributes
>      are up to date.

That seems an odd way to do it. Wouldn't it be cleaner and more flexible
to give a timestamp of the oldest time you consider acceptable (and
obviously passing 0 indicates whatever you have)

>  (4) Allow the filesystem to indicate what it can/cannot provide: A
>      filesystem can now say it doesn't support a standard stat feature if
>      that isn't available.
> 
>  (5) Make the fields a consistent size on all arches, and make them large.
> 
>  (6) Can be extended by using more request flags and using up the padding
>      space in the statx struct.
> 
> Note that no lstat() equivalent is required as that can be implemented
> through statx() with atflag == 0.  There is also no fstat() equivalent as
> that can be implemented through statx() with filename == NULL and the
> relevant fd passed as dfd.

and dfd + a name gives you fstatat() ? The cover note could be clearer on
this.

Should the fields really be split the way they are for times rather than
a struct for each one so you can write code generically to handle one of
those rather than having to have a 4 way switch statement all the time.

Another attribute that would be nice (but migt need some trivial device
layer tweaking) would be STATX_ATTR_VOLATILE for filesystems that will
probably evaporate on a reboot. That's useful information for tools like
installers and also for sanity checking things like backup paths.

Remote needs to have clear semantics: is ext4fs over nbd 'remote' for
example ?

Alan

[toc] | [prev] | [next] | [standalone]

#1524895 — Re: [PATCH 1/4] statx: Add a system call to make enhanced file info available

On Thu, Nov 17, 2016 at 01:35:03PM +0000, David Howells wrote:
> Add a system call to make extended file information available, including
> file creation, data version and some attribute flags where available
> through the underlying filesystem.
> 
> 
> ========
> OVERVIEW
> ========
> 
> The idea was initially proposed as a set of xattrs that could be retrieved
> with getxattr(), but the general preferance proved to be for a new syscall
> with an extended stat structure.
> 
> This has a number of uses:
> 
....
>  (5) Data version number: Could be used by userspace NFS servers [Aneesh
>      Kumar].
> 
>      Can also be used to modify fill_post_wcc() in NFSD which retrieves
>      i_version directly, but has just called vfs_getattr().  It could get
>      it from the kstat struct if it used vfs_xgetattr() instead.

This needs a much clearer name that stx_version as "version" is
entirely ambiguous. e.g. Inodes have internal version numbers to
disambiguate life cycles. and there are versioning filesystems
which have multiple versions of the same file. 

So if stx_version this is intended to export the internal filesystem
inode change counter (i.e. inode->i_version) then lets call it that:
stx_modification_count. It's clear and unambiguous as to what it
represents, especially as this counter is more than just a "data
modification" counter - inode metadata modifications will also
cause it to change....


....

> (13) FS_IOC_GETFLAGS value.  These could be translated to BSD's st_flags.
>      Note that the Linux IOC flags are a mess and filesystems such as Ext4
>      define flags that aren't in linux/fs.h, so translation in the kernel
>      may be a necessity (or, possibly, we provide the filesystem type too).

And we now also have FS_IOC_FSGETXATTR that extends the flags
and information userspace can get from filesystems. It makes little
sense to now add xstat() and not add everything this interface
also exports...

> The defined bits in request_mask and stx_mask are:
> 
> 	STATX_TYPE		Want/got stx_mode & S_IFMT
> 	STATX_MODE		Want/got stx_mode & ~S_IFMT
> 	STATX_NLINK		Want/got stx_nlink
> 	STATX_UID		Want/got stx_uid
> 	STATX_GID		Want/got stx_gid
> 	STATX_ATIME		Want/got stx_atime{,_ns}
> 	STATX_MTIME		Want/got stx_mtime{,_ns}
> 	STATX_CTIME		Want/got stx_ctime{,_ns}
> 	STATX_INO		Want/got stx_ino
> 	STATX_SIZE		Want/got stx_size
> 	STATX_BLOCKS		Want/got stx_blocks
> 	STATX_BASIC_STATS	[The stuff in the normal stat struct]
> 	STATX_BTIME		Want/got stx_btime{,_ns}
> 	STATX_VERSION		Want/got stx_version
> 	STATX_ALL		[All currently available stuff]

What happens when an application uses STATX_ALL from a future kernel
that defines more flags than are initially supported, and that
application then is run on a kernel that onyl supports the initial
fields?

> stx_btime is the file creation time; stx_version is the data version number
> (i_version); stx_mask is a bitmask indicating the data provided; and
> __spares*[] are where as-yet undefined fields can be placed.
> 
> Time fields are split into separate seconds and nanoseconds fields to make
> packing easier and the granularities can be queried with the filesystem
> info system call.  Note that times will be negative if before 1970; in such
> a case, the nanosecond fields will also be negative if not zero.

So what happens in ten years time when we want to support
femptosecond resolution in the timestamp interface? We've got to
change everything to 64 bit? Shouldn't we just make everything
timestamp related 64 bit?

> 
> The bits defined in the stx_attributes field convey information about a
> file, how it is accessed, where it is and what it does.  The following
> attributes map to FS_*_FL flags and are the same numerical value:

Please isolate the new interface flags completely from the FS_*_FL
values. We should not repeat the mistake of tying values derived
from filesystem specific on-disk values to a user interface. 

> 	STATX_ATTR_COMPRESSED		File is compressed by the fs
> 	STATX_ATTR_IMMUTABLE		File is marked immutable
> 	STATX_ATTR_APPEND		File is append-only
> 	STATX_ATTR_NODUMP		File is not to be dumped
> 	STATX_ATTR_ENCRYPTED		File requires key to decrypt in fs
> 
> The supported flags are listed by:
> 
> 	STATX_ATTR_FS_IOC_FLAGS

Again, we have many more common and extended flags than this.
NOATIME and SYNC are two that immediately come to mind as generic
flags that should be in this...

> 
> [Are any other IOC flags of sufficient general interest to be exposed
> through this interface?]
> 
> New flags include:
> 
> 	STATX_ATTR_NONUNIX_OWNERSHIP	File doesn't have Unixy ownership
> 	STATX_ATTR_HAS_ACL		File has an ACL

So statx will require us to do ACL lookups? i.e. instead of just
reading the inode to get the information, we'll also have to do
extended attribute lookups? That's potentially very expensive if
the extended attribute is not stored in the inode....

> 	STATX_ATTR_KERNEL_API		File is kernel API (eg: procfs/sysfs)
> 	STATX_ATTR_REMOTE		File is remote and needs network
> 	STATX_ATTR_FABRICATED		File was made up by fs

Every file is fabricated by a filesystem :P

Perhaps you're wanting "virtual file" because it is has no physical
presence?

> Fields in struct statx come in a number of classes:
> 
>  (0) stx_dev_*, stx_blksize.
> 
>      These are local system information and are always available.

What does stx_blksize actually mean? It's completely ambiguous in
stat() because we don't actually report the physical block size
here - we report the "minimum unit of efficient IO" that we expect
applications to use. Please define :P


> =======
> TESTING
> =======
> 
> The following test program can be used to test the statx system call:
> 
> 	samples/statx/test-statx.c
> 
> Just compile and run, passing it paths to the files you want to examine.
> The file is built automatically if CONFIG_SAMPLES is enabled.

Can we get xfstests written to exercise and validate all this
functionality, please? I'd suggest that adding xfs_io support for
the statx syscall would be far more useful for xfstests than a
standalone  test program, too. We already have equivalent stat()
functionality in xfs_io and that's used quite a bit in xfstests....

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

[toc] | [prev] | [next] | [standalone]

#1524992 — Re: [PATCH 1/4] statx: Add a system call to make enhanced file info available

[Multipart message — attachments visible in raw view] — view raw

On Nov 17, 2016, at 4:40 PM, Dave Chinner <david@fromorbit.com> wrote:
> 
> On Thu, Nov 17, 2016 at 01:35:03PM +0000, David Howells wrote:
>> Add a system call to make extended file information available, including
>> file creation, data version and some attribute flags where available
>> through the underlying filesystem.
>> 
>> 
>> ========
>> OVERVIEW
>> ========
>> 
>> The idea was initially proposed as a set of xattrs that could be retrieved
>> with getxattr(), but the general preferance proved to be for a new syscall
>> with an extended stat structure.
>> 
>> This has a number of uses:
>> 
> ....
>> (5) Data version number: Could be used by userspace NFS servers [Aneesh
>>     Kumar].
>> 
>>     Can also be used to modify fill_post_wcc() in NFSD which retrieves
>>     i_version directly, but has just called vfs_getattr().  It could get
>>     it from the kstat struct if it used vfs_xgetattr() instead.
> 
> This needs a much clearer name that stx_version as "version" is
> entirely ambiguous. e.g. Inodes have internal version numbers to
> disambiguate life cycles. and there are versioning filesystems
> which have multiple versions of the same file.
> 
> So if stx_version this is intended to export the internal filesystem
> inode change counter (i.e. inode->i_version) then lets call it that:
> stx_modification_count. It's clear and unambiguous as to what it
> represents, especially as this counter is more than just a "data
> modification" counter - inode metadata modifications will also
> cause it to change...

Honestly, I don't think "modification count" is necessarily more clear
than "version".  This value isn't necessarily a counter incremented
to hold the number of times the inode was modified, but is used by NFS
only to determine whether the inode has been modified from one access to
the next.  It may not be incremented sequentially for each inode, but
rather be a global value across all inodes in the filesystem.  It is
much more important to clearly document what this version field means.
Maybe "stx_modification_version", but I'm fine with "version" as well,
since this is what the field is named in the kernel.

>> (13) FS_IOC_GETFLAGS value.  These could be translated to BSD's st_flags.
>>     Note that the Linux IOC flags are a mess and filesystems such as Ext4
>>     define flags that aren't in linux/fs.h, so translation in the kernel
>>     may be a necessity (or, possibly, we provide the filesystem type too).
> 
> And we now also have FS_IOC_FSGETXATTR that extends the flags
> and information userspace can get from filesystems. It makes little
> sense to now add xstat() and not add everything this interface
> also exports...

The number of flags available will always be a moving target.  Better
to get the core functionality landed, and then add in new flags in a
later patch, otherwise this patch will never be landed.

>> The defined bits in request_mask and stx_mask are:
>> 
>> 	STATX_TYPE		Want/got stx_mode & S_IFMT
>> 	STATX_MODE		Want/got stx_mode & ~S_IFMT
>> 	STATX_NLINK		Want/got stx_nlink
>> 	STATX_UID		Want/got stx_uid
>> 	STATX_GID		Want/got stx_gid
>> 	STATX_ATIME		Want/got stx_atime{,_ns}
>> 	STATX_MTIME		Want/got stx_mtime{,_ns}
>> 	STATX_CTIME		Want/got stx_ctime{,_ns}
>> 	STATX_INO		Want/got stx_ino
>> 	STATX_SIZE		Want/got stx_size
>> 	STATX_BLOCKS		Want/got stx_blocks
>> 	STATX_BASIC_STATS	[The stuff in the normal stat struct]
>> 	STATX_BTIME		Want/got stx_btime{,_ns}
>> 	STATX_VERSION		Want/got stx_version
>> 	STATX_ALL		[All currently available stuff]
> 
> What happens when an application uses STATX_ALL from a future kernel
> that defines more flags than are initially supported, and that
> application then is run on a kernel that onyl supports the initial
> fields?

Fields that are unknown by the current kernel/filesystem will not be set,
and this is reflected in the flags that are returned to userspace.

The minimum required flags are the intersection of the requested flags
from userspace and the flags known by the kernel/filesystem.  It is
possible for the filesystem to optionally return extra flags/fields if
they are free to provide, but it should always return the requested
flags *if* it knows what those flags mean.

>> stx_btime is the file creation time; stx_version is the data version
>> number (i_version); stx_mask is a bitmask indicating the data provided;
>> and __spares*[] are where as-yet undefined fields can be placed.
>> 
>> Time fields are split into separate seconds and nanoseconds fields to make
>> packing easier and the granularities can be queried with the filesystem
>> info system call.  Note that times will be negative if before 1970; in
>> such a case, the nanosecond fields will also be negative if not zero.
> 
> So what happens in ten years time when we want to support
> femptosecond resolution in the timestamp interface? We've got to
> change everything to 64 bit? Shouldn't we just make everything
> timestamp related 64 bit?

Is this a serious request?  Are we going to need to multiply everything
by 10e9 to convert to/from nanoseconds for the next 10 years on the off
chance that we have timestamps more accurate than this in the future?
We could just as easily add extra 32-bit fields in the future to hold
the fraction of nanoseconds if we ever actually need this.  Given that
we've totally bailed on keeping accurate atimes below a day, I'm not
really worried about needing this.

>> The bits defined in the stx_attributes field convey information about a
>> file, how it is accessed, where it is and what it does.  The following
>> attributes map to FS_*_FL flags and are the same numerical value:
> 
> Please isolate the new interface flags completely from the FS_*_FL
> values. We should not repeat the mistake of tying values derived
> from filesystem specific on-disk values to a user interface.

Using the existing FS_*_FL flags as initial values is not worse than
starting with any other arbitrary values for the flags.

>> 	STATX_ATTR_COMPRESSED		File is compressed by the fs
>> 	STATX_ATTR_IMMUTABLE		File is marked immutable
>> 	STATX_ATTR_APPEND		File is append-only
>> 	STATX_ATTR_NODUMP		File is not to be dumped
>> 	STATX_ATTR_ENCRYPTED		File requires key to decrypt in fs
>> 
>> The supported flags are listed by:
>> 
>> 	STATX_ATTR_FS_IOC_FLAGS
> 
> Again, we have many more common and extended flags than this.
> NOATIME and SYNC are two that immediately come to mind as generic
> flags that should be in this...

Sure, and they can be added incrementally in a later patch.  I'm not
sure why NOATIME and SYNC are missing, and I'm not against adding them,
but it is equally likely that they were removed in a previous round of
bikeshedding to avoid some real or perceived issue, so that this patch
can finally land rather than being in limbo for another 5 years.

>> [Are any other IOC flags of sufficient general interest to be exposed
>> through this interface?]
>> 
>> New flags include:
>> 
>> 	STATX_ATTR_NONUNIX_OWNERSHIP	File doesn't have Unixy ownership
>> 	STATX_ATTR_HAS_ACL		File has an ACL
> 
> So statx will require us to do ACL lookups? i.e. instead of just
> reading the inode to get the information, we'll also have to do
> extended attribute lookups? That's potentially very expensive if
> the extended attribute is not stored in the inode....

No, there is no requirement to return anything that the caller didn't
ask for.  Only fields that are explicitly requested need to be returned,
and others can optionally be returned if it is easy for the filesystem
to do so.

>> 	STATX_ATTR_KERNEL_API		File is kernel API (eg: procfs/sysfs)
>> 	STATX_ATTR_REMOTE		File is remote and needs network
>> 	STATX_ATTR_FABRICATED		File was made up by fs
> 
> Every file is fabricated by a filesystem :P
> 
> Perhaps you're wanting "virtual file" because it is has no physical
> presence?
> 
>> Fields in struct statx come in a number of classes:
>> 
>> (0) stx_dev_*, stx_blksize.
>> 
>>     These are local system information and are always available.
> 
> What does stx_blksize actually mean? It's completely ambiguous in
> stat() because we don't actually report the physical block size
> here - we report the "minimum unit of efficient IO" that we expect
> applications to use. Please define :P
> 
> 
>> =======
>> TESTING
>> =======
>> 
>> The following test program can be used to test the statx system call:
>> 
>> 	samples/statx/test-statx.c
>> 
>> Just compile and run, passing it paths to the files you want to examine.
>> The file is built automatically if CONFIG_SAMPLES is enabled.
> 
> Can we get xfstests written to exercise and validate all this
> functionality, please? I'd suggest that adding xfs_io support for
> the statx syscall would be far more useful for xfstests than a
> standalone  test program, too. We already have equivalent stat()
> functionality in xfs_io and that's used quite a bit in xfstests....
> 
> Cheers,
> 
> Dave.
> --
> Dave Chinner
> david@fromorbit.com
> --
> To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html


Cheers, Andreas

[toc] | [prev] | [next] | [standalone]

#1525138 — Re: [PATCH 1/4] statx: Add a system call to make enhanced file info available

Andreas Dilger <adilger@dilger.ca> wrote:

> > What happens when an application uses STATX_ALL from a future kernel
> > that defines more flags than are initially supported, and that
> > application then is run on a kernel that onyl supports the initial
> > fields?
> 
> Fields that are unknown by the current kernel/filesystem will not be set,
> and this is reflected in the flags that are returned to userspace.

Yep.

A userspace program can stick 0xffffffff in there if it wants.  No error will
be incurred.  It just won't necessarily get anything back for each of those
bits.  That said, if we, say, want to reserve bit 31 as a struct extension
bit, sticking in 0xffffffff without knowing what this is going to do to you on
a kernel that supports a longer struct might give you a problem.

But, basically, STATX_ALL indicates what flags have fields in the copy of the
struct you got from the header file.

There's an extra scenario: you could compile your userspace program against
the headers for a particular kernel and then run against a later kernel.  In
such a case, you may find bits set that are outside STATX_ALL in stx_mask.
However, you don't have definitions for those bits and can only ignore them.

> > Again, we have many more common and extended flags than this.
> > NOATIME and SYNC are two that immediately come to mind as generic
> > flags that should be in this...
> 
> Sure, and they can be added incrementally in a later patch.  I'm not
> sure why NOATIME and SYNC are missing, and I'm not against adding them,
> but it is equally likely that they were removed in a previous round of
> bikeshedding to avoid some real or perceived issue, so that this patch
> can finally land rather than being in limbo for another 5 years.

Does it make sense to return them through statx?  Note that NOATIME might be
considered superfluous given that STATX_ATIME is cleared in such a case.

> >> New flags include:
> >> 
> >> 	STATX_ATTR_NONUNIX_OWNERSHIP	File doesn't have Unixy ownership
> >> 	STATX_ATTR_HAS_ACL		File has an ACL
> > 
> > So statx will require us to do ACL lookups? i.e. instead of just
> > reading the inode to get the information, we'll also have to do
> > extended attribute lookups? That's potentially very expensive if
> > the extended attribute is not stored in the inode....
> 
> No, there is no requirement to return anything that the caller didn't
> ask for.  Only fields that are explicitly requested need to be returned,
> and others can optionally be returned if it is easy for the filesystem
> to do so.

Actually, Dave might have a point.  We don't necessarily know that the file
has an ACL without doing a getxattr() to probe for it - on the other hand, I
would expect the permissions check to have done precisely that.

David

[toc] | [prev] | [next] | [standalone]

#1525721 — Re: [PATCH 1/4] statx: Add a system call to make enhanced file info available

On Thu, Nov 17, 2016 at 08:28:57PM -0700, Andreas Dilger wrote:
> On Nov 17, 2016, at 4:40 PM, Dave Chinner <david@fromorbit.com> wrote:
> >> 
> >> Time fields are split into separate seconds and nanoseconds fields to make
> >> packing easier and the granularities can be queried with the filesystem
> >> info system call.  Note that times will be negative if before 1970; in
> >> such a case, the nanosecond fields will also be negative if not zero.
> > 
> > So what happens in ten years time when we want to support
> > femptosecond resolution in the timestamp interface? We've got to
> > change everything to 64 bit? Shouldn't we just make everything
> > timestamp related 64 bit?
> 
> Is this a serious request?  Are we going to need to multiply everything
> by 10e9 to convert to/from nanoseconds for the next 10 years on the off
> chance that we have timestamps more accurate than this in the future?

We've been stuck with the stat() interface since, what, the early
1980s? And it will still be used in 10-15 years time. That's a
/50-year lifetime/ for a syscall interface.

So it's not unreasonable to think that statx() might have a similar
lifetime. statx() is clearly intended to support >y2038 dates
cleanly, so clearly we're intending statx() to still be around in
20-25 years. And when we start thinking in those timeframes, an
increase in timestamp resoultion of at least another 10e-3 is
likely....

> > Please isolate the new interface flags completely from the FS_*_FL
> > values. We should not repeat the mistake of tying values derived
> > from filesystem specific on-disk values to a user interface.
> 
> Using the existing FS_*_FL flags as initial values is not worse than
> starting with any other arbitrary values for the flags.

Except it starts with a sparse set of flags for no good reason.
Someone comes along needed to add a new flag and wonders WTF there
are holes in the flags space, and whether it is because flags have
been removed and whether it's unsafe to use the flag space in the
holes...

New user facing APIs should be clean and neat and not carry any
unnecessary historical baggage with them....

> >> 	STATX_ATTR_NONUNIX_OWNERSHIP	File doesn't have Unixy ownership
> >> 	STATX_ATTR_HAS_ACL		File has an ACL
> > 
> > So statx will require us to do ACL lookups? i.e. instead of just
> > reading the inode to get the information, we'll also have to do
> > extended attribute lookups? That's potentially very expensive if
> > the extended attribute is not stored in the inode....
> 
> No, there is no requirement to return anything that the caller didn't
> ask for.

Applications are going to use STATX_ALL because it's simpler than
specifying 10 different flags on every statx() call and then
checking them on return.  i.e. the set/check feature flags API
sounds good until you have to write the boiler plate code it
requires time you want to stat a file...

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

[toc] | [prev] | [next] | [standalone]

Page 1 of 3  [1] 2 3  Next page →

Back to top | Article view | linux.kernel

csiph-web