Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1356035 > unrolled thread

Variant symlink filesystem

Back to article view | Back to linux.kernel

Contents

  Variant symlink filesystem Cole <cole@opteqint.net> - 2016-03-11 17:30 +0100
    Re: Variant symlink filesystem "Austin S. Hemmelgarn" <ahferroin7@gmail.com> - 2016-03-11 17:50 +0100
    Re: Variant symlink filesystem Richard Weinberger <richard.weinberger@gmail.com> - 2016-03-11 21:10 +0100
      Re: Variant symlink filesystem Cole <cole@opteqint.net> - 2016-03-11 21:20 +0100
        Re: Variant symlink filesystem Richard Weinberger <richard@nod.at> - 2016-03-11 21:20 +0100
          Re: Variant symlink filesystem Richard Weinberger <richard@nod.at> - 2016-03-11 21:30 +0100
            Re: Variant symlink filesystem "Austin S. Hemmelgarn" <ahferroin7@gmail.com> - 2016-03-11 21:40 +0100
              Re: Variant symlink filesystem Cole <cole@opteqint.net> - 2016-03-11 22:00 +0100
                Re: Variant symlink filesystem Al Viro <viro@ZenIV.linux.org.uk> - 2016-03-11 23:00 +0100
                  Re: Variant symlink filesystem Cole <cole@opteqint.net> - 2016-03-11 23:10 +0100
                    Re: Variant symlink filesystem Al Viro <viro@ZenIV.linux.org.uk> - 2016-03-11 23:30 +0100
                      Re: Variant symlink filesystem Cole <cole@opteqint.net> - 2016-03-11 23:40 +0100
                        Re: Variant symlink filesystem David Lang <david@lang.hm> - 2016-03-11 23:50 +0100
                          Re: Variant symlink filesystem Cole <cole@opteqint.net> - 2016-03-12 00:00 +0100
                    Re: Variant symlink filesystem David Lang <david@lang.hm> - 2016-03-11 23:50 +0100
            Re: Variant symlink filesystem Cole <cole@opteqint.net> - 2016-03-11 21:40 +0100
              Re: Variant symlink filesystem Richard Weinberger <richard@nod.at> - 2016-03-11 21:40 +0100
              Re: Variant symlink filesystem David Lang <david@lang.hm> - 2016-03-11 23:40 +0100
          Re: Variant symlink filesystem Cole <cole@opteqint.net> - 2016-03-11 21:30 +0100

#1356035 — Variant symlink filesystem

Hi,

I have written a Variant Symlink Filesystem for linux, currently
implemented as a kernel module:
https://github.com/onslauth/varsymfs
The code was written for the 3.x kernel.

I would like to try to get this included into the linux kernel, and am
willing to hand over all copyright and change the license as needed.
As such, I would like to know what I can do to try to make this
happen.

If the code quality or standards are not up to par with those of the
linux kernel, or code needs to change due to newer changes introduced
into the kernel, please let me know and I will endeavour to make the
necessary changes.

Please can you also cc me in any replies, as I am not currently
subscribed to the list.

Thanks
/Cole

[toc] | [next] | [standalone]

#1356048

On 2016-03-11 11:20, Cole wrote:
> Hi,
>
> I have written a Variant Symlink Filesystem for linux, currently
> implemented as a kernel module:
> https://github.com/onslauth/varsymfs
> The code was written for the 3.x kernel.
>
> I would like to try to get this included into the linux kernel, and am
> willing to hand over all copyright and change the license as needed.
> As such, I would like to know what I can do to try to make this
> happen.
The first two steps for this to happen are making sure it works on newer 
kernels (I haven't looked in-depth at your code, but there are not 
likely to be many changes needed to get it working on 4.4.x assuming you 
have it working already on late 3.x cycle kernels), and then get it 
converted to an in-tree module (this also shouldn't be too difficult 
either, once you figure out which tree to base it on and where to put it 
in the source tree).

Once you get to that point, read through Documentation/SubmittingPatches 
and follow the steps there.  Aside from what's there, you'll probably 
want to include a couple of well described test cases.

[toc] | [prev] | [next] | [standalone]

#1356163

On Fri, Mar 11, 2016 at 5:20 PM, Cole <cole@opteqint.net> wrote:
> Hi,
>
> I have written a Variant Symlink Filesystem for linux, currently
> implemented as a kernel module:
> https://github.com/onslauth/varsymfs
> The code was written for the 3.x kernel.
>
> I would like to try to get this included into the linux kernel, and am
> willing to hand over all copyright and change the license as needed.
> As such, I would like to know what I can do to try to make this
> happen.
>
> If the code quality or standards are not up to par with those of the
> linux kernel, or code needs to change due to newer changes introduced
> into the kernel, please let me know and I will endeavour to make the
> necessary changes.
>
> Please can you also cc me in any replies, as I am not currently
> subscribed to the list.

Why does this need to be a kernel filesystem and not a filesystem in
userspace (FUSE)?
Especially as you are dealing with environment variables which are
owned and controlled
by userspace.

-- 
Thanks,
//richard

[toc] | [prev] | [next] | [standalone]

#1356169

On 11 March 2016 at 22:04, Richard Weinberger
<richard.weinberger@gmail.com> wrote:
> On Fri, Mar 11, 2016 at 5:20 PM, Cole <cole@opteqint.net> wrote:
>> Hi,
>>
>> I have written a Variant Symlink Filesystem for linux, currently
>> implemented as a kernel module:
>> https://github.com/onslauth/varsymfs
>> The code was written for the 3.x kernel.
>>
>> I would like to try to get this included into the linux kernel, and am
>> willing to hand over all copyright and change the license as needed.
>> As such, I would like to know what I can do to try to make this
>> happen.
>>
>> If the code quality or standards are not up to par with those of the
>> linux kernel, or code needs to change due to newer changes introduced
>> into the kernel, please let me know and I will endeavour to make the
>> necessary changes.
>>
>> Please can you also cc me in any replies, as I am not currently
>> subscribed to the list.
>
> Why does this need to be a kernel filesystem and not a filesystem in
> userspace (FUSE)?
> Especially as you are dealing with environment variables which are
> owned and controlled
> by userspace.

The original implementation was in fuse, to prove the concept. However,
because we are compiling, as well as running programs and reading/writing
files inside of this path, the performance loss is too great. Therefore we
moved to this solution.

Regards
/Cole

[toc] | [prev] | [next] | [standalone]

#1356170

Am 11.03.2016 um 21:15 schrieb Cole:
> On 11 March 2016 at 22:04, Richard Weinberger
> <richard.weinberger@gmail.com> wrote:
>> On Fri, Mar 11, 2016 at 5:20 PM, Cole <cole@opteqint.net> wrote:
>>> Hi,
>>>
>>> I have written a Variant Symlink Filesystem for linux, currently
>>> implemented as a kernel module:
>>> https://github.com/onslauth/varsymfs
>>> The code was written for the 3.x kernel.
>>>
>>> I would like to try to get this included into the linux kernel, and am
>>> willing to hand over all copyright and change the license as needed.
>>> As such, I would like to know what I can do to try to make this
>>> happen.
>>>
>>> If the code quality or standards are not up to par with those of the
>>> linux kernel, or code needs to change due to newer changes introduced
>>> into the kernel, please let me know and I will endeavour to make the
>>> necessary changes.
>>>
>>> Please can you also cc me in any replies, as I am not currently
>>> subscribed to the list.
>>
>> Why does this need to be a kernel filesystem and not a filesystem in
>> userspace (FUSE)?
>> Especially as you are dealing with environment variables which are
>> owned and controlled
>> by userspace.
> 
> The original implementation was in fuse, to prove the concept. However,
> because we are compiling, as well as running programs and reading/writing
> files inside of this path, the performance loss is too great. Therefore we
> moved to this solution.

Before giving up and going to kernelspace you could try improving the root cause of the performance loss. :)
Maybe the kernel interface for finding the env variables can be speeded up.

Thanks,
//richard

[toc] | [prev] | [next] | [standalone]

#1356172

Am 11.03.2016 um 21:22 schrieb Cole:
> If I remember correctly, when we were testing the fuse version, we hard coded
> the path to see if that solved the problem, and the difference between
> the env lookup
> code and the hard coded path was almost the same, but substantially slower than
> the native file system.

And where exactly as the performance problem?

Anyway, if you submit your filesystem also provide a decent use case for it. :-)

Thanks,
//richard

[toc] | [prev] | [next] | [standalone]

#1356177

On 2016-03-11 15:24, Richard Weinberger wrote:
> Am 11.03.2016 um 21:22 schrieb Cole:
>> If I remember correctly, when we were testing the fuse version, we hard coded
>> the path to see if that solved the problem, and the difference between
>> the env lookup
>> code and the hard coded path was almost the same, but substantially slower than
>> the native file system.
>
> And where exactly as the performance problem?
>
> Anyway, if you submit your filesystem also provide a decent use case for it. :-)
>
I don't know that this qualifies as a use case, but I've seen a number 
of capability based systems that have a similar concept they usually 
refer to as 'context dependent symbolic links'.  In such cases, the 
resolution is usually based on what capabilities you posses, and is more 
of a mapping than a value expansion most of the time, but such usage 
could be emulated (albeit much less securely) with this.  If this could 
be extended to expand other values (for example, process bit width, or 
SELinux context, or even what namespace the process is in), it could 
provide the same functionality almost as securely.

[toc] | [prev] | [next] | [standalone]

#1356183

On 11 March 2016 at 22:38, Austin S. Hemmelgarn <ahferroin7@gmail.com> wrote:
> On 2016-03-11 15:24, Richard Weinberger wrote:
>>
>> Am 11.03.2016 um 21:22 schrieb Cole:
>>>
>>> If I remember correctly, when we were testing the fuse version, we hard
>>> coded
>>> the path to see if that solved the problem, and the difference between
>>> the env lookup
>>> code and the hard coded path was almost the same, but substantially
>>> slower than
>>> the native file system.
>>
>>
>> And where exactly as the performance problem?
>>
>> Anyway, if you submit your filesystem also provide a decent use case for
>> it. :-)
>>
> I don't know that this qualifies as a use case, but I've seen a number of
> capability based systems that have a similar concept they usually refer to
> as 'context dependent symbolic links'.  In such cases, the resolution is
> usually based on what capabilities you posses, and is more of a mapping than
> a value expansion most of the time, but such usage could be emulated (albeit
> much less securely) with this.  If this could be extended to expand other
> values (for example, process bit width, or SELinux context, or even what
> namespace the process is in), it could provide the same functionality almost
> as securely.


The implementation doesn't necessarily have to continue to work with
env variables. On FreeBSD, the variant symlinks function by using
variables stored in kernel memory, and have a hierarchical lookup,
starting with user defined values and terminating with global entries.
I am not aware of such functionality existing on linux, but if someone
could point me at something similar to that, I would much prefer to
use that, as there are issues with variables that are exported or
modified during process execution.

Regards
/Cole

[toc] | [prev] | [next] | [standalone]

#1356215

On Fri, Mar 11, 2016 at 10:52:52PM +0200, Cole wrote:

> The implementation doesn't necessarily have to continue to work with
> env variables. On FreeBSD, the variant symlinks function by using
> variables stored in kernel memory, and have a hierarchical lookup,
> starting with user defined values and terminating with global entries.
> I am not aware of such functionality existing on linux, but if someone
> could point me at something similar to that, I would much prefer to
> use that, as there are issues with variables that are exported or
> modified during process execution.

Put your processes into a separate namespace and use mount --bind in it...

[toc] | [prev] | [next] | [standalone]

#1356220

On 11 March 2016 at 23:51, Al Viro <viro@zeniv.linux.org.uk> wrote:
> On Fri, Mar 11, 2016 at 10:52:52PM +0200, Cole wrote:
>
>> The implementation doesn't necessarily have to continue to work with
>> env variables. On FreeBSD, the variant symlinks function by using
>> variables stored in kernel memory, and have a hierarchical lookup,
>> starting with user defined values and terminating with global entries.
>> I am not aware of such functionality existing on linux, but if someone
>> could point me at something similar to that, I would much prefer to
>> use that, as there are issues with variables that are exported or
>> modified during process execution.
>
> Put your processes into a separate namespace and use mount --bind in it...

This was one of the first solutions we looked at, and using various
namespaces. However we would like to be able to have multiple terminal
sessions open, and be able to have each session using a different
mount point, or be able to use the other terminals mount point, i.e.
switching the mount point to that of the other terminals. We would
also like the shell to be able to make use of these, and use shell
commands such as 'ls'.

When we originally looked at namespaces and containers, we could not
find a solution to achieve the above. Is this possible using
namespaces?

Regards
/Cole

[toc] | [prev] | [next] | [standalone]

#1356239

On Sat, Mar 12, 2016 at 12:03:11AM +0200, Cole wrote:

> This was one of the first solutions we looked at, and using various
> namespaces. However we would like to be able to have multiple terminal
> sessions open, and be able to have each session using a different
> mount point, or be able to use the other terminals mount point, i.e.
> switching the mount point to that of the other terminals. We would
> also like the shell to be able to make use of these, and use shell
> commands such as 'ls'.
> 
> When we originally looked at namespaces and containers, we could not
> find a solution to achieve the above. Is this possible using
> namespaces?

I'd try to look at setns(2) if you want processes joinging existing namespaces.
I'm afraid that I'll need to get some sleep before I'll be up to asking
the right questions for figuring out what requirements do you have and
what's the best way to do it - after a while coffee stops being efficient
and I'm already several hours past that ;-/

[toc] | [prev] | [next] | [standalone]

#1356244

On 12 March 2016 at 00:24, Al Viro <viro@zeniv.linux.org.uk> wrote:
> On Sat, Mar 12, 2016 at 12:03:11AM +0200, Cole wrote:
>
>> This was one of the first solutions we looked at, and using various
>> namespaces. However we would like to be able to have multiple terminal
>> sessions open, and be able to have each session using a different
>> mount point, or be able to use the other terminals mount point, i.e.
>> switching the mount point to that of the other terminals. We would
>> also like the shell to be able to make use of these, and use shell
>> commands such as 'ls'.
>>
>> When we originally looked at namespaces and containers, we could not
>> find a solution to achieve the above. Is this possible using
>> namespaces?
>
> I'd try to look at setns(2) if you want processes joinging existing namespaces.
> I'm afraid that I'll need to get some sleep before I'll be up to asking
> the right questions for figuring out what requirements do you have and
> what's the best way to do it - after a while coffee stops being efficient
> and I'm already several hours past that ;-/


Sure, not a problem, when you have time to reply I will gladly welcome
any feed back.

As for the usage, I'll explain it a bit so that you have something to
work off of when you get a chance to read it.

The problem we encountered with namespaces when we looked at it more
than a year ago was 'how do you get the shell' to join them, or into
one. And also how do you move the shell in one terminal session into a
namespace that another shell is currently in. We wanted a solution
that doesn't require modifying existing programs to make them
namespace aware. However, as I said, this was more than a year ago
that we looked at it, and we could easily have misunderstood
something, or not understood the full functionality available. If you
say this is possible, without modifying programs such as bash, could
you please point me in the direction of the documentation describing
this, and I will try to educate myself.

Thanks
/Cole

[toc] | [prev] | [next] | [standalone]

#1356246

On Sat, 12 Mar 2016, Cole wrote:

> On 12 March 2016 at 00:24, Al Viro <viro@zeniv.linux.org.uk> wrote:
>> On Sat, Mar 12, 2016 at 12:03:11AM +0200, Cole wrote:
>>
>>> This was one of the first solutions we looked at, and using various
>>> namespaces. However we would like to be able to have multiple terminal
>>> sessions open, and be able to have each session using a different
>>> mount point, or be able to use the other terminals mount point, i.e.
>>> switching the mount point to that of the other terminals. We would
>>> also like the shell to be able to make use of these, and use shell
>>> commands such as 'ls'.
>>>
>>> When we originally looked at namespaces and containers, we could not
>>> find a solution to achieve the above. Is this possible using
>>> namespaces?
>>
>> I'd try to look at setns(2) if you want processes joinging existing namespaces.
>> I'm afraid that I'll need to get some sleep before I'll be up to asking
>> the right questions for figuring out what requirements do you have and
>> what's the best way to do it - after a while coffee stops being efficient
>> and I'm already several hours past that ;-/
>
>
> Sure, not a problem, when you have time to reply I will gladly welcome
> any feed back.
>
> As for the usage, I'll explain it a bit so that you have something to
> work off of when you get a chance to read it.
>
> The problem we encountered with namespaces when we looked at it more
> than a year ago was 'how do you get the shell' to join them, or into
> one. And also how do you move the shell in one terminal session into a
> namespace that another shell is currently in. We wanted a solution
> that doesn't require modifying existing programs to make them
> namespace aware. However, as I said, this was more than a year ago
> that we looked at it, and we could easily have misunderstood
> something, or not understood the full functionality available. If you
> say this is possible, without modifying programs such as bash, could
> you please point me in the direction of the documentation describing
> this, and I will try to educate myself.

looking at the setns() function, it seems like you could have a suid helper 
program that you run in one session that changes the namespace and then invokes 
a bash shell in that namespace that you then run unmodified stuff in.

it seems like there should be a way for a root program to change the namespace 
of another, but I'm not finding it at the moment.

There is the nsenter program that will run a program inside an existing 
namespace. It looks like you need something similar that implements some 
permission checking (only let you go into namespaces of other programs for the 
same user or similar), but you should be able to make proof-of-concept scripts 
with nsenter.

David Lang

[toc] | [prev] | [next] | [standalone]

#1356252

On 12 March 2016 at 00:48, David Lang <david@lang.hm> wrote:
> On Sat, 12 Mar 2016, Cole wrote:
>
>> On 12 March 2016 at 00:24, Al Viro <viro@zeniv.linux.org.uk> wrote:
>>>
>>> On Sat, Mar 12, 2016 at 12:03:11AM +0200, Cole wrote:
>>>
>>>> This was one of the first solutions we looked at, and using various
>>>> namespaces. However we would like to be able to have multiple terminal
>>>> sessions open, and be able to have each session using a different
>>>> mount point, or be able to use the other terminals mount point, i.e.
>>>> switching the mount point to that of the other terminals. We would
>>>> also like the shell to be able to make use of these, and use shell
>>>> commands such as 'ls'.
>>>>
>>>> When we originally looked at namespaces and containers, we could not
>>>> find a solution to achieve the above. Is this possible using
>>>> namespaces?
>>>
>>>
>>> I'd try to look at setns(2) if you want processes joinging existing
>>> namespaces.
>>> I'm afraid that I'll need to get some sleep before I'll be up to asking
>>> the right questions for figuring out what requirements do you have and
>>> what's the best way to do it - after a while coffee stops being efficient
>>> and I'm already several hours past that ;-/
>>
>>
>>
>> Sure, not a problem, when you have time to reply I will gladly welcome
>> any feed back.
>>
>> As for the usage, I'll explain it a bit so that you have something to
>> work off of when you get a chance to read it.
>>
>> The problem we encountered with namespaces when we looked at it more
>> than a year ago was 'how do you get the shell' to join them, or into
>> one. And also how do you move the shell in one terminal session into a
>> namespace that another shell is currently in. We wanted a solution
>> that doesn't require modifying existing programs to make them
>> namespace aware. However, as I said, this was more than a year ago
>> that we looked at it, and we could easily have misunderstood
>> something, or not understood the full functionality available. If you
>> say this is possible, without modifying programs such as bash, could
>> you please point me in the direction of the documentation describing
>> this, and I will try to educate myself.
>
>
> looking at the setns() function, it seems like you could have a suid helper
> program that you run in one session that changes the namespace and then
> invokes a bash shell in that namespace that you then run unmodified stuff
> in.
>
> it seems like there should be a way for a root program to change the
> namespace of another, but I'm not finding it at the moment.
>
> There is the nsenter program that will run a program inside an existing
> namespace. It looks like you need something similar that implements some
> permission checking (only let you go into namespaces of other programs for
> the same user or similar), but you should be able to make proof-of-concept
> scripts with nsenter.

Thank you very much for the explanation and links.

If I understand you correctly, you have described something similar to
how docker functions, with a single daemon that runs, and the various
support binaries make calls to it to perform the necessary functions.
The only concern as you mentioned is that there does not seem to be
any functionality available to move a process from one namespace to
another. However, the only process in my design that would need to do
that would be the shell, so nsenter might be sufficient.

I will look into your suggestions, as it then means I do not need to
have anything inlined into the kernel.

Thank you for all the help and advice.

Regards
/Cole

[toc] | [prev] | [next] | [standalone]

#1356245

On Sat, 12 Mar 2016, Cole wrote:

> On 11 March 2016 at 23:51, Al Viro <viro@zeniv.linux.org.uk> wrote:
>> On Fri, Mar 11, 2016 at 10:52:52PM +0200, Cole wrote:
>>
>>> The implementation doesn't necessarily have to continue to work with
>>> env variables. On FreeBSD, the variant symlinks function by using
>>> variables stored in kernel memory, and have a hierarchical lookup,
>>> starting with user defined values and terminating with global entries.
>>> I am not aware of such functionality existing on linux, but if someone
>>> could point me at something similar to that, I would much prefer to
>>> use that, as there are issues with variables that are exported or
>>> modified during process execution.
>>
>> Put your processes into a separate namespace and use mount --bind in it...
>
> This was one of the first solutions we looked at, and using various
> namespaces. However we would like to be able to have multiple terminal
> sessions open, and be able to have each session using a different
> mount point, or be able to use the other terminals mount point, i.e.
> switching the mount point to that of the other terminals. We would
> also like the shell to be able to make use of these, and use shell
> commands such as 'ls'.

you should be able to have multiple sessions using the same namespace. There is 
the lwn.net series on namespaces at https://lwn.net/Articles/531114/

from what I'm looking at, this should be possible with the right mount options. 
It's not as trivial as setting an environment variable, but if it's all 
scripted, that shouldn't matter to the user.

you would need to use the setns() call to have one session join an existing 
namespace rather than creating a new one.

now, changing namespaces does require CAP_SYS_ADMIN, so if you are not running 
things as root, you may need to create a small daemon to run as root that 
reassigns your different sessions from one ns to another.

David Lang

> When we originally looked at namespaces and containers, we could not
> find a solution to achieve the above. Is this possible using
> namespaces?
>
> Regards
> /Cole
>

[toc] | [prev] | [next] | [standalone]

#1356178

On 11 March 2016 at 22:24, Richard Weinberger <richard@nod.at> wrote:
> Am 11.03.2016 um 21:22 schrieb Cole:
>> If I remember correctly, when we were testing the fuse version, we hard coded
>> the path to see if that solved the problem, and the difference between
>> the env lookup
>> code and the hard coded path was almost the same, but substantially slower than
>> the native file system.
>
> And where exactly as the performance problem?
>
> Anyway, if you submit your filesystem also provide a decent use case for it. :-)

Thank you, I will do so. One example as a use case could be to allow
for multiple
package repositories to exist on a single computer, all in different
locations, but with
a fixed path so as not to break the package manager, the correct
repository then is
selected based on ENV variable. That way each user could have their own packages
installed that would be separate from the system packages, and no
collisions would
occur.

If you don't mind me asking, are fuse based file systems meant to be as fast or
almost as fast as native or in-kernel filesystems? My last experience
with them was
that they were substantially slower. I also believe with our version
of the fuse filesytem
that we wrote, the env variable was only being looked up during mount, and then
remained static from there onwards. Do you believe that we should have
been able to
achieve performance almost as good as the in-kernel filesystems?

Regards
/Cole

[toc] | [prev] | [next] | [standalone]

#1356179

Hi!

Am 11.03.2016 um 21:32 schrieb Cole:
> On 11 March 2016 at 22:24, Richard Weinberger <richard@nod.at> wrote:
>> Am 11.03.2016 um 21:22 schrieb Cole:
>>> If I remember correctly, when we were testing the fuse version, we hard coded
>>> the path to see if that solved the problem, and the difference between
>>> the env lookup
>>> code and the hard coded path was almost the same, but substantially slower than
>>> the native file system.
>>
>> And where exactly as the performance problem?
>>
>> Anyway, if you submit your filesystem also provide a decent use case for it. :-)
> 
> Thank you, I will do so. One example as a use case could be to allow
> for multiple
> package repositories to exist on a single computer, all in different
> locations, but with
> a fixed path so as not to break the package manager, the correct
> repository then is
> selected based on ENV variable. That way each user could have their own packages
> installed that would be separate from the system packages, and no
> collisions would
> occur.
> 
> If you don't mind me asking, are fuse based file systems meant to be as fast or
> almost as fast as native or in-kernel filesystems? My last experience
> with them was
> that they were substantially slower. I also believe with our version
> of the fuse filesytem
> that we wrote, the env variable was only being looked up during mount, and then
> remained static from there onwards. Do you believe that we should have
> been able to
> achieve performance almost as good as the in-kernel filesystems?

FUSE filesystems are slower.
But IMHO your use case cries for FUSE and it does not seem to be very performance critical
as all you do is managing a symlink farm and redirecting.
IOW all file io can go through the native filesystem.

Thanks,
//richard

[toc] | [prev] | [next] | [standalone]

#1356242

On Fri, 11 Mar 2016, Cole wrote:

> On 11 March 2016 at 22:24, Richard Weinberger <richard@nod.at> wrote:
>> Am 11.03.2016 um 21:22 schrieb Cole:
>>> If I remember correctly, when we were testing the fuse version, we hard coded
>>> the path to see if that solved the problem, and the difference between
>>> the env lookup
>>> code and the hard coded path was almost the same, but substantially slower than
>>> the native file system.
>>
>> And where exactly as the performance problem?
>>
>> Anyway, if you submit your filesystem also provide a decent use case for it. :-)
>
> Thank you, I will do so. One example as a use case could be to allow
> for multiple
> package repositories to exist on a single computer, all in different
> locations, but with
> a fixed path so as not to break the package manager, the correct
> repository then is
> selected based on ENV variable. That way each user could have their own packages
> installed that would be separate from the system packages, and no
> collisions would
> occur.

why would this not be a case to use filesystem namespaces and bind mounts?

David Lang

[toc] | [prev] | [next] | [standalone]

#1356173

On 11 March 2016 at 22:18, Richard Weinberger <richard@nod.at> wrote:
> Am 11.03.2016 um 21:15 schrieb Cole:
>> On 11 March 2016 at 22:04, Richard Weinberger
>> <richard.weinberger@gmail.com> wrote:
>>> On Fri, Mar 11, 2016 at 5:20 PM, Cole <cole@opteqint.net> wrote:
>>>> Hi,
>>>>
>>>> I have written a Variant Symlink Filesystem for linux, currently
>>>> implemented as a kernel module:
>>>> https://github.com/onslauth/varsymfs
>>>> The code was written for the 3.x kernel.
>>>>
>>>> I would like to try to get this included into the linux kernel, and am
>>>> willing to hand over all copyright and change the license as needed.
>>>> As such, I would like to know what I can do to try to make this
>>>> happen.
>>>>
>>>> If the code quality or standards are not up to par with those of the
>>>> linux kernel, or code needs to change due to newer changes introduced
>>>> into the kernel, please let me know and I will endeavour to make the
>>>> necessary changes.
>>>>
>>>> Please can you also cc me in any replies, as I am not currently
>>>> subscribed to the list.
>>>
>>> Why does this need to be a kernel filesystem and not a filesystem in
>>> userspace (FUSE)?
>>> Especially as you are dealing with environment variables which are
>>> owned and controlled
>>> by userspace.
>>
>> The original implementation was in fuse, to prove the concept. However,
>> because we are compiling, as well as running programs and reading/writing
>> files inside of this path, the performance loss is too great. Therefore we
>> moved to this solution.
>
> Before giving up and going to kernelspace you could try improving the root cause of the performance loss. :)
> Maybe the kernel interface for finding the env variables can be speeded up.

If I remember correctly, when we were testing the fuse version, we hard coded
the path to see if that solved the problem, and the difference between
the env lookup
code and the hard coded path was almost the same, but substantially slower than
the native file system.

Regards
/Cole

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web