Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.user > #262529 > unrolled thread

A file synchronization tool that respects hardlinks

Back to article view | Back to linux.debian.user

Contents

  A file synchronization tool that respects hardlinks Itay <debian@itayf.fastmail.fm> - 2023-10-22 16:50 +0200
    Re: A file synchronization tool that respects hardlinks Michael Kjörling <2695bd53d63c@ewoof.net> - 2023-10-22 19:40 +0200
      Re: A file synchronization tool that respects hardlinks Itay <debian@itayf.fastmail.fm> - 2023-10-24 16:20 +0200
    Re: A file synchronization tool that respects hardlinks Mike Castle <dalgoda+debian@gmail.com> - 2023-10-22 19:40 +0200
      Re: A file synchronization tool that respects hardlinks Itay <debian@itayf.fastmail.fm> - 2023-10-24 16:20 +0200
        Re: A file synchronization tool that respects hardlinks Charles Curley <charlescurley@charlescurley.com> - 2023-10-24 17:20 +0200
          Re: A file synchronization tool that respects hardlinks Itay <debian@itayf.fastmail.fm> - 2023-10-25 09:00 +0200
            Re: A file synchronization tool that respects hardlinks Charles Curley <charlescurley@charlescurley.com> - 2023-10-25 15:30 +0200
              Re: A file synchronization tool that respects hardlinks David Wright <deblis@lionunicorn.co.uk> - 2023-10-26 01:30 +0200
    Re: A file synchronization tool that respects hardlinks Michael Kjörling <2695bd53d63c@ewoof.net> - 2023-10-22 19:40 +0200
      Re: A file synchronization tool that respects hardlinks Itay <debian@itayf.fastmail.fm> - 2023-10-24 16:20 +0200
    Re: A file synchronization tool that respects hardlinks Charles Curley <charlescurley@charlescurley.com> - 2023-10-22 20:20 +0200
      Re: A file synchronization tool that respects hardlinks Itay <debian@itayf.fastmail.fm> - 2023-10-24 16:30 +0200
        Re: A file synchronization tool that respects hardlinks Itay <debian@itayf.fastmail.fm> - 2023-10-25 09:10 +0200

#262529 — A file synchronization tool that respects hardlinks

[Multipart message — attachments visible in raw view] — view raw

Hi

I have two machines for which I need to sync my home directory.
The sync must be bidirectional and *respect hardlinks* because many subdirs are kept under the control of darcs[1] which by default uses hardlinks.  The sync tool I used in the past, unison[2], does not respect hardlinks[3].

According to wikipedia[4] the following tools are bidirectional:
     FreeFileSync / NextCloud / Owncloud / SyncThing
Please -- can someone quickly tell me if they respect hardlinks?
Or recommend another tool(s) that respect hardlinks?

Thanks in advance.
Itay

[1] darcs.net/
[2] www.cis.upenn.edu/~bcpierce/unison/
[3] See manual, page 18, top.
     raw.githubusercontent.com/bcpierce00/unison/documentation/unison-manual.pdf
[4] en.wikipedia.org/wiki/Comparison_of_file_synchronization_software

[toc] | [next] | [standalone]

#262545

On 22 Oct 2023 10:32 -0700, from dalgoda+debian@gmail.com (Mike Castle):
> rsync supports hardlinks.
> 
> --hard-links, -H         preserve hard links

It does, but on the other hand it's very hard to wrangle rsync into
doing safe bidirectional syncing. At least I didn't find a way to do
that, which led me to eventually settle on unison for where I need
bidirectional sync.

-- 
Michael Kjörling                     🔗 https://michael.kjorling.se
“Remember when, on the Internet, nobody cared that you were a dog?”

[toc] | [prev] | [next] | [standalone]

#262663

[Multipart message — attachments visible in raw view] — view raw

On Sun, 22 Oct 2023, at 20:36, Michael Kjörling wrote:
> On 22 Oct 2023 10:32 -0700, from dalgoda+debian@gmail.com <mailto:dalgoda%2Bdebian@gmail.com> (Mike Castle):
>> rsync supports hardlinks.
>> 
>> --hard-links, -H         preserve hard links
>
> It does, but on the other hand it's very hard to wrangle rsync into
> doing safe bidirectional syncing. At least I didn't find a way to do
> that, which led me to eventually settle on unison for where I need
> bidirectional sync.
>

Same here.  But this was before I started using darcs...
Thanks for the answer.

> -- 
> Michael Kjörling                     🔗 https://michael.kjorling.se
> “Remember when, on the Internet, nobody cared that you were a dog?”

[toc] | [prev] | [next] | [standalone]

#262548

rsync supports hardlinks.

--hard-links, -H         preserve hard links

Though, in general, the purpose of something like darcs is to
*provide* the syncing.

mrc

[toc] | [prev] | [next] | [standalone]

#262664

[Multipart message — attachments visible in raw view] — view raw

On Sun, 22 Oct 2023, at 20:32, Mike Castle wrote:
> rsync supports hardlinks.
>
> --hard-links, -H         preserve hard links
>

Yes.  This is what I use for unidirectional syncing.

> Though, in general, the purpose of something like darcs is to
> *provide* the syncing.
>

True.  But my home dir includes many subdirs that are not under darcs control.

Thanks for the answer.

> mrc

[toc] | [prev] | [next] | [standalone]

#262670

On Tue, 24 Oct 2023 17:14:21 +0300
Itay <debian@itayf.fastmail.fm> wrote:

> > Though, in general, the purpose of something like darcs is to
> > *provide* the syncing.
> >  
> 
> True.  But my home dir includes many subdirs that are not under darcs
> control.

Then perhaps you should consider what I do.

I put stuff under version control under a directory dedicated to
version controlled material, ~/versioned. Those I have to synchronize
to such other computers as I want synchronized manually with the
relevant VCS.

I then put other stuff I want synchronized to other computers under a
special directory, ~/projects. ~/projects is subject to syncthing, so
other computers are updated automatically. (It also makes ~/ a lot
cleaner.) No hard links so far.

Stuff outside of ~/projects which I want synchronized I symlink into a
tree under ~/projects/home. So ~/.emacs is a symlink to
~/projects/home/emacs. So now every time I change .emacs on one
computer it gets updated on the others where I have syncthing.

Other stuff I want synchronized, but rarely, such as music or
photographs, I synchronize manually with rsync.

-- 
Does anybody read signatures any more?

https://charlescurley.com
https://charlescurley.com/blog/

[toc] | [prev] | [next] | [standalone]

#262706

On Tue, 24 Oct 2023, at 18:09, Charles Curley wrote:
> On Tue, 24 Oct 2023 17:14:21 +0300
> Itay <debian@itayf.fastmail.fm> wrote:
>
>> > Though, in general, the purpose of something like darcs is to
>> > *provide* the syncing.
>> >  
>> 
>> True.  But my home dir includes many subdirs that are not under darcs
>> control.
>
> Then perhaps you should consider what I do.
>
> I put stuff under version control under a directory dedicated to
> version controlled material, ~/versioned. Those I have to synchronize
> to such other computers as I want synchronized manually with the
> relevant VCS.
>
> I then put other stuff I want synchronized to other computers under a
> special directory, ~/projects. ~/projects is subject to syncthing, so
> other computers are updated automatically. (It also makes ~/ a lot
> cleaner.) No hard links so far.
>
> Stuff outside of ~/projects which I want synchronized I symlink into a
> tree under ~/projects/home. So ~/.emacs is a symlink to
> ~/projects/home/emacs. So now every time I change .emacs on one
> computer it gets updated on the others where I have syncthing.
>
> Other stuff I want synchronized, but rarely, such as music or
> photographs, I synchronize manually with rsync.
>

Maybe this would be the best way to go.
Perhaps I will grab the chance to separate private stuff from work stuff :-)

One point to remember is to avoid using hardlinks anywhere outside ~/versioned.

Thanks!

> -- 
> Does anybody read signatures any more?
>
> https://charlescurley.com
> https://charlescurley.com/blog/

[toc] | [prev] | [next] | [standalone]

#262720

On Wed, 25 Oct 2023 09:57:19 +0300
Itay <debian@itayf.fastmail.fm> wrote:

> Perhaps I will grab the chance to separate private stuff from work
> stuff :-)

Indeed! I don't know where you are located, but I will tell you that in
parts of the US commingling the two can become a legal nightmare. I
would consider having a separate computer for each.

-- 
Does anybody read signatures any more?

https://charlescurley.com
https://charlescurley.com/blog/

[toc] | [prev] | [next] | [standalone]

#262729

On Wed 25 Oct 2023 at 07:28:44 (-0600), Charles Curley wrote:
> On Wed, 25 Oct 2023 09:57:19 +0300
> Itay <debian@itayf.fastmail.fm> wrote:
> 
> > Perhaps I will grab the chance to separate private stuff from work
> > stuff :-)
> 
> Indeed! I don't know where you are located, but I will tell you that in
> parts of the US commingling the two can become a legal nightmare. I
> would consider having a separate computer for each.

The same for phones.

Cheers,
David.

[toc] | [prev] | [next] | [standalone]

#262549

On 22 Oct 2023 17:40 +0300, from debian@itayf.fastmail.fm (Itay):
> The sync must be bidirectional and *respect hardlinks* because many
> subdirs are kept under the control of darcs[1] which by default uses
> hardlinks. The sync tool I used in the past, unison[2], does not
> respect hardlinks[3].

What do you mean by "respect hardlinks"? To preserve the multiple
names as pointing to the same file system object (inode in classic
*nix parlace)?

-- 
Michael Kjörling                     🔗 https://michael.kjorling.se
“Remember when, on the Internet, nobody cared that you were a dog?”

[toc] | [prev] | [next] | [standalone]

#262662

[Multipart message — attachments visible in raw view] — view raw

On Sun, 22 Oct 2023, at 20:38, Michael Kjörling wrote:
> On 22 Oct 2023 17:40 +0300, from debian@itayf.fastmail.fm (Itay):
>> The sync must be bidirectional and *respect hardlinks* because many
>> subdirs are kept under the control of darcs[1] which by default uses
>> hardlinks. The sync tool I used in the past, unison[2], does not
>> respect hardlinks[3].
>
> What do you mean by "respect hardlinks"? To preserve the multiple
> names as pointing to the same file system object (inode in classic
> *nix parlace)?
>

Exactly.  As 'rsync -H' does.

> -- 
> Michael Kjörling                     🔗 https://michael.kjorling.se
> “Remember when, on the Internet, nobody cared that you were a dog?”

[toc] | [prev] | [next] | [standalone]

#262559

On Sun, 22 Oct 2023 17:40:43 +0300
Itay <debian@itayf.fastmail.fm> wrote:

> According to wikipedia[4] the following tools are bidirectional:
>      FreeFileSync / NextCloud / Owncloud / SyncThing
> Please -- can someone quickly tell me if they respect hardlinks?
> Or recommend another tool(s) that respect hardlinks?

I can tell you that nextcloud and syncthing do not appear to respect
hard links. They will copy two hard linked files (the same inode), but
on the receiving computer the files will not be hard linked.

I suspect the same for owncloud, as nextcloud is a fork of owncloud.

-- 
Does anybody read signatures any more?

https://charlescurley.com
https://charlescurley.com/blog/

[toc] | [prev] | [next] | [standalone]

#262665

[Multipart message — attachments visible in raw view] — view raw

On Sun, 22 Oct 2023, at 21:10, Charles Curley wrote:
> On Sun, 22 Oct 2023 17:40:43 +0300
> Itay <debian@itayf.fastmail.fm> wrote:
>
>> According to wikipedia[4] the following tools are bidirectional:
>>      FreeFileSync / NextCloud / Owncloud / SyncThing
>> Please -- can someone quickly tell me if they respect hardlinks?
>> Or recommend another tool(s) that respect hardlinks?
>
> I can tell you that nextcloud and syncthing do not appear to respect
> hard links. They will copy two hard linked files (the same inode), but
> on the receiving computer the files will not be hard linked.
>
> I suspect the same for owncloud, as nextcloud is a fork of owncloud.
>

Thank you.  Your answer narrows down the options to FreeFileSync.
I'll search the documentation.

> -- 
> Does anybody read signatures any more?
>
> https://charlescurley.com
> https://charlescurley.com/blog/

[toc] | [prev] | [next] | [standalone]

#262707

[Multipart message — attachments visible in raw view] — view raw

On Tue, 24 Oct 2023, at 17:19, Itay wrote:
> 
> 
> On Sun, 22 Oct 2023, at 21:10, Charles Curley wrote:
> > On Sun, 22 Oct 2023 17:40:43 +0300
> > Itay <debian@itayf.fastmail.fm> wrote:
> >
> >> According to wikipedia[4] the following tools are bidirectional:
> >>      FreeFileSync / NextCloud / Owncloud / SyncThing
> >> Please -- can someone quickly tell me if they respect hardlinks?
> >> Or recommend another tool(s) that respect hardlinks?
> >
> > I can tell you that nextcloud and syncthing do not appear to respect
> > hard links. They will copy two hard linked files (the same inode), but
> > on the receiving computer the files will not be hard linked.
> >
> > I suspect the same for owncloud, as nextcloud is a fork of owncloud.
> >
> 
> Thank you.  Your answer narrows down the options to FreeFileSync.
> I'll search the documentation.

According to FreeFileSync forum the tool does *not* respect hardlinks[1,2].
However, in the same forum they recommend the commercial tool SynCovery[2,3].
There are downloads packaged for debian, and a free one month trial.
Haven't tried it, yet.

My thanks to all the responders.

[1] freefilesync.org/forum/viewtopic.php?t=6087&hilit=hardlinks#p20016
[2] freefilesync.org/forum/viewtopic.php?t=6633&hilit=hardlinks
[3] https://www.syncovery.com/

> 
> > -- 
> > Does anybody read signatures any more?
> >
> > https://charlescurley.com
> > https://charlescurley.com/blog/

[toc] | [prev] | [standalone]

Back to top | Article view | linux.debian.user

csiph-web