Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.user > #255644

Re: what method do you prefer for data transfer between nodes?

Show all headers | View raw

[Multipart message — attachments visible in raw view] - view raw

That's smart. Thanks

Sincerely,
Ken Young


On Mon, Mar 6, 2023 at 3:43 AM Linux-Fan <Ma_Sys.ma@web.de> wrote:

> Ken Young writes:
>
> > Hello,[1;5B
> >
> >
> > The methods I know,
> >
> > 1. scp
> > pros: the native tool in the OS
> > cons: you will either input password or put key pairs into servers for
> > authentication.
>
> Works for simple cases.
>
> > 2. rsync
> > pros: it can transfer data by increasement
> > cons: you need to setup rsyncd server and make the correct authorization.
>
> Works for simple and complex cases.
>
> > 3. ftp/ftps
> > pros: easy to use
> > cons: need to setup ftpd server, and the way is not that secure?
>
> Whenever possible, I'd prefer 1 or 2 over this.
>
> > 4. rclone
> > pros:easy to use
> > cons: hard to setup (you may need a cloud storage for middleware).
>
> I only use rclone when I want to target a cloud storage.
> A „cloud storage for middleware” does not seem sensible to me when I can
> copy using methods 1 and 2 without using such a middleware.
>
> > For me I most often use scp + rsync. and what's your choice?
>
> These are my standard choices, too. In automated scenarios I often prefer
> rsync over scp due to more flexibility in configuration.
>
> My additional tools for special purposes:
>
> 5. lsyncd
> If you need to keep directories in sync continuously, there is a tool
> called
> `lsyncd` that automates repeated invocation of `rsync` in a smart way.
>
> 6. tar + netcat (or tar + ssh in very rare cases)
> Using tar sacrifices all the flexibility of rsync but may attain a
> significantly higher performance and does not need a lot of flags to do
> the
> right thing by default (i.e. preserve everything when acting as root). I
> prefer this variant when migrating to a new disk or PC because it seems
> to
> be the most efficient variant in a "local trusted network and no speedup
> from incremental copying" scenario.
>
> I documented my approach to this here:
> https://masysma.net/37/data_transfer_netcat_tar.xhtml
>
> HTH and YMMV
> Linux-Fan
>
> öö
>

Back to linux.debian.user | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

what method do you prefer for data transfer between nodes? Ken Young <ken@highwinds.cloud> - 2023-03-05 09:30 +0100
  Re: what method do you prefer for data transfer between nodes? <tomas@tuxteam.de> - 2023-03-05 10:20 +0100
    Re: what method do you prefer for data transfer between nodes? Michel Verdier <mv524@free.fr> - 2023-03-05 11:40 +0100
      Re: what method do you prefer for data transfer between nodes? <tomas@tuxteam.de> - 2023-03-05 12:00 +0100
        Re: what method do you prefer for data transfer between nodes? Ken Young <ken@highwinds.cloud> - 2023-03-05 12:30 +0100
          Re: what method do you prefer for data transfer between nodes? tomas@tuxteam.de - 2023-03-05 14:10 +0100
            Re: what method do you prefer for data transfer between nodes? Ken Young <ken@highwinds.cloud> - 2023-03-05 14:50 +0100
              Re: what method do you prefer for data transfer between nodes? Rodrigo Cunha <rodrigo.root.rj@gmail.com> - 2023-03-05 20:00 +0100
            Re: what method do you prefer for data transfer between nodes? Celejar <celejar@gmail.com> - 2023-03-07 02:30 +0100
              Re: what method do you prefer for data transfer between nodes? Ken Young <ken@highwinds.cloud> - 2023-03-07 02:30 +0100
  Re: what method do you prefer for data transfer between nodes? Michel Verdier <mv524@free.fr> - 2023-03-05 11:40 +0100
  Re: what method do you prefer for data transfer between          nodes? Linux-Fan <Ma_Sys.ma@web.de> - 2023-03-05 20:50 +0100
    Re: what method do you prefer for data transfer between nodes? Ken Young <ken@highwinds.cloud> - 2023-03-06 00:50 +0100
  Re: what method do you prefer for data transfer between nodes? Charles Curley <charlescurley@charlescurley.com> - 2023-03-06 01:20 +0100
  Re: what method do you prefer for data transfer between nodes? David Christensen <dpchrist@holgerdanske.com> - 2023-03-06 12:10 +0100
  Re: what method do you prefer for data transfer between nodes? Max Nikulin <manikulin@gmail.com> - 2023-03-06 12:50 +0100
    Re: what method do you prefer for data transfer between nodes? <tomas@tuxteam.de> - 2023-03-06 13:20 +0100
  Re: what method do you prefer for data transfer between nodes? Vincent Lefevre <vincent@vinc17.net> - 2023-03-07 03:40 +0100

csiph-web