Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.os.linux.development.system > #797

Re: File transfer for fastness

Show all headers | View raw

On Sunday, 14 June 2020 20:09:24 UTC+5:30, Jorgen Grahn  wrote:
> On Sun, 2020-06-14, t.sathyanarayanan@gmail.com wrote:
> > Hi,
> >
> > Im trying to find best solution for parallel file transfer, which is
> > event driven after the file written to a directory with extension
> > like .done, means the file is ready for file transfer.
> >
> > 1. Monitoring the directory for .done is cpu cycle time waster
> > 2. running a cron on particular interval  to transfer files will
> >    not help fasteness to the data.
> 
> I cannot understand what you want to do, or how you want to do it;
> you would have to explain better.
> 
> > what are other options existing for  file transfer.
> 
> Personally I always use rsync over ssh.  Unless I download files over
> http, then I use firefox or curl.  Bittorrent is another option.
> 
> /Jorgen
> 
> -- 
>   // Jorgen Grahn <grahn@  Oo  o.   .     .
> \X/     snipabacken.se>   O  o   .




Thanks for your help.


we have a distrubuted systems which collects peta bytes of data and deliver the  data to netstorage, the problem we have is transmission of files at faster rate,   the one problem we sees as soon as the file comes in, we are not delivering the data, we have to wait for cron to run and pick those files and to deliver netstorage, the problem in cron is we schedule a time to run but practically this will lead to big data backlog, my question here is there any event based for monitoring file system to initiate the job as soons as files that comes into the directory, like (epoll mechanism)

Back to comp.os.linux.development.system | Previous | Next — Previous in thread | Next in thread | Find similar

Thread

File transfer for fastness t.sathyanarayanan@gmail.com - 2020-06-14 06:43 -0700
  Re: File transfer for fastness Jorgen Grahn <grahn+nntp@snipabacken.se> - 2020-06-14 14:39 +0000
    Re: File transfer for fastness Sathya Narayanan Thangavelu <t.sathyanarayanan@gmail.com> - 2020-06-15 02:14 -0700
      Re: File transfer for fastness Jorgen Grahn <grahn+nntp@snipabacken.se> - 2020-06-15 11:42 +0000
  Re: File transfer for fastness Grant Edwards <invalid@invalid.invalid> - 2020-06-15 01:31 +0000
    Re: File transfer for fastness Jorgen Grahn <grahn+nntp@snipabacken.se> - 2020-06-15 08:41 +0000

csiph-web