Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.php > #11037

Re: Asynchronous FTP Upload

Show all headers | View raw

On 30-01-11 15:20, Jerry Stuckle wrote:
> On 1/30/2011 8:41 AM, Luuk wrote:
>> On 28-01-11 20:48, Jerry Stuckle wrote:
>>> On 1/28/2011 10:56 AM, duderion wrote:
>>>> On Jan 28, 4:29 pm, Jerry Stuckle<jstuck...@attglobal.net>   wrote:
>>>>> On 1/28/2011 10:12 AM, duderion wrote:
>>>>>
>>>>>> hi guys,
>>>>>
>>>>>> could anyone tell me how i can handle an ftp upload to several
>>>>>> servers
>>>>>> at once?
>>>>>
>>>>>> I found fb_nb_put, but i dont know how to combine 5 connections with
>>>>>> this.
>>>>>
>>>>>> any help would be nice :)
>>>>>
>>>>>> dude
>>>>>
>>>>> You'll need to open 5 different streams and start each transfer.  Keep
>>>>> track of the status of each transfer in an array and loop while any of
>>>>> them need to continue. In the loop, continue those which have not
>>>>> finished.
>>>>>
>>>>> Not sure what this is going to do though for you though, other than
>>>>> take
>>>>> a lot of unnecessary CPU because you're effectively polling
>>>>> constantly.
>>>>>     Why don't you just upload each file individually?
>>>>>
>>>> Hi Jerry
>>>> thanks for the quick and nice reply,
>>>>
>>>> i need to do this, because i have to trasfer videos to around 500
>>>> hosts during one night. i have a 1gb upload line, and thats why i want
>>>> to run those uploads simultaniously....
>>>>
>>> <Top posting fixed>
>>>
>>> That doesn't mean you'll get anywhere near 1gb upload.  Your limit in
>>> this case is likely going to be disk access speed (assuming the other
>>> hosts are replying in a timely manner, of course).  And forcing the disk
>>> to jump around to fetch data from different areas of the disk is likely
>>> to be slower then accessing the data in a contiguous file.
>>>
>>> The point being - even if you open 5 parallel connections, you are not
>>> going to get 5x the speed; in fact, depending on what you're doing, you
>>> may actually slow down the processing.  And error recovery becomes much
>>> harder.
>>
>> You might be true, but most of it depends on the download speed at the
>> receiving site. If its lower then 1/5 of your uploadspeed than you
>> should not worry ;)
>>
> 
> 
> Not really.  You're not going to get 1gb/sec. or even 200mb/sec. from
> the disk drive, especially not continuously.  So even if the download
> speed on the other end is 200mb/sec, that's still not going to be a
> limiting factor.
> 
> And forcing the disk to pull data from several different files on the
> disk will slow disk overall disk access even more, especially if the
> files are contiguous.

But if the files are send to 500 hosts, the file might be in cache,  if
enough memory is available, which should speed up disk-access again.. ;)

> 
>>>
>>> You need to test and find out.  The "sweet spot" may be anywhere from 1
>>> to 500 parallel connections (although I highly doubt the latter :) ).
>>> And it may vary depending on exactly which hosts you're currently
>>> accessing and how quickly they respond.
>>>
>>> P.S. Please don't top post.  Thanks.
>>>
>>
>>


-- 
Luuk

Back to comp.lang.php | Previous | Next — Next in thread | Find similar | Unroll thread

Thread

Re: Asynchronous FTP Upload Luuk <Luuk@invalid.lan> - 2011-01-30 15:48 +0100
  Re: Asynchronous FTP Upload The Natural Philosopher <tnp@invalid.invalid> - 2011-01-31 12:29 +0000
  Re: Asynchronous FTP Upload bill <nobody@spamcop.net> - 2011-01-31 07:38 -0500
  Re: Asynchronous FTP Upload "Peter H. Coffin" <hellsop@ninehells.com> - 2011-01-30 19:07 -0600
  Re: Asynchronous FTP Upload "Peter H. Coffin" <hellsop@ninehells.com> - 2011-01-30 10:02 -0600
    Re: Asynchronous FTP Upload The Natural Philosopher <tnp@invalid.invalid> - 2011-01-30 17:06 +0000
  Re: Asynchronous FTP Upload The Natural Philosopher <tnp@invalid.invalid> - 2011-01-30 17:04 +0000

csiph-web