Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.php > #11100 > unrolled thread

Re: Asynchronous FTP Upload

Back to article view | Back to comp.lang.php

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Re: Asynchronous FTP Upload Jerry Stuckle <jstucklex@attglobal.net> - 2011-01-28 14:48 -0500
    Re: Asynchronous FTP Upload Luuk <Luuk@invalid.lan> - 2011-01-30 14:41 +0100
    Re: Asynchronous FTP Upload duderion <adrian.gier@googlemail.com> - 2011-01-30 03:33 -0800

#11100 — Re: Asynchronous FTP Upload

On 1/28/2011 10:56 AM, duderion wrote:
> On Jan 28, 4:29 pm, Jerry Stuckle<jstuck...@attglobal.net>  wrote:
>> On 1/28/2011 10:12 AM, duderion wrote:
>>
>>> hi guys,
>>
>>> could anyone tell me how i can handle an ftp upload to several servers
>>> at once?
>>
>>> I found fb_nb_put, but i dont know how to combine 5 connections with
>>> this.
>>
>>> any help would be nice :)
>>
>>> dude
>>
>> You'll need to open 5 different streams and start each transfer.  Keep
>> track of the status of each transfer in an array and loop while any of
>> them need to continue. In the loop, continue those which have not finished.
>>
>> Not sure what this is going to do though for you though, other than take
>> a lot of unnecessary CPU because you're effectively polling constantly.
>>    Why don't you just upload each file individually?
>>
 > Hi Jerry
 > thanks for the quick and nice reply,
 >
 > i need to do this, because i have to trasfer videos to around 500
 > hosts during one night. i have a 1gb upload line, and thats why i want
 > to run those uploads simultaniously....
 >
<Top posting fixed>

That doesn't mean you'll get anywhere near 1gb upload.  Your limit in 
this case is likely going to be disk access speed (assuming the other 
hosts are replying in a timely manner, of course).  And forcing the disk 
to jump around to fetch data from different areas of the disk is likely 
to be slower then accessing the data in a contiguous file.

The point being - even if you open 5 parallel connections, you are not 
going to get 5x the speed; in fact, depending on what you're doing, you 
may actually slow down the processing.  And error recovery becomes much 
harder.

You need to test and find out.  The "sweet spot" may be anywhere from 1 
to 500 parallel connections (although I highly doubt the latter :) ). 
And it may vary depending on exactly which hosts you're currently 
accessing and how quickly they respond.

P.S. Please don't top post.  Thanks.

-- 
==================
Remove the "x" from my email address
Jerry Stuckle
JDS Computer Training Corp.
jstucklex@attglobal.net
==================

[toc] | [next] | [standalone]

#11107

On 28-01-11 20:48, Jerry Stuckle wrote:
> On 1/28/2011 10:56 AM, duderion wrote:
>> On Jan 28, 4:29 pm, Jerry Stuckle<jstuck...@attglobal.net>  wrote:
>>> On 1/28/2011 10:12 AM, duderion wrote:
>>>
>>>> hi guys,
>>>
>>>> could anyone tell me how i can handle an ftp upload to several servers
>>>> at once?
>>>
>>>> I found fb_nb_put, but i dont know how to combine 5 connections with
>>>> this.
>>>
>>>> any help would be nice :)
>>>
>>>> dude
>>>
>>> You'll need to open 5 different streams and start each transfer.  Keep
>>> track of the status of each transfer in an array and loop while any of
>>> them need to continue. In the loop, continue those which have not
>>> finished.
>>>
>>> Not sure what this is going to do though for you though, other than take
>>> a lot of unnecessary CPU because you're effectively polling constantly.
>>>    Why don't you just upload each file individually?
>>>
>> Hi Jerry
>> thanks for the quick and nice reply,
>>
>> i need to do this, because i have to trasfer videos to around 500
>> hosts during one night. i have a 1gb upload line, and thats why i want
>> to run those uploads simultaniously....
>>
> <Top posting fixed>
> 
> That doesn't mean you'll get anywhere near 1gb upload.  Your limit in
> this case is likely going to be disk access speed (assuming the other
> hosts are replying in a timely manner, of course).  And forcing the disk
> to jump around to fetch data from different areas of the disk is likely
> to be slower then accessing the data in a contiguous file.
> 
> The point being - even if you open 5 parallel connections, you are not
> going to get 5x the speed; in fact, depending on what you're doing, you
> may actually slow down the processing.  And error recovery becomes much
> harder.

You might be true, but most of it depends on the download speed at the
receiving site. If its lower then 1/5 of your uploadspeed than you
should not worry ;)

> 
> You need to test and find out.  The "sweet spot" may be anywhere from 1
> to 500 parallel connections (although I highly doubt the latter :) ).
> And it may vary depending on exactly which hosts you're currently
> accessing and how quickly they respond.
> 
> P.S. Please don't top post.  Thanks.
> 


-- 
Luuk

[toc] | [prev] | [next] | [standalone]

#11111

On Jan 28, 8:48 pm, Jerry Stuckle <jstuck...@attglobal.net> wrote:
> On 1/28/2011 10:56 AM, duderion wrote:
>
>
>
>
>
>
>
> > On Jan 28, 4:29 pm, Jerry Stuckle<jstuck...@attglobal.net>  wrote:
> >> On 1/28/2011 10:12 AM, duderion wrote:
>
> >>> hi guys,
>
> >>> could anyone tell me how i can handle an ftp upload to several servers
> >>> at once?
>
> >>> I found fb_nb_put, but i dont know how to combine 5 connections with
> >>> this.
>
> >>> any help would be nice :)
>
> >>> dude
>
> >> You'll need to open 5 different streams and start each transfer.  Keep
> >> track of the status of each transfer in an array and loop while any of
> >> them need to continue. In the loop, continue those which have not finished.
>
> >> Not sure what this is going to do though for you though, other than take
> >> a lot of unnecessary CPU because you're effectively polling constantly.
> >>    Why don't you just upload each file individually?
>
>  > Hi Jerry
>  > thanks for the quick and nice reply,
>  >
>  > i need to do this, because i have to trasfer videos to around 500
>  > hosts during one night. i have a 1gb upload line, and thats why i want
>  > to run those uploads simultaniously....
>  >
> <Top posting fixed>
>
> That doesn't mean you'll get anywhere near 1gb upload.  Your limit in
> this case is likely going to be disk access speed (assuming the other
> hosts are replying in a timely manner, of course).  And forcing the disk
> to jump around to fetch data from different areas of the disk is likely
> to be slower then accessing the data in a contiguous file.
>
> The point being - even if you open 5 parallel connections, you are not
> going to get 5x the speed; in fact, depending on what you're doing, you
> may actually slow down the processing.  And error recovery becomes much
> harder.
>
> You need to test and find out.  The "sweet spot" may be anywhere from 1
> to 500 parallel connections (although I highly doubt the latter :) ).
> And it may vary depending on exactly which hosts you're currently
> accessing and how quickly they respond.
>
> P.S. Please don't top post.  Thanks.
>
> --
> ==================
> Remove the "x" from my email address
> Jerry Stuckle
> JDS Computer Training Corp.
> jstuck...@attglobal.net
> ==================

Thanks Jerry,

ill pass your useful information to my boss, who had this idea. i
guess its better to upload them in a row...

THANKS ALOT
and ill never top-post again :D

dude

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.php

csiph-web