Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.soft-sys.math.mathematica > #16416 > unrolled thread

Downloading Multiple Files Via Import and FTP

Back to article view | Back to comp.soft-sys.math.mathematica

Contents

  Downloading Multiple Files Via Import and FTP Gregory Lypny <gregory.lypny@videotron.ca> - 2014-01-13 06:57 +0000
    Re: Downloading Multiple Files Via Import and FTP Roland Franzius <roland.franzius@uos.de> - 2014-01-15 08:39 +0000

#16416 — Downloading Multiple Files Via Import and FTP

Hello everyone,

Mathematica's Import function works well for downloading single files from a remote computer using ftp, but it gets stuck when downloading anything more than about a 20 files in succession, even when those files are just few bytes.

If I define the path to a folder on my remote mac as

	ftpPath="ftp://user:password@domainName//filePath/=93

I can easily download a single file, myFile, using

	Import[ftpPath<>myFile, "CSV=94]

But if I try to download, say, 60 small files whose names are contained in myFileNames by mapping across the names (or by using Table)

	Import[ftpPath<> #, "CSV=94] &/@ myFileNames

I get two errors: (1) URLFetch::invhttp : Failure when receiving data from the peer; and (2) FetchURL::conopen.

I thought that maybe Mathematica was sending in the requests too quickly, so I tried slowing things down by inserting a one-second pause between each call to Import but that didn't help. Maybe I have to close a socket between each import but I don't know how to do that.

Anyone know of a fix?

Much obliged,

Gregory

[toc] | [next] | [standalone]

#16420

Am 13.01.2014 07:57, schrieb Gregory Lypny:
> Hello everyone,
>
> Mathematica's Import function works well for downloading single files from a remote computer using ftp, but it gets stuck when downloading anything more than about a 20 files in succession, even when those files are just few bytes.
>
> If I define the path to a folder on my remote mac as
>
> 	ftpPath="ftp://user:password@domainName//filePath/=93
>
> I can easily download a single file, myFile, using
>
> 	Import[ftpPath<>myFile, "CSV=94]
>
> But if I try to download, say, 60 small files whose names are contained in myFileNames by mapping across the names (or by using Table)
>
> 	Import[ftpPath<> #, "CSV=94] &/@ myFileNames
>
> I get two errors: (1) URLFetch::invhttp : Failure when receiving data from the peer; and (2) FetchURL::conopen.
>
> I thought that maybe Mathematica was sending in the requests too quickly, so I tried slowing things down by inserting a one-second pause between each call to Import but that didn't help. Maybe I have to close a socket between each import but I don't know how to do that.
>
> Anyone know of a fix?

Uploading and downloading many files or a very large file using  a 
ftp-browser like fileZilla, in the protocoll window you observe that the 
server rather often breaks the connection.

Its the browsers duty to reconnect and and resume the broken transfer 
session.

The rather short connection timeout seems to be a security feature of 
ftp servers against automated data suction. The ftp password security is 
an illusion, it applies to the naive users only.

-- 

Roland Franzius

[toc] | [prev] | [standalone]

Back to top | Article view | comp.soft-sys.math.mathematica

csiph-web