Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]


Groups > comp.soft-sys.math.mathematica > #9499

Import HTTP data in asynchronous/parallel way

From Rodrigo Murta <murta@rodrigomurta.com>
Newsgroups comp.soft-sys.math.mathematica
Subject Import HTTP data in asynchronous/parallel way
Date 2012-04-07 09:58 +0000
Organization Steven M. Christensen and Associates, Inc and MathTensor, Inc.
Message-ID <jlp34d$14s$1@smc.vnet.net> (permalink)

Show all headers | View raw


Hi All

    I'm testing some web scraping using Mathematica and would like to
know how to work with Import in an asynchronous/parallel way.
    Now I'm using Parallelize to do that. It works but it doesn't look
like the best way to do that, due to the kernel number limitation.

   My code is like that:
   result = Parallelize[Import/@urlList]

  How can I do it in another asynchronous way? Something like
backgroud process using & in bash?
  I can do it inside Mathematica? I know that I could speed up a lot
my scrap with that.

tks in advance
Murta

Back to comp.soft-sys.math.mathematica | Previous | Next — Next in thread | Find similar | Unroll thread


Thread

Import HTTP data in asynchronous/parallel way Rodrigo Murta <murta@rodrigomurta.com> - 2012-04-07 09:58 +0000
  Re: Import HTTP data in asynchronous/parallel way David Bailey <dave@removedbailey.co.uk> - 2012-04-10 06:31 +0000
    Re: Import HTTP data in asynchronous/parallel way Murta <rodrigomurtax@gmail.com> - 2012-04-11 22:19 +0000
      Re: Import HTTP data in asynchronous/parallel way David Bailey <dave@removedbailey.co.uk> - 2012-04-13 09:01 +0000

csiph-web