Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > comp.soft-sys.math.mathematica > #10034
| From | Armand Tamzarian <mike.honeychurch@gmail.com> |
|---|---|
| Newsgroups | comp.soft-sys.math.mathematica |
| Subject | Re: Problem Importing web site in Mathematica: How to by pass pages asking for login credentials |
| Date | 2012-05-11 04:16 +0000 |
| Organization | Steven M. Christensen and Associates, Inc and MathTensor, Inc. |
| Message-ID | <joi3ri$n5i$1@smc.vnet.net> (permalink) |
On May 9, 5:53 pm, Mark Coleman <markspcole...@gmail.com> wrote: > Hi, > > I'm using Mathematica v8 for some text mining/classification analysis of web > sites. As part of this I first Import[] the hyperlinks from the web > site 's home page into a list, and then systematically traverse this > list and Import each URL. In some cases, I hit a page or set of pages > that requires a user to enter login credentials. At this point my code > pops up the site's login screen and waits for manual input before > proceeding. This obviously makes importing a large set of URLs > infeasible. > > I'm wondering if it's possible to identify these pages in advance, so > I can filter them out of my list of URLs. allowing me to automatically > Import the remaining pages? > > Thanks, > > Mark I use wget in combination with Mathematica to work around logins and cookies. Mike
Back to comp.soft-sys.math.mathematica | Previous | Next | Find similar | Unroll thread
Re: Problem Importing web site in Mathematica: How to by pass pages asking for login credentials Armand Tamzarian <mike.honeychurch@gmail.com> - 2012-05-11 04:16 +0000
csiph-web