Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > microsoft.public.excel.programming > #109324
| From | GS <gs@v.invalid> |
|---|---|
| Newsgroups | microsoft.public.excel.programming, microsoft.public.excel |
| Subject | Re: Read (and parse) file on the web |
| Date | 2016-09-17 11:06 -0400 |
| Organization | A noiseless patient Spider |
| Message-ID | <nrjm5a$34a$1@dont-email.me> (permalink) |
| References | (3 earlier) <nrcv9f$ibs$1@dont-email.me> <_GACz.3064$YK2.1842@fx25.iad> <l9LCz.44525$7G1.10736@fx10.iad> <nrh4bn$pbq$1@dont-email.me> <TM5Dz.20215$HR2.7527@fx43.iad> |
Cross-posted to 2 groups.
> Maybe i was not too clear.
> Case one:
> Using a browser, log to https://www.nsncenter.com/ and give it a
> search term: 5960®ULATOR&"ELECTRON TUBE" in the NSN box, and click
> on the WebFLIS Search green button.
> Then use the browser "File" pulldown, select Save Page As and
> modify the extension to .TXT
> The resulting file is a bit different than what one sees in other
> methods.
> Case two:
> Choose a method of getting the search results; a given search term
> will always produce the same results (ie: reproducible), and small
> changes of the search term may give different results - and THOSE
> DIFFERENCES are some of what i am talking about.
> Case three:
> Choose a given search term, and compare results between various
> methods; DIFFERENCES may be huge, also some of what i am talking
> about.
>
> In case three, with your program, whatever is happening gives a
> radically different result. And that result is VERY useful.
>
> For some unknown reason, your program/macro refuses to run, and
> gives the following error message: "Can't find project or library".
>
> Would you be so kind as to modify the search term in your program
> to 5960®ULATOR&"ELECTRON TUBE" and run it? and please send the
> results
Robert,
I'm fully aware of what you're saying. That's just how search engines
work! The most important statement I made to you is...
"You need to decide what URL string you want to run with and set
both the URL cell and gsUrl1 strings to that."
..because that string determines the resulting page source structure
for the search. ANY URL string other than what is hard-coded in gsUrl1
will not likely work with my code because that code is designed to work
with the structure of the resulting page source it uses. That's why
there are 2 methods of parsing the same webpage; AutoParse and Sheet1
are both coded differently to reflect the fact that...
even with the same URL string, both methods have to be structured
to work with the underlying document.
So if you change the search criteria then expect a different page
source for which you'll need to find out how the structure needs to be
parsed in terms of what line numbers the data you want rests on *in
relation to* the string the code searches for to establish 'lStart' and
'lEnd' in their respective "ParseParentPage" and "ParseChildPage"
procedures. For example, when you added "ELECTRON TUBE" to the URL
string I had to revise the hard-code start position of the 1st loop in
both procedures to match the new page source...
Sheet1.ParseParentPage
<snip>
vDoc = Split(sTxt, vbNewLine)
For n = 120 To UBound(vDoc)
If InStr(vDoc(n), sLinkTxt) > 0 Then _
lStart = n: lEnd = n + 60: Exit For
Next 'n
m_OPenClose.ParseParentPage
<snip>
vDoc = Split(StripTags(ReadTextFile(sTxt)), vbNewLine)
For n = 7900 To UBound(vDoc)
If InStr(vDoc(n), sLinkTxt) > 0 Then _
lStart = n: lEnd = n + 160: Exit For
Next 'n
So then, entering 5960®ULATOR&"ELECTRON TUBE" into the search box
returns this URL string...
NSNSearch?q=5960%26REGULATOR%26%22ELECTRON+TUBE%22
appended to...
https://www.nsncenter.com/
..in the address bar. This returns a browser page source containing
8147 lines. Using SaveAs "txt" creates a 411 line file with all the
html tags stripped out of this page source. *This is why I show both
methods* of parsing: *They code differently to match the page source
'structure' they are parsing*!
The browser source finds the 1st item link on line 7920;
(This is what AutoParse works with via URLDownloadToFile)
Sheet1.txtPgSrc finds that same link on line 129;
(This is what clicking btnViewSrc returns as
WebBrowser1.Document.body.innerHTML)
The SaveAs "txt" file finds that same item link on line 296;
If you click btnViewSrc and then in the VBE Immediate Window:
Typing "sheet1.txtPgSrc.Text=striptags(sheet1.txtPgSrc.Text)"
then typing "?sheet1.txtPgSrc.LineCount"
returns 274 lines where the same item link is on line 127.
(This is essentially same as using SaveAs "txt" without the
ignored HTML tags "<https://...>")
Same webpage, different page sources, so different code is needed
to parse each according to how the data sits within the text!
So whatever search criteria you stumble onto as being the best to run
with, revising my code to work with the URL string is simple as 1-2-3:
1 Hard-code the base string; gsUrl2$ = "https://www.nsncenter.com/"
2 Hard-code the search string appended to gsUrl2 as gsUrl1$;
"NSNSearch?q=5960%20regulator%20%22ELECTRON%20TUBE%22&PageNumber=";
Optionally, just go to gsUrl2 and enter your search criteria, then
copy the AddressBar contents and append "&PageNumber=" to it!
3 Revise existing code to work with its page source for both
parent/child pages in m_OpenClose and Sheet1.
The 2 methods my project demos are what I find to be most reliable and
consistent in terms of structure for parsing. Problem today is the NSN
search engine returns different page source than it did when I sent you
the last revision. Perfect time for you to get your hands dirty
revising the code to work with the new page source. Though, it
shouldn't matter what the source is if we have consistent html tags for
the data.
You get your hands dirty; I'll write generic code so it doesn't matter
what the page source structure is...
--
Garry
Free usenet access at http://www.eternal-september.org
Classic VB Users Regroup!
comp.lang.basic.visual.misc
microsoft.public.vb.general.discussion
---
This email has been checked for viruses by Avast antivirus software.
https://www.avast.com/antivirus
Back to microsoft.public.excel.programming | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread
Read (and parse) file on the web Robert Baer <robertbaer@localnet.com> - 2016-08-26 10:57 -0800
Re: Read (and parse) file on the web "Auric__" <not.my.real@email.address> - 2016-08-26 15:21 +0000
Re: Read (and parse) file on the web "Auric__" <not.my.real@email.address> - 2016-08-26 15:22 +0000
Re: Read (and parse) file on the web Robert Baer <robertbaer@localnet.com> - 2016-08-27 13:04 -0800
Re: Read (and parse) file on the web Robert Baer <robertbaer@localnet.com> - 2016-08-27 13:00 -0800
Re: Read (and parse) file on the web Robert Baer <robertbaer@localnet.com> - 2016-08-27 23:14 -0800
Re: Read (and parse) file on the web CORRECTION Robert Baer <robertbaer@localnet.com> - 2016-08-28 09:42 -0800
Re: Read (and parse) file on the web CORRECTION#2 Robert Baer <robertbaer@localnet.com> - 2016-08-28 10:08 -0800
Re: Read (and parse) file on the web CORRECTION#2 "Auric__" <not.my.real@email.address> - 2016-08-29 01:51 +0000
Re: Read (and parse) file on the web CORRECTION#2 Robert Baer <robertbaer@localnet.com> - 2016-08-28 21:32 -0800
Re: Read (and parse) file on the web CORRECTION#2 "Auric__" <not.my.real@email.address> - 2016-08-29 19:41 +0000
Re: Read (and parse) file on the web CORRECTION#2 Robert Baer <robertbaer@localnet.com> - 2016-08-30 00:21 -0800
Re: Read (and parse) file on the web CORRECTION#2 GS <gs@v.invalid> - 2016-08-30 06:06 -0400
Re: Read (and parse) file on the web CORRECTION#2 Robert Baer <robertbaer@localnet.com> - 2016-08-30 14:20 -0800
Re: Read (and parse) file on the web CORRECTION#2 GS <gs@v.invalid> - 2016-08-30 18:43 -0400
Re: Read (and parse) file on the web CORRECTION#2 GS <gs@v.invalid> - 2016-08-31 16:47 -0400
Re: Read (and parse) file on the web CORRECTION#2 Robert Baer <robertbaer@localnet.com> - 2016-09-01 04:37 -0800
Re: Read (and parse) file on the web CORRECTION#2 GS <gs@v.invalid> - 2016-09-01 13:13 -0400
Re: Read (and parse) file on the web CORRECTION#2 Robert Baer <robertbaer@localnet.com> - 2016-09-01 16:32 -0800
Re: Read (and parse) file on the web CORRECTION#2 GS <gs@v.invalid> - 2016-09-01 19:43 -0400
Re: Read (and parse) file on the web CORRECTION#2 Robert Baer <robertbaer@localnet.com> - 2016-09-01 04:23 -0800
Re: Read (and parse) file on the web CORRECTION#2 GS <gs@v.invalid> - 2016-09-05 10:10 -0400
Re: Read (and parse) file on the web CORRECTION#2 GS <gs@v.invalid> - 2016-09-05 10:17 -0400
Re: Read (and parse) file on the web CORRECTION#3 Robert Baer <robertbaer@localnet.com> - 2016-09-05 20:50 -0800
Re: Read (and parse) file on the web CORRECTION#2 Robert Baer <robertbaer@localnet.com> - 2016-09-05 20:32 -0800
Re: Read (and parse) file on the web CORRECTION#2 GS <gs@v.invalid> - 2016-09-06 10:49 -0400
Re: Read (and parse) file on the web CORRECTION#2 Robert Baer <robertbaer@localnet.com> - 2016-09-06 12:05 -0800
Re: Read (and parse) file on the web CORRECTION#2 GS <gs@v.invalid> - 2016-09-06 15:15 -0400
Re: Read (and parse) file on the web CORRECTION#2 Robert Baer <robertbaer@localnet.com> - 2016-09-06 22:32 -0800
Re: Read (and parse) file on the web CORRECTION#2 GS <gs@v.invalid> - 2016-09-07 11:44 -0400
Re: Read (and parse) file on the web CORRECTION#2 Robert Baer <robertbaer@localnet.com> - 2016-09-07 09:33 -0800
Re: Read (and parse) file on the web CORRECTION#2 Robert Baer <robertbaer@localnet.com> - 2016-09-08 20:18 -0800
Re: Read (and parse) file on the web CORRECTION#2 GS <gs@v.invalid> - 2016-09-09 06:07 -0400
Re: Read (and parse) file on the web CORRECTION#2 GS <gs@v.invalid> - 2016-09-09 22:05 -0400
Re: Read (and parse) file on the web CORRECTION#2 Robert Baer <robertbaer@localnet.com> - 2016-09-14 17:16 -0800
Re: Read (and parse) file on the web CORRECTION#2 GS <gs@v.invalid> - 2016-09-14 20:23 -0400
Re: Read (and parse) file on the web CORRECTION#2 "Auric__" <not.my.real@email.address> - 2016-09-07 02:37 +0000
Re: Read (and parse) file on the web CORRECTION#2 Robert Baer <robertbaer@localnet.com> - 2016-09-06 22:43 -0800
Re: Read (and parse) file on the web CORRECTION#2 "Auric__" <not.my.real@email.address> - 2016-09-08 05:49 +0000
Re: Read (and parse) file on the web CORRECTION#2 Robert Baer <robertbaer@localnet.com> - 2016-09-08 01:07 -0800
Re: Read (and parse) file on the web CORRECTION#2 "Auric__" <not.my.real@email.address> - 2016-09-09 06:40 +0000
Re: Read (and parse) file on the web WGET versions Robert Baer <robertbaer@localnet.com> - 2016-09-08 01:19 -0800
Re: Read (and parse) file on the web WGET versions Robert Baer <robertbaer@localnet.com> - 2016-09-08 20:09 -0800
Re: Read (and parse) file on the web CORRECTION#2 Robert Baer <robertbaer@localnet.com> - 2016-09-06 23:49 -0800
Re: Read (and parse) file on the web GS <gs@v.invalid> - 2016-08-28 15:51 -0400
Re: Read (and parse) file on the web Robert Baer <robertbaer@localnet.com> - 2016-08-28 21:37 -0800
Re: Read (and parse) file on the web Robert Baer <robertbaer@localnet.com> - 2016-09-02 19:09 -0800
Re: Read (and parse) file on the web GS <gs@v.invalid> - 2016-08-29 01:56 -0400
Re: Read (and parse) file on the web GS <gs@v.invalid> - 2016-08-29 02:21 -0400
Re: Read (and parse) file on the web Robert Baer <robertbaer@localnet.com> - 2016-08-29 19:55 -0800
Re: Read (and parse) file on the web GS <gs@v.invalid> - 2016-08-29 23:35 -0400
Re: Read (and parse) file on the web Robert Baer <robertbaer@localnet.com> - 2016-08-29 22:25 -0800
Re: Read (and parse) file on the web Robert Baer <robertbaer@localnet.com> - 2016-08-29 19:21 -0800
Re: Read (and parse) file on the web GS <gs@v.invalid> - 2016-08-29 23:21 -0400
Re: Read (and parse) file on the web Robert Baer <robertbaer@localnet.com> - 2016-08-29 22:10 -0800
Re: Read (and parse) file on the web GS <gs@v.invalid> - 2016-08-30 04:04 -0400
Re: Read (and parse) file on the web Robert Baer <robertbaer@localnet.com> - 2016-08-30 13:48 -0800
Re: Read (and parse) file on the web GS <gs@v.invalid> - 2016-08-29 23:37 -0400
Re: Read (and parse) file on the web Robert Baer <robertbaer@localnet.com> - 2016-08-29 22:25 -0800
Re: Read (and parse) file on the web lovexes17816 <lovexes17816@gmail.com> - 2016-08-28 14:00 +0100
Re: Read (and parse) file on the web lovexes17816 <lovexes17816@gmail.com> - 2016-08-29 02:17 +0100
Re: Read (and parse) file on the web Robert Baer <robertbaer@localnet.com> - 2016-09-02 19:11 -0800
Re: Read (and parse) file on the web "Auric__" <not.my.real@email.address> - 2016-09-03 04:12 +0000
Re: Read (and parse) file on the web Tikivn23143 <Tikivn23143@gmail.com> - 2016-08-30 02:28 +0100
Re: Read (and parse) file on the web everonvietnam2016 <everonvietnam2016@gmail.com> - 2016-09-07 15:03 +0100
Re: Read (and parse) file on the web Robert Baer <robertbaer@localnet.com> - 2016-09-07 08:43 -0800
Re: Read (and parse) file on the web "Auric__" <not.my.real@email.address> - 2016-09-08 05:43 +0000
Re: Read (and parse) file on the web Robert Baer <robertbaer@localnet.com> - 2016-09-08 01:09 -0800
Re: Read (and parse) file on the web "Auric__" <not.my.real@email.address> - 2016-09-09 06:49 +0000
Re: Read (and parse) file on the web everonvietnam2016 <everonvietnam2016@gmail.com> - 2016-09-08 11:11 +0100
Re: Read (and parse) file on the web GS <gs@v.invalid> - 2016-09-14 17:14 -0400
Re: Read (and parse) file on the web Robert Baer <robertbaer@localnet.com> - 2016-09-14 17:19 -0800
Re: Read (and parse) file on the web GS <gs@v.invalid> - 2016-09-14 20:26 -0400
Re: Read (and parse) file on the web Robert Baer <robertbaer@localnet.com> - 2016-09-15 09:59 -0800
Re: Read (and parse) file on the web GS <gs@v.invalid> - 2016-09-15 14:33 -0400
Re: Read (and parse) file on the web GS <gs@v.invalid> - 2016-09-14 21:28 -0400
Re: Read (and parse) file on the web Robert Baer <robertbaer@localnet.com> - 2016-09-15 10:05 -0800
Re: Read (and parse) file on the web Robert Baer <robertbaer@localnet.com> - 2016-09-14 18:48 -0800
Re: Read (and parse) file on the web GS <gs@v.invalid> - 2016-09-14 21:59 -0400
Re: Read (and parse) file on the web Robert Baer <robertbaer@localnet.com> - 2016-09-15 10:15 -0800
Re: Read (and parse) file on the web Robert Baer <robertbaer@localnet.com> - 2016-09-15 22:10 -0800
Re: Read (and parse) file on the web GS <gs@v.invalid> - 2016-09-16 11:50 -0400
Re: Read (and parse) file on the web Robert Baer <robertbaer@localnet.com> - 2016-09-16 23:54 -0800
Re: Read (and parse) file on the web GS <gs@v.invalid> - 2016-09-17 11:06 -0400
Re: Read (and parse) file on the web Robert Baer <robertbaer@localnet.com> - 2016-09-17 23:59 -0800
Re: Read (and parse) file on the web GS <gs@v.invalid> - 2016-09-18 03:22 -0400
Re: Read (and parse) file on the web Robert Baer <robertbaer@localnet.com> - 2016-09-20 00:20 -0800
Re: Read (and parse) file on the web GS <gs@v.invalid> - 2016-09-17 11:14 -0400
Re: Read (and parse) file on the web Robert Baer <robertbaer@localnet.com> - 2016-09-18 00:18 -0800
Re: Read (and parse) file on the web GS <gs@v.invalid> - 2016-09-18 03:30 -0400
Re: Read (and parse) file on the web GS <gs@v.invalid> - 2016-09-15 05:02 -0400
Re: Read (and parse) file on the web - revised GS <gs@v.invalid> - 2016-09-17 17:15 -0400
csiph-web