Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > microsoft.public.excel.programming > #109281
| From | Robert Baer <robertbaer@localnet.com> |
|---|---|
| Newsgroups | microsoft.public.excel.programming, microsoft.public.excel |
| Subject | Re: Read (and parse) file on the web CORRECTION#2 |
| References | (5 earlier) <XnsA672BFE492FC2auricauricauricauric@213.239.209.88> <0OPwz.40087$%M4.39822@fx36.iad> <XnsA673811EF8D4Fauricauricauricauric@213.239.209.88> <kuaxz.15921$Xc3.6064@fx03.iad> <nqjuc4$7j8$1@dont-email.me> |
| Message-ID | <JNqzz.334$OT5.113@fx19.iad> (permalink) |
| Date | 2016-09-05 20:32 -0800 |
Cross-posted to 2 groups.
GS wrote:
>> I wish to read and parse every page of
>> "https://www.nsncenter.com/NSNSearch?q=5960%20regulator&PageNumber="
>> where the page number goes from 5 to 999.
>> On each page, find "<a href="/NSN/5960 [it is longer, but that is the
>> start].
>> Given the full number (eg: <a href="/NSN/5960-00-831-8683"), open a
>> new related page "https://www.nsncenter.com/NSN/5960-00-831-8683" and
>> find the line ending "(MCRL)".
>> Read abut 4 lines to <a href="/PartNumber/ which is <a
>> href="/PartNumber/GV4S1400"> in this case.
>> save/write that line plus the next three; close this secondary online
>> URL and step to next "<a href="/NSN/5960 to process the same way.
>> Continue to end of the page, close that URL and open the next page.
>
> Robert,
> Here's what I have after parsing 'parent' pages for a list of its links:
>
> N/A
> 5960-00-503-9529
> 5960-00-504-8401
> 5960-01-035-3901
> 5960-01-029-2766
> 5960-00-617-4105
> 5960-00-729-5602
> 5960-00-826-1280
> 5960-00-754-5316
> 5960-00-962-5391
> 5960-00-944-4671
>
> This is pg1 where the 1st link doesn't contain "5960" and so will be
> ignored.
>
> Each link's text is appended to this URL to bring up its 'child' pg:
>
> https://www.nsncenter.com/NSN/
>
> Each child page is parsed for the following 4 lines:
>
> <TD style="VERTICAL-ALIGN: middle" align=center><A
> href="/PartNumber/GV3S2800">GV3S2800</A></TD>
> <TD style="HEIGHT: 60px; WIDTH: 125px; VERTICAL-ALIGN: middle" noWrap
> align=center> <A href="/CAGE/63060">63060</A> </TD>
> <TD style="VERTICAL-ALIGN: middle" align=center> <A
> href="/CAGE/63060"><IMG class=img-thumbnail
> src="https://placehold.it/90x45?text=No%0DImage%0DYet" width=90
> height=45></A> </TD>
> <TD style="VERTICAL-ALIGN: middle" text-align="center"><A title="CAGE
> 63060" href="/CAGE/63060">HEICO OHMITE LLC</A></TD>
>
> I'm stripping html syntax to get this data:
>
> Line1: PartNumber/GV3S2800
> Line2: CAGE/63060
> Line3: https://placehold.it/90x45?text=No%0DImage%0DYet
> Line4: HEICO OHMITE LLC
>
>
> The output file has these filenames in the 1st line:
>
> NSN Item#,Description,Part#,MCRL,CAGE,Source
>
> I left the 3rd line URL out since, outside its host webpage, it'll be
> useless to you. I need to know from you if the 3rd line URL is needed!
>
> Otherwise, the output file will have 1 line per item so it can be used
> as the db file "NSN_5960_ElectronTube.dat". I invite your suggestion for
> filename...
>
> I could extend the collected data to include...
>
> Reference Number/DRN_3570
> Entity Code/DRN_9250
> Category Code/DRN_2910
> Variation Code/DRN_4780
>
> ..where the fieldnames would then be:
>
> Item#,Part#,MCRL,CAGE,Source,Ref,Entity,Category,Variation
>
> The 1st record will be:
>
> 5960-00-503-9529,GV3S2800,3302008,63060,HEICO OHMITE
> LLC,DRN_3570,DRN_9250,DRN_2910,DRN_4780
>
> Output file size for 1 parent pg is 1Kb; for 10 parent pgs is 10Kb. You
> could have 1000 parent pgs of data stored in a 1Mb file.
>
> Your feedback is appreciated...
>
WOW!
Absolutely PERFECT!
You are correct, #1) do not need that line 3, and #2) do not need the
extended info.
File name(s) for PageNumber=1 I would use 5960_001.TXT,..to
PageNumber=999 I would use 5960_999.TXT and that would preserve order.
*OR*
Reading & parsing from PageNumber=1 to PageNumber=999,one could
append to same file (name NSN_5960.TXT); might as well - makes it easier
to pour into a single Excel file.
Either way is fine.
I have found a way to get rid of items that are not strictly electron
tubes and/or not regulators; that way you do not have to parse out these
"unfit" items from first page description. Use:
"https://www.nsncenter.com/NSNSearch?q=5960%20regulator%20and%20%22ELECTRON%20TUBE%22&PageNumber=1"
Naturally, PageNumber still goes from 1 to 999.
Note the implied "(", ")" and " "; human-readable "5960 regulator and
(ELECTRON TUBE)".
As far as i can tell, using that shows no undesirable parts.
Thanks!
PS: i found WGET to be non-useful (a) it truncates the filename (b) it
buggers it to partial gibberish.
Back to microsoft.public.excel.programming | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread
Read (and parse) file on the web Robert Baer <robertbaer@localnet.com> - 2016-08-26 10:57 -0800
Re: Read (and parse) file on the web "Auric__" <not.my.real@email.address> - 2016-08-26 15:21 +0000
Re: Read (and parse) file on the web "Auric__" <not.my.real@email.address> - 2016-08-26 15:22 +0000
Re: Read (and parse) file on the web Robert Baer <robertbaer@localnet.com> - 2016-08-27 13:04 -0800
Re: Read (and parse) file on the web Robert Baer <robertbaer@localnet.com> - 2016-08-27 13:00 -0800
Re: Read (and parse) file on the web Robert Baer <robertbaer@localnet.com> - 2016-08-27 23:14 -0800
Re: Read (and parse) file on the web CORRECTION Robert Baer <robertbaer@localnet.com> - 2016-08-28 09:42 -0800
Re: Read (and parse) file on the web CORRECTION#2 Robert Baer <robertbaer@localnet.com> - 2016-08-28 10:08 -0800
Re: Read (and parse) file on the web CORRECTION#2 "Auric__" <not.my.real@email.address> - 2016-08-29 01:51 +0000
Re: Read (and parse) file on the web CORRECTION#2 Robert Baer <robertbaer@localnet.com> - 2016-08-28 21:32 -0800
Re: Read (and parse) file on the web CORRECTION#2 "Auric__" <not.my.real@email.address> - 2016-08-29 19:41 +0000
Re: Read (and parse) file on the web CORRECTION#2 Robert Baer <robertbaer@localnet.com> - 2016-08-30 00:21 -0800
Re: Read (and parse) file on the web CORRECTION#2 GS <gs@v.invalid> - 2016-08-30 06:06 -0400
Re: Read (and parse) file on the web CORRECTION#2 Robert Baer <robertbaer@localnet.com> - 2016-08-30 14:20 -0800
Re: Read (and parse) file on the web CORRECTION#2 GS <gs@v.invalid> - 2016-08-30 18:43 -0400
Re: Read (and parse) file on the web CORRECTION#2 GS <gs@v.invalid> - 2016-08-31 16:47 -0400
Re: Read (and parse) file on the web CORRECTION#2 Robert Baer <robertbaer@localnet.com> - 2016-09-01 04:37 -0800
Re: Read (and parse) file on the web CORRECTION#2 GS <gs@v.invalid> - 2016-09-01 13:13 -0400
Re: Read (and parse) file on the web CORRECTION#2 Robert Baer <robertbaer@localnet.com> - 2016-09-01 16:32 -0800
Re: Read (and parse) file on the web CORRECTION#2 GS <gs@v.invalid> - 2016-09-01 19:43 -0400
Re: Read (and parse) file on the web CORRECTION#2 Robert Baer <robertbaer@localnet.com> - 2016-09-01 04:23 -0800
Re: Read (and parse) file on the web CORRECTION#2 GS <gs@v.invalid> - 2016-09-05 10:10 -0400
Re: Read (and parse) file on the web CORRECTION#2 GS <gs@v.invalid> - 2016-09-05 10:17 -0400
Re: Read (and parse) file on the web CORRECTION#3 Robert Baer <robertbaer@localnet.com> - 2016-09-05 20:50 -0800
Re: Read (and parse) file on the web CORRECTION#2 Robert Baer <robertbaer@localnet.com> - 2016-09-05 20:32 -0800
Re: Read (and parse) file on the web CORRECTION#2 GS <gs@v.invalid> - 2016-09-06 10:49 -0400
Re: Read (and parse) file on the web CORRECTION#2 Robert Baer <robertbaer@localnet.com> - 2016-09-06 12:05 -0800
Re: Read (and parse) file on the web CORRECTION#2 GS <gs@v.invalid> - 2016-09-06 15:15 -0400
Re: Read (and parse) file on the web CORRECTION#2 Robert Baer <robertbaer@localnet.com> - 2016-09-06 22:32 -0800
Re: Read (and parse) file on the web CORRECTION#2 GS <gs@v.invalid> - 2016-09-07 11:44 -0400
Re: Read (and parse) file on the web CORRECTION#2 Robert Baer <robertbaer@localnet.com> - 2016-09-07 09:33 -0800
Re: Read (and parse) file on the web CORRECTION#2 Robert Baer <robertbaer@localnet.com> - 2016-09-08 20:18 -0800
Re: Read (and parse) file on the web CORRECTION#2 GS <gs@v.invalid> - 2016-09-09 06:07 -0400
Re: Read (and parse) file on the web CORRECTION#2 GS <gs@v.invalid> - 2016-09-09 22:05 -0400
Re: Read (and parse) file on the web CORRECTION#2 Robert Baer <robertbaer@localnet.com> - 2016-09-14 17:16 -0800
Re: Read (and parse) file on the web CORRECTION#2 GS <gs@v.invalid> - 2016-09-14 20:23 -0400
Re: Read (and parse) file on the web CORRECTION#2 "Auric__" <not.my.real@email.address> - 2016-09-07 02:37 +0000
Re: Read (and parse) file on the web CORRECTION#2 Robert Baer <robertbaer@localnet.com> - 2016-09-06 22:43 -0800
Re: Read (and parse) file on the web CORRECTION#2 "Auric__" <not.my.real@email.address> - 2016-09-08 05:49 +0000
Re: Read (and parse) file on the web CORRECTION#2 Robert Baer <robertbaer@localnet.com> - 2016-09-08 01:07 -0800
Re: Read (and parse) file on the web CORRECTION#2 "Auric__" <not.my.real@email.address> - 2016-09-09 06:40 +0000
Re: Read (and parse) file on the web WGET versions Robert Baer <robertbaer@localnet.com> - 2016-09-08 01:19 -0800
Re: Read (and parse) file on the web WGET versions Robert Baer <robertbaer@localnet.com> - 2016-09-08 20:09 -0800
Re: Read (and parse) file on the web CORRECTION#2 Robert Baer <robertbaer@localnet.com> - 2016-09-06 23:49 -0800
Re: Read (and parse) file on the web GS <gs@v.invalid> - 2016-08-28 15:51 -0400
Re: Read (and parse) file on the web Robert Baer <robertbaer@localnet.com> - 2016-08-28 21:37 -0800
Re: Read (and parse) file on the web Robert Baer <robertbaer@localnet.com> - 2016-09-02 19:09 -0800
Re: Read (and parse) file on the web GS <gs@v.invalid> - 2016-08-29 01:56 -0400
Re: Read (and parse) file on the web GS <gs@v.invalid> - 2016-08-29 02:21 -0400
Re: Read (and parse) file on the web Robert Baer <robertbaer@localnet.com> - 2016-08-29 19:55 -0800
Re: Read (and parse) file on the web GS <gs@v.invalid> - 2016-08-29 23:35 -0400
Re: Read (and parse) file on the web Robert Baer <robertbaer@localnet.com> - 2016-08-29 22:25 -0800
Re: Read (and parse) file on the web Robert Baer <robertbaer@localnet.com> - 2016-08-29 19:21 -0800
Re: Read (and parse) file on the web GS <gs@v.invalid> - 2016-08-29 23:21 -0400
Re: Read (and parse) file on the web Robert Baer <robertbaer@localnet.com> - 2016-08-29 22:10 -0800
Re: Read (and parse) file on the web GS <gs@v.invalid> - 2016-08-30 04:04 -0400
Re: Read (and parse) file on the web Robert Baer <robertbaer@localnet.com> - 2016-08-30 13:48 -0800
Re: Read (and parse) file on the web GS <gs@v.invalid> - 2016-08-29 23:37 -0400
Re: Read (and parse) file on the web Robert Baer <robertbaer@localnet.com> - 2016-08-29 22:25 -0800
Re: Read (and parse) file on the web lovexes17816 <lovexes17816@gmail.com> - 2016-08-28 14:00 +0100
Re: Read (and parse) file on the web lovexes17816 <lovexes17816@gmail.com> - 2016-08-29 02:17 +0100
Re: Read (and parse) file on the web Robert Baer <robertbaer@localnet.com> - 2016-09-02 19:11 -0800
Re: Read (and parse) file on the web "Auric__" <not.my.real@email.address> - 2016-09-03 04:12 +0000
Re: Read (and parse) file on the web Tikivn23143 <Tikivn23143@gmail.com> - 2016-08-30 02:28 +0100
Re: Read (and parse) file on the web everonvietnam2016 <everonvietnam2016@gmail.com> - 2016-09-07 15:03 +0100
Re: Read (and parse) file on the web Robert Baer <robertbaer@localnet.com> - 2016-09-07 08:43 -0800
Re: Read (and parse) file on the web "Auric__" <not.my.real@email.address> - 2016-09-08 05:43 +0000
Re: Read (and parse) file on the web Robert Baer <robertbaer@localnet.com> - 2016-09-08 01:09 -0800
Re: Read (and parse) file on the web "Auric__" <not.my.real@email.address> - 2016-09-09 06:49 +0000
Re: Read (and parse) file on the web everonvietnam2016 <everonvietnam2016@gmail.com> - 2016-09-08 11:11 +0100
Re: Read (and parse) file on the web GS <gs@v.invalid> - 2016-09-14 17:14 -0400
Re: Read (and parse) file on the web Robert Baer <robertbaer@localnet.com> - 2016-09-14 17:19 -0800
Re: Read (and parse) file on the web GS <gs@v.invalid> - 2016-09-14 20:26 -0400
Re: Read (and parse) file on the web Robert Baer <robertbaer@localnet.com> - 2016-09-15 09:59 -0800
Re: Read (and parse) file on the web GS <gs@v.invalid> - 2016-09-15 14:33 -0400
Re: Read (and parse) file on the web GS <gs@v.invalid> - 2016-09-14 21:28 -0400
Re: Read (and parse) file on the web Robert Baer <robertbaer@localnet.com> - 2016-09-15 10:05 -0800
Re: Read (and parse) file on the web Robert Baer <robertbaer@localnet.com> - 2016-09-14 18:48 -0800
Re: Read (and parse) file on the web GS <gs@v.invalid> - 2016-09-14 21:59 -0400
Re: Read (and parse) file on the web Robert Baer <robertbaer@localnet.com> - 2016-09-15 10:15 -0800
Re: Read (and parse) file on the web Robert Baer <robertbaer@localnet.com> - 2016-09-15 22:10 -0800
Re: Read (and parse) file on the web GS <gs@v.invalid> - 2016-09-16 11:50 -0400
Re: Read (and parse) file on the web Robert Baer <robertbaer@localnet.com> - 2016-09-16 23:54 -0800
Re: Read (and parse) file on the web GS <gs@v.invalid> - 2016-09-17 11:06 -0400
Re: Read (and parse) file on the web Robert Baer <robertbaer@localnet.com> - 2016-09-17 23:59 -0800
Re: Read (and parse) file on the web GS <gs@v.invalid> - 2016-09-18 03:22 -0400
Re: Read (and parse) file on the web Robert Baer <robertbaer@localnet.com> - 2016-09-20 00:20 -0800
Re: Read (and parse) file on the web GS <gs@v.invalid> - 2016-09-17 11:14 -0400
Re: Read (and parse) file on the web Robert Baer <robertbaer@localnet.com> - 2016-09-18 00:18 -0800
Re: Read (and parse) file on the web GS <gs@v.invalid> - 2016-09-18 03:30 -0400
Re: Read (and parse) file on the web GS <gs@v.invalid> - 2016-09-15 05:02 -0400
Re: Read (and parse) file on the web - revised GS <gs@v.invalid> - 2016-09-17 17:15 -0400
csiph-web