Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > microsoft.public.excel.programming > #109248

Re: Read (and parse) file on the web

From GS <gs@v.invalid>
Newsgroups microsoft.public.excel.programming, microsoft.public.excel
Subject Re: Read (and parse) file on the web
Date 2016-08-30 04:04 -0400
Organization A noiseless patient Spider
Message-ID <nq3els$as9$1@dont-email.me> (permalink)
References (2 earlier) <6bwwz.1429$EP2.1313@fx12.iad> <nq0iqm$65d$1@dont-email.me> <C46xz.16588$oU2.7722@fx09.iad> <nq2u36$224$1@dont-email.me> <Oy8xz.31343$6d.20556@fx26.iad>

Cross-posted to 2 groups.

Show all headers | View raw

>  I note that in Windoze, that "\" is used,and on the web, "/" is 
> used.

For clarity.., a Windows file path is not the same as a URL.

  Windows file paths allow spaces; URLs do not.
  Windows path delimiter is "\"; Web path delimiter is "/".

The function URLDownloadToFile() downloads szURL to szFilename.

Once downloaded, szFilename needs to be opened, parsed, and result 
stored locally. Then the next file needs to be downloaded, parsed, and 
stored. And so on until all files have been downloaded and parsed.

Since the actual page contents comprise only a small portion of the 
files being downloaded, there size should be considerably smaller after 
parsing. If you extract the data (text) only (no images) and save this 
to a txt file you should be able to 'append' to a single file which 
would result in occupying far less disc space. (For example, pg5 is 
less than 1kb) I suspect, though, that you need the image to identify 
the item source (Raytheon, RCA, Lucent Tech, NAWC, MIL STD, etc) 
because this info is not stored in the image file metadata. Otherwise, 
the txt file after parsing pg5's text is the following 53 lines:

NSN 5960-00-509-3171
    5960-00-509-3171

    ELECTRON TUBE

NSN 5960-00-569-9531
    5960-00-569-9531

    ELECTRON TUBE

NSN 5960-00-553-3770
    5960-00-553-3770

    ELECTRON TUBE

NSN 5960-00-682-8624
    5960-00-682-8624

    ELECTRON TUBE

NSN 5960-00-808-6928
    5960-00-808-6928

    ELECTRON TUBE

NSN 5960-00-766-1953
    5960-00-766-1953

    ELECTRON TUBE

NSN 5960-00-850-6169
    5960-00-850-6169

    ELECTRON TUBE

NSN 5960-00-679-8153
    5960-00-679-8153

    ELECTRON TUBE

NSN 5960-00-134-6884
    5960-00-134-6884

    ELECTRON TUBE

NSN 5960-00-061-8610
    5960-00-061-8610

    ELECTRON TUBE

    5960-00-067-9636

    ELECTRON TUBE

The file size is 711 bytes, and lists 11 items. Note the last item has 
no image and so no filler text (NSN line). This inconsistency makes 
parsing the contents difficult since you don't know which items do not 
have images.

If you copy/paste pg5 into Excel you get both text and image. You could 
then do something to construct the info in a database fashion...

  Col Headers:
  Source :: PartNum :: Description

..and put the data in the respective columns. This seems very 
inefficient but is probably less daunting than what you've been doing 
manually thus far. Auto Complete should be helpful with this, and you 
could sort the list by Source. Note that clicking the image or part# on 
the worksheet takes you to the same page as does clicking it on the web 
page. In the case of pg5, the data will occupy 11 rows.

Seems like your approach is the long way; -I'd find a better data 
source myself! Perhaps subscribe to an electronics database utility 
(such as my CAD software would use) that I can update by downloading a 
single db file<g>

-- 
Garry

Free usenet access at http://www.eternal-september.org
Classic VB Users Regroup!
  comp.lang.basic.visual.misc
  microsoft.public.vb.general.discussion

---
This email has been checked for viruses by Avast antivirus software.
https://www.avast.com/antivirus

Back to microsoft.public.excel.programming | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Read (and parse)  file on the web Robert Baer <robertbaer@localnet.com> - 2016-08-26 10:57 -0800
  Re: Read (and parse)  file on the web "Auric__" <not.my.real@email.address> - 2016-08-26 15:21 +0000
    Re: Read (and parse)  file on the web "Auric__" <not.my.real@email.address> - 2016-08-26 15:22 +0000
      Re: Read (and parse)  file on the web Robert Baer <robertbaer@localnet.com> - 2016-08-27 13:04 -0800
    Re: Read (and parse)  file on the web Robert Baer <robertbaer@localnet.com> - 2016-08-27 13:00 -0800
    Re: Read (and parse)  file on the web Robert Baer <robertbaer@localnet.com> - 2016-08-27 23:14 -0800
      Re: Read (and parse)  file on the web CORRECTION Robert Baer <robertbaer@localnet.com> - 2016-08-28 09:42 -0800
        Re: Read (and parse)  file on the web CORRECTION#2 Robert Baer <robertbaer@localnet.com> - 2016-08-28 10:08 -0800
          Re: Read (and parse)  file on the web CORRECTION#2 "Auric__" <not.my.real@email.address> - 2016-08-29 01:51 +0000
            Re: Read (and parse)  file on the web CORRECTION#2 Robert Baer <robertbaer@localnet.com> - 2016-08-28 21:32 -0800
              Re: Read (and parse)  file on the web CORRECTION#2 "Auric__" <not.my.real@email.address> - 2016-08-29 19:41 +0000
                Re: Read (and parse)  file on the web CORRECTION#2 Robert Baer <robertbaer@localnet.com> - 2016-08-30 00:21 -0800
                Re: Read (and parse)  file on the web CORRECTION#2 GS <gs@v.invalid> - 2016-08-30 06:06 -0400
                Re: Read (and parse)  file on the web CORRECTION#2 Robert Baer <robertbaer@localnet.com> - 2016-08-30 14:20 -0800
                Re: Read (and parse)  file on the web CORRECTION#2 GS <gs@v.invalid> - 2016-08-30 18:43 -0400
                Re: Read (and parse)  file on the web CORRECTION#2 GS <gs@v.invalid> - 2016-08-31 16:47 -0400
                Re: Read (and parse)  file on the web CORRECTION#2 Robert Baer <robertbaer@localnet.com> - 2016-09-01 04:37 -0800
                Re: Read (and parse)  file on the web CORRECTION#2 GS <gs@v.invalid> - 2016-09-01 13:13 -0400
                Re: Read (and parse)  file on the web CORRECTION#2 Robert Baer <robertbaer@localnet.com> - 2016-09-01 16:32 -0800
                Re: Read (and parse)  file on the web CORRECTION#2 GS <gs@v.invalid> - 2016-09-01 19:43 -0400
                Re: Read (and parse)  file on the web CORRECTION#2 Robert Baer <robertbaer@localnet.com> - 2016-09-01 04:23 -0800
                Re: Read (and parse)  file on the web CORRECTION#2 GS <gs@v.invalid> - 2016-09-05 10:10 -0400
                Re: Read (and parse)  file on the web CORRECTION#2 GS <gs@v.invalid> - 2016-09-05 10:17 -0400
                Re: Read (and parse)  file on the web CORRECTION#3 Robert Baer <robertbaer@localnet.com> - 2016-09-05 20:50 -0800
                Re: Read (and parse)  file on the web CORRECTION#2 Robert Baer <robertbaer@localnet.com> - 2016-09-05 20:32 -0800
                Re: Read (and parse)  file on the web CORRECTION#2 GS <gs@v.invalid> - 2016-09-06 10:49 -0400
                Re: Read (and parse)  file on the web CORRECTION#2 Robert Baer <robertbaer@localnet.com> - 2016-09-06 12:05 -0800
                Re: Read (and parse)  file on the web CORRECTION#2 GS <gs@v.invalid> - 2016-09-06 15:15 -0400
                Re: Read (and parse)  file on the web CORRECTION#2 Robert Baer <robertbaer@localnet.com> - 2016-09-06 22:32 -0800
                Re: Read (and parse)  file on the web CORRECTION#2 GS <gs@v.invalid> - 2016-09-07 11:44 -0400
                Re: Read (and parse)  file on the web CORRECTION#2 Robert Baer <robertbaer@localnet.com> - 2016-09-07 09:33 -0800
                Re: Read (and parse)  file on the web CORRECTION#2 Robert Baer <robertbaer@localnet.com> - 2016-09-08 20:18 -0800
                Re: Read (and parse)  file on the web CORRECTION#2 GS <gs@v.invalid> - 2016-09-09 06:07 -0400
                Re: Read (and parse)  file on the web CORRECTION#2 GS <gs@v.invalid> - 2016-09-09 22:05 -0400
                Re: Read (and parse)  file on the web CORRECTION#2 Robert Baer <robertbaer@localnet.com> - 2016-09-14 17:16 -0800
                Re: Read (and parse)  file on the web CORRECTION#2 GS <gs@v.invalid> - 2016-09-14 20:23 -0400
                Re: Read (and parse)  file on the web CORRECTION#2 "Auric__" <not.my.real@email.address> - 2016-09-07 02:37 +0000
                Re: Read (and parse)  file on the web CORRECTION#2 Robert Baer <robertbaer@localnet.com> - 2016-09-06 22:43 -0800
                Re: Read (and parse)  file on the web CORRECTION#2 "Auric__" <not.my.real@email.address> - 2016-09-08 05:49 +0000
                Re: Read (and parse)  file on the web CORRECTION#2 Robert Baer <robertbaer@localnet.com> - 2016-09-08 01:07 -0800
                Re: Read (and parse)  file on the web CORRECTION#2 "Auric__" <not.my.real@email.address> - 2016-09-09 06:40 +0000
                Re: Read (and parse)  file on the web WGET versions Robert Baer <robertbaer@localnet.com> - 2016-09-08 01:19 -0800
                Re: Read (and parse)  file on the web WGET versions Robert Baer <robertbaer@localnet.com> - 2016-09-08 20:09 -0800
                Re: Read (and parse)  file on the web CORRECTION#2 Robert Baer <robertbaer@localnet.com> - 2016-09-06 23:49 -0800
      Re: Read (and parse)  file on the web GS <gs@v.invalid> - 2016-08-28 15:51 -0400
        Re: Read (and parse)  file on the web Robert Baer <robertbaer@localnet.com> - 2016-08-28 21:37 -0800
        Re: Read (and parse)  file on the web Robert Baer <robertbaer@localnet.com> - 2016-09-02 19:09 -0800
      Re: Read (and parse)  file on the web GS <gs@v.invalid> - 2016-08-29 01:56 -0400
        Re: Read (and parse)  file on the web GS <gs@v.invalid> - 2016-08-29 02:21 -0400
          Re: Read (and parse)  file on the web Robert Baer <robertbaer@localnet.com> - 2016-08-29 19:55 -0800
            Re: Read (and parse)  file on the web GS <gs@v.invalid> - 2016-08-29 23:35 -0400
              Re: Read (and parse)  file on the web Robert Baer <robertbaer@localnet.com> - 2016-08-29 22:25 -0800
        Re: Read (and parse)  file on the web Robert Baer <robertbaer@localnet.com> - 2016-08-29 19:21 -0800
          Re: Read (and parse)  file on the web GS <gs@v.invalid> - 2016-08-29 23:21 -0400
            Re: Read (and parse)  file on the web Robert Baer <robertbaer@localnet.com> - 2016-08-29 22:10 -0800
              Re: Read (and parse)  file on the web GS <gs@v.invalid> - 2016-08-30 04:04 -0400
                Re: Read (and parse)  file on the web Robert Baer <robertbaer@localnet.com> - 2016-08-30 13:48 -0800
          Re: Read (and parse)  file on the web GS <gs@v.invalid> - 2016-08-29 23:37 -0400
            Re: Read (and parse)  file on the web Robert Baer <robertbaer@localnet.com> - 2016-08-29 22:25 -0800
    Re: Read (and parse)  file on the web lovexes17816 <lovexes17816@gmail.com> - 2016-08-28 14:00 +0100
    Re: Read (and parse)  file on the web lovexes17816 <lovexes17816@gmail.com> - 2016-08-29 02:17 +0100
      Re: Read (and parse)  file on the web Robert Baer <robertbaer@localnet.com> - 2016-09-02 19:11 -0800
        Re: Read (and parse)  file on the web "Auric__" <not.my.real@email.address> - 2016-09-03 04:12 +0000
    Re: Read (and parse)  file on the web Tikivn23143 <Tikivn23143@gmail.com> - 2016-08-30 02:28 +0100
    Re: Read (and parse)  file on the web everonvietnam2016 <everonvietnam2016@gmail.com> - 2016-09-07 15:03 +0100
      Re: Read (and parse)  file on the web Robert Baer <robertbaer@localnet.com> - 2016-09-07 08:43 -0800
        Re: Read (and parse)  file on the web "Auric__" <not.my.real@email.address> - 2016-09-08 05:43 +0000
          Re: Read (and parse)  file on the web Robert Baer <robertbaer@localnet.com> - 2016-09-08 01:09 -0800
            Re: Read (and parse)  file on the web "Auric__" <not.my.real@email.address> - 2016-09-09 06:49 +0000
    Re: Read (and parse)  file on the web everonvietnam2016 <everonvietnam2016@gmail.com> - 2016-09-08 11:11 +0100
  Re: Read (and parse)  file on the web GS <gs@v.invalid> - 2016-09-14 17:14 -0400
    Re: Read (and parse)  file on the web Robert Baer <robertbaer@localnet.com> - 2016-09-14 17:19 -0800
      Re: Read (and parse)  file on the web GS <gs@v.invalid> - 2016-09-14 20:26 -0400
        Re: Read (and parse)  file on the web Robert Baer <robertbaer@localnet.com> - 2016-09-15 09:59 -0800
          Re: Read (and parse)  file on the web GS <gs@v.invalid> - 2016-09-15 14:33 -0400
      Re: Read (and parse)  file on the web GS <gs@v.invalid> - 2016-09-14 21:28 -0400
        Re: Read (and parse)  file on the web Robert Baer <robertbaer@localnet.com> - 2016-09-15 10:05 -0800
    Re: Read (and parse)  file on the web Robert Baer <robertbaer@localnet.com> - 2016-09-14 18:48 -0800
      Re: Read (and parse)  file on the web GS <gs@v.invalid> - 2016-09-14 21:59 -0400
        Re: Read (and parse)  file on the web Robert Baer <robertbaer@localnet.com> - 2016-09-15 10:15 -0800
          Re: Read (and parse)  file on the web Robert Baer <robertbaer@localnet.com> - 2016-09-15 22:10 -0800
            Re: Read (and parse)  file on the web GS <gs@v.invalid> - 2016-09-16 11:50 -0400
              Re: Read (and parse)  file on the web Robert Baer <robertbaer@localnet.com> - 2016-09-16 23:54 -0800
                Re: Read (and parse)  file on the web GS <gs@v.invalid> - 2016-09-17 11:06 -0400
                Re: Read (and parse)  file on the web Robert Baer <robertbaer@localnet.com> - 2016-09-17 23:59 -0800
                Re: Read (and parse)  file on the web GS <gs@v.invalid> - 2016-09-18 03:22 -0400
                Re: Read (and parse)  file on the web Robert Baer <robertbaer@localnet.com> - 2016-09-20 00:20 -0800
                Re: Read (and parse)  file on the web GS <gs@v.invalid> - 2016-09-17 11:14 -0400
                Re: Read (and parse)  file on the web Robert Baer <robertbaer@localnet.com> - 2016-09-18 00:18 -0800
                Re: Read (and parse)  file on the web GS <gs@v.invalid> - 2016-09-18 03:30 -0400
      Re: Read (and parse)  file on the web GS <gs@v.invalid> - 2016-09-15 05:02 -0400
        Re: Read (and parse)  file on the web - revised GS <gs@v.invalid> - 2016-09-17 17:15 -0400

csiph-web