Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.javascript > #37022

Re: Firefox changed IFRAME behaviour - I need a remedy??

Show all headers | View raw

On Thursday, 1 August 2019 02:58:06 UTC+1, Michael Haufe (TNO)  wrote:
> On Wednesday, July 31, 2019 at 6:14:15 AM UTC-5, dr.j.r...@gmail.com wrote:
> > On Wednesday, 31 July 2019 02:24:10 UTC+1, Michael Haufe (TNO)  wrote:
> > > I had three different answers for you, but before I could finish each one a new question comes up. Ultimately: Why do you not want to use IIS? It takes minutes to get your site running. Is it only because of distribution? Are these other machines on a shared network? Context, context. What is this project?
> > 
> > Well, there is a simple answer to the first question - to the best of my recollection, this is the first time I've heard of such a thing being part of Windows.
> > 
> > I wrote LINXCHEK.HTM for me, to run on my main PC (and I want it to run on any machine I designate as "main" - and late last year I had to redesignate, as my main machine's C hard drive had failed).  I had, IIRC, coded LINXCHEK to be disabled before doing anything useful if loaded from my Web site, since for bandwidth reasons I did not want anyone to download it very frequently (anyone could disable that bit of code; but they would not be able to put the page in MY website) - therefore, I don't know what changes might be needed to make it run fully from a server).
> > 
> > Other machines might be on networks, but don't share with me.
> > 
> > The page LINXCHEK.HTM runs a local spider.
> > 
> > Way back when I had only dial-up Internet, I wanted to be able to check links and anchors on the local master copy of my Web site; I wrote a Borland Pascal 7 program (CHEKLINX.PAS, 16-bit EXE) to empirically parse an HTM page (to help it, I write in a large subset of HTML), get the links and anchors, and check; and it would follow local links to HTM pages, recursively.  When I got Delphi 3 I made a CHEKLINX.EXE (32-bit) with it - and that still runs.
> > 
> > But when I discovered that HTML has IFRAME, and IFRAME has contentDocument.body, I realised that I can spider the properly-parsed bodies of my local INDEX.HTM page and all local pages that can be reached from it, and do the above much better with many added features (such as checking YYYY-MM-DD DoW) to see if DoW is calendrically correct <g>).  That is  LINXCHEK.HTM.
> > 
> > And, now that the C drive is a solid-state disc, Firefox processes^Hd  over 200 pages in about 20 seconds.
> > 
> > LINXCHEK.HTM starts by reading a Directory File, essentially as obtained by previously running Prompt>DIR /b /s > $DIR.TXT .
> > 
> > 
> > For (much) more description, see an older version of the page itself at
> > https://web.archive.org/web/20140809171940/http://www.merlyn.demon.co.uk/linxchek.htm
> > 
> > That will not run when loaded from that server, as the GO-type button is disabled; but one can read the page, and one can download it with Save As and then run it (after making a suitable Directory File).  It would like a stylesheet and an include file, but (IIRC) neither is really needed.
> > 
> > 
> > 
> > I've gutted a copy of one of the HTAs that you helped me to write, and intend next to see what happens if I get it to read a whole HTM page into a string
> > and assign that string to the innerHTML of a DIV.  If I get a DOM-like tree in the DIV, I can proceed ...
> > 
> > 
> > Must stop, routine medical appointment imminent.
> > 
> > Thanks, regards,


 
> IIS has been part of windows [1] since Windows NT 3.51. I highly suggest
> you use it.
> 
> 1. Navigate to C:\inetpub\wwwroot.
> 2. Create a folder for your application/website and copy/move your files there.
>    - example: 'merlyn.local'
> 3. Open the start menu and type "iis". You'll see the application. open it.
>   - Assuming Windows 10
>   - If on older version of windows, from the Start -> Run command type: "inetmgr" and hit ENTER.
> 4. On the left side you'll see a tree. Expand it and locate the "Sites" folder
> 5. Right click on the folder and select "Add Website..."
> 6. Fill out the form (example entries):
>   - Site name: 'merlyn.local'
>   - Physical path: The path from step 2: 'C:\inetpub\wwwroot\merlyn.local'
>   - Host name: 'merlyn.local'
> 7. update hosts file for your website
>   - Open (as an administrator) 'C:\Windows\System32\drivers\etc\hosts'
>   - Follow the example and add a new line to this file:
> 	127.0.0.1	merlyn.local
>   -  save and close. Agai, you need to be an administrator to do this
> 8. Open your web browser and navigate to: 'http://merlyn.local'
> 	- or if you are missing an index.html page: 'http://merlyn.local/foo.html'
> 
> DONE!


But I want to be able to run my process on an "as delivered"
computer, without having to "install" little-known software,
even if that is actually already present.  To that end, for
example, while my page LINXCHEK.HTM contains references to
an include file and a CSS file, I have ensured that neither
of those is necessary for operation.


> You now have a static website on your machine. You also have an option for server side programming as well.
> 
> You can host this as is for free in a number of places. For example on Azure.
> 
> If you choose Azure as your hosting provider, you'd have a website like:
> <http://merlyn.azurewebsites.net/>
> 
> For example if you don't want or can't afford a host name of your own.
> 
> Github pages is another free option, but you'll have to understand and use Git.
> <https://pages.github.com/>
> 
> Other people in the group may have other options for hosting.
> 
> Now on to your original problem:
> 
> You should be able to display your files like you used to since you now have a proper domain instead of <file://...>.
> 
> In regards to checking links, you can use the DOM instead of the file system:


I don't follow. My Pascal CHEKLINX.EXE knows nothing about the
DOM, because I knew nothing about the DOM when I wrote it. But
LINXCHEK.HTM relies on the DOM in the IFRAME.


> <script>
> // This is impressionistic, and not meant for usage as-is
> let visited = new Set()
> 
> function crawlSite(url) {
>     fetch(url).then(function(response){
>         // Assuming textual response
>         return response.text()
>     }).then(function(htmlText){
>         var parser = new DOMParser();
>         var doc = parser.parseFromString(htmlText, "text/html");
>         var links = Array.from(doc.querySelectorAll('a'))
>         return links
>     }).then(function(links) {
>         // Only crawl links inside your own website
>         var filteredLinks = links.filter(function(link){
>             return link.hostname === document.location.hostname 
>                 && !visited.has(link.href)
>         })
>         return filteredLinks
>     }).then(function(filteredLinks){
>         filteredLinks.forEach(function(link){
>             visited.add(link.href)
>             crawlSite(link.href)
>         })
>     }).catch(function(error){
>         // report error where appropriate
>         console.log('Failed to fetch page: ', err);  
>     })
> }
> 
> //start at root
> crawlSite(document.location.hostname)
> 
> </script>

That contains too much grammar that I'm not familiar with.
And I had no idea that DOMParser existed, apart from a thought
that it would be nice if it did.  I've found descriptions of
how to call DOMParser, but nothing about just what its result
is.

 
> You'll want to do this as a Breadth First Search so you don't look at the same page more than once,
?  I don't follow.  The pages being scanned are of course from
 tree structure, but the links-and-anchors which connect them are
a network.  I just work through the links collection of a page
recursing into only those destinations not yet visited - and I
think yours does similarly.  I keep (I think) the anchors
collection of each page found, since a page found subsequently
may want to check an anchor of a previously found page.

> and additionally you'll want to filter the links to stay in your own domain. Also in my impressionistic example I left out other checks and such to keep it relatively clear. I'm certain other group members can point out bugs and such in my pseudo-code where appropriate.

I'm sure that your code would be an excellent restarting point
if I needed to return to square one, and it would very probably
take less time than my equivalent.  But I'm confident that the
majority of the run time is taken in the actual fetching and
parsing of the pages being scanned.


> [1] https://en.wikipedia.org/wiki/Internet_Information_Services


-- 
  (c) John Stockton, near London, UK.  Using Google Groups.           |
 Mail: J.R.""""""""@physics.org - or as Reply-To, if any.             |

Back to comp.lang.javascript | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Firefox changed IFRAME behaviour - I need a remedy?? dr.j.r.stockton@gmail.com - 2019-07-20 12:53 -0700
  Re: Firefox changed IFRAME behaviour - I need a remedy?? Andrew Poulos <ap_prog@hotmail.com> - 2019-07-21 15:15 +1000
  Re: Firefox changed IFRAME behaviour - I need a remedy?? "Michael Haufe (TNO)" <tno@thenewobjective.com> - 2019-07-22 03:07 -0700
  Re: Firefox changed IFRAME behaviour - I need a remedy?? Martin Honnen <martin.honnen@gmx.de> - 2019-07-22 16:20 +0200
    Re: Firefox changed IFRAME behaviour - I need a remedy?? dr.j.r.stockton@gmail.com - 2019-07-30 08:20 -0700
  Re: Firefox changed IFRAME behaviour - I need a remedy?? "Michael Haufe (TNO)" <tno@thenewobjective.com> - 2019-07-30 18:24 -0700
    Re: Firefox changed IFRAME behaviour - I need a remedy?? dr.j.r.stockton@gmail.com - 2019-07-31 04:14 -0700
      Re: Firefox changed IFRAME behaviour - I need a remedy?? dr.j.r.stockton@gmail.com - 2019-07-31 05:32 -0700
        Re: Firefox changed IFRAME behaviour - I need a remedy?? "R.Wieser" <address@not.available> - 2019-07-31 18:22 +0200
          Re: Firefox changed IFRAME behaviour - I need a remedy?? dr.j.r.stockton@gmail.com - 2019-08-02 02:37 -0700
            Re: Firefox changed IFRAME behaviour - I need a remedy?? "R.Wieser" <address@not.available> - 2019-08-02 13:38 +0200
              Re: Firefox changed IFRAME behaviour - I need a remedy?? dr.j.r.stockton@gmail.com - 2019-08-02 05:59 -0700
                Re: Firefox changed IFRAME behaviour - I need a remedy?? "R.Wieser" <address@not.available> - 2019-08-02 17:55 +0200
                Re: Firefox changed IFRAME behaviour - I need a remedy?? "Michael Haufe (TNO)" <tno@thenewobjective.com> - 2019-08-02 15:57 -0700
                Re: Firefox changed IFRAME behaviour - I need a remedy?? dr.j.r.stockton@gmail.com - 2019-08-03 06:11 -0700
                Re: Firefox changed IFRAME behaviour - I need a remedy?? Thomas 'PointedEars' Lahn <PointedEars@web.de> - 2019-08-04 11:44 +0200
              Re: Firefox changed IFRAME behaviour - I need a remedy?? "Michael Haufe (TNO)" <tno@thenewobjective.com> - 2019-08-02 15:10 -0700
                Re: Firefox changed IFRAME behaviour - I need a remedy?? "R.Wieser" <address@not.available> - 2019-08-03 08:59 +0200
                OT: Re: Firefox changed IFRAME behaviour - I need a remedy?? dr.j.r.stockton@gmail.com - 2019-08-03 05:18 -0700
                Re: OT: Re: Firefox changed IFRAME behaviour - I need a remedy?? "Evertjan." <exxjxw.hannivoort@inter.nl.net> - 2019-08-03 15:27 +0200
                Re: OT: Re: Firefox changed IFRAME behaviour - I need a remedy?? dr.j.r.stockton@gmail.com - 2019-08-03 09:28 -0700
                Re: OT: Re: Firefox changed IFRAME behaviour - I need a remedy?? "Evertjan." <exxjxw.hannivoort@inter.nl.net> - 2019-08-04 12:32 +0200
                Re: Firefox changed IFRAME behaviour - I need a remedy?? Thomas 'PointedEars' Lahn <PointedEars@web.de> - 2019-08-04 11:30 +0200
                Re: Firefox changed IFRAME behaviour - I need a remedy?? "Michael Haufe (TNO)" <tno@thenewobjective.com> - 2019-08-04 15:31 -0700
                Re: Firefox changed IFRAME behaviour - I need a remedy?? Thomas 'PointedEars' Lahn <PointedEars@web.de> - 2019-08-05 13:48 +0200
                Re: Firefox changed IFRAME behaviour - I need a remedy?? "Michael Haufe (TNO)" <tno@thenewobjective.com> - 2019-08-05 16:51 -0700
                Re: Firefox changed IFRAME behaviour - I need a remedy?? Thomas 'PointedEars' Lahn <PointedEars@web.de> - 2019-08-07 01:42 +0200
                Re: Firefox changed IFRAME behaviour - I need a remedy?? "Michael Haufe (TNO)" <tno@thenewobjective.com> - 2019-08-07 01:47 -0700
                Re: Firefox changed IFRAME behaviour - I need a remedy?? Thomas 'PointedEars' Lahn <PointedEars@web.de> - 2019-08-08 20:56 +0200
                Re: Firefox changed IFRAME behaviour - I need a remedy?? "Michael Haufe (TNO)" <tno@thenewobjective.com> - 2019-08-08 13:16 -0700
                Re: Firefox changed IFRAME behaviour - I need a remedy?? Martin Leese <please@see.Web.for.e-mail.INVALID> - 2019-08-08 23:57 -0600
                Re: Firefox changed IFRAME behaviour - I need a remedy?? Thomas 'PointedEars' Lahn <PointedEars@web.de> - 2019-08-09 12:48 +0200
                Re: Firefox changed IFRAME behaviour - I need a remedy?? dr.j.r.stockton@gmail.com - 2019-08-05 06:37 -0700
                Re: Firefox changed IFRAME behaviour - I need a remedy?? John G Harris <niam@jghnorth.org.uk.invalid> - 2019-08-05 18:46 +0100
                Re: Firefox changed IFRAME behaviour - I need a remedy?? dr.j.r.stockton@gmail.com - 2019-08-06 03:39 -0700
                Re: Firefox changed IFRAME behaviour - I need a remedy?? "Michael Haufe (TNO)" <tno@thenewobjective.com> - 2019-08-05 17:30 -0700
            Re: Firefox changed IFRAME behaviour - I need a remedy?? "Michael Haufe (TNO)" <tno@thenewobjective.com> - 2019-08-02 15:05 -0700
      Re: Firefox changed IFRAME behaviour - I need a remedy?? "Michael Haufe (TNO)" <tno@thenewobjective.com> - 2019-07-31 18:58 -0700
        Re: Firefox changed IFRAME behaviour - I need a remedy?? dr.j.r.stockton@gmail.com - 2019-08-02 05:01 -0700
          Re: Firefox changed IFRAME behaviour - I need a remedy?? "Michael Haufe (TNO)" <tno@thenewobjective.com> - 2019-08-02 16:07 -0700
            Re: Firefox changed IFRAME behaviour - I need a remedy?? dr.j.r.stockton@gmail.com - 2019-08-03 08:19 -0700
              Re: Firefox changed IFRAME behaviour - I need a remedy?? "Michael Haufe (TNO)" <tno@thenewobjective.com> - 2019-08-03 18:42 -0700
                Re: Firefox changed IFRAME behaviour - I need a remedy?? Thomas 'PointedEars' Lahn <PointedEars@web.de> - 2019-08-04 11:27 +0200

csiph-web