Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.javascript > #30536 > unrolled thread

Nofollow javascript

Back to article view | Back to comp.lang.javascript

Contents

  Nofollow javascript Andre Trevas <andretrevas@gmail.com> - 2016-05-24 23:11 -0700
    Re: Nofollow javascript "Evertjan." <exxjxw.hannivoort@inter.nl.net> - 2016-05-25 09:23 +0200
      Re: Nofollow javascript Andre Trevas <andretrevas@gmail.com> - 2016-05-25 10:32 -0700
        Re: Nofollow javascript "Evertjan." <exxjxw.hannivoort@inter.nl.net> - 2016-05-25 21:08 +0200
          Re: Nofollow javascript Andre Trevas <andretrevas@gmail.com> - 2016-05-25 15:09 -0700
            Re: Nofollow javascript "Evertjan." <exxjxw.hannivoort@inter.nl.net> - 2016-05-26 00:39 +0200
            Re: Nofollow javascript Thomas 'PointedEars' Lahn <PointedEars@web.de> - 2016-05-27 18:27 +0200
              Re: Nofollow javascript "Evertjan." <exxjxw.hannivoort@inter.nl.net> - 2016-05-28 11:48 +0200
    Re: Nofollow javascript "Michael Haufe (TNO)" <tno@thenewobjective.com> - 2016-05-25 06:10 -0700
      Re: Nofollow javascript Andre Trevas <andretrevas@gmail.com> - 2016-05-25 10:36 -0700
        Re: Nofollow javascript "Michael Haufe (TNO)" <tno@thenewobjective.com> - 2016-05-25 16:44 -0700
          Re: Nofollow javascript "Michael Haufe (TNO)" <tno@thenewobjective.com> - 2016-05-25 16:50 -0700
            Re: Nofollow javascript "Evertjan." <exxjxw.hannivoort@inter.nl.net> - 2016-05-26 12:55 +0200
              Re: Nofollow javascript "Michael Haufe (TNO)" <tno@thenewobjective.com> - 2016-05-26 05:14 -0700
                Re: Nofollow javascript "Evertjan." <exxjxw.hannivoort@inter.nl.net> - 2016-05-26 14:20 +0200
                  Re: Nofollow javascript "Michael Haufe (TNO)" <tno@thenewobjective.com> - 2016-05-26 05:55 -0700
    Re: Nofollow javascript Andre Trevas <andretrevas@gmail.com> - 2016-06-09 08:29 -0700

#30536 — Nofollow javascript

Hello,

In my website i have these lines:

<script type="text/javascript">
$.get( "{{URL::route("addFavorite",$realty->id)}}", function(data) {
	...										});
</script>

I can't move to a separate file because it is dynamically generated. How can i stop google from tracking these URLs? I'm facing a lot of troubles because of this (more than 10000 errors in google console).

[toc] | [next] | [standalone]

#30537

Andre Trevas <andretrevas@gmail.com> wrote on 25 May 2016 in
comp.lang.javascript: 

> Hello,
> 
> In my website i have these lines:
> 
> <script type="text/javascript">
> $.get( "{{URL::route("addFavorite",$realty->id)}}", function(data) {
>      ...  

THAT is not javascript in sensu strictior, 
it perhaps presupposes a library?

                                                });
> </script>
> 
> I can't move to a separate file because it is dynamically generated. 

Why would you want to move to a separate file?

If it is on your website you must be the master of such dynamic generation?

> How can i stop google from tracking these URLs?

What URLs? Your website? Use a robots-meta.

> I'm facing a lot of troubles
> because of this (more than 10000 errors in google console). 

And why are these troubles, you are facing?

Surely it is your visitors, not google console, you should be concerned 
with?

> Re: Nofollow javascript

Indeed, I don't.

-- 
Evertjan.
The Netherlands.
(Please change the x'es to dots in my emailaddress)

[toc] | [prev] | [next] | [standalone]

#30539

> > Hello,
> > 
> > In my website i have these lines:
> > 
> > <script type="text/javascript">
> > $.get( "{{URL::route("addFavorite",$realty->id)}}", function(data) {
> >      ...  
> 
> THAT is not javascript in sensu strictior, 
> it perhaps presupposes a library?
> 

Jquery library (jQuery v1.11.1), .get = jQuery.get()

>                                                 });
> > </script>
> > 
> > I can't move to a separate file because it is dynamically generated. 
> 
> Why would you want to move to a separate file?
> 

If i have one file "script.js" with this code i can use <script src="script.js" rel="nofollow"> and google won't track this URL inside the script (according to google developers)

> If it is on your website you must be the master of such dynamic generation?
> 

Yes, i'm the master but can i have one .js file with php functions? This code is integrate with php, so i need to dynamic build it (if in php, some code in js, else php, code js...)

> > How can i stop google from tracking these URLs?
> 
> What URLs? Your website? Use a robots-meta.

$.get( "{{URL::route("external.add.favorite",$realty[0]->idSINAI)}}", function(data) {
												});

My system will generate www.example.com.br/addFavorite/12111 and make one request. In this moment, google will add one line in Crawl Errors report
www.example.com.br/addFavorite/12111 error 500. You can't access this url directly from chrome, or google bots.


> 
> > I'm facing a lot of troubles
> > because of this (more than 10000 errors in google console). 
> 
> And why are these troubles, you are facing?
> 
> Surely it is your visitors, not google console, you should be concerned 
> with?

Error in https://www.google.com/webmasters/tools/crawl-errors?utm_source=support.google.com/webmasters/...


1   addFavoritos/94302      500    2/28/16
I have more than 4000 lines in crawl errors report

> 
> > Re: Nofollow javascript
> 
> Indeed, I don't.
> 
> -- 
> Evertjan.
> The Netherlands.
> (Please change the x'es to dots in my emailaddress)


What do i need? 
Every url with www.example.com.br/addFavorite/id can't be follow by google.

Do you understand now?
Errors image:
http://postimg.org/image/4jzbzqpqz/

[toc] | [prev] | [next] | [standalone]

#30541

Andre Trevas <andretrevas@gmail.com> wrote on 25 May 2016 in 
comp.lang.javascript:

> What do i need? 
> Every url with www.example.com.br/addFavorite/id can't be follow by 
google.

Methinks [!!!!] everything that you put on the web without a login 
CAN be indexed by others, like google.

However, Google has some voluntarily agreed restrictions, it seems [!!!]:

Any url with "...?id=1234" [any number]
won't be indexed and followed.

Any page with 
<meta name="robots" content="noindex, nofollow">
won't be indexed and followed.

Any page disallowed in the /robots.txt file as such
won't be indexed and followed.

And I doubt Google will follow URLs
mentioned in a piece of javascript in a page,
independent on if it has a .js, .php, .htmp, .asp extention.

And of course, google cannot see any serverside code in such pages,
when their extentions are flagged in the server to be executed by a 
serverside engine in stead of just downloaded to the user's browser.

So methinks you have a lot of choices.

> Every url with www.example.com.br/addFavorite/id 
> can't be follow by google.

Use "/robots.txt" disallow.
See <http://www.robotstxt.org/robotstxt.html>

-- 
Evertjan.
The Netherlands.
(Please change the x'es to dots in my emailaddress)

[toc] | [prev] | [next] | [standalone]

#30542

> > What do i need? 
> > Every url with www.example.com.br/addFavorite/id can't be follow by 
> google.
> 
> Methinks [!!!!] everything that you put on the web without a login 
> CAN be indexed by others, like google.
> 
> However, Google has some voluntarily agreed restrictions, it seems [!!!]:
> 
> Any url with "...?id=1234" [any number]
> won't be indexed and followed.
> 
> Any page with 
> <meta name="robots" content="noindex, nofollow">
> won't be indexed and followed.
> 
> Any page disallowed in the /robots.txt file as such
> won't be indexed and followed.
> 
> And I doubt Google will follow URLs
> mentioned in a piece of javascript in a page,
> independent on if it has a .js, .php, .htmp, .asp extention.
> 
> And of course, google cannot see any serverside code in such pages,
> when their extentions are flagged in the server to be executed by a 
> serverside engine in stead of just downloaded to the user's browser.
> 
> So methinks you have a lot of choices.
> 
> > Every url with www.example.com.br/addFavorite/id 
> > can't be follow by google.
> 
> Use "/robots.txt" disallow.
> See <http://www.robotstxt.org/robotstxt.html>
> 
> -- 
> Evertjan.
> The Netherlands.
> (Please change the x'es to dots in my emailaddress)

Evertjan, thank you very much.

Can i add this code in robots.txt

User-agent: *
Disallow: /addFavoritos/

And fix it?

[toc] | [prev] | [next] | [standalone]

#30543

Andre Trevas <andretrevas@gmail.com> wrote on 26 May 2016 in 
comp.lang.javascript:

>> Use "/robots.txt" disallow.
>> See <http://www.robotstxt.org/robotstxt.html>
> 
> Evertjan, thank you very much.
> 
> Can i add this code in robots.txt
> 
> User-agent: *
> Disallow: /addFavoritos/
> 
> And fix it?

Probably.

-- 
Evertjan.
The Netherlands.
(Please change the x'es to dots in my emailaddress)

[toc] | [prev] | [next] | [standalone]

#30570

Andre Trevas wrote:

>> > What do i need?
>> > Every url with www.example.com.br/addFavorite/id can't be follow by
>> > google.
>> 
>> Methinks [!!!!] everything that you put on the web without a login
>> CAN be indexed by others, like google.

Yes, of course.
 
>> However, Google has some voluntarily agreed restrictions, it seems [!!!]:
>> 
>> Any url with "...?id=1234" [any number]
>> won't be indexed and followed.

How did you get that idea?
 
>> Any page with
>> <meta name="robots" content="noindex, nofollow">
>> won't be indexed and followed.

Correct.

<https://support.google.com/webmasters/answer/79812>
 
>> Any page disallowed in the /robots.txt file as such
>> won't be indexed and followed.

Correct.

>> And I doubt Google will follow URLs
>> mentioned in a piece of javascript in a page,
>> independent on if it has a .js, .php, .htmp, .asp extention.

Given that Google even *executes* "javascript" [1], how did you get that 
idea?

[1] <https://webmasters.googleblog.com/2014/05/understanding-web-pages-better.html>
 
> Can i add this code in robots.txt
> 
> User-agent: *
> Disallow: /addFavoritos/
> 
> And fix it?

SEO is off-topic here.  

Also, this is not an Internet chat, but a Usenet discussion group; you 
should keep one attribution line per quotation level.

Please read the FAQ, and adjust your settings and behavior accordingly.

-- 
PointedEars
FAQ: <http://PointedEars.de/faq> | SVN: <http://PointedEars.de/wsvn/>
Twitter: @PointedEars2 | ES Matrix: <http://PointedEars.de/es-matrix>
Please do not cc me. / Bitte keine Kopien per E-Mail.

[toc] | [prev] | [next] | [standalone]

#30572

Thomas 'PointedEars' Lahn <PointedEars@web.de> wrote on 27 May 2016 in 
comp.lang.javascript:

>>> Any url with "...?id=1234" [any number]
>>> won't be indexed and followed.
> 
> How did you get that idea?

There used to be a Googlebot rule, 
[perhaps Google does not follow that anymore]: 

"Don't use "&id=" as a parameter in your URLs, 
as we don't include these pages in our index."

============================

Still, you can induce this behavour, it seems:

User-agent: Googlebot
Disallow: *id=

with this in robots.txt the google-bot not touch:

http://www.example.com/mypage.html?id=1234
and
http://www.example.com/mypage.html?c=d&id=1234&a=b

[not tested]

-- 
Evertjan.
The Netherlands.
(Please change the x'es to dots in my emailaddress)

[toc] | [prev] | [next] | [standalone]

#30538

On Wednesday, May 25, 2016 at 1:11:32 AM UTC-5, Andre Trevas wrote:
> Hello,
> 
> In my website i have these lines:
> 
> <script type="text/javascript">
> $.get( "{{URL::route("addFavorite",$realty->id)}}", function(data) {
> 	...										});
> </script>
> 
> I can't move to a separate file because it is dynamically generated. How can i stop google from tracking these URLs? I'm facing a lot of troubles because of this (more than 10000 errors in google console).

use robots.txt

Also, it is not true that you can't move this to a separate file, you just have to change your approach.

[toc] | [prev] | [next] | [standalone]

#30540

> > Hello,
> > 
> > In my website i have these lines:
> > 
> > <script type="text/javascript">
> > $.get( "{{URL::route("addFavorite",$realty->id)}}", function(data) {
> > 	...										});
> > </script>
> > 
> > I can't move to a separate file because it is dynamically generated. How can i stop google from tracking these URLs? I'm facing a lot of troubles because of this (more than 10000 errors in google console).
> 
> use robots.txt
> 
> Also, it is not true that you can't move this to a separate file, you just have to change your approach.

Can i create one rule in robots.txt? Because i have more than 20000 urls.

What do you suggest? How can i move it to a separate file? The code have php and javascript functions, and it is dynamically generated( it will be different from each realty).

[toc] | [prev] | [next] | [standalone]

#30544

On Wednesday, May 25, 2016 at 12:36:52 PM UTC-5, Andre Trevas wrote:

> Can i create one rule in robots.txt? Because i have more than 20000 urls.

Look at some examples syntax and you should be able to figure out what you're looking for:

<https://en.wikipedia.org/wiki/Robots_exclusion_standard#Examples>

> What do you suggest? How can i move it to a separate file? The code have php and javascript functions, and it is dynamically generated( it will be different from each realty).

I suspect your entire JavaScript file doesn't need to be generated. Just some variables

[toc] | [prev] | [next] | [standalone]

#30545

On Wednesday, May 25, 2016 at 6:44:28 PM UTC-5, Michael Haufe (TNO) wrote:
> On Wednesday, May 25, 2016 at 12:36:52 PM UTC-5, Andre Trevas wrote:
> 
> > Can i create one rule in robots.txt? Because i have more than 20000 urls.
> 
> Look at some examples syntax and you should be able to figure out what you're looking for:
> 
> <https://en.wikipedia.org/wiki/Robots_exclusion_standard#Examples>
> 
> > What do you suggest? How can i move it to a separate file? The code have php and javascript functions, and it is dynamically generated( it will be different from each realty).
> 
> I suspect your entire JavaScript file doesn't need to be generated. Just some variables

If that is the case, then simple generate those variable elsewhere. For example:

<script type="text/javascript" src="myFile.js" data-var1='<?php echo "foo"; ?>'></script>

Then inside myFile.js

(function(){
    "use strict"

    var currentScript = document.scripts[document.scripts.length - 1],
        foo = currentScript.getAttribute("data-var1");

    ...
})()

[toc] | [prev] | [next] | [standalone]

#30560

"Michael Haufe (TNO)" <tno@thenewobjective.com> wrote on 26 May 2016 in
comp.lang.javascript: 

>> I suspect your entire JavaScript file doesn't need to be generated.
>> Just some variables 
> 
> If that is the case, 

But is it?

> then simple generate those variable elsewhere. For
> example: 
> 
> <script type="text/javascript" src="myFile.js" data-var1='<?php echo
> "foo"; ?>'></script> 

Why not:

<script type="text/javascript" src="myJsFile.php"></script> 

and then inside that myJsFile.php file:

var fooVar = '<?php echo "foo"; ?>';

Do you realy think Google will extract urls from a js-variable?


-- 
Evertjan.
The Netherlands.
(Please change the x'es to dots in my emailaddress)

[toc] | [prev] | [next] | [standalone]

#30563

On Thursday, May 26, 2016 at 5:56:14 AM UTC-5, Evertjan. wrote:
> "Michael Haufe (TNO)" wrote on 26 May 2016 in
> comp.lang.javascript: 
> 
> >> I suspect your entire JavaScript file doesn't need to be generated.
> >> Just some variables 
> > 
> > If that is the case, 
> 
> But is it?

If/When the OP replies, we'll know.

> > then simple generate those variable elsewhere. For
> > example: 
> > 
> > <script type="text/javascript" src="myFile.js" data-var1='<?php echo
> > "foo"; ?>'></script> 
> 
> Why not:
> 
> <script type="text/javascript" src="myJsFile.php"></script> 
> 
> and then inside that myJsFile.php file:
> 
> var fooVar = '<?php echo "foo"; ?>';

I prefer not to interleave client and server code (Loose Coupling principle). 

> Do you realy think Google will extract urls from a js-variable?

Google will execute JavaScript

[toc] | [prev] | [next] | [standalone]

#30564

"Michael Haufe (TNO)" <tno@thenewobjective.com> wrote on 26 May 2016 in
comp.lang.javascript: 

>> var fooVar = '<?php echo "foo"; ?>';
> 
> I prefer not to interleave client and server code (Loose Coupling
> principle). 

I love it, it gives so many simple possibilities!

Perhaps I am more practical and you more principled?

>> Do you realy think Google will extract urls from a js-variable?
> 
> Google will execute JavaScript

Any substantiation?

And if so, which I doubt, 
will Google extract urls from a js-variable and follow that url?

-- 
Evertjan.
The Netherlands.
(Please change the x'es to dots in my emailaddress)

[toc] | [prev] | [next] | [standalone]

#30565

On Thursday, May 26, 2016 at 7:20:45 AM UTC-5, Evertjan. wrote:
> "Michael Haufe (TNO)" wrote on 26 May 2016 in
> comp.lang.javascript: 
> 
> >> var fooVar = '<?php echo "foo"; ?>';
> > 
> > I prefer not to interleave client and server code (Loose Coupling
> > principle). 
> 
> I love it, it gives so many simple possibilities!
> 
> Perhaps I am more practical and you more principled?

More principled? Probably.

Less Practical? That depends. If you define practical as prioritizing short term goals: then yes, I'd agree with you here as well.

> >> Do you realy think Google will extract urls from a js-variable?
> > 
> > Google will execute JavaScript
> 
> Any substantiation?

<https://webmasters.googleblog.com/2015/10/deprecating-our-ajax-crawling-scheme.html>

There may be a newer reference available

> And if so, which I doubt, 
> will Google extract urls from a js-variable and follow that url?

I don't think so. I "believe" it still looks at DOM elements and simulates clicks across the page (avoiding non-GET requests).

[toc] | [prev] | [next] | [standalone]

#30647

I tried some options, and in the end i used Robots.txt, because it work better for me. I thought that it was a javascript problem, but i understand that is only a SEO problem.

I'm sorry for posting it here, should be in a SEO topic. 

Thanks to all who posted here.

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.javascript

csiph-web