Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.misc > #13396 > unrolled thread

DuckDuckGo: finally tracks me?

Back to article view | Back to comp.misc

Contents

  DuckDuckGo: finally tracks me? Ivan Shmakov <ivan@siamics.net> - 2017-03-31 07:13 +0000
    Re: DuckDuckGo: finally tracks me? Andy Burns <usenet@andyburns.uk> - 2017-03-31 08:33 +0100
      Re: DuckDuckGo: finally tracks me? Ivan Shmakov <ivan@siamics.net> - 2017-03-31 08:07 +0000
    Re: DuckDuckGo: finally tracks me? Kees Nuyt <k.nuyt@nospam.demon.nl> - 2017-03-31 18:28 +0200
    Re: DuckDuckGo: finally tracks me? Michael Forsythe <forsythe@example.com> - 2017-03-31 17:34 +0000
      Re: DuckDuckGo: finally tracks me? Ivan Shmakov <ivan@siamics.net> - 2017-03-31 18:33 +0000
    Re: DuckDuckGo: finally tracks me? not@telling.you.invalid (Computer Nerd Kev) - 2017-04-01 02:08 +0000
      Re: DuckDuckGo: finally tracks me? not@telling.you.invalid (Computer Nerd Kev) - 2017-04-01 02:17 +0000
      Re: DuckDuckGo: finally tracks me? Computer Nerd Kev <not@telling.you.invalid> - 2017-04-01 05:51 +0000
      Re: DuckDuckGo: finally tracks me? Ivan Shmakov <ivan@siamics.net> - 2017-04-05 05:57 +0000
        Re: DuckDuckGo: finally tracks me? not@telling.you.invalid (Computer Nerd Kev) - 2017-04-05 23:08 +0000
          Re: DuckDuckGo: finally tracks me? Ivan Shmakov <ivan@siamics.net> - 2017-04-06 17:25 +0000

#13396 — DuckDuckGo: finally tracks me?

	Contrary to virtually all the other major search engines out
	there, DuckDuckGo used to provide "direct" links to the pages
	found, like: http://example.com/.

	As it seems, it changed around January 12th, and now the links
	are redirected via their server instead, like [1]:

https://duckduckgo.com/l/?kh=-1&uddg=http%3A%2F%2Fexample.com%2F

	So, whenever I choose to follow the link, I do that by informing
	their server that I am, indeed, following that link, and only
	then the server kindly redirects me to the resource proper.

	Any idea on what may be going on?  (Are they gone evil at last?)

	FWIW, there doesn't seem to be anything relevant on this new
	behavior at their blog [2].

[1] https://duckduckgo.com/html/?q=example.com
[2] https://duck.co/blog

-- 
FSF associate member #7257  58F8 0F47 53F5 2EB2 F6A5  8916 3013 B6A0 230E 334A

[toc] | [next] | [standalone]

#13397

Ivan Shmakov wrote:

> https://duckduckgo.com/l/?kh=-1&uddg=http%3A%2F%2Fexample.com%2F

Maybe something like a greasemonkey script run after the page loads 
could search for links matching
https://duckduckgo.com/l/ and alter them to point direct to the part in 
the uddg parameter?

[toc] | [prev] | [next] | [standalone]

#13398

>>>>> Andy Burns <usenet@andyburns.uk> writes:
>>>>> Ivan Shmakov wrote:

 >> https://duckduckgo.com/l/?kh=-1&uddg=http%3A%2F%2Fexample.com%2F

 > Maybe something like a greasemonkey script run after the page loads
 > could search for links matching https://duckduckgo.com/l/ and alter
 > them to point direct to the part in the uddg parameter?

	Of course.  Although as I use Lynx for my primary browser,
	I'd need to come with something along the lines of a Perl
	"clean-up HTTP proxy" script instead.  (Or 'also'.)

	Still curious on why they've decided to break what was not
	needing to be fixed.  (Referer: issues aside.)

	(Makes me wonder if much of the "modern Web" could even be
	usable without Stylish and Greasemonkey.  And NoScript.)

-- 
FSF associate member #7257  58F8 0F47 53F5 2EB2 F6A5  8916 3013 B6A0 230E 334A

[toc] | [prev] | [next] | [standalone]

#13399

On Fri, 31 Mar 2017 07:13:23 +0000, Ivan Shmakov
<ivan@siamics.net> wrote:

>	Contrary to virtually all the other major search engines out
>	there, DuckDuckGo used to provide "direct" links to the pages
>	found, like: http://example.com/.
>
>	As it seems, it changed around January 12th, and now the links
>	are redirected via their server instead, like [1]:
>
> https://duckduckgo.com/l/?kh=-1&uddg=http%3A%2F%2Fexample.com%2F
>
>	So, whenever I choose to follow the link, I do that by informing
>	their server that I am, indeed, following that link, and only
>	then the server kindly redirects me to the resource proper.
>
>	Any idea on what may be going on?  (Are they gone evil at last?)

No idea. IXQuick https://www.ixquick.com/ still does the right
thing, except for the "ads by Google" above the search results.

>	FWIW, there doesn't seem to be anything relevant on this new
>	behavior at their blog [2].
>
> [1] https://duckduckgo.com/html/?q=example.com
> [2] https://duck.co/blog

-- 
Regards,
Kees Nuyt

[toc] | [prev] | [next] | [standalone]

#13400

Ivan Shmakov <ivan@siamics.net> wrote:
>         Contrary to virtually all the other major search engines out
>         there, DuckDuckGo used to provide "direct" links to the pages
>         found, like: http://example.com/.
> 
>         As it seems, it changed around January 12th, and now the links
>         are redirected via their server instead, like [1]:
> 
> https://duckduckgo.com/l/?kh=-1&uddg=http%3A%2F%2Fexample.com%2F
> 
>         So, whenever I choose to follow the link, I do that by informing
>         their server that I am, indeed, following that link, and only
>         then the server kindly redirects me to the resource proper.
> 
>         Any idea on what may be going on?  (Are they gone evil at last?)
> 
>         FWIW, there doesn't seem to be anything relevant on this new
>         behavior at their blog [2].
> 
> [1] https://duckduckgo.com/html/?q=example.com
> [2] https://duck.co/blog

For what it's worth, I don't see this when browsing normally - I
get direct links.

    $ curl -s https://duckduckgo.com/html -F "q=example.com"|grep uddg
    $ 

But I do see this if I GET the page.

    $ curl -s "https://duckduckgo.com/html?q=example.com"|grep uddg|wc -l
    108
    $ 

Also, somebody on stackexchange asked about this a few years ago[0],
so it's not new. The answer on stackexchange ("DDG is really helping
you by sanitizing referers") doesn't sit well with me - my web
browser doesn't send referers thankyouverymuch - but I think it
actually checks out; the referer can't include parameters sent over
POST, so there's no need for DDG to sanitize if I POSTed my query.

[0] http://webapps.stackexchange.com/questions/73312/are-duckduckgo-redirects-a-privacy-issue 

[toc] | [prev] | [next] | [standalone]

#13401

>>>>> Michael Forsythe <forsythe@example.com> writes:
>>>>> Ivan Shmakov <ivan@siamics.net> wrote:

 >> Contrary to virtually all the other major search engines out there,
 >> DuckDuckGo used to provide "direct" links to the pages found, like:
 >> http://example.com/.

 >> As it seems, it changed around January 12th, and now the links are
 >> redirected via their server instead, like [1]:

 >> https://duckduckgo.com/l/?kh=-1&uddg=http%3A%2F%2Fexample.com%2F

[...]

 > For what it's worth, I don't see this when browsing normally - I get
 > direct links.

 > $ curl -s https://duckduckgo.com/html -F "q=example.com"|grep uddg 
 > $ 

 > But I do see this if I GET the page.

 > $ curl -s "https://duckduckgo.com/html?q=example.com"|grep uddg|wc -l 
 > 108
 > $ 

	Indeed, that seems to be the case.  Thanks!

 > Also, somebody on stackexchange asked about this a few years ago[0],
 > so it's not new.  The answer on stackexchange ("DDG is really helping
 > you by sanitizing referers") doesn't sit well with me - my web
 > browser doesn't send referers thankyouverymuch

	FWIW, I've seen a site or two that pretty much require Referer:
	for at least some of the queries.

 > - but I think it actually checks out; the referer can't include
 > parameters sent over POST, so there's no need for DDG to sanitize if
 > I POSTed my query.

	Yes.  I think I may've seen such behavior for some User-Agent:
	values before.  So, I guess they've dropped whatever logic they
	had there and use redirects for all the GET searches instead.

 > [0] http://webapps.stackexchange.com/questions/73312/
 > are-duckduckgo-redirects-a-privacy-issue

-- 
FSF associate member #7257  np. Blue -- Fortissimo    8916 3013 B6A0 230E 334A

[toc] | [prev] | [next] | [standalone]

#13402

Ivan Shmakov <ivan@siamics.net> wrote:
>        Contrary to virtually all the other major search engines out
>        there, DuckDuckGo used to provide "direct" links to the pages
>        found, like: http://example.com/.
> 
>        As it seems, it changed around January 12th, and now the links
>        are redirected via their server instead, like [1]:
> 
> https://duckduckgo.com/l/?kh=-1&uddg=http%3A%2F%2Fexample.com%2F
> 
>        So, whenever I choose to follow the link, I do that by informing
>        their server that I am, indeed, following that link, and only
>        then the server kindly redirects me to the resource proper.
> 
>        Any idea on what may be going on?

I ran into this problem late last year. I always use the "lite" version
of the website ( https://duckduckgo.com/lite/ ) along with various
"parameters" in the search string configured in my web browsers. The
parameter "kd=-1" disables the redirects and gives you real direct
links:

" https://duckduckgo.com/lite/?kd=-1&q=[search query] "
" https://duckduckgo.com/html/kd=-1&q=[search query] "

I soon found this switch when they originally started referring links on
the "lite" site. It's rather ambiguously called "redirect" on the URL
parameters list ( https://duckduckgo.com/params ). I was very eager to
find a way to turn the function off because my most used web browser,
Dillo, didn't work with their redirect system (it used some, possibly
non-standard, script-free referrer tag) and I was either manually
editing links or giving up and going back to Google (whose redirects do
work in Dillo).

I sent a long, grumpy, message to their black hole... sorry "complaints
page", outlining all the issues with forcing URL redirects (the extra
connection also delayed every page load by a couple of seconds at my
internet speeds) and a week or so later the "kd=-1" flag magically
started working.

>  (Are they gone evil at last?)

Possibly, Yahoo seem to have got very involved lately.

>        FWIW, there doesn't seem to be anything relevant on this new
>        behavior at their blog [2].
> 
> [1] https://duckduckgo.com/html/?q=example.com
> [2] https://duck.co/blog

There was a post somewhere. They started doing it on the non-lite
pages a long time before I dicovered the problem on the "lite" site.

Ah, that's right. It was somewhere on their Reddit, err... page?
forum? domain? It's about the only time I've been to Reddit so I
don't know how it works, but it's there somewhere (if I find my
notes on all this, I'll post a link). They passed it off as a
security feature because it meant that referrers weren't passed
to sites visited from the search results - but like you I
consider the solution far more concerning than the problem.

-- 
__          __
#_ < |\| |< _#

[toc] | [prev] | [next] | [standalone]

#13403

Computer Nerd Kev <not@telling.you.invalid> wrote:
> 
> " https://duckduckgo.com/lite/?kd=-1&q=[search query] "
> " https://duckduckgo.com/html/kd=-1&q=[search query] "

Bah, I tested the top one but not the bottom one:
" https://duckduckgo.com/html/?kd=-1&q=[search query] "

-- 
__          __
#_ < |\| |< _#

[toc] | [prev] | [next] | [standalone]

#13404

Computer Nerd Kev <not@telling.you.invalid> wrote:
> Ivan Shmakov <ivan@siamics.net> wrote:
> 
>>        FWIW, there doesn't seem to be anything relevant on this new
>>        behavior at their blog [2].
>> 
>> [1] https://duckduckgo.com/html/?q=example.com
>> [2] https://duck.co/blog
> 
> There was a post somewhere. They started doing it on the non-lite
> pages a long time before I dicovered the problem on the "lite" site.
> 
> Ah, that's right. It was somewhere on their Reddit, err... page?
> forum? domain? It's about the only time I've been to Reddit so I
> don't know how it works, but it's there somewhere (if I find my
> notes on all this, I'll post a link). They passed it off as a
> security feature because it meant that referrers weren't passed
> to sites visited from the search results - but like you I
> consider the solution far more concerning than the problem.

https://www.reddit.com/r/duckduckgo/comments/5lbv5z/ddg_redirecting_bug/

Which links to:

https://duck.co/help/results/rduckduckgocom

-- 
__          __
#_ < |\| |< _#

[toc] | [prev] | [next] | [standalone]

#13409

>>>>> Computer Nerd Kev <not@telling.you.invalid> writes:
>>>>> Ivan Shmakov <ivan@siamics.net> wrote:

 >> Contrary to virtually all the other major search engines out there,
 >> DuckDuckGo used to provide "direct" links to the pages found, like:
 >> http://example.com/.

 >> As it seems, it changed around January 12th, and now the links are
 >> redirected via their server instead, like [1]:

 >> https://duckduckgo.com/l/?kh=-1&uddg=http%3A%2F%2Fexample.com%2F

 >> So, whenever I choose to follow the link, I do that by informing
 >> their server that I am, indeed, following that link, and only then
 >> the server kindly redirects me to the resource proper.

 >> Any idea on what may be going on?

 > I ran into this problem late last year.  I always use the "lite"
 > version of the website ( https://duckduckgo.com/lite/ )

	Makes me wonder how "lite" differs from "HTML".  (If anything,
	it doesn't seem to be covered in the help [2].)

[1] https://duckduckgo.com/html/?q=example.com
[2] https://duck.co/help/features/non-javascript

 > along with various "parameters" in the search string configured in my
 > web browsers.  The parameter "kd=-1" disables the redirects and gives
 > you real direct links:

 > " https://duckduckgo.com/lite/?kd=-1&q=[search query] "
 > " https://duckduckgo.com/html/?kd=-1&q=[search query] "

	Indeed, that does the trick.  Thanks!

 > I soon found this switch when they originally started referring links
 > on the "lite" site. It's rather ambiguously called "redirect" on the
 > URL parameters list ( https://duckduckgo.com/params ).

	They do not seem to advertise that page much, now do they?

 > I was very eager to find a way to turn the function off because my
 > most used web browser, Dillo, didn't work with their redirect system
 > (it used some, possibly non-standard, script-free referrer tag) and I
 > was either manually editing links or giving up and going back to
 > Google (whose redirects do work in Dillo).

	Actually, they use both a trivial <script /> and a "refresh"
	<meta /> on the redirect pages.  The latter is standard per
	HTML5 4.2.5 [3].

[3] http://www.w3.org/TR/html5/document-metadata.html#attr-meta-http-equiv-refresh

[...]

-- 
FSF associate member #7257  58F8 0F47 53F5 2EB2 F6A5  8916 3013 B6A0 230E 334A

[toc] | [prev] | [next] | [standalone]

#13413

Ivan Shmakov <ivan@siamics.net> wrote:
>>>>>> Computer Nerd Kev <not@telling.you.invalid> writes:
>>>>>> Ivan Shmakov <ivan@siamics.net> wrote:
> 
> >> Contrary to virtually all the other major search engines out there,
> >> DuckDuckGo used to provide "direct" links to the pages found, like:
> >> http://example.com/.
> 
> >> As it seems, it changed around January 12th, and now the links are
> >> redirected via their server instead, like [1]:
> 
> >> https://duckduckgo.com/l/?kh=-1&uddg=http%3A%2F%2Fexample.com%2F
> 
> >> So, whenever I choose to follow the link, I do that by informing
> >> their server that I am, indeed, following that link, and only then
> >> the server kindly redirects me to the resource proper.
> 
> >> Any idea on what may be going on?
> 
> > I ran into this problem late last year.  I always use the "lite"
> > version of the website ( https://duckduckgo.com/lite/ )
> 
>        Makes me wonder how "lite" differs from "HTML".  (If anything,
>        it doesn't seem to be covered in the help [2].)
> 
> [1] https://duckduckgo.com/html/?q=example.com
> [2] https://duck.co/help/features/non-javascript

A quick look at the page source seems to indicate that "/html/" uses
css, while "/lite/" puts all the results in a table. There may well
be other concents to compatibility as well. "/lite/" looks much nicer
in Dillo, and I use it in Firefox as well because I think I worked out
that it was quicker (it's smaller in any case).

> > along with various "parameters" in the search string configured in my
> > web browsers.  The parameter "kd=-1" disables the redirects and gives
> > you real direct links:
> 
> > " https://duckduckgo.com/lite/?kd=-1&q=[search query] "
> > " https://duckduckgo.com/html/?kd=-1&q=[search query] "
> 
>        Indeed, that does the trick.  Thanks!
> 
> > I soon found this switch when they originally started referring links
> > on the "lite" site. It's rather ambiguously called "redirect" on the
> > URL parameters list ( https://duckduckgo.com/params ).
> 
>        They do not seem to advertise that page much, now do they?

To be fair, advertising is one of the things I use a parameter to
turn off. :)

> > I was very eager to find a way to turn the function off because my
> > most used web browser, Dillo, didn't work with their redirect system
> > (it used some, possibly non-standard, script-free referrer tag) and I
> > was either manually editing links or giving up and going back to
> > Google (whose redirects do work in Dillo).
> 
>        Actually, they use both a trivial <script /> and a "refresh"
>        <meta /> on the redirect pages.  The latter is standard per
>        HTML5 4.2.5 [3].
> 
> [3] http://www.w3.org/TR/html5/document-metadata.html#attr-meta-http-equiv-refresh

I see, Dillo and myself haven't caught up with HTML5 yet.

-- 
__          __
#_ < |\| |< _#

[toc] | [prev] | [next] | [standalone]

#13419

>>>>> Computer Nerd Kev <not@telling.you.invalid> writes:
>>>>> Ivan Shmakov <ivan@siamics.net> wrote:
>>>>> Computer Nerd Kev <not@telling.you.invalid> writes:

[...]

 > A quick look at the page source seems to indicate that "/html/" uses
 > css, while "/lite/" puts all the results in a table.  There may well
 > be other concents to compatibility as well.  "/lite/" looks much
 > nicer in Dillo, and I use it in Firefox as well because I think I
 > worked out that it was quicker (it's smaller in any case).

	ACK, thanks.

[...]

 >>> I was very eager to find a way to turn the function off because my
 >>> most used web browser, Dillo, didn't work with their redirect
 >>> system (it used some, possibly non-standard, script-free referrer
 >>> tag) and I was either manually editing links or giving up and going
 >>> back to Google (whose redirects do work in Dillo).

 >> Actually, they use both a trivial <script /> and a "refresh"
 >> <meta /> on the redirect pages.  The latter is standard per HTML5
 >> 4.2.5 [3].

 >> [3] http://www.w3.org/TR/html5/document-metadata.html#attr-meta-http-equiv-refresh

 > I see, Dillo and myself haven't caught up with HTML5 yet.

	... Which reminds me that I should try to invest some effort
	into improving the Lynx HTML5 compatibility.  For one thing, it
	doesn't seem to properly handle the "subpage" <div />-in-a-<a />
	links used at https://duck.co/help.

	Though in this case, HTML5 only documented the existing practice
	of providing an equivalent to the non-standard HTTP Refresh:
	header via <meta http-equiv= />.

-- 
FSF associate member #7257  np. Pathfinder -- Kubbi   8916 3013 B6A0 230E 334A

[toc] | [prev] | [standalone]

Back to top | Article view | comp.misc

csiph-web