Re: [OT] Free software versus software idea patents

References	(13 earlier) <cArop.17646$sS4.7406@newsfe11.iad> <mailman.219.1302542844.9059.python-list@python.org> <sRUop.2627$7N3.197@newsfe10.iad> <mailman.279.1302637061.9059.python-list@python.org> <D7wpp.14226$sy5.2295@newsfe22.iad>
Date	2011-04-14 19:15 +1000
Subject	Re: [OT] Free software versus software idea patents
From	Chris Angelico <rosuav@gmail.com>
Newsgroups	comp.lang.python
Message-ID	<mailman.353.1302772509.9059.python-list@python.org> (permalink)

Show all headers | View raw

On Thu, Apr 14, 2011 at 4:04 PM, harrismh777 <harrismh777@charter.net> wrote:
>
>    How many web crawlers have you built? Are there any web programmers out
> there who need a web bot to hit multiple sites zillions of times a month
> from different places on earth to 'up' the number of hits for economic
> reasons? I've seen my share of this.

A well-behaved spider will (a) have a UA that identifies itself (as a
bot, and preferably as itself - eg "GoogleBot", etc - some even go so
far as to include a URL for more info), and (b) start by fetching
/robots.txt before they go any further. Servers can recognize
properly-built crawlers. And improperly-built crawlers, deliberately
trying to hammer a server to lie about browser stats? Seriously, do
you think people actually care THAT much?

>    How mamy times have you altered the identity of your web browser so that
> the web site would 'work'? You know, stupid messages from the server that
> say, "We only support IE 6+, upgrade your browser...",  so you tell it
> you're using IE 6 and, well no problem.

Yep. Which means that the figures will always be skewed toward IE a
bit. But it's a lot less than you might think; most people don't leave
UA switchers active all the time, and the number of web sites that
require them is dropping. It's true that UA switching will tip toward
IE (I've never seen a site where you have to pretend to be Google
Chrome), but the epidemology is, I believe, not all that high.

>    Web site data is bogus. It assumes even distributions... it assumes even
> usage of the site from all surfers, it assumes no web crawlers and no bots,
> it assumes no browser identity tampering, and it assumes that there aren't
> those who for economic reasons are not inflating the numbers deliberately
> (no, really??) from world-owned bot farms.

Even distributions of what?

1) Assuming nothing, it merely gives data. About one site. That's why
overall "browser marketshare" stats have to be done by averaging
multiple sites.

2) Web crawlers - see above. If you've ever looked at AWStats or
Webalizer or *insert stats engine here*, you'll have seen that it will
identify them. AWStats goes a bit further and will identify "viewed
traffic" and "not viewed traffic" even if it's unable to identify the
specific bot.

3) Yes, it assumes no UA switchers, obviously. It's just based on
headers. But I reckon you could easily identify someone who's using a
switcher, based on other headers - for instance, I doubt very much
that IE6 will send "Accept-Encoding: gzip,deflate,sdch" (which my
Chrome does).

4) Assumes people aren't deliberately fiddling the figures. Yeah, that
would be correct. We're in the realm of conspiracy theories here...
does anyone seriously think that browser stats are THAT important that
they'd go to multiple web servers with deceitful hits? Not forgetting
that they'd have to mix up the IPs, make plausible "browsing sessions"
(with referers and image retrieval and so on), vary the date/times,
etc, etc, etc, etc... and generate enough hits to make a reasonable
dent in the figures.

>    There is no reliable way to measure free software usage. But, there sure
> is a lot of posturing going on in the market place ...  wonder why?

Sure, and there's no reliable way to measure non-free software usage
either. What's the difference? You could count sales of Microsoft
Office, and you could count downloads of Open Office. Neither is any
more accurate than the other; although I think the 24-hour figures for
Firefox 4 / IE 9 downloads are fairly indicative, since people can't
get them off their respective OS install CDs. And this isn't
restricted to electronica either. Which is more popular, Coca-Cola or
Pepsi? Do more people vote Liberal or Labour, Republican or Democrat,
Whig or Tory?

Statisticking is a huge science. Most of it involves figuring out
what's important - anyone can get data, but getting useful information
out of the data takes some work.

Chris Angelico

Back to comp.lang.python | Previous | Next — Previous in thread | Next in thread | Find similar

Thread

Re: Python benefits over Cobra harrismh777 <harrismh777@charter.net> - 2011-04-06 03:03 -0500
  Re: Python benefits over Cobra Chris Angelico <rosuav@gmail.com> - 2011-04-06 18:26 +1000
    Re: Python benefits over Cobra Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2011-04-06 12:29 +0000
      Re: Python benefits over Cobra Chris Angelico <rosuav@gmail.com> - 2011-04-06 23:06 +1000
      [OT] Free software versus software idea patents (was: Python benefits over Cobra) Ben Finney <ben+python@benfinney.id.au> - 2011-04-07 07:50 +1000
        Re: [OT] Free software versus software idea patents harrismh777 <harrismh777@charter.net> - 2011-04-07 00:03 -0500
          Re: [OT] Free software versus software idea patents Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2011-04-07 07:19 +0000
            Re: [OT] Free software versus software idea patents harrismh777 <harrismh777@charter.net> - 2011-04-07 11:33 -0500
              Re: [OT] Free software versus software idea patents Mel <mwilson@the-wire.com> - 2011-04-07 17:15 +0000
              Re: [OT] Free software versus software idea patents Chris Angelico <rosuav@gmail.com> - 2011-04-08 05:55 +1000
                Re: [OT] Free software versus software idea patents harrismh777 <harrismh777@charter.net> - 2011-04-10 21:49 -0500
                Re: [OT] Free software versus software idea patents geremy condra <debatem1@gmail.com> - 2011-04-11 00:53 -0700
                Re: Free software versus software idea patents rusi <rustompmody@gmail.com> - 2011-04-11 01:36 -0700
                Re: Free software versus software idea patents harrismh777 <harrismh777@charter.net> - 2011-04-12 03:53 -0500
                Re: Free software versus software idea patents Terry Reedy <tjreedy@udel.edu> - 2011-04-12 13:32 -0400
                Re: Free software versus software idea patents Dan Stromberg <drsalists@gmail.com> - 2011-04-12 11:44 -0700
                Re: Free software versus software idea patents Terry Reedy <tjreedy@udel.edu> - 2011-04-12 15:39 -0400
                Re: [OT] Free software versus software idea patents Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2011-04-11 09:10 +0000
                Re: [OT] Free software versus software idea patents geremy condra <debatem1@gmail.com> - 2011-04-11 11:17 -0700
                Re: [OT] Free software versus software idea patents Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2011-04-11 22:28 +0000
                Re: [OT] Free software versus software idea patents geremy condra <debatem1@gmail.com> - 2011-04-11 15:55 -0700
                Re: [even more OT than before] Arithmetic [was Free software versus software idea patents] Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2011-04-12 11:34 +0000
                Re: [even more OT than before] Arithmetic [was Free software versus software idea patents] geremy condra <debatem1@gmail.com> - 2011-04-12 11:11 -0700
                Re: [OT] Free software versus software idea patents Ben Finney <ben+python@benfinney.id.au> - 2011-04-12 10:02 +1000
                Re: [OT] Free software versus software idea patents harrismh777 <harrismh777@charter.net> - 2011-04-12 03:15 -0500
                Re: [OT] Free software versus software idea patents geremy condra <debatem1@gmail.com> - 2011-04-12 09:54 -0700
                Re: [OT] Free software versus software idea patents harrismh777 <harrismh777@charter.net> - 2011-04-14 00:35 -0500
                Re: [OT] Free software versus software idea patents geremy condra <debatem1@gmail.com> - 2011-04-13 23:36 -0700
                Re: [OT] Free software versus software idea patents harrismh777 <harrismh777@charter.net> - 2011-04-14 02:22 -0500
                Re: [OT] Free software versus software idea patents geremy condra <debatem1@gmail.com> - 2011-04-14 09:51 -0700
                Re: [OT] Free software versus software idea patents harrismh777 <harrismh777@charter.net> - 2011-04-16 00:21 -0500
                Re: [OT] Free software versus software idea patents geremy condra <debatem1@gmail.com> - 2011-04-15 23:46 -0700
                Re: [OT] Free software versus software idea patents Algis Kabaila <akabaila@pcug.org.au> - 2011-04-16 21:01 +1000
                Re: [OT] Free software versus software idea patents Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2011-04-16 23:36 +0000
                Re: [OT] Free software versus software idea patents Chris Angelico <rosuav@gmail.com> - 2011-04-17 10:31 +1000
                Re: [OT] Free software versus software idea patents geremy condra <debatem1@gmail.com> - 2011-04-16 18:35 -0700
                Re: [OT] Free software versus software idea patents harrismh777 <harrismh777@charter.net> - 2011-04-18 01:29 -0500
                Re: [OT] Free software versus software idea patents geremy condra <debatem1@gmail.com> - 2011-04-18 00:34 -0700
                Re: [OT] Free software versus software idea patents Terry Reedy <tjreedy@udel.edu> - 2011-04-12 13:43 -0400
                Re: [OT] Free software versus software idea patents Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2011-04-12 23:56 +0000
                Re: [OT] Free software versus software idea patents geremy condra <debatem1@gmail.com> - 2011-04-12 17:05 -0700
        Re: [OT] Free software versus software idea patents (was: Python benefits over Cobra) Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2011-04-07 07:39 +0000
          Re: [OT] Free software versus software idea patents (was: Python benefits over Cobra) Chris Angelico <rosuav@gmail.com> - 2011-04-07 18:17 +1000
            Re: Free software versus software idea patents (was: Python benefits over Cobra) flebber <flebber.crue@gmail.com> - 2011-04-07 03:51 -0700
          Re: [OT] Free software versus software idea patents harrismh777 <harrismh777@charter.net> - 2011-04-07 10:31 -0500
            Re: [OT] Free software versus software idea patents Benjamin Kaplan <benjamin.kaplan@case.edu> - 2011-04-07 11:50 -0400
            Re: [OT] Free software versus software idea patents Adam Tauno Williams <awilliam@whitemice.org> - 2011-04-07 13:55 -0400
              Re: [OT] Free software versus software idea patents harrismh777 <harrismh777@charter.net> - 2011-04-07 14:37 -0500
                Re: [OT] Free software versus software idea patents Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2011-04-08 01:30 +0000
                Re: [OT] Free software versus software idea patents harrismh777 <harrismh777@charter.net> - 2011-04-08 01:37 -0500
                Re: [OT] Free software versus software idea patents Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2011-04-09 23:55 +0000
                Re: [OT] Free software versus software idea patents harrismh777 <harrismh777@charter.net> - 2011-04-10 19:04 -0500
                Re: [OT] Free software versus software idea patents Chris Angelico <rosuav@gmail.com> - 2011-04-11 10:18 +1000
                Re: [OT] Free software versus software idea patents harrismh777 <harrismh777@charter.net> - 2011-04-10 20:48 -0500
                Re: [OT] Free software versus software idea patents Westley Martínez <anikom15@gmail.com> - 2011-04-10 21:01 -0700
                Re: [OT] Free software versus software idea patents Ian Kelly <ian.g.kelly@gmail.com> - 2011-04-11 11:26 -0600
                Re: [OT] Free software versus software idea patents harrismh777 <harrismh777@charter.net> - 2011-04-12 04:22 -0500
                Re: [OT] Free software versus software idea patents Westley Martínez <anikom15@gmail.com> - 2011-04-12 06:59 -0700
                Re: Free software versus software idea patents CM <cmpython@gmail.com> - 2011-04-12 10:48 -0700
                Re: Free software versus software idea patents geremy condra <debatem1@gmail.com> - 2011-04-12 11:13 -0700
                Re: Free software versus software idea patents Westley Martínez <anikom15@gmail.com> - 2011-04-12 15:56 -0700
                Re: Free software versus software idea patents harrismh777 <harrismh777@charter.net> - 2011-04-14 00:50 -0500
                Re: Free software versus software idea patents geremy condra <debatem1@gmail.com> - 2011-04-13 23:38 -0700
                Re: Free software versus software idea patents harrismh777 <harrismh777@charter.net> - 2011-04-14 02:29 -0500
                Re: Free software versus software idea patents Ethan Furman <ethan@stoneleaf.us> - 2011-04-14 08:42 -0700
                Re: Free software versus software idea patents CM <cmpython@gmail.com> - 2011-04-14 09:48 -0700
                Re: Free software versus software idea patents harrismh777 <harrismh777@charter.net> - 2011-04-15 23:27 -0500
                Re: [OT] Free software versus software idea patents Ian Kelly <ian.g.kelly@gmail.com> - 2011-04-12 13:37 -0600
                Re: [OT] Free software versus software idea patents Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2011-04-12 23:45 +0000
                Re: [OT] Free software versus software idea patents harrismh777 <harrismh777@charter.net> - 2011-04-14 01:04 -0500
                Re: [OT] Free software versus software idea patents Chris Angelico <rosuav@gmail.com> - 2011-04-14 19:15 +1000
                Re: [OT] Free software versus software idea patents Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2011-04-14 14:02 +0000
                Re: [OT] Free software versus software idea patents Chris Angelico <rosuav@gmail.com> - 2011-04-15 00:11 +1000
                Re: [OT] Free software versus software idea patents Westley Martínez <anikom15@gmail.com> - 2011-04-14 13:46 -0700
                Re: [OT] Free software versus software idea patents Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2011-04-15 02:21 +0000
                Re: [OT] Free software versus software idea patents Roy Smith <roy@panix.com> - 2011-04-14 22:52 -0400
                Re: [OT] Free software versus software idea patents Westley Martínez <anikom15@gmail.com> - 2011-04-14 13:50 -0700
                Re: [OT] Free software versus software idea patents Martin Gregorie <martin@address-in-sig.invalid> - 2011-04-14 21:36 +0000
                Re: [OT] Free software versus software idea patents Chris Angelico <rosuav@gmail.com> - 2011-04-15 08:01 +1000
                Re: [OT] Free software versus software idea patents Martin Gregorie <martin@address-in-sig.invalid> - 2011-04-14 22:13 +0000
                Re: [OT] Free software versus software idea patents Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2011-04-15 01:51 +0000
                Re: [OT] Free software versus software idea patents Westley Martínez <anikom15@gmail.com> - 2011-04-14 15:23 -0700
                Re: [OT] Free software versus software idea patents Martin Gregorie <martin@address-in-sig.invalid> - 2011-04-14 22:55 +0000
                Re: [OT] Free software versus software idea patents Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2011-04-15 02:09 +0000
                Re: [OT] Free software versus software idea patents Chris Angelico <rosuav@gmail.com> - 2011-04-15 12:31 +1000
                Re: [OT] Free software versus software idea patents Roy Smith <roy@panix.com> - 2011-04-14 22:43 -0400
                Re: [OT] Free software versus software idea patents Ethan Furman <ethan@stoneleaf.us> - 2011-04-14 14:20 -0700
                Re: [OT] Free software versus software idea patents Dan Stromberg <drsalists@gmail.com> - 2011-04-15 02:28 -0700
                Re: [OT] Free software versus software idea patents Ian Kelly <ian.g.kelly@gmail.com> - 2011-04-14 10:49 -0600
                Re: [OT] Free software versus software idea patents Dan Stromberg <drsalists@gmail.com> - 2011-04-12 13:51 -0700
                Re: [OT] Free software versus software idea patents Ian Kelly <ian.g.kelly@gmail.com> - 2011-04-12 15:05 -0600
                Re: [OT] Free software versus software idea patents Chris Angelico <rosuav@gmail.com> - 2011-04-12 03:31 +1000
                Re: [OT] Free software versus software idea patents harrismh777 <harrismh777@charter.net> - 2011-04-08 01:41 -0500
                Re: [OT] Free software versus software idea patents Westley Martínez <anikom15@gmail.com> - 2011-04-08 07:14 -0700
                Re: [OT] Free software versus software idea patents Ethan Furman <ethan@stoneleaf.us> - 2011-04-08 09:10 -0700
                Re: [OT] Free software versus software idea patents Bob Martin <bob.martin@excite.com> - 2011-04-09 07:45 +0100
          Re: [OT] Free software versus software idea patents (was: Python benefits over Cobra) Ross Ridge <rridge@csclub.uwaterloo.ca> - 2011-04-07 14:18 -0400
          Re: [OT] Free software versus software idea patents Ben Finney <ben+python@benfinney.id.au> - 2011-04-10 11:53 +1000
          Re: Free software versus software idea patents (was: Python benefits over Cobra) sturlamolden <sturlamolden@yahoo.no> - 2011-04-12 09:04 -0700
    Re: Python benefits over Cobra harrismh777 <harrismh777@charter.net> - 2011-04-07 00:25 -0500
      Re: Python benefits over Cobra Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2011-04-07 06:43 +0000

csiph-web