Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #55549 > unrolled thread

Use the Source Luke

Back to article view | Back to comp.lang.python

Contents

  Use the Source Luke Raymond Hettinger <python@rcn.com> - 2011-01-28 10:32 -0800
    Re: Use the Source Luke rantingrick <rantingrick@gmail.com> - 2011-01-31 12:45 -0800
    Re: Use the Source Luke rusi <rustompmody@gmail.com> - 2011-01-30 08:50 -0800
    Re: Use the Source Luke Raymond Hettinger <python@rcn.com> - 2011-01-29 12:23 -0800
    Re: Use the Source Luke Paul Rubin <no.email@nospam.invalid> - 2011-01-28 12:30 -0800
    Re: Use the Source Luke rusi <rustompmody@gmail.com> - 2011-02-04 08:58 -0800
    Re: Use the Source Luke Rui Maciel <rui.maciel@gmail.com> - 2011-01-28 20:49 +0000
    Re: Use the Source Luke rantingrick <rantingrick@gmail.com> - 2011-01-31 22:38 -0800
    Re: Use the Source Luke David Boddie <david@boddie.org.uk> - 2011-01-30 14:19 +0100
    Re: Use the Source Luke Giampaolo Rodolà <g.rodola@gmail.com> - 2011-01-28 21:32 +0100
    Re: Use the Source Luke rantingrick <rantingrick@gmail.com> - 2011-01-30 08:42 -0800
    Re: Use the Source Luke Emile van Sebille <emile@fenx.com> - 2011-01-31 14:07 -0800
    Re: Use the Source Luke Tim Wintle <tim.wintle@teamrubber.com> - 2011-01-30 14:47 +0000
      Re: Use the Source Luke Raymond Hettinger <python@rcn.com> - 2011-01-31 12:35 -0800
        Re: Use the Source Luke Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2011-01-31 22:39 +0000
          Re: Use the Source Luke rusi <rustompmody@gmail.com> - 2011-01-31 22:25 -0800
      Re: Use the Source Luke rusi <rustompmody@gmail.com> - 2011-02-04 08:34 -0800
        Re: Use the Source Luke "OKB (not okblacke)" <brenNOSPAMbarn@NObrenSPAMbarn.net> - 2011-02-04 19:11 +0000
          Re: Use the Source Luke rusi <rustompmody@gmail.com> - 2011-02-04 21:20 -0800
      Re: Use the Source Luke "OKB (not okblacke)" <brenNOSPAMbarn@NObrenSPAMbarn.net> - 2011-02-01 19:32 +0000
    Re: Use the Source Luke Jack Diederich <jackdied@gmail.com> - 2011-01-28 19:02 -0500
      Re: Use the Source Luke Daniel Fetchinson <fetchinson@googlemail.com> - 2011-02-05 12:46 +0100
      Re: Use the Source Luke Raymond Hettinger <python@rcn.com> - 2011-01-28 17:52 -0800
    Re: Use the Source Luke Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2011-01-30 04:21 +0000
      Re: Use the Source Luke rusi <rustompmody@gmail.com> - 2011-01-29 20:50 -0800
        Re: Use the Source Luke Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2011-01-30 08:53 +0000
    Re: Use the Source Luke Raymond Hettinger <python@rcn.com> - 2011-01-29 21:17 -0800
    Re: Use the Source Luke TP <wingusr@gmail.com> - 2011-01-29 03:22 -0800
    Re: Use the Source Luke Ben Finney <ben+python@benfinney.id.au> - 2011-01-29 10:10 +1100
      Re: Use the Source Luke rusi <rustompmody@gmail.com> - 2011-01-29 19:59 -0800
      Re: Use the Source Luke rusi <rustompmody@gmail.com> - 2011-01-28 23:50 -0800
        Re: Use the Source Luke Ben Finney <ben+python@benfinney.id.au> - 2011-01-30 08:22 +1100

Page 2 of 2 — ← Prev page 1 [2]

#55777

On Fri, Jan 28, 2011 at 1:32 PM, Raymond Hettinger <python@rcn.com> wrote:
> I hoping a new trend will start with dev's putting direct
> source code links in their documentation:
>
>  http://rhettinger.wordpress.com/2011/01/28/open-your-source-more/
>
> I'm looking for more examples of projects that routinely
> link their docs back into relavant sections of code.
> Have any of you all seen other examples besides
> the Go language docs and the Python docs?

I think you overestimate how common it used to be to carry around the
sourcecode for the software you use compared to now;  In the past it
wasn't even always possible - if the Sun cc compiler core dumps you
have no recourse to code.  Promoting the idea of doing it is good
because it /is/ a novel idea to many people.  Promoting the idea of
making it extremely easy via documentation links is good because it is
new as well.

Modern tools are making this easier than it used to be so your call
for making it easier still is well timed.  Github/bitbucket/launchpad
have combined the source with documentation; github especially because
the README on github is the canonical documentation and the source is
only one mouse click away.  ack-grep has changed my life.  Sure, I
could always do the same thing in the past with find+grep but ack-grep
makes it so easy (switches for language file types!) that I use it
much more;  I have "ag" aliased to "ack-grep --python" and I use it
all the f'ing time because it costs me near zero to do so.  Likewise I
have an alias "cdp" that "cd"s me into the directory where any given
python module lives.  "cdp collections" puts me straight into
"/usr/local/lib/python2.6" - again, it makes it so easy to look at
sourcecode that I do it all the time.  It is usually quicker to do
cdp/python/import module_name/help(module_name) than to look up the
docs.  Worst case the docstrings suck and I just read the code.

* Anecdote.  I was in a room with Tim Peters and has some questions
about the interface to code he wrote so I thought "Hey, I'll just ask
Tim!"  I asked him and he replied "I'm not sure what you're asking -
do you want me to read the code aloud to you?"  So I just went and
read it.

-Jack

[toc] | [prev] | [next] | [standalone]

#56013

> For the Python world though, there does seem
> to have been a change.  A decade ago in this newsgroup, there were
> frequent references to standard library source.  I don't see that
> much anymore.

Popularity has a price. A decade ago only hackers were exposed to
python who are happy to chat about the source code but these days
average computer users (my grandma) frequently come across python and
ask questions from a user perspective.

So I'd think your observation is correct that the ratio of source
related posts to non-source related posts on this list went down but
the reason is not because the total number of source related questions
decreased but because the total number of non-source related questions
increased.

Cheers,
Daniel


-- 
Psss, psss, put it down! - http://www.cafepress.com/putitdown

[toc] | [prev] | [next] | [standalone]

#56046

[Jack Diedrich]
> I think you overestimate how common it used to be to carry around the
> sourcecode for the software you use compared to now;  In the past it
> wasn't even always possible - if the Sun cc compiler core dumps you
> have no recourse to code.

You're right of course.  For the Python world though, there does seem
to have been a change.  A decade ago in this newsgroup, there were
frequent references to standard library source.  I don't see that
much anymore.


> Promoting the idea of doing it is good
> because it /is/ a novel idea to many people.  Promoting the idea of
> making it extremely easy via documentation links is good because it is
> new as well.

Judging from the comments so far, it looks like everyone here agrees.

> Modern tools are making this easier than it used to be so your call
> for making it easier still is well timed.  Github/bitbucket/launchpad
> have combined the source with documentation;

Do they typically feature multiple links from documentation specifics
to corresponding code specifics?

Part of my thesis is that it is not enough to make docs and source
available, they need to be linked in a way that helps people
answer specific problems without having to invest a week in learning
the gestalt of a foreign code base.


> Worst case the docstrings suck and I just read the code.

That's a good habit to have.  I find that my willingness to do it
varies across projects -- I'm happy to look at the Mercurial source
but could never being myself to look at the innards of Git or CouchDB.


> * Anecdote.  I was in a room with Tim Peters and has some questions
> about the interface to code he wrote so I thought "Hey, I'll just ask
> Tim!"  I asked him and he replied "I'm not sure what you're asking -
> do you want me to read the code aloud to you?"  So I just went and
> read it.

Thanks for the anecdote.  I love that story :-)
Uncle Timmy's style is both clever and pointed.


Raymond

[toc] | [prev] | [next] | [standalone]

#55788

On Sat, 29 Jan 2011 19:59:33 -0800, rusi wrote:

> On Jan 30, 2:22 am, Ben Finney <ben+pyt...@benfinney.id.au> wrote:
>>
>> The “problem”, which I don't consider to be a problem per se, is one of
>> OS-wide policy, not “installers”. The policy is a matter of tradeoffs
>> across the system, and isn't “tucking the code away in a dark corner”.
> 
> Earlier mail:
> 
>> If you want to blame anything for this (though I think it’s inaccurate
>> to frame it as a problem), the correct target of your accusation is the
>> fact that a filesystem path is the identifier for these modules that
>> will be used by programs to find them.
> 
> I think this is a fairly accurate description of (one aspect of) the
> problem.
> If you dont see it as a problem how do you explain that google can
> search the World Wide Web better than we can search our individual hard
> disks?

I fail to see any connection between the location that operating systems 
store files, and the ability of Google to index publicly available 
websites. It sounds to me like the equivalent of "If you don't think 
Python programmers are a problem, how do you explain that it takes me 45 
minutes to drive to work in the morning but nearly an hour to drive home 
in the evening?"

Google has approximately one million servers indexing the web, and 
they're willing to use hundreds of terabytes of disk space to store the 
indexes. My desktop is *one* PC with little free space, and I'd rather 
trade off time for storage, so I don't keep any indexes of file content 
on my system. If I *wanted* to index my files, I could do so, although in 
fairness I'm not aware of any Linux tools which do this -- I know of 
`locate`, which indexes file *names* but not content, and `grep`, which 
searches file content but doesn't index what it finds.

On Windows and Mac, though, I believe there are standard utilities which 
will index file content if you allow them.

So... Google can search the web better than we can search our local hard 
drives because Google has invested tens or hundreds of millions into 
building a world-class search engine, and we haven't.


-- 
Steven

[toc] | [prev] | [next] | [standalone]

#55891

On Jan 30, 9:21 am, Steven D'Aprano <steve
+comp.lang.pyt...@pearwood.info> wrote:
>
> > I think this is a fairly accurate description of (one aspect of) the
> > problem.
> > If you dont see it as a problem how do you explain that google can
> > search the World Wide Web better than we can search our individual hard
> > disks?
>
> I fail to see any connection between the location that operating systems
> store files, and the ability of Google to index publicly available
> websites.

http://en.wikipedia.org/wiki/Content-addressable_storage#Content-addressed_vs._Location-addressed

[toc] | [prev] | [next] | [standalone]

#56068

On Sat, 29 Jan 2011 20:50:20 -0800, rusi wrote:

> On Jan 30, 9:21 am, Steven D'Aprano <steve
> +comp.lang.pyt...@pearwood.info> wrote:
>>
>> > I think this is a fairly accurate description of (one aspect of) the
>> > problem.
>> > If you dont see it as a problem how do you explain that google can
>> > search the World Wide Web better than we can search our individual
>> > hard disks?
>>
>> I fail to see any connection between the location that operating
>> systems store files, and the ability of Google to index publicly
>> available websites.
> 
> http://en.wikipedia.org/wiki/Content-addressable_storage#Content-
addressed_vs._Location-addressed


Nope, sorry, doesn't help. Both local files on your hard drive, and most 
remote websites on the Internet, are location addressed. Google indexes 
the content, but they don't provide content-addresses. Indeed, they 
*can't* do so (except, possibly, for content they control such as Google 
Books), since they can't prevent content owners from modifying either the 
location address or the content. And as I've mentioned, there are desktop 
utilities that index content for Windows and Macintosh. In fact, Google 
themselves offer a desktop app that does just that:

http://desktop.google.com/features.html



-- 
Steven

[toc] | [prev] | [next] | [standalone]

#55790

On Jan 28, 3:10 pm, Ben Finney <ben+pyt...@benfinney.id.au> wrote:
> Raymond Hettinger <pyt...@rcn.com> writes:
> > The rest of the blame lies with installers. They all treat
> > human-readable scripts like they were binaries and tuck the code away
> > in a dark corner.
>
> That’s hardly a “blame” of installers. The modules are placed in such
> locations because they need to be accessible in a hierarchy at a
> location that is known to not conflict with anything else, and be
> predictable for the Python interpreter on the system.

Sure.  Installers are just installing where they're supposed to.
And good people have given a lot of thought to the preferred
target directories.  I'm just observing that the source files
are often ending-up out-of-sight and out-of-mind so that fewer
users ever see the source.

It's not deep a problem -- it would only take a symlink to
provide quick access.

My thesis is that we can do even better than that by adding
direct links from the docs to the relevant code with nice
syntax highlighting.


Raymond


P.S.  Making it easy to get to relevant source is only half of
the solution.  The remaining problem is cultural.  Perhaps every
project should have a recommended reading list.

As a start, I think the following are instructive and make for a good
read:

http://svn.python.org/view/python/branches/py3k/Lib/ftplib.py?view=markup
http://svn.python.org/view/python/branches/py3k/Lib/heapq.py?view=markup
http://svn.python.org/view/python/branches/py3k/Lib/collections.py?view=markup
http://svn.python.org/view/python/branches/py3k/Lib/queue.py?view=markup
http://svn.python.org/view/python/branches/py3k/Lib/functools.py?view=markup

[toc] | [prev] | [next] | [standalone]

#55806

On Fri, Jan 28, 2011 at 10:32 AM, Raymond Hettinger <python@rcn.com> wrote:
> I hoping a new trend will start with dev's putting direct
> source code links in their documentation:
>
>  http://rhettinger.wordpress.com/2011/01/28/open-your-source-more/
>
> I'm looking for more examples of projects that routinely
> link their docs back into relavant sections of code.
> Have any of you all seen other examples besides
> the Go language docs and the Python docs?
>
>
> Raymond
> --
> http://mail.python.org/mailman/listinfo/python-list
>

The Sphinx Python Documentation Generator
(http://sphinx.pocoo.org/index.html), used for documenting lots of
things other than Python, has an extension called "sphinx.ext.viewcode
– Add links to highlighted source code"
(http://sphinx.pocoo.org/ext/viewcode.html).

You can see my use of it here on a small PyQt project:
http://tpgit.github.com/MDIImageViewer/imageviewer.html. You can even
view the Sphinx documentation "code" by clicking the "Show Source"
link on the left.

-- TP

[toc] | [prev] | [next] | [standalone]

#55838

Raymond Hettinger <python@rcn.com> writes:

> I hoping a new trend will start with dev's putting direct
> source code links in their documentation:
>
>  http://rhettinger.wordpress.com/2011/01/28/open-your-source-more/

That's a good article overall.

I have a quibble with the framing:

> The rest of the blame lies with installers. They all treat
> human-readable scripts like they were binaries and tuck the code away
> in a dark corner.

That’s hardly a “blame” of installers. The modules are placed in such
locations because they need to be accessible in a hierarchy at a
location that is known to not conflict with anything else, and be
predictable for the Python interpreter on the system.

If you want to blame anything for this (though I think it’s inaccurate
to frame it as a problem), the correct target of your accusation is the
fact that a filesystem path is the identifier for these modules that
will be used by programs to find them.


As for reading the source and making it more available to programmers,
yes, I agree wholeheartedly. Encouraging the routine reading of other
projects’s source code is a good thing, and thank you for beating the
drum.

-- 
 \       “I distrust those people who know so well what God wants them |
  `\    to do to their fellows, because it always coincides with their |
_o__)                      own desires.” —Susan Brownell Anthony, 1896 |
Ben Finney

[toc] | [prev] | [next] | [standalone]

#56059

On Jan 30, 2:22 am, Ben Finney <ben+pyt...@benfinney.id.au> wrote:
>
> The “problem”, which I don't consider to be a problem per se, is one of
> OS-wide policy, not “installers”. The policy is a matter of tradeoffs
> across the system, and isn't “tucking the code away in a dark corner”.

Earlier mail:

> If you want to blame anything for this (though I think it’s inaccurate
> to frame it as a problem), the correct target of your accusation is the
> fact that a filesystem path is the identifier for these modules that
> will be used by programs to find them.

I think this is a fairly accurate description of (one aspect of) the
problem.
If you dont see it as a problem how do you explain that google can
search the World Wide Web better than we can search our individual
hard disks?

[toc] | [prev] | [next] | [standalone]

#56085

On Jan 29, 4:10 am, Ben Finney <ben+pyt...@benfinney.id.au> wrote:

> Note that Raymond is speaking specifically in the context of free
> software, where the license is by definition permitting free
> redistribution of the source code.

It is an obvious necessary condition that for code to be opened it
should be open (source).
However the necessary condition is not sufficient.

>
> I have a quibble with the framing:
>
> > The rest of the blame lies with installers. They all treat
> > human-readable scripts like they were binaries and tuck the code away
> > in a dark corner.

Consider this example:
The emacs source if compiled from source will give you help for each
lisp or even builtin (C) function out of the box from inside emacs.
However if you get the emacs package from debian/ubuntu you will get
neither unless you install el files -- which is fine -- just install
the el package.  About the C sources?  I dont believe that one can get
that linkage from within apt; one just hast to compile oneself.  (I
would be happy to be surprised on this).

Isn't this an instance of the problem that Raymond is talking of?

[Personal note:  Ive been a python user and teacher for nearly 10
years and would eagerly welcome more easy code-open-ness]

[toc] | [prev] | [next] | [standalone]

#56092

rusi <rustompmody@gmail.com> writes:

> On Jan 29, 4:10 am, Ben Finney <ben+pyt...@benfinney.id.au> wrote:
> > I have a quibble with the framing:
> >
> > > The rest of the blame lies with installers. They all treat
> > > human-readable scripts like they were binaries and tuck the code
> > > away in a dark corner.
>
> Consider this example:
> The emacs source if compiled from source will give you help for each
> lisp or even builtin (C) function out of the box from inside emacs.
> However if you get the emacs package from debian/ubuntu you will get
> neither unless you install el files -- which is fine -- just install
> the el package. […]

That's an example of the point I made in what followed in my message you
quoted. The description can be interpreted as accurate, but it's framed
poorly.

> Isn't this an instance of the problem that Raymond is talking of?

The “problem”, which I don't consider to be a problem per se, is one of
OS-wide policy, not “installers”. The policy is a matter of tradeoffs
across the system, and isn't “tucking the code away in a dark corner”.

> [Personal note:  Ive been a python user and teacher for nearly 10
> years and would eagerly welcome more easy code-open-ness]

Agreed.

-- 
 \     “When people believe that they have absolute knowledge, with no |
  `\     test in reality, this [the Auschwitz crematorium] is how they |
_o__)             behave.” —Jacob Bronowski, _The Ascent of Man_, 1973 |
Ben Finney

[toc] | [prev] | [standalone]

Page 2 of 2 — ← Prev page 1 [2]

Back to top | Article view | comp.lang.python

csiph-web