Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #15378

Re: A Python script to put CTAN into git (from DVDs)

Path csiph.com!x330-a1.tempe.blueboxinc.net!usenet.pasdenom.info!gegeweb.org!de-l.enfer-du-nord.net!feeder2.enfer-du-nord.net!newsfeed.eweka.nl!eweka.nl!feeder3.eweka.nl!newsfeed.xs4all.nl!newsfeed6.news.xs4all.nl!xs4all!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail
Return-Path <jnareb@gmail.com>
X-Original-To python-list@python.org
Delivered-To python-list@mail.python.org
X-Spam-Status OK 0.012
X-Spam-Evidence '*H*': 0.98; '*S*': 0.00; 'python)': 0.05; 'subject:Python': 0.06; 'git': 0.07; 'url:bitbucket': 0.07; 'versions.': 0.07; '[1]:': 0.09; 'of)': 0.09; 'url:36': 0.09; 'url:github': 0.09; 'things.': 0.13; 'cc:addr:python-list': 0.15; '(set': 0.16; 'mercurial.': 0.16; 'meant': 0.16; 'archive': 0.21; 'file,': 0.21; 'tool,': 0.23; 'header:In-Reply-To:1': 0.23; 'writes:': 0.25; '(in': 0.26; 'fine': 0.26; '"the': 0.26; "i'm": 0.26; 'putting': 0.27; 'martin': 0.28; 'import': 0.28; 'script': 0.28; "he's": 0.28; "skip:' 30": 0.28; 'cc:addr:python.org': 0.29; 'keeps': 0.30; 'thanks': 0.31; 'received:209.85.161.46': 0.31; 'received:mail-fx0-f46.google.com': 0.31; 'version': 0.31; 'earlier': 0.32; 'quite': 0.32; "won't": 0.32; 'there': 0.33; 'header:User-Agent:1': 0.33; 'done': 0.33; 'checking': 0.34; 'probably': 0.34; 'here,': 0.34; 'similar': 0.35; 'received:209.85.161': 0.35; 'rather': 0.35; 'cc:2**1': 0.36; 'skip:" 10': 0.36; 'jonathan': 0.36; 'hosting': 0.36; 'but': 0.37; 'using': 0.37; 'received:google.com': 0.38; 'received:209.85': 0.38; 'url:org': 0.38; "i'd": 0.39; 'subject: (': 0.39; 'subject:: ': 0.39; 'might': 0.39; 'files': 0.40; 'received:209': 0.40; 'user': 0.40; 'per': 0.61; 'hope': 0.61; 'published': 0.62; 'forward': 0.63; 'direction.': 0.64; 'p.s.': 0.66; 'collection': 0.73; 'message-id:@localhost.localdomain': 0.78; 'snapshots': 0.84; 'subject:put': 0.84; 'tex': 0.84; '100,000': 0.93
DKIM-Signature v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=x-authentication-warning:subject:references:to:cc:from:user-agent :date:in-reply-to:message-id:lines:mime-version:content-type :content-transfer-encoding; bh=7w8BKmBv3yC11NWAj3jN/bG0reUkHy1mrnk7mNp+TyY=; b=ihY0jy0R2GOYC2OfJaJZOcaaLMJP3O1dnX+LlZmg20sRi2JtheU7weeiqIeJXuZtZV QGcrU8oNVlXhVFXcxZd36b89Blj2TyvlX7dNvdq/2+FpRHKo09ylaWp4ju9w1ncaPBQ2 bYT2rofcuXYzsgrdrYrMy6CNEmNP5UNbdVpUA=
X-Authentication-Warning localhost.localdomain: jnareb set sender to jnareb@gmail.com using -f
Subject Re: A Python script to put CTAN into git (from DVDs)
References <4EB6A522.3020909@pytex.org>
To Jonathan Fine <jfine@pytex.org>
From Jakub Narebski <jnareb@gmail.com>
User-Agent Gnus/5.09 (Gnus v5.9.0) Emacs/21.4
Date Sun, 06 Nov 2011 08:42:23 -0800 (PST)
In-Reply-To <4EB6A522.3020909@pytex.org>
MIME-Version 1.0
Content-Type text/plain; charset=iso-8859-4
Content-Transfer-Encoding quoted-printable
Cc python-list@python.org, git@vger.kernel.org
X-BeenThere python-list@python.org
X-Mailman-Version 2.1.12
Precedence list
List-Id General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe <http://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive <http://mail.python.org/pipermail/python-list>
List-Post <mailto:python-list@python.org>
List-Help <mailto:python-list-request@python.org?subject=help>
List-Subscribe <http://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
Newsgroups comp.lang.python
Message-ID <mailman.2464.1320597747.27778.python-list@python.org> (permalink)
Lines 49
NNTP-Posting-Host 2001:888:2000:d::a6
X-Trace 1320597747 news.xs4all.nl 6907 [2001:888:2000:d::a6]:47444
X-Complaints-To abuse@xs4all.nl
Xref x330-a1.tempe.blueboxinc.net comp.lang.python:15378

Show key headers only | View raw

Jonathan Fine <jfine@pytex.org> writes:

> Hi
> 
> This it to let you know that I'm writing (in Python) a script that
> places the content of CTAN into a git repository.
>      https://bitbucket.org/jfine/python-ctantools

I hope that you meant "repositories" (plural) here, one per tool,
rather than putting all of CTAN into single Git repository.
 
> I'm working from the TeX Collection DVDs that are published each year
> by the TeX user groups, which contain a snapshot of CTAN (about
> 100,000 files occupying 4Gb), which means I have to unzip folders and
> do a few other things.

There is 'contrib/fast-import/import-zips.py' in git.git repository.
If you are not using it, or its equivalent, it might be worth checking
out.
 
> CTAN is the Comprehensive TeX Archive Network.  CTAN keeps only the
> latest version of each file, but old CTAN snapshots will provide many
> earlier versions.

There was similar effort done in putting CPAN (Comprehensive _Perl_
Archive Network) in Git, hosting repositories on GitHub[1], by the name
of gitPAN, see e.g.:

  "The gitPAN Import is Complete"
  http://perlisalive.com/articles/36
 
[1]: https://github.com/gitpan

> I'm working on putting old CTAN files into modern version
> control. Martin Scharrer is working in the other direction.  He's
> putting new files added to CTAN into Mercurial.
>      http://ctanhg.scharrer-online.de/

Nb. thanks to tools such as git-hg and fast-import / fast-export
we have quite good interoperability and convertability between
Git and Mercurial.

P.S. I'd point to reposurgeon tool, which can be used to do fixups
after import, but it would probably won't work on such large (set of)
repositories.

P.P.S. Can you forward it to comp.text.tex?
-- 
Jakub Narębski

Back to comp.lang.python | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

A Python script to put CTAN into git (from DVDs) Jonathan Fine <jfine@pytex.org> - 2011-11-06 15:17 +0000
  Re: A Python script to put CTAN into git (from DVDs) Jakub Narebski <jnareb@gmail.com> - 2011-11-06 08:42 -0800
    Re: A Python script to put CTAN into git (from DVDs) Jonathan Fine <jfine@pytex.org> - 2011-11-06 18:19 +0000
      Re: A Python script to put CTAN into git (from DVDs) Jakub Narebski <jnareb@gmail.com> - 2011-11-06 21:28 +0100
        Re: A Python script to put CTAN into git (from DVDs) Jonathan Fine <jfine@pytex.org> - 2011-11-07 20:21 +0000
        Re: A Python script to put CTAN into git (from DVDs) Jonathan Fine <jfine@pytex.org> - 2011-11-07 20:21 +0000
      Re: A Python script to put CTAN into git (from DVDs) Jakub Narebski <jnareb@gmail.com> - 2011-11-06 12:29 -0800
    Re: A Python script to put CTAN into git (from DVDs) Jonathan Fine <jfine@pytex.org> - 2011-11-06 18:19 +0000

csiph-web