Path: csiph.com!usenet.pasdenom.info!gegeweb.org!de-l.enfer-du-nord.net!feeder1.enfer-du-nord.net!feeds.phibee-telecom.net!newsfeed.xs4all.nl!newsfeed6.news.xs4all.nl!xs4all!post.news.xs4all.nl!not-for-mail
Date: Fri, 27 Jul 2012 03:50:39 -0700
From: Ethan Furman <ethan@stoneleaf.us>
User-Agent: Thunderbird 2.0.0.24 (Windows/20100228)
MIME-Version: 1.0
To: python-list@python.org
Subject: Re: ANN: dbf.py 0.94.003
References: <50119AF5.6020904@stoneleaf.us> <5011ED43.1050809@fossworkflowguides.com>
In-Reply-To: <5011ED43.1050809@fossworkflowguides.com>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
Precedence: list
Newsgroups: comp.lang.python
Message-ID: <mailman.2655.1343387770.4697.python-list@python.org>
Lines: 53
NNTP-Posting-Host: 2001:888:2000:d::a6
Xref: csiph.com comp.lang.python:26141

Simon Cropper wrote:
> On 27/07/12 05:31, Ethan Furman wrote:
>> A few more bug fixes, and I actually included the documentation this
>> time.  :)  It can be found at http://python.org/pypi/dbf, and has been
>> tested on CPythons 2.4 - 2.7, and PyPy 1.8.
> 
> [snip]
> 
> Ethan,
> 
> That's great.
> 
> Can you comment on the ultimate aim of the project?

To provide read/write access to the dbf format, from
dBase III to dBase 7, including memos and index files.
Currently supports dBase III, FoxPro, Clipper,
and Visual Foxpro (but not autoincrement nor varchar).


> Is this package primarily a "universal dbf translator" that allows the 
> data stored in DBFs (which I might add I have many in legacy VFP 
> applications and GIS Shapefiles) to be accessed and extracted or is the 
> module being designed to be used interactively to extract data from and 
> update tables?

Some folk use it as a dbf translator, some folk use it for interactive 
work.  I use it for both those purposes as well as creating new dbf 
files which get processed by our in-house software as well as 
third-party software every day.


> I remember on the last thread that someone mentioned that indexes are 
> not supported. I presume then that moving around a table with a couple 
> of million records might be a tad slow. Have you tested the package on 
> large datasets, both DBFs with a large number of records as well as a 
> large number of fields?

The largest tables I've had at my disposal so far were about 300,000 
records with roughly 50 fields with a total record length of about 
1,500.  Processing (for me) involves going through every single record, 
and yes it was a tad slow.  This is my most common scenario, and index 
files would not help at all.

For more typical work (for others) of selecting and using a subset of 
the dbf, an in-memory index can be created -- initial creation can take 
a few moments, but searches afterwards are quite quick.

This is a pure-python implementation, so speed is not the first goal. 
At some point in the future I would like to create a C accelerator, but 
that's pretty far down the to-do list.

~Ethan~