Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > comp.lang.python > #108116
| From | DFS <nospam@dfs.com> |
|---|---|
| Newsgroups | comp.lang.python |
| Subject | Re: Fastest way to retrieve and write html contents to file |
| Date | 2016-05-04 02:10 -0400 |
| Organization | A noiseless patient Spider |
| Message-ID | <ngc3hi$3ql$1@dont-email.me> (permalink) |
| References | (13 earlier) <20160503102850.63cec18a@bigbox.christie.dr> <mailman.352.1462294245.32212.python-list@python.org> <ngal91$8c4$2@dont-email.me> <20160503134121.19f71792@bigbox.christie.dr> <mailman.362.1462309634.32212.python-list@python.org> |
On 5/3/2016 2:41 PM, Tim Chase wrote:
> On 2016-05-03 13:00, DFS wrote:
>> On 5/3/2016 11:28 AM, Tim Chase wrote:
>>> On 2016-05-03 00:24, DFS wrote:
>>>> One small comparison I was able to make was VBA vs python/pyodbc
>>>> to summarize an Access database. Not quite a fair test, but
>>>> interesting nonetheless.
>>>>
>>>> Access 2003 file
>>>> Access 2003 VBA code
>>>> Time: 0.18 seconds
>>>>
>>>> same Access 2003 file
>>>> 32-bit python 2.7.11 + 32-bit pyodbc 3.0.6
>>>> Time: 0.49 seconds
>>>
>>> Curious whether you're forcing Access VBA to talk over ODBC or
>>> whether Access is using native access/file-handling (and thus
>>> bypassing the ODBC overhead)?
>>
>> The latter, which is why I said "not quite a fair test".
>
> Can you try the same tests, getting Access/VBA to use ODBC instead to
> see how much overhead ODBC entails?
>
> -tkc
Done.
I dropped a few extraneous tables from the database (was 114 tables):
Access 2003 .mdb file
2,009,164 rows
97 tables (max row = 600288)
725 columns
text: 389
boolean: 4
numeric: 261
date-time: 69
binary: 2
264 indexes (25 foreign keys)*
299,167,744 bytes on disk
1. DAO
Time: 0.15 seconds
2. ADODB, Access ODBC driver, OpenSchema method**
Time: 0.26 seconds
3. python, pyodbc, Access ODBC driver
Time: 0.42 seconds
* despite being written by Microsoft, the Access ODBC driver doesn't
support the ODBC SQLForeignKeys function, so the python code doesn't
show a count of foreign keys
** the Access ODBC driver doesn't support the adSchemaIndexes or
adSchemaForeignKeys query types, so I used DAO code to count
indexes and foreign keys.
Back to comp.lang.python | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread
Fastest way to retrieve and write html contents to file DFS <nospam@dfs.com> - 2016-05-02 00:06 -0400
Re: Fastest way to retrieve and write html contents to file Stephen Hansen <me+python@ixokai.io> - 2016-05-01 21:34 -0700
Re: Fastest way to retrieve and write html contents to file Chris Angelico <rosuav@gmail.com> - 2016-05-02 14:40 +1000
Re: Fastest way to retrieve and write html contents to file DFS <nospam@dfs.com> - 2016-05-02 00:50 -0400
Re: Fastest way to retrieve and write html contents to file Stephen Hansen <me+python@ixokai.io> - 2016-05-01 22:00 -0700
Re: Fastest way to retrieve and write html contents to file DFS <nospam@dfs.com> - 2016-05-02 01:04 -0400
Re: Fastest way to retrieve and write html contents to file Chris Angelico <rosuav@gmail.com> - 2016-05-02 15:12 +1000
Re: Fastest way to retrieve and write html contents to file Stephen Hansen <me+python@ixokai.io> - 2016-05-01 22:17 -0700
Re: Fastest way to retrieve and write html contents to file Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2016-05-02 15:57 +1000
Re: Fastest way to retrieve and write html contents to file Ben Finney <ben+python@benfinney.id.au> - 2016-05-02 14:49 +1000
Re: Fastest way to retrieve and write html contents to file DFS <nospam@dfs.com> - 2016-05-02 01:00 -0400
Re: Fastest way to retrieve and write html contents to file Stephen Hansen <me+python@ixokai.io> - 2016-05-01 22:15 -0700
Re: Fastest way to retrieve and write html contents to file DFS <nospam@dfs.com> - 2016-05-02 01:59 -0400
Re: Fastest way to retrieve and write html contents to file Stephen Hansen <me+python@ixokai.io> - 2016-05-01 23:27 -0700
Re: Fastest way to retrieve and write html contents to file DFS <nospam@dfs.com> - 2016-05-02 03:37 -0400
Re: Fastest way to retrieve and write html contents to file Stephen Hansen <me+python@ixokai.io> - 2016-05-02 00:58 -0700
Re: Fastest way to retrieve and write html contents to file Michael Torrie <torriem@gmail.com> - 2016-05-02 22:06 -0600
Re: Fastest way to retrieve and write html contents to file DFS <nospam@dfs.com> - 2016-05-03 00:24 -0400
Re: Fastest way to retrieve and write html contents to file Tim Chase <python.list@tim.thechases.com> - 2016-05-03 10:28 -0500
Re: Fastest way to retrieve and write html contents to file DFS <nospam@dfs.com> - 2016-05-03 13:00 -0400
Re: Fastest way to retrieve and write html contents to file Tim Chase <python.list@tim.thechases.com> - 2016-05-03 13:41 -0500
Re: Fastest way to retrieve and write html contents to file DFS <nospam@dfs.com> - 2016-05-04 02:10 -0400
Re: Fastest way to retrieve and write html contents to file Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2016-05-02 16:05 +1000
Re: Fastest way to retrieve and write html contents to file DFS <nospam@dfs.com> - 2016-05-02 02:47 -0400
Re: Fastest way to retrieve and write html contents to file Chris Angelico <rosuav@gmail.com> - 2016-05-02 17:19 +1000
Re: Fastest way to retrieve and write html contents to file DFS <nospam@dfs.com> - 2016-05-02 21:51 -0400
Re: Fastest way to retrieve and write html contents to file Chris Angelico <rosuav@gmail.com> - 2016-05-03 12:00 +1000
Re: Fastest way to retrieve and write html contents to file DFS <nospam@dfs.com> - 2016-05-02 22:01 -0400
Re: Fastest way to retrieve and write html contents to file Peter Otten <__peter__@web.de> - 2016-05-02 10:42 +0200
Re: Fastest way to retrieve and write html contents to file DFS <nospam@dfs.com> - 2016-05-02 21:52 -0400
Re: Fastest way to retrieve and write html contents to file Chris Angelico <rosuav@gmail.com> - 2016-05-02 14:53 +1000
Re: Fastest way to retrieve and write html contents to file Tim Chase <python.list@tim.thechases.com> - 2016-05-02 07:38 -0500
csiph-web