Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > comp.os.linux.advocacy > #413236
| From | Snit <usenet@gallopinginsanity.com> |
|---|---|
| Newsgroups | comp.os.linux.advocacy |
| Subject | Re: Snit hitting the glue bag early today |
| Date | 2017-05-05 13:44 -0700 |
| Message-ID | <D5323259.A3AF5%usenet@gallopinginsanity.com> (permalink) |
| References | (11 earlier) <zcv903ahbue.3u8@rooftop.invalid> <oegs1s$b3k$3@dont-email.me> <acv993.by93ap@rooftop.invalid> <oeias1$v75$2@dont-email.me> <zvz8v03.bu8ie@rooftop.invalid> |
On 5/5/17, 1:14 PM, in article zvz8v03.bu8ie@rooftop.invalid, "owl"
<owl@rooftop.invalid> wrote:
> DFS <nospam@dfs.com> wrote:
>> On 5/5/2017 2:18 AM, owl wrote:
>>> DFS <nospam@dfs.com> wrote:
>>
> ...
>
>
>> f.write("%s,%s,%s\n" % (dt,cnt,nm))
>> print "...\n.csv file written"
>>
>> db.close()
>> conn.close()
>> print "\n%.2fs prog exec"%(time.clock() - startmaster)
>> --------------------------------------------------------------------
>>
>> That's 35 lines, for 'readability' and timing and feedback.
>>
>> I can shorten it to 17 lines and still meet the 7 requirements.
>>
>
> Mine's about 60-70 lines now, but I'm still leaving out a lot
> of error-checking code that really should be in there.
>
> ...
>
>
>>
>>> I guess these are the queries you want. I have to say, it's pretty
>>> slow. Nearly 8.5 seconds for a measly 1000 inserts.
>>
>> try
>>
>> BEGIN TRANSACTION;
>> END TRANSACTION;
>> or
>> BEGIN TRANSACTION;
>> conn.commit()
>>
>
> Thanks. begin/end tansaction did the trick. I think I remember
> that now from something else I was working on way back.
> ...
>
>
>>> fprintf(stdout,"\"%s\"\n",PQgetvalue(result,i,1));
>>> }
>>> PQclear(result);
>>> PQfinish(conn);
>>> fclose(fp1);
>>> fclose(fp2);
>>> return 0;
>>> }
>>> -----------------------------------------------
>>
>> Thanks. Not as 'unwieldy' as I expected C/postgres to be.
>>
>>
>
> Also got the "copy from" to work now. Needed to close the file
> before doing the copy. Here's how things show on my machine now:
>
> Python:
> anon@lowtide:~/code/pg$ time ./dfs.py 1000 >/dev/null
>
> real 0m0.226s
> user 0m0.032s
> sys 0m0.032s
> anon@lowtide:~/code/pg$
>
> C with inserts wrapped in transaction:
> anon@lowtide:~/code/pg$ time ./glah >/dev/null
>
> real 0m0.197s
> user 0m0.016s
> sys 0m0.008s
> anon@lowtide:~/code/pg$
>
> C creating a csv and using "copy from" instead of inserts:
> anon@lowtide:~/code/pg$ time ./flah >/dev/null
> PGRES_COMMAND_OK
>
> real 0m0.111s
> user 0m0.008s
> sys 0m0.000s
> anon@lowtide:~/code/pg$
>
> Without using "copy from", the Python and C have similar performance.
> The postgresql connection is the bottleneck. However, depending on the
> complexity and volume of work necessary beforehand to build the queries,
> C could easily stomp the crap out of Python. :)
>
I just want to note that with the obsession about me and trolling of me
reduced the discussion of technology is growing. This happens to be about a
topic I have little knowledge in (programming) and I have nothing really to
add to it, but it is just great to see.
I am sincerely impressed and hope it continues.
--
Personal attacks from those who troll show their own insecurity. They cannot
use reason to show the message to be wrong so they try to feel somehow
superior by attacking the messenger.
They cling to their attacks and ignore the message time and time again.
<https://youtu.be/H4NW-Cqh308>
Back to comp.os.linux.advocacy | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread
Snit hitting the glue bag early today Charlie <pipedroner3@kismet45.org> - 2017-04-28 13:18 -0400
Re: Snit hitting the glue bag early today Silver Slimer <sl@im.er> - 2017-04-28 16:06 -0400
Re: Snit hitting the glue bag early today Snit <usenet@gallopinginsanity.com> - 2017-04-28 13:08 -0700
Re: Snit hitting the glue bag early today Sandman <mr@sandman.net> - 2017-05-01 07:10 +0000
Re: Snit hitting the glue bag early today Steve Carroll <fretwizzer@gmail.com> - 2017-05-01 07:17 -0700
Re: Snit hitting the glue bag early today Glenn Harrison <glannharrison0@gmail.com> - 2017-05-01 16:24 +0000
Re: Snit hitting the glue bag early today Snit <usenet@gallopinginsanity.com> - 2017-05-01 09:38 -0700
Re: Snit hitting the glue bag early today Sandman <mr@sandman.net> - 2017-05-01 21:51 +0000
Re: Snit hitting the glue bag early today Snit <usenet@gallopinginsanity.com> - 2017-05-01 15:14 -0700
Re: Snit hitting the glue bag early today vallor <vallor@cultnix.org> - 2017-05-01 22:53 +0000
Re: Snit hitting the glue bag early today Snit <usenet@gallopinginsanity.com> - 2017-05-01 16:34 -0700
Re: Snit hitting the glue bag early today Sandman <mr@sandman.net> - 2017-05-01 18:00 +0000
Re: Snit hitting the glue bag early today DFS <nospam@dfs.com> - 2017-05-01 18:35 -0400
Re: Snit hitting the glue bag early today owl <owl@rooftop.invalid> - 2017-05-02 02:23 +0000
Re: Snit hitting the glue bag early today DFS <nospam@dfs.com> - 2017-05-02 11:26 -0400
Re: Snit hitting the glue bag early today owl <owl@rooftop.invalid> - 2017-05-02 22:12 +0000
Re: Snit hitting the glue bag early today DFS <nospam@dfs.com> - 2017-05-03 09:22 -0400
Re: Snit hitting the glue bag early today owl <owl@rooftop.invalid> - 2017-05-03 14:07 +0000
Re: Snit hitting the glue bag early today DFS <nospam@dfs.com> - 2017-05-04 09:29 -0400
Re: Snit hitting the glue bag early today Sandman <mr@sandman.net> - 2017-05-04 16:42 +0000
Re: Snit hitting the glue bag early today owl <owl@rooftop.invalid> - 2017-05-04 20:06 +0000
Re: Snit hitting the glue bag early today owl <owl@rooftop.invalid> - 2017-05-04 20:12 +0000
Re: Snit hitting the glue bag early today chrisv <chrisv@nospam.invalid> - 2017-05-05 06:52 -0500
Re: Snit hitting the glue bag early today Marek Novotny <marek.novotny@marspolar.com> - 2017-05-05 10:32 -0500
Re: Snit hitting the glue bag early today DFS <nospam@dfs.com> - 2017-05-05 13:22 -0400
Re: Snit hitting the glue bag early today DFS <nospam@dfs.com> - 2017-05-05 13:01 -0400
Re: Snit hitting the glue bag early today Steve Carroll <fretwizzer@gmail.com> - 2017-05-04 13:16 -0700
Re: Snit hitting the glue bag early today Sandman <mr@sandman.net> - 2017-05-04 21:36 +0000
Re: Snit hitting the glue bag early today DFS <nospam@dfs.com> - 2017-05-06 09:44 -0400
Re: Snit hitting the glue bag early today owl <owl@rooftop.invalid> - 2017-05-04 19:49 +0000
Re: Snit hitting the glue bag early today DFS <nospam@dfs.com> - 2017-05-04 19:28 -0400
Re: Snit hitting the glue bag early today owl <owl@rooftop.invalid> - 2017-05-05 02:56 +0000
Re: Snit hitting the glue bag early today DFS <nospam@dfs.com> - 2017-05-04 23:40 -0400
Re: Snit hitting the glue bag early today owl <owl@rooftop.invalid> - 2017-05-05 06:18 +0000
Re: Snit hitting the glue bag early today owl <owl@rooftop.invalid> - 2017-05-05 06:34 +0000
Re: Snit hitting the glue bag early today Sandman <mr@sandman.net> - 2017-05-05 07:07 +0000
Re: Snit hitting the glue bag early today owl <owl@rooftop.invalid> - 2017-05-05 07:29 +0000
Re: Snit hitting the glue bag early today owl <owl@rooftop.invalid> - 2017-05-05 16:48 +0000
Re: Snit hitting the glue bag early today owl <owl@rooftop.invalid> - 2017-05-05 20:49 +0000
Re: Snit hitting the glue bag early today DFS <nospam@dfs.com> - 2017-05-05 13:02 -0400
Re: Snit hitting the glue bag early today Sandman <mr@sandman.net> - 2017-05-05 19:12 +0000
Re: Snit hitting the glue bag early today Snit <usenet@gallopinginsanity.com> - 2017-05-05 13:45 -0700
Re: Snit hitting the glue bag early today DFS <nospam@dfs.com> - 2017-05-05 17:09 -0400
Re: Snit hitting the glue bag early today Sandman <mr@sandman.net> - 2017-05-05 21:41 +0000
Re: Snit hitting the glue bag early today DFS <nospam@dfs.com> - 2017-05-05 13:00 -0400
Re: Snit hitting the glue bag early today owl <owl@rooftop.invalid> - 2017-05-05 20:14 +0000
Re: Snit hitting the glue bag early today Snit <usenet@gallopinginsanity.com> - 2017-05-05 13:44 -0700
Re: Snit hitting the glue bag early today DFS <nospam@dfs.com> - 2017-05-05 23:51 -0400
Re: Snit hitting the glue bag early today owl <owl@rooftop.invalid> - 2017-05-06 04:48 +0000
Re: Snit hitting the glue bag early today DFS <nospam@dfs.com> - 2017-05-06 10:06 -0400
Re: Snit hitting the glue bag early today owl <owl@rooftop.invalid> - 2017-05-06 15:50 +0000
Re: Snit hitting the glue bag early today DFS <nospam@dfs.com> - 2017-05-07 13:09 -0400
Re: Snit hitting the glue bag early today owl <owl@rooftop.invalid> - 2017-05-07 18:39 +0000
Re: Snit hitting the glue bag early today owl <owl@rooftop.invalid> - 2017-05-09 22:01 +0000
Re: Snit hitting the glue bag early today Marek Novotny <marek.novotny@marspolar.com> - 2017-05-09 17:17 -0500
Re: Snit hitting the glue bag early today owl <owl@rooftop.invalid> - 2017-05-09 22:45 +0000
Re: Snit hitting the glue bag early today Snit <usenet@gallopinginsanity.com> - 2017-05-09 16:21 -0700
Re: Snit hitting the glue bag early today Marek Novotny <marek.novotny@marspolar.com> - 2017-05-09 19:20 -0500
Re: Snit hitting the glue bag early today owl <owl@rooftop.invalid> - 2017-05-10 03:01 +0000
Re: Snit hitting the glue bag early today Marek Novotny <marek.novotny@marspolar.com> - 2017-05-09 22:04 -0500
Re: Snit hitting the glue bag early today DFS <nospam@dfs.com> - 2017-05-09 23:13 -0400
Re: Snit hitting the glue bag early today DFS <nospam@dfs.com> - 2017-05-09 23:41 -0400
Re: Snit hitting the glue bag early today owl <owl@rooftop.invalid> - 2017-05-10 04:56 +0000
Re: Snit hitting the glue bag early today DFS <nospam@dfs.com> - 2017-05-11 11:21 -0400
Re: Snit hitting the glue bag early today DFS <nospam@dfs.com> - 2017-05-12 14:21 -0400
Re: Snit hitting the glue bag early today Snit <usenet@gallopinginsanity.com> - 2017-05-12 11:34 -0700
Re: Snit hitting the glue bag early today Steve Carroll <fretwizzer@gmail.com> - 2017-05-12 11:39 -0700
Re: Snit hitting the glue bag early today DFS <nospam@dfs.com> - 2017-05-12 19:06 -0400
Re: Snit hitting the glue bag early today Steve Carroll <fretwizzer@gmail.com> - 2017-05-13 09:14 -0700
Re: Snit hitting the glue bag early today DFS <nospam@dfs.com> - 2017-05-14 08:15 -0400
Re: Snit hitting the glue bag early today Steve Carroll <fretwizzer@gmail.com> - 2017-05-14 09:46 -0700
Re: Snit hitting the glue bag early today Marek Novotny <marek.novotny@marspolar.com> - 2017-05-14 12:31 -0500
Re: Snit hitting the glue bag early today Steve Carroll <fretwizzer@gmail.com> - 2017-05-14 10:42 -0700
Re: Snit hitting the glue bag early today Marek Novotny <marek.novotny@marspolar.com> - 2017-05-14 12:45 -0500
Re: Snit hitting the glue bag early today Steve Carroll <fretwizzer@gmail.com> - 2017-05-14 10:50 -0700
Re: Snit hitting the glue bag early today Marek Novotny <marek.novotny@marspolar.com> - 2017-05-14 12:54 -0500
Re: Snit hitting the glue bag early today owl <owl@rooftop.invalid> - 2017-05-14 19:28 +0000
Re: Snit hitting the glue bag early today DFS <nospam@dfs.com> - 2017-05-14 23:08 -0400
Re: Snit hitting the glue bag early today Snit <usenet@gallopinginsanity.com> - 2017-05-14 20:16 -0700
Re: Snit hitting the glue bag early today Steve Carroll <fretwizzer@gmail.com> - 2017-05-15 09:46 -0700
Re: Snit hitting the glue bag early today owl <owl@rooftop.invalid> - 2017-05-15 19:51 +0000
Re: Snit hitting the glue bag early today Snit <usenet@gallopinginsanity.com> - 2017-05-15 12:59 -0700
Re: Snit hitting the glue bag early today Steve Carroll <fretwizzer@gmail.com> - 2017-05-15 15:26 -0700
Re: Snit hitting the glue bag early today DFS <nospam@dfs.com> - 2017-05-16 14:02 -0400
Re: Snit hitting the glue bag early today Snit <usenet@gallopinginsanity.com> - 2017-05-16 11:11 -0700
Re: Snit hitting the glue bag early today Steve Carroll <fretwizzer@gmail.com> - 2017-05-16 12:26 -0700
Re: Snit hitting the glue bag early today DFS <nospam@dfs.com> - 2017-05-17 09:06 -0400
It takes me 8 Seconds to Download 999 articles ( full header and body ). Jeff-Relf.Me @. - 2017-05-19 04:24 -0700
Re: It takes me 8 Seconds to Download 999 articles ( full header and body ). tlvp <mPiOsUcB.EtLlLvEp@att.net> - 2017-05-19 23:17 -0400
Re: Snit hitting the glue bag early today owl <owl@rooftop.invalid> - 2017-05-13 16:24 +0000
Re: Snit hitting the glue bag early today DFS <nospam@dfs.com> - 2017-05-13 13:39 -0400
Re: Snit hitting the glue bag early today owl <owl@rooftop.invalid> - 2017-05-13 18:24 +0000
Re: Snit hitting the glue bag early today DFS <nospam@dfs.com> - 2017-05-13 23:47 -0400
Re: Snit hitting the glue bag early today DFS <nospam@dfs.com> - 2017-05-15 11:23 -0400
Re: Snit hitting the glue bag early today owl <owl@rooftop.invalid> - 2017-05-15 20:49 +0000
Re: Snit hitting the glue bag early today owl <owl@rooftop.invalid> - 2017-05-15 21:05 +0000
Re: Snit hitting the glue bag early today DFS <nospam@dfs.com> - 2017-05-15 17:30 -0400
Re: Snit hitting the glue bag early today owl <owl@rooftop.invalid> - 2017-05-15 22:08 +0000
Re: Snit hitting the glue bag early today DFS <nospam@dfs.com> - 2017-05-15 19:50 -0400
Re: Snit hitting the glue bag early today vallor <vallor@cultnix.org> - 2017-05-17 21:04 +0000
Re: Snit hitting the glue bag early today Marek Novotny <marek.novotny@marspolar.com> - 2017-05-15 17:24 -0500
Re: Snit hitting the glue bag early today DFS <nospam@dfs.com> - 2017-05-15 18:45 -0400
Re: Snit hitting the glue bag early today Marek Novotny <marek.novotny@marspolar.com> - 2017-05-15 17:57 -0500
Re: Snit hitting the glue bag early today DFS <nospam@dfs.com> - 2017-05-15 11:30 -0400
Re: Snit hitting the glue bag early today DFS <nospam@dfs.com> - 2017-05-15 20:53 -0400
Re: Snit hitting the glue bag early today owl <owl@rooftop.invalid> - 2017-05-16 01:16 +0000
Re: Snit hitting the glue bag early today DFS <nospam@dfs.com> - 2017-05-16 09:32 -0400
Re: Snit hitting the glue bag early today owl <owl@rooftop.invalid> - 2017-05-16 14:22 +0000
Re: Snit hitting the glue bag early today DFS <nospam@dfs.com> - 2017-05-16 14:03 -0400
Re: Snit hitting the glue bag early today DFS <nospam@dfs.com> - 2017-05-10 09:23 -0400
Re: Snit hitting the glue bag early today owl <owl@rooftop.invalid> - 2017-05-10 22:21 +0000
Re: Snit hitting the glue bag early today DFS <nospam@dfs.com> - 2017-05-11 11:21 -0400
Re: Snit hitting the glue bag early today owl <owl@rooftop.invalid> - 2017-05-11 22:42 +0000
Re: Snit hitting the glue bag early today DFS <nospam@dfs.com> - 2017-05-11 22:43 -0400
Re: Snit hitting the glue bag early today owl <owl@rooftop.invalid> - 2017-05-12 04:43 +0000
Re: Snit hitting the glue bag early today DFS <nospam@dfs.com> - 2017-05-12 14:20 -0400
Re: Snit hitting the glue bag early today owl <owl@rooftop.invalid> - 2017-05-12 20:44 +0000
Re: Snit hitting the glue bag early today DFS <nospam@dfs.com> - 2017-05-13 09:06 -0400
Re: Snit hitting the glue bag early today owl <owl@rooftop.invalid> - 2017-05-13 16:20 +0000
Re: Snit hitting the glue bag early today Chris Ahlstrom <OFeem1987@teleworm.us> - 2017-05-05 05:28 -0400
Re: Snit hitting the glue bag early today DFS <nospam@dfs.com> - 2017-05-05 13:02 -0400
Re: Snit hitting the glue bag early today DFS <nospam@dfs.com> - 2017-05-01 14:44 -0400
Re: Snit hitting the glue bag early today Snit <usenet@gallopinginsanity.com> - 2017-05-01 11:50 -0700
Re: Snit hitting the glue bag early today Steve Carroll <fretwizzer@gmail.com> - 2017-05-01 12:12 -0700
Re: Snit hitting the glue bag early today DFS <nospam@dfs.com> - 2017-05-01 17:37 -0400
Re: Snit hitting the glue bag early today Sandman <mr@sandman.net> - 2017-05-01 22:06 +0000
Re: Snit hitting the glue bag early today chrisv <chrisv@nospam.invalid> - 2017-05-02 08:08 -0500
Re: Snit hitting the glue bag early today DFS <nospam@dfs.com> - 2017-05-02 11:26 -0400
Re: Snit hitting the glue bag early today Silver Slimer <sl@im.er> - 2017-05-02 12:25 -0400
Re: Snit hitting the glue bag early today Snit <usenet@gallopinginsanity.com> - 2017-05-02 09:52 -0700
Re: Snit hitting the glue bag early today Sandman <mr@sandman.net> - 2017-05-02 19:39 +0000
Re: Snit hitting the glue bag early today Snit <usenet@gallopinginsanity.com> - 2017-05-02 14:14 -0700
Snit digest 432 / 2017-05-02 Sandman <mr@sandman.net> - 2017-05-02 21:37 +0000
Re: Snit hitting the glue bag early today "Alistair Fitzpatrick" <fitz@gmail.com> - 2017-05-03 09:32 -0400
Re: Snit hitting the glue bag early today chrisv <chrisv@nospam.invalid> - 2017-05-03 08:55 -0500
Re: Snit hitting the glue bag early today DFS <nospam@dfs.com> - 2017-05-03 10:20 -0400
Re: Snit hitting the glue bag early today William Poaster <wp@dev.null> - 2017-05-03 15:22 +0100
Re: Snit hitting the glue bag early today DFS <nospam@dfs.com> - 2017-05-02 10:28 -0400
Re: Snit hitting the glue bag early today Silver Slimer <sl@im.er> - 2017-05-01 19:25 -0400
Re: Snit hitting the glue bag early today Sandman <mr@sandman.net> - 2017-05-01 21:50 +0000
Re: Snit hitting the glue bag early today Snit <usenet@gallopinginsanity.com> - 2017-05-01 08:11 -0700
Re: Snit hitting the glue bag early today Sandman <mr@sandman.net> - 2017-05-01 18:06 +0000
Re: Snit hitting the glue bag early today Snit <usenet@gallopinginsanity.com> - 2017-05-01 11:48 -0700
Re: Snit hitting the glue bag early today Sandman <mr@sandman.net> - 2017-05-01 19:20 +0000
Re: Snit hitting the glue bag early today Snit <usenet@gallopinginsanity.com> - 2017-05-01 12:31 -0700
Re: Snit hitting the glue bag early today Sandman <mr@sandman.net> - 2017-05-01 21:37 +0000
Re: Snit hitting the glue bag early today Snit <usenet@gallopinginsanity.com> - 2017-05-01 15:14 -0700
Re: Snit hitting the glue bag early today Sandman <mr@sandman.net> - 2017-05-02 06:28 +0000
Re: Snit hitting the glue bag early today Sandman <mr@sandman.net> - 2017-05-02 06:47 +0000
Re: Snit hitting the glue bag early today vallor <vallor@cultnix.org> - 2017-05-02 06:53 +0000
Re: Snit hitting the glue bag early today Sandman <mr@sandman.net> - 2017-05-02 07:10 +0000
Re: Snit hitting the glue bag early today Snit <usenet@gallopinginsanity.com> - 2017-05-02 09:52 -0700
Re: Snit hitting the glue bag early today Sandman <mr@sandman.net> - 2017-05-02 19:37 +0000
Re: Snit hitting the glue bag early today Snit <usenet@gallopinginsanity.com> - 2017-05-02 14:12 -0700
Snit digest 431 / 2017-05-02 Sandman <mr@sandman.net> - 2017-05-02 21:34 +0000
Re: Snit hitting the glue bag early today DFS <nospam@dfs.com> - 2017-05-02 11:27 -0400
Re: Snit hitting the glue bag early today Snit <usenet@gallopinginsanity.com> - 2017-05-02 09:49 -0700
Re: Snit hitting the glue bag early today Sandman <mr@sandman.net> - 2017-05-02 19:36 +0000
Re: Snit hitting the glue bag early today Snit <usenet@gallopinginsanity.com> - 2017-05-02 14:11 -0700
Snit digest 430 / 2017-05-02 Sandman <mr@sandman.net> - 2017-05-02 21:33 +0000
Re: Snit hitting the glue bag early today Sandman <mr@sandman.net> - 2017-05-02 19:23 +0000
Re: Snit hitting the glue bag early today Snit <usenet@gallopinginsanity.com> - 2017-05-02 14:09 -0700
Snit digest 429 / 2017-05-02 Sandman <mr@sandman.net> - 2017-05-02 21:32 +0000
Re: Snit digest 429 / 2017-05-02 chrisv <chrisv@nospam.invalid> - 2017-05-03 07:31 -0500
Re: Snit digest 429 / 2017-05-02 Silver Slimer <sl@im.er> - 2017-05-03 10:16 -0400
Re: Snit hitting the glue bag early today Snit <usenet@gallopinginsanity.com> - 2017-05-02 09:48 -0700
Re: Snit hitting the glue bag early today Sandman <mr@sandman.net> - 2017-05-02 19:33 +0000
csiph-web