Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > comp.os.linux.advocacy > #411925 > unrolled thread
| Started by | Charlie <pipedroner3@kismet45.org> |
|---|---|
| First post | 2017-04-28 13:18 -0400 |
| Last post | 2017-05-02 19:33 +0000 |
| Articles | 20 on this page of 167 — 16 participants |
Back to article view | Back to comp.os.linux.advocacy
Snit hitting the glue bag early today Charlie <pipedroner3@kismet45.org> - 2017-04-28 13:18 -0400
Re: Snit hitting the glue bag early today Silver Slimer <sl@im.er> - 2017-04-28 16:06 -0400
Re: Snit hitting the glue bag early today Snit <usenet@gallopinginsanity.com> - 2017-04-28 13:08 -0700
Re: Snit hitting the glue bag early today Sandman <mr@sandman.net> - 2017-05-01 07:10 +0000
Re: Snit hitting the glue bag early today Steve Carroll <fretwizzer@gmail.com> - 2017-05-01 07:17 -0700
Re: Snit hitting the glue bag early today Glenn Harrison <glannharrison0@gmail.com> - 2017-05-01 16:24 +0000
Re: Snit hitting the glue bag early today Snit <usenet@gallopinginsanity.com> - 2017-05-01 09:38 -0700
Re: Snit hitting the glue bag early today Sandman <mr@sandman.net> - 2017-05-01 21:51 +0000
Re: Snit hitting the glue bag early today Snit <usenet@gallopinginsanity.com> - 2017-05-01 15:14 -0700
Re: Snit hitting the glue bag early today vallor <vallor@cultnix.org> - 2017-05-01 22:53 +0000
Re: Snit hitting the glue bag early today Snit <usenet@gallopinginsanity.com> - 2017-05-01 16:34 -0700
Re: Snit hitting the glue bag early today Sandman <mr@sandman.net> - 2017-05-01 18:00 +0000
Re: Snit hitting the glue bag early today DFS <nospam@dfs.com> - 2017-05-01 18:35 -0400
Re: Snit hitting the glue bag early today owl <owl@rooftop.invalid> - 2017-05-02 02:23 +0000
Re: Snit hitting the glue bag early today DFS <nospam@dfs.com> - 2017-05-02 11:26 -0400
Re: Snit hitting the glue bag early today owl <owl@rooftop.invalid> - 2017-05-02 22:12 +0000
Re: Snit hitting the glue bag early today DFS <nospam@dfs.com> - 2017-05-03 09:22 -0400
Re: Snit hitting the glue bag early today owl <owl@rooftop.invalid> - 2017-05-03 14:07 +0000
Re: Snit hitting the glue bag early today DFS <nospam@dfs.com> - 2017-05-04 09:29 -0400
Re: Snit hitting the glue bag early today Sandman <mr@sandman.net> - 2017-05-04 16:42 +0000
Re: Snit hitting the glue bag early today owl <owl@rooftop.invalid> - 2017-05-04 20:06 +0000
Re: Snit hitting the glue bag early today owl <owl@rooftop.invalid> - 2017-05-04 20:12 +0000
Re: Snit hitting the glue bag early today chrisv <chrisv@nospam.invalid> - 2017-05-05 06:52 -0500
Re: Snit hitting the glue bag early today Marek Novotny <marek.novotny@marspolar.com> - 2017-05-05 10:32 -0500
Re: Snit hitting the glue bag early today DFS <nospam@dfs.com> - 2017-05-05 13:22 -0400
Re: Snit hitting the glue bag early today DFS <nospam@dfs.com> - 2017-05-05 13:01 -0400
Re: Snit hitting the glue bag early today Steve Carroll <fretwizzer@gmail.com> - 2017-05-04 13:16 -0700
Re: Snit hitting the glue bag early today Sandman <mr@sandman.net> - 2017-05-04 21:36 +0000
Re: Snit hitting the glue bag early today DFS <nospam@dfs.com> - 2017-05-06 09:44 -0400
Re: Snit hitting the glue bag early today owl <owl@rooftop.invalid> - 2017-05-04 19:49 +0000
Re: Snit hitting the glue bag early today DFS <nospam@dfs.com> - 2017-05-04 19:28 -0400
Re: Snit hitting the glue bag early today owl <owl@rooftop.invalid> - 2017-05-05 02:56 +0000
Re: Snit hitting the glue bag early today DFS <nospam@dfs.com> - 2017-05-04 23:40 -0400
Re: Snit hitting the glue bag early today owl <owl@rooftop.invalid> - 2017-05-05 06:18 +0000
Re: Snit hitting the glue bag early today owl <owl@rooftop.invalid> - 2017-05-05 06:34 +0000
Re: Snit hitting the glue bag early today Sandman <mr@sandman.net> - 2017-05-05 07:07 +0000
Re: Snit hitting the glue bag early today owl <owl@rooftop.invalid> - 2017-05-05 07:29 +0000
Re: Snit hitting the glue bag early today owl <owl@rooftop.invalid> - 2017-05-05 16:48 +0000
Re: Snit hitting the glue bag early today owl <owl@rooftop.invalid> - 2017-05-05 20:49 +0000
Re: Snit hitting the glue bag early today DFS <nospam@dfs.com> - 2017-05-05 13:02 -0400
Re: Snit hitting the glue bag early today Sandman <mr@sandman.net> - 2017-05-05 19:12 +0000
Re: Snit hitting the glue bag early today Snit <usenet@gallopinginsanity.com> - 2017-05-05 13:45 -0700
Re: Snit hitting the glue bag early today DFS <nospam@dfs.com> - 2017-05-05 17:09 -0400
Re: Snit hitting the glue bag early today Sandman <mr@sandman.net> - 2017-05-05 21:41 +0000
Re: Snit hitting the glue bag early today DFS <nospam@dfs.com> - 2017-05-05 13:00 -0400
Re: Snit hitting the glue bag early today owl <owl@rooftop.invalid> - 2017-05-05 20:14 +0000
Re: Snit hitting the glue bag early today Snit <usenet@gallopinginsanity.com> - 2017-05-05 13:44 -0700
Re: Snit hitting the glue bag early today DFS <nospam@dfs.com> - 2017-05-05 23:51 -0400
Re: Snit hitting the glue bag early today owl <owl@rooftop.invalid> - 2017-05-06 04:48 +0000
Re: Snit hitting the glue bag early today DFS <nospam@dfs.com> - 2017-05-06 10:06 -0400
Re: Snit hitting the glue bag early today owl <owl@rooftop.invalid> - 2017-05-06 15:50 +0000
Re: Snit hitting the glue bag early today DFS <nospam@dfs.com> - 2017-05-07 13:09 -0400
Re: Snit hitting the glue bag early today owl <owl@rooftop.invalid> - 2017-05-07 18:39 +0000
Re: Snit hitting the glue bag early today owl <owl@rooftop.invalid> - 2017-05-09 22:01 +0000
Re: Snit hitting the glue bag early today Marek Novotny <marek.novotny@marspolar.com> - 2017-05-09 17:17 -0500
Re: Snit hitting the glue bag early today owl <owl@rooftop.invalid> - 2017-05-09 22:45 +0000
Re: Snit hitting the glue bag early today Snit <usenet@gallopinginsanity.com> - 2017-05-09 16:21 -0700
Re: Snit hitting the glue bag early today Marek Novotny <marek.novotny@marspolar.com> - 2017-05-09 19:20 -0500
Re: Snit hitting the glue bag early today owl <owl@rooftop.invalid> - 2017-05-10 03:01 +0000
Re: Snit hitting the glue bag early today Marek Novotny <marek.novotny@marspolar.com> - 2017-05-09 22:04 -0500
Re: Snit hitting the glue bag early today DFS <nospam@dfs.com> - 2017-05-09 23:13 -0400
Re: Snit hitting the glue bag early today DFS <nospam@dfs.com> - 2017-05-09 23:41 -0400
Re: Snit hitting the glue bag early today owl <owl@rooftop.invalid> - 2017-05-10 04:56 +0000
Re: Snit hitting the glue bag early today DFS <nospam@dfs.com> - 2017-05-11 11:21 -0400
Re: Snit hitting the glue bag early today DFS <nospam@dfs.com> - 2017-05-12 14:21 -0400
Re: Snit hitting the glue bag early today Snit <usenet@gallopinginsanity.com> - 2017-05-12 11:34 -0700
Re: Snit hitting the glue bag early today Steve Carroll <fretwizzer@gmail.com> - 2017-05-12 11:39 -0700
Re: Snit hitting the glue bag early today DFS <nospam@dfs.com> - 2017-05-12 19:06 -0400
Re: Snit hitting the glue bag early today Steve Carroll <fretwizzer@gmail.com> - 2017-05-13 09:14 -0700
Re: Snit hitting the glue bag early today DFS <nospam@dfs.com> - 2017-05-14 08:15 -0400
Re: Snit hitting the glue bag early today Steve Carroll <fretwizzer@gmail.com> - 2017-05-14 09:46 -0700
Re: Snit hitting the glue bag early today Marek Novotny <marek.novotny@marspolar.com> - 2017-05-14 12:31 -0500
Re: Snit hitting the glue bag early today Steve Carroll <fretwizzer@gmail.com> - 2017-05-14 10:42 -0700
Re: Snit hitting the glue bag early today Marek Novotny <marek.novotny@marspolar.com> - 2017-05-14 12:45 -0500
Re: Snit hitting the glue bag early today Steve Carroll <fretwizzer@gmail.com> - 2017-05-14 10:50 -0700
Re: Snit hitting the glue bag early today Marek Novotny <marek.novotny@marspolar.com> - 2017-05-14 12:54 -0500
Re: Snit hitting the glue bag early today owl <owl@rooftop.invalid> - 2017-05-14 19:28 +0000
Re: Snit hitting the glue bag early today DFS <nospam@dfs.com> - 2017-05-14 23:08 -0400
Re: Snit hitting the glue bag early today Snit <usenet@gallopinginsanity.com> - 2017-05-14 20:16 -0700
Re: Snit hitting the glue bag early today Steve Carroll <fretwizzer@gmail.com> - 2017-05-15 09:46 -0700
Re: Snit hitting the glue bag early today owl <owl@rooftop.invalid> - 2017-05-15 19:51 +0000
Re: Snit hitting the glue bag early today Snit <usenet@gallopinginsanity.com> - 2017-05-15 12:59 -0700
Re: Snit hitting the glue bag early today Steve Carroll <fretwizzer@gmail.com> - 2017-05-15 15:26 -0700
Re: Snit hitting the glue bag early today DFS <nospam@dfs.com> - 2017-05-16 14:02 -0400
Re: Snit hitting the glue bag early today Snit <usenet@gallopinginsanity.com> - 2017-05-16 11:11 -0700
Re: Snit hitting the glue bag early today Steve Carroll <fretwizzer@gmail.com> - 2017-05-16 12:26 -0700
Re: Snit hitting the glue bag early today DFS <nospam@dfs.com> - 2017-05-17 09:06 -0400
It takes me 8 Seconds to Download 999 articles ( full header and body ). Jeff-Relf.Me @. - 2017-05-19 04:24 -0700
Re: It takes me 8 Seconds to Download 999 articles ( full header and body ). tlvp <mPiOsUcB.EtLlLvEp@att.net> - 2017-05-19 23:17 -0400
Re: Snit hitting the glue bag early today owl <owl@rooftop.invalid> - 2017-05-13 16:24 +0000
Re: Snit hitting the glue bag early today DFS <nospam@dfs.com> - 2017-05-13 13:39 -0400
Re: Snit hitting the glue bag early today owl <owl@rooftop.invalid> - 2017-05-13 18:24 +0000
Re: Snit hitting the glue bag early today DFS <nospam@dfs.com> - 2017-05-13 23:47 -0400
Re: Snit hitting the glue bag early today DFS <nospam@dfs.com> - 2017-05-15 11:23 -0400
Re: Snit hitting the glue bag early today owl <owl@rooftop.invalid> - 2017-05-15 20:49 +0000
Re: Snit hitting the glue bag early today owl <owl@rooftop.invalid> - 2017-05-15 21:05 +0000
Re: Snit hitting the glue bag early today DFS <nospam@dfs.com> - 2017-05-15 17:30 -0400
Re: Snit hitting the glue bag early today owl <owl@rooftop.invalid> - 2017-05-15 22:08 +0000
Re: Snit hitting the glue bag early today DFS <nospam@dfs.com> - 2017-05-15 19:50 -0400
Re: Snit hitting the glue bag early today vallor <vallor@cultnix.org> - 2017-05-17 21:04 +0000
Re: Snit hitting the glue bag early today Marek Novotny <marek.novotny@marspolar.com> - 2017-05-15 17:24 -0500
Re: Snit hitting the glue bag early today DFS <nospam@dfs.com> - 2017-05-15 18:45 -0400
Re: Snit hitting the glue bag early today Marek Novotny <marek.novotny@marspolar.com> - 2017-05-15 17:57 -0500
Re: Snit hitting the glue bag early today DFS <nospam@dfs.com> - 2017-05-15 11:30 -0400
Re: Snit hitting the glue bag early today DFS <nospam@dfs.com> - 2017-05-15 20:53 -0400
Re: Snit hitting the glue bag early today owl <owl@rooftop.invalid> - 2017-05-16 01:16 +0000
Re: Snit hitting the glue bag early today DFS <nospam@dfs.com> - 2017-05-16 09:32 -0400
Re: Snit hitting the glue bag early today owl <owl@rooftop.invalid> - 2017-05-16 14:22 +0000
Re: Snit hitting the glue bag early today DFS <nospam@dfs.com> - 2017-05-16 14:03 -0400
Re: Snit hitting the glue bag early today DFS <nospam@dfs.com> - 2017-05-10 09:23 -0400
Re: Snit hitting the glue bag early today owl <owl@rooftop.invalid> - 2017-05-10 22:21 +0000
Re: Snit hitting the glue bag early today DFS <nospam@dfs.com> - 2017-05-11 11:21 -0400
Re: Snit hitting the glue bag early today owl <owl@rooftop.invalid> - 2017-05-11 22:42 +0000
Re: Snit hitting the glue bag early today DFS <nospam@dfs.com> - 2017-05-11 22:43 -0400
Re: Snit hitting the glue bag early today owl <owl@rooftop.invalid> - 2017-05-12 04:43 +0000
Re: Snit hitting the glue bag early today DFS <nospam@dfs.com> - 2017-05-12 14:20 -0400
Re: Snit hitting the glue bag early today owl <owl@rooftop.invalid> - 2017-05-12 20:44 +0000
Re: Snit hitting the glue bag early today DFS <nospam@dfs.com> - 2017-05-13 09:06 -0400
Re: Snit hitting the glue bag early today owl <owl@rooftop.invalid> - 2017-05-13 16:20 +0000
Re: Snit hitting the glue bag early today Chris Ahlstrom <OFeem1987@teleworm.us> - 2017-05-05 05:28 -0400
Re: Snit hitting the glue bag early today DFS <nospam@dfs.com> - 2017-05-05 13:02 -0400
Re: Snit hitting the glue bag early today DFS <nospam@dfs.com> - 2017-05-01 14:44 -0400
Re: Snit hitting the glue bag early today Snit <usenet@gallopinginsanity.com> - 2017-05-01 11:50 -0700
Re: Snit hitting the glue bag early today Steve Carroll <fretwizzer@gmail.com> - 2017-05-01 12:12 -0700
Re: Snit hitting the glue bag early today DFS <nospam@dfs.com> - 2017-05-01 17:37 -0400
Re: Snit hitting the glue bag early today Sandman <mr@sandman.net> - 2017-05-01 22:06 +0000
Re: Snit hitting the glue bag early today chrisv <chrisv@nospam.invalid> - 2017-05-02 08:08 -0500
Re: Snit hitting the glue bag early today DFS <nospam@dfs.com> - 2017-05-02 11:26 -0400
Re: Snit hitting the glue bag early today Silver Slimer <sl@im.er> - 2017-05-02 12:25 -0400
Re: Snit hitting the glue bag early today Snit <usenet@gallopinginsanity.com> - 2017-05-02 09:52 -0700
Re: Snit hitting the glue bag early today Sandman <mr@sandman.net> - 2017-05-02 19:39 +0000
Re: Snit hitting the glue bag early today Snit <usenet@gallopinginsanity.com> - 2017-05-02 14:14 -0700
Snit digest 432 / 2017-05-02 Sandman <mr@sandman.net> - 2017-05-02 21:37 +0000
Re: Snit hitting the glue bag early today "Alistair Fitzpatrick" <fitz@gmail.com> - 2017-05-03 09:32 -0400
Re: Snit hitting the glue bag early today chrisv <chrisv@nospam.invalid> - 2017-05-03 08:55 -0500
Re: Snit hitting the glue bag early today DFS <nospam@dfs.com> - 2017-05-03 10:20 -0400
Re: Snit hitting the glue bag early today William Poaster <wp@dev.null> - 2017-05-03 15:22 +0100
Re: Snit hitting the glue bag early today DFS <nospam@dfs.com> - 2017-05-02 10:28 -0400
Re: Snit hitting the glue bag early today Silver Slimer <sl@im.er> - 2017-05-01 19:25 -0400
Re: Snit hitting the glue bag early today Sandman <mr@sandman.net> - 2017-05-01 21:50 +0000
Re: Snit hitting the glue bag early today Snit <usenet@gallopinginsanity.com> - 2017-05-01 08:11 -0700
Re: Snit hitting the glue bag early today Sandman <mr@sandman.net> - 2017-05-01 18:06 +0000
Re: Snit hitting the glue bag early today Snit <usenet@gallopinginsanity.com> - 2017-05-01 11:48 -0700
Re: Snit hitting the glue bag early today Sandman <mr@sandman.net> - 2017-05-01 19:20 +0000
Re: Snit hitting the glue bag early today Snit <usenet@gallopinginsanity.com> - 2017-05-01 12:31 -0700
Re: Snit hitting the glue bag early today Sandman <mr@sandman.net> - 2017-05-01 21:37 +0000
Re: Snit hitting the glue bag early today Snit <usenet@gallopinginsanity.com> - 2017-05-01 15:14 -0700
Re: Snit hitting the glue bag early today Sandman <mr@sandman.net> - 2017-05-02 06:28 +0000
Re: Snit hitting the glue bag early today Sandman <mr@sandman.net> - 2017-05-02 06:47 +0000
Re: Snit hitting the glue bag early today vallor <vallor@cultnix.org> - 2017-05-02 06:53 +0000
Re: Snit hitting the glue bag early today Sandman <mr@sandman.net> - 2017-05-02 07:10 +0000
Re: Snit hitting the glue bag early today Snit <usenet@gallopinginsanity.com> - 2017-05-02 09:52 -0700
Re: Snit hitting the glue bag early today Sandman <mr@sandman.net> - 2017-05-02 19:37 +0000
Re: Snit hitting the glue bag early today Snit <usenet@gallopinginsanity.com> - 2017-05-02 14:12 -0700
Snit digest 431 / 2017-05-02 Sandman <mr@sandman.net> - 2017-05-02 21:34 +0000
Re: Snit hitting the glue bag early today DFS <nospam@dfs.com> - 2017-05-02 11:27 -0400
Re: Snit hitting the glue bag early today Snit <usenet@gallopinginsanity.com> - 2017-05-02 09:49 -0700
Re: Snit hitting the glue bag early today Sandman <mr@sandman.net> - 2017-05-02 19:36 +0000
Re: Snit hitting the glue bag early today Snit <usenet@gallopinginsanity.com> - 2017-05-02 14:11 -0700
Snit digest 430 / 2017-05-02 Sandman <mr@sandman.net> - 2017-05-02 21:33 +0000
Re: Snit hitting the glue bag early today Sandman <mr@sandman.net> - 2017-05-02 19:23 +0000
Re: Snit hitting the glue bag early today Snit <usenet@gallopinginsanity.com> - 2017-05-02 14:09 -0700
Snit digest 429 / 2017-05-02 Sandman <mr@sandman.net> - 2017-05-02 21:32 +0000
Re: Snit digest 429 / 2017-05-02 chrisv <chrisv@nospam.invalid> - 2017-05-03 07:31 -0500
Re: Snit digest 429 / 2017-05-02 Silver Slimer <sl@im.er> - 2017-05-03 10:16 -0400
Re: Snit hitting the glue bag early today Snit <usenet@gallopinginsanity.com> - 2017-05-02 09:48 -0700
Re: Snit hitting the glue bag early today Sandman <mr@sandman.net> - 2017-05-02 19:33 +0000
Page 3 of 9 — ← Prev page 1 2 [3] 4 5 6 7 8 9 Next page →
| From | Sandman <mr@sandman.net> |
|---|---|
| Date | 2017-05-05 19:12 +0000 |
| Message-ID | <sandman-9d8004cc061d14fa800516a65d864533@individual.net> |
| In reply to | #413169 |
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 In article <oeiau5$v75$4@dont-email.me>, DFS wrote: > > Sandman: > > I thought this was slow, but I'm using SQLite: > > You only met requirements 2,4,5,6 > 1 PostgreSQL 2 connection 3 create simple table (name, date) with PK > of your choice 4 add 1000 rows (5 random names, random dates within > a range of one month) 5 extract rows (group by query: name, date, > count(date)) 6 print summary to stdout 7 save summary to .csv file If you think writing to CSV takes more than 7 seconds with PHP you're sadly mistaken :) -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (GNU/Linux) iQEcBAEBAgAGBQJZDM64AAoJECFEwfBaA+jRaYEH/3z6QMN5b4kJKsy60G/eTUV2 +7+WIXW9wyGQ+/jv057UdyTf9hikFnGpNJgybOC9nRKKuBdzDdF21AGyycCOYtBj 5d+FnrOfszyj1Dql/z8xBqIUWoL5EgiMM/10DMiRYdi7pDYlLEYeja4VU28wVD// Oc7FgKeL1qZ2d54LywwRfezosxJatl0ZgQeQ0isQhr4CmKH+aanlkRkOIpZEz1iN ZTbiewwceC6FqM98wI51O7PhPvTIN9TtEWNVrlF764rGu7pMgvBGrOFf1qo9AOef /BwHS9bBslJzMuM5zx3jSU0vzb9esje/0FbmbG8oWermszzGWwTpr/MfjWnddSs= =WzjQ -----END PGP SIGNATURE----- -- Sandman
[toc] | [prev] | [next] | [standalone]
| From | Snit <usenet@gallopinginsanity.com> |
|---|---|
| Date | 2017-05-05 13:45 -0700 |
| Message-ID | <D532327C.A3AFB%usenet@gallopinginsanity.com> |
| In reply to | #413222 |
On 5/5/17, 12:12 PM, in article sandman-9d8004cc061d14fa800516a65d864533@individual.net, "Sandman" <mr@sandman.net> wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > In article <oeiau5$v75$4@dont-email.me>, DFS wrote: > >>> Sandman: >>> I thought this was slow, but I'm using SQLite: >> >> You only met requirements 2,4,5,6 > >> 1 PostgreSQL 2 connection 3 create simple table (name, date) with PK >> of your choice 4 add 1000 rows (5 random names, random dates within >> a range of one month) 5 extract rows (group by query: name, date, >> count(date)) 6 print summary to stdout 7 save summary to .csv file > > If you think writing to CSV takes more than 7 seconds with PHP you're sadly > mistaken :) Well, if you are using one of Apple's slow drives who knows. :) -- Personal attacks from those who troll show their own insecurity. They cannot use reason to show the message to be wrong so they try to feel somehow superior by attacking the messenger. They cling to their attacks and ignore the message time and time again. <https://youtu.be/H4NW-Cqh308>
[toc] | [prev] | [next] | [standalone]
| From | DFS <nospam@dfs.com> |
|---|---|
| Date | 2017-05-05 17:09 -0400 |
| Message-ID | <oeipd0$j7a$2@dont-email.me> |
| In reply to | #413222 |
On 5/5/2017 3:12 PM, Sandman wrote: > > In article <oeiau5$v75$4@dont-email.me>, DFS wrote: > >>> Sandman: >>> I thought this was slow, but I'm using SQLite: >> >> You only met requirements 2,4,5,6 > >> 1 PostgreSQL 2 connection 3 create simple table (name, date) with PK >> of your choice 4 add 1000 rows (5 random names, random dates within >> a range of one month) 5 extract rows (group by query: name, date, >> count(date)) 6 print summary to stdout 7 save summary to .csv file #1 Why no Postgres? #3 threw you for a loop... again ;) > If you think writing to CSV takes more than 7 seconds with PHP you're sadly > mistaken :) 7 seconds is 6.75 too many
[toc] | [prev] | [next] | [standalone]
| From | Sandman <mr@sandman.net> |
|---|---|
| Date | 2017-05-05 21:41 +0000 |
| Message-ID | <sandman-4817053dda068c51aba13794498f86af@individual.net> |
| In reply to | #413242 |
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 In article <oeipd0$j7a$2@dont-email.me>, DFS wrote: > > > > Sandman: > > > > I thought this was slow, but I'm using SQLite: > > > > > > DFS: > > > You only met requirements 2,4,5,6 > > > > > 1 PostgreSQL 2 connection 3 create simple table (name, date) > > > with PK of your choice 4 add 1000 rows (5 random names, random > > > dates within a range of one month) 5 extract rows (group by > > > query: name, date, count(date)) 6 print summary to stdout 7 save > > > summary to .csv file > > > #1 Why no Postgres? Don't have it. Don't care about it. > #3 threw you for a loop... again ;) Huh? -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (GNU/Linux) iQEcBAEBAgAGBQJZDPGmAAoJECFEwfBaA+jR9gsIAL1fty2mqIPMo1nnbwJdut73 5NSU8Rh3sFzpvow/4j32E0Y34efBUR0tcEokvoIU1pPJi7JBfxEw+B8I5RNjyMPR 5sF+Q4ZLJcjUGHU6trpls0Sv7qWNZZcWIUzsLqGAs3sDei8aqlQ8BQTGB+6gfQEd HtbBr2q/ZmHmzioPa+k/ZtKXHcfLrVMqBL4bceS0OrvqcRxe8dtp7jrlvYXsp6Pp UkzjPIc/HcHYLmyGEfrDrjmVkXE3YECSq6+8iYCbT4L3/ns9cYezzz/U5SXL85CC 0p77i6EYBKamO2576i/FEW1F/9yruJ2b4xoGcx2ii6lDwczTgBxwUe6LI5x3hr0= =wvBo -----END PGP SIGNATURE----- -- Sandman
[toc] | [prev] | [next] | [standalone]
| From | DFS <nospam@dfs.com> |
|---|---|
| Date | 2017-05-05 13:00 -0400 |
| Message-ID | <oeias1$v75$2@dont-email.me> |
| In reply to | #413087 |
On 5/5/2017 2:18 AM, owl wrote:
> DFS <nospam@dfs.com> wrote:
>> * connection
>> * create simple table (name, date) with PK of your choice
>> * add 1000 rows (5 random names, random dates within a range of one
>> month)
>> * extract rows (group by query: name, date, count(date))
>> * print summary to stdout
>> * save summary to .csv file
>>
>> All that stuff is easy and quick in python.
>
> Have you posted python code for this?
Not all in one place, but most of it yeah.
$ python colatest.py 1000
1000 rows inserted in 0.23s
2017-05-02 14 Moe
2017-05-12 14 Ralph
2017-05-02 13 Alice
2017-05-07 12 Norton
2017-05-16 12 Ralph
2017-05-03 11 Alice
2017-05-20 11 Alice
2017-05-20 11 Moe
2017-05-13 11 Norton
2017-05-24 11 Trixie
...
.csv file written
0.28s prog exec
--------------------------------------------------------------------
import sys, psycopg2, random, time
startmaster = time.clock()
connstr = "host='localhost' dbname='USENET' "
connstr += "user='DFS' password='evermore'"
conn = psycopg2.connect(connstr)
db = conn.cursor()
cSQL = " CREATE TABLE IF NOT EXISTS COLATEST "
cSQL += " (NID SERIAL PRIMARY KEY, NAME VARCHAR (10), DATE DATE);"
db.execute(cSQL)
startinserts = time.clock()
nrows=[]
for r in range(int(sys.argv[1])):
nm = random.choice(['Ralph','Alice','Norton','Trixie','Moe'])
dt = '5/'+str(random.randint(1,31))+'/2017'
nrows.append((nm,dt,))
db.executemany("INSERT INTO COLATEST (NAME,DATE) VALUES(%s,%s);",nrows)
conn.commit()
timed = "%.2fs"%(time.clock() - startinserts)
db.execute("SELECT COUNT(NAME) AS CNT FROM COLATEST;")
print db.fetchone()[0],'rows inserted in',timed,'\n'
cSQL = " SELECT DATE, COUNT(DATE) AS CNT, NAME "
cSQL += " FROM COLATEST "
cSQL += " GROUP BY DATE, NAME "
cSQL += " ORDER BY COUNT(DATE) DESC, NAME, DATE "
cSQL += " LIMIT 10;"
db.execute(cSQL)
with open('colatest.csv','w') as f:
f.write("Date,Count,Name\n")
for dt,cnt,nm in db.fetchall():
print dt,cnt,nm
f.write("%s,%s,%s\n" % (dt,cnt,nm))
print "...\n.csv file written"
db.close()
conn.close()
print "\n%.2fs prog exec"%(time.clock() - startmaster)
--------------------------------------------------------------------
That's 35 lines, for 'readability' and timing and feedback.
I can shorten it to 17 lines and still meet the 7 requirements.
> I guess these are the queries you want. I have to say, it's pretty
> slow. Nearly 8.5 seconds for a measly 1000 inserts.
try
BEGIN TRANSACTION;
END TRANSACTION;
or
BEGIN TRANSACTION;
conn.commit()
> Code at the bottom. You said you wanted it short, so I left
> out all error checks. (Wouldn't want to spaghettify it).
>
> -------------------- blah.c -------------------
> #include <stdio.h>
> #include <stdlib.h>
> #include <postgresql/libpq-fe.h>
> #include <string.h>
> #include <time.h>
>
> int main(int argc, char *argv[])
> {
> PGconn *conn;
> PGresult *result;
> char query[200]={0};
> int i;
> char *names[]={"ralph","alice","norton","trixie","moe"};
> FILE *fp1;
> FILE *fp2;
>
> fp1=fopen("dates.csv","w+");
> fp2=fopen("names.csv","w+");
>
> srand(time(NULL));
> conn=PQconnectdb("dbname=dummydb host=localhost");
> sprintf(query,"create table dummy (id serial primary key,"
> "name varchar(20), date date)");
> result=PQexec(conn,query);
> for(i=0;i<1000;i++)
> {
> sprintf(query,"insert into dummy(name,date) values('%s','2017-01-%d')",
> names[rand()%5],
> rand()%31 + 1);
> result=PQexec(conn,query);
> }
> sprintf(query,"select date,count(name) from dummy group by date");
> result=PQexec(conn,query);
> for(i=0;i<PQntuples(result);i++)
> {
> fprintf(fp1,"\"%s\",",PQgetvalue(result,i,0));
> fprintf(stdout,"\"%s\",",PQgetvalue(result,i,0));
> fprintf(fp1,"\"%s\"\n",PQgetvalue(result,i,1));
> fprintf(stdout,"\"%s\"\n",PQgetvalue(result,i,1));
> }
> sprintf(query,"select name,count(date) from dummy group by name");
> result=PQexec(conn,query);
> for(i=0;i<PQntuples(result);i++)
> {
> fprintf(fp2,"\"%s\",",PQgetvalue(result,i,0));
> fprintf(stdout,"\"%s\",",PQgetvalue(result,i,0));
> fprintf(fp2,"\"%s\"\n",PQgetvalue(result,i,1));
> fprintf(stdout,"\"%s\"\n",PQgetvalue(result,i,1));
> }
> PQclear(result);
> PQfinish(conn);
> fclose(fp1);
> fclose(fp2);
> return 0;
> }
> -----------------------------------------------
Thanks. Not as 'unwieldy' as I expected C/postgres to be.
[toc] | [prev] | [next] | [standalone]
| From | owl <owl@rooftop.invalid> |
|---|---|
| Date | 2017-05-05 20:14 +0000 |
| Message-ID | <zvz8v03.bu8ie@rooftop.invalid> |
| In reply to | #413167 |
DFS <nospam@dfs.com> wrote:
> On 5/5/2017 2:18 AM, owl wrote:
>> DFS <nospam@dfs.com> wrote:
>
...
> f.write("%s,%s,%s\n" % (dt,cnt,nm))
> print "...\n.csv file written"
>
> db.close()
> conn.close()
> print "\n%.2fs prog exec"%(time.clock() - startmaster)
> --------------------------------------------------------------------
>
> That's 35 lines, for 'readability' and timing and feedback.
>
> I can shorten it to 17 lines and still meet the 7 requirements.
>
Mine's about 60-70 lines now, but I'm still leaving out a lot
of error-checking code that really should be in there.
...
>
>> I guess these are the queries you want. I have to say, it's pretty
>> slow. Nearly 8.5 seconds for a measly 1000 inserts.
>
> try
>
> BEGIN TRANSACTION;
> END TRANSACTION;
> or
> BEGIN TRANSACTION;
> conn.commit()
>
Thanks. begin/end tansaction did the trick. I think I remember
that now from something else I was working on way back.
...
>> fprintf(stdout,"\"%s\"\n",PQgetvalue(result,i,1));
>> }
>> PQclear(result);
>> PQfinish(conn);
>> fclose(fp1);
>> fclose(fp2);
>> return 0;
>> }
>> -----------------------------------------------
>
> Thanks. Not as 'unwieldy' as I expected C/postgres to be.
>
>
Also got the "copy from" to work now. Needed to close the file
before doing the copy. Here's how things show on my machine now:
Python:
anon@lowtide:~/code/pg$ time ./dfs.py 1000 >/dev/null
real 0m0.226s
user 0m0.032s
sys 0m0.032s
anon@lowtide:~/code/pg$
C with inserts wrapped in transaction:
anon@lowtide:~/code/pg$ time ./glah >/dev/null
real 0m0.197s
user 0m0.016s
sys 0m0.008s
anon@lowtide:~/code/pg$
C creating a csv and using "copy from" instead of inserts:
anon@lowtide:~/code/pg$ time ./flah >/dev/null
PGRES_COMMAND_OK
real 0m0.111s
user 0m0.008s
sys 0m0.000s
anon@lowtide:~/code/pg$
Without using "copy from", the Python and C have similar performance.
The postgresql connection is the bottleneck. However, depending on the
complexity and volume of work necessary beforehand to build the queries,
C could easily stomp the crap out of Python. :)
[toc] | [prev] | [next] | [standalone]
| From | Snit <usenet@gallopinginsanity.com> |
|---|---|
| Date | 2017-05-05 13:44 -0700 |
| Message-ID | <D5323259.A3AF5%usenet@gallopinginsanity.com> |
| In reply to | #413233 |
On 5/5/17, 1:14 PM, in article zvz8v03.bu8ie@rooftop.invalid, "owl"
<owl@rooftop.invalid> wrote:
> DFS <nospam@dfs.com> wrote:
>> On 5/5/2017 2:18 AM, owl wrote:
>>> DFS <nospam@dfs.com> wrote:
>>
> ...
>
>
>> f.write("%s,%s,%s\n" % (dt,cnt,nm))
>> print "...\n.csv file written"
>>
>> db.close()
>> conn.close()
>> print "\n%.2fs prog exec"%(time.clock() - startmaster)
>> --------------------------------------------------------------------
>>
>> That's 35 lines, for 'readability' and timing and feedback.
>>
>> I can shorten it to 17 lines and still meet the 7 requirements.
>>
>
> Mine's about 60-70 lines now, but I'm still leaving out a lot
> of error-checking code that really should be in there.
>
> ...
>
>
>>
>>> I guess these are the queries you want. I have to say, it's pretty
>>> slow. Nearly 8.5 seconds for a measly 1000 inserts.
>>
>> try
>>
>> BEGIN TRANSACTION;
>> END TRANSACTION;
>> or
>> BEGIN TRANSACTION;
>> conn.commit()
>>
>
> Thanks. begin/end tansaction did the trick. I think I remember
> that now from something else I was working on way back.
> ...
>
>
>>> fprintf(stdout,"\"%s\"\n",PQgetvalue(result,i,1));
>>> }
>>> PQclear(result);
>>> PQfinish(conn);
>>> fclose(fp1);
>>> fclose(fp2);
>>> return 0;
>>> }
>>> -----------------------------------------------
>>
>> Thanks. Not as 'unwieldy' as I expected C/postgres to be.
>>
>>
>
> Also got the "copy from" to work now. Needed to close the file
> before doing the copy. Here's how things show on my machine now:
>
> Python:
> anon@lowtide:~/code/pg$ time ./dfs.py 1000 >/dev/null
>
> real 0m0.226s
> user 0m0.032s
> sys 0m0.032s
> anon@lowtide:~/code/pg$
>
> C with inserts wrapped in transaction:
> anon@lowtide:~/code/pg$ time ./glah >/dev/null
>
> real 0m0.197s
> user 0m0.016s
> sys 0m0.008s
> anon@lowtide:~/code/pg$
>
> C creating a csv and using "copy from" instead of inserts:
> anon@lowtide:~/code/pg$ time ./flah >/dev/null
> PGRES_COMMAND_OK
>
> real 0m0.111s
> user 0m0.008s
> sys 0m0.000s
> anon@lowtide:~/code/pg$
>
> Without using "copy from", the Python and C have similar performance.
> The postgresql connection is the bottleneck. However, depending on the
> complexity and volume of work necessary beforehand to build the queries,
> C could easily stomp the crap out of Python. :)
>
I just want to note that with the obsession about me and trolling of me
reduced the discussion of technology is growing. This happens to be about a
topic I have little knowledge in (programming) and I have nothing really to
add to it, but it is just great to see.
I am sincerely impressed and hope it continues.
--
Personal attacks from those who troll show their own insecurity. They cannot
use reason to show the message to be wrong so they try to feel somehow
superior by attacking the messenger.
They cling to their attacks and ignore the message time and time again.
<https://youtu.be/H4NW-Cqh308>
[toc] | [prev] | [next] | [standalone]
| From | DFS <nospam@dfs.com> |
|---|---|
| Date | 2017-05-05 23:51 -0400 |
| Message-ID | <oejgv5$btg$2@dont-email.me> |
| In reply to | #413233 |
On 5/5/2017 4:14 PM, owl wrote:
> C creating a csv and using "copy from" instead of inserts:
> anon@lowtide:~/code/pg$ time ./flah >/dev/null
> PGRES_COMMAND_OK
>
> real 0m0.111s
> user 0m0.008s
> sys 0m0.000s
> anon@lowtide:~/code/pg$
>
> Without using "copy from", the Python and C have similar performance.
> The postgresql connection is the bottleneck. However, depending on the
> complexity and volume of work necessary beforehand to build the queries,
> C could easily stomp the crap out of Python. :)
Write .csv then COPY FROM it is sneaky. You must've gotten desperate
when it was 8 seconds to insert 1K rows.
But I didn't specify INSERTs so it's legal for the challenge.
After doing the COPY FROM, are you writing back out the summary data to
a new .csv (last requirement)?
my previous python: 0.28s (0.226s on your system)
new python (writes random data to file, imports it via StringIO module)
$ python colatest3.py 1000
2017-05-25 45
2017-05-03 44
2017-05-15 44
2017-05-21 40
2017-05-18 39
2017-05-01 38
2017-05-27 38
2017-05-30 38
2017-05-26 36
2017-05-13 35
...
.csv file written
0.127s prog exec
50% faster. hoo-ahh.
Please test it on your system and see what kind of beatdown it puts on
your C (note: to save bytes I'm only outputting 10 rows of summary
data, as did the previous code)
colatest3.py (23 lines)
import sys, psycopg2, random, StringIO, time
startmaster = time.clock()
conn = psycopg2.connect("host='localhost' dbname='USENET' user='DFS'
password='evermore'")
db = conn.cursor()
db.execute(" CREATE TABLE IF NOT EXISTS COLATEST (NID SERIAL PRIMARY
KEY, NAME VARCHAR (10), DATE DATE);")
db.execute("DELETE FROM COLATEST;")
testfile = "D:\\colatest.csv"
with open(testfile,'w') as f:
for r in range(int(sys.argv[1])):
f.write("%d,%s,%s\n" %
(r,random.choice(['Ralph','Alice','Norton','Trixie','Moe']),'5/'+str(random.randint(1,31))+'/2017'))
io = open(testfile, 'r')
db.copy_from(io,'COLATEST',',')
io.close()
db.execute("SELECT DATE, COUNT(DATE) AS CNT FROM COLATEST GROUP BY DATE
ORDER BY COUNT(DATE) DESC, DATE LIMIT 10;")
with open(testfile,'w') as f:
f.write("Date,Count\n")
for dt,cnt in db.fetchall():
print dt,cnt
f.write("%s,%s\n" % (dt,cnt))
print "...\n.csv file written"
db.close()
conn.close()
print "\n%.3fs prog exec"%(time.clock() - startmaster)
[toc] | [prev] | [next] | [standalone]
| From | owl <owl@rooftop.invalid> |
|---|---|
| Date | 2017-05-06 04:48 +0000 |
| Message-ID | <pa80b.3yvevv45gv7j@rooftop.invalid> |
| In reply to | #413275 |
DFS <nospam@dfs.com> wrote: > On 5/5/2017 4:14 PM, owl wrote: > >> C creating a csv and using "copy from" instead of inserts: >> anon@lowtide:~/code/pg$ time ./flah >/dev/null >> PGRES_COMMAND_OK >> >> real 0m0.111s >> user 0m0.008s >> sys 0m0.000s >> anon@lowtide:~/code/pg$ >> >> Without using "copy from", the Python and C have similar performance. >> The postgresql connection is the bottleneck. However, depending on the >> complexity and volume of work necessary beforehand to build the queries, >> C could easily stomp the crap out of Python. :) > > > Write .csv then COPY FROM it is sneaky. You must've gotten desperate > when it was 8 seconds to insert 1K rows. > heh > But I didn't specify INSERTs so it's legal for the challenge. > > After doing the COPY FROM, are you writing back out the summary data to > a new .csv (last requirement)? > Yes. Actually doing two queries and writing to a names.csv and dates.csv. anon@lowtide:~/code/pg$ cat names.csv "trixie","221" "norton","186" "ralph","210" "alice","179" "moe","204" anon@lowtide:~/code/pg$ head dates.csv "2017-01-01","29" "2017-01-13","29" "2017-01-26","38" "2017-01-04","41" "2017-01-03","24" "2017-01-30","30" "2017-01-27","36" "2017-01-19","32" "2017-01-10","45" "2017-01-15","32" anon@lowtide:~/code/pg$ > my previous python: 0.28s (0.226s on your system) > new python (writes random data to file, imports it via StringIO module) > > $ python colatest3.py 1000 > 2017-05-25 45 > 2017-05-03 44 > 2017-05-15 44 > 2017-05-21 40 > 2017-05-18 39 > 2017-05-01 38 > 2017-05-27 38 > 2017-05-30 38 > 2017-05-26 36 > 2017-05-13 35 > ... > .csv file written > > 0.127s prog exec > > > 50% faster. hoo-ahh. > > Please test it on your system and see what kind of beatdown it puts on > your C (note: to save bytes I'm only outputting 10 rows of summary > data, as did the previous code) > When I run it, it doesn't write to the db and there is no record of the csv file that contained the 1000 records, which appears to be the same file you write your summary to, so... The first code you had worked for writing to the db. Also, the timer function you are using shows about 1/10 the usage that "time" reports as "real". anon@lowtide:~/code/pg$ time ./dfs_copy 1000 2017-05-27 44 2017-05-04 42 2017-05-21 41 2017-05-26 40 2017-05-01 39 2017-05-30 39 2017-05-12 37 2017-05-15 36 2017-05-20 35 2017-05-05 34 ... .csv file written 0.020s prog exec real 0m0.231s user 0m0.096s sys 0m0.008s anon@lowtide:~/code/pg$ dummydb=# \dt No relations found. dummydb=# The only changes I made were to dbname and the path for colatest.csv.
[toc] | [prev] | [next] | [standalone]
| From | DFS <nospam@dfs.com> |
|---|---|
| Date | 2017-05-06 10:06 -0400 |
| Message-ID | <oekkrm$eco$2@dont-email.me> |
| In reply to | #413279 |
On 5/6/2017 12:48 AM, owl wrote: > When I run it, it doesn't write to the db and there is no > record of the csv file that contained the 1000 records, > which appears to be the same file you write your summary > to, so... > > The first code you had worked for writing to the db. > Also, the timer function you are using shows about 1/10 > the usage that "time" reports as "real". > > dummydb=# \dt > No relations found. > dummydb=# > > The only changes I made were to dbname and the path for colatest.csv. Thanks for testing it. Probably just need a conn.commit() statement at line 14, right after the io.close() call. Later today I'll fix and update it to run 2 summaries/2 csv for a better comparison.
[toc] | [prev] | [next] | [standalone]
| From | owl <owl@rooftop.invalid> |
|---|---|
| Date | 2017-05-06 15:50 +0000 |
| Message-ID | <acv893qf.jig@rooftop.invalid> |
| In reply to | #413327 |
DFS <nospam@dfs.com> wrote:
> On 5/6/2017 12:48 AM, owl wrote:
>
>> When I run it, it doesn't write to the db and there is no
>> record of the csv file that contained the 1000 records,
>> which appears to be the same file you write your summary
>> to, so...
>>
>> The first code you had worked for writing to the db.
>> Also, the timer function you are using shows about 1/10
>> the usage that "time" reports as "real".
>
>>
>> dummydb=# \dt
>> No relations found.
>> dummydb=#
>>
>> The only changes I made were to dbname and the path for colatest.csv.
>
>
> Thanks for testing it.
>
> Probably just need a conn.commit() statement at line 14, right after the
> io.close() call.
>
> Later today I'll fix and update it to run 2 summaries/2 csv for a better
> comparison.
>
1,000,000 records:
Python 21.402 sec
C 14.495 sec
anon@lowtide:~/code/pg$ time ./dfscopy 1000000
2017-05-11 32541
2017-05-03 32530
2017-05-09 32480
2017-05-23 32440
2017-05-14 32414
2017-05-13 32393
2017-05-06 32386
2017-05-25 32380
2017-05-26 32379
2017-05-17 32355
...
.csv file written
3.340s prog exec
real 0m21.402s
user 0m3.360s
sys 0m0.072s
anon@lowtide:~/code/pg$ time ./tlah 1000000
NOTICE: relation "dummy" already exists, skipping
2017-01-04 32739
2017-01-25 32596
2017-01-24 32552
2017-01-06 32490
2017-01-15 32448
2017-01-13 32438
2017-01-29 32437
2017-01-05 32419
2017-01-02 32384
2017-01-07 32352
real 0m14.495s
user 0m0.324s
sys 0m0.028s
anon@lowtide:~/code/pg$
dummydb=# \dt
List of relations
Schema | Name | Type | Owner
--------+----------+-------+-------
public | colatest | table | anon
public | dummy | table | anon
(2 rows)
dummydb=# select count(*) from colatest;
count
---------
1000000
(1 row)
dummydb=# select count(*) from dummy;
count
---------
1000000
(1 row)
dummydb=#
----------------------- tlah.c -------------------------
#include <stdio.h>
#include <stdlib.h>
#include <postgresql/libpq-fe.h>
#include <string.h>
#include <time.h>
int main(int argc, char *argv[])
{
PGconn *conn;
PGresult *result;
char query[200]={0};
int i;
char *names[]={"ralph","alice","norton","trixie","moe"};
FILE *fp3;
FILE *fp4;
fp3=fopen("jah.csv","w+");
fp4=fopen("report.csv","w+");
if(argc!=2)
{
fprintf(stderr,"usage: %s <count>\n",argv[0]);
exit(1);
}
srand(time(NULL));
conn=PQconnectdb("dbname=dummydb host=localhost");
sprintf(query,"create table if not exists dummy (id serial primary key,"
"name varchar(20), date date)");
result=PQexec(conn,query);
PQexec(conn,"delete from dummy");
for(i=0;i<atoi(argv[1]);i++)
{
fprintf(fp3,"%s, 2017-01-%02d\n",names[rand()%5],rand()%31+1);
}
fclose(fp3);
result=PQexec(conn,"copy dummy(name,date) from \
'/home/anon/code/pg/jah.csv' with csv");
sprintf(query,"select date,count(date) as cnt from dummy \
group by date order by count(date) desc, date limit 10;");
result=PQexec(conn,query);
for(i=0;i<PQntuples(result);i++)
{
fprintf(fp4,"%s %s\n",PQgetvalue(result,i,0), PQgetvalue(result,i,0));
fprintf(stdout,"%s %s\n",PQgetvalue(result,i,0), PQgetvalue(result,i,1));
}
PQclear(result);
PQfinish(conn);
fclose(fp4);
return 0;
}
--------------------------------------------------------
[toc] | [prev] | [next] | [standalone]
| From | DFS <nospam@dfs.com> |
|---|---|
| Date | 2017-05-07 13:09 -0400 |
| Message-ID | <oenk6u$uf8$2@dont-email.me> |
| In reply to | #413331 |
On 5/6/2017 11:50 AM, owl wrote:
> 1,000,000 records:
> Python 21.402 sec
> C 14.495 sec
Only 50% slower? That's called a victory for python...
> anon@lowtide:~/code/pg$ time ./dfscopy 1000000
> 3.340s prog exec
The python timer says 3.340s and Linux reports 21.402s?
Wonder what's going on there.
Back at the home base:
$ python colatest.py 1000000
2017-05-12 32673
2017-05-25 32670
2017-05-29 32541
2017-05-04 32462
2017-05-16 32452
2017-05-19 32399
2017-05-17 32346
2017-05-07 32339
2017-05-08 32329
2017-05-30 32323
24.299s
$ tcc owlpg.c libpq.dll -o paschetti.exe
$ paschetti 1000000
2017-01-26 32717
2017-01-20 32570
2017-01-25 32447
2017-01-13 32440
2017-01-15 32420
2017-01-16 32385
2017-01-09 32381
2017-01-10 32368
2017-01-31 32367
2017-01-06 32363
13.626s
few mods: Windows timer code, paths, dbname, drop table if exists
fixed bug line 47: you were outputting the date 2x to report.csv
owlpg.c
---------------------------------------------------------------------
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <C:\Program Files\PostgreSQL\9.6\include\libpq-fe.h>
#include <Windows.h>
int main(int argc, char *argv[])
{
LARGE_INTEGER frequency,start,end;
double elapsed=0;
QueryPerformanceFrequency(&frequency);
QueryPerformanceCounter(&start);
PGconn *conn;
PGresult *result;
char query[200]={0};
int i;
char *names[]={"ralph","alice","norton","trixie","moe"};
FILE *fp3;
FILE *fp4;
fp3=fopen("jah.csv","w+");
fp4=fopen("report.csv","w+");
if(argc!=2)
{
fprintf(stderr,"usage: %s <count>\n",argv[0]);
exit(1);
}
srand(time(NULL));
conn=PQconnectdb("dbname=USENET host=localhost user='DFS'
password='evermore'");
PQexec(conn,"drop table if exists dummy");
sprintf(query,"create table if not exists dummy (id serial primary
key, name varchar(20), date date)");
result=PQexec(conn,query);
for(i=0;i<atoi(argv[1]);i++)
{
fprintf(fp3,"%s, 2017-01-%02d\n",names[rand()%5],rand()%31+1);
}
fclose(fp3);
result=PQexec(conn,"copy dummy(name,date) from
'D:\\DFS\\Personal\\computer\\development\\C\\owl\\jah.csv' with csv");
sprintf(query,"select date,count(date) as cnt from dummy group by date
order by count(date) desc, date limit 10;");
result=PQexec(conn,query);
for(i=0;i<PQntuples(result);i++)
{
fprintf(fp4,"%s %s\n",PQgetvalue(result,i,0), PQgetvalue(result,i,1));
fprintf(stdout,"%s %s\n",PQgetvalue(result,i,0),
PQgetvalue(result,i,1));
}
PQclear(result);
PQfinish(conn);
fclose(fp4);
QueryPerformanceCounter(&end);
elapsed = (end.QuadPart - start.QuadPart) / (double)frequency.QuadPart;
printf("%.3fs",elapsed);
return 0;
}
---------------------------------------------------------------------
> for(i=0;i<atoi(argv[1]);i++)
> {
> fprintf(fp3,"%s, 2017-01-%02d\n",names[rand()%5],rand()%31+1);
> }
> fclose(fp3);
Here's where C blasts by python - elsewhere it's constrained by the db.
[toc] | [prev] | [next] | [standalone]
| From | owl <owl@rooftop.invalid> |
|---|---|
| Date | 2017-05-07 18:39 +0000 |
| Message-ID | <zx5cvu93.ve0@rooftop.invalid> |
| In reply to | #413509 |
DFS <nospam@dfs.com> wrote:
> On 5/6/2017 11:50 AM, owl wrote:
>
>
>> for(i=0;i<atoi(argv[1]);i++)
>> {
>> fprintf(fp3,"%s, 2017-01-%02d\n",names[rand()%5],rand()%31+1);
>> }
>> fclose(fp3);
>
> Here's where C blasts by python - elsewhere it's constrained by the db.
Yeah, performance would depend on what you had to do to prepare the
data beforehand. Times are very close for just bringing in the
1,000,000 records:
anon@lowtide:~/code/pg$ time ./ulah
NOTICE: table "dummy" does not exist, skipping
2017-01-02 32686
2017-01-20 32665
2017-01-31 32620
2017-01-12 32548
2017-01-30 32536
2017-01-01 32473
2017-01-25 32401
2017-01-09 32377
2017-01-21 32371
2017-01-17 32356
real 0m8.558s
user 0m0.024s
sys 0m0.008s
anon@lowtide:~/code/pg$ time ./dfscopyonly
2017-01-02 32686
2017-01-20 32665
2017-01-31 32620
2017-01-12 32548
2017-01-30 32536
2017-01-01 32473
2017-01-25 32401
2017-01-09 32377
2017-01-21 32371
2017-01-17 32356
...
.csv file written
0.048s prog exec
real 0m8.587s
user 0m0.080s
sys 0m0.040s
anon@lowtide:~/code/pg$
On an unrelated note, this is nice to know if you like C:
https://www.postgresql.org/docs/9.4/static/xfunc-c.html
[toc] | [prev] | [next] | [standalone]
| From | owl <owl@rooftop.invalid> |
|---|---|
| Date | 2017-05-09 22:01 +0000 |
| Message-ID | <ac89v3.agh8tt@rooftop.invalid> |
| In reply to | #413509 |
DFS <nospam@dfs.com> wrote:
> On 5/6/2017 11:50 AM, owl wrote:
>
>> 1,000,000 records:
>> Python 21.402 sec
>> C 14.495 sec
>
> Only 50% slower? That's called a victory for python...
>
>
>> anon@lowtide:~/code/pg$ time ./dfscopy 1000000
>> 3.340s prog exec
>
> The python timer says 3.340s and Linux reports 21.402s?
> Wonder what's going on there.
>
>
>
> Back at the home base:
> $ python colatest.py 1000000
> 2017-05-12 32673
> 2017-05-25 32670
> 2017-05-29 32541
> 2017-05-04 32462
> 2017-05-16 32452
> 2017-05-19 32399
> 2017-05-17 32346
> 2017-05-07 32339
> 2017-05-08 32329
> 2017-05-30 32323
> 24.299s
>
> $ tcc owlpg.c libpq.dll -o paschetti.exe
> $ paschetti 1000000
> 2017-01-26 32717
> 2017-01-20 32570
> 2017-01-25 32447
> 2017-01-13 32440
> 2017-01-15 32420
> 2017-01-16 32385
> 2017-01-09 32381
> 2017-01-10 32368
> 2017-01-31 32367
> 2017-01-06 32363
> 13.626s
>
>
>
> few mods: Windows timer code, paths, dbname, drop table if exists
> fixed bug line 47: you were outputting the date 2x to report.csv
>
> owlpg.c
> ---------------------------------------------------------------------
> #include <stdio.h>
> #include <stdlib.h>
> #include <string.h>
> #include <C:\Program Files\PostgreSQL\9.6\include\libpq-fe.h>
> #include <Windows.h>
>
> int main(int argc, char *argv[])
> {
>
> LARGE_INTEGER frequency,start,end;
> double elapsed=0;
> QueryPerformanceFrequency(&frequency);
> QueryPerformanceCounter(&start);
>
> PGconn *conn;
> PGresult *result;
> char query[200]={0};
> int i;
> char *names[]={"ralph","alice","norton","trixie","moe"};
> FILE *fp3;
> FILE *fp4;
> fp3=fopen("jah.csv","w+");
> fp4=fopen("report.csv","w+");
> if(argc!=2)
> {
> fprintf(stderr,"usage: %s <count>\n",argv[0]);
> exit(1);
> }
> srand(time(NULL));
>
> conn=PQconnectdb("dbname=USENET host=localhost user='DFS'
> password='evermore'");
> PQexec(conn,"drop table if exists dummy");
> sprintf(query,"create table if not exists dummy (id serial primary
> key, name varchar(20), date date)");
> result=PQexec(conn,query);
>
> for(i=0;i<atoi(argv[1]);i++)
> {
> fprintf(fp3,"%s, 2017-01-%02d\n",names[rand()%5],rand()%31+1);
> }
> fclose(fp3);
> result=PQexec(conn,"copy dummy(name,date) from
> 'D:\\DFS\\Personal\\computer\\development\\C\\owl\\jah.csv' with csv");
>
> sprintf(query,"select date,count(date) as cnt from dummy group by date
> order by count(date) desc, date limit 10;");
> result=PQexec(conn,query);
> for(i=0;i<PQntuples(result);i++)
> {
> fprintf(fp4,"%s %s\n",PQgetvalue(result,i,0), PQgetvalue(result,i,1));
> fprintf(stdout,"%s %s\n",PQgetvalue(result,i,0),
> PQgetvalue(result,i,1));
> }
> PQclear(result);
> PQfinish(conn);
> fclose(fp4);
>
> QueryPerformanceCounter(&end);
> elapsed = (end.QuadPart - start.QuadPart) / (double)frequency.QuadPart;
> printf("%.3fs",elapsed);
>
> return 0;
> }
> ---------------------------------------------------------------------
>
>
>
>> for(i=0;i<atoi(argv[1]);i++)
>> {
>> fprintf(fp3,"%s, 2017-01-%02d\n",names[rand()%5],rand()%31+1);
>> }
>> fclose(fp3);
>
> Here's where C blasts by python - elsewhere it's constrained by the db.
First stab at a GUI for this 1,000,000 record database (expanded to fifteen
names) using xforms:
https://vid.me/Yq32
The xforms package comes with a crapload of demos for most widgets,
with source code, along with at GUI form builder. I played with the
builder for a while but ended up just tweaking one of the demos to work
with postgres. It's nowhere near complete, but shows how easy it is to
get started with xforms.
http://xforms-toolkit.org/
[toc] | [prev] | [next] | [standalone]
| From | Marek Novotny <marek.novotny@marspolar.com> |
|---|---|
| Date | 2017-05-09 17:17 -0500 |
| Message-ID | <9L-dnYpyg_Floo_EnZ2dnUU7-dednZ2d@giganews.com> |
| In reply to | #414159 |
On 2017-05-09, owl <owl@rooftop.invalid> wrote: // snip > First stab at a GUI for this 1,000,000 record database (expanded to fifteen > names) using xforms: > > https://vid.me/Yq32 > > The xforms package comes with a crapload of demos for most widgets, > with source code, along with at GUI form builder. I played with the > builder for a while but ended up just tweaking one of the demos to work > with postgres. It's nowhere near complete, but shows how easy it is to > get started with xforms. > > http://xforms-toolkit.org/ nicely done, Owl. -- Marek Novotny https://github.com/marek-novotny
[toc] | [prev] | [next] | [standalone]
| From | owl <owl@rooftop.invalid> |
|---|---|
| Date | 2017-05-09 22:45 +0000 |
| Message-ID | <abzvc03.abyt@rooftop.invalid> |
| In reply to | #414161 |
Marek Novotny <marek.novotny@marspolar.com> wrote: > On 2017-05-09, owl <owl@rooftop.invalid> wrote: > > // snip > >> First stab at a GUI for this 1,000,000 record database (expanded to fifteen >> names) using xforms: >> >> https://vid.me/Yq32 >> >> The xforms package comes with a crapload of demos for most widgets, >> with source code, along with at GUI form builder. I played with the >> builder for a while but ended up just tweaking one of the demos to work >> with postgres. It's nowhere near complete, but shows how easy it is to >> get started with xforms. >> >> http://xforms-toolkit.org/ > > nicely done, Owl. > Thanks. This xforms stuff is kinda fun. It's so clean and simple. I'm tempted to dig deeper into it.
[toc] | [prev] | [next] | [standalone]
| From | Snit <usenet@gallopinginsanity.com> |
|---|---|
| Date | 2017-05-09 16:21 -0700 |
| Message-ID | <D5379D23.A448D%usenet@gallopinginsanity.com> |
| In reply to | #414167 |
On 5/9/17, 3:45 PM, in article abzvc03.abyt@rooftop.invalid, "owl" <owl@rooftop.invalid> wrote: > Marek Novotny <marek.novotny@marspolar.com> wrote: >> On 2017-05-09, owl <owl@rooftop.invalid> wrote: >> >> // snip >> >>> First stab at a GUI for this 1,000,000 record database (expanded to fifteen >>> names) using xforms: >>> >>> https://vid.me/Yq32 >>> >>> The xforms package comes with a crapload of demos for most widgets, >>> with source code, along with at GUI form builder. I played with the >>> builder for a while but ended up just tweaking one of the demos to work >>> with postgres. It's nowhere near complete, but shows how easy it is to >>> get started with xforms. >>> >>> http://xforms-toolkit.org/ >> >> nicely done, Owl. >> > > Thanks. This xforms stuff is kinda fun. It's so clean and simple. > I'm tempted to dig deeper into it. > Makes me want to get FileMaker Pro 16... I am still on 8.x. Sigh. -- Personal attacks from those who troll show their own insecurity. They cannot use reason to show the message to be wrong so they try to feel somehow superior by attacking the messenger. They cling to their attacks and ignore the message time and time again. <https://youtu.be/H4NW-Cqh308>
[toc] | [prev] | [next] | [standalone]
| From | Marek Novotny <marek.novotny@marspolar.com> |
|---|---|
| Date | 2017-05-09 19:20 -0500 |
| Message-ID | <2c2dneG7t-50wY_EnZ2dnUU7-ROdnZ2d@giganews.com> |
| In reply to | #414167 |
On 2017-05-09, owl <owl@rooftop.invalid> wrote: > Marek Novotny <marek.novotny@marspolar.com> wrote: >> On 2017-05-09, owl <owl@rooftop.invalid> wrote: >> >> // snip >> >>> First stab at a GUI for this 1,000,000 record database (expanded to fifteen >>> names) using xforms: >>> >>> https://vid.me/Yq32 >>> >>> The xforms package comes with a crapload of demos for most widgets, >>> with source code, along with at GUI form builder. I played with the >>> builder for a while but ended up just tweaking one of the demos to work >>> with postgres. It's nowhere near complete, but shows how easy it is to >>> get started with xforms. >>> >>> http://xforms-toolkit.org/ >> >> nicely done, Owl. >> > > Thanks. This xforms stuff is kinda fun. It's so clean and simple. > I'm tempted to dig deeper into it. Tempted to look into it myself. Trying not to get sidetracked. Looks pretty good. -- Marek Novotny https://github.com/marek-novotny
[toc] | [prev] | [next] | [standalone]
| From | owl <owl@rooftop.invalid> |
|---|---|
| Date | 2017-05-10 03:01 +0000 |
| Message-ID | <zcvi309tg.bki@rooftop.invalid> |
| In reply to | #414170 |
Marek Novotny <marek.novotny@marspolar.com> wrote: > On 2017-05-09, owl <owl@rooftop.invalid> wrote: >> Marek Novotny <marek.novotny@marspolar.com> wrote: >>> On 2017-05-09, owl <owl@rooftop.invalid> wrote: >>> >>> // snip >>> >>>> First stab at a GUI for this 1,000,000 record database (expanded to fifteen >>>> names) using xforms: >>>> >>>> https://vid.me/Yq32 >>>> >>>> The xforms package comes with a crapload of demos for most widgets, >>>> with source code, along with at GUI form builder. I played with the >>>> builder for a while but ended up just tweaking one of the demos to work >>>> with postgres. It's nowhere near complete, but shows how easy it is to >>>> get started with xforms. >>>> >>>> http://xforms-toolkit.org/ >>> >>> nicely done, Owl. >>> >> >> Thanks. This xforms stuff is kinda fun. It's so clean and simple. >> I'm tempted to dig deeper into it. > > Tempted to look into it myself. Trying not to get sidetracked. Looks > pretty good. > Now with 10,000,000 records: http://imgur.com/a/NM3bc How many does it take to break Access? :)
[toc] | [prev] | [next] | [standalone]
| From | Marek Novotny <marek.novotny@marspolar.com> |
|---|---|
| Date | 2017-05-09 22:04 -0500 |
| Message-ID | <l8idnXYS6MDRHo_EnZ2dnUU7-a-dnZ2d@giganews.com> |
| In reply to | #414175 |
On 2017-05-10, owl <owl@rooftop.invalid> wrote: > Marek Novotny <marek.novotny@marspolar.com> wrote: >> On 2017-05-09, owl <owl@rooftop.invalid> wrote: >>> Marek Novotny <marek.novotny@marspolar.com> wrote: >>>> On 2017-05-09, owl <owl@rooftop.invalid> wrote: >>>> >>>> // snip >>>> >>>>> First stab at a GUI for this 1,000,000 record database (expanded to fifteen >>>>> names) using xforms: >>>>> >>>>> https://vid.me/Yq32 >>>>> >>>>> The xforms package comes with a crapload of demos for most widgets, >>>>> with source code, along with at GUI form builder. I played with the >>>>> builder for a while but ended up just tweaking one of the demos to work >>>>> with postgres. It's nowhere near complete, but shows how easy it is to >>>>> get started with xforms. >>>>> >>>>> http://xforms-toolkit.org/ >>>> >>>> nicely done, Owl. >>>> >>> >>> Thanks. This xforms stuff is kinda fun. It's so clean and simple. >>> I'm tempted to dig deeper into it. >> >> Tempted to look into it myself. Trying not to get sidetracked. Looks >> pretty good. >> > > Now with 10,000,000 records: > http://imgur.com/a/NM3bc > How many does it take to break Access? :) This is great. I guess we're going to have to call you an X-man. ;) -- Marek Novotny https://github.com/marek-novotny
[toc] | [prev] | [next] | [standalone]
Page 3 of 9 — ← Prev page 1 2 [3] 4 5 6 7 8 9 Next page →
Back to top | Article view | comp.os.linux.advocacy
csiph-web