Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > comp.os.linux.advocacy > #413239
| From | owl <owl@rooftop.invalid> |
|---|---|
| Newsgroups | comp.os.linux.advocacy |
| Subject | Re: Snit hitting the glue bag early today |
| Date | 2017-05-05 20:49 +0000 |
| Organization | O.W.L. |
| Message-ID | <szc99b83.abyu@rooftop.invalid> (permalink) |
| References | (10 earlier) <acv993.by93ap@rooftop.invalid> <czcvz8993.ahu@rooftop.invalid> <sandman-e69f935d64a51f758f8e7fedabd1c619@individual.net> <avzc9v03.ahu@rooftop.invalid> <szc9v903.vu@rooftop.invalid> |
owl <owl@rooftop.invalid> wrote:
> owl <owl@rooftop.invalid> wrote:
>> Sandman <mr@sandman.net> wrote:
>>> }
>>> if ($results = $sql->query("select name, COUNT(date) AS nr from names GROUP BY name")) {
>>> while ($row = $results->fetchArray()) printf("%-10s: %d\n", $row["name"], $row["nr"]);
>>> }
>>>
>>
>> I agree it seems really slow at 8+ seconds. That could be from my code.
>> I have very little experience with database stuff. But from what I
>> see online, sqlite is much faster with inserts that postgresql. So,
>> I guess it could be either or both.
>>
>
> OK, I found that I can use the COPY FROM command to load a csv file
> into the table rather than using multiple inserts, so I have the code
> writing to a csv, then doing that. That brings the run time down to
> about 0.05 seconds. The problem is that the thing fails with a fatal
> error more often than it works, and when it "works" it only brings in
> about 960 some odd records. The exact same query on the exact same
> file works perfectly 100% of the time when done from a psql terminal,
> whether done as 'copy' or '\copy'. Debugging it is driving me crazy.
>
> But anyway, the "almost" doing it (with 960 records) gives some idea
> of the how fast it should be if I ever get it to work right. I'm
> leaning toward it being a UID issue, since the 'server' version of
> the command (the one I'm using in the code, since '\copy' won't work
> that way) runs as the UID of the postgres user. The file is already
> world-readable though...
>
The solution here was to close the file before doing the "copy from".
Back to comp.os.linux.advocacy | Previous | Next — Previous in thread | Next in thread | Find similar
Re: Snit hitting the glue bag early today DFS <nospam@dfs.com> - 2017-05-04 09:29 -0400
Re: Snit hitting the glue bag early today Sandman <mr@sandman.net> - 2017-05-04 16:42 +0000
Re: Snit hitting the glue bag early today owl <owl@rooftop.invalid> - 2017-05-04 20:06 +0000
Re: Snit hitting the glue bag early today owl <owl@rooftop.invalid> - 2017-05-04 20:12 +0000
Re: Snit hitting the glue bag early today chrisv <chrisv@nospam.invalid> - 2017-05-05 06:52 -0500
Re: Snit hitting the glue bag early today Marek Novotny <marek.novotny@marspolar.com> - 2017-05-05 10:32 -0500
Re: Snit hitting the glue bag early today DFS <nospam@dfs.com> - 2017-05-05 13:22 -0400
Re: Snit hitting the glue bag early today DFS <nospam@dfs.com> - 2017-05-05 13:01 -0400
Re: Snit hitting the glue bag early today Steve Carroll <fretwizzer@gmail.com> - 2017-05-04 13:16 -0700
Re: Snit hitting the glue bag early today Sandman <mr@sandman.net> - 2017-05-04 21:36 +0000
Re: Snit hitting the glue bag early today DFS <nospam@dfs.com> - 2017-05-06 09:44 -0400
Re: Snit hitting the glue bag early today owl <owl@rooftop.invalid> - 2017-05-04 19:49 +0000
Re: Snit hitting the glue bag early today DFS <nospam@dfs.com> - 2017-05-04 19:28 -0400
Re: Snit hitting the glue bag early today owl <owl@rooftop.invalid> - 2017-05-05 02:56 +0000
Re: Snit hitting the glue bag early today DFS <nospam@dfs.com> - 2017-05-04 23:40 -0400
Re: Snit hitting the glue bag early today owl <owl@rooftop.invalid> - 2017-05-05 06:18 +0000
Re: Snit hitting the glue bag early today owl <owl@rooftop.invalid> - 2017-05-05 06:34 +0000
Re: Snit hitting the glue bag early today Sandman <mr@sandman.net> - 2017-05-05 07:07 +0000
Re: Snit hitting the glue bag early today owl <owl@rooftop.invalid> - 2017-05-05 07:29 +0000
Re: Snit hitting the glue bag early today owl <owl@rooftop.invalid> - 2017-05-05 16:48 +0000
Re: Snit hitting the glue bag early today owl <owl@rooftop.invalid> - 2017-05-05 20:49 +0000
Re: Snit hitting the glue bag early today DFS <nospam@dfs.com> - 2017-05-05 13:02 -0400
Re: Snit hitting the glue bag early today Sandman <mr@sandman.net> - 2017-05-05 19:12 +0000
Re: Snit hitting the glue bag early today Snit <usenet@gallopinginsanity.com> - 2017-05-05 13:45 -0700
Re: Snit hitting the glue bag early today DFS <nospam@dfs.com> - 2017-05-05 17:09 -0400
Re: Snit hitting the glue bag early today Sandman <mr@sandman.net> - 2017-05-05 21:41 +0000
Re: Snit hitting the glue bag early today DFS <nospam@dfs.com> - 2017-05-05 13:00 -0400
Re: Snit hitting the glue bag early today owl <owl@rooftop.invalid> - 2017-05-05 20:14 +0000
Re: Snit hitting the glue bag early today Snit <usenet@gallopinginsanity.com> - 2017-05-05 13:44 -0700
Re: Snit hitting the glue bag early today DFS <nospam@dfs.com> - 2017-05-05 23:51 -0400
Re: Snit hitting the glue bag early today owl <owl@rooftop.invalid> - 2017-05-06 04:48 +0000
Re: Snit hitting the glue bag early today DFS <nospam@dfs.com> - 2017-05-06 10:06 -0400
Re: Snit hitting the glue bag early today owl <owl@rooftop.invalid> - 2017-05-06 15:50 +0000
Re: Snit hitting the glue bag early today DFS <nospam@dfs.com> - 2017-05-07 13:09 -0400
Re: Snit hitting the glue bag early today owl <owl@rooftop.invalid> - 2017-05-07 18:39 +0000
Re: Snit hitting the glue bag early today Chris Ahlstrom <OFeem1987@teleworm.us> - 2017-05-05 05:28 -0400
Re: Snit hitting the glue bag early today DFS <nospam@dfs.com> - 2017-05-05 13:02 -0400
csiph-web