Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.databases.postgresql > #388

Re: scan into many tables

Show all headers | View raw

On 2012-07-13, Greg Hennessy <greg.hennessy@cox.net> wrote:
> I have a rather large table (3 TB) that I need to split up.
> I can do a bunch of 
> select yada into tmp_00 from centroids where value=0;
> select yada into tmp_01 from centroids where value=1;
> select yada into tmp_02 from centroids where value=2;
>
> but since a scan on the database takes about 40 hours,
> and I have values between 0 and 48, I'm looking at multiple 
> weeks just to copy things into the smaller tables.
>
> Is there a way to do this while going through the
> database once? 

set up table partitioning (or something like it) 
and have your insert trigger function do the 
demultiplexing,

on the other hand that's a lot of data to handle in a single
transaction. dumping it and spltting it externally may work
better.

-- 
⚂⚃ 100% natural

--- Posted via news://freenews.netfront.net/ - Complaints to news@netfront.net ---

Back to comp.databases.postgresql | Previous | Next — Previous in thread | Find similar | Unroll thread

Thread

scan into many tables Greg Hennessy <greg.hennessy@cox.net> - 2012-07-13 16:11 +0000
  Re: scan into many tables Hans Castorp <REWYRLXHEGHO@spammotel.com> - 2012-07-13 18:37 +0200
    Re: scan into many tables Greg Hennessy <greg.hennessy@cox.net> - 2012-07-13 18:54 +0000
    Re: scan into many tables Greg Hennessy <greg.hennessy@cox.net> - 2012-07-13 20:48 +0000
  Re: scan into many tables Matthew Woodcraft <mattheww@chiark.greenend.org.uk> - 2012-07-13 17:51 +0100
    Re: scan into many tables Greg Hennessy <greg.hennessy@cox.net> - 2012-07-13 19:00 +0000
      Re: scan into many tables Matthew Woodcraft <mattheww@chiark.greenend.org.uk> - 2012-07-13 22:15 +0100
      Re: scan into many tables Graham Murray <newspost@gmurray.org.uk> - 2012-07-22 11:46 +0100
        Re: scan into many tables Matthew Woodcraft <mattheww@chiark.greenend.org.uk> - 2012-07-22 13:09 +0100
          Re: scan into many tables Robert Klemme <shortcutter@googlemail.com> - 2012-07-22 18:55 +0200
            Re: scan into many tables Matthew Woodcraft <mattheww@chiark.greenend.org.uk> - 2012-07-22 21:07 +0100
          Re: scan into many tables Don Y <this@isnotme.com> - 2012-07-22 10:30 -0700
            Re: scan into many tables Hans Castorp <REWYRLXHEGHO@spammotel.com> - 2012-07-22 20:24 +0200
              Re: scan into many tables Don Y <this@isnotme.com> - 2012-07-22 12:54 -0700
        Re: scan into many tables Greg Hennessy <greg.hennessy@cox.net> - 2012-07-22 22:57 +0000
  Re: scan into many tables Jasen Betts <jasen@xnet.co.nz> - 2012-07-14 02:55 +0000

csiph-web