Re: Multiprocessing, shared memory vs. pickled copies

From	John Ladasky <ladasky@my-deja.com>
Newsgroups	comp.lang.python
Subject	Re: Multiprocessing, shared memory vs. pickled copies
Date	2011-04-07 11:39 -0700
Organization	http://groups.google.com
Message-ID	<df11f2e9-19e7-4713-a933-aa6161f8bb5d@p16g2000yqb.googlegroups.com> (permalink)
References	(1 earlier) <mailman.27.1301960081.9059.python-list@python.org> <cad2405b-378c-43e4-b79e-edf81e5701e2@w7g2000pre.googlegroups.com> <mailman.56.1302026842.9059.python-list@python.org> <5d0fcd63-cebf-4b6e-a176-20cb7bbec389@n12g2000yqc.googlegroups.com> <mailman.115.1302198262.9059.python-list@python.org>

Show all headers | View raw

On Apr 7, 10:44 am, Robert Kern <robert.k...@gmail.com> wrote:
> On 4/7/11 1:40 AM, John Ladasky wrote:
>
> > On Apr 5, 10:43 am, Philip Semanchuk<phi...@semanchuk.com>  wrote:
> >> And as Robert Kern pointed out, numpy arrays are also pickle-able.
>
> > OK, but SUBCLASSES of numpy.ndarray are not, in my hands, pickling as
> > I would expect.  I already have lots of code that is based on such
> > subclasses, and they do everything I want EXCEPT pickle correctly.  I
> > may have to direct this line of questions to the numpy discussion
> > group after all.
>
> Define the __reduce_ex__() method, not __getstate__(), __setstate__().
>
> http://docs.python.org/release/2.6.6/library/pickle.html#pickling-and...
>
> ndarrays are extension types, so they use that mechanism.

Thanks, Robert, as you can see, I got on that track shortly after I
posted my code example.  This is apparently NOT a numpy issue, it's an
issue for pickling all C extension types.

Is there a way to examine a Python object, and determine whether it's
a C extension type or not?  Or do you have to deduce that from the
documentation and/or the source code?

I started hunting through the numpy source code last night.  It's
complicated.  I haven't found the ndarray object definition yet.
Perhaps because I was looking through .py files, when I actually
should have been looking through .c files?

Thread

Multiprocessing, shared memory vs. pickled copies John Ladasky <ladasky@my-deja.com> - 2011-04-04 13:20 -0700
  Re: Multiprocessing, shared memory vs. pickled copies Philip Semanchuk <philip@semanchuk.com> - 2011-04-04 19:34 -0400
    Re: Multiprocessing, shared memory vs. pickled copies John Ladasky <ladasky@my-deja.com> - 2011-04-05 09:58 -0700
      Re: Multiprocessing, shared memory vs. pickled copies Philip Semanchuk <philip@semanchuk.com> - 2011-04-05 13:43 -0400
        Re: Multiprocessing, shared memory vs. pickled copies John Ladasky <ladasky@my-deja.com> - 2011-04-06 23:40 -0700
          Re: Multiprocessing, shared memory vs. pickled copies John Ladasky <ladasky@my-deja.com> - 2011-04-07 00:41 -0700
            Re: Multiprocessing, shared memory vs. pickled copies Philip Semanchuk <philip@semanchuk.com> - 2011-04-07 09:23 -0400
          Re: Multiprocessing, shared memory vs. pickled copies Robert Kern <robert.kern@gmail.com> - 2011-04-07 12:44 -0500
            Re: Multiprocessing, shared memory vs. pickled copies John Ladasky <ladasky@my-deja.com> - 2011-04-07 11:39 -0700
              Re: Multiprocessing, shared memory vs. pickled copies Robert Kern <robert.kern@gmail.com> - 2011-04-07 15:01 -0500
  Re: Multiprocessing, shared memory vs. pickled copies Robert Kern <robert.kern@gmail.com> - 2011-04-04 19:05 -0500
    Re: Multiprocessing, shared memory vs. pickled copies sturlamolden <sturlamolden@yahoo.no> - 2011-04-07 16:39 -0700
  Re: Multiprocessing, shared memory vs. pickled copies Philip Semanchuk <philip@semanchuk.com> - 2011-04-04 21:16 -0400
  Re: Multiprocessing, shared memory vs. pickled copies Robert Kern <robert.kern@gmail.com> - 2011-04-05 10:47 -0500
  Re: Multiprocessing, shared memory vs. pickled copies sturlamolden <sturlamolden@yahoo.no> - 2011-04-07 17:03 -0700
    Re: Multiprocessing, shared memory vs. pickled copies sturlamolden <sturlamolden@yahoo.no> - 2011-04-07 17:38 -0700
      Re: Multiprocessing, shared memory vs. pickled copies sturlamolden <sturlamolden@yahoo.no> - 2011-04-07 18:10 -0700
        Re: Multiprocessing, shared memory vs. pickled copies John Ladasky <ladasky@my-deja.com> - 2011-04-09 00:36 -0700
          Re: Multiprocessing, shared memory vs. pickled copies sturlamolden <sturlamolden@yahoo.no> - 2011-04-09 10:15 -0700
            Re: Multiprocessing, shared memory vs. pickled copies John Ladasky <ladasky@my-deja.com> - 2011-04-09 13:18 -0700
              Re: Multiprocessing, shared memory vs. pickled copies sturlamolden <sturlamolden@yahoo.no> - 2011-04-10 08:01 -0700
        Re: Multiprocessing, shared memory vs. pickled copies sturlamolden <sturlamolden@yahoo.no> - 2011-04-10 15:35 -0700

csiph-web