Path: csiph.com!v102.xanadu-bbs.net!xanadu-bbs.net!feeder.erje.net!eu.feeder.erje.net!feeds.phibee-telecom.net!newsfeed.xs4all.nl!newsfeed4.news.xs4all.nl!xs4all!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail Return-Path: X-Original-To: python-list@python.org Delivered-To: python-list@mail.python.org X-Spam-Status: OK 0.025 X-Spam-Evidence: '*H*': 0.95; '*S*': 0.00; 'dynamically': 0.07; 'modify': 0.07; 'currently,': 0.09; 'subject:modules': 0.09; '(which,': 0.16; 'all?': 0.16; 'detected': 0.16; 'obviously,': 0.16; 'overridden': 0.16; 'overriding': 0.16; 'subclasses': 0.16; 'fix': 0.17; 'module': 0.19; 'trying': 0.19; 'normally': 0.19; 'creating': 0.23; 'load': 0.23; 'header:User-Agent:1': 0.23; "aren't": 0.24; 'module,': 0.24; 'along': 0.30; 'code': 0.31; 'pickle': 0.31; 'class': 0.32; 'interface': 0.32; 'subject:from': 0.34; 'could': 0.34; 'problem': 0.35; 'subject:with': 0.35; "can't": 0.35; 'classes': 0.35; 'skip:u 20': 0.35; 'but': 0.35; 'there': 0.35; 'executing': 0.36; 'subject:data': 0.36; 'charset :us-ascii': 0.36; 'to:addr:python-list': 0.38; 'list,': 0.38; 'that,': 0.38; 'to:addr:python.org': 0.39; 'either': 0.39; 'even': 0.60; 'skip:u 10': 0.60; 'course': 0.61; 'such': 0.63; 'dear': 0.65; '(that': 0.65; 'containing': 0.69; 'wish': 0.70; 'compiling': 0.84; 'itself?': 0.84 Date: Wed, 21 Aug 2013 17:46:32 +0200 (CEST) From: Fredrik Tolf To: python-list@python.org Subject: Unpickling data with classes from dynamic modules User-Agent: Alpine 2.10 (DEB 1266 2009-07-14) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; format=flowed; charset=US-ASCII X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.3.7 (nerv.dolda2000.com [IPv6:2002:54d9:e26d:200::1]); Wed, 21 Aug 2013 17:46:32 +0200 (CEST) X-BeenThere: python-list@python.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: General discussion list for the Python programming language List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Newsgroups: comp.lang.python Message-ID: Lines: 30 NNTP-Posting-Host: 2001:888:2000:d::a6 X-Trace: 1377101923 news.xs4all.nl 15920 [2001:888:2000:d::a6]:45733 X-Complaints-To: abuse@xs4all.nl Xref: csiph.com comp.lang.python:52771 Dear list, I have a system in which I load modules dynamically every now and then (that is, creating a module with types.ModuleType, compiling the code for the module and then executing it in the module with exec()), and where I would wish to be able to have classes in those modules containing classes that could pickled, and then unpickled again. The problem with that, currently, is of course two-fold: The dynamically loaded module may not be loaded at the time when the unpickling takes place, but even if it were, it isn't registered in sys.modules, so the unpickler can't find it either way. And, of course, that is detected already when pickling. Is there any way to fix this, at all? I considered trying to create subclasses of the pickler and unpickler that pickle a reference to a module loader and data for the particular module along with a class that comes from such a module, by overriding Pickler.save_global and Unpickler.load_global. Unfortunately, however, those functions aren't part of the public interface and can't be overridden when the C-pickle module is used instead (which, obviously, is what normally happens). Is there any way around this without having to modify the pickle module itself? -- Fredrik Tolf