Path: csiph.com!usenet.pasdenom.info!gegeweb.org!de-l.enfer-du-nord.net!feeder1.enfer-du-nord.net!feeds.phibee-telecom.net!newsfeed.xs4all.nl!newsfeed5.news.xs4all.nl!xs4all!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail
MIME-Version: 1.0
In-Reply-To: <CAF_E5JaBXs51oHFLO7aYZ4sksUGvzF2M8rO5AQ+2ztr+x6oPxQ@mail.gmail.com>
References: <CAF_E5Jb=_H1D9YwLWGzE+D40U9ji1kFzo1hz=VwB3yzb8=43CA@mail.gmail.com> <CAPTjJmoUkKCe3q_qa7iACw0B0oVdZB-_Mo7xvgJT0nEyaOTdwg@mail.gmail.com> <CAF_E5Ja+3wMdm-jvx9C3fWk9Nwrreu7Q6r9FDwPr4YDFWt5gVQ@mail.gmail.com> <CAPTjJmrDqZWH0OBX1ExKYc0csieQvV0Kk3CVRNzbMmttmYKpjA@mail.gmail.com> <CAF_E5JaBXs51oHFLO7aYZ4sksUGvzF2M8rO5AQ+2ztr+x6oPxQ@mail.gmail.com>
Date: Fri, 27 Jul 2012 12:01:03 +0100
Subject: Re: reloading code and multiprocessing
From: andrea crotti <andrea.crotti.0@gmail.com>
To: Chris Angelico <rosuav@gmail.com>
Content-Type: text/plain; charset=ISO-8859-1
Cc: python-list@python.org
Precedence: list
Newsgroups: comp.lang.python
Message-ID: <mailman.2654.1343386865.4697.python-list@python.org>
Lines: 60
NNTP-Posting-Host: 2001:888:2000:d::a6
Xref: csiph.com comp.lang.python:26140

2012/7/25 andrea crotti <andrea.crotti.0@gmail.com>:
>
> I would also like to avoid this in general, but we have many
> subprocesses to launch and some of them might take weeks, so we need
> to have a process which is always running, because there is never a
> point in time where we can just say let's stop everything and start again..
>
> Anyway if there are better solutions I'm still glad to hear them, but
> I would also like to keep it simple..
>
> Another thing which now we need to figure out is how to communicate
> with the live process..  For example we might want to submit something
> manually, which should pass from the main process.
>
> The first idea is to have a separate process that opens a socket and
> listens for data on a local port, with a defined protocol.
>
> Then the main process can parse these commands and run them.
> Are there easier ways otherwise?


So I was trying to do this, removing the module from sys.modules and
starting a new process (after modifying the file), but it doesn't work
as I expected.
The last assertion fails, but how?

The pyc file is not generated, the module is actually not in
sys.modules, and the function doesn't in the subprocess doesn't fail
but still returns the old value.
Any idea?

old_a = "def ret(): return 0"
new_a = "def ret(): return 1"


def func_no_import(queue):
    queue.put(a_glob.ret())


class TestMultiProc(unittest.TestCase):

    def test_reloading_with_global_import(self):
        """In this case the import is done before the process are started,
        so we need to clean sys.modules to make sure we reload everything
        """
        queue = Queue()
        open(path.join(CUR_DIR, 'old_a.py'), 'w').write(old_a)

        p1 = Process(target=func_no_import, args=(queue, ))
        p1.start()
        p1.join()
        self.assertEqual(queue.get(), 0)

        open(path.join(CUR_DIR, 'old_a.py'), 'w').write(new_a)
        del sys.modules['auto_tester.tests.a_glob']

        p2 = Process(target=func_no_import, args=(queue, ))
        p2.start()
        p2.join()
        self.assertEqual(queue.get(), 1)