Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #25617 > unrolled thread

reloading code and multiprocessing

Back to article view | Back to comp.lang.python

Contents

  reloading code and multiprocessing andrea crotti <andrea.crotti.0@gmail.com> - 2012-07-19 11:15 +0100
    Re: reloading code and multiprocessing 88888 Dihedral <dihedral88888@googlemail.com> - 2012-07-19 08:08 -0700
    Re: reloading code and multiprocessing 88888 Dihedral <dihedral88888@googlemail.com> - 2012-07-19 08:08 -0700

#25617 — reloading code and multiprocessing

We need to be able to reload code on a live system.  This live system
has a daemon process always running but it runs many subprocesses with
multiprocessing, and the subprocesses might have a short life...

Now I found a way to reload the code successfully, as you can see from
this testcase:


def func():
    from . import a
    print(a.ret())


class TestMultiProc(unittest.TestCase):
    def setUp(self):
        open(path.join(cur_dir, 'a.py'), 'w').write(old_a)

    def tearDown(self):
        remove(path.join(cur_dir, 'a.py'))

    def test_reloading(self):
        """Starting a new process gives a different result
        """
        p1 = Process(target=func)
        p2 = Process(target=func)
        p1.start()
        res = p1.join()
        open(path.join(cur_dir, 'a.py'), 'w').write(new_a)
        remove(path.join(cur_dir, 'a.pyc'))

        p2.start()
        res = p2.join()


As long as I import the code in the function and make sure to remove the
"pyc" files everything seems to work..
Are there any possible problems which I'm not seeing in this approach or
it's safe?

Any other better ways otherwise?

[toc] | [next] | [standalone]

#25628

andrea crotti於 2012年7月19日星期四UTC+8下午6時15分11秒寫道：
> We need to be able to reload code on a live system.  This live system
> has a daemon process always running but it runs many subprocesses with
> multiprocessing, and the subprocesses might have a short life...
> 
> Now I found a way to reload the code successfully, as you can see from
> this testcase:
> 
> 
> def func():
>     from . import a
>     print(a.ret())
> 
> 
> class TestMultiProc(unittest.TestCase):
>     def setUp(self):
>         open(path.join(cur_dir, 'a.py'), 'w').write(old_a)
> 
>     def tearDown(self):
>         remove(path.join(cur_dir, 'a.py'))
> 
>     def test_reloading(self):
>         """Starting a new process gives a different result
>         """
>         p1 = Process(target=func)
>         p2 = Process(target=func)
>         p1.start()
>         res = p1.join()
>         open(path.join(cur_dir, 'a.py'), 'w').write(new_a)
>         remove(path.join(cur_dir, 'a.pyc'))
> 
>         p2.start()
>         res = p2.join()
> 
> 
> As long as I import the code in the function and make sure to remove the
> "pyc" files everything seems to work..
> Are there any possible problems which I'm not seeing in this approach or
> it's safe?
> 
> Any other better ways otherwise?

If a byte code interpreter is embedded in the executable, then the program 
can obtain or reload code objects in the run time.

In C/C++, unless one can swap some DLL safely or spawn in another process with 
new executables obtained from other places, or a script interpreter is embedded, otherwise it's not easy for an instance in C/C++ to get new methods dynamically.

or 

[toc] | [prev] | [next] | [standalone]

#25629

andrea crotti於 2012年7月19日星期四UTC+8下午6時15分11秒寫道：
> We need to be able to reload code on a live system.  This live system
> has a daemon process always running but it runs many subprocesses with
> multiprocessing, and the subprocesses might have a short life...
> 
> Now I found a way to reload the code successfully, as you can see from
> this testcase:
> 
> 
> def func():
>     from . import a
>     print(a.ret())
> 
> 
> class TestMultiProc(unittest.TestCase):
>     def setUp(self):
>         open(path.join(cur_dir, 'a.py'), 'w').write(old_a)
> 
>     def tearDown(self):
>         remove(path.join(cur_dir, 'a.py'))
> 
>     def test_reloading(self):
>         """Starting a new process gives a different result
>         """
>         p1 = Process(target=func)
>         p2 = Process(target=func)
>         p1.start()
>         res = p1.join()
>         open(path.join(cur_dir, 'a.py'), 'w').write(new_a)
>         remove(path.join(cur_dir, 'a.pyc'))
> 
>         p2.start()
>         res = p2.join()
> 
> 
> As long as I import the code in the function and make sure to remove the
> "pyc" files everything seems to work..
> Are there any possible problems which I'm not seeing in this approach or
> it's safe?
> 
> Any other better ways otherwise?

If a byte code interpreter is embedded in the executable, then the program 
can obtain or reload code objects in the run time.

In C/C++, unless one can swap some DLL safely or spawn in another process with 
new executables obtained from other places, or a script interpreter is embedded, otherwise it's not easy for an instance in C/C++ to get new methods dynamically.

or 

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web