Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #72529 > unrolled thread

multiprocess (and paramiko)

Back to article view | Back to comp.lang.python

Contents

  multiprocess (and paramiko) mennis <michaelian.ennis@gmail.com> - 2014-06-03 08:43 -0700
    Re: multiprocess (and paramiko) Chris Angelico <rosuav@gmail.com> - 2014-06-04 02:08 +1000
    Re: multiprocess (and paramiko) Roy Smith <roy@panix.com> - 2014-06-03 12:43 -0400
      Re: multiprocess (and paramiko) mennis <michaelian.ennis@gmail.com> - 2014-06-03 11:29 -0700

#72529 — multiprocess (and paramiko)

I was able to work around this by using a completely different design but I still don''t understand why this doesn't work.  It appears that the process that launches the process doesn't get access to updated object attributes.  When I set and check them in the object itself it behaves as expected.  When I check them from outside the object instance I get the initial values only.  Could someone explain what I'm missing?

Here I have a simple multiprocessing class that when initializes takes a connected SSHClient instance and a command to run on the associated host in a new channel.

import multiprocessing

from time import time
from Crypto import Random
import paramiko        


class Nonblock(multiprocessing.Process):

    def __init__(self, connection, cmd):
        Random.atfork()
        multiprocessing.Process.__init__(self)

        self.transport = connection.get_transport()
        if self.transport is None:
            raise ConnectionError("connection.get_transport() returned None ")
        self.channel = self.transport.open_session()

        self.command = cmd
        self.done = False
        self.stdin = None
        self.stdout = None
        self.stderr = None
        self.status = None
        self.message = str()
        self.time = float()

    def _read(self, channelobj):
        """read until EOF"""
        buf = channelobj.readline()
        output = str(buf)
        while buf:
            buf = channelobj.readline()
            output += buf
        return output

    def run(self):

        start = time()
        stdin, stdout, stderr = self.channel.exec_command(command=self.command)

        self.stderr = self._read(stderr)
        self.status = stdout.channel.recv_exit_status()

        if self.status != 0:
            self.status = False
            self.message = self.stderr
        else:
            self.status = True
            self.message = self._read(stdout)

        self.time = time() - start
        stdin.close()

        self.done = True


I expect to use it in the following manner:

from simplelib import Nonblock
from time import sleep
from paramiko import SSHClient, AutoAddPolicy


if __name__== "__main__":
    connection = SSHClient()
    connection.set_missing_host_key_policy(AutoAddPolicy())

    username = "uname"
    hostname = "hostname"
    password = "password"

    connection.connect(hostname, 22, username, password)
    print connection.exec_command("sleep 1; echo test 0")[1].read()

    n = Nonblock(connection,"sleep 20; echo test 2")
    n.start()

    print connection.exec_command("sleep 1; echo test 1")[1].read()
    while not n.done:
        sleep(1)
    print n.message
    print "done"

[toc] | [next] | [standalone]

#72530

On Wed, Jun 4, 2014 at 1:43 AM, mennis <michaelian.ennis@gmail.com> wrote:
> I was able to work around this by using a completely different design but I still don''t understand why this doesn't work.  It appears that the process that launches the process doesn't get access to updated object attributes.  When I set and check them in the object itself it behaves as expected.  When I check them from outside the object instance I get the initial values only.  Could someone explain what I'm missing?
>

When you fork into two processes, the child gets a copy of the
parent's state, but after that, changes happen completely
independently. You need to use actual multiprocessing features like
Queue and such to pass information from one process to another.

ChrisA

[toc] | [prev] | [next] | [standalone]

#72536

In article <3c0be3a7-9d2d-4530-958b-13be97db3765@googlegroups.com>,
 mennis <michaelian.ennis@gmail.com> wrote:

> Here I have a simple multiprocessing class that when initializes takes a 
> connected SSHClient instance and a command to run on the associated host in a 
> new channel.

ChrisA has already answered your question, but to answer the question 
you didn't ask, you probably want to ditch working directly with 
paramiko and take a look at fabric (http://www.fabfile.org/).  It layers 
a really nice interface on top of paramiko.  Instead of gobs of 
low-level paramiko code, you just do something like:

from fabric.api import env, run
env.host_string = "my-remote-hostname.com"
output = run("my command")

and you're done.

[toc] | [prev] | [next] | [standalone]

#72542

I'm familiar with and have learned much from fabric.  Its execution model don't work for this specific interface I'm working on.  I use fabric for other things though and it's great.

Ian

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web