Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #40049 > unrolled thread

"Daemonizing" an application.

Back to article view | Back to comp.lang.python

Contents

  "Daemonizing" an application. Gilles Lenfant <gilles.lenfant@gmail.com> - 2013-02-27 02:52 -0800
    Re: "Daemonizing" an application. Chris Angelico <rosuav@gmail.com> - 2013-02-27 22:03 +1100
    Re: "Daemonizing" an application. Werner Thie <werner@thieprojects.ch> - 2013-02-27 12:08 +0100
    Re: "Daemonizing" an application. Sven <svenito@gmail.com> - 2013-02-27 11:15 +0000
    Re: "Daemonizing" an application. Gilles Lenfant <gilles.lenfant@gmail.com> - 2013-02-27 05:06 -0800
      Re: "Daemonizing" an application. Chris Angelico <rosuav@gmail.com> - 2013-02-28 00:21 +1100
      Re: "Daemonizing" an application. "Vytas D." <vytasd2013@gmail.com> - 2013-02-27 13:46 +0000
    Re: "Daemonizing" an application. Tarek Ziadé <tarek@ziade.org> - 2013-02-27 14:55 +0100
      Re: "Daemonizing" an application. Gilles Lenfant <gilles.lenfant@gmail.com> - 2013-02-27 06:52 -0800
      Re: "Daemonizing" an application. Gilles Lenfant <gilles.lenfant@gmail.com> - 2013-02-27 06:52 -0800
    Re: "Daemonizing" an application. Tarek Ziadé <tarek@ziade.org> - 2013-02-27 15:00 +0100

#40049 — "Daemonizing" an application.

Hello,

Sorry for the obscure title, but I can't make short to explain what I'm searching for. :)

I made an app (kind of proxy) that works without UI within it's process. So far, so good.

Now I need to change "live" some controls of this application, without stopping it.

So my app will be split in two :

* A control app (say "appctl") for a console UI
* A daemon (or agent ?) that runs the core of the app (say "appd"), processing inputs and outputs

What are the best practices to do this ? Examples in a well known Pyhon app I could hack ? Is it possible with standard packages only ?

Thanks in advance fo any pointer.

-- 
Gilles Lenant

[toc] | [next] | [standalone]

#40050

On Wed, Feb 27, 2013 at 9:52 PM, Gilles Lenfant
<gilles.lenfant@gmail.com> wrote:
> Hello,
>
> Sorry for the obscure title, but I can't make short to explain what I'm searching for. :)
>
> I made an app (kind of proxy) that works without UI within it's process. So far, so good.
>
> Now I need to change "live" some controls of this application, without stopping it.
>
> So my app will be split in two :
>
> * A control app (say "appctl") for a console UI
> * A daemon (or agent ?) that runs the core of the app (say "appd"), processing inputs and outputs

Daemonizing is a fairly specific operation (forking and disconnecting
from the console), which may well be a part of what you're asking for,
but on the other hand may be unnecessary (if, for instance, you let
your core app be invoked by Upstart directly).

What form of control do you need? With many apps of this nature, the
only control required is Unix signals - particularly SIGHUP, to say
"I've edited your config files, go reread them". Your front end might
do the editing, or you could even abolish the control app altogether
and simply edit the configs manually. But if you need more, you'll
need to work out how you want them to communicate with each other.

What platform or platforms do you need this to run on?

Regardless of your answers to the above, I would say that in all
probability *yes*, you will be able to do this with just Python and
the standard library. There are a lot of batteries included with
Python :)

ChrisA

[toc] | [prev] | [next] | [standalone]

#40053

Hi

Might be an overkill, but have a look at twisted, 
http://www.twistedmatrix.com

I usually use the spread package for structured communication between 
partners via localhost

> What are the best practices to do this ? Examples in a well known Pyhon app I could hack ? Is it possible with standard packages only ?

Have fun, Werner

[toc] | [prev] | [next] | [standalone]

#40054

[Multipart message — attachments visible in raw view] — view raw

On 27 February 2013 11:03, Chris Angelico <rosuav@gmail.com> wrote:

> On Wed, Feb 27, 2013 at 9:52 PM, Gilles Lenfant
> <gilles.lenfant@gmail.com> wrote:
> > Hello,
> >
> > Sorry for the obscure title, but I can't make short to explain what I'm
> searching for. :)
> >
> > I made an app (kind of proxy) that works without UI within it's process.
> So far, so good.
> >
> > Now I need to change "live" some controls of this application, without
> stopping it.
> >
> > So my app will be split in two :
> >
> > * A control app (say "appctl") for a console UI
> > * A daemon (or agent ?) that runs the core of the app (say "appd"),
> processing inputs and outputs
>
> Daemonizing is a fairly specific operation (forking and disconnecting
> from the console), which may well be a part of what you're asking for,
> but on the other hand may be unnecessary (if, for instance, you let
> your core app be invoked by Upstart directly).
>
> What form of control do you need? With many apps of this nature, the
> only control required is Unix signals - particularly SIGHUP, to say
> "I've edited your config files, go reread them". Your front end might
> do the editing, or you could even abolish the control app altogether
> and simply edit the configs manually. But if you need more, you'll
> need to work out how you want them to communicate with each other.
>
> What platform or platforms do you need this to run on?
>
> Regardless of your answers to the above, I would say that in all
> probability *yes*, you will be able to do this with just Python and
> the standard library. There are a lot of batteries included with
> Python :)
>

One solution is to use XMLRPC to communicate with the daemonized process.
This does involve the daemon listening on the local network. The GUI can
then connect to the daemon and send it commands which essentially is
calling functions on the daemon. These can then change certain behaviors of
the daemon process without restarting it. If you want to be hardcore you
can use sockets. There are other networking libs like Twisted and RabbitMQ,
but they might be a bit overkill for your purpose.

But whether any of this is ok depends on your use case. The signals Chris
mentioned will also work. Alternatively there are a number of filewatching
libraries like watchdog or https://github.com/seb-m/pyinotify which you can
use to monitor changes to a config file if that's the way you want to go.

-- 
./Sven

[toc] | [prev] | [next] | [standalone]

#40057

Le mercredi 27 février 2013 11:52:19 UTC+1, Gilles Lenfant a écrit :
> Hello,
> 
Hello again,

And thanks to all for your pointers and ideas.

As the app is already tied to an Unix like OS, I'll go with signal handling first since I can do all I need through reconfiguration (SIGHUP).

If I need other controls, I shall use an XML-RPC, since it's an OTB feature.

Thanks again
-- 
Gilles Lenfant

[toc] | [prev] | [next] | [standalone]

#40059

On Thu, Feb 28, 2013 at 12:06 AM, Gilles Lenfant
<gilles.lenfant@gmail.com> wrote:
> Le mercredi 27 février 2013 11:52:19 UTC+1, Gilles Lenfant a écrit :
>> Hello,
>>
> Hello again,
>
> And thanks to all for your pointers and ideas.
>
> As the app is already tied to an Unix like OS, I'll go with signal handling first since I can do all I need through reconfiguration (SIGHUP).
>
> If I need other controls, I shall use an XML-RPC, since it's an OTB feature.

Sounds good! Definitely go with the simplest possible solution first;
you can always complicate things later.

ChrisA

[toc] | [prev] | [next] | [standalone]

#40062

[Multipart message — attachments visible in raw view] — view raw

Hello,

Just recently learned about memory-mapped files (module mmap). If two
processes open the same file, this file is mapped for both processes. So at
the same time the same memory space can be accesses by two processes. The
documentation stated that memory-mapped files can (and are) used for
inter-process communication. Also, you can work with the memory-mapped
files using indexing (just like strings).

So you might want to have a look at memory-mapped files if not considered
before. Might help to solve your problem.

Vytas


On Wed, Feb 27, 2013 at 1:21 PM, Chris Angelico <rosuav@gmail.com> wrote:

> On Thu, Feb 28, 2013 at 12:06 AM, Gilles Lenfant
> <gilles.lenfant@gmail.com> wrote:
> > Le mercredi 27 février 2013 11:52:19 UTC+1, Gilles Lenfant a écrit :
> >> Hello,
> >>
> > Hello again,
> >
> > And thanks to all for your pointers and ideas.
> >
> > As the app is already tied to an Unix like OS, I'll go with signal
> handling first since I can do all I need through reconfiguration (SIGHUP).
> >
> > If I need other controls, I shall use an XML-RPC, since it's an OTB
> feature.
>
> Sounds good! Definitely go with the simplest possible solution first;
> you can always complicate things later.
>
> ChrisA
> --
> http://mail.python.org/mailman/listinfo/python-list
>

[toc] | [prev] | [next] | [standalone]

#40063

On 2/27/13 11:52 AM, Gilles Lenfant wrote:
> Hello,
>
> Sorry for the obscure title, but I can't make short to explain what I'm searching for. :)
>
> I made an app (kind of proxy) that works without UI within it's process. So far, so good.
>
> Now I need to change "live" some controls of this application, without stopping it.
>
> So my app will be split in two :
>
> * A control app (say "appctl") for a console UI
> * A daemon (or agent ?) that runs the core of the app (say "appd"), processing inputs and outputs
>
> What are the best practices to do this ? Examples in a well known Pyhon app I could hack ? Is it possible with standard packages only ?
>
> Thanks in advance fo any pointer.
>
You can have a look at Circus - https://circus.readthedocs.org which is 
a process manager.

"circusctl" is used to control "circusd" using ZeroMQ

The nice thing about zmq as opposed to signals is that you can code your 
thing independantly from the transport
then choose which transport fits a situation: TPC (then the ctl can be 
on another box), IPC or even ITC

That also means your ctl part can be portable to any platform

Cheers
Tarek

-- 
Tarek Ziadé · http://ziade.org · @tarek_ziade

[toc] | [prev] | [next] | [standalone]

#40065

Le mercredi 27 février 2013 14:55:42 UTC+1, Tarek Ziadé a écrit :
> On 2/27/13 11:52 AM, Gilles Lenfant wrote:
> 
> > Hello,

[...]

> 
> > Thanks in advance fo any pointer.
> 
> >
> 
> You can have a look at Circus - https://circus.readthedocs.org which is 
> 
> a process manager.
> 
> 
> 
> "circusctl" is used to control "circusd" using ZeroMQ
> 
> 
> 
> The nice thing about zmq as opposed to signals is that you can code your 
> 
> thing independantly from the transport
> 
> then choose which transport fits a situation: TPC (then the ctl can be 
> 
> on another box), IPC or even ITC
> 
> 
> 
> That also means your ctl part can be portable to any platform

Hi Tarek,

Great stuff. Exactly what I was looking for. The various processes of my app already chat with each other using the great ZeroMQ power sockets. And can potentially be powered in as many physical servers since the shared persistent data are provided through SQLAlchemy + eXist-db.

I can read that Circus can monitor sockets too. Should I understand that I can monitor the state of ZMQ listening sockets with Circus too ?

Cheers
-- 
Gilles

> 
> 
> 
> Cheers
> 
> Tarek
> 
> 
> 
> -- 
> 
> Tarek Ziadé · http://ziade.org · @tarek_ziade

[toc] | [prev] | [next] | [standalone]

#40066

Le mercredi 27 février 2013 14:55:42 UTC+1, Tarek Ziadé a écrit :
> On 2/27/13 11:52 AM, Gilles Lenfant wrote:
> 
> > Hello,

[...]

> 
> > Thanks in advance fo any pointer.
> 
> >
> 
> You can have a look at Circus - https://circus.readthedocs.org which is 
> 
> a process manager.
> 
> 
> 
> "circusctl" is used to control "circusd" using ZeroMQ
> 
> 
> 
> The nice thing about zmq as opposed to signals is that you can code your 
> 
> thing independantly from the transport
> 
> then choose which transport fits a situation: TPC (then the ctl can be 
> 
> on another box), IPC or even ITC
> 
> 
> 
> That also means your ctl part can be portable to any platform

Hi Tarek,

Great stuff. Exactly what I was looking for. The various processes of my app already chat with each other using the great ZeroMQ power sockets. And can potentially be powered in as many physical servers since the shared persistent data are provided through SQLAlchemy + eXist-db.

I can read that Circus can monitor sockets too. Should I understand that I can monitor the state of ZMQ listening sockets with Circus too ?

Cheers
-- 
Gilles

> 
> 
> 
> Cheers
> 
> Tarek
> 
> 
> 
> -- 
> 
> Tarek Ziadé · http://ziade.org · @tarek_ziade

[toc] | [prev] | [next] | [standalone]

#40064

s/TPC/TCP   ;)

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web