Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #100154 > unrolled thread

Is Python a good choice for a data logger?

Back to article view | Back to comp.lang.python

Contents

  Is Python a good choice for a data logger? villascape@gmail.com - 2015-12-08 05:30 -0800
    Re: Is Python a good choice for a data logger? Chris Angelico <rosuav@gmail.com> - 2015-12-09 01:17 +1100

#100154 — Is Python a good choice for a data logger?

Let's say there are multiple data-loggers (PCs probably running Linux) behind various firewalls which communicate to other devices on their respective LAN via a proprietary protocol using UDP/IP to collect data.  On perhaps a sixty second periodic basis, the data-loggers will send the data to a server located outside of their firewalls (via TCP/IP and probably using HTTP).  The server (probably running Linux) will store the data monitored by the data-loggers.  Web clients will then make HTTP requests to the server to retrieve the data.


Is Python a good choice for the following:

1) The data-loggers to pole data and/or receive data from the UDP devices and send the data to the server?

2) The daemon/application running on the server which receives the data from the data-loggers, and stores it in the database?


If Python is a good choice, please explain why.  Also, please provide a very high level strategy to implement (I am not asking for script, but just enough so I can initially focus on  learning those parts of Python).

Thanks!

[toc] | [next] | [standalone]

#100157

On Wed, Dec 9, 2015 at 12:30 AM,  <villascape@gmail.com> wrote:
> Is Python a good choice for the following:
>
> 1) The data-loggers to pole data and/or receive data from the UDP devices and send the data to the server?
>
> 2) The daemon/application running on the server which receives the data from the data-loggers, and stores it in the database?
>
>
> If Python is a good choice, please explain why.
>

Yep! Both of these applications are going to spend virtually all their
time *waiting*. That means you don't have to stress about performance,
and can concentrate on writing code in the easiest and cleanest way
possible. To my mind, that says either Python or Pike; both languages
have excellent networking libraries, but in this case, Python
definitely takes the edge, because you already have broad familiarity
with it.

> Also, please provide a very high level strategy to implement (I am not asking for script, but just enough so I can initially focus on  learning those parts of Python).
>

Start with the socket module:

https://docs.python.org/3/library/socket.html

Get to know something of how TCP/IP works, if you don't already. Then
once you're ready to do your actual implementation, since you've
considered using HTTP, I would suggest looking into the third-party
'requests' library for the HTTP clients (you can do everything with
the standard library if you prefer, but you owe it to yourself to at
least have a quick _look_ at requests), and one of the standard web
frameworks for the server (personally, I use Flask, but any would do).
You'll probably want a database back end to store the actual content;
I strongly recommend PostgreSQL, which you can access from Python
using psycopg2. If you don't like hand-writing SQL, you could use
something like SQLAlchemy as middleware; otherwise, direct use of
psycopg2 works just fine.

Every part in this can be replaced without affecting the rest, but if
I were doing this kind of job, these would be the tools I'd first
reach for.

requests: http://requests.readthedocs.org/en/latest/
Flask: http://flask.pocoo.org/docs/
psycopg2: http://initd.org/psycopg/docs/
SQLAlchemy: http://www.sqlalchemy.org/

You asked for a "very high level" strategy. This is so high level it's
practically in LEO, but it's a start :)

ChrisA

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web