Path: csiph.com!fu-berlin.de!uni-berlin.de!not-for-mail
From: Matt Ruffalo <matt.ruffalo@gmail.com>
Newsgroups: comp.lang.python
Subject: Using SSL socket as stdin for subprocess.Popen
Date: Sat, 19 Mar 2016 13:44:58 -0400
Lines: 59
Message-ID: <mailman.435.1458559693.12893.python-list@python.org>
Mime-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Thunderbird/38.6.0
Precedence: list
Xref: csiph.com comp.lang.python:105348

Hi all-

I'm writing a backup client for automating the synchronization of btrfs
snapshots between machines -- essentially piping the output of `btrfs
send` on my laptop/desktop to `btrfs receive` on a server. I've been
doing this manually for quite a while, and something automated would be
much more convenient.

I've started implementing this in terms of a daemon that will run on a
server, and client code that will be invoked every hour or so on client
machines. Every machine I use is configured to take btrfs snapshots of
all available subvolumes on every boot, so it's my expectation that most
of the time the client code runs it will see that it has nothing to do,
and exit with a message like "most recent snapshot is already on remote
server". I'll also implement some checks like "skip backups if running
on battery power" and "only back up over Ethernet instead of wifi", but
one thing at a time.

I've been using SSL for the communication between the client and server,
both in a static "control port" and a dynamic data port that is assigned
for each individual call to btrfs send | btrfs receive. In addition to
verifying the client certificates against a trusted CA, this also allows
things like selecting the backup destination based on the common name in
the client cert.

I've hit an issue that I'm not sure how to work through, though. I'm
attempting to use a SSL socket (and/or the result of its 'makefile'
method) directly as the `stdin` argument to subprocess.Popen, but it
seems that the *encrypted* data is used by the subprocess.

The WIP code is at https://github.com/mruffalo/btrfs-sync-daemon , and
the specific functionality that I'm describing is at
https://github.com/mruffalo/btrfs-sync-daemon/blob/master/server.py#L64
. The SSL socket behaves exactly as I expect in most cases, e.g. calling
its read() method, or calling ssl_sock.makefile('rb') and then read()ing
the result of the makefile() call. This gives me the decrypted data that
I send in the client code.

What seems to be consistently malfunctioning is the usage of the SSL
socket wrappers as `stdin` in a Popen call. While I'm getting all of
this working, I'm replacing the `btrfs receive` call with effectively
`cat > test` in the working directory of the code, and the 'test' file
seems to contain encrypted data instead of the b'hello' I'm sending over
the SSL socket. In addition to using `conn` as `stdin`, I also attempted
using conn.makefile('rb') with the same results: encrypted data in the
output file.

I could conceivably just use a non-encrypted socket for the data
connection since I don't intend this to be used outside of a home
network, but after implementing SSL communication for the control
connection I didn't see a reason *not* to use it for everything.

At the moment I'm running this under Python 3.5.1 on Kubuntu 16.04. I
did some searching in the Python documentation, on Google, and in the
Python bug tracker, but haven't seen anyone else report an issue like
this. Does this sound like a bug I should report?

Thank you,
MMR...