Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #93096 > unrolled thread

Re: Pure Python Data Mangling or Encrypting

Back to article view | Back to comp.lang.python

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Re: Pure Python Data Mangling or Encrypting Randall Smith <randall@tnr.cc> - 2015-06-24 13:36 -0500
    Re: Pure Python Data Mangling or Encrypting Steven D'Aprano <steve@pearwood.info> - 2015-06-25 14:07 +1000
      Re: Pure Python Data Mangling or Encrypting Devin Jeanpierre <jeanpierreda@gmail.com> - 2015-06-24 21:27 -0700
        Re: Pure Python Data Mangling or Encrypting Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2015-06-25 19:25 +1000
          Re: Pure Python Data Mangling or Encrypting Devin Jeanpierre <jeanpierreda@gmail.com> - 2015-06-25 02:41 -0700
          Re: Pure Python Data Mangling or Encrypting Chris Angelico <rosuav@gmail.com> - 2015-06-25 19:57 +1000
          Re: Pure Python Data Mangling or Encrypting Jon Ribbens <jon+usenet@unequivocal.co.uk> - 2015-06-25 10:03 +0000
            Re: Pure Python Data Mangling or Encrypting Steven D'Aprano <steve@pearwood.info> - 2015-06-26 01:13 +1000
              Re: Pure Python Data Mangling or Encrypting Jon Ribbens <jon+usenet@unequivocal.co.uk> - 2015-06-25 15:26 +0000
                Re: Pure Python Data Mangling or Encrypting Randall Smith <randall@tnr.cc> - 2015-06-25 13:58 -0500
                Re: Pure Python Data Mangling or Encrypting Chris Angelico <rosuav@gmail.com> - 2015-06-26 10:33 +1000
                  Re: Pure Python Data Mangling or Encrypting Jon Ribbens <jon+usenet@unequivocal.co.uk> - 2015-06-26 10:49 +0000
                Re: Pure Python Data Mangling or Encrypting Ian Kelly <ian.g.kelly@gmail.com> - 2015-06-25 19:01 -0600
                  Re: Pure Python Data Mangling or Encrypting Steven D'Aprano <steve@pearwood.info> - 2015-06-27 03:06 +1000
                    Re: Pure Python Data Mangling or Encrypting Randall Smith <randall@tnr.cc> - 2015-06-26 15:09 -0500
                      Re: Pure Python Data Mangling or Encrypting Johannes Bauer <dfnsonfsduifb@gmx.de> - 2015-06-26 23:07 +0200
                        Re: Pure Python Data Mangling or Encrypting Jon Ribbens <jon+usenet@unequivocal.co.uk> - 2015-06-26 21:29 +0000
                          Re: Pure Python Data Mangling or Encrypting Mark Lawrence <breamoreboy@yahoo.co.uk> - 2015-06-26 22:55 +0100
                          Re: Pure Python Data Mangling or Encrypting Johannes Bauer <dfnsonfsduifb@gmx.de> - 2015-06-27 00:42 +0200
                            Re: Pure Python Data Mangling or Encrypting Devin Jeanpierre <jeanpierreda@gmail.com> - 2015-06-26 16:26 -0700
                            Re: Pure Python Data Mangling or Encrypting Jon Ribbens <jon+usenet@unequivocal.co.uk> - 2015-06-27 00:21 +0000
                            Re: Pure Python Data Mangling or Encrypting Randall Smith <randall@tnr.cc> - 2015-06-26 19:55 -0500
                              Re: Pure Python Data Mangling or Encrypting Johannes Bauer <dfnsonfsduifb@gmx.de> - 2015-06-27 07:24 +0200
                          Re: Pure Python Data Mangling or Encrypting Randall Smith <randall@tnr.cc> - 2015-06-26 19:12 -0500
                        Re: Pure Python Data Mangling or Encrypting Ian Kelly <ian.g.kelly@gmail.com> - 2015-06-26 15:58 -0600
                        Re: Pure Python Data Mangling or Encrypting Randall Smith <randall@tnr.cc> - 2015-06-26 19:23 -0500
                      Re: Pure Python Data Mangling or Encrypting Johannes Bauer <dfnsonfsduifb@gmx.de> - 2015-06-26 23:11 +0200
                        Re: Pure Python Data Mangling or Encrypting Michael Torrie <torriem@gmail.com> - 2015-06-27 11:02 -0600
                          Re: Pure Python Data Mangling or Encrypting Paul Rubin <no.email@nospam.invalid> - 2015-06-27 10:45 -0700
                      Re: Pure Python Data Mangling or Encrypting Steven D'Aprano <steve@pearwood.info> - 2015-06-27 13:38 +1000
                        Re: Pure Python Data Mangling or Encrypting Devin Jeanpierre <jeanpierreda@gmail.com> - 2015-06-26 21:05 -0700
                          Re: Pure Python Data Mangling or Encrypting Steven D'Aprano <steve@pearwood.info> - 2015-06-27 16:16 +1000
                            Re: Pure Python Data Mangling or Encrypting Devin Jeanpierre <jeanpierreda@gmail.com> - 2015-06-27 13:30 -0700
                              Re: Pure Python Data Mangling or Encrypting Steven D'Aprano <steve@pearwood.info> - 2015-06-28 11:18 +1000
                                Re: Pure Python Data Mangling or Encrypting Devin Jeanpierre <jeanpierreda@gmail.com> - 2015-06-27 19:11 -0700
                        Re: Pure Python Data Mangling or Encrypting Ian Kelly <ian.g.kelly@gmail.com> - 2015-06-26 23:47 -0600
                          Re: Pure Python Data Mangling or Encrypting Steven D'Aprano <steve@pearwood.info> - 2015-06-27 18:38 +1000
                            Re: Pure Python Data Mangling or Encrypting Chris Angelico <rosuav@gmail.com> - 2015-06-27 18:53 +1000
                              Re: Pure Python Data Mangling or Encrypting Johannes Bauer <dfnsonfsduifb@gmx.de> - 2015-06-27 11:07 +0200
                                Re: Pure Python Data Mangling or Encrypting Chris Angelico <rosuav@gmail.com> - 2015-06-27 19:17 +1000
                                  Re: Pure Python Data Mangling or Encrypting Jon Ribbens <jon+usenet@unequivocal.co.uk> - 2015-06-27 09:27 +0000
                                    Re: Pure Python Data Mangling or Encrypting Johannes Bauer <dfnsonfsduifb@gmx.de> - 2015-06-27 12:05 +0200
                                      Re: Pure Python Data Mangling or Encrypting Chris Angelico <rosuav@gmail.com> - 2015-06-27 20:16 +1000
                                        Re: Pure Python Data Mangling or Encrypting Johannes Bauer <dfnsonfsduifb@gmx.de> - 2015-06-27 12:55 +0200
                                      Re: Pure Python Data Mangling or Encrypting Jon Ribbens <jon+usenet@unequivocal.co.uk> - 2015-06-27 10:26 +0000
                                    Re: Pure Python Data Mangling or Encrypting Laura Creighton <lac@openend.se> - 2015-06-27 14:27 +0200
                                  Re: Pure Python Data Mangling or Encrypting Johannes Bauer <dfnsonfsduifb@gmx.de> - 2015-06-27 12:18 +0200
                                    Re: Pure Python Data Mangling or Encrypting Chris Angelico <rosuav@gmail.com> - 2015-06-27 21:33 +1000
                                    Re: Pure Python Data Mangling or Encrypting Ian Kelly <ian.g.kelly@gmail.com> - 2015-06-27 08:59 -0600
                                  Re: Pure Python Data Mangling or Encrypting Laura Creighton <lac@openend.se> - 2015-06-27 13:25 +0200
                                    Re: Pure Python Data Mangling or Encrypting Jussi Piitulainen <jpiitula@ling.helsinki.fi> - 2015-06-27 15:23 +0300
                                    Re: Pure Python Data Mangling or Encrypting Laura Creighton <lac@openend.se> - 2015-06-27 14:48 +0200
                            Re: Pure Python Data Mangling or Encrypting Johannes Bauer <dfnsonfsduifb@gmx.de> - 2015-06-27 11:12 +0200
                            Re: Pure Python Data Mangling or Encrypting Ian Kelly <ian.g.kelly@gmail.com> - 2015-06-27 09:09 -0600
                              Re: Pure Python Data Mangling or Encrypting Steven D'Aprano <steve@pearwood.info> - 2015-06-28 03:35 +1000
                                Re: Pure Python Data Mangling or Encrypting Steven D'Aprano <steve@pearwood.info> - 2015-06-28 03:58 +1000
                                Re: Pure Python Data Mangling or Encrypting Ian Kelly <ian.g.kelly@gmail.com> - 2015-06-27 14:16 -0600
                                Re: Pure Python Data Mangling or Encrypting Jon Ribbens <jon+usenet@unequivocal.co.uk> - 2015-06-28 13:41 +0000
                        Re: Pure Python Data Mangling or Encrypting Robert Kern <robert.kern@gmail.com> - 2015-06-27 08:58 +0100
                        Re: Pure Python Data Mangling or Encrypting Robert Kern <robert.kern@gmail.com> - 2015-06-27 09:07 +0100
                        Re: Pure Python Data Mangling or Encrypting Dennis Lee Bieber <wlfraed@ix.netcom.com> - 2015-06-27 10:39 -0400
                      Re: Pure Python Data Mangling or Encrypting Grant Edwards <invalid@invalid.invalid> - 2015-06-27 12:38 +0000
                        Re: Pure Python Data Mangling or Encrypting Randall Smith <randall@tnr.cc> - 2015-06-27 13:22 -0500
                          Re: Pure Python Data Mangling or Encrypting Steven D'Aprano <steve@pearwood.info> - 2015-06-28 04:51 +1000
                            Re: Pure Python Data Mangling or Encrypting Chris Angelico <rosuav@gmail.com> - 2015-06-28 09:05 +1000
                    Re: Pure Python Data Mangling or Encrypting Chris Angelico <rosuav@gmail.com> - 2015-06-27 11:21 +1000
                    Re: Pure Python Data Mangling or Encrypting Ian Kelly <ian.g.kelly@gmail.com> - 2015-06-26 23:59 -0600
                      Re: Pure Python Data Mangling or Encrypting Jon Ribbens <jon+usenet@unequivocal.co.uk> - 2015-06-27 09:26 +0000
                    Re: Pure Python Data Mangling or Encrypting Chris Angelico <rosuav@gmail.com> - 2015-06-27 16:52 +1000
                    Re: Pure Python Data Mangling or Encrypting Randall Smith <randall@tnr.cc> - 2015-06-27 12:08 -0500
                      Re: Pure Python Data Mangling or Encrypting Steven D'Aprano <steve@pearwood.info> - 2015-06-28 04:50 +1000
                        Re: Pure Python Data Mangling or Encrypting Randall Smith <randall@tnr.cc> - 2015-06-29 15:52 -0500
                          Re: Pure Python Data Mangling or Encrypting Steven D'Aprano <steve@pearwood.info> - 2015-06-30 13:00 +1000
                            Re: Pure Python Data Mangling or Encrypting Jon Ribbens <jon+usenet@unequivocal.co.uk> - 2015-06-30 12:19 +0000
                              Re: Pure Python Data Mangling or Encrypting Steven D'Aprano <steve@pearwood.info> - 2015-07-01 04:17 +1000
                                Re: Pure Python Data Mangling or Encrypting Chris Angelico <rosuav@gmail.com> - 2015-07-01 04:33 +1000
                                Re: Pure Python Data Mangling or Encrypting Jon Ribbens <jon+usenet@unequivocal.co.uk> - 2015-06-30 18:37 +0000
                                Re: Pure Python Data Mangling or Encrypting Randall Smith <randall@tnr.cc> - 2015-07-01 09:38 -0500
                            Re: Pure Python Data Mangling or Encrypting Randall Smith <randall@tnr.cc> - 2015-06-30 12:39 -0500
                              Re: Pure Python Data Mangling or Encrypting Steven D'Aprano <steve@pearwood.info> - 2015-07-01 04:59 +1000
                                Re: Pure Python Data Mangling or Encrypting Chris Angelico <rosuav@gmail.com> - 2015-07-01 05:20 +1000
                                Re: Pure Python Data Mangling or Encrypting Jon Ribbens <jon+usenet@unequivocal.co.uk> - 2015-06-30 23:25 +0000
                                  Re: Pure Python Data Mangling or Encrypting alister <alister.nospam.ware@ntlworld.com> - 2015-07-01 08:06 +0000
                      Re: Pure Python Data Mangling or Encrypting Jon Ribbens <jon+usenet@unequivocal.co.uk> - 2015-06-28 14:21 +0000
                        Re: Pure Python Data Mangling or Encrypting Randall Smith <randall@tnr.cc> - 2015-06-29 15:46 -0500
                          Re: Pure Python Data Mangling or Encrypting Jon Ribbens <jon+usenet@unequivocal.co.uk> - 2015-06-29 20:49 +0000
                            Re: Pure Python Data Mangling or Encrypting Randall Smith <randall@tnr.cc> - 2015-06-30 12:43 -0500
                      Re: Pure Python Data Mangling or Encrypting Gregory Ewing <greg.ewing@canterbury.ac.nz> - 2015-07-02 10:31 +1200
                Re: Pure Python Data Mangling or Encrypting Mark Lawrence <breamoreboy@yahoo.co.uk> - 2015-06-26 02:17 +0100
                Re: Pure Python Data Mangling or Encrypting Chris Angelico <rosuav@gmail.com> - 2015-06-26 12:06 +1000
                Re: Pure Python Data Mangling or Encrypting Chris Angelico <rosuav@gmail.com> - 2015-06-26 12:05 +1000
                Re: Pure Python Data Mangling or Encrypting Mark Lawrence <breamoreboy@yahoo.co.uk> - 2015-06-26 03:24 +0100
                Re: Pure Python Data Mangling or Encrypting Chris Angelico <rosuav@gmail.com> - 2015-06-26 12:29 +1000
          Re: Pure Python Data Mangling or Encrypting Joonas Liik <liik.joonas@gmail.com> - 2015-06-25 13:00 +0300
          Re: Pure Python Data Mangling or Encrypting Devin Jeanpierre <jeanpierreda@gmail.com> - 2015-06-25 03:18 -0700
      Re: Pure Python Data Mangling or Encrypting Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2015-06-25 17:05 +1000
      Re: Pure Python Data Mangling or Encrypting Randall Smith <randall@tnr.cc> - 2015-06-25 14:09 -0500

Page 4 of 5 — ← Prev page 1 2 3 [4] 5  Next page →

#93248

On Sat, 27 Jun 2015 13:38:46 +1000, Steven D'Aprano <steve@pearwood.info>
declaimed the following:


>
>Sometimes you say the user is supposed to encrypt the data themselves:
>
>    While the data senders are supposed to encrypt data, that's not
>    guaranteed
>
>
>Now you say that the application encrypts the data, except that the user can
>turn that option off.
>
	<SNIP>
>
>Great to hear it! Just make sure your application always encrypts the
>uploaded files, and you protect both the sender of the files and the
>receiver.
>

	Just an aside: I'm still not clear on just where this application
resides!

	Some posts make it seem like it is all on the storage server, with
nothing running on the sender (a web page file submittal page, for
example).

	If the "application" consists of both client side and server side
programs, with some defined protocol (proprietary to minimize the odds of
someone spoofing as a client) between them, then yes -- just do the AES
encryption in the client before sending... The server never needs to know
the key [the client installation will need a key, preferably generated
locally, and saved so the file can be decrypted later]. If the files will
be retrievable by others than the originator, a public key system (PGP/GPG)
might need to replace the AES encryption.

	For the web-page submittal scheme -- as far as I'm concerned it is too
late to bother encrypting the received file. The only excuse would be that
the server machine itself is insecure and anyone could log-in and peruse
the file system looking at stuff. For this I wouldn't even consider using
the file system to store these snippets... Use a database and store them as
BLOB data, along with suitable information for keying on the content
(submitter ID, submitter filename), make sure the database storage and the
DBMS run as some limited privilege process, and have all access SQL pass
through a limited user account.

-- 
	Wulfraed                 Dennis Lee Bieber         AF6VN
    wlfraed@ix.netcom.com    HTTP://wlfraed.home.netcom.com/

[toc] | [prev] | [next] | [standalone]

#93246

On 2015-06-26, Randall Smith <randall@tnr.cc> wrote:

> The only person who can read a file is the owner.

That's always the plan, but many a successful exploit has been based
on breaking that assumption.  If privacy actually matters, that's not
a good assumption to rely on as a single point of failure.

--
Grant

[toc] | [prev] | [next] | [standalone]

#93257

On 06/27/2015 07:38 AM, Grant Edwards wrote:
> On 2015-06-26, Randall Smith <randall@tnr.cc> wrote:
>
>> The only person who can read a file is the owner.
>
> That's always the plan, but many a successful exploit has been based
> on breaking that assumption.  If privacy actually matters, that's not
> a good assumption to rely on as a single point of failure.
>
> --
> Grant
>


The owner (client software) encrypts the data using AES.  This is the 
default behavior of the client software.  If the client chooses to 
disable encryption, that's their issue for sure.  I'm trying to make 
sure it doesn't become the storage server's issue too.

-Randall

[toc] | [prev] | [next] | [standalone]

#93260

On Sun, 28 Jun 2015 04:22 am, Randall Smith wrote:

> The owner (client software) encrypts the data using AES.  This is the
> default behavior of the client software.  If the client chooses to
> disable encryption, that's their issue for sure.

I cannot imagine what you think you gain from allowing that to be optional.
Apart from privacy and security breaches.



-- 
Steven

[toc] | [prev] | [next] | [standalone]

#93263

On Sun, Jun 28, 2015 at 4:51 AM, Steven D'Aprano <steve@pearwood.info> wrote:
> On Sun, 28 Jun 2015 04:22 am, Randall Smith wrote:
>
>> The owner (client software) encrypts the data using AES.  This is the
>> default behavior of the client software.  If the client chooses to
>> disable encryption, that's their issue for sure.
>
> I cannot imagine what you think you gain from allowing that to be optional.
> Apart from privacy and security breaches.

I've no idea whether this is the case or not, but one thing you might
gain is independence from a third-party module. You could, for
instance, automatically AES-encrypt your data, but only if "from
Crypto.Cipher import AES" didn't raise ImportError. That effectively
makes encryption optional (the program won't barf for lack of pycrypto
installation), while still clearly being the default - and if you have
a nice loud warning, then it's clear that encryption is the normal
state, and the fallback is a lesser state.

ChrisA

[toc] | [prev] | [next] | [standalone]

#93216

On Sat, Jun 27, 2015 at 6:09 AM, Randall Smith <randall@tnr.cc> wrote:
> Give me one plausible scenario where an attacker can cause malware to hit
> the disk after bytearray.translate with a 256 byte translation table and
> I'll be thankful to you.

The entire 256-byte translation table is significant ONLY if you need
all 256 possible bytes. Suppose I want to generate the following byte
sequence:

"\xCD\x19"

(Okay, this is a slightly oversimplified example, as this attack
doesn't work on a modern Windows. But back in the days of DOS, this
program would reboot your computer.)

How many truly different translation tables are there if I'm trying to
produce this? Just 256*255, or 65280. If I send random two-byte files,
there is one chance in that of my "malware" successfully landing. Once
I've sent about 45,000 of those files, I have a fifty-fifty chance of
having hit it. Send twice as many, I have a 75% chance of success,
etc.

Malware can be crafted to fit within certain restrictions. I saw a
proof-of-concept and analysis document detailing a particular remote
code execution/privilege escalation attack that involved stuffing
"text" into an entry field and then inducing the program to read that
into its stack, finally triggering it by some sort of buffer overflow,
I think. The text had to be no more than X bytes long (because that's
all the entry field was set to accept - it'd truncate after that), and
had to not contain any NUL bytes, and there might have been other
restrictions too. Sure, it makes it harder to write your malware...
but imagine if you can write something in just a handful of different
bytes, which then goes and triggers something else. You could have an
extremely plausible attack that might need only a day's uploading to
deliver.

It makes no difference that there are 256! possible encryption keys,
if most of them have the same result.

ChrisA

[toc] | [prev] | [next] | [standalone]

#93223

On Fri, Jun 26, 2015 at 7:21 PM, Chris Angelico <rosuav@gmail.com> wrote:
> On Sat, Jun 27, 2015 at 6:09 AM, Randall Smith <randall@tnr.cc> wrote:
>> Give me one plausible scenario where an attacker can cause malware to hit
>> the disk after bytearray.translate with a 256 byte translation table and
>> I'll be thankful to you.
>
> The entire 256-byte translation table is significant ONLY if you need
> all 256 possible bytes. Suppose I want to generate the following byte
> sequence:
>
> "\xCD\x19"
>
> (Okay, this is a slightly oversimplified example, as this attack
> doesn't work on a modern Windows. But back in the days of DOS, this
> program would reboot your computer.)

Nice! When I suggested the possibility of a two byte value malicious
payload, I thought it an extreme example of the hypothetical attack. I
didn't expect that somebody might actually produce one.

[toc] | [prev] | [next] | [standalone]

#93235

On 2015-06-27, Ian Kelly <ian.g.kelly@gmail.com> wrote:
> On Fri, Jun 26, 2015 at 7:21 PM, Chris Angelico <rosuav@gmail.com> wrote:
>> On Sat, Jun 27, 2015 at 6:09 AM, Randall Smith <randall@tnr.cc> wrote:
>>> Give me one plausible scenario where an attacker can cause malware to hit
>>> the disk after bytearray.translate with a 256 byte translation table and
>>> I'll be thankful to you.
>>
>> The entire 256-byte translation table is significant ONLY if you need
>> all 256 possible bytes. Suppose I want to generate the following byte
>> sequence:
>>
>> "\xCD\x19"
>>
>> (Okay, this is a slightly oversimplified example, as this attack
>> doesn't work on a modern Windows. But back in the days of DOS, this
>> program would reboot your computer.)
>
> Nice! When I suggested the possibility of a two byte value malicious
> payload, I thought it an extreme example of the hypothetical attack. I
> didn't expect that somebody might actually produce one.

It's a good example of the interesting things that people can come up
with (for example, "binary" executable files that in fact are
comprised entirely of printable ASCII characters), but it isn't in
any sense an "attack" on the system described in this thread.

[toc] | [prev] | [next] | [standalone]

#93226

On Sat, Jun 27, 2015 at 3:59 PM, Ian Kelly <ian.g.kelly@gmail.com> wrote:
> On Fri, Jun 26, 2015 at 7:21 PM, Chris Angelico <rosuav@gmail.com> wrote:
>> On Sat, Jun 27, 2015 at 6:09 AM, Randall Smith <randall@tnr.cc> wrote:
>>> Give me one plausible scenario where an attacker can cause malware to hit
>>> the disk after bytearray.translate with a 256 byte translation table and
>>> I'll be thankful to you.
>>
>> The entire 256-byte translation table is significant ONLY if you need
>> all 256 possible bytes. Suppose I want to generate the following byte
>> sequence:
>>
>> "\xCD\x19"
>>
>> (Okay, this is a slightly oversimplified example, as this attack
>> doesn't work on a modern Windows. But back in the days of DOS, this
>> program would reboot your computer.)
>
> Nice! When I suggested the possibility of a two byte value malicious
> payload, I thought it an extreme example of the hypothetical attack. I
> didn't expect that somebody might actually produce one.

I'm fairly sure this won't actually work on a modern system (I tried
it and all that happened was that debug.exe terminated), but it's
entirely possible there are other attacks. Or attacks that require
only a small number of bytes - maybe create a gzip bomb that will
expand to petabytes of data, that probably wouldn't need many unique
byte values.

ChrisA

[toc] | [prev] | [next] | [standalone]

#93252

On 06/26/2015 08:21 PM, Chris Angelico wrote:
> On Sat, Jun 27, 2015 at 6:09 AM, Randall Smith <randall@tnr.cc> wrote:
>> Give me one plausible scenario where an attacker can cause malware to hit
>> the disk after bytearray.translate with a 256 byte translation table and
>> I'll be thankful to you.
>
> The entire 256-byte translation table is significant ONLY if you need
> all 256 possible bytes. Suppose I want to generate the following byte
> sequence:
>
> "\xCD\x19"
>
> (Okay, this is a slightly oversimplified example, as this attack
> doesn't work on a modern Windows. But back in the days of DOS, this
> program would reboot your computer.)
>
> How many truly different translation tables are there if I'm trying to
> produce this? Just 256*255, or 65280. If I send random two-byte files,
> there is one chance in that of my "malware" successfully landing. Once
> I've sent about 45,000 of those files, I have a fifty-fifty chance of
> having hit it. Send twice as many, I have a 75% chance of success,
> etc.
>

Yes, that's true.  It's even an issue with AES, which uses padding to 
overcome.  That said, remember these are bytes going straight to disk. 
I'm really not concerned about 2 or 3 byte malware and the probability 
plunges after that.  And remember, normal use case is AES encrypted data.

Quite interesting.

Though I didn't mention it in the description, the storage server is 
appending a CRC32 checksum for routine integrity checks.  So by the time 
the data hits the disk, it will have added both a 256 byte translation 
table and a 4 byte checksum.  I think that would interfere with any 
extremely short malware.


> Malware can be crafted to fit within certain restrictions. I saw a
> proof-of-concept and analysis document detailing a particular remote
> code execution/privilege escalation attack that involved stuffing
> "text" into an entry field and then inducing the program to read that
> into its stack, finally triggering it by some sort of buffer overflow,
> I think. The text had to be no more than X bytes long (because that's
> all the entry field was set to accept - it'd truncate after that), and
> had to not contain any NUL bytes, and there might have been other
> restrictions too. Sure, it makes it harder to write your malware...
> but imagine if you can write something in just a handful of different
> bytes, which then goes and triggers something else. You could have an
> extremely plausible attack that might need only a day's uploading to
> deliver.
>
> It makes no difference that there are 256! possible encryption keys,
> if most of them have the same result.
>
> ChrisA
>


Thankyou.  Nice points. I do think given the risks (there are always 
risks) discussed, a successful attack of this nature is not very likely. 
  Worse case, something that looks like this would land on the disk.

crc32 checksum + translation table + malware

with a generated base64 name and no extension.


Doesn't seem like much of a threat.   Much less likely than a bug in the 
standard Crytpo libraries.

-Randall

[toc] | [prev] | [next] | [standalone]

#93259

On Sun, 28 Jun 2015 03:08 am, Randall Smith wrote:

> Though I didn't mention it in the description, the storage server is
> appending a CRC32 checksum for routine integrity checks.  So by the time
> the data hits the disk, it will have added both a 256 byte translation
> table and a 4 byte checksum.


http://stackoverflow.com/questions/1515914/crc32-collision




-- 
Steven

[toc] | [prev] | [next] | [standalone]

#93295

On 06/27/2015 01:50 PM, Steven D'Aprano wrote:
> On Sun, 28 Jun 2015 03:08 am, Randall Smith wrote:
>
>> Though I didn't mention it in the description, the storage server is
>> appending a CRC32 checksum for routine integrity checks.  So by the time
>> the data hits the disk, it will have added both a 256 byte translation
>> table and a 4 byte checksum.
>
>
> http://stackoverflow.com/questions/1515914/crc32-collision
>
>
>
>


Not sure why you posted the link.  The crc32 checksum is just to check 
for possible filesystem corruption.  The system does periodic data 
corruption checks.  BTRFS uses crc32 checksums also.  Please explain.

-Randall

[toc] | [prev] | [next] | [standalone]

#93309

On Tue, 30 Jun 2015 06:52 am, Randall Smith wrote:

> Not sure why you posted the link.  The crc32 checksum is just to check
> for possible filesystem corruption.  The system does periodic data
> corruption checks.  BTRFS uses crc32 checksums also.  Please explain.

The file system can trust that anything writing to a file is allowed to
write to it, in doesn't have to defend against malicious writes. As I
understand it, your application does.

Here is the attack scenario I have in mind:

- you write a file to my computer, and neglect to encrypt it;
- and record the checksum for later;
- I insert malware into your file;
- you retrieve the file from me;
- if the checksum matches what you have on record, you accept the file;
- since you are using CRC, it is quite easy for me to ensure the 
  checksums match after inserting malware;
- and I have now successfully injected malware into your computer.

I'm making an assumption here -- I assume that the sender records a checksum
for uploaded files so that when they get something back again they can tell
whether or not it is the same content they uploaded.

* * * 

By the way, regarding the use of a substitution cipher, I spoke to the
crypto guy at work, and "preimage attack" is not quite the right
terminology, since that's normally used in the context of hash functions.
It's almost a "known ciphertext attack", but not quite, since that
terminology refers to guessing the key from the ciphertext.

I was wrong: cryptographically strong ciphers are generally NOT resistant to
what I described as a preimage attack. If the key leaks, using AES won't
save you: an attacker with access to the key can produce a ciphertext that
decrypts to the malware of his choice, regardless of whether you use
AES-256 or rot-13. There may be other encryption methods which don't suffer
from that, but he doesn't know of them off the top of his head.

His comment was, "don't leak the key".

The other threat I mentioned is that the receiver will read the content of
the file. For that, a strong cipher is much to be preferred over a weak
one, and it needs to be encrypted by the sending end, not the receiving
end. (If the receiving end does it, it has to keep the key so it can
decrypt before sending back, which means the computer's owner can just grab
the key and read the files.)



-- 
Steven

[toc] | [prev] | [next] | [standalone]

#93326

On 2015-06-30, Steven D'Aprano <steve@pearwood.info> wrote:
> On Tue, 30 Jun 2015 06:52 am, Randall Smith wrote:
>> Not sure why you posted the link.  The crc32 checksum is just to check
>> for possible filesystem corruption.  The system does periodic data
>> corruption checks.  BTRFS uses crc32 checksums also.  Please explain.
>
> The file system can trust that anything writing to a file is allowed to
> write to it, in doesn't have to defend against malicious writes. As I
> understand it, your application does.
>
> Here is the attack scenario I have in mind:
>
> - you write a file to my computer, and neglect to encrypt it;

Eh? The game is over right there. I don't trust you, and yet
I have just given you my private data, unencrypted. Checksums
don't even come into it, we have failed utterly at step 1.

> - since you are using CRC, it is quite easy for me to ensure the 
>   checksums match after inserting malware;

No, you have yet *again* misunderstood the difference between the
client and the server.

> I was wrong: cryptographically strong ciphers are generally NOT resistant to
> what I described as a preimage attack. If the key leaks, using AES won't
> save you: an attacker with access to the key can produce a ciphertext that
> decrypts to the malware of his choice, regardless of whether you use
> AES-256 or rot-13. There may be other encryption methods which don't suffer
> from that, but he doesn't know of them off the top of his head.

lol. I suspected as much. You and Johannes were even more wrong than
was already obvious.

> The other threat I mentioned is that the receiver will read the content of
> the file. For that, a strong cipher is much to be preferred over a weak
> one, and it needs to be encrypted by the sending end, not the receiving
> end. (If the receiving end does it, it has to keep the key so it can
> decrypt before sending back, which means the computer's owner can just grab
> the key and read the files.)

Yes, that is utterly basic.

[toc] | [prev] | [next] | [standalone]

#93345

On Tue, 30 Jun 2015 10:19 pm, Jon Ribbens wrote:

> On 2015-06-30, Steven D'Aprano <steve@pearwood.info> wrote:
>> On Tue, 30 Jun 2015 06:52 am, Randall Smith wrote:
>>> Not sure why you posted the link.  The crc32 checksum is just to check
>>> for possible filesystem corruption.  The system does periodic data
>>> corruption checks.  BTRFS uses crc32 checksums also.  Please explain.
>>
>> The file system can trust that anything writing to a file is allowed to
>> write to it, in doesn't have to defend against malicious writes. As I
>> understand it, your application does.
>>
>> Here is the attack scenario I have in mind:
>>
>> - you write a file to my computer, and neglect to encrypt it;
> 
> Eh? The game is over right there. I don't trust you, and yet
> I have just given you my private data, unencrypted.

Yes. That is exactly the problem. If the application doesn't encrypt the
data for me, *it isn't going to happen*. We are in violent agreement that
the sender needs to encrypt the data.

I think Randall has been somewhat less than clear about what the application
actually does and how it works. He probably thinks he doesn't need to
explain, that its none of our business, and wishes we'd just shut up about
it. That's his right.

It's also my right to discuss the possible security implications of some
hypothetical peer-to-peer dropbox-like application which may, or may not,
be similar to Randall's application. Whether Randall learns anything from
that discussion, or just tunes it out, is irrelevant. I've already learned
at least one thing from this discussion, so as far as I'm concerned it's a
win.

Randall has suggested that encryption is optional. It isn't clear whether he
means there is an option to turn encryption off, or whether he means I can
hack the application and disable it, or write my own application. I don't
expect him to be responsible for rogue applications that have been hacked
or written independently, which (out of malice or stupidity) don't encrypt
the uploaded data. But I think that it is foolish to support an unencrypted
mode of operation.

It's not unreasonable to raise this issue. The default state of security
among IT professionals is something worse than awful:

https://medium.com/message/everything-is-broken-81e5f33a24e1

One of Australia's largest ISPs recently was hacked, and guess how they
stored their customer's passwords? Yes, you got it: in plain text. There is
no security mistake so foolish that IT professionals won't make it.


> Checksums don't even come into it, we have failed utterly at step 1.

*shrug*

You're right. But having failed at step 1, there are multiple attacks that
can follow. The first attack is the obvious one: the ability to read the
unencrypted data.

If you can trick me into turning encryption off (say, you use a social
engineering attack on me and convince me to delete "the virus crypto.py"),
then I might inadvertently upload unencrypted data to you. Or maybe you
find an attack on the application that can fool it into dropping down to
unencrypted mode. If there's no unencrypted mode in the first place, that's
much harder.

Earlier, Chris suggested that the application might choose to import the
crypto module, and if it's not available, just keep working without
encryption. This hypothetical attack demonstrates that this would be a
mistake. It's hard for an attacker to convince a naive user to open up the
application source code and edit the code. It's easier to convince them to
delete a file.

Or, the application just has a bug in it. It accidentally flips the sense of
the "use encryption" flag. That's a failure mode that simply cannot occur
if there is no such flag in the first place.

If our attacker has managed to disable encryption in the sender's
application, then they can not only read my data, but tamper with it. These
are *separate attacks* with the same underlying cause. I can mitigate one
without mitigating the other.

We can mitigate against the second attack by using a cryptographically
strong hash function to detect tampering. These *are* resistant to preimage
attacks. If I give you a SHA512 checksum, there is no known practical
method to generate a file with that same checksum. If I give you a CRC
checksum, you can.

(Naturally the checksum has to be under the sender's control. If the
receiver has the checksum and the data, they can just replace the checksum
with one of their choosing.)

That's a separate issue from detecting non-malicious data corruption,
although of course a SHA512 checksum will detect that as well.


>> - since you are using CRC, it is quite easy for me to ensure the
>>   checksums match after inserting malware;
> 
> No, you have yet *again* misunderstood the difference between the
> client and the server.

This was described as a peer-to-peer application. You even stated that it
was a "pretty obvious" use-case, a "peer-to-peer dropbox". So is it
peer-to-peer or client-server?

In any case, since Randall has refused to go into specific details of how
his application works, as is his right, we can treat this as an
intellectual exercise for how it *might* work. If such an application keeps
a checksum of the files uploaded, it should be one which can detect
deliberate tampering and not just accidental corruption.


>> I was wrong: cryptographically strong ciphers are generally NOT resistant
>> to what I described as a preimage attack. If the key leaks, using AES
>> won't save you: an attacker with access to the key can produce a
>> ciphertext that decrypts to the malware of his choice, regardless of
>> whether you use AES-256 or rot-13. There may be other encryption methods
>> which don't suffer from that, but he doesn't know of them off the top of
>> his head.
> 
> lol. I suspected as much. You and Johannes were even more wrong than
> was already obvious.

You "suspected as much"? Such a pity you didn't speak up earlier and explain
that cryptographic ciphers aren't generally resistant to preimage attacks.

In any case, I have learned something from this discussion, which makes it
worthwhile.



-- 
Steven

[toc] | [prev] | [next] | [standalone]

#93346

On Wed, Jul 1, 2015 at 4:17 AM, Steven D'Aprano <steve@pearwood.info> wrote:
> If you can trick me into turning encryption off (say, you use a social
> engineering attack on me and convince me to delete "the virus crypto.py"),
> then I might inadvertently upload unencrypted data to you. Or maybe you
> find an attack on the application that can fool it into dropping down to
> unencrypted mode. If there's no unencrypted mode in the first place, that's
> much harder.
>
> Earlier, Chris suggested that the application might choose to import the
> crypto module, and if it's not available, just keep working without
> encryption. This hypothetical attack demonstrates that this would be a
> mistake. It's hard for an attacker to convince a naive user to open up the
> application source code and edit the code. It's easier to convince them to
> delete a file.

And I'm sure Steven knows about this, but if anyone else isn't
convinced that this is a serious vulnerability, look into various
forms of downgrade attack, such as the recent POODLE. Security doesn't
exist if an attacker can convince your program to turn it off without
your knowledge.

>>> - since you are using CRC, it is quite easy for me to ensure the
>>>   checksums match after inserting malware;
>>
>> No, you have yet *again* misunderstood the difference between the
>> client and the server.
>
> This was described as a peer-to-peer application. You even stated that it
> was a "pretty obvious" use-case, a "peer-to-peer dropbox". So is it
> peer-to-peer or client-server?

I've never managed to get any sort of grasp of what this application
actually *is*, but "peer-to-peer Dropbox" is certainly something that
it *might be*. It could be simultaneously peer-to-peer from the
human's point of view, and client-server from the application's -
imagine BitTorrent protocol, but where one end connects to a socket
that the other's listening on, and the active socket always pushes
data to the passive socket. (With BitTorrent, it's truly symmetrical -
doesn't matter who listens and who connects. But imagine if it weren't
that way.) From the software's point of view, it has two distinct
modes: server, in which it listens on a socket and receives data, and
client, in which it connects to other people's sockets and sends data.
As such, the "server" mode is the only one that receives untrusted
data from another user and stores it on the hard disk.

But this is just one theory of what the program *might* be, based on
what I've gathered in this thread. Or rather, it's a vague theory of
something that's mostly plausible, without necessarily even being
useful.

ChrisA

[toc] | [prev] | [next] | [standalone]

#93347

On 2015-06-30, Steven D'Aprano <steve@pearwood.info> wrote:
> On Tue, 30 Jun 2015 10:19 pm, Jon Ribbens wrote:
>> Eh? The game is over right there. I don't trust you, and yet
>> I have just given you my private data, unencrypted.
>
> Yes. That is exactly the problem. If the application doesn't encrypt the
> data for me, *it isn't going to happen*. We are in violent agreement that
> the sender needs to encrypt the data.

It's a good thing that he's said it will then.

> Randall has suggested that encryption is optional.

No he hasn't. You just keep creatively misreading what he says, for
some reason.

> It's not unreasonable to raise this issue.

It is unreasonable to raise it over and over again however,
especially when there's no reason at all to think it's relevant,
and nothing has changed from the last time you raised it.

> We can mitigate against the second attack by using a cryptographically
> strong hash function to detect tampering.

Not on the server you can't. If the attacker can edit the files he can
edit the hashes too.

> These *are* resistant to preimage attacks. If I give you a SHA512
> checksum, there is no known practical method to generate a file with
> that same checksum. If I give you a CRC checksum, you can.

Randall didn't suggest any usage of CRCs where preimage attacks are
relevant. You just made that bit up.

>>> - since you are using CRC, it is quite easy for me to ensure the
>>>   checksums match after inserting malware;
>> 
>> No, you have yet *again* misunderstood the difference between the
>> client and the server.
>
> This was described as a peer-to-peer application. You even stated that it
> was a "pretty obvious" use-case, a "peer-to-peer dropbox". So is it
> peer-to-peer or client-server?

Both. It sounds a bit like there are clients which upload files
to a cloud of servers which are peers of each other. But seriously,
is this the source of all your confusion? Even if all the nodes
are pure "peers" (which it doesn't sound like they are), any
particular file will still have a source node which is therefore
the "client" for that file. You're trying to draw a hard distinction
where there is none.

>> lol. I suspected as much. You and Johannes were even more wrong than
>> was already obvious.
>
> You "suspected as much"? Such a pity you didn't speak up earlier and
> explain that cryptographic ciphers aren't generally resistant to
> preimage attacks.

I think you're misusing that phrase. But taking what you meant,
I suspected it was true (would they be reistant, after all?)
but I couldn't be bothered to check because the whole "crypto" bit
was a complete red-herring in the first place. The original discussion
wasn't about crypto, all the discussion about that was only because
you and Johannes wrongly insisted it was necessary.

[toc] | [prev] | [next] | [standalone]

#93374

On 06/30/2015 01:33 PM, Chris Angelico wrote:
>  From the software's point of view, it has two distinct
> modes: server, in which it listens on a socket and receives data, and
> client, in which it connects to other people's sockets and sends data.
> As such, the "server" mode is the only one that receives untrusted
> data from another user and stores it on the hard disk.


That's close.  There are 3 types: storage nodes, client nodes, and 
control nodes.

Communication:

storage node <--> control node
storage node <--> storage node
client node   --> storage node
client node   --> control node

Data is uploaded by clients and distributed among storage nodes.

Everything is coordinated by the control nodes (plural for redundancy).

-Randall

[toc] | [prev] | [next] | [standalone]

#93343

On 06/29/2015 10:00 PM, Steven D'Aprano wrote:
> On Tue, 30 Jun 2015 06:52 am, Randall Smith wrote:
>
>> Not sure why you posted the link.  The crc32 checksum is just to check
>> for possible filesystem corruption.  The system does periodic data
>> corruption checks.  BTRFS uses crc32 checksums also.  Please explain.
>
> The file system can trust that anything writing to a file is allowed to
> write to it, in doesn't have to defend against malicious writes. As I
> understand it, your application does.
>
> Here is the attack scenario I have in mind:
>
> - you write a file to my computer, and neglect to encrypt it;
> - and record the checksum for later;
> - I insert malware into your file;
> - you retrieve the file from me;
> - if the checksum matches what you have on record, you accept the file;
> - since you are using CRC, it is quite easy for me to ensure the
>    checksums match after inserting malware;
> - and I have now successfully injected malware into your computer.
>
> I'm making an assumption here -- I assume that the sender records a checksum
> for uploaded files so that when they get something back again they can tell
> whether or not it is the same content they uploaded.

Yes.  The client software computes sha256 checksums.

>
> * * *
>
> By the way, regarding the use of a substitution cipher, I spoke to the
> crypto guy at work, and "preimage attack" is not quite the right
> terminology, since that's normally used in the context of hash functions.
> It's almost a "known ciphertext attack", but not quite, since that
> terminology refers to guessing the key from the ciphertext.
>
> I was wrong: cryptographically strong ciphers are generally NOT resistant to
> what I described as a preimage attack. If the key leaks, using AES won't
> save you: an attacker with access to the key can produce a ciphertext that
> decrypts to the malware of his choice, regardless of whether you use
> AES-256 or rot-13. There may be other encryption methods which don't suffer
> from that, but he doesn't know of them off the top of his head.
>
> His comment was, "don't leak the key".

I'm pretty sure all encryption hinges on guarding the key.

>
> The other threat I mentioned is that the receiver will read the content of
> the file. For that, a strong cipher is much to be preferred over a weak
> one, and it needs to be encrypted by the sending end, not the receiving
> end. (If the receiving end does it, it has to keep the key so it can
> decrypt before sending back, which means the computer's owner can just grab
> the key and read the files.)
>


And again, that's why the client (data owner) uses AES.

[toc] | [prev] | [next] | [standalone]

#93348

On Wed, 1 Jul 2015 03:39 am, Randall Smith wrote:

> On 06/29/2015 10:00 PM, Steven D'Aprano wrote:

>> I'm making an assumption here -- I assume that the sender records a
>> checksum for uploaded files so that when they get something back again
>> they can tell whether or not it is the same content they uploaded.
> 
> Yes.  The client software computes sha256 checksums.

Thanks for clarifying.


[...]
>> His comment was, "don't leak the key".
> 
> I'm pretty sure all encryption hinges on guarding the key.

That would be Kerckhoffs' Principle, also known as Shannon's Maxim.

I don't think there has been much research into keeping at least *some*
security even when keys have been compromised, apart from as it relates to
two-factor authentication. (Assume that other people know the password to
your bank account. They can read your balance, but they can't steal your
money unless they first steal your phone or RSA token.)

In the past, and still today among people who don't understand Kerckhoffs'
principle, people have tried to keep the cipher secret and not have a key
at all. E.g. atbash, or caesar cipher, which once upon a time were cutting
edge ciphers, as laughably insecure as they are today. If the method was
compromised, all was lost. 

Today, if the key is compromised, all is lost. Is it possible that there are
ciphers that are resistant to discovery of the key? Obviously if you know
the key you can read encrypted messages, that's what the key is for, but
there are scenarios where you would want security to degrade gracefully
instead of in a brittle all-or-nothing manner:

- even if the attacker can read my messages, he cannot tamper with 
  them or write new ones as me.

(I'm pretty sure that, for example, the military would consider it horrible
if the enemy could listen in on their communications, but *even worse* if
the enemy could send false orders that appear to be legitimate.)

Sixty years ago, the idea of having a separate encryption key that you keep
secret and a decryption key that you can give out to everyone (public key
encryption) probably would have seemed ridiculous too.



-- 
Steven

[toc] | [prev] | [next] | [standalone]

Page 4 of 5 — ← Prev page 1 2 3 [4] 5  Next page →

Back to top | Article view | comp.lang.python

csiph-web