Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #93096 > unrolled thread

Re: Pure Python Data Mangling or Encrypting

Back to article view | Back to comp.lang.python

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Re: Pure Python Data Mangling or Encrypting Randall Smith <randall@tnr.cc> - 2015-06-24 13:36 -0500
    Re: Pure Python Data Mangling or Encrypting Steven D'Aprano <steve@pearwood.info> - 2015-06-25 14:07 +1000
      Re: Pure Python Data Mangling or Encrypting Devin Jeanpierre <jeanpierreda@gmail.com> - 2015-06-24 21:27 -0700
        Re: Pure Python Data Mangling or Encrypting Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2015-06-25 19:25 +1000
          Re: Pure Python Data Mangling or Encrypting Devin Jeanpierre <jeanpierreda@gmail.com> - 2015-06-25 02:41 -0700
          Re: Pure Python Data Mangling or Encrypting Chris Angelico <rosuav@gmail.com> - 2015-06-25 19:57 +1000
          Re: Pure Python Data Mangling or Encrypting Jon Ribbens <jon+usenet@unequivocal.co.uk> - 2015-06-25 10:03 +0000
            Re: Pure Python Data Mangling or Encrypting Steven D'Aprano <steve@pearwood.info> - 2015-06-26 01:13 +1000
              Re: Pure Python Data Mangling or Encrypting Jon Ribbens <jon+usenet@unequivocal.co.uk> - 2015-06-25 15:26 +0000
                Re: Pure Python Data Mangling or Encrypting Randall Smith <randall@tnr.cc> - 2015-06-25 13:58 -0500
                Re: Pure Python Data Mangling or Encrypting Chris Angelico <rosuav@gmail.com> - 2015-06-26 10:33 +1000
                  Re: Pure Python Data Mangling or Encrypting Jon Ribbens <jon+usenet@unequivocal.co.uk> - 2015-06-26 10:49 +0000
                Re: Pure Python Data Mangling or Encrypting Ian Kelly <ian.g.kelly@gmail.com> - 2015-06-25 19:01 -0600
                  Re: Pure Python Data Mangling or Encrypting Steven D'Aprano <steve@pearwood.info> - 2015-06-27 03:06 +1000
                    Re: Pure Python Data Mangling or Encrypting Randall Smith <randall@tnr.cc> - 2015-06-26 15:09 -0500
                      Re: Pure Python Data Mangling or Encrypting Johannes Bauer <dfnsonfsduifb@gmx.de> - 2015-06-26 23:07 +0200
                        Re: Pure Python Data Mangling or Encrypting Jon Ribbens <jon+usenet@unequivocal.co.uk> - 2015-06-26 21:29 +0000
                          Re: Pure Python Data Mangling or Encrypting Mark Lawrence <breamoreboy@yahoo.co.uk> - 2015-06-26 22:55 +0100
                          Re: Pure Python Data Mangling or Encrypting Johannes Bauer <dfnsonfsduifb@gmx.de> - 2015-06-27 00:42 +0200
                            Re: Pure Python Data Mangling or Encrypting Devin Jeanpierre <jeanpierreda@gmail.com> - 2015-06-26 16:26 -0700
                            Re: Pure Python Data Mangling or Encrypting Jon Ribbens <jon+usenet@unequivocal.co.uk> - 2015-06-27 00:21 +0000
                            Re: Pure Python Data Mangling or Encrypting Randall Smith <randall@tnr.cc> - 2015-06-26 19:55 -0500
                              Re: Pure Python Data Mangling or Encrypting Johannes Bauer <dfnsonfsduifb@gmx.de> - 2015-06-27 07:24 +0200
                          Re: Pure Python Data Mangling or Encrypting Randall Smith <randall@tnr.cc> - 2015-06-26 19:12 -0500
                        Re: Pure Python Data Mangling or Encrypting Ian Kelly <ian.g.kelly@gmail.com> - 2015-06-26 15:58 -0600
                        Re: Pure Python Data Mangling or Encrypting Randall Smith <randall@tnr.cc> - 2015-06-26 19:23 -0500
                      Re: Pure Python Data Mangling or Encrypting Johannes Bauer <dfnsonfsduifb@gmx.de> - 2015-06-26 23:11 +0200
                        Re: Pure Python Data Mangling or Encrypting Michael Torrie <torriem@gmail.com> - 2015-06-27 11:02 -0600
                          Re: Pure Python Data Mangling or Encrypting Paul Rubin <no.email@nospam.invalid> - 2015-06-27 10:45 -0700
                      Re: Pure Python Data Mangling or Encrypting Steven D'Aprano <steve@pearwood.info> - 2015-06-27 13:38 +1000
                        Re: Pure Python Data Mangling or Encrypting Devin Jeanpierre <jeanpierreda@gmail.com> - 2015-06-26 21:05 -0700
                          Re: Pure Python Data Mangling or Encrypting Steven D'Aprano <steve@pearwood.info> - 2015-06-27 16:16 +1000
                            Re: Pure Python Data Mangling or Encrypting Devin Jeanpierre <jeanpierreda@gmail.com> - 2015-06-27 13:30 -0700
                              Re: Pure Python Data Mangling or Encrypting Steven D'Aprano <steve@pearwood.info> - 2015-06-28 11:18 +1000
                                Re: Pure Python Data Mangling or Encrypting Devin Jeanpierre <jeanpierreda@gmail.com> - 2015-06-27 19:11 -0700
                        Re: Pure Python Data Mangling or Encrypting Ian Kelly <ian.g.kelly@gmail.com> - 2015-06-26 23:47 -0600
                          Re: Pure Python Data Mangling or Encrypting Steven D'Aprano <steve@pearwood.info> - 2015-06-27 18:38 +1000
                            Re: Pure Python Data Mangling or Encrypting Chris Angelico <rosuav@gmail.com> - 2015-06-27 18:53 +1000
                              Re: Pure Python Data Mangling or Encrypting Johannes Bauer <dfnsonfsduifb@gmx.de> - 2015-06-27 11:07 +0200
                                Re: Pure Python Data Mangling or Encrypting Chris Angelico <rosuav@gmail.com> - 2015-06-27 19:17 +1000
                                  Re: Pure Python Data Mangling or Encrypting Jon Ribbens <jon+usenet@unequivocal.co.uk> - 2015-06-27 09:27 +0000
                                    Re: Pure Python Data Mangling or Encrypting Johannes Bauer <dfnsonfsduifb@gmx.de> - 2015-06-27 12:05 +0200
                                      Re: Pure Python Data Mangling or Encrypting Chris Angelico <rosuav@gmail.com> - 2015-06-27 20:16 +1000
                                        Re: Pure Python Data Mangling or Encrypting Johannes Bauer <dfnsonfsduifb@gmx.de> - 2015-06-27 12:55 +0200
                                      Re: Pure Python Data Mangling or Encrypting Jon Ribbens <jon+usenet@unequivocal.co.uk> - 2015-06-27 10:26 +0000
                                    Re: Pure Python Data Mangling or Encrypting Laura Creighton <lac@openend.se> - 2015-06-27 14:27 +0200
                                  Re: Pure Python Data Mangling or Encrypting Johannes Bauer <dfnsonfsduifb@gmx.de> - 2015-06-27 12:18 +0200
                                    Re: Pure Python Data Mangling or Encrypting Chris Angelico <rosuav@gmail.com> - 2015-06-27 21:33 +1000
                                    Re: Pure Python Data Mangling or Encrypting Ian Kelly <ian.g.kelly@gmail.com> - 2015-06-27 08:59 -0600
                                  Re: Pure Python Data Mangling or Encrypting Laura Creighton <lac@openend.se> - 2015-06-27 13:25 +0200
                                    Re: Pure Python Data Mangling or Encrypting Jussi Piitulainen <jpiitula@ling.helsinki.fi> - 2015-06-27 15:23 +0300
                                    Re: Pure Python Data Mangling or Encrypting Laura Creighton <lac@openend.se> - 2015-06-27 14:48 +0200
                            Re: Pure Python Data Mangling or Encrypting Johannes Bauer <dfnsonfsduifb@gmx.de> - 2015-06-27 11:12 +0200
                            Re: Pure Python Data Mangling or Encrypting Ian Kelly <ian.g.kelly@gmail.com> - 2015-06-27 09:09 -0600
                              Re: Pure Python Data Mangling or Encrypting Steven D'Aprano <steve@pearwood.info> - 2015-06-28 03:35 +1000
                                Re: Pure Python Data Mangling or Encrypting Steven D'Aprano <steve@pearwood.info> - 2015-06-28 03:58 +1000
                                Re: Pure Python Data Mangling or Encrypting Ian Kelly <ian.g.kelly@gmail.com> - 2015-06-27 14:16 -0600
                                Re: Pure Python Data Mangling or Encrypting Jon Ribbens <jon+usenet@unequivocal.co.uk> - 2015-06-28 13:41 +0000
                        Re: Pure Python Data Mangling or Encrypting Robert Kern <robert.kern@gmail.com> - 2015-06-27 08:58 +0100
                        Re: Pure Python Data Mangling or Encrypting Robert Kern <robert.kern@gmail.com> - 2015-06-27 09:07 +0100
                        Re: Pure Python Data Mangling or Encrypting Dennis Lee Bieber <wlfraed@ix.netcom.com> - 2015-06-27 10:39 -0400
                      Re: Pure Python Data Mangling or Encrypting Grant Edwards <invalid@invalid.invalid> - 2015-06-27 12:38 +0000
                        Re: Pure Python Data Mangling or Encrypting Randall Smith <randall@tnr.cc> - 2015-06-27 13:22 -0500
                          Re: Pure Python Data Mangling or Encrypting Steven D'Aprano <steve@pearwood.info> - 2015-06-28 04:51 +1000
                            Re: Pure Python Data Mangling or Encrypting Chris Angelico <rosuav@gmail.com> - 2015-06-28 09:05 +1000
                    Re: Pure Python Data Mangling or Encrypting Chris Angelico <rosuav@gmail.com> - 2015-06-27 11:21 +1000
                    Re: Pure Python Data Mangling or Encrypting Ian Kelly <ian.g.kelly@gmail.com> - 2015-06-26 23:59 -0600
                      Re: Pure Python Data Mangling or Encrypting Jon Ribbens <jon+usenet@unequivocal.co.uk> - 2015-06-27 09:26 +0000
                    Re: Pure Python Data Mangling or Encrypting Chris Angelico <rosuav@gmail.com> - 2015-06-27 16:52 +1000
                    Re: Pure Python Data Mangling or Encrypting Randall Smith <randall@tnr.cc> - 2015-06-27 12:08 -0500
                      Re: Pure Python Data Mangling or Encrypting Steven D'Aprano <steve@pearwood.info> - 2015-06-28 04:50 +1000
                        Re: Pure Python Data Mangling or Encrypting Randall Smith <randall@tnr.cc> - 2015-06-29 15:52 -0500
                          Re: Pure Python Data Mangling or Encrypting Steven D'Aprano <steve@pearwood.info> - 2015-06-30 13:00 +1000
                            Re: Pure Python Data Mangling or Encrypting Jon Ribbens <jon+usenet@unequivocal.co.uk> - 2015-06-30 12:19 +0000
                              Re: Pure Python Data Mangling or Encrypting Steven D'Aprano <steve@pearwood.info> - 2015-07-01 04:17 +1000
                                Re: Pure Python Data Mangling or Encrypting Chris Angelico <rosuav@gmail.com> - 2015-07-01 04:33 +1000
                                Re: Pure Python Data Mangling or Encrypting Jon Ribbens <jon+usenet@unequivocal.co.uk> - 2015-06-30 18:37 +0000
                                Re: Pure Python Data Mangling or Encrypting Randall Smith <randall@tnr.cc> - 2015-07-01 09:38 -0500
                            Re: Pure Python Data Mangling or Encrypting Randall Smith <randall@tnr.cc> - 2015-06-30 12:39 -0500
                              Re: Pure Python Data Mangling or Encrypting Steven D'Aprano <steve@pearwood.info> - 2015-07-01 04:59 +1000
                                Re: Pure Python Data Mangling or Encrypting Chris Angelico <rosuav@gmail.com> - 2015-07-01 05:20 +1000
                                Re: Pure Python Data Mangling or Encrypting Jon Ribbens <jon+usenet@unequivocal.co.uk> - 2015-06-30 23:25 +0000
                                  Re: Pure Python Data Mangling or Encrypting alister <alister.nospam.ware@ntlworld.com> - 2015-07-01 08:06 +0000
                      Re: Pure Python Data Mangling or Encrypting Jon Ribbens <jon+usenet@unequivocal.co.uk> - 2015-06-28 14:21 +0000
                        Re: Pure Python Data Mangling or Encrypting Randall Smith <randall@tnr.cc> - 2015-06-29 15:46 -0500
                          Re: Pure Python Data Mangling or Encrypting Jon Ribbens <jon+usenet@unequivocal.co.uk> - 2015-06-29 20:49 +0000
                            Re: Pure Python Data Mangling or Encrypting Randall Smith <randall@tnr.cc> - 2015-06-30 12:43 -0500
                      Re: Pure Python Data Mangling or Encrypting Gregory Ewing <greg.ewing@canterbury.ac.nz> - 2015-07-02 10:31 +1200
                Re: Pure Python Data Mangling or Encrypting Mark Lawrence <breamoreboy@yahoo.co.uk> - 2015-06-26 02:17 +0100
                Re: Pure Python Data Mangling or Encrypting Chris Angelico <rosuav@gmail.com> - 2015-06-26 12:06 +1000
                Re: Pure Python Data Mangling or Encrypting Chris Angelico <rosuav@gmail.com> - 2015-06-26 12:05 +1000
                Re: Pure Python Data Mangling or Encrypting Mark Lawrence <breamoreboy@yahoo.co.uk> - 2015-06-26 03:24 +0100
                Re: Pure Python Data Mangling or Encrypting Chris Angelico <rosuav@gmail.com> - 2015-06-26 12:29 +1000
          Re: Pure Python Data Mangling or Encrypting Joonas Liik <liik.joonas@gmail.com> - 2015-06-25 13:00 +0300
          Re: Pure Python Data Mangling or Encrypting Devin Jeanpierre <jeanpierreda@gmail.com> - 2015-06-25 03:18 -0700
      Re: Pure Python Data Mangling or Encrypting Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2015-06-25 17:05 +1000
      Re: Pure Python Data Mangling or Encrypting Randall Smith <randall@tnr.cc> - 2015-06-25 14:09 -0500

Page 2 of 5 — ← Prev page 1 [2] 3 4 5  Next page →

#93213

On 2015-06-26, Johannes Bauer <dfnsonfsduifb@gmx.de> wrote:
> On 26.06.2015 23:29, Jon Ribbens wrote:
>>> While you seem to think that Steven is rampaging about nothing, he does
>>> have a fair point: You consistently were vague about wheter you want to
>>> have encryption, authentication or obfuscation of data. This suggests
>>> that you may not be so sure yourself what it is you actually want.
>> 
>> He hasn't been vague, you and Steven just haven't been paying
>> attention.
>
> Bullshit. Even the topic indicates that he doesn't know what he wants:
> "data mangling" or "encryption", which one is it?

He wants data mangling and he was asking whether he needed encryption
to achieve it. The answer is no, he doesn't.

> I could go into detail about how the assumtion that the ciphertext is
> secret is not a smart one in the context of cryptography.

But, and I've already pointed this out and you don't seem to have
quite got the picture yet, we're not in the context of cryptography.

> And how side channels and other leakage may affect overall system
> security. But I'm going to save my time on that. I do get paid to
> review cryptographic systems and part of the job is dealing with
> belligerent people who have read Schneier's blog and think they can
> outsmart anyone else.

You seem to be describing your own attitude to a tee.

> Since I don't get paid to convice you, it's absolutely fine that you
> think your substitution scheme is the grand prize.

"My" scheme? It wasn't my suggestion.

> So the topic says "Encrypting". If you look really closely at the word,
> the part "crypt" might give away to you that cryptography is involved.

If you were to actually read past the subject line and continue on to
read the text of the articles, you would discover that cryptography is
not involved. No wonder you're confused if you're disengaging your
brain the instant you get past the subject line.

>> He's just trying to avoid letting third parties write completely
>> arbitrary data to the disk.
>
> There's your requirement.

"My" requirement?

>> You know what would be a perfectly good solution to his problem?
>> Base 64 encoding. That would solve the issue pretty much
>> completely, the only reason it's not an ideal solution is that it
>> of course increases the size of the data.
>
> ...wow.
>
> That's a nice interpretation of not letting a third party write
> completely arbitrary data.

It's an accurate interpretation. Something that seems not to be your
forte.

> According to your definition, this would be: It's okay if the
> attacker can control 6 of 8 bits.

Yes, it probably is ok. Add a bit of random gunk at the top and tail
of the file and it's almost certainly ok. Why do you think it's not?

> Oh I understand your "solutions" plenty well.

Evidently not.

> The only thing I don't understand is why you don't own a Fields
> medal yet for your groundbreaking work on bulletproof obfuscation.

That is clearly a very long way from the only thing you don't
understand.

[toc] | [prev] | [next] | [standalone]

#93215

On 06/26/2015 05:42 PM, Johannes Bauer wrote:
> On 26.06.2015 23:29, Jon Ribbens wrote:
>
>>> While you seem to think that Steven is rampaging about nothing, he does
>>> have a fair point: You consistently were vague about wheter you want to
>>> have encryption, authentication or obfuscation of data. This suggests
>>> that you may not be so sure yourself what it is you actually want.
>>
>> He hasn't been vague, you and Steven just haven't been paying
>> attention.
>
> Bullshit. Even the topic indicates that he doesn't know what he wants:
> "data mangling" or "encryption", which one is it?

I knew exactly what I wanted and spelled it out.

"protect the recipient against exposure to nefarious data ... before it 
is written to disk"

You shouldn't need to make assumptions about other parts of the system. 
  Just prevent potential malware from hitting the disk as such.

Before this thread, I knew that encryption would definitely work and 
data mangling might.  Now I know that data mangling is a really nice 
solution for the given requirements.


>
>>> You always play around with the 256! which would be a ridiculously high
>>> security margin (1684 bits of security, woooo!). You totally ignore that
>>> the system can be broken in a linear fashion.
>>
>> No, it can't, because the attacker does not have access to the
>> ciphertext.
>
> Or so you claim.

No the attacker does not have access to the ciphertext.  What would lead 
you to think they did?

This statement is central to the problem:

"I'd like to protect the recipient against exposure to nefarious data by 
mangling or encrypting the data before it is written to disk"

This makes it clear I'm not trying to encrypt data to protect the data. 
  I'm trying to protect the recipient (storage server) from an attack. 
This specific attack being malware.  Yes, AES encryption would have 
worked here, but encryption is not the objective.

>
> I could go into detail about how the assumtion that the ciphertext is
> secret is not a smart one in the context of cryptography. And how side
> channels and other leakage may affect overall system security. But I'm
> going to save my time on that. I do get paid to review cryptographic
> systems and part of the job is dealing with belligerent people who have
> read Schneier's blog and think they can outsmart anyone else. Since I
> don't get paid to convice you, it's absolutely fine that you think your
> substitution scheme is the grand prize.


All of which has nothing to do with this thread.  Actual encryption is 
handled using AES and TLS.  This is not about encryption.

>
>>> Nobody assumes you're a moron. But it's safe to assume that you're a
>>> crypto layman, because only laymen have no clue on how difficult it is
>>> to get cryptography even remotely right.
>>
>> Amateur crypto is indeed a bad idea. But what you're still not getting
>> is that what he's doing here *isn't crypto*.
>
> So the topic says "Encrypting". If you look really closely at the word,
> the part "crypt" might give away to you that cryptography is involved.
>


This isn't about encrypting data to protect the data.  All the 
encryption I do uses standard AES and TLS.  Yes, I do understand that 
crypto is best left to experts.  The topic says "Encrypting" because I 
knew that encrypting the data would properly obfuscate it.


>> He's just trying to avoid
>> letting third parties write completely arbitrary data to the disk.
>
> There's your requirement. Then there's obviously some kind of
> implication when a third party *can* write arbitrary data to disk. And
> your other solution to that problem...

It's a network protocol.  Just like when writing a web app, you have to 
deal with bad actors.  That's what I'm doing here.  The entire service 
is about handling arbitrary data.  Just like Amazon S3 handles people's 
"arbitrary data".  Not sure what you mean by "third party".  It would be 
a registered peer.  But registration doesn't prevent the scenario in 
discussion.

[toc] | [prev] | [next] | [standalone]

#93221

On 27.06.2015 02:55, Randall Smith wrote:

> No the attacker does not have access to the ciphertext.  What would lead
> you to think they did?

Years of practical experience in the field of applied cryptography.
Knowledge of how side channels work and how easily they can be
constructed for bad schemes.

Rest snipped, explanation futile.
Cheers,
Johannes

-- 
>> Wo hattest Du das Beben nochmal GENAU vorhergesagt?
> Zumindest nicht öffentlich!
Ah, der neueste und bis heute genialste Streich unsere großen
Kosmologen: Die Geheim-Vorhersage.
 - Karl Kaos über Rüdiger Thomas in dsa <hidbv3$om2$1@speranza.aioe.org>

[toc] | [prev] | [next] | [standalone]

#93212

On 06/26/2015 04:55 PM, Mark Lawrence wrote:

>
> To be perfectly blunt I gave up days ago trying to follow what was being
> said, just too many words from all angles and too few diagrams for me to
> follow.  I sincerely hope it doesn't end in tears.
>

Mark.

There's not much to follow.  The solution was simple and complete.

The original description was limited to a small part of a large, more 
complex system.  The reason you've had trouble following is because 
several people made (very bad) assumptions about what the rest of the 
system did.  Everything required for the solution was present in the 
original post.

-Randall

[toc] | [prev] | [next] | [standalone]

#93209

On Fri, Jun 26, 2015 at 3:07 PM, Johannes Bauer <dfnsonfsduifb@gmx.de> wrote:
> That people in 2015 actually defend inventing a substitution-cipher
> "crypto"system sends literally shivers down my spine.

I think that the people defending this have been reasonably consistent
about using the word "obfuscation", not "crypto".

[toc] | [prev] | [next] | [standalone]

#93214

On 06/26/2015 04:07 PM, Johannes Bauer wrote:
> You consistently were vague about wheter you want to
> have encryption, authentication or obfuscation of data.

I knew (possibly extra) encryption wasn't necessary at this stage, but I 
also knew that encryption would provide good obfuscation.  Problem is, I 
didn't want an extra C library to install. See the original post.

"... I'd like to protect the recipient against exposure to nefarious 
data by mangling or encrypting the data before it is written to disk. My 
original idea was for the recipient to encrypt using AES.  But I want to 
keep this software pure Python "batteries included" and not require 
installation of other platform-dependent software ... I don't know that 
I really need encryption here, but some type of fast mangling algorithm 
where a bad actor sending a payload can't guess the output ahead of time."

-Randall

[toc] | [prev] | [next] | [standalone]

#93206

On 26.06.2015 22:09, Randall Smith wrote:

> And that's why we're having this discussion.  Do you know of an attack
> in which you can control the output (say at least 100 consecutive bytes)
> for data which goes through a 256 byte translation table, chosen
> randomly from 256! permutations after the data is sent.  If you do, I'm
> all ears!  But at this point you're just setting up straw men and
> knocking them down.

Oh and I wanted to comment on this as well, but sent my reply too soon.

You misunderstand. This is now how it works, this is not how any of this
works. Steven does not *at all* have to prove to you your system is
breakable or show actual attacks. YOU have to prove that your system is
secure. Either analytically or you wait until you have peer review and
cryptanalysis by actual experts.

It's *very* easy to set up a badly flawed obfuscation system that can't
be broken by laymen in a Python newsgroup and which appers to be secure.
This does not imply one bit that it is even remotely secure.

Cheers,
Johannes

-- 
>> Wo hattest Du das Beben nochmal GENAU vorhergesagt?
> Zumindest nicht öffentlich!
Ah, der neueste und bis heute genialste Streich unsere großen
Kosmologen: Die Geheim-Vorhersage.
 - Karl Kaos über Rüdiger Thomas in dsa <hidbv3$om2$1@speranza.aioe.org>

[toc] | [prev] | [next] | [standalone]

#93251

On 06/26/2015 03:11 PM, Johannes Bauer wrote:
> You misunderstand. This is now how it works, this is not how any of this
> works. Steven does not *at all* have to prove to you your system is
> breakable or show actual attacks. YOU have to prove that your system is
> secure. 

Ahh the holy grail of computer science.  Now it's been a while since I
finished my CS degree, but I recall spending a lot of time in class
talking about the proving code correctness, which is a similar problem,
and learning that that was thought to be NP complete.  Furthermore you
cannot prove a negative, which is what proving security is for anything
but the trivial case.

Are you saying this is untrue?

Obviously there are best practices, which you are an expert in.  But how
does one prove a system is secure except by enumerating attack vectors
and addressing each one, preferably in the design phase?

[toc] | [prev] | [next] | [standalone]

#93254

Michael Torrie <torriem@gmail.com> writes:
> Furthermore you cannot prove a negative, which is what proving
> security is for anything but the trivial case. Are you saying this is
> untrue?

I've always thought that there are no two even numbers that when you add
them together, give you an odd number.  Are you saying that statement
can't be proven?

> But how does one prove a system is secure except by enumerating attack
> vectors

In the case of encryption, you do a reduction proof to a recognized
primitive like AES.  That is, you show that if your system is breakable,
you can transform the break into a break against AES itself.  That's the
best you can do at the moment, because the open status of the P!=NP
problem means that no one knows how to prove that any primitive (such as
AES) is secure.  The reduction proof means that the evidence for AES's
security also applies to your system.

Of course that's just for the cipher itself.  For the entire surrounding
software/hardware/process system which is mostly not mathematical,
you're right, there's no way to (mathematically) prove security or even
to define it.

[toc] | [prev] | [next] | [standalone]

#93217

On Sat, 27 Jun 2015 06:09 am, Randall Smith wrote:

> On 06/26/2015 12:06 PM, Steven D'Aprano wrote:
>> On Fri, 26 Jun 2015 11:01 am, Ian Kelly wrote:
>>
>>> You're making the same mistake that Steven did in misunderstanding the
>>> threat model.
>>
>> I don't think I'm misunderstanding the threat, I think I'm pointing out a
>> threat which the OP is hoping to just ignore.
> 
> I'm not hoping to ignore anything.  I didn't explain the entire system,
> as it was not necessary to find a solution to the problem at hand.  But
> since you want to make negative assumptions about what I didn't tell
> you, I'll gladly address your accusations of negligence.

"Negligence" is *your* word, not mine. I've never said that. And I'm not
*assuming* anything, everything I've stated has been based on the evidence
of what you have written. I've even gone so far as to EXPLICITLY say that I
cannot know for a fact that your application is vulnerable to these
threats, since I'm only going from a description rather than the app
itself. But your responses don't suggest that you have these threats under
control, on the contrary, they indicate that you are *far* underestimating
the seriousness of them and overestimating the difficulty of running a
secure application on a machine you cannot trust.

If your application has any saving grace, it is that there are easier ways
to get malware onto somebody else's computer. There are a hundred million
unsecured Windows boxen out there, if I were malicious I would just hire a
bot net rather than spend the time trying to hack your system. But maybe
somebody else will do it just for the lulz, or to prove it can be done.
Some black hats like a challenge, and yours appears to fall nicely into
that middle ground of hard enough to be interesting but not hard enough to
be really difficult.


>> In an earlier post, I suggested that the threat model should involve at
>> least *three* different attacks, apart from the usual man-in-the-model
>> attacks of data in transit.
> 
> All communication is secured using TLS and authentication handled by
> X.509 certificates.  This prevents man in the middle attacks.
> Certificates are signed by CAs I control.

You control the CAs? Presumably that means they're self-signed (unless you
mean you get to choose the CA). I don't know if that makes a difference or
not.


>> One is that the attacker is the person sending the data. E.g. I want to
>> send a nasty payload (say, malware, or an offensive image). Another is
>> that the attacker is the recipient of the file, who wants to read the
>> sender's data.
> 
> The only person who can read a file is the owner.   AES encryption is
> built into the client software.  The only way data can be uploaded
> unencrypted is if encryption is intentionally disabled.

With respect Randall, you contradict yourself. Is there any wonder that some
of us (well, me at least) is suspicious and confused, when your story
changes as often as the weather?

Sometimes you say that the client software uses AES encryption. Sometimes
you say that you don't want to use AES encryption because you want the
client to be pure Python, and a pure-Python implementation would be too
slow. Your very first post says:

    My original idea was for the recipient to encrypt using AES.  But
    I want to keep this software pure Python "batteries included" and
    not require installation of other platform-dependent software.  
    Pure Python AES and even DES are just way too slow.


Sometimes you say the user is supposed to encrypt the data themselves:

    While the data senders are supposed to encrypt data, that's not
    guaranteed


Now you say that the application encrypts the data, except that the user can
turn that option off.

Just make the AES encryption mandatory, not optional. Then the user cannot
upload unencrypted malicious data, and the receiver cannot read the data.
That's two problems solved.

Making AES or similarly strong encryption mandatory protects both the sender
of data and the receiver of data. I cannot imagine why you are considering
making it optional, since that only adds more work for you and reduces the
security of your users.

Oh, and DES is not good enough.


>> As far as I can tell, the OP's plan to defend the sender's privacy is to
>> dump responsibility for encrypting the files in the sender's lap. As far
>> as I'm concerned, perhaps as many as one user in 20000 will pre-encrypt
>> their files. (Early adopters will be unrepresentative of the eventual
>> user base of this system. If this takes off, the user base will likely
>> end up dominated by people who think that "qwerty" is the epitome of
>> unguessable passwords.)
> 
> Making assumptions again.  See above.  The client software encrypts by
> default.  You're also assuming there is no password strength checking.

My comment about "qwerty" as a password was a comment on the majority of
people on the internet, not an assumption about your application.


>> Users just don't use crypto unless their applications do it for them.
> 
> And it does.

Great to hear it! Just make sure your application always encrypts the
uploaded files, and you protect both the sender of the files and the
receiver.

At least from these threats. There are others.

Just to indicate how hard this is, here is a ten year old timing attack
against AES:

http://cr.yp.to/antiforgery/cachetiming-20050414.pdf

Is your application vulnerable to timing attacks? No idea. Talk to a
security expert.



>> My opinion is that the application ought to do so, and not expect Aunt
>> Tillie to learn how to correctly use encryption software before uploading
>> her files.
>>
>> http://www.catb.org/jargon/html/A/Aunt-Tillie.html
>>
>> It is the OP's prerogative to disagree, of course, but to me, if the OP's
>> app doesn't use strong crypto to encrypt users' data, that's tantamount
>> to saying they don't care about their users' data privacy. Using a
>> monoalphabetic substitution cipher to obfuscate the data is not strong
>> crypto.
> 
> You've gone on a rampage about nothing.  My original description said
> the client was supposed to encrypt the data, but you want to assume the
> opposite for some unknown reason.

Hardly a "rampage", and not "some unknown reason". End-users don't do
crypto. About one person in a hundred thousand digitally signs their
emails. Even people who know better don't use crypto. I don't use crypto.
If you are relying on people to encrypt their files, as you have suggested
in the past, *it won't happen*.

If the app does encrypt the data with AES before sending, then you don't
gain any benefit by obfuscating an encrypted file with a classical
monoalphabetic substitution cipher.


>>> The goal isn't to prevent the attacker from working out
>>> the key for a file that has already been obfuscated. Any real data
>>> that might be exposed by a vulnerability in the server is presumed to
>>> have already been strongly encrypted by the user.
>>
>> I think that's a ridiculously unrealistic presumption, unless your
>> user-base is entirely taken from a very small subset of security savvy
>> and pedantically careful users.
> 
> The difference is he's not assuming I'm a moron.  He's giving me the
> benefit of the doubt.  That plus I actually said, "data senders are
> supposed to encrypt data".

I'm not assuming you are a moron, I'm making a judgement based on your posts
that you might be out of your depth when it comes to crypto. Maybe I'm
wrong, and you know what you're doing, you just don't know how to
communicate that fact. This is an example: the sender is supposed to
encrypt the data, or the application encrypts it for them? Sometimes you
say the sender encrypts the data, sometimes you say the application
encrypts the data. It makes a big difference whether or not Aunt Tillie has
to run a separate encryption application before uploading her files.

If the application does it (a very good thing!) then there is the mystery of
why you would allow people to turn that option off. That can only make more
work for you and reduce the security of your application for both senders
and receivers.


> In a networked system, you can't make assumptions about what the other
> peers are doing.  You have to handle what comes across the wire.  You
> also have to consider that you may come under attack.  That's what this
> is about.

Right.

And you're in a difficult situation because your users (sender and receiver)
can't trust each other. This is harder than (say) Bittorrent.

With BT, the receiver can't trust the sender, but there's no threat to the
sender. (Well, legal threats, but that's a social issue, not a technical
one.) If I (let's assume legally) upload a file to a BT network, I don't
worry about downloaders reading the file. I want them to read it. That's
not the case here.


>>> The goal is to prevent the attacker from guessing a key that hasn't
>>> even been generated yet, which could be exploited to engineer the
>>> obfuscated content into something malicious.
>>
>> They don't need to predict the key exactly. If they can predict that the
>> key will be, lets say, one of these thousand values, then they can
>> generate one thousand files and upload them. One of them will match the
>> key, and there's your exploit. That's one attack.
> 
> Thousand Values ???  Isn't it 256!, which is just freaking huge!  import
> math; math.factorial(256)

No. There are 256! possible keys *in total*, but that doesn't necessarily
mean that the keys are unpredictable.

Suppose that you hire an intern to write the "choose key" function, and not
knowing any better, he simply iterates through the keys in numeric order,
one after the other. So the first upload will use key 0, the second key 1,
the third key 2, and so on, until key 256! - 1, then start again. In that
case, predicting the next key is *trivial*. If I can work out what key you
send now (I just upload a file containing "\x00\x01\x02...\xFF" to myself
and see what I get), then I know what key the app will use next.

Now obviously you're not going to let the new intern write such a critical
part of the application, not without a senior developer doing a code
review. He reviews the code, and mixes the keys up in some fashion, the
more unpredictable the better. So he uses a random number generator to
choose the key. 

Maybe you use Python's standard library and the Mersenne Twister. The period
of that is huge, possibly bigger than 256! (or not, I forget, and I'm too
lazy to look it up). So you think that's safe. But it's not: Mersenne
Twister is not a cryptographically secure pseudorandom number generator. If
I can get some small number of values from the Twister (by memory,
something of the order of 100 such values) then I can predict the rest for
ever.

Even if I can't do that, I might be able to guess the seed: I know what time
the application started up, to within a few milliseconds, and I know (or
can guess) how many random numbers you have used, so I can predict that it
will be so far into the period, give or take a few hundred or a thousand
values. Instead of having to guess the key out of 256! possible values, I
now only have to guess the key out of 1000 possible values. I can keep
sending files to myself until I work out which key it is, and from that
point on I can now predict every key with 100% certainty.

Okay, so you're smarter than that. You know that Python's Mersenne Twister
is not a CSPRNG, and you use /dev/urandom or its Windows equivalent.
Excellent.

Except... you're getting your random numbers from a system *I* control. I
can just feed you whatever "random" numbers I want.

I don't know how to solve that one. Writing secure code where you cannot
trust the machine you are running on is *immeasurably tougher* than writing
secure code on a trusted machine.


>> A second attack is to force the key. The attacker controls the machine
>> the application is running on, they control /dev/urandom and can feed
>> your app whatever not-so-random numbers they like, so potentially they
>> can force the app to use the key of their choosing. Then they don't need
>> 1000 files, they just need one.
>>
> 
> If the attacker controlled the machine the app was on, why would it fool
> with /dev/urandom?  I think he'd just plant the files he wanted to plant
> and be done.  This is non-nonsensical anyway.

No, you don't understand the nature of the attack. In this scenario, the
sender is the attacker. I want to upload malicious files to the receiver.
You are trying to stop me, that's the whole point of "mangling or
encrypting" the files. (Your words.) So I, the sender, prepare a file such
that when you mangle it, the resulting mangled content is the malicious
content I want.

If you use a substitution cipher, I can do this if I can guess or force the
key. If you use strong crypto, I can't.

However, I can hack the application. The client sits on my computer, it's
pure Python, even if it isn't I can still hack the application, I don't
need access to the source code.


>> That's two. Does anyone think that I've thought of all the possible
>> attacks?
>>
>> (Well, hypothetical attacks. I acknowledge that I don't know the
>> application, and cannot be sure that it *actually is* vulnerable to these
>> attacks.)
>>
>> The problem here is that a monoalphabetic substitution cipher is not
>> resistant to preimage attacks. Your only defence is that the key is
>> unknown. If the attacker can force the key, or predict the key, or guess
>> a small range of keys, they can exploit your weak cipher.
>>
>> (Technically, "preimage attack" is usually used to refer to attacks on
>> hash functions. I'm not sure if the same name is used for attacks on
>> ciphers.)
>>
>> https://en.wikipedia.org/wiki/Preimage_attack
>>
>> With a strong crypto cipher, there are no known preimage attacks. Even if
>> the attacker knows exactly what key you are using, they cannot predict
>> what preimage they need to supply in order to generate the malicious
>> payload they want after encryption. (As far as I know.)
>>
>> That is the critical issue right there. The sort of simple monoalphabetic
>> substitution cipher using bytes.translate that the OP is using is
>> vulnerable to preimage attacks. Strong crypto is not.
> 
> It isn't vulnerable to preimage attacks unless you can guess the key out
> of 256! possibilities.

Yes. Do you think that's hard for an attacker who has access to your
application, possibly including the source code, and controls all the
sources of entropy on the system your application is running on?

I don't have to *randomly* guess. I control what time your application
starts, I control what randomness you get from /dev/urandom, I control how
many keys you go through, I might even be able to read the source code of
the application (not that I need to, that just makes it easier).


> The key doesn't even exist until after the data 
> is sent.  Give me one plausible scenario where an attacker can cause
> malware to hit the disk after bytearray.translate with a 256 byte
> translation table and I'll be thankful to you.  As it stands now, you're
> either ignoring information I've already given, assuming I've made
> moronic design decisions the pouncing on them, or completely
> misunderstanding the issue at hand.

If you still don't understand the security threats after I've explained them
repeatedly, then your app is doomed to join the multitude of insecure and
unsafe applications on the internet.

Actually, the more I think about this, the more I come to think that the
only way this can be secure is for both the sending client application and
the receiving client appl to *both* encrypt the data. The sender can't
trust the receiver not to read the files, so the sender has to encrypt; the
receiver can't trust the sender not to send malicious files, so the
receiver has to encrypt too.


>>> There are no
>>> frequency-based attacks possible here, because you can't do frequency
>>> analysis on the result of a key that hasn't even been generated yet.
>>
>> Frequency-based attacks apply to a different threat. I'm referring to at
>> least two different attacks here, with different attackers and different
>> victims. Don't mix them up.
>>
>>
>>> Assuming that you have no attack on the key generation itself, the
>>
>> Not a safe assumption!
> 
> For this case it is a safe assumption.  For the same reason you're
> assuming the PSU isn't defective.

A detective power supply isn't a security threat. It doesn't enable me to
upload malicious files to an unsuspecting receiver.


> An attack on /dev/urandom for 
> instance would also compromise TLS, and every sort of secure key
> generation, not just the key for byte translation.  That is a separate
> problem altogether with a separate solution.

Er, yes? And how does that make it invalid?



>>> best you can do is send a file deobfuscated with a random key and hope
>>> that the recipient randomly chooses the same key; the odds of that
>>> happening are 1 in 256!.
>>
>> It's easy to come up with attacks which are no better than brute force.
>> It's the attacks which are better than brute force that you have to watch
>> out for.
>>
> 
> And that's why we're having this discussion.  Do you know of an attack
> in which you can control the output (say at least 100 consecutive bytes)
> for data which goes through a 256 byte translation table, chosen
> randomly from 256! permutations after the data is sent.  If you do, I'm
> all ears!  But at this point you're just setting up straw men and
> knocking them down.

Ah yes, the good ol' "any argument I do not have an answer for must be a
straw man" gambit.

Whatever you say dude. Obviously you've got this security thing down to a
fine art. After all, if you can't think of a way to break the system,
nobody else can either.



-- 
Steven

[toc] | [prev] | [next] | [standalone]

#93218

On Fri, Jun 26, 2015 at 8:38 PM, Steven D'Aprano <steve@pearwood.info> wrote:
> Now you say that the application encrypts the data, except that the user can
> turn that option off.
>
> Just make the AES encryption mandatory, not optional. Then the user cannot
> upload unencrypted malicious data, and the receiver cannot read the data.
> That's two problems solved.

No, because another application could pretend to be the file-sending
application, but send unencrypted data instead of encrypted data.

-- Devin

[toc] | [prev] | [next] | [standalone]

#93224

On Sat, 27 Jun 2015 02:05 pm, Devin Jeanpierre wrote:

> On Fri, Jun 26, 2015 at 8:38 PM, Steven D'Aprano <steve@pearwood.info>
> wrote:
>> Now you say that the application encrypts the data, except that the user
>> can turn that option off.
>>
>> Just make the AES encryption mandatory, not optional. Then the user
>> cannot upload unencrypted malicious data, and the receiver cannot read
>> the data. That's two problems solved.
> 
> No, because another application could pretend to be the file-sending
> application, but send unencrypted data instead of encrypted data.

Did you stop reading my post when you got to that? Because I went on to say:

"Actually, the more I think about this, the more I come to think that the
only way this can be secure is for both the sending client application and
the receiving client appl to both encrypt the data. The sender can't
trust the receiver not to read the files, so the sender has to encrypt; the
receiver can't trust the sender not to send malicious files, so the
receiver has to encrypt too."




-- 
Steven

[toc] | [prev] | [next] | [standalone]

#93262

On Fri, Jun 26, 2015 at 11:16 PM, Steven D'Aprano <steve@pearwood.info> wrote:
> On Sat, 27 Jun 2015 02:05 pm, Devin Jeanpierre wrote:
>
>> On Fri, Jun 26, 2015 at 8:38 PM, Steven D'Aprano <steve@pearwood.info>
>> wrote:
>>> Now you say that the application encrypts the data, except that the user
>>> can turn that option off.
>>>
>>> Just make the AES encryption mandatory, not optional. Then the user
>>> cannot upload unencrypted malicious data, and the receiver cannot read
>>> the data. That's two problems solved.
>>
>> No, because another application could pretend to be the file-sending
>> application, but send unencrypted data instead of encrypted data.
>
> Did you stop reading my post when you got to that? Because I went on to say:

At that point I quit in frustration, yeah.

> "Actually, the more I think about this, the more I come to think that the
> only way this can be secure is for both the sending client application and
> the receiving client appl to both encrypt the data. The sender can't
> trust the receiver not to read the files, so the sender has to encrypt; the
> receiver can't trust the sender not to send malicious files, so the
> receiver has to encrypt too."

When you realize you've said something completely wrong, you should
edit your email.

-- Devin

[toc] | [prev] | [next] | [standalone]

#93264

On Sun, 28 Jun 2015 06:30 am, Devin Jeanpierre wrote:

> On Fri, Jun 26, 2015 at 11:16 PM, Steven D'Aprano <steve@pearwood.info>
> wrote:
>> On Sat, 27 Jun 2015 02:05 pm, Devin Jeanpierre wrote:
>>
>>> On Fri, Jun 26, 2015 at 8:38 PM, Steven D'Aprano <steve@pearwood.info>
>>> wrote:
>>>> Now you say that the application encrypts the data, except that the
>>>> user can turn that option off.
>>>>
>>>> Just make the AES encryption mandatory, not optional. Then the user
>>>> cannot upload unencrypted malicious data, and the receiver cannot read
>>>> the data. That's two problems solved.
>>>
>>> No, because another application could pretend to be the file-sending
>>> application, but send unencrypted data instead of encrypted data.
>>
>> Did you stop reading my post when you got to that? Because I went on to
>> say:
> 
> At that point I quit in frustration, yeah.
> 
>> "Actually, the more I think about this, the more I come to think that the
>> only way this can be secure is for both the sending client application
>> and the receiving client appl to both encrypt the data. The sender can't
>> trust the receiver not to read the files, so the sender has to encrypt;
>> the receiver can't trust the sender not to send malicious files, so the
>> receiver has to encrypt too."
> 
> When you realize you've said something completely wrong, you should
> edit your email.

If both the sender and receiver encrypt the data, how is is "completely
wrong" to say that encrypting data should be mandatory?




-- 
Steven

[toc] | [prev] | [next] | [standalone]

#93265

On Sat, Jun 27, 2015 at 6:18 PM, Steven D'Aprano <steve@pearwood.info> wrote:
> On Sun, 28 Jun 2015 06:30 am, Devin Jeanpierre wrote:
>
>> On Fri, Jun 26, 2015 at 11:16 PM, Steven D'Aprano <steve@pearwood.info>
>> wrote:
>>> On Sat, 27 Jun 2015 02:05 pm, Devin Jeanpierre wrote:
>>>
>>>> On Fri, Jun 26, 2015 at 8:38 PM, Steven D'Aprano <steve@pearwood.info>
>>>> wrote:
>>>>> Now you say that the application encrypts the data, except that the
>>>>> user can turn that option off.
>>>>>
>>>>> Just make the AES encryption mandatory, not optional. Then the user
>>>>> cannot upload unencrypted malicious data, and the receiver cannot read
>>>>> the data. That's two problems solved.
>>>>
>>>> No, because another application could pretend to be the file-sending
>>>> application, but send unencrypted data instead of encrypted data.
>>>
>>> Did you stop reading my post when you got to that? Because I went on to
>>> say:
>>
>> At that point I quit in frustration, yeah.
>>
>>> "Actually, the more I think about this, the more I come to think that the
>>> only way this can be secure is for both the sending client application
>>> and the receiving client appl to both encrypt the data. The sender can't
>>> trust the receiver not to read the files, so the sender has to encrypt;
>>> the receiver can't trust the sender not to send malicious files, so the
>>> receiver has to encrypt too."
>>
>> When you realize you've said something completely wrong, you should
>> edit your email.
>
> If both the sender and receiver encrypt the data, how is is "completely
> wrong" to say that encrypting data should be mandatory?

That isn't what I was calling completely wrong. This is:

>>>>> Just make the AES encryption mandatory, not optional. Then the user
>>>>> cannot upload unencrypted malicious data, and the receiver cannot read
>>>>> the data. That's two problems solved.

The user can still upload unencrypted malicious data by writing their
own client that doesn't have mandatory AES encryption.

You realized this later in the email, apparently, which is why you
should have edited your own email to delete your original, insecure,
suggestion. :(

That said, I appreciate the work you've done here asking for a
specific threat model and pushing back on the idea that it's up to
python-list to prove something is insecure, not the other way around.
That's important. I think, for the same reasons, it's also important
to be really careful what cryptosystems we discuss, and not suggest or
appear to suggest ones that won't work.


P.S. FWIW, the base64 idea has a lot of promise and is probably
fundamentally better than a crypto algorithm. With something along the
lines of base64 -- say, encoding a file using just the letters 'a' and
'b' -- one might try to make it it literally impossible to write "bad"
things to disk, whereas with any crypto, it is always possible to
obtain the key, so one has to be careful with key management to
prevent/mitigate that.  (One might add: why not both? Beats me. I like
using extension modules.)

P.P.S.: of course, I'm not an expert.

-- Devin

[toc] | [prev] | [next] | [standalone]

#93222

On Fri, Jun 26, 2015 at 9:38 PM, Steven D'Aprano <steve@pearwood.info> wrote:
> With respect Randall, you contradict yourself. Is there any wonder that some
> of us (well, me at least) is suspicious and confused, when your story
> changes as often as the weather?
>
> Sometimes you say that the client software uses AES encryption. Sometimes
> you say that you don't want to use AES encryption because you want the
> client to be pure Python, and a pure-Python implementation would be too
> slow. Your very first post says:
>
>     My original idea was for the recipient to encrypt using AES.  But
>     I want to keep this software pure Python "batteries included" and
>     not require installation of other platform-dependent software.
>     Pure Python AES and even DES are just way too slow.

In the context of the initial post, this was referring to the data
mangling done by the receiver; it has no bearing on the form of the
data sent by the application.

> Sometimes you say the user is supposed to encrypt the data themselves:
>
>     While the data senders are supposed to encrypt data, that's not
>     guaranteed

Whereas this clearly describes the behavior of the application itself.

> Now you say that the application encrypts the data, except that the user can
> turn that option off.
>
> Just make the AES encryption mandatory, not optional. Then the user cannot
> upload unencrypted malicious data, and the receiver cannot read the data.
> That's two problems solved.

And what if somebody else writes a competing version of the client
software that doesn't bother with the encryption step at all? The
point was that while encryption is expected, it cannot be assumed by
the receiver, and in fact if the data is actually malicious, then it
likely is not even being sent by the client software in the first
place.

> If the app does encrypt the data with AES before sending, then you don't
> gain any benefit by obfuscating an encrypted file with a classical
> monoalphabetic substitution cipher.

Only if the recipient can *trust* the sender to have performed the
encryption, which it can't, no matter how mandatory the OP tries to
make it.

> Suppose that you hire an intern to write the "choose key" function, and not
> knowing any better, he simply iterates through the keys in numeric order,
> one after the other. So the first upload will use key 0, the second key 1,
> the third key 2, and so on, until key 256! - 1, then start again. In that
> case, predicting the next key is *trivial*. If I can work out what key you
> send now (I just upload a file containing "\x00\x01\x02...\xFF" to myself
> and see what I get), then I know what key the app will use next.

If you upload a file to yourself, the result that you get will have no
bearing on what key might be chosen when you upload a file to somebody
else.

> Even if I can't do that, I might be able to guess the seed: I know what time
> the application started up, to within a few milliseconds,

How?

> and I know (or
> can guess) how many random numbers you have used,

How?

> Except... you're getting your random numbers from a system *I* control.

No you don't. If you did already control the target system, then as
already suggested, you have no need to attack the data upload; you can
just write whatever data you want to disk. This is like suggesting
that the sudoers file is insecure because a user with root access
would be able to add themselves to it.

>> If the attacker controlled the machine the app was on, why would it fool
>> with /dev/urandom?  I think he'd just plant the files he wanted to plant
>> and be done.  This is non-nonsensical anyway.
>
> No, you don't understand the nature of the attack. In this scenario, the
> sender is the attacker. I want to upload malicious files to the receiver.
> You are trying to stop me, that's the whole point of "mangling or
> encrypting" the files. (Your words.) So I, the sender, prepare a file such
> that when you mangle it, the resulting mangled content is the malicious
> content I want.
>
> If you use a substitution cipher, I can do this if I can guess or force the
> key. If you use strong crypto, I can't.
>
> However, I can hack the application. The client sits on my computer, it's
> pure Python, even if it isn't I can still hack the application, I don't
> need access to the source code.

If the recipient system is using the system random to generate the
key, then you can hack the application all you want, and it will give
you precisely zero information about the state of the entropy pool on
the remote system.

> Yes. Do you think that's hard for an attacker who has access to your
> application, possibly including the source code, and controls all the
> sources of entropy on the system your application is running on?
>
> I don't have to *randomly* guess. I control what time your application
> starts, I control what randomness you get from /dev/urandom, I control how
> many keys you go through, I might even be able to read the source code of
> the application (not that I need to, that just makes it easier).

I think what's going on here is that you've missed the point that the
obfuscation is done by the recipient, not by the sender. The sender
has no control over any of those things, and no access to either the
key or the obfuscated data unless they can gain it through some other
attack vector (which is certainly a factor to consider before
implementing this).

[toc] | [prev] | [next] | [standalone]

#93230

On Sat, 27 Jun 2015 03:47 pm, Ian Kelly wrote:

[...]
>> Just make the AES encryption mandatory, not optional. Then the user
>> cannot upload unencrypted malicious data, and the receiver cannot read
>> the data. That's two problems solved.
> 
> And what if somebody else writes a competing version of the client
> software that doesn't bother with the encryption step at all? The
> point was that while encryption is expected, it cannot be assumed by
> the receiver, and in fact if the data is actually malicious, then it
> likely is not even being sent by the client software in the first
> place.

Right. As I said later in my post, you have a situation where neither party
can trust the other. I'm trying to store data on your computer, and I can't
trust you not to snoop on it, and you can't trust me not to send you
malware.


>> If the app does encrypt the data with AES before sending, then you don't
>> gain any benefit by obfuscating an encrypted file with a classical
>> monoalphabetic substitution cipher.
> 
> Only if the recipient can *trust* the sender to have performed the
> encryption, which it can't, no matter how mandatory the OP tries to
> make it.

True. But in either case, a classical (i.e. insecure) cipher doesn't do the
job.


>> Suppose that you hire an intern to write the "choose key" function, and
>> not knowing any better, he simply iterates through the keys in numeric
>> order, one after the other. So the first upload will use key 0, the
>> second key 1, the third key 2, and so on, until key 256! - 1, then start
>> again. In that case, predicting the next key is *trivial*. If I can work
>> out what key you send now (I just upload a file containing
>> "\x00\x01\x02...\xFF" to myself and see what I get), then I know what key
>> the app will use next.
> 
> If you upload a file to yourself, the result that you get will have no
> bearing on what key might be chosen when you upload a file to somebody
> else.

I admit it: I was getting a bit confused between attacks on the sender side
and attacks on the receiver side. The attacks I describe depend on the
sender's application doing encryption, but given that a malicious uploader
can just write their own client, that's redundant. There are easier attacks
on sender-side encryption. A back-and-forth argument on Usenet is no
substitute between a careful security analysis.

Can the sender attack encryption on the client side? Well, Chris has already
demonstrated one actual attack, based on a two-byte malicious payload. That
proves that the concept is at least possible, even if nobody uses DOS any
more.

As you go on to say:

> If the recipient system is using the system random to generate the
> key, then you can hack the application all you want, and it will give
> you precisely zero information about the state of the entropy pool on
> the remote system.

You're right. Are there other attacks where I, the sender, can get the
recipient to leak information about the key from the receiver?

Would you like to bet the answer is always No? I wouldn't.

Can you say "timing attack"?

http://codahale.com/a-lesson-in-timing-attacks/

Can you [generic you] believe that attackers can *reliably* attack remote
systems based on a 20µs timing differences? If you say "No", then you fail
Security 101 and should step away from the computer until a security expert
can be called in to review your code.

I'm not a security expert. I'm not even a talented amateur. *Every time* I
suggest that "X is secure", the security guy at work shoots me down in
flames. But nicely, because I pay his wages <wink>

If I say "such-and-such an attack is impossible", feel free to scoff and
laugh, because I'm probably wrong. But if I say "I don't know what it is,
but there's probably an attack you haven't thought of", unless you're a
security guy yourself, you probably ought to listen. (And if you are a
security guy, then you know how hard it is to secure against unknown
attacks.)

Tens of millions of zombie computers in botnets are proof that there are
exploitable attacks that programmers didn't think of. Or rather, *some*
programmers didn't think of them. Some other guys did.

I've said it before, and I will say it again: a classical substitution
cipher is trivially vulnerable to a preimage attack, strong crypto ciphers
are not. You're betting everything on the key being secret. If the keys
leaks, or is predictable, the attacker can successfully write malware on
the receiver's system. If the keys leak with AES, the system is still
secure against a preimage attack.


"Nobody will be able to guess the key, we don't need strong crypto."

"The Titanic is unsinkable, we don't need lifeboats for everyone."




-- 
Steven

[toc] | [prev] | [next] | [standalone]

#93231

On Sat, Jun 27, 2015 at 6:38 PM, Steven D'Aprano <steve@pearwood.info> wrote:
> I'm not a security expert. I'm not even a talented amateur. *Every time* I
> suggest that "X is secure", the security guy at work shoots me down in
> flames. But nicely, because I pay his wages <wink>

Just out of interest, is _anybody_ active in this thread an expert on
security? I certainly am not, which means that the proposal I'm
currently putting together probably has a whole bunch of
vulnerabilities that I haven't thought of. (Though there's no emphasis
on encryption anywhere, just signing. I'm *hoping* that RSA public key
verification is sufficient, but if it isn't, it would be possible for
a malicious user to make a serious mess of stuff.) But I'm under no
delusions. I don't say "this is secure" - all I'm saying is "this
works in proof-of-concept".

ChrisA

[toc] | [prev] | [next] | [standalone]

#93232

On 27.06.2015 10:53, Chris Angelico wrote:
> On Sat, Jun 27, 2015 at 6:38 PM, Steven D'Aprano <steve@pearwood.info> wrote:
>> I'm not a security expert. I'm not even a talented amateur. *Every time* I
>> suggest that "X is secure", the security guy at work shoots me down in
>> flames. But nicely, because I pay his wages <wink>
> 
> Just out of interest, is _anybody_ active in this thread an expert on
> security?

Yes. I've done a good 10 years of work in the field doing security
(mostly applied cryptography on embedded systems with a focus on side
channels like DPA, but also security concepts and threat/risk analysis)
and spent the last 3-4 years working on my PhD in the field of IT
security. My thesis is almost(tm) finished. I would claim to be an
expert, yes.

> I certainly am not, which means that the proposal I'm
> currently putting together probably has a whole bunch of
> vulnerabilities that I haven't thought of. (Though there's no emphasis
> on encryption anywhere, just signing. I'm *hoping* that RSA public key
> verification is sufficient, but if it isn't, it would be possible for
> a malicious user to make a serious mess of stuff.) But I'm under no
> delusions. I don't say "this is secure" - all I'm saying is "this
> works in proof-of-concept".

I must admit that I haven't seen your ideas in this thread?

Best regards,
Johannes

-- 
>> Wo hattest Du das Beben nochmal GENAU vorhergesagt?
> Zumindest nicht öffentlich!
Ah, der neueste und bis heute genialste Streich unsere großen
Kosmologen: Die Geheim-Vorhersage.
 - Karl Kaos über Rüdiger Thomas in dsa <hidbv3$om2$1@speranza.aioe.org>

[toc] | [prev] | [next] | [standalone]

#93234

On Sat, Jun 27, 2015 at 7:07 PM, Johannes Bauer <dfnsonfsduifb@gmx.de> wrote:
> On 27.06.2015 10:53, Chris Angelico wrote:
>> On Sat, Jun 27, 2015 at 6:38 PM, Steven D'Aprano <steve@pearwood.info> wrote:
>>> I'm not a security expert. I'm not even a talented amateur. *Every time* I
>>> suggest that "X is secure", the security guy at work shoots me down in
>>> flames. But nicely, because I pay his wages <wink>
>>
>> Just out of interest, is _anybody_ active in this thread an expert on
>> security?
>
> Yes. I've done a good 10 years of work in the field doing security
> (mostly applied cryptography on embedded systems with a focus on side
> channels like DPA, but also security concepts and threat/risk analysis)
> and spent the last 3-4 years working on my PhD in the field of IT
> security. My thesis is almost(tm) finished. I would claim to be an
> expert, yes.

Good, so this isn't like that episode of Yes Minister when they were
trying to figure out whether to allow a chemical factory to be built.

>> I certainly am not, which means that the proposal I'm
>> currently putting together probably has a whole bunch of
>> vulnerabilities that I haven't thought of. (Though there's no emphasis
>> on encryption anywhere, just signing. I'm *hoping* that RSA public key
>> verification is sufficient, but if it isn't, it would be possible for
>> a malicious user to make a serious mess of stuff.) But I'm under no
>> delusions. I don't say "this is secure" - all I'm saying is "this
>> works in proof-of-concept".
>
> I must admit that I haven't seen your ideas in this thread?

No, the proposal I'm putting together is unrelated. You'll see the
*vast* extent of my security skills here:

https://github.com/Rosuav/ThirdSquare

My contribution to this thread has been fairly minor, just suggesting
one attack that doesn't even work any more, not much else.

ChrisA

[toc] | [prev] | [next] | [standalone]

Page 2 of 5 — ← Prev page 1 [2] 3 4 5  Next page →

Back to top | Article view | comp.lang.python

csiph-web