Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.user > #164502 > unrolled thread

Warning �~@~T Linux Mint Website Hacked and ISOs replaced with Backdoored Operating System)

Back to article view | Back to linux.debian.user

Contents

   Warning �~@~T Linux Mint Website Hacked and ISOs replaced with Backdoored Operating System) Karen Lewellen <klewellen@shellworld.net> - 2016-02-22 19:10 +0100
    Re: Warning Linux Mint Website Hacked and ISOs replaced with Backdoored Operating System "Thomas Schmitt" <scdbackup@gmx.net> - 2016-02-22 20:30 +0100
      Re: Warning Linux Mint Website Hacked and ISOs replaced with Backdoored  Operating System Karen Lewellen <klewellen@shellworld.net> - 2016-02-22 20:40 +0100
        Re: Warning Linux Mint Website Hacked and ISOs replaced with Backdoored Operating System "Thomas Schmitt" <scdbackup@gmx.net> - 2016-02-22 21:00 +0100
          Re: Warning Linux Mint Website Hacked and ISOs replaced with  Backdoored Operating System Charlie Kravetz <cjk@teamcharliesangels.com> - 2016-02-22 22:00 +0100
          Re: Warning Linux Mint Website Hacked and ISOs replaced with  Backdoored Operating System Stuart Longland <stuartl@longlandclan.id.au> - 2016-02-25 03:40 +0100
      Re: Warning Linux Mint Website Hacked and ISOs replaced with  Backdoored Operating System Dalios <dalios@eumx.net> - 2016-02-22 23:00 +0100
        Re: Warning Linux Mint Website Hacked and ISOs replaced with Backdoored Operating System "Thomas Schmitt" <scdbackup@gmx.net> - 2016-02-22 23:20 +0100
          Re: Warning Linux Mint Website Hacked and ISOs replaced with Backdoored Operating System Sven Hartge <sven@svenhartge.de> - 2016-02-22 23:40 +0100
            Re: Warning Linux Mint Website Hacked and ISOs replaced with Backdoored Operating System "Thomas Schmitt" <scdbackup@gmx.net> - 2016-02-23 00:00 +0100
              Re: Warning Linux Mint Website Hacked and ISOs replaced with  Backdoored Operating System Andrew McGlashan <andrew.mcglashan@affinityvision.com.au> - 2016-02-24 10:30 +0100
                Re: Warning Linux Mint Website Hacked and ISOs replaced with Backdoored Operating System "Thomas Schmitt" <scdbackup@gmx.net> - 2016-02-24 13:50 +0100
                  Re: Warning Linux Mint Website Hacked and ISOs replaced with  Backdoored Operating System <tomas@tuxteam.de> - 2016-02-24 14:10 +0100
                  Re: Warning Linux Mint Website Hacked and ISOs replaced with  Backdoored Operating System Andrew McGlashan <andrew.mcglashan@affinityvision.com.au> - 2016-02-24 14:20 +0100
                    OT: ownership of Mega.co.nz (was Re: Warning Linux Mint Website  Hacked and ISOs replaced with Backdoored Operating System) Jonathan Dowland <jmtd@debian.org> - 2016-02-24 15:00 +0100
      Re: Warning Linux Mint Website Hacked and ISOs replaced with  Backdoored Operating System Gener Badenas <gener.ong.badenas@gmail.com> - 2016-02-23 02:40 +0100
        Re: Warning Linux Mint Website Hacked and ISOs replaced with Backdoored Operating System Glenn English <ghe@srv.slsware.net> - 2016-02-23 05:20 +0100
        Re: Warning Linux Mint Website Hacked and ISOs replaced with Backdoored Operating System "Thomas Schmitt" <scdbackup@gmx.net> - 2016-02-23 08:10 +0100
        Re: Warning Linux Mint Website Hacked and ISOs replaced with  Backdoored Operating System Darac Marjal <mailinglist@darac.org.uk> - 2016-02-23 11:00 +0100
          Re: Warning Linux Mint Website Hacked and ISOs replaced with  Backdoored Operating System Nicolas George <george@nsup.org> - 2016-02-23 11:30 +0100
            Re: Warning Linux Mint Website Hacked and ISOs replaced with Backdoored Operating System "Thomas Schmitt" <scdbackup@gmx.net> - 2016-02-23 12:10 +0100
              Re: Warning Linux Mint Website Hacked and ISOs replaced with  Backdoored Operating System Nicolas George <george@nsup.org> - 2016-02-23 12:10 +0100
                Re: Warning Linux Mint Website Hacked and ISOs replaced with Backdoored Operating System "Thomas Schmitt" <scdbackup@gmx.net> - 2016-02-23 13:10 +0100
                  Re: Warning Linux Mint Website Hacked and ISOs replaced with  Backdoored Operating System Nicolas George <george@nsup.org> - 2016-02-23 13:20 +0100
                    Re: Warning Linux Mint Website Hacked and ISOs replaced with  Backdoored Operating System David Wright <deblis@lionunicorn.co.uk> - 2016-02-23 16:00 +0100
                      Re: Warning Linux Mint Website Hacked and ISOs replaced with  Backdoored Operating System Nicolas George <george@nsup.org> - 2016-02-23 16:10 +0100
                        Re: Warning Linux Mint Website Hacked and ISOs replaced with  Backdoored Operating System David Wright <deblis@lionunicorn.co.uk> - 2016-02-23 17:00 +0100
                          Re: Warning Linux Mint Website Hacked and ISOs replaced with  Backdoored Operating System Nicolas George <george@nsup.org> - 2016-02-23 17:00 +0100
                            Re: Warning Linux Mint Website Hacked and ISOs replaced with  Backdoored Operating System arian <debian@semioptimal.net> - 2016-02-23 17:50 +0100
                              Re: Warning Linux Mint Website Hacked and ISOs replaced with  Backdoored Operating System arian <debian@semioptimal.net> - 2016-02-23 17:50 +0100
                              Re: Warning Linux Mint Website Hacked and ISOs replaced with  Backdoored Operating System Nicolas George <george@nsup.org> - 2016-02-23 18:00 +0100
                                Re: Warning Linux Mint Website Hacked and ISOs replaced with  Backdoored Operating System arian <debian@semioptimal.net> - 2016-02-23 18:10 +0100
                            Re: Warning Linux Mint Website Hacked and ISOs replaced with  Backdoored Operating System David Wright <deblis@lionunicorn.co.uk> - 2016-02-23 21:00 +0100
                              Re: Warning Linux Mint Website Hacked and ISOs replaced with  Backdoored Operating System Nicolas George <george@nsup.org> - 2016-02-24 11:00 +0100
                                Re: Warning Linux Mint Website Hacked and ISOs replaced with  Backdoored Operating System David Wright <deblis@lionunicorn.co.uk> - 2016-02-27 06:30 +0100
                          Re: Warning Linux Mint Website Hacked and ISOs replaced with  Backdoored Operating System Henrique de Moraes Holschuh <hmh@debian.org> - 2016-02-25 15:00 +0100
                            Re: Warning Linux Mint Website Hacked and ISOs replaced with Backdoored Operating System Stefan Monnier <monnier@iro.umontreal.ca> - 2016-02-25 15:10 +0100
                              Re: Warning Linux Mint Website Hacked and ISOs replaced with Backdoored Operating System Stefan Monnier <monnier@iro.umontreal.ca> - 2016-02-25 15:20 +0100
                              Re: Warning Linux Mint Website Hacked and ISOs replaced with  Backdoored Operating System Christian Seiler <christian@iwakd.de> - 2016-02-25 16:20 +0100
                            Re: Warning Linux Mint Website Hacked and ISOs replaced with Backdoored Operating System "Thomas Schmitt" <scdbackup@gmx.net> - 2016-02-25 21:10 +0100
                      Re: Warning Linux Mint Website Hacked and ISOs replaced with Backdoored Operating System "Thomas Schmitt" <scdbackup@gmx.net> - 2016-02-23 16:40 +0100
                        Re: Warning Linux Mint Website Hacked and ISOs replaced with  Backdoored Operating System Nicolas George <george@nsup.org> - 2016-02-23 17:00 +0100
                          Re: Warning Linux Mint Website Hacked and ISOs replaced with Backdoored Operating System "Thomas Schmitt" <scdbackup@gmx.net> - 2016-02-23 17:50 +0100
                            Re: Warning Linux Mint Website Hacked and ISOs replaced with  Backdoored Operating System Nicolas George <george@nsup.org> - 2016-02-23 18:40 +0100
                          Re: Warning Linux Mint Website Hacked and ISOs replaced with  Backdoored Operating System Christian Seiler <christian@iwakd.de> - 2016-02-23 19:20 +0100
                            Re: Warning Linux Mint Website Hacked and ISOs replaced with  Backdoored Operating System Nicolas George <george@nsup.org> - 2016-02-23 20:00 +0100
                              Re: Warning Linux Mint Website Hacked and ISOs replaced with  Backdoored Operating System Christian Seiler <christian@iwakd.de> - 2016-02-23 20:40 +0100
                                Re: Warning Linux Mint Website Hacked and ISOs replaced with  Backdoored Operating System Nicolas George <george@nsup.org> - 2016-02-24 11:00 +0100
                                  Re: Warning Linux Mint Website Hacked and ISOs replaced with  Backdoored Operating System Christian Seiler <christian@iwakd.de> - 2016-02-24 12:40 +0100
                                    Re: Warning Linux Mint Website Hacked and ISOs replaced with  Backdoored Operating System Nicolas George <george@nsup.org> - 2016-02-24 13:50 +0100
                                      Re: Warning Linux Mint Website Hacked and ISOs replaced with  Backdoored Operating System Christian Seiler <christian@iwakd.de> - 2016-02-24 15:30 +0100
                              Re: Warning Linux Mint Website Hacked and ISOs replaced with  Backdoored Operating System Richard Hector <richard@walnut.gen.nz> - 2016-02-24 10:50 +0100
                                Re: Warning Linux Mint Website Hacked and ISOs replaced with  Backdoored Operating System Nicolas George <george@nsup.org> - 2016-02-24 11:00 +0100
                                  Re: Warning Linux Mint Website Hacked and ISOs replaced with  Backdoored Operating System Richard Hector <richard@walnut.gen.nz> - 2016-02-24 11:10 +0100
                Re: Warning Linux Mint Website Hacked and ISOs replaced with  Backdoored Operating System Seeker <seeker5528@comcast.net> - 2016-02-23 17:30 +0100
                  Re: Warning Linux Mint Website Hacked and ISOs replaced with  Backdoored Operating System Nicolas George <george@nsup.org> - 2016-02-23 17:40 +0100
              Re: Warning Linux Mint Website Hacked and ISOs replaced with  Backdoored Operating System jdd <jdd@dodin.org> - 2016-02-23 12:10 +0100
              Re: Warning Linux Mint Website Hacked and ISOs replaced with Backdoored Operating System "Thomas Schmitt" <scdbackup@gmx.net> - 2016-02-23 12:40 +0100
                Re: Warning Linux Mint Website Hacked and ISOs replaced with  Backdoored Operating System Nicolas George <george@nsup.org> - 2016-02-23 13:30 +0100
              Re: Warning Linux Mint Website Hacked and ISOs replaced with  Backdoored Operating System "Andrew M.A. Cater" <amacater@galactic.demon.co.uk> - 2016-02-23 13:00 +0100
              Re: Warning Linux Mint Website Hacked and ISOs replaced with  Backdoored Operating System Andrew McGlashan <andrew.mcglashan@affinityvision.com.au> - 2016-02-24 10:40 +0100
          Re: Warning Linux Mint Website Hacked and ISOs replaced with Backdoored Operating System "Thomas Schmitt" <scdbackup@gmx.net> - 2016-02-23 11:50 +0100

Page 2 of 4 — ← Prev page 1 [2] 3 4  Next page →

#164540 — Re: Warning Linux Mint Website Hacked and ISOs replaced with Backdoored Operating System

Hi,

Nicolas George wrote:
> Of course, that does not mean MD5 and SHA-1 should be used nowadays. New
> theoretical attacks are found, keeping using hashes with known weaknesses is
> stupid.

The ISO checksums are provided more for transport verification than
for the fight against intentional mainpulation.
Signing the hash lists by PGP still seems a bit weak as protection.

But well, if Debian armors its ISOs, then it would have to scrutinize
the source of its packages, too.

Most important seems a permanent supervision of the web site content
from not publicly known client machines. Hash sums may be manipulated.
But the whole content of an ISO is either original or not. Easy to spot.


Have a nice day :)

Thomas

[toc] | [prev] | [next] | [standalone]

#164541 — Re: Warning Linux Mint Website Hacked and ISOs replaced with Backdoored Operating System

[Multipart message — attachments visible in raw view] — view raw

Le quintidi 5 ventôse, an CCXXIV, Thomas Schmitt a écrit :
> The ISO checksums are provided more for transport verification than
> for the fight against intentional mainpulation.

If that were true, CRC32 would be enough.

> Signing the hash lists by PGP still seems a bit weak as protection.

Signing hashes will get you a spanking from any cryptographer.
Cryptographic signatures must be applied on the file itself; it works
internally by signing a hash of the file, but the hash is done in a way that
prevents most attacks even with weak hashes.

Regards,

-- 
  Nicolas George

[toc] | [prev] | [next] | [standalone]

#164545 — Re: Warning Linux Mint Website Hacked and ISOs replaced with Backdoored Operating System

Hi,

i wrote:
> > The ISO checksums are provided more for transport verification than
> > for the fight against intentional mainpulation.

Nicolas George wrote:
> If that were true, CRC32 would be enough.

For detecting most glitches, yes.
But not if we want to use it for identifying files in benevolent
environments.

The birthday paradox predicts that CRC32 will produce the first
collision after square root of 32 bit, i.e. 16 bit = 65536 tries.

MD5 gets square rooted to 64 bit, which gives a collision roughly
the same probability as that a 10 km asteroid hits you (not me)
directly within a day. Assumed that such a thing happens every
100 million years.

(And well, who would respect a person that divides polynomials on
a Galois field.)


> Signing hashes will get you a spanking from any cryptographer.

I sign my release tarballs by a gpg command which i really don't
understand, except that Karl Berry of FSF teached me to use it for
GNU uploads.
(How many bits worth is Karl's reputation and my illusion that it
was him who wrote to me ?)


I wonder whether there are performance reasons why only the hashs
of Debian ISOs are PGP signed.


Have a nice day :)

Thomas

[toc] | [prev] | [next] | [standalone]

#164546 — Re: Warning Linux Mint Website Hacked and ISOs replaced with Backdoored Operating System

[Multipart message — attachments visible in raw view] — view raw

Le quintidi 5 ventôse, an CCXXIV, Thomas Schmitt a écrit :
> i wrote:
> > > The ISO checksums are provided more for transport verification than
> > > for the fight against intentional mainpulation.

> Nicolas George wrote:
> > If that were true, CRC32 would be enough.

> For detecting most glitches, yes.
> But not if we want to use it for identifying files in benevolent
> environments.

You are changing the terms of the problem at each messages, it does not make
for a constructive discussion.

I am sure that you can come up with enough far-fetched conditions to build a
situation where MD5 would be the best choice, but this is not relevant for
real world situations.

For real world situations, to protect against hostile tampering MD5 is
broken, and to protect against accidental tampering, faster hashes exist.

[toc] | [prev] | [next] | [standalone]

#164548 — Re: Warning Linux Mint Website Hacked and ISOs replaced with Backdoored Operating System

On Tue 23 Feb 2016 at 13:15:38 (+0100), Nicolas George wrote:
> Le quintidi 5 ventôse, an CCXXIV, Thomas Schmitt a écrit :
> > i wrote:
> > > > The ISO checksums are provided more for transport verification than
> > > > for the fight against intentional mainpulation.
> 
> > Nicolas George wrote:
> > > If that were true, CRC32 would be enough.
> 
> > For detecting most glitches, yes.
> > But not if we want to use it for identifying files in benevolent
> > environments.
> 
> You are changing the terms of the problem at each messages, it does not make
> for a constructive discussion.

But it's raised an issue that interests me...

> I am sure that you can come up with enough far-fetched conditions to build a
> situation where MD5 would be the best choice, but this is not relevant for
> real world situations.
> 
> For real world situations, to protect against hostile tampering MD5 is
> broken, and to protect against accidental tampering, faster hashes exist.

Any faster ones that you recommend from the lists below? (I've rolled
my own implementation of fdupes (which uses MD5) in python.)

Python 3.4.2 (default, Oct  8 2014, 13:14:40) 
...
>>> hashlib.algorithms_guaranteed
{'md5', 'sha1', 'sha224', 'sha512', 'sha384', 'sha256'}
>>> hashlib.algorithms_available
{'MD4', 'md5', 'md4', 'sha1', 'MD5', 'dsaWithSHA', 'whirlpool', 'sha',
'SHA512', 'SHA256', 'ripemd160', 'sha512', 'SHA384', 'sha384',
'dsaEncryption', 'RIPEMD160', 'sha256', 'SHA224', 'SHA1',
'ecdsa-with-SHA1', 'DSA', 'SHA', 'sha224', 'DSA-SHA'}
>>> 

Cheers,
David.

[toc] | [prev] | [next] | [standalone]

#164550 — Re: Warning Linux Mint Website Hacked and ISOs replaced with Backdoored Operating System

[Multipart message — attachments visible in raw view] — view raw

Le quintidi 5 ventôse, an CCXXIV, David Wright a écrit :
> Any faster ones that you recommend from the lists below? (I've rolled
> my own implementation of fdupes (which uses MD5) in python.)

Nobody can recommend anything without knowing the intended use.

Regards,

-- 
  Nicolas George

[toc] | [prev] | [next] | [standalone]

#164554 — Re: Warning Linux Mint Website Hacked and ISOs replaced with Backdoored Operating System

On Tue 23 Feb 2016 at 16:04:37 (+0100), Nicolas George wrote:
> Le quintidi 5 ventôse, an CCXXIV, David Wright a écrit :
> > Any faster ones that you recommend from the lists below? (I've rolled
> > my own implementation of fdupes (which uses MD5) in python.)
> 
> Nobody can recommend anything without knowing the intended use.

I don't understand. The intended use is in the previous line:

"my own implementation of fdupes (which uses MD5) in python"

and also in Thomas's statement:

"we want to use it for identifying files in benevolent environments"

(which you snipped).

So to summarise:

1) I do what fdupes does, ie identify files (in a benevolent
   environment) using the MD5 signature to detect duplicate
   contents.
2) In view of your statement that faster hashes exist, I would
   like to explore replacing my use of MD5 by such a hash.
3) My python implementation has the following hashes in its own
   library, either "available on this platform" or "guaranteed
   on all python platforms". Using one of these makes altering
   my program easier than having to find out how to call an
   external hashing program (and the calls might slow things
   back down again).
4) As you're far more familiar with hashing than I am (and
   many people here), would you have any recommendations from
   these two lists?

Python 3.4.2 (default, Oct  8 2014, 13:14:40)
...
>>> hashlib.algorithms_guaranteed
{'md5', 'sha1', 'sha224', 'sha512', 'sha384', 'sha256'}
>>> hashlib.algorithms_available
{'MD4', 'md5', 'md4', 'sha1', 'MD5', 'dsaWithSHA', 'whirlpool', 'sha',
'SHA512', 'SHA256', 'ripemd160', 'sha512', 'SHA384', 'sha384',
'dsaEncryption', 'RIPEMD160', 'sha256', 'SHA224', 'SHA1',
'ecdsa-with-SHA1', 'DSA', 'SHA', 'sha224', 'DSA-SHA'}
>>> 

I hope that explains things better than my previous attempt.

Cheers,
David.

[toc] | [prev] | [next] | [standalone]

#164555 — Re: Warning Linux Mint Website Hacked and ISOs replaced with Backdoored Operating System

[Multipart message — attachments visible in raw view] — view raw

Le quintidi 5 ventôse, an CCXXIV, David Wright a écrit :
> 1) I do what fdupes does, ie identify files (in a benevolent
>    environment) using the MD5 signature to detect duplicate
>    contents.

You did not specify the average size of files nor how sure you want to be.

If the files are large, I would suggest to use a sparse hash function, i.e.
a hash function that only reads small parts of the file, and do a full
comparison or compute a strong hash only for files that have a collision on
that.

> >>> hashlib.algorithms_guaranteed
> {'md5', 'sha1', 'sha224', 'sha512', 'sha384', 'sha256'}
> >>> hashlib.algorithms_available
> {'MD4', 'md5', 'md4', 'sha1', 'MD5', 'dsaWithSHA', 'whirlpool', 'sha',
> 'SHA512', 'SHA256', 'ripemd160', 'sha512', 'SHA384', 'sha384',
> 'dsaEncryption', 'RIPEMD160', 'sha256', 'SHA224', 'SHA1',
> 'ecdsa-with-SHA1', 'DSA', 'SHA', 'sha224', 'DSA-SHA'}

These are all cryptographic hash functions: too strong for a preliminary
test, insufficient for absolute certainty.

Still, you can easily benchmark.

[toc] | [prev] | [next] | [standalone]

#164558 — Re: Warning Linux Mint Website Hacked and ISOs replaced with Backdoored Operating System

[Multipart message — attachments visible in raw view] — view raw

> These are all cryptographic hash functions: too strong for a preliminary
> test, insufficient for absolute certainty.

where do you get that these are "insufficient for absolute certainty"? (beside maybe md4)
there are no known collisions in sha1 and better, and even md4's preimage attack has complexity 2^102. [1,2]

OTOH harddrive manufacturers state MTBF of around 500'000 h, and that's just a fraction of the error sources you have. HDDs read at around 150 MB/s, so in those 500'000h they read:
5e5 h*3600 s/h*150e6 B/s *8 b/B = 2.16e18 = 2^55
- so from that you have 2^47 higher probability of random errors over the worst algorithm's failure. That's probably around the number of bits of your personal storage. Stated differently: until hash functions flip one bit of your storage wrong, storage itself has eaten it all.

of course, those estimates are off the real (unknown) numbers, you probably have unknown attacks on any hash, which may be far better than 2^102. But how many of 1960s harddrives and computer around it are still running and have produced only one wrong bit in their lifetime? MTBFs are far off too.

[toc] | [prev] | [next] | [standalone]

#164559 — Re: Warning Linux Mint Website Hacked and ISOs replaced with Backdoored Operating System

[Multipart message — attachments visible in raw view] — view raw

> and even md4's preimage attack has complexity 2^102. [1,2]

sorry, forgot the quotes:
[1] https://en.wikipedia.org/wiki/Preimage_attack
[2] https://en.wikipedia.org/wiki/MD4#Security

[toc] | [prev] | [next] | [standalone]

#164561 — Re: Warning Linux Mint Website Hacked and ISOs replaced with Backdoored Operating System

[Multipart message — attachments visible in raw view] — view raw

Le quintidi 5 ventôse, an CCXXIV, arian a écrit :
> where do you get that these are "insufficient for absolute certainty"?
> (beside maybe md4)
> there are no known collisions in sha1 and better, and even md4's preimage
> attack has complexity 2^102. [1,2]

There are collisions for SHA1 as soon as the files are larger than 20 bytes.
The same applies to SHA-2 with the corresponding size.

The collisions are not known, and very unlikely, but "absolute" means
absolute, not "very likely".

Regards,

-- 
  Nicolas George

[toc] | [prev] | [next] | [standalone]

#164563 — Re: Warning Linux Mint Website Hacked and ISOs replaced with Backdoored Operating System

[Multipart message — attachments visible in raw view] — view raw

> The collisions are not known, and very unlikely, but "absolute" means
> absolute, not "very likely".

from the way you stated:
> These are all cryptographic hash functions: too strong for a preliminary
> test, insufficient for absolute certainty.

I understood you suggest there is a relevant level of uncertainty, one that will bite someone and should therefore be avoided (for the purpose of comparing files from sources that they trust to hold their data for that matter)

[toc] | [prev] | [next] | [standalone]

#164576 — Re: Warning Linux Mint Website Hacked and ISOs replaced with Backdoored Operating System

On Tue 23 Feb 2016 at 16:58:38 (+0100), Nicolas George wrote:
> Le quintidi 5 ventôse, an CCXXIV, David Wright a écrit :
> > 1) I do what fdupes does, ie identify files (in a benevolent
> >    environment) using the MD5 signature to detect duplicate
> >    contents.
> 
> You did not specify the average size of files nor how sure you want to be.

Just the usual mix of normal user files. Nothing specialised here.

> If the files are large, I would suggest to use a sparse hash function, i.e.
> a hash function that only reads small parts of the file, and do a full
> comparison or compute a strong hash only for files that have a collision on
> that.

After eliminating uniquely-sized files, I do check the first chunk of
files with identical lengths before I hash them. I think the following
methods are the only ones easily available.

> > >>> hashlib.algorithms_guaranteed
> > {'md5', 'sha1', 'sha224', 'sha512', 'sha384', 'sha256'}
> > >>> hashlib.algorithms_available
> > {'MD4', 'md5', 'md4', 'sha1', 'MD5', 'dsaWithSHA', 'whirlpool', 'sha',
> > 'SHA512', 'SHA256', 'ripemd160', 'sha512', 'SHA384', 'sha384',
> > 'dsaEncryption', 'RIPEMD160', 'sha256', 'SHA224', 'SHA1',
> > 'ecdsa-with-SHA1', 'DSA', 'SHA', 'sha224', 'DSA-SHA'}
> 
> These are all cryptographic hash functions: too strong for a preliminary
> test, insufficient for absolute certainty.

Good enough for me (apart from the specially-crafted pair of letters
from Julius Caesar).

> Still, you can easily benchmark.

Well, md5 beats md4 and sha1, so I guess I'll stick with that for the
time being.

Cheers,
David.

[toc] | [prev] | [next] | [standalone]

#164602 — Re: Warning Linux Mint Website Hacked and ISOs replaced with Backdoored Operating System

[Multipart message — attachments visible in raw view] — view raw

Le quintidi 5 ventôse, an CCXXIV, David Wright a écrit :
> Well, md5 beats md4

There is something wrong in your library.

Regards,

-- 
  Nicolas George

[toc] | [prev] | [next] | [standalone]

#164733 — Re: Warning Linux Mint Website Hacked and ISOs replaced with Backdoored Operating System

On Wed 24 Feb 2016 at 10:58:56 (+0100), Nicolas George wrote:
> Le quintidi 5 ventôse, an CCXXIV, David Wright a écrit :
> > Well, md5 beats md4
> 
> There is something wrong in your library.

Thanks for your misplaced confidence in me. It was my timing
that wasn't rigorous enough.

Cheers,
David.

[toc] | [prev] | [next] | [standalone]

#164655 — Re: Warning Linux Mint Website Hacked and ISOs replaced with Backdoored Operating System

On Tue, 23 Feb 2016, David Wright wrote:
> 1) I do what fdupes does, ie identify files (in a benevolent
>    environment) using the MD5 signature to detect duplicate
>    contents.

MD5 alone can be somewhat dangerous even in benevolent environments: if the
data sets are large enough or you are just unlucky, you are going to hit a
colision and corrupt-or-lose-data-on-dedup sooner or later.

At least use data-size + hash.  But even that won't save you for
colisions... the "full fix" is to use the hash (or size + hash) as a screen
to detect possible matches: when it matches, compare the two data-sets to
ensure they're really equal before you trigger the dedup.

I am not going to bother with the detail that you need to ensure one of the
data sets can't/didn't change under you between the comparison and the dedup
getting commited to storage.

> 2) In view of your statement that faster hashes exist, I would
>    like to explore replacing my use of MD5 by such a hash.

Any wide-enough hash will do if you use it just for screening, where you
don't care for for any security properties of the hash.  And at that point,
you might as well use a wide-enough CRC (ensure it is vectorizable and get
the compiler to vectorize it!) if it proves to be faster than crypto
hashes...

-- 
  "One disk to rule them all, One disk to find them. One disk to bring
  them all and in the darkness grind them. In the Land of Redmond
  where the shadows lie." -- The Silicon Valley Tarot
  Henrique Holschuh

[toc] | [prev] | [next] | [standalone]

#164656 — Re: Warning Linux Mint Website Hacked and ISOs replaced with Backdoored Operating System

> MD5 alone can be somewhat dangerous even in benevolent environments: if the
> data sets are large enough or you are just unlucky, you are going to hit a
> colision and corrupt-or-lose-data-on-dedup sooner or later.

it doesn't seem worried about this.  Admittedly, they use sha1 rather
than md5, so they have 160bit instead of 128bit, with a correspondingly
lower probability of collisions, but I'd be interested to know about
cases where md5 lead to accidental collisions.


        Stefan

[toc] | [prev] | [next] | [standalone]

#164657 — Re: Warning Linux Mint Website Hacked and ISOs replaced with Backdoored Operating System

>> MD5 alone can be somewhat dangerous even in benevolent environments: if the
>> data sets are large enough or you are just unlucky, you are going to hit a
>> colision and corrupt-or-lose-data-on-dedup sooner or later.
> it doesn't seem worried about this.  Admittedly, they use sha1 rather
 ^
 G


        Stefan "Damn concise names!"

[toc] | [prev] | [next] | [standalone]

#164660 — Re: Warning Linux Mint Website Hacked and ISOs replaced with Backdoored Operating System

[Multipart message — attachments visible in raw view] — view raw

On 02/25/2016 03:07 PM, Stefan Monnier wrote:
>> MD5 alone can be somewhat dangerous even in benevolent environments: if the
>> data sets are large enough or you are just unlucky, you are going to hit a
>> colision and corrupt-or-lose-data-on-dedup sooner or later.
> 
> [G]it doesn't seem worried about this.  Admittedly, they use sha1 rather
> than md5, so they have 160bit instead of 128bit, with a correspondingly
> lower probability of collisions, but I'd be interested to know about
> cases where md5 lead to accidental collisions.

Well, I wouldn't necessarily use that as a benchmark: git could have used
SHA2-256 from the start - it's not like SHA2 is something brand new, it
was already 4 years old when git was developed.

I haven't heard of any _accidental_ collision of either MD5 or SHA1 so
far, but I might be mistaken. (There are of course famous intentional
collisions in MD5, see <http://www.mscs.dal.ca/~selinger/md5collision/>.)

From a mathematical standpoint: if we assume that the values a hash may
produce are uniformly distributed and cover the entire range of possible
outputs, due to the birthday paradox accidental collisions occur every
2^(bitsize/2) inputs; for MD5 that would be 2^64, for SHA1 that would be
2^80.

Whether you can live with that is up to you.

Regards,
Christian

[toc] | [prev] | [next] | [standalone]

#164675 — Re: Warning Linux Mint Website Hacked and ISOs replaced with Backdoored Operating System

Hi,

Henrique de Moraes Holschuh wrote:
> MD5 alone can be somewhat dangerous even in benevolent environments: if the
> data sets are large enough or you are just unlucky,

The size of the data set does not matter much.
As already stated, there is the Pidgeon Hole Principle, which tells
us that a 128 bit hashsum cannot be the mapping result of all inputs
of 129 bit without having 1 exp 128 collisions. Roughly each MD5
will appear twice among the messages of 129 bit length.

Since MD5 is computed bytewise, you will get about 256 times
each possible MD5 value from the set of all 17 byte inputs.


Nevertheless the size of 128 is considered sufficient for UUID.
  https://en.wikipedia.org/wiki/Universally_unique_identifier
says:

  A UUID is simply a 128-bit value. [...]
  The intent of UUIDs is to enable distributed systems to uniquely
  identify information without significant central coordination.
  In this context the word unique should be taken to mean
  "practically unique" rather than "guaranteed unique".

then comes some math
  https://en.wikipedia.org/wiki/Universally_unique_identifier#Random_UUID_probability_of_duplicates

  "only after generating 1 billion UUIDs every second for the next 100 years,
   the probability of creating just one duplicate would be about 50%."

The only "practical" objection against MD5 in this context would be
that we first have to prove its quality of uniformly distributing the
results over the space of 2 exp 128 possible values.
  https://en.wikipedia.org/wiki/Hash_function#Uniformity

I am not aware that MD5 was accused of not having enough of this property.
This also yields the "practical" randomness which UUID presumes.
I understand the table in
  http://michiel.buddingh.eu/distribution-of-hash-values
that MD5 has denser and sparser regions, but still is quite well uniform.


As already stated too, the Birthday Paradox hits when you create a large
collection of MD5 and add more. You have to expect the first collision
after about 2 exp 64 values. (That is about the probability to get stomped
into the earth by a 10 km asteroid until tomorrow evening. So take this
as benchmark whether you should be worried about MD5 collisions.)

I also have read a good argument in this thread about the lifetime
of contemporary hardware. It gives you few chances to process 2 exp 64
data sets of 16 bytes (256 billion gigabytes).


Have a nice day :)

Thomas

[toc] | [prev] | [next] | [standalone]

Page 2 of 4 — ← Prev page 1 [2] 3 4  Next page →

Back to top | Article view | linux.debian.user

csiph-web