Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.java.programmer > #38396

Re: Why is Java 7 converting the Password toCharArray instead of Byte in it's PBKDF2 function?

Cross-posted to 2 groups.

Show all headers | View raw

On 19.06.18 23:15, Robert Klemme wrote:
> On 19.06.2018 12:54, Karl.Frank wrote:
>> On 18.06.18 18:41, Robert Klemme wrote:
>>> On 15.05.2018 23:37, Karl.Frank wrote:
>>>> Holy moley! It's a buggy Java Library I'm currently using!!
>>>
>>> You are converting a byte array to a String without explicit mention of
>>> the encoding. Maybe the way Java picks up the default encoding has
>>> changed between versions. Anyway, this is an unsafe practice.
>>>
>> Robert, probably that might be the case.
>
> It is, it is.
>
>> In regards of cryptography I always tend to stay on the byte level. This
>> way my own implementation of HMAC and PBKDF2 has no need to bother with
>> string encoding when reading in from stdin or files. Curiously Java's
>> PBKDF2 expect a string or char array rather than a byte array and
>> therefore I need the extra conversion. Actually from now on my own
>> functions will be used instead.
>
> The point is not your usage of byte array nor your need for a
> conversion. The point is that your conversion is unsafe because the
> default encoding is used implicitly instead of making your choice of
> encoding explicit. There are various factors that might influence the
> JVM's choice of default encoding and they might not be the same as
> govern the encoding of the user input (what you send via echo). This is
> a source of errors and the bad thing is it might actually work in one
> environment - but not in others. Such bugs can go unnoticed for a long
> time before they hit you. And then you're left with wrongly encoded data
> which can be very painful.
>
Whatever comes in from stdin has to be handled and converted by the
piping program as the reading program normally does expect byte - so is
my understanding and experience in the past with C, Python and other
languages.

In fact my test program get the byte properly from stdin because they
are converted by the piping program - in case of the example by "echo".

Due to the fact that the PBKDF2 function of Java expect a string which
has to be converted to a char array for the password (but a byte array
for the salt) I need to convert the byte array first. Unclear for me
however why the mixup of char array and byte array is necessary here.
But to be honest, I'm not really keen to dig deeper because my own
implementations are working properly.


   public static String byteArrToString(byte[] byteArr) {
     String bs = new String(byteArr);

     return bs;
   }

   Password = byteArrToString(pwByte);

And here it comes:

certainly I do *not* know in advance what kind of string conversion to 
choose in the byteArrToString() function simply because I do not know 
what kind of data the piping program might pass over.

For that reason I'll *always* read in byte and never ever anything else, 
because one thing I can be sure, the piped data will be byte. So never 
ever read in data as strings or what ever else crude format they might be.


>> It might be a different matter if the user type in password and/or salt
>> in some GUI. In this case however I would define the kind of strings
>> accepted and later convert them into byte arrays before processing them
>> further.
>
> Graphical inputs usually emit String, char[] or CharSequence so you can
> be sure the input is properly encoded (modulo some issues) and you can
> pick whatever encoding you want for the transformation into bytes.
>
That's what I prefer. If I'd had to read in user typed data I define the 
encoding in the GUI forms input field.


>>> Also, you are reading only once, there is the chance that there is still
>>> data in the pipe which you will miss. Even worse, depending on how data
>>> is written to stdin you might even get different results. This might not
>>> be an issue for your test program, but could be if you use this code in
>>> production.
>>>
>> Well, yes I read only once from stdin and store the result in a byte
>> array. How do you suppose to read twice?
>>
>> The amount of byte to read is exactly defined by
>>
>>   int BuffSize = System.in.available();
>>   byte[] pwByte = new byte[BuffSize];
>>   System.in.read(pwByte, 0, BuffSize);
>>
>> How would the program read more byte than necessary?
>
> Please read carefully: I wrote it might read _less_ bytes. You should
> really read the JavaDocs of InputStream#available(). For example it is
> perfectly legal for an implementation to always return 0 yet still yield
> data via read(). Basically, available() is quite useless most of the
> time if you want to read the complete input anyway.
>
Right, missed that. Explained that in another answer in this thread
already. In case of the test program it seems fairly sufficient to use
System.in.available().


>> As data on stdin is always byte and all my programs read in byte and
>> never ever strings in my opinion it doesn't matter how these byte are
>> written and piped by any previous instance or program.
>
> Again, the point is a different one: your reading does not ensure you
> get the complete input. For this you need a loop that reads until EOF.
> Using just a single read() is not sufficient as an implementation is
> free to choose how many bytes to return for a single read().
>
> There is no way you can get away without a loop.
>
> final byte[] buffer = new byte[1024];
> final ByteArrayOutputStream out = new ByteArrayOutputStream();
>
> for (int len; (len = in.read(buffer)) != -1;) {
> out.write(buffer, 0, len);
> }
>
> byte[] data = out.toByteArray();
>
> There are other more elegant variants depending on which libs you use
> and on which Java version you are.
>
To be honest again, all these different versions and libs coming with
Java are a real pain in the ass. And exactly these incompatibilities are
the reason why I try to rely as much as possible only on the bare
minimum of Java's inherent commands and implement critical functions on
my own in pure Java.


>>>> So my own implementation works correctly - what a relieve!
>>>
>>> There are cases where wrong code emits correct results - and this
>>> surfaces only later, when the input or some other environmental factor
>>> changes.
>>>
>> That's one reason why I tend to run large and extensive tests before my
>> program go into a production environment. Until today I've never
>> experienced any real problem. Even in this case I'd implement my own
>> HMAC and PBKDF2 and therefore found the "buggy" Java lib. Sadly the
>> developer of the original program some years ago didn't check that and
>> now everyone is baffled that we can't decrypt old files with the
>> correctly working PBKDF2 implementation.
>
> I am not convinced you have identified a "'buggy' Java lib".
>
Well, I am.

> Kind regards
>
> robert
>

Cheers,
Karl


-- 
cHNiMUBACG0HAAAAAAAAAAAAAABIZVbDdKVM0w1kM9vxQHw+bkLxsY/Z0czY0uv8/Ks6WULxJVua
zjvpoYvtEwDVhP7RGTCBVlzZ+VBWPHg5rqmKWvtzsuVmMSDxAIS6Db6YhtzT+RStzoG9ForBcG8k
G97Q3Jml/aBun8Kyf+XOBHpl5gNW4YqhiM0=

Back to comp.lang.java.programmer | Previous | Next — Previous in thread | Find similar | Unroll thread

Thread

Re: Why is Java 7 converting the Password toCharArray instead of Byte in it's PBKDF2 function? "Karl.Frank" <Karl.Frank@Freecx.co.uk> - 2018-05-15 23:37 +0200
  Re: Why is Java 7 converting the Password toCharArray instead of Byte in it's PBKDF2 function? "Karl.Frank" <Karl.Frank@Freecx.co.uk> - 2018-05-16 15:58 +0200
  Re: Why is Java 7 converting the Password toCharArray instead of Byte in it's PBKDF2 function? Robert Klemme <shortcutter@googlemail.com> - 2018-06-18 18:41 +0200
    Re: Why is Java 7 converting the Password toCharArray instead of Byte in it's PBKDF2 function? "Chris M. Thomasson" <invalid_chris_thomasson@invalid.invalid> - 2018-06-18 13:33 -0700
    Re: Why is Java 7 converting the Password toCharArray instead of Byte in it's PBKDF2 function? "Karl.Frank" <Karl.Frank@Freecx.co.uk> - 2018-06-19 12:54 +0200
      Re: Why is Java 7 converting the Password toCharArray instead of Byte in it's PBKDF2 function? Lothar Kimmeringer <news201705@kimmeringer.de> - 2018-06-19 22:18 +0200
        Re: Why is Java 7 converting the Password toCharArray instead of Byte in it's PBKDF2 function? Robert Klemme <shortcutter@googlemail.com> - 2018-06-19 23:20 +0200
        Re: Why is Java 7 converting the Password toCharArray instead of Byte in it's PBKDF2 function? "Karl.Frank" <Karl.Frank@Freecx.co.uk> - 2018-06-19 23:23 +0200
      Re: Why is Java 7 converting the Password toCharArray instead of Byte in it's PBKDF2 function? Robert Klemme <shortcutter@googlemail.com> - 2018-06-19 23:15 +0200
        Re: Why is Java 7 converting the Password toCharArray instead of Byte in it's PBKDF2 function? "Karl.Frank" <Karl.Frank@Freecx.co.uk> - 2018-06-20 00:03 +0200

csiph-web