Path: csiph.com!usenet.pasdenom.info!aioe.org!.POSTED!not-for-mail
From: qwertmonkey@syberianoutpost.ru
Newsgroups: comp.lang.java.programmer
Subject: greatly differing processing time between java and Linux while calculating hashes?
Date: Sun, 9 Sep 2012 14:35:26 +0000 (UTC)
Organization: Aioe.org NNTP Server
Lines: 146
Message-ID: <k2i9fd$vo0$1@speranza.aioe.org>
NNTP-Posting-Host: z9Rq+Ge+SLJPEkk9TGcnaw.user.speranza.aioe.org
X-Complaints-To: abuse@aioe.org
X-Notice: Filtered by postfilter v. 0.8.2
X-Newsreader: NetComponents
Xref: csiph.com comp.lang.java.programmer:18627

~ 
> you want to make sure the size of the byte / char array given to the digest
calculator is big so that you have the overhead of the call as infrequently
as possible.
~ 
 is an increase from 512 (2^9) to 65536 (2^16) enough?
~ 
 I didn't see any differences whatsoever in the processing times
~ 
> You might want to do a trace to be sure you are changing that, not just some
other buffering. 
~ 
 Well, I did trace the loaded classes using the java options:
~ 
 -verbose:jni -verbose:class
~ 
 and using jstack but I couldn't see anything explaining that much of a 
difference. From jstack I got:
~ 
 sun.jvm.hotspot.debugger.{UnmappedAddressException, UnalignedAddressException}
~ 
 which I couldn't quickly troubleshoot
~ 
 I still think those issues may relate in weird (not straight/explicit) ways in
which the jvm may be chosing checksumming algorithms
~ 
 lbrtchx
 comp.lang.java.programmer: 
greatly differing processing time between java and Linux while calculating
hashes?
~ 
~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ 
~ 
$ java -verbose:jni -verbose:class CheckSum00Test SHA-512 DQfUaXLk_sw.mp4 > 
`date +%Y%m%d%H%M%S`"_SHA-512.log"

$ java -verbose:jni -verbose:class CheckSum00Test MD5 DQfUaXLk_sw.mp4 > 
`date +%Y%m%d%H%M%S`"_MD5.log"

$ java -verbose:jni -verbose:class CheckSum00Test MD5 DQfUaXLk_sw.mp4 > 
`date +%Y%m%d%H%M%S`"_MD5.log"

$ java -verbose:jni -verbose:class CheckSum00Test SHA-512 DQfUaXLk_sw.mp4 > 
`date +%Y%m%d%H%M%S`"_SHA-512.log"

$ diff 20120909100112_MD5.log 20120909100215_MD5.log > diff_MD5.log

$ wc -l *_MD5.log
   852 20120909100112_MD5.log
   852 20120909100215_MD5.log
     4 diff_MD5.log
  1708 total

$ cat diff_MD5.log
850c850
< // __ MD5 encrypting  "DQfUaXLk_sw.mp4":"ecbcbe3d0a2fda4ebfddc41836cee528"
  took: 5921 (ms)
---
> // __ MD5 encrypting  "DQfUaXLk_sw.mp4":"ecbcbe3d0a2fda4ebfddc41836cee528"
  took: 5830 (ms)

$ diff 20120909095915_SHA-512.log 20120909100225_SHA-512.log > SHA-512.log

$ wc -l *SHA-512.log
   853 20120909095915_SHA-512.log
   853 20120909100225_SHA-512.log
     4 SHA-512.log
  1710 total

$ cat SHA-512.log
851c851
< // __ SHA-512 encrypting  "DQfUaXLk_sw.mp4":"f6f2f2b88faf3cef5a5de0cb0bc8f1ed
15969490ef785673d3f7e325ab947c8df8b30edf6636657e945b27eb5202300244b8acef874a11e
630b2053f956982d8"  took: 46852 (ms)
---
> // __ SHA-512 encrypting  "DQfUaXLk_sw.mp4":"f6f2f2b88faf3cef5a5de0cb0bc8f1ed
15969490ef785673d3f7e325ab947c8df8b30edf6636657e945b27eb5202300244b8acef874a11e
630b2053f956982d8"  took: 45030 (ms)

$ diff 20120909100225_SHA-512.log 20120909100112_MD5.log > SHA-512_MD5.log

$ wc -l SHA-512_MD5.log
9 SHA-512_MD5.log

$ cat SHA-512_MD5.log
848,849c848
< [Loaded sun.security.provider.SHA5 from /media/sdb1/inst/sw/jdk1.7.0_07/jre/
lib/rt.jar]
< [Loaded sun.security.provider.SHA5$SHA512 from /media/sdb1/inst/sw/
jdk1.7.0_07/jre/lib/rt.jar]
---
> [Loaded sun.security.provider.MD5 from /media/sdb1/inst/sw/jdk1.7.0_07/jre/
lib/rt.jar]
851c850
< // __ SHA-512 encrypting  "DQfUaXLk_sw.mp4":"f6f2f2b88faf3cef5a5de0cb0bc8f1ed
15969490ef785673d3f7e325ab947c8df8b30edf6636657e945b27eb5202300244b8acef874a11e
630b2053f956982d8"  took: 45030 (ms)
---
> // __ MD5 encrypting  "DQfUaXLk_sw.mp4":"ecbcbe3d0a2fda4ebfddc41836cee528"
  took: 5921 (ms)
$ 
~ 
~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ 
~ 
#!/bin/bash

echo "// __ java -version:"
java -version

_KDP="java CheckSum00Test"

_DT=`date +%Y%m%d%H%M%S`

_PID=`ps aux | grep "${_KDP}" | grep -v grep | awk '{print $2}'`

echo "// __ process: "
ps aux | grep "${_KDP}" | grep -v grep

echo "// __ process id: "${_PID}

_OFL=${_DT}"_jstack_-m_-l_"${_PID}".log"

echo "// __ java -version:" >> ${_OFL} 2>&1
java -version >> ${_OFL} 2>&1
echo "~" >> ${_OFL} 2>&1


echo "// __ readlink -f $(which jmap):" >> ${_OFL} 2>&1
readlink -f $(which jmap) >> ${_OFL} 2>&1
echo "~" >> ${_OFL} 2>&1


echo "// __ process: " >> ${_OFL} 2>&1
ps aux | grep "${_KDP}" | grep -v grep >> ${_OFL} 2>&1
echo "~" >> ${_OFL} 2>&1


echo "// __ process id: "${_PID} >> ${_OFL} 2>&1
echo "~" >> ${_OFL} 2>&1


echo "// __ jstack -m -l ${_PID} >> ${_OFL} 2>&1" >> ${_OFL} 2>&1
echo "~" >> ${_OFL} 2>&1

jstack -m -l ${_PID} >> ${_OFL} 2>&1
~