Path: csiph.com!x330-a1.tempe.blueboxinc.net!newsfeed.hal-mli.net!feeder1.hal-mli.net!weretis.net!feeder4.news.weretis.net!news-transit.tcx.org.uk!aioe.org!.POSTED!not-for-mail From: Abu Yahya Newsgroups: comp.lang.java.programmer,comp.programming,comp.lang.java.databases Subject: Storing large strings for future equality checks Date: Wed, 08 Jun 2011 22:05:30 +0530 Organization: Aioe.org NNTP Server Lines: 19 Message-ID: NNTP-Posting-Host: LePVoNEtezBuiMA9+cM5gA.user.speranza.aioe.org Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Complaints-To: abuse@aioe.org User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.17) Gecko/20110414 Thunderbird/3.1.10 X-Notice: Filtered by postfilter v. 0.8.2 Xref: x330-a1.tempe.blueboxinc.net comp.lang.java.programmer:5113 comp.programming:436 comp.lang.java.databases:458 A small application that I'm making requires me to store very long strings (>1000 characters) in a database. I will need to use these strings later to compare for equality to incoming strings from another application. I will also want to add some of the incoming strings to the storage, if they meet certain criteria. For my application, I get a feeling that storing these strings in my table will be a waste of space, and will impact performance due to retrieval and storage times, as well as comparison times. I considered using an SHA-512 hash of these strings and storing them in the database. However, while these will save on storage space, it will take time to do the hashing before comparing an incoming string. So I'm still wasting time. (Collisions due to hashing will not be a problem, since an occasional false positive will not be fatal for my application). What would be the best approach?