Path: csiph.com!2.us.feeder.erje.net!feeder.erje.net!2.eu.feeder.erje.net!border1.nntp.ams1.giganews.com!nntp.giganews.com!takemy.news.telefonica.de!storethat.news.telefonica.de!telefonica.de!news-2.dfn.de!news.dfn.de!news.informatik.hu-berlin.de!fu-berlin.de!uni-berlin.de!not-for-mail
From: Mok-Kong Shen <mok-kong.shen@t-online.de>
Newsgroups: comp.lang.python.announce
Subject: ANN: TEXTCOMBINE-SP
Date: Sun, 9 Jul 2017 12:33:12 +0200
Organization: albasani.net
Lines: 26
Approved: python-announce-list@python.org
Message-ID: <mailman.6.1499688607.29743.python-announce-list@python.org>
References: <ojt0p9$4hj$1@news.albasani.net>
Mime-Version: 1.0
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit
X-Trace: news.uni-berlin.de c/7TR4/Q7BGUrT2Cwtw17QHZHHSNQ6ePvjvLx3+QLGhw==
Cancel-Lock: sha1:Gf1f6mT96k54Ot2bHoo289PBoNk=
Return-Path: <news@four.schnuerpel.eu>
X-Original-To: python-announce@python.org
Delivered-To: python-announce-list@mail.python.org
X-Spam-Status: OK 0.012
X-Spam-Evidence: '*H*': 0.98; '*S*': 0.00; 'plenty': 0.05; 'bits': 0.07; 'practice,': 0.07; 'subject:ANN': 0.07; 'english,': 0.09; 'implies': 0.09; 'logic': 0.09; 'underlying': 0.09; 'files.': 0.15; '32)': 0.16; '[1].': 0.16; 'bits,': 0.16; 'received:eu': 0.16; 'seems': 0.21; 'simpler': 0.22; 'bit': 0.22; 'header:X -Complaints-To:1': 0.24; 'header:User-Agent:1': 0.26; 'e.g.': 0.27; 'values': 0.28; 'coded': 0.29; 'mode.': 0.29; 'thus': 0.29; '[1]': 0.33; 'could': 0.33; 'schemes': 0.33; 'text': 0.36; 'there': 0.36; 'subject:: ': 0.36; 'files': 0.36; 'that,': 0.36; 'method': 0.38; 'subject:-': 0.38; 'received:io': 0.39; 'received:psf.io': 0.39; 'received:ams1.psf.io': 0.40; 'received:mail1.ams1.psf.io': 0.40; 'per': 0.62; 'high': 0.64; 'our': 0.64; 'obtaining': 0.66; 'to:addr:comp-lang-python- announce': 0.66; 'letters': 0.70; 'sum': 0.73; 'quality': 0.73; 'to:addr:moderators.isc.org': 0.76; 'viable': 0.84
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.2.1
X-Mozilla-News-Host: news://news.albasani.net:119
X-Mailman-Approved-At: Mon, 10 Jul 2017 08:10:05 -0400
X-BeenThere: python-announce-list@python.org
X-Mailman-Version: 2.1.24
Precedence: list
List-Id: Announcement-only list for the Python programming language <python-announce-list.python.org>
List-Unsubscribe: <https://mail.python.org/mailman/options/python-announce-list>, <mailto:python-announce-list-request@python.org?subject=unsubscribe>
List-Archive: <http://mail.python.org/pipermail/python-announce-list/>
List-Post: <mailto:python-announce-list@python.org>
List-Help: <mailto:python-announce-list-request@python.org?subject=help>
List-Subscribe: <https://mail.python.org/mailman/listinfo/python-announce-list>, <mailto:python-announce-list-request@python.org?subject=subscribe>
X-Mailman-Original-Message-ID: <ojt0p9$4hj$1@news.albasani.net>
Xref: csiph.com comp.lang.python.announce:2562

An estimate of entropy of English texts is 1.34 bits per letter [1]. This
implies that, if the letters are coded into 5 bits, one needs to 
appropriately
combine 4 text files in order to obtain bit sequences of full entropy, since
4*1.34 = 5.36 > 5. The method used in our software is to sum (mod 32) 
the coded
values of a-z (mapped to 0-25) as 5 bits of the corresponding letters of 
the
text files.

There are plenty of other schemes for obtaining high quality pseudo-random
sequences in practice, e.g. AES in counter mode. However our scheme seems to
be much simpler both in the underlying logic (understandability) and in
implementation and is thus a viable alternative that one could use/need 
under
circumstances.

The software is available at mok-kong-shen.de

M. K. Shen
------------------------------------------------------------------------

[1] T. M. Cover, R. C. King, A Convergent Gambling Estimate of the 
Entropy of
English, IEEE Trans. Inf. Theory, vol. 24, 1978, pp. 413-421.