Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #85921

Re: python implementation of a new integer encoding algorithm.

Path csiph.com!usenet.pasdenom.info!gegeweb.org!usenet-fr.net!nerim.net!novso.com!newsfeed.xs4all.nl!newsfeed4.news.xs4all.nl!xs4all!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail
Return-Path <davea@davea.name>
X-Original-To python-list@python.org
Delivered-To python-list@mail.python.org
X-Spam-Status OK 0.047
X-Spam-Evidence '*H*': 0.91; '*S*': 0.00; 'encoding': 0.05; '"no"': 0.09; 'bits': 0.09; 'tcp/ip': 0.09; 'will,': 0.09; 'worse': 0.09; 'times,': 0.14; 'advice,': 0.16; 'assumptions': 0.16; 'blocking': 0.16; 'enough.': 0.16; 'integers,': 0.16; 'length,': 0.16; 'worse.': 0.16; 'subject:python': 0.16; 'elements': 0.16; 'wrote:': 0.18; 'else,': 0.19; 'seems': 0.21; 'feb': 0.22; 'header :User-Agent:1': 0.23; '"you': 0.24; 'byte': 0.24; 'bytes': 0.24; 'diverse': 0.24; "haven't": 0.24; '(or': 0.24; 'suggested': 0.26; 'values': 0.27; 'header:In-Reply-To:1': 0.27; 'chris': 0.29; 'am,': 0.29; 'primarily': 0.30; 'streaming': 0.30; 'that.': 0.31; 'subject:skip:i 10': 0.31; 'values.': 0.31; 'probably': 0.32; 'fri,': 0.33; 'raw': 0.33; 'could': 0.34; 'something': 0.35; 'but': 0.35; 'data,': 0.36; 'scheme': 0.36; 'subject:new': 0.38; 'to:addr:python-list': 0.38; 'pm,': 0.38; 'does': 0.39; 'extremely': 0.39; 'to:addr:python.org': 0.39; 'enough': 0.39; 'even': 0.60; 'read': 0.60; 'dave': 0.60; 'number,': 0.60; 'length': 0.61; 'numbers': 0.61; 'entire': 0.61; 'making': 0.63; 'such': 0.63; 'our': 0.64; 'more': 0.64; 'charset:windows-1252': 0.65; '20,': 0.68; 'reads': 0.68; 'received:74.208': 0.68; 'eight': 0.74; '2015': 0.84; 'angel': 0.91; 'that),': 0.91; 'hand,': 0.93; 'responses': 0.93; 'thereafter': 0.93
Date Thu, 19 Feb 2015 13:41:48 -0500
From Dave Angel <davea@davea.name>
User-Agent Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.4.0
MIME-Version 1.0
To python-list@python.org
Subject Re: python implementation of a new integer encoding algorithm.
References <e45a71b2-7ec0-4c7d-88ae-c48aebe154b7@googlegroups.com> <mailman.18784.1424182393.18130.python-list@python.org> <515047c1-a20d-430e-a029-1c2d77db2f1a@googlegroups.com> <mailman.18823.1424278025.18130.python-list@python.org> <2a717ffb-d61d-4407-9082-1c17cd7ee573@googlegroups.com> <mailman.18839.1424298000.18130.python-list@python.org> <993f64c3-fc85-48a7-9d02-a4f12ecb33c6@googlegroups.com> <54E62A68.4010603@davea.name> <CAPTjJmos-_yGWuco-u-vRgimcnQ=CUYVB9b9pW5-AYFtzrCM=Q@mail.gmail.com>
In-Reply-To <CAPTjJmos-_yGWuco-u-vRgimcnQ=CUYVB9b9pW5-AYFtzrCM=Q@mail.gmail.com>
Content-Type text/plain; charset=windows-1252; format=flowed
Content-Transfer-Encoding 7bit
X-Provags-ID V02:K0:ACjeHDTmet2lwSfwm9+VN/d8RMCmdL7tovM28xjcnX5 bcWsCg2wLN0pym6nvYDAs1pggcXwX66Xc+AwHQgZ7GSR2jdKBL gueRY2jT4+YotG7YE8E9w26pQbuxtFYMAkXLshghJkjdK3sxl7 sV+tva0nubAdBBrb+oQN+czdlaL/JvQZvGXOI7zlM0i7PmnWhB KVtpht5dGgWOg2YjnR3Lxjvz9nvc+TIyMvjJM8w37nEJ4tLXrS zHchMNurx5q5ENPnG5Pkd0oUSL90CIpbDjGJhdAgz8Z/xEEfmj /TqslmcSn+/OyM2j6Q/18qqUDzr+aGKIjWPfcAwadxZ+nZWDhB cxPX9dMiKa3Rco+gqQcE=
X-UI-Out-Filterresults notjunk:1;
X-BeenThere python-list@python.org
X-Mailman-Version 2.1.15
Precedence list
List-Id General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe <https://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive <http://mail.python.org/pipermail/python-list/>
List-Post <mailto:python-list@python.org>
List-Help <mailto:python-list-request@python.org?subject=help>
List-Subscribe <https://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
Newsgroups comp.lang.python
Message-ID <mailman.18897.1424371314.18130.python-list@python.org> (permalink)
Lines 39
NNTP-Posting-Host 2001:888:2000:d::a6
X-Trace 1424371314 news.xs4all.nl 2913 [2001:888:2000:d::a6]:57979
X-Complaints-To abuse@xs4all.nl
Xref csiph.com comp.lang.python:85921

Show key headers only | View raw

On 02/19/2015 01:34 PM, Chris Angelico wrote:
> On Fri, Feb 20, 2015 at 5:24 AM, Dave Angel <davea@davea.name> wrote:
>> In all my experimenting, I haven't found any values where the 7bit scheme
>> does worse.  It seems likely that for extremely large integers, it will, but
>> if those are to be the intended distribution, the 7bit scheme could be
>> replaced by something else, like just encoding a length at the beginning,
>> and using raw bytes after that.
>
> Encoding a length (as varlen) and then using eight bits to the byte
> thereafter is worse for small numbers,

I only suggested this if it turns out that the distribution is primarily 
extremely large numbers, large enough that 7bit isn't good enough.

As I (and others) have said many times, making it optimal means making 
some assumptions about the distribution of likely values.

> breaks even around 2**56, and
> then is better. So unless your numbers are mainly going to be above
> 2**56, it's better to just use varlen for the entire number. On the
> other hand, if you have to stream this without over-reading (imagine
> streaming from a TCP/IP socket; you want to block until you have the
> whole number, but not block after that), it may be more efficient to
> take the length, and then do a blocking read for the main data,
> instead of a large number of single-byte reads. But on the gripping
> hand, you can probably just do those one-byte reads and rely on (or
> implement) lower-level buffering.
>
> Ask not the python-list for advice, because they will say both "yes"
> and "no" and "maybe"... because they will say all three of "yes",
> "no", "maybe", and "you don't need to do that"... erm, AMONG our
> responses will be such diverse elements as...
>
> ChrisA
>


-- 
DaveA

Back to comp.lang.python | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

python implementation of a new integer encoding algorithm. janhein.vanderburg@gmail.com - 2015-02-17 03:22 -0800
  Re: python implementation of a new integer encoding algorithm. Chris Angelico <rosuav@gmail.com> - 2015-02-18 00:16 +1100
    Re: python implementation of a new integer encoding algorithm. janhein.vanderburg@gmail.com - 2015-02-18 00:55 -0800
      Re: python implementation of a new integer encoding algorithm. Chris Angelico <rosuav@gmail.com> - 2015-02-18 20:36 +1100
        Re: python implementation of a new integer encoding algorithm. janhein.vanderburg@gmail.com - 2015-02-18 11:29 -0800
      Re: python implementation of a new integer encoding algorithm. Laura Creighton <lac@openend.se> - 2015-02-18 11:32 +0100
        Re: python implementation of a new integer encoding algorithm. janhein.vanderburg@gmail.com - 2015-02-18 11:48 -0800
      People hated it for the same reasons I found them cool (was: python implementation of a new integer encoding algorithm.) Ben Finney <ben+python@benfinney.id.au> - 2015-02-18 21:57 +1100
  Re: python implementation of a new integer encoding algorithm. Dave Angel <davea@davea.name> - 2015-02-17 09:12 -0500
    Re: python implementation of a new integer encoding algorithm. janhein.vanderburg@gmail.com - 2015-02-18 00:59 -0800
      Re: python implementation of a new integer encoding algorithm. Dave Angel <davea@davea.name> - 2015-02-18 11:46 -0500
        Re: python implementation of a new integer encoding algorithm. Grant Edwards <invalid@invalid.invalid> - 2015-02-18 17:30 +0000
          Re: python implementation of a new integer encoding algorithm. Mark Lawrence <breamoreboy@yahoo.co.uk> - 2015-02-18 18:12 +0000
        Re: python implementation of a new integer encoding algorithm. janhein.vanderburg@gmail.com - 2015-02-18 11:55 -0800
          Re: python implementation of a new integer encoding algorithm. Marko Rauhamaa <marko@pacujo.net> - 2015-02-18 23:54 +0200
            Re: python implementation of a new integer encoding algorithm. Marko Rauhamaa <marko@pacujo.net> - 2015-02-19 00:08 +0200
            Re: python implementation of a new integer encoding algorithm. Grant Edwards <invalid@invalid.invalid> - 2015-02-18 22:58 +0000
          Re: python implementation of a new integer encoding algorithm. Dave Angel <davea@davea.name> - 2015-02-18 17:19 -0500
            Re: python implementation of a new integer encoding algorithm. janhein.vanderburg@gmail.com - 2015-02-19 07:45 -0800
              Re: python implementation of a new integer encoding algorithm. Ian Kelly <ian.g.kelly@gmail.com> - 2015-02-19 11:04 -0700
              Re: python implementation of a new integer encoding algorithm. Ian Kelly <ian.g.kelly@gmail.com> - 2015-02-19 11:16 -0700
              Re: python implementation of a new integer encoding algorithm. Dave Angel <davea@davea.name> - 2015-02-19 13:24 -0500
              Re: python implementation of a new integer encoding algorithm. Chris Angelico <rosuav@gmail.com> - 2015-02-20 05:34 +1100
              Re: python implementation of a new integer encoding algorithm. Ian Kelly <ian.g.kelly@gmail.com> - 2015-02-19 11:32 -0700
              Re: python implementation of a new integer encoding algorithm. Dave Angel <davea@davea.name> - 2015-02-19 13:41 -0500
              Re: python implementation of a new integer encoding algorithm. Dave Angel <davea@davea.name> - 2015-02-19 13:46 -0500
              Re: python implementation of a new integer encoding algorithm. Chris Angelico <rosuav@gmail.com> - 2015-02-20 05:49 +1100
      Re: python implementation of a new integer encoding algorithm. Mark Lawrence <breamoreboy@yahoo.co.uk> - 2015-02-18 17:00 +0000
  Re: python implementation of a new integer encoding algorithm. Chris Angelico <rosuav@gmail.com> - 2015-02-18 01:34 +1100
    Re: python implementation of a new integer encoding algorithm. janhein.vanderburg@gmail.com - 2015-02-18 01:04 -0800
      Re: python implementation of a new integer encoding algorithm. Dave Angel <davea@davea.name> - 2015-02-18 08:54 -0500
        Re: python implementation of a new integer encoding algorithm. janhein.vanderburg@gmail.com - 2015-02-18 11:52 -0800
      Re: python implementation of a new integer encoding algorithm. Chris Angelico <rosuav@gmail.com> - 2015-02-19 01:16 +1100
  Re: python implementation of a new integer encoding algorithm. Dave Angel <davea@davea.name> - 2015-02-17 09:50 -0500
  Re: python implementation of a new integer encoding algorithm. Chris Angelico <rosuav@gmail.com> - 2015-02-18 01:58 +1100
  Re: python implementation of a new integer encoding algorithm. Dave Angel <davea@davea.name> - 2015-02-17 10:18 -0500
  Re: python implementation of a new integer encoding algorithm. Chris Angelico <rosuav@gmail.com> - 2015-02-18 02:25 +1100
  Re: python implementation of a new integer encoding algorithm. Paul Rubin <no.email@nospam.invalid> - 2015-02-17 08:43 -0800
    Re: python implementation of a new integer encoding algorithm. janhein.vanderburg@gmail.com - 2015-02-18 01:06 -0800
  Re: python implementation of a new integer encoding algorithm. Mario Figueiredo <marfig@gmail.com> - 2015-02-19 08:44 +0100
    Re: python implementation of a new integer encoding algorithm. Mark Lawrence <breamoreboy@yahoo.co.uk> - 2015-02-19 08:06 +0000
      Re: python implementation of a new integer encoding algorithm. Marko Rauhamaa <marko@pacujo.net> - 2015-02-19 10:36 +0200
        Re: python implementation of a new integer encoding algorithm. Mark Lawrence <breamoreboy@yahoo.co.uk> - 2015-02-19 09:33 +0000
        Re: python implementation of a new integer encoding algorithm. Terry Reedy <tjreedy@udel.edu> - 2015-02-19 14:50 -0500
          Re: python implementation of a new integer encoding algorithm. Marko Rauhamaa <marko@pacujo.net> - 2015-02-19 21:55 +0200
    Re: python implementation of a new integer encoding algorithm. Chris Angelico <rosuav@gmail.com> - 2015-02-19 19:36 +1100
    Re: python implementation of a new integer encoding algorithm. Mario Figueiredo <marfig@gmail.com> - 2015-02-19 10:42 +0100
    Re: python implementation of a new integer encoding algorithm. Mark Lawrence <breamoreboy@yahoo.co.uk> - 2015-02-19 10:28 +0000
    Re: python implementation of a new integer encoding algorithm. Mario Figueiredo <marfig@gmail.com> - 2015-02-19 14:27 +0100
  Re: python implementation of a new integer encoding algorithm. Jonas Wielicki <jonas@wielicki.name> - 2015-02-19 09:38 +0100
    Re: python implementation of a new integer encoding algorithm. janhein.vanderburg@gmail.com - 2015-02-19 07:58 -0800
  Re: python implementation of a new integer encoding algorithm. Denis McMahon <denismfmcmahon@gmail.com> - 2015-02-20 02:46 +0000
    Re: python implementation of a new integer encoding algorithm. wxjmfauth@gmail.com - 2015-02-20 00:58 -0800

csiph-web