Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #70220

Re: MemoryError in data conversion

Show all headers | View raw

Mok-Kong Shen wrote:

> Am 14.04.2014 09:46, schrieb Peter Otten:
> 
>> You ran into a limitation of the compiler. For us to suggest a workaround
>> you'd have to explain why you want to convert the list returned from
>> buildhuffmantree() into python source code and back.
> 
> That list gives the Huffman encoding tree for compressing a given piece
> of source text. I am writing a Python code to implement an algorithm
> (not new, being first sketched in the literature since decades but yet
> having no publically available implementation as far as I am aware) of
> encryption processing that has Huffman data compression as its major
> constituent. Now, for good security against cryptanalysis, this list
> (which has to be included in the ciphertext for decryption by the
> recipient) has to be well scrambled in some way. I choose to use 8-bit
> bytes as units for the scrambling. Hence I convert the list to a
> bytearray for performing scrambling. On decryption I reverse the
> scrambling and get back the original bytearray and use ast to recover
> from it the list so as to be able to do the decompression. Hopefully
> this description is sufficiently clear.

You could use json, but you may run into the same problem with that, too 
(only later):

>>> import json
>>> items = []
>>> for i in range(1000):
...     s = json.dumps(items)
...     items = [items]
... 
Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
  File "/usr/lib/python3.3/json/__init__.py", line 236, in dumps
    return _default_encoder.encode(obj)
  File "/usr/lib/python3.3/json/encoder.py", line 191, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "/usr/lib/python3.3/json/encoder.py", line 249, in iterencode
    return _iterencode(o, 0)
RuntimeError: maximum recursion depth exceeded while encoding a JSON object
>>> i
995

The safest option is probably to serialize the original flist and slist, and 
use them to create the tree on the fly.

Back to comp.lang.python | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

MemoryError in data conversion Mok-Kong Shen <mok-kong.shen@t-online.de> - 2014-04-14 03:46 +0200
  Re: MemoryError in data conversion dieter <dieter@handshake.de> - 2014-04-14 08:14 +0200
  Re: MemoryError in data conversion Peter Otten <__peter__@web.de> - 2014-04-14 09:46 +0200
    Re: MemoryError in data conversion Mok-Kong Shen <mok-kong.shen@t-online.de> - 2014-04-14 14:26 +0200
      Re: MemoryError in data conversion Peter Otten <__peter__@web.de> - 2014-04-14 15:59 +0200
        Re: MemoryError in data conversion Mok-Kong Shen <mok-kong.shen@t-online.de> - 2014-04-14 23:20 +0200
          Re: MemoryError in data conversion Gregory Ewing <greg.ewing@canterbury.ac.nz> - 2014-04-15 11:51 +1200
            Re: MemoryError in data conversion Mok-Kong Shen <mok-kong.shen@t-online.de> - 2014-04-15 10:55 +0200
              Re: MemoryError in data conversion Gregory Ewing <greg.ewing@canterbury.ac.nz> - 2014-04-16 00:08 +1200
            Re: MemoryError in data conversion Peter Otten <__peter__@web.de> - 2014-04-15 11:59 +0200
  Re:MemoryError in data conversion Dave Angel <davea@davea.name> - 2014-04-14 08:42 -0400

csiph-web