Path: csiph.com!newsfeed.hal-mli.net!feeder3.hal-mli.net!newsfeed.hal-mli.net!feeder1.hal-mli.net!newsfeed.xs4all.nl!newsfeed4.news.xs4all.nl!xs4all!post.news.xs4all.nl!not-for-mail
To: python-list@python.org
From: Dennis Lee Bieber <wlfraed@ix.netcom.com>
Subject: Re: Reading c struct via python
Date: Tue, 12 Nov 2013 20:40:07 -0500
Organization: IISS Elusive Unicorn
References: <CAKuJGC_-i66xtk0aitHKpBQ74ufDKzRn7vXEBjiar+hSGTFZfg@mail.gmail.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Precedence: list
Newsgroups: comp.lang.python
Message-ID: <mailman.2519.1384306817.18130.python-list@python.org>
Lines: 117
NNTP-Posting-Host: 2001:888:2000:d::a6
Xref: csiph.com comp.lang.python:59274

On Tue, 12 Nov 2013 09:44:51 +0530, "Lakshmipathi.G"
<lakshmipathi.g@gmail.com> declaimed the following:

>Hi -
>We have C code which writes following struct into berkeley db ("my_db.db").
>
>struct my_info {
>    unsigned long int i, e;
>    int o;
>    char *f;
>    char *s;
>};
>
>How to read this via Python? Google search gave this code

	Where is the description of the BDB write? What is the key, what is the
data...



>
>While storing an entry (o=>500,f=>/home/laks/abcde,s=>OSr,i=>4668368
>,e=>10000) and reading it back i get
>
>$ python pybsd2.py
>("\x10'\x00\x00\x00\x00\x00\x00",
>"\xf4\x01\x00\x00\xd0;G\x00\x00\x00\x00\x00\x10'\x00\x00\x00\x00\x00\x00/home/laks/abcde\x00OSr\x00")
>
	Looks normal for the representation of a binary "string"

>>> import struct
>>> a = "\x10'\x00\x00\x00\x00\x00\x00"
>>> a
"\x10'\x00\x00\x00\x00\x00\x00"
>>> struct.unpack("Q", a)
(10000,)

	Looks like your "e"... Is that the key field?

>>> b = "\xf4\x01\x00\x00\xd0;G\x00\x00\x00\x00\x00\x10'\x00\x00\x00\x00\x00\x00/home/laks/abcde\x00OSr\x00"
>>> b
"\xf4\x01\x00\x00\xd0;G\x00\x00\x00\x00\x00\x10'\x00\x00\x00\x00\x00\x00/home/laks/abcde\x00OSr\x00"
>>> struct.unpack("<lQ", b[:12])
(500, 4668368)


	That looks like your "o" and "i", but note that I had to specify
little-endian packed data... AND the order of the items is "o" before "i"
-- that doesn't seem to match your C struct definition.

>>> c = b[12:]
>>> c
"\x10'\x00\x00\x00\x00\x00\x00/home/laks/abcde\x00OSr\x00"
>>> struct.unpack("Q", c[:8])
(10000,)

	Hmmm, and here is your "e" value again... So far your structure appears
to be

o	32-bit int
i	64-bit uint
e	64-bit unit

NOT 

i	64 uint
e	64 uint
o	32 int

	Now comes the big problem... The struct module understands two types of
strings. "s" format in which you precode the length of the string in the
format ("10s" is a 10 character string), and "p" format in which the first
byte of the string is the length (0..255) of the rest of the string. The
struct module doesn't handle C-type null terminated strings directly.
	
>>> d = c[8:]
>>> d
'/home/laks/abcde\x00OSr\x00'
>>> n = d.index("\x00")
>>> n
16
>>> e = d[:n]
>>> e
'/home/laks/abcde'

	We've already got the string now, but to continue with struct module,
we'll use the above "n"

>>> s = struct.unpack("16s", e)
>>> s
('/home/laks/abcde',)

	And now to get the other one skipping the previous null termination

>>> f = d[n+1:]
>>> f
'OSr\x00'
>>> n2 = f.index("\x00")
>>> f[:n2]
'OSr'
>>> n2
3
>>> g = f[:n2]
>>> s2 = struct.unpack("3s", g)
>>> s2
('OSr',)
>>>

If you really want to do it in one struct.unpack, you'll need to modify
your C output structure to do fixed length strings without null termination
OR for variable length write the length as a byte and again drop the null
terminator.
 
-- 
	Wulfraed                 Dennis Lee Bieber         AF6VN
    wlfraed@ix.netcom.com    HTTP://wlfraed.home.netcom.com/