Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.c > #29255

Re: Serialization library, request for feedback

Show all headers | View raw

On 12/14/2012 5:25 PM, Ian Collins wrote:
> BGB wrote:
>> On 12/14/2012 2:31 PM, James Kuyper wrote:
>>> On 12/14/2012 03:23 PM, Ian Collins wrote:
>>>> Ben Bacarisse wrote:
>>>>> Rui Maciel <rui.maciel@gmail.com> writes:
>>>>>
>>>>>> Ben Bacarisse wrote:
>>>>>>
>>>>>>> I don't think JSON is a good fit for the OP's task.  There are
>>>>>>> lots of C
>>>>>>> data structures that have no obvious JSON representation at all.
>>>>>>
>>>>>> What data structures do you have in mind?
>>>>>
>>>>> My ring_buffer example is one.  In fact, any structure where
>>>>> pointers are
>>>>> "shared" would seem to be a bad fit with JSON.
>>>>
>>>> How often would you want to serialise a structure that contains
>>>> pointers?  Pointer values are only useful within the current executing
>>>> program, so you have to use some form of conversion.
>>>
>>> Yes, and ideally the serialization system should handle that conversion
>>> for you, if you give it enough information to do so. I don't claim to
>>> know of any serialization system that meets that ideal.
>>
>> if I understand what is meant here, this case would likely a *very* hard
>> problem for a general-purpose serialization API...
>
> Don't loose sight of the difference between the serialisation technique
> and the representation of the serialised data.  The same problems (such
> as what to do with pointers) exist with any general purpose
> serialisation technique or sufficiently expressive data representation.
>

usually, the matter of pointers is transformed into some sort of object 
index, but the problem becomes, what if the pointer doesn't point at a 
well-defined object?...

say, a person is pointing to raw memory gained via "mmap()" or 
"malloc()" or similar.


a lot of my stuff doesn't work correctly in these cases, and if given a 
pointer to such memory, it will have NULL as the dynamic type.


my scripting language can deal with it, so far as it is also capable of 
seeing the types "C-style" (and so will work with the data according to 
its struct declaration and/or pointer types). (when doing so, it will no 
longer have any notion of array-bounds though, since there may not be 
any information available to tell the VM where the beginning or end of 
the array are located, ...).

but, the serialization code will have no idea, and will generally encode 
it as an "UNDEFINED" value.



> How the data types are described is another, independent, problem.  Even
> in C code, a structure object requires meta-data (the struct
> declaration) to describe its type.
>

even as such, C has many holes in its natural type representations:
information which may be needed to serialize an object with a given type 
may not be present within the types as understood by the language itself.

hence, why some information may be needed in terms of special 
annotations, and run-time type-tags, because even though the tool can 
understand the types (struct declarations, ...), some information may 
still not be knowable from the declarations themselves.


like, say:
"int *a;"

given the 'a' pointer, how do you know where the beginning or end of its 
associated memory object is at?... run-time information may be needed in 
this case (in the sense that the memory manager may know where the array 
is located, and how big it is).

but, the memory manager may only know this if the memory for 'a' was 
allocated via the appropriate functions. (if it came directly from 
"mmap()" or similar, the memory manager may not have any idea).

granted, yes, the memory manager gets its own memory initially from 
"mmap()" or "VirtualAlloc()" or similar, but it remembers which memory 
chunks exist, and can identify individual memory objects within these 
chunks.


a problem though is that, usually, there is not sufficient information 
to work with a union, either within C, or within run-time tags, since if 
all we know is that it is a union, it doesn't help much knowing which 
possibility regarding its contents is the "correct" one.


> In my experience serialising data containing pointers is a very unusual
> occurrence.  The only time I have done this is where a set of
> cooperating processes are using data in shared memory.  In this case,
> pointers can either be serialised as an offset from the segment base or
> as plain integer values if all processes use the same base address for
> the segment.
>

most often, IME, the pointers are actually object references (either 
pointing to another structure, or an array of values, ...).

in other cases, there may be other problems.


granted, in my case, nearly all of my program's memory is allocated via 
the special memory manager (I very rarely use "malloc()" or similar), 
so, in this case, there is usually sufficient information to work with.

Back to comp.lang.c | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Serialization library, request for feedback Ulf Åström <ulf.astrom@gmail.com> - 2012-12-13 13:46 -0800
  Re: Serialization library, request for feedback Keith Thompson <kst-u@mib.org> - 2012-12-13 13:57 -0800
    Re: Serialization library, request for feedback Ulf Åström <ulf.astrom@gmail.com> - 2012-12-13 14:30 -0800
      Re: Serialization library, request for feedback Ian Collins <ian-news@hotmail.com> - 2012-12-14 19:38 +1300
        Re: Serialization library, request for feedback BGB <cr88192@hotmail.com> - 2012-12-14 03:47 -0600
          Re: Serialization library, request for feedback Ian Collins <ian-news@hotmail.com> - 2012-12-14 23:23 +1300
            Re: Serialization library, request for feedback BGB <cr88192@hotmail.com> - 2012-12-14 11:29 -0600
          Re: Serialization library, request for feedback Ulf Åström <ulf.astrom@gmail.com> - 2012-12-14 03:51 -0800
            Re: Serialization library, request for feedback BGB <cr88192@hotmail.com> - 2012-12-14 12:02 -0600
        Re: Serialization library, request for feedback Ben Bacarisse <ben.usenet@bsb.me.uk> - 2012-12-14 12:18 +0000
          Re: Serialization library, request for feedback Rui Maciel <rui.maciel@gmail.com> - 2012-12-14 14:27 +0000
            Re: Serialization library, request for feedback Ben Bacarisse <ben.usenet@bsb.me.uk> - 2012-12-14 15:22 +0000
              Re: Serialization library, request for feedback BGB <cr88192@hotmail.com> - 2012-12-14 13:00 -0600
                Re: Serialization library, request for feedback Greg Martin <greg@softsprocket.com> - 2012-12-14 11:58 -0800
                Re: Serialization library, request for feedback BGB <cr88192@hotmail.com> - 2012-12-14 15:08 -0600
                Re: Serialization library, request for feedback William Ahern <william@wilbur.25thandClement.com> - 2012-12-14 20:07 -0800
                Re: Serialization library, request for feedback Nick Keighley <nick_keighley_nospam@hotmail.com> - 2012-12-16 03:49 -0800
                Re: Serialization library, request for feedback BGB <cr88192@hotmail.com> - 2012-12-16 13:17 -0600
                Re: Serialization library, request for feedback Greg Martin <greg@softsprocket.com> - 2012-12-17 09:07 -0800
                Re: Serialization library, request for feedback BGB <cr88192@hotmail.com> - 2012-12-17 13:25 -0600
              Re: Serialization library, request for feedback Ian Collins <ian-news@hotmail.com> - 2012-12-15 09:23 +1300
                Re: Serialization library, request for feedback James Kuyper <jameskuyper@verizon.net> - 2012-12-14 15:31 -0500
                Re: Serialization library, request for feedback BGB <cr88192@hotmail.com> - 2012-12-14 15:26 -0600
                Re: Serialization library, request for feedback Ian Collins <ian-news@hotmail.com> - 2012-12-15 12:25 +1300
                Re: Serialization library, request for feedback BGB <cr88192@hotmail.com> - 2012-12-14 18:41 -0600
                Re: Serialization library, request for feedback Ian Collins <ian-news@hotmail.com> - 2012-12-15 14:56 +1300
                Re: Serialization library, request for feedback BGB <cr88192@hotmail.com> - 2012-12-14 23:36 -0600
                Re: Serialization library, request for feedback Ian Collins <ian-news@hotmail.com> - 2012-12-15 20:13 +1300
                Re: Serialization library, request for feedback Shao Miller <sha0.miller@gmail.com> - 2012-12-14 21:42 -0500
                Re: Serialization library, request for feedback BGB <cr88192@hotmail.com> - 2012-12-14 23:47 -0600
              Re: Serialization library, request for feedback Rui Maciel <rui.maciel@gmail.com> - 2012-12-14 21:12 +0000
                Re: Serialization library, request for feedback Johann Klammer <klammerj@NOSPAM.a1.net> - 2012-12-15 05:06 +0100
    Re: Serialization library, request for feedback Willem <willem@turtle.stack.nl> - 2012-12-14 22:20 +0000
  Re: Serialization library, request for feedback Ben Bacarisse <ben.usenet@bsb.me.uk> - 2012-12-13 23:11 +0000
    Re: Serialization library, request for feedback Ulf Åström <ulf.astrom@gmail.com> - 2012-12-13 15:57 -0800
      Re: Serialization library, request for feedback Ben Bacarisse <ben.usenet@bsb.me.uk> - 2012-12-14 02:53 +0000
        Re: Serialization library, request for feedback Ulf Åström <ulf.astrom@gmail.com> - 2012-12-14 04:25 -0800
      Re: Serialization library, request for feedback Bart van Ingen Schenau <bart@ingen.ddns.info.invalid> - 2012-12-14 08:04 +0000
  Re: Serialization library, request for feedback Rui Maciel <rui.maciel@gmail.com> - 2012-12-14 14:25 +0000
    Re: Serialization library, request for feedback BGB <cr88192@hotmail.com> - 2012-12-14 13:58 -0600
      Re: Serialization library, request for feedback "BartC" <bc@freeuk.com> - 2012-12-14 20:44 +0000
        Re: Serialization library, request for feedback BGB <cr88192@hotmail.com> - 2012-12-14 16:58 -0600
        Re: Serialization library, request for feedback Ulf Åström <ulf.astrom@gmail.com> - 2012-12-14 17:39 -0800
    Re: Serialization library, request for feedback Malcolm McLean <malcolm.mclean5@btinternet.com> - 2012-12-14 14:19 -0800

csiph-web