Re: Serialization library, request for feedback

From	Ian Collins <ian-news@hotmail.com>
Newsgroups	comp.lang.c
Subject	Re: Serialization library, request for feedback
Date	2012-12-15 14:56 +1300
Message-ID	<aj23n6Fjn61U4@mid.individual.net> (permalink)
References	(7 earlier) <aj1g5kFjn61U1@mid.individual.net> <50CB8C8B.8080608@verizon.net> <kag5nv$5fo$1@news.albasani.net> <aj1qqfFjn61U2@mid.individual.net> <kagh56$qb2$1@news.albasani.net>

Show all headers | View raw

BGB wrote:
> On 12/14/2012 5:25 PM, Ian Collins wrote:
>> BGB wrote:
>>> On 12/14/2012 2:31 PM, James Kuyper wrote:
>>>> On 12/14/2012 03:23 PM, Ian Collins wrote:.
>>>>>
>>>>> How often would you want to serialise a structure that contains
>>>>> pointers?  Pointer values are only useful within the current executing
>>>>> program, so you have to use some form of conversion.
>>>>
>>>> Yes, and ideally the serialization system should handle that conversion
>>>> for you, if you give it enough information to do so. I don't claim to
>>>> know of any serialization system that meets that ideal.
>>>
>>> if I understand what is meant here, this case would likely a *very* hard
>>> problem for a general-purpose serialization API...
>>
>> Don't loose sight of the difference between the serialisation technique
>> and the representation of the serialised data.  The same problems (such
>> as what to do with pointers) exist with any general purpose
>> serialisation technique or sufficiently expressive data representation.
>
> usually, the matter of pointers is transformed into some sort of object
> index, but the problem becomes, what if the pointer doesn't point at a
> well-defined object?...

Then the object isn't a suitable candidate for serialisation.

I don't agree with your "usually".  Unless you are dealing with the very 
rare case where a pointer value (even expressed as a base + offset) is 
meaningful to the reader of the serialised data as well as the writer, 
the only sensible thing to do is the equivalent of a deep copy and 
serialise the data pointed to and not the pointer value.

>> How the data types are described is another, independent, problem.  Even
>> in C code, a structure object requires meta-data (the struct
>> declaration) to describe its type.
>>
>
> even as such, C has many holes in its natural type representations:
> information which may be needed to serialize an object with a given type
> may not be present within the types as understood by the language itself.

Eh?

> hence, why some information may be needed in terms of special
> annotations, and run-time type-tags, because even though the tool can
> understand the types (struct declarations, ...), some information may
> still not be knowable from the declarations themselves.
>
> like, say:
> "int *a;"
>
> given the 'a' pointer, how do you know where the beginning or end of its
> associated memory object is at?... run-time information may be needed in
> this case (in the sense that the memory manager may know where the array
> is located, and how big it is).

Ah-ha.  See above.

>> In my experience serialising data containing pointers is a very unusual
>> occurrence.  The only time I have done this is where a set of
>> cooperating processes are using data in shared memory.  In this case,
>> pointers can either be serialised as an offset from the segment base or
>> as plain integer values if all processes use the same base address for
>> the segment.
>>
>
> most often, IME, the pointers are actually object references (either
> pointing to another structure, or an array of values, ...).
>
> in other cases, there may be other problems.
>
> granted, in my case, nearly all of my program's memory is allocated via
> the special memory manager (I very rarely use "malloc()" or similar),
> so, in this case, there is usually sufficient information to work with.

Not if you are sending the data to another language....

-- 
Ian Collins

Thread

Serialization library, request for feedback Ulf Åström <ulf.astrom@gmail.com> - 2012-12-13 13:46 -0800
  Re: Serialization library, request for feedback Keith Thompson <kst-u@mib.org> - 2012-12-13 13:57 -0800
    Re: Serialization library, request for feedback Ulf Åström <ulf.astrom@gmail.com> - 2012-12-13 14:30 -0800
      Re: Serialization library, request for feedback Ian Collins <ian-news@hotmail.com> - 2012-12-14 19:38 +1300
        Re: Serialization library, request for feedback BGB <cr88192@hotmail.com> - 2012-12-14 03:47 -0600
          Re: Serialization library, request for feedback Ian Collins <ian-news@hotmail.com> - 2012-12-14 23:23 +1300
            Re: Serialization library, request for feedback BGB <cr88192@hotmail.com> - 2012-12-14 11:29 -0600
          Re: Serialization library, request for feedback Ulf Åström <ulf.astrom@gmail.com> - 2012-12-14 03:51 -0800
            Re: Serialization library, request for feedback BGB <cr88192@hotmail.com> - 2012-12-14 12:02 -0600
        Re: Serialization library, request for feedback Ben Bacarisse <ben.usenet@bsb.me.uk> - 2012-12-14 12:18 +0000
          Re: Serialization library, request for feedback Rui Maciel <rui.maciel@gmail.com> - 2012-12-14 14:27 +0000
            Re: Serialization library, request for feedback Ben Bacarisse <ben.usenet@bsb.me.uk> - 2012-12-14 15:22 +0000
              Re: Serialization library, request for feedback BGB <cr88192@hotmail.com> - 2012-12-14 13:00 -0600
                Re: Serialization library, request for feedback Greg Martin <greg@softsprocket.com> - 2012-12-14 11:58 -0800
                Re: Serialization library, request for feedback BGB <cr88192@hotmail.com> - 2012-12-14 15:08 -0600
                Re: Serialization library, request for feedback William Ahern <william@wilbur.25thandClement.com> - 2012-12-14 20:07 -0800
                Re: Serialization library, request for feedback Nick Keighley <nick_keighley_nospam@hotmail.com> - 2012-12-16 03:49 -0800
                Re: Serialization library, request for feedback BGB <cr88192@hotmail.com> - 2012-12-16 13:17 -0600
                Re: Serialization library, request for feedback Greg Martin <greg@softsprocket.com> - 2012-12-17 09:07 -0800
                Re: Serialization library, request for feedback BGB <cr88192@hotmail.com> - 2012-12-17 13:25 -0600
              Re: Serialization library, request for feedback Ian Collins <ian-news@hotmail.com> - 2012-12-15 09:23 +1300
                Re: Serialization library, request for feedback James Kuyper <jameskuyper@verizon.net> - 2012-12-14 15:31 -0500
                Re: Serialization library, request for feedback BGB <cr88192@hotmail.com> - 2012-12-14 15:26 -0600
                Re: Serialization library, request for feedback Ian Collins <ian-news@hotmail.com> - 2012-12-15 12:25 +1300
                Re: Serialization library, request for feedback BGB <cr88192@hotmail.com> - 2012-12-14 18:41 -0600
                Re: Serialization library, request for feedback Ian Collins <ian-news@hotmail.com> - 2012-12-15 14:56 +1300
                Re: Serialization library, request for feedback BGB <cr88192@hotmail.com> - 2012-12-14 23:36 -0600
                Re: Serialization library, request for feedback Ian Collins <ian-news@hotmail.com> - 2012-12-15 20:13 +1300
                Re: Serialization library, request for feedback Shao Miller <sha0.miller@gmail.com> - 2012-12-14 21:42 -0500
                Re: Serialization library, request for feedback BGB <cr88192@hotmail.com> - 2012-12-14 23:47 -0600
              Re: Serialization library, request for feedback Rui Maciel <rui.maciel@gmail.com> - 2012-12-14 21:12 +0000
                Re: Serialization library, request for feedback Johann Klammer <klammerj@NOSPAM.a1.net> - 2012-12-15 05:06 +0100
    Re: Serialization library, request for feedback Willem <willem@turtle.stack.nl> - 2012-12-14 22:20 +0000
  Re: Serialization library, request for feedback Ben Bacarisse <ben.usenet@bsb.me.uk> - 2012-12-13 23:11 +0000
    Re: Serialization library, request for feedback Ulf Åström <ulf.astrom@gmail.com> - 2012-12-13 15:57 -0800
      Re: Serialization library, request for feedback Ben Bacarisse <ben.usenet@bsb.me.uk> - 2012-12-14 02:53 +0000
        Re: Serialization library, request for feedback Ulf Åström <ulf.astrom@gmail.com> - 2012-12-14 04:25 -0800
      Re: Serialization library, request for feedback Bart van Ingen Schenau <bart@ingen.ddns.info.invalid> - 2012-12-14 08:04 +0000
  Re: Serialization library, request for feedback Rui Maciel <rui.maciel@gmail.com> - 2012-12-14 14:25 +0000
    Re: Serialization library, request for feedback BGB <cr88192@hotmail.com> - 2012-12-14 13:58 -0600
      Re: Serialization library, request for feedback "BartC" <bc@freeuk.com> - 2012-12-14 20:44 +0000
        Re: Serialization library, request for feedback BGB <cr88192@hotmail.com> - 2012-12-14 16:58 -0600
        Re: Serialization library, request for feedback Ulf Åström <ulf.astrom@gmail.com> - 2012-12-14 17:39 -0800
    Re: Serialization library, request for feedback Malcolm McLean <malcolm.mclean5@btinternet.com> - 2012-12-14 14:19 -0800

csiph-web