Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.c > #29270

Re: Serialization library, request for feedback

Show all headers | View raw

BGB wrote:
> On 12/14/2012 7:56 PM, Ian Collins wrote:
>> BGB wrote:

>>> usually, the matter of pointers is transformed into some sort of object
>>> index, but the problem becomes, what if the pointer doesn't point at a
>>> well-defined object?...
>>
>> Then the object isn't a suitable candidate for serialisation.
>>
>
> I thought someone here was objecting here to serialization not working
> with structures mapped into a ring-buffer?... (IOW: a big glob of memory
> treated like a ring, with structures allocated "around" the inside of
> the ring...).

My point was an object that contains a pointer to a blob of unspecified 
size isn't suitable for serialisation.  If the object is designed to be 
serialised (of if the requirement is retrofitted), it would have an 
embedded size, or provide a means to determine the size of the blob.  In 
practice the conditions for serialising are pretty much the same as 
those for copying an object.

>> I don't agree with your "usually".  Unless you are dealing with the very
>> rare case where a pointer value (even expressed as a base + offset) is
>> meaningful to the reader of the serialised data as well as the writer,
>> the only sensible thing to do is the equivalent of a deep copy and
>> serialise the data pointed to and not the pointer value.
>>
>
> the "usually" is because there are some ways to implement persistence
> mechanisms which do depend on pointer addresses... (usually by
> implementing it via "mmap()" or "CreateFileMapping()" or similar...).

That's basically what I described up thread as the case of a set of 
cooperating processes are using data in shared memory.

> in most other cases, any pointer is mapped to an object index or
> similar, and the original address is not preserved. (hence, "usually").

Again, I disagree.  it makes more sense (in a general scheme) to 
serialise the data pointed to, not the pointer.  Unless you also intend 
including the entire memory space in the serialised data.  The only time 
I see that is in a core file.

>>>> How the data types are described is another, independent, problem.  Even
>>>> in C code, a structure object requires meta-data (the struct
>>>> declaration) to describe its type.
>>>>
>>>
>>> even as such, C has many holes in its natural type representations:
>>> information which may be needed to serialize an object with a given type
>>> may not be present within the types as understood by the language itself.
>>
>> Eh?
>>
>
> like, array bounds for raw-memory arrays.
>
> if a person is like:
> int *a;
> a=malloc(256*sizeof(int));

If you wanted to serialise a, you would hang on to the size.

> other possibilities include a pointer to a struct physically embedded
> within another struct.
>
> typedef struct
> {
> int x, y;
> }Foo;
>
> typedef struct
> {
> int w;
> Foo foo;
> int z;
> }Bar;
>
> Bar *bar;
> Foo *foo;
> bar=malloc(sizeof(Bar));
> foo=&(bar->foo);
>
> if we try to serialize with nothing more than 'foo', there is not enough
> information available (within the C type-system), for a serialization
> API to realize that 'foo' is contained within an instance of Bar, and
> that the object pointed to by 'bar' should be serialized instead.

That sounds like a programming error rather than a deficiency in the C 
type system.

> another similar example, would be allocating a buffer for the contents
> of a file, then casting various parts of this buffer to various structs
> pointers.
>
> a serializer working simply by walking the pointers would likely be
> entirely unaware that the structs are held within a common buffer (and
> would not preserve their physical spatial relationship within this buffer).

If you want a portable, persistent representation of the data, that is 
probably a good thing!

> granted, a lot of these are cases which my APIs don't really address,
> nor would I really want to try to address them...
>
> this is part of the reason for some of the "restrictions" mentioned
> before, which mostly serve to place more limits of what can be called
> "well defined" data structures.

In most of what I do the serialisation API is "convert to JSON, stream". 
  So if something can't be represented as JSON, it can't be serialised. 
  This rule came about in order to share data between JavaScript clients 
and C++, C or PHP servers.  I have subsequently found internal JSON 
objects incredibly useful for building and passing dynamic types, 
especially as I can use almost identical code in JavaScript, PHP and C++ 
(but alas, not C) to manipulate them.

-- 
Ian Collins

Back to comp.lang.c | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Serialization library, request for feedback Ulf Åström <ulf.astrom@gmail.com> - 2012-12-13 13:46 -0800
  Re: Serialization library, request for feedback Keith Thompson <kst-u@mib.org> - 2012-12-13 13:57 -0800
    Re: Serialization library, request for feedback Ulf Åström <ulf.astrom@gmail.com> - 2012-12-13 14:30 -0800
      Re: Serialization library, request for feedback Ian Collins <ian-news@hotmail.com> - 2012-12-14 19:38 +1300
        Re: Serialization library, request for feedback BGB <cr88192@hotmail.com> - 2012-12-14 03:47 -0600
          Re: Serialization library, request for feedback Ian Collins <ian-news@hotmail.com> - 2012-12-14 23:23 +1300
            Re: Serialization library, request for feedback BGB <cr88192@hotmail.com> - 2012-12-14 11:29 -0600
          Re: Serialization library, request for feedback Ulf Åström <ulf.astrom@gmail.com> - 2012-12-14 03:51 -0800
            Re: Serialization library, request for feedback BGB <cr88192@hotmail.com> - 2012-12-14 12:02 -0600
        Re: Serialization library, request for feedback Ben Bacarisse <ben.usenet@bsb.me.uk> - 2012-12-14 12:18 +0000
          Re: Serialization library, request for feedback Rui Maciel <rui.maciel@gmail.com> - 2012-12-14 14:27 +0000
            Re: Serialization library, request for feedback Ben Bacarisse <ben.usenet@bsb.me.uk> - 2012-12-14 15:22 +0000
              Re: Serialization library, request for feedback BGB <cr88192@hotmail.com> - 2012-12-14 13:00 -0600
                Re: Serialization library, request for feedback Greg Martin <greg@softsprocket.com> - 2012-12-14 11:58 -0800
                Re: Serialization library, request for feedback BGB <cr88192@hotmail.com> - 2012-12-14 15:08 -0600
                Re: Serialization library, request for feedback William Ahern <william@wilbur.25thandClement.com> - 2012-12-14 20:07 -0800
                Re: Serialization library, request for feedback Nick Keighley <nick_keighley_nospam@hotmail.com> - 2012-12-16 03:49 -0800
                Re: Serialization library, request for feedback BGB <cr88192@hotmail.com> - 2012-12-16 13:17 -0600
                Re: Serialization library, request for feedback Greg Martin <greg@softsprocket.com> - 2012-12-17 09:07 -0800
                Re: Serialization library, request for feedback BGB <cr88192@hotmail.com> - 2012-12-17 13:25 -0600
              Re: Serialization library, request for feedback Ian Collins <ian-news@hotmail.com> - 2012-12-15 09:23 +1300
                Re: Serialization library, request for feedback James Kuyper <jameskuyper@verizon.net> - 2012-12-14 15:31 -0500
                Re: Serialization library, request for feedback BGB <cr88192@hotmail.com> - 2012-12-14 15:26 -0600
                Re: Serialization library, request for feedback Ian Collins <ian-news@hotmail.com> - 2012-12-15 12:25 +1300
                Re: Serialization library, request for feedback BGB <cr88192@hotmail.com> - 2012-12-14 18:41 -0600
                Re: Serialization library, request for feedback Ian Collins <ian-news@hotmail.com> - 2012-12-15 14:56 +1300
                Re: Serialization library, request for feedback BGB <cr88192@hotmail.com> - 2012-12-14 23:36 -0600
                Re: Serialization library, request for feedback Ian Collins <ian-news@hotmail.com> - 2012-12-15 20:13 +1300
                Re: Serialization library, request for feedback Shao Miller <sha0.miller@gmail.com> - 2012-12-14 21:42 -0500
                Re: Serialization library, request for feedback BGB <cr88192@hotmail.com> - 2012-12-14 23:47 -0600
              Re: Serialization library, request for feedback Rui Maciel <rui.maciel@gmail.com> - 2012-12-14 21:12 +0000
                Re: Serialization library, request for feedback Johann Klammer <klammerj@NOSPAM.a1.net> - 2012-12-15 05:06 +0100
    Re: Serialization library, request for feedback Willem <willem@turtle.stack.nl> - 2012-12-14 22:20 +0000
  Re: Serialization library, request for feedback Ben Bacarisse <ben.usenet@bsb.me.uk> - 2012-12-13 23:11 +0000
    Re: Serialization library, request for feedback Ulf Åström <ulf.astrom@gmail.com> - 2012-12-13 15:57 -0800
      Re: Serialization library, request for feedback Ben Bacarisse <ben.usenet@bsb.me.uk> - 2012-12-14 02:53 +0000
        Re: Serialization library, request for feedback Ulf Åström <ulf.astrom@gmail.com> - 2012-12-14 04:25 -0800
      Re: Serialization library, request for feedback Bart van Ingen Schenau <bart@ingen.ddns.info.invalid> - 2012-12-14 08:04 +0000
  Re: Serialization library, request for feedback Rui Maciel <rui.maciel@gmail.com> - 2012-12-14 14:25 +0000
    Re: Serialization library, request for feedback BGB <cr88192@hotmail.com> - 2012-12-14 13:58 -0600
      Re: Serialization library, request for feedback "BartC" <bc@freeuk.com> - 2012-12-14 20:44 +0000
        Re: Serialization library, request for feedback BGB <cr88192@hotmail.com> - 2012-12-14 16:58 -0600
        Re: Serialization library, request for feedback Ulf Åström <ulf.astrom@gmail.com> - 2012-12-14 17:39 -0800
    Re: Serialization library, request for feedback Malcolm McLean <malcolm.mclean5@btinternet.com> - 2012-12-14 14:19 -0800

csiph-web