Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]


Groups > comp.lang.c > #143904

Re: Static JSON parser for embedded

From pozz <pozzugno@gmail.com>
Newsgroups comp.lang.c
Subject Re: Static JSON parser for embedded
Date 2019-05-18 00:11 +0200
Organization A noiseless patient Spider
Message-ID <qbnbjc$js3$1@dont-email.me> (permalink)
References (3 earlier) <sdzDE.389927$v12.254932@fx29.am4> <qbmgvk$n78$1@dont-email.me> <XIBDE.241096$SH1.40745@fx03.am4> <d3002c74-f4d0-4a50-a524-a4dfe608d4fb@googlegroups.com> <9pCDE.1896414$oQj.765689@fx12.am4>

Show all headers | View raw


Il 17/05/2019 19:43, Bart ha scritto:
> On 17/05/2019 18:21, Rick C. Hodgin wrote:
>> On Friday, May 17, 2019 at 12:56:33 PM UTC-4, Bart wrote:
>>> I think I'll have a go at doing this, although I don't know if I'll come
>>> up with anything that will be of any use to you.
>>>
>>> But some questions about the spec:
>>>
>>> * Tags in JSON can be in any order?
>>>
>>> * Unknown tags to be ignored?
>>>
>>> * Missing tags to be ignored? (Allowing both of these I think allows any
>>> JSON object to match any struct)
>>>
>>> * No char* fields? (These might need heap memory)
>>>
>>> * Float fields needed?
>>>
>>> * Will arrays fields be only 1D, or can be 2D and 3D?
>>
>> Might make a good comp.lang.c coding challenge.
> 
> Maybe, but the whole thing is rather open-ended. Even forgetting about 
> converting a C struct to C parsing code, just the parsing part, even for 
> a struct format that is hard-coded, is full of choices that need to be 
> made.
> 
> The specifications are not clear enough (the above is just part of it).
> 
> Having data come in in any order makes it entirely different: reading an 
> expected string or number is easier than reading an known value of half 
> a dozen type categories. What about underflow in array elements? Too 
> many array elements? 

You define a maximum number of elements of an array in the C struct 
definition. If there are less elements in the JSON message, we can use a 
new meta-member size. If there are more elements, the extra elements are 
simply ignored.

struct {
   int array[MAX_ARRAY_SIZE];
   size_t array_size;

   int array2[MAX_ARRAY2_SIZE];
   size_t array2_size;
} myStruct;


> Mixed types within an array? (All are possible in 
> JSON data.)

No, I wouldn't support this.


> How to distinguish between 'char[32]' which can be initialised with a 
> string, and one which is an array of integers, or allow either?

If we decide to define the C struct with a newer language (not pure C), 
we can define the exact type of the member (string or array of chars).

Another possibility is to use char[] for strings and (unsigned) int for 
integers (no short or long).


> It is rather a can of worms. (I'm doing it in a soft language first to 
> see what is possible.)
> 
> The OP added a few extra bits in as well, like /adding/ optional flag 
> members to the struct, with the implication that the struct definition 
> now additionally has to be part of the generated code. 

Considering it is very difficult to read the definition of a C struct in 
a pure C code (think of comments, preprocessor macros...), I think we 
start with a new struct definition language (.cj). The output of the 
generator will be .c and .h file, latter with C struct definition.

> (Itself full of 
> problems, if the struct has dependencies of a 1000s of lines of C 
> headers; you don't want to drag in all that.)
> 
> (The way it's coming out, the struct definition doesn't need to be 
> visible in the generated parser; it just needs tables of names, types, 
> offsets and sizes.)

Take a look at jsmn[1]. The parser generates an array of tokens. Each 
token has a type, a name, the offset in the message and the size of the 
token.

However, after parsing, it's difficult to search for a precise tag and 
retrive its value to save in the struct.


Another nice feature that some JSON parsers have and that can be very 
useful in restricted environments, is the possibility to parse a message 
incrementally.
Suppose you have a 4kB JSON message stored in an external memory module. 
The processor can't read directly from the external memory that isn't on 
a different bus (SPI, I2C).

The application could read part of the message, maybe 256 bytes, and 
feed the parser for each part, until the end.


[1] https://github.com/zserge/jsmn

Back to comp.lang.c | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread


Thread

Static JSON parser for embedded pozz <pozzugno@gmail.com> - 2019-05-17 09:20 +0200
  Re: Static JSON parser for embedded Jorgen Grahn <grahn+nntp@snipabacken.se> - 2019-05-17 08:48 +0000
    Re: Static JSON parser for embedded pozz <pozzugno@gmail.com> - 2019-05-17 11:06 +0200
      Re: Static JSON parser for embedded Jorgen Grahn <grahn+nntp@snipabacken.se> - 2019-05-17 11:22 +0000
        Re: Static JSON parser for embedded Thiago Adams <thiago.adams@gmail.com> - 2019-05-17 04:40 -0700
          Re: Static JSON parser for embedded pozz <pozzugno@gmail.com> - 2019-05-17 14:10 +0200
            Re: Static JSON parser for embedded Thiago Adams <thiago.adams@gmail.com> - 2019-05-17 05:18 -0700
              Re: Static JSON parser for embedded Jorgen Grahn <grahn+nntp@snipabacken.se> - 2019-05-18 18:18 +0000
        Re: Static JSON parser for embedded pozz <pozzugno@gmail.com> - 2019-05-17 14:10 +0200
          Re: Static JSON parser for embedded Jorgen Grahn <grahn+nntp@snipabacken.se> - 2019-05-18 18:22 +0000
  Re: Static JSON parser for embedded Bart <bc@freeuk.com> - 2019-05-17 11:39 +0100
    Re: Static JSON parser for embedded pozz <pozzugno@gmail.com> - 2019-05-17 13:19 +0200
      Re: Static JSON parser for embedded Thiago Adams <thiago.adams@gmail.com> - 2019-05-17 05:09 -0700
        Re: Static JSON parser for embedded Thiago Adams <thiago.adams@gmail.com> - 2019-05-17 05:15 -0700
      Re: Static JSON parser for embedded Bart <bc@freeuk.com> - 2019-05-17 15:06 +0100
        Re: Static JSON parser for embedded pozz <pozzugno@gmail.com> - 2019-05-17 16:37 +0200
          Re: Static JSON parser for embedded Bart <bc@freeuk.com> - 2019-05-17 17:56 +0100
            Re: Static JSON parser for embedded "Rick C. Hodgin" <rick.c.hodgin@gmail.com> - 2019-05-17 10:21 -0700
              Re: Static JSON parser for embedded Bart <bc@freeuk.com> - 2019-05-17 18:43 +0100
                Re: Static JSON parser for embedded Thiago Adams <thiago.adams@gmail.com> - 2019-05-17 10:58 -0700
                Re: Static JSON parser for embedded "Rick C. Hodgin" <rick.c.hodgin@gmail.com> - 2019-05-17 11:16 -0700
                Re: Static JSON parser for embedded Bart <bc@freeuk.com> - 2019-05-17 19:43 +0100
                Re: Static JSON parser for embedded "Rick C. Hodgin" <rick.c.hodgin@gmail.com> - 2019-05-17 11:57 -0700
                Re: Static JSON parser for embedded Bart <bc@freeuk.com> - 2019-05-17 20:11 +0100
                Re: Static JSON parser for embedded "Rick C. Hodgin" <rick.c.hodgin@gmail.com> - 2019-05-17 12:22 -0700
                Re: Static JSON parser for embedded Ben Bacarisse <ben.usenet@bsb.me.uk> - 2019-05-17 21:33 +0100
                Re: Static JSON parser for embedded pozz <pozzugno@gmail.com> - 2019-05-18 00:11 +0200
            Re: Static JSON parser for embedded Bart <bc@freeuk.com> - 2019-05-17 22:33 +0100
              Re: Static JSON parser for embedded pozz <pozzugno@gmail.com> - 2019-05-18 00:17 +0200
                Re: Static JSON parser for embedded Bart <bc@freeuk.com> - 2019-05-18 01:25 +0100
            Re: Static JSON parser for embedded pozz <pozzugno@gmail.com> - 2019-05-17 23:43 +0200
  Re: Static JSON parser for embedded Thiago Adams <thiago.adams@gmail.com> - 2019-05-17 04:38 -0700
  Re: Static JSON parser for embedded "Rick C. Hodgin" <rick.c.hodgin@gmail.com> - 2019-05-17 07:38 -0700
    Re: Static JSON parser for embedded Malcolm McLean <malcolm.arthur.mclean@gmail.com> - 2019-05-17 08:05 -0700
      Re: Static JSON parser for embedded "Rick C. Hodgin" <rick.c.hodgin@gmail.com> - 2019-05-17 08:15 -0700
      Re: Static JSON parser for embedded Lew Pitcher <lew.pitcher@digitalfreehold.ca> - 2019-05-17 11:53 -0400
  Re: Static JSON parser for embedded Ian Collins <ian-news@hotmail.com> - 2019-05-18 15:33 +1200
    Re: Static JSON parser for embedded Bart <bc@freeuk.com> - 2019-05-18 11:57 +0100
  Re: Static JSON parser for embedded Clifford Heath <no.spam@please.net> - 2019-05-18 17:14 +1000
    Re: Static JSON parser for embedded pozz <pozzugno@gmail.com> - 2019-05-18 16:03 +0200
      Re: Static JSON parser for embedded Clifford Heath <no.spam@please.net> - 2019-05-19 09:12 +1000
        Re: Static JSON parser for embedded Niklas Holsti <niklas.holsti@tidorum.invalid> - 2019-05-19 17:13 +0300
          Re: Static JSON parser for embedded Clifford Heath <no.spam@please.net> - 2019-05-20 09:25 +1000
        Re: Static JSON parser for embedded pozz <pozzugno@gmail.com> - 2019-06-06 11:12 +0200
          Re: Static JSON parser for embedded Clifford Heath <no.spam@please.net> - 2019-06-07 09:11 +1000
            Re: Static JSON parser for embedded pozz <pozzugno@gmail.com> - 2019-06-07 08:59 +0200
              Re: Static JSON parser for embedded Clifford Heath <no.spam@please.net> - 2019-06-12 09:41 +1000
    Re: Static JSON parser for embedded pozz <pozzugno@gmail.com> - 2019-06-05 16:44 +0200
      Re: Static JSON parser for embedded Clifford Heath <no.spam@please.net> - 2019-06-06 09:27 +1000
        Re: Static JSON parser for embedded pozz <pozzugno@gmail.com> - 2019-06-06 09:50 +0200
          Re: Static JSON parser for embedded Clifford Heath <no.spam@please.net> - 2019-06-07 09:08 +1000
      Re: Static JSON parser for embedded Clifford Heath <no.spam@please.net> - 2019-06-06 09:48 +1000
        Re: Static JSON parser for embedded pozz <pozzugno@gmail.com> - 2019-06-06 09:54 +0200

csiph-web