Re: What is your strategy to save non volatile data?

From	pozz <pozzugno@gmail.com>
Newsgroups	comp.arch.embedded
Subject	Re: What is your strategy to save non volatile data?
Date	2016-08-12 00:16 +0200
Organization	A noiseless patient Spider
Message-ID	<noitgh$8fa$1@dont-email.me> (permalink)
References	<nohs7p$ff3$1@dont-email.me> <noi142$ih$1@dont-email.me>

Show all headers | View raw

Il 11/08/2016 16:12, David Brown ha scritto:
 > On 11/08/16 14:48, pozz wrote:
 >> Many times I work on embedded projects where it is required to save some
 >> data in a non volatile way.  Usualy I use external (to the main
 >> microcontroller) serial/parallel EEPROM or internal Flash memory (with a
 >> software layer for EEPROM emulation).
 >>
 >> However my goal isn't to discuss the hw alternatives, but the data
 >> format and strategy.
 >>
 >> I know there are many serialization data format (XML, JSON, Protocol
 >> Buffer, SQLite), but I don't think they are valid solutions for
 >> medium/low end microcontrollers.  They are complex to implement and they
 >> aren't really necessary.
 >> Data are written and read from the same platform, so the endianess and
 >> byte order are not a problem.
 >>
 >> IMHO directly saving a C structure to the non-volatile memory is the
 >> best method.  When you will load bytes from the non volatile memory, you
 >> will have magically your C structure in RAM filled with the saved 
values.
 >>
 >> Is it the same strategy that you use?
 >
 > As Don says, there is no single best method.  The answer to almost every
 > question in embedded development is "it depends".

Of course, yes.  However if we define and restrict the requirements, the 
number of good strategies is reduced.


 > But for the common case of storing a small set of parameters,

Indeed my intent is that situation.


 > I usually
 > work with a struct definition.  Along with the parameters in question, I
 > include a "magic number" for identification, a "parameter structure
 > version number" (very useful for upgrading the parameter set if a later
 > version of the firmware uses more parameters), an update serial number,
 > a CRC check field, and some space reserved for future additional 
parameters.
 >
 > Typically I have a statically allocated struct in ram, and two copies in
 > eeprom or flash.  On startup, I check both non-volatile copies to see
 > which is the latest version (highest serial number) that has a valid
 > CRC, and copy that into RAM.  If the parameter structure version number
 > does not match the current firmware requirements, I upgrade the 
parameters.
 >
 > When I need to save the parameters, I use the oldest of the two NV sets.
 >  First, I erase it (for eeprom, that's just clearing the magic number).
 >  Then I copy in most of the data and CRC, leaving the magic number 
for last.

Interesting approach.

I couldn't understand what is "magic number" for identification purpose. 
  CRC can be used to check if the saved struct is valid or not, so the 
"magic number" is useless for this goal.

Maybe you have more than one struct saved to the non volatile memory, 
and the "magic number" identifies the type of the struct?
In this case, I think the physical address where you save the struct 
automatically identifies the struct itsel.
Do you have a different "magic number", "serial number", CRC for each 
saved struct?

Or do you have a dynamic list of structs that could be saved at 
different places of non volatile memory (non fixed address)?  In this 
case, you have a file list somewhere, similar to a FAT table.  Maybe you 
have implemented a minimal filesystem where a file is a struct.


 > I have had occasion to use a log structured parameter "file system" in
 > flash, but usually such complications are unnecessary.  And certainly
 > SQLite, or JSON, etc., are normally overkill.
 >

Thread

What is your strategy to save non volatile data? pozz <pozzugno@gmail.com> - 2016-08-11 14:48 +0200
  Re: What is your strategy to save non volatile data? Don Y <blockedofcourse@foo.invalid> - 2016-08-11 06:27 -0700
  Re: What is your strategy to save non volatile data? David Brown <david.brown@hesbynett.no> - 2016-08-11 16:12 +0200
    Re: What is your strategy to save non volatile data? Don Y <blockedofcourse@foo.invalid> - 2016-08-11 07:53 -0700
      Re: What is your strategy to save non volatile data? Don Y <blockedofcourse@foo.invalid> - 2016-08-11 08:27 -0700
        Re: What is your strategy to save non volatile data? bitrex <bitrex@de.lete.earthlink.net> - 2016-08-11 12:11 -0400
          Re: What is your strategy to save non volatile data? bitrex <bitrex@de.lete.earthlink.net> - 2016-08-11 12:12 -0400
            Re: What is your strategy to save non volatile data? Don Y <blockedofcourse@foo.invalid> - 2016-08-11 11:35 -0700
              Re: What is your strategy to save non volatile data? bitrex <bitrex@de.lete.earthlink.net> - 2016-08-11 20:59 -0400
          Re: What is your strategy to save non volatile data? Don Y <blockedofcourse@foo.invalid> - 2016-08-11 11:33 -0700
        Re: What is your strategy to save non volatile data? pozz <pozzugno@gmail.com> - 2016-08-12 00:21 +0200
          Re: What is your strategy to save non volatile data? Don Y <blockedofcourse@foo.invalid> - 2016-08-11 15:40 -0700
      Re: What is your strategy to save non volatile data? David Brown <david.brown@hesbynett.no> - 2016-08-11 23:07 +0200
        Re: What is your strategy to save non volatile data? pozz <pozzugno@gmail.com> - 2016-08-12 00:17 +0200
          Re: What is your strategy to save non volatile data? Don Y <blockedofcourse@foo.invalid> - 2016-08-11 15:30 -0700
            Re: What is your strategy to save non volatile data? pozz <pozzugno@gmail.com> - 2016-08-12 09:02 +0200
              Re: What is your strategy to save non volatile data? Don Y <blockedofcourse@foo.invalid> - 2016-08-12 09:29 -0700
            Re: What is your strategy to save non volatile data? David Brown <david.brown@hesbynett.no> - 2016-08-12 11:54 +0200
              Re: What is your strategy to save non volatile data? John Devereux <john@devereux.me.uk> - 2016-08-12 13:14 +0100
                Re: What is your strategy to save non volatile data? David Brown <david.brown@hesbynett.no> - 2016-08-12 15:22 +0200
              Re: What is your strategy to save non volatile data? Don Y <blockedofcourse@foo.invalid> - 2016-08-12 09:40 -0700
                Re: What is your strategy to save non volatile data? David Brown <david.brown@hesbynett.no> - 2016-08-12 22:40 +0200
                Re: What is your strategy to save non volatile data? Don Y <blockedofcourse@foo.invalid> - 2016-08-13 09:44 -0700
                Re: What is your strategy to save non volatile data? David Brown <david.brown@hesbynett.no> - 2016-08-14 23:01 +0200
          Re: What is your strategy to save non volatile data? David Brown <david.brown@hesbynett.no> - 2016-08-12 09:52 +0200
            Re: What is your strategy to save non volatile data? pozz <pozzugno@gmail.com> - 2016-08-12 16:30 +0200
              Re: What is your strategy to save non volatile data? David Brown <david.brown@hesbynett.no> - 2016-08-12 17:09 +0200
        Re: What is your strategy to save non volatile data? Don Y <blockedofcourse@foo.invalid> - 2016-08-11 15:36 -0700
        Re: What is your strategy to save non volatile data? Paul <paul@pcserviceselectronics.co.uk> - 2016-08-12 03:21 +0100
    Re: What is your strategy to save non volatile data? bitrex <bitrex@de.lete.earthlink.net> - 2016-08-11 12:19 -0400
      Re: What is your strategy to save non volatile data? pozz <pozzugno@gmail.com> - 2016-08-11 23:49 +0200
        Re: What is your strategy to save non volatile data? Don Y <blockedofcourse@foo.invalid> - 2016-08-11 15:37 -0700
    Re: What is your strategy to save non volatile data? John Devereux <john@devereux.me.uk> - 2016-08-11 20:50 +0100
    Re: What is your strategy to save non volatile data? pozz <pozzugno@gmail.com> - 2016-08-12 00:16 +0200
      Re: What is your strategy to save non volatile data? Paul <paul@pcserviceselectronics.co.uk> - 2016-08-12 03:34 +0100
      Re: What is your strategy to save non volatile data? David Brown <david.brown@hesbynett.no> - 2016-08-12 11:59 +0200
  Re: What is your strategy to save non volatile data? Paul Rubin <no.email@nospam.invalid> - 2016-08-11 11:29 -0700
  Re: What is your strategy to save non volatile data? Les Cargill <lcargill99@comcast.com> - 2016-08-12 12:15 -0500
  Re: What is your strategy to save non volatile data? Joe Chisolm <jchisolm6@earthlink.net> - 2016-08-12 12:10 -0500
    Re: What is your strategy to save non volatile data? Paul <paul@pcserviceselectronics.co.uk> - 2016-08-12 22:00 +0100
  Re: What is your strategy to save non volatile data? Tim Wescott <tim@seemywebsite.com> - 2016-08-13 12:06 -0500
    Re: What is your strategy to save non volatile data? pozz <pozzugno@gmail.com> - 2016-08-13 23:42 +0200
      Re: What is your strategy to save non volatile data? Don Y <blockedofcourse@foo.invalid> - 2016-08-13 15:55 -0700
      Re: What is your strategy to save non volatile data? Paul <paul@pcserviceselectronics.co.uk> - 2016-08-14 00:08 +0100
      Re: What is your strategy to save non volatile data? Tim Wescott <tim@seemywebsite.com> - 2016-08-13 19:06 -0500
        Re: What is your strategy to save non volatile data? pozz <pozzugno@gmail.com> - 2016-08-22 09:46 +0200

csiph-web