Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > comp.arch.embedded > #27818
| From | pozz <pozzugno@gmail.com> |
|---|---|
| Newsgroups | comp.arch.embedded |
| Subject | Re: Serial EEPROM or Serial Flash? |
| Date | 2018-06-15 17:20 +0200 |
| Organization | A noiseless patient Spider |
| Message-ID | <pg0lfs$es1$1@dont-email.me> (permalink) |
| References | (1 earlier) <pfth8a$vpp$1@dont-email.me> <pfvqei$9t7$1@dont-email.me> <pg00mh$cmj$1@dont-email.me> <pg0ds2$sal$1@dont-email.me> <pg0i0n$nlr$1@dont-email.me> |
Il 15/06/2018 16:21, David Brown ha scritto:
> On 15/06/18 15:10, pozz wrote:
>> Il 15/06/2018 11:25, David Brown ha scritto:
>>> On 15/06/18 09:38, pozz wrote:
>>>> Il 14/06/2018 12:49, David Brown ha scritto:
>>>>> On 14/06/18 12:20, pozz wrote:
>>>>>> I need to save some data on a non-volatile memory. They are some
>>>>>> parameters that the user could change infrequently, for example 10
>>>>>> times
>>>>>> per day at a maximum. In the range 100-2kB.
>>>>>>
> <snip>
>>>>>
>>>>> Multiple copies (at least 2) are key, along with timestamps or counters
>>>>
>>>> Two should be sufficient to prevent corruption caused by interruption
>>>> during the writing of a block of data.
>>>> Maybe three (or more) are needed to face memory *physical* corruption
>>>> (maybe for many writings).
>>>> I think I can ignore this event in my actual project.
>>>>
>>>
>>> You use more for wear levelling. Often your flash chip is /way/ bigger
>>> than you need - perhaps by a factor of 1000 simply because that's what
>>> you have on stock, or that's the cheapest device in the package you
>>> want. Spread your writes over 1000 blocks instead of 2, and you have
>>> 500 times the endurance. Use a good checksum and accept that sometimes
>>> a block will be worn out and move onto the next one, and you have
>>> perhaps a million times the endurance (because most blocks last a lot
>>> longer than the guaranteed minimum).
>>
>> So the writing process should be:
>>
>> 1. write the data at block i+1 (where i is the block of the current
>> data in RAM)
>> 2. read back the block i+1 and check if checksum is ok
>> 3. if ok, writing process is finished
>> 4. if not, go to block i+2 and start again from 1.
>>
>
> Yes.
>
> Then comes step 5 (for flash with separate erasing) :
>
> 5. If you have written block x, check if block x+1 (modulo the size of
> the device) is erased. If not, then erase it ready for the next write.
>
> Note that it does not matter if the erase block size is bigger than the
> program block size - if it is, then your "erase block x+1" command will
> cover the next few program blocks.
>
>>>>> and checksums.
>>>>
>>>> This is interesting. Why do I need a checksum? My approach is to use
>>>> only a magic number plus a counter... and two memory areas.
>>>> At first startup magic number isn't found on any areas, so the device
>>>> starts with default and write data on Area1 (magic/0, where 0 is the
>>>> counter).
>>>
>>> You use checksums to ensure that you haven't had a power-out or reset in
>>> the middle of writing,
>>
>> Only for this thing, you can write the counter as the last byte. If the
>> writing is interrupted in the middle, counter hasn't written yet, so the
>> block is not valid (because considered too old or empty).
>
> Nope. You can't rely on that, unless you are absolutely sure that you
> have a simple, old-fashioned E²PROM device. Many devices are /not/ like
> that, even if they provide an interface that matches it logically.
>
> A common structure for a modern device is to have 32-byte pages as a
> compromise between density, cost, and flexibility. (Bigger pages are
> more efficient in device area and cost.) When you send a command to
> write a byte, the device reads the old 32-byte page into ram, erases the
> old page, updates the ram with the new data, the writes the whole 32
> byte page back in.
>
> The write process is done by a loop that writes all the data, reads it
> back at a low voltage to see if it has stuck, and writes again as needed
> until the data looks good. Then it writes again a few times for safety
> - either a fixed number, or a percentage of the number of writes taken.
>
> So it is /entirely/ possible for an interrupted write to give you a
> valid counter, but invalid data. It is also entirely possible to get
> some bits of the counter as valid while others are still erased (giving
> ones on most devices).
>
> And that is just for simple devices that don't do any fancy wear
> levelling, packing, garbage collection, etc.
>
>
>>
>>> and that the flash has not worn out.
>>>
>>>>
>>>> When the configuration is changed, Area2 is written with magic/1, being
>>>> careful to save magic/1 only at the end of area writing.
>>>>
>>>> At startup magics and counters from both area are loaded, and one area
>>>> is chosen (magic should be valid and counter should be the maximum).
>>>>
>>>> I think this approach works, even when the area writing is interrupted
>>>> at the middle.
>>>>
>>>> Why do I need checksum? The only thing that comes in mind is to prevent
>>>> writing errors: for example, I want to write 0x00 but the value really
>>>> written is 0x01, maybe for a noise on the serial bus.
>>>>
>>>> To solve this situation, I need checksum... but also I need to re-read
>>>> and re-calculate the checksum at *every* area writing... and start a new
>>>> writing if something was wrong.
>>>>
>>>> Do you have a better strategy?
>>>
>>> You calculate the checksum for a block before writing it, and you check
>>> it when reading it. Simple.
>>
>> Do you calc the checksum of all the data block in RAM, including padding
>> bytes?
>
> Yes, of course. The trick is not to have unknown padding bytes. I make
> a point of /never/ having compiler-generated padding in my structs.
>
> So you have something like this:
>
> #define sizeOfRawBlock 32
> #define noOfRawBlocks 4
> #define magicNumber 0x9185be91
> #define dataStructVersionExpected 0x0001
>
> typedef union {
> uint8_t raw8[sizeOfRawBlock * noOfRawBlocks];
> uint16_t raw16[sizeOfRawBlock * noOfRawBlocks / 2];
> struct {
> uint32_t magic;
> uint16_t dataStructVersion;
> uint16_t crc;
> uint32_t count;
>
> // real data
> }
> } nvmData_t;
>
> static_assert(sizeof(nvmData_t) == (sizeofRawBlock * noOfRawBlocks),
> "Check size of nvnData!");
Why raw8[]?
I think you can avoid raw16[] too. If you have the function:
uint16_t calc_crc(const void *data, size_t size);
you can simply call:
nvmData.crc =
calc_crc( ((unsigned char *)&nvmData) + 8, sizeof(nvmData) );
> When you want to store the data, run your crc calculation over the
> raw16[] from index 4 upwards, and set it in the crc field. Set the
> magic field with a different fixed value per program. Use the
> dataStructVersion to track versions of the structure of the data - so
> that updated firmware can recognise old data and quietly update it.
This is interesting. Suppose you have a new struct version and you are
reading an old version from the memory. How do you read it? Do you put
it completely in RAM and after copy it in the final struct making the
changes from one version to the other? In this case you need a double
space in RAM, for old and new version.
Or you declare a "unsigned char reserved_for_future[1024]" big array in
the struct, so the CRC is calculated over the full area?
> Set the count to a new number each time.
>
> Leave plenty of room to add new fields to your nvm structure.
>
>
>>
>>
>>>>>> This means I need to copy&paste an entire block everytime, even for a
>>>>>> single byte change. And this is similar to Flash approach, where I
>>>>>> *need* a sector erase before changing a single byte.
>>>>>
>>>>> Yes.
>>>>>
>>>>> Of course, it is possible to have multiple smaller blocks rather than
>>>>> just one big one.
>>>>
>>>> This is good to reduce writing time of a block of data.
>>>> In my case I have 2kB of data, around 62 32-bytes EEPROM pages, that is
>>>> 5*62=300ms. I don't like to block a task (in a cooperative OS) for so
>>>> long time.
>>>
>>> Your writing should be done in the background. The background writer
>>> task is blocked - the system is not.
>>
>> Are you thinking of a preemption OS? I'm using a cooperative OS.
>>
>
> It doesn't matter. In a pre-emptive OS, your background writer is:
^^^^^^^^^^^
You mean cooperative.
>
> static enum { idle, writing, checking, erasing } writerState;
>
> void doWriter(void) {
> switch (writerState) {
> case idle :
> if (writeTriggered) {
> writeTriggered = false;
> startWriting();
> writerState = writing;
> }
> break;
> case writing :
> bool stillWorking = pollNVMdevice();
> if (stillWorking) return;
> startChecking();
> writerState = checking;
> break;
> case checking :
> bool stillWorking = pollNVMdevice();
> if (stillWorking) return;
> startErasingNextBlock();
> writerState = erasing;
> break;
> case erasing :
> bool stillWorking = pollNVMdevice();
> if (stillWorking) return;
> writerState = idle;
> break;
> }
> }
>
> There - half your program is done :-)
>
> Cooperative multitasking does not mean busy waiting for long-running
> tasks. It means using state machines, interrupt-driven SPI/I²C drivers,
> polling, etc., so that you can mix fast and slow tasks.
Yes, ok.
Back to comp.arch.embedded | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread
Serial EEPROM or Serial Flash? pozz <pozzugno@gmail.com> - 2018-06-14 12:20 +0200
Re: Serial EEPROM or Serial Flash? David Brown <david.brown@hesbynett.no> - 2018-06-14 12:49 +0200
Re: Serial EEPROM or Serial Flash? pozz <pozzugno@gmail.com> - 2018-06-15 09:38 +0200
Re: Serial EEPROM or Serial Flash? Jack <jack4747@gmail.com> - 2018-06-15 00:54 -0700
Re: Serial EEPROM or Serial Flash? Reinhardt Behm <rbehm@hushmail.com> - 2018-06-15 16:26 +0800
Re: Serial EEPROM or Serial Flash? Jack <jack4747@gmail.com> - 2018-06-15 01:31 -0700
Re: Serial EEPROM or Serial Flash? pozz <pozzugno@gmail.com> - 2018-06-15 10:50 +0200
Re: Serial EEPROM or Serial Flash? Reinhardt Behm <rbehm@hushmail.com> - 2018-06-15 17:03 +0800
Re: Serial EEPROM or Serial Flash? pozz <pozzugno@gmail.com> - 2018-06-15 14:59 +0200
Re: Serial EEPROM or Serial Flash? David Brown <david.brown@hesbynett.no> - 2018-06-15 15:51 +0200
Re: Serial EEPROM or Serial Flash? pozz <pozzugno@gmail.com> - 2018-06-15 10:47 +0200
Re: Serial EEPROM or Serial Flash? Jack <jack4747@gmail.com> - 2018-06-15 02:20 -0700
Re: Serial EEPROM or Serial Flash? Stef <stef33d@yahooI-N-V-A-L-I-D.com.invalid> - 2018-06-15 13:41 +0200
Re: Serial EEPROM or Serial Flash? pozz <pozzugno@gmail.com> - 2018-06-15 15:01 +0200
Re: Serial EEPROM or Serial Flash? David Brown <david.brown@hesbynett.no> - 2018-06-15 11:25 +0200
Re: Serial EEPROM or Serial Flash? pozz <pozzugno@gmail.com> - 2018-06-15 15:10 +0200
Re: Serial EEPROM or Serial Flash? Jack <jack4747@gmail.com> - 2018-06-15 06:51 -0700
Re: Serial EEPROM or Serial Flash? David Brown <david.brown@hesbynett.no> - 2018-06-15 16:21 +0200
Re: Serial EEPROM or Serial Flash? pozz <pozzugno@gmail.com> - 2018-06-15 17:20 +0200
Re: Serial EEPROM or Serial Flash? Richard Damon <Richard@Damon-Family.org> - 2018-06-15 13:11 -0400
Re: Serial EEPROM or Serial Flash? pozz <pozzugno@gmail.com> - 2018-06-15 22:54 +0200
Re: Serial EEPROM or Serial Flash? Richard Damon <Richard@Damon-Family.org> - 2018-06-15 18:25 -0400
Re: Serial EEPROM or Serial Flash? pozz <pozzugno@gmail.com> - 2018-06-16 08:26 +0200
Re: Serial EEPROM or Serial Flash? Richard Damon <Richard@Damon-Family.org> - 2018-06-16 09:20 -0400
Re: Serial EEPROM or Serial Flash? pozz <pozzugno@gmail.com> - 2018-06-16 19:38 +0200
Re: Serial EEPROM or Serial Flash? Richard Damon <Richard@Damon-Family.org> - 2018-06-16 16:41 -0400
Re: Serial EEPROM or Serial Flash? pozz <pozzugno@gmail.com> - 2018-06-16 23:05 +0200
Re: Serial EEPROM or Serial Flash? David Brown <david.brown@hesbynett.no> - 2018-06-17 18:41 +0200
Re: Serial EEPROM or Serial Flash? pozz <pozzugno@gmail.com> - 2018-06-16 20:12 +0200
Re: Serial EEPROM or Serial Flash? David Brown <david.brown@hesbynett.no> - 2018-06-17 19:27 +0200
Re: Serial EEPROM or Serial Flash? pozz <pozzugno@gmail.com> - 2018-06-19 17:40 +0200
Re: Serial EEPROM or Serial Flash? Richard Damon <Richard@Damon-Family.org> - 2018-06-17 17:05 -0400
Re: Serial EEPROM or Serial Flash? pozz <pozzugno@gmail.com> - 2018-06-19 17:44 +0200
Re: Serial EEPROM or Serial Flash? Richard Damon <Richard@Damon-Family.org> - 2018-06-19 23:14 -0400
Re: Serial EEPROM or Serial Flash? pozz <pozzugno@gmail.com> - 2018-06-20 11:03 +0200
Re: Serial EEPROM or Serial Flash? Richard Damon <Richard@Damon-Family.org> - 2018-06-20 08:03 -0400
Re: Serial EEPROM or Serial Flash? Michael Karas <mkaras@carousel-design.com> - 2018-06-23 09:43 -0700
csiph-web