Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.arch.embedded > #30741

Re: Shutdown behavior and flash storage devices

Show all headers | View raw

On 7/25/2021 2:00 PM, Dimiter_Popoff wrote:
> On 7/25/2021 23:27, Don Y wrote:
>> On 7/25/2021 12:51 PM, Dimiter_Popoff wrote:
>> ......
>>
>>> Of course again
>>> you will be prone to data loss on unexpected power loss but updating
>>> the write cache will be done at (much) longer intervals and in
>>> relatively brief bursts, thus the probability to be hit gets lower.
>>
>> The usual trick is to just have a mirror of the FLASH contents
>> and flush that to the media when you're pretty sure the ship
>> is sinking.
>>
>> But, if this is *large*, then you risk only partial writes before
>> power drops to an unusable/unreliable level.
> 
> Well I think tens of gigabytes for small drives nowadays, what can you
> do. Cacheing it all just is not feasible.

Right.  OTOH, if your software is pushing large objects to long term
storage, it likely WANTS them to be written, there.  Caching is
a more prudent strategy for small things that may change often
(e.g., configuration parameters, "state", etc.)  Pushing them
to the physical store is LOGICALLY correct (from the application's
standpoint) but usually a performance hit and a durability hit
(for media that wear).

> What DPS does is cache directories and CATs on a 4k piece basis
> (regardless of the disk block size, 512 2048, 4k) and maintains a
> bitmap for its dirty regions - so when it updates to the medium
> it writes at least 4k and no more than twice 4095 excessive bytes
> or something like that.

I only have persistent memory in my RDBMS -- everything else is
just RAM (save for small bootloaders).  But, the RDBMS node can
afford to have gobs of disk cache to avoid physical writes.

And, I can use different "tablespaces" (the "disk" on which
particular "tables" reside) to deliberately avoid backing certain
tables (like temporary ones) to durable memory.

>>> The rest can only be got rid of by having large enough caps on
>>> the power supply and an early warning that power has been lost
>>> (usually at the input of the stepdowns and you are running on fumes.
>>
>> I'm not sure that will work with consumer devices.  You have no way of
>> knowing how long it will take to reliably flush ALL of the data
>> out to the physical store.
> 
> Oh with consumer devices you have not built yourself it will probably
> never work, of course. They are unlikely to give you an early warning
> in the first place. If you want to be safer you just use good rotating
> disks.... I think I read somewhere they do their soft shutdown using the
> energy in the rotor (and do that since times immemorial), but I have
> never verified this.

I think the first step is questioning what you *need* to store.

E.g., I trade memory for time with a lot of my algorithms -- pulling
data from disk and using that to build large structures in RAM that
I can use to expedite the algorithms.  It would be *nice* to be
able to save those structures in their "large" state -- to save
the time/effort of having to rebuild them when I restart those
applications.  But, this gets to be costly if you try to adopt
that practice universally (with all applications).

And, of course, making sure you can "recognize" data that is
corrupt...

Back to comp.arch.embedded | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Shutdown behavior and flash storage devices Dave Nadler <drn@nadler.com> - 2021-07-24 11:19 -0400
  Re: Shutdown behavior and flash storage devices Theo <theom+news@chiark.greenend.org.uk> - 2021-07-24 16:37 +0100
  Re: Shutdown behavior and flash storage devices Don Y <blockedofcourse@foo.invalid> - 2021-07-24 18:10 -0700
    Re: Shutdown behavior and flash storage devices Dimiter_Popoff <dp@tgi-sci.com> - 2021-07-25 14:18 +0300
      Re: Shutdown behavior and flash storage devices Don Y <blockedofcourse@foo.invalid> - 2021-07-25 09:20 -0700
    Re: Shutdown behavior and flash storage devices Dave Nadler <drn@nadler.com> - 2021-07-25 08:49 -0400
      Re: Shutdown behavior and flash storage devices Don Y <blockedofcourse@foo.invalid> - 2021-07-25 09:30 -0700
      Re: Shutdown behavior and flash storage devices Dimiter_Popoff <dp@tgi-sci.com> - 2021-07-25 22:51 +0300
        Re: Shutdown behavior and flash storage devices Don Y <blockedofcourse@foo.invalid> - 2021-07-25 13:27 -0700
          Re: Shutdown behavior and flash storage devices Dimiter_Popoff <dp@tgi-sci.com> - 2021-07-26 00:00 +0300
            Re: Shutdown behavior and flash storage devices Don Y <blockedofcourse@foo.invalid> - 2021-07-25 17:35 -0700
  Re: Shutdown behavior and flash storage devices kegs@provalid.com (Kent Dickey) - 2021-07-25 13:53 -0500
    Re: Shutdown behavior and flash storage devices Don Y <blockedofcourse@foo.invalid> - 2021-07-25 13:20 -0700
  Re: Shutdown behavior and flash storage devices Jaded Hobo <badboy@heaven.org> - 2021-07-27 19:44 +0200
    Re: Shutdown behavior and flash storage devices Don Y <blockedofcourse@foo.invalid> - 2021-07-27 11:27 -0700

csiph-web