Path: csiph.com!usenet.pasdenom.info!weretis.net!feeder4.news.weretis.net!rt.uk.eu.org!newsfeed.xs4all.nl!newsfeed2.news.xs4all.nl!xs4all!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail
Return-Path: <rosuav@gmail.com>
X-Original-To: python-list@python.org
Delivered-To: python-list@mail.python.org
X-Spam-Status: OK 0.040
X-Spam-Evidence: '*H*': 0.92; '*S*': 0.00; 'subject:: [': 0.04; 'parameters': 0.04; 'root': 0.05; 'linux,': 0.07; 'subject:skip:s 10': 0.07; 'subject: [': 0.09; 'url:blog': 0.10; 'all...': 0.16; 'caching': 0.16; 'from:addr:rosuav': 0.16; 'from:name:chris angelico': 0.16; 'great.)': 0.16; 'lie,': 0.16; 'simple.': 0.16; 'worse.': 0.16; 'worst': 0.16; 'followed': 0.16; 'sat,': 0.16; 'wrote:': 0.18; 'subject:] ': 0.20; 'written': 0.21; 'example': 0.22; '(in': 0.22; 'documented': 0.24; 'of.': 0.24; "i've": 0.25; 'mention': 0.26; 'this:': 0.26; 'header:In-Reply-To:1': 0.27; 'especially': 0.30; 'message-id:@mail.gmail.com': 0.30; 'url:wiki': 0.31; '(on': 0.31; '+0100,': 0.31; '13,': 0.31; "d'aprano": 0.31; 'steven': 0.31; 'url:wikipedia': 0.31; 'probably': 0.32; 'quite': 0.32; 'noticed': 0.34; 'subject: (': 0.35; 'received:209.85': 0.35; 'something': 0.35; 'but': 0.35; 'received:google.com': 0.35; 'version': 0.36; 'disk': 0.36; 'url:org': 0.36; 'reports': 0.37; 'received:209': 0.37; 'skip:o 20': 0.38; 'to:addr:python-list': 0.38; 'pm,': 0.38; 'anything': 0.39; 'obtain': 0.39; 'to:addr:python.org': 0.39; 'according': 0.40; 'even': 0.60; 'easy': 0.60; 'most': 0.60; 'new': 0.61; 'matter': 0.61; 'times': 0.62; "you've": 0.63; 'details': 0.65; 'minutes': 0.67; 'close': 0.67; 'nobody': 0.68; 'respect': 0.70; 'safe': 0.72; 'apart': 0.72; 'brand': 0.72; 'physical': 0.72; 'url:2011': 0.75; '100%': 0.77; 'article,': 0.84; 'lying': 0.84; 'disks,': 0.91; 'light.': 0.93; 'reliable,': 0.93; '2013': 0.98
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:in-reply-to:references:date:message-id :subject:from:to:content-type; bh=7A27LYWSVHxJkboBGHkjWecMedXPXAEDt28i0r72k54=; b=QzOCPlToKKqzkDNdOFDdzev+/IV4YMBKrqhslxupGyUuUS2nqpGSlJmQhLAdekCGu3 qpK1UPMg0SVLBvcLVTfl722bEh1riwNZkeOyBAPiFliE3CyIHK2MIheCiwfPRaifaazA DkLbgqj1yRe6TLyGc5u8Vot1rOLyf1SphD4Arvp+3DApSPCqpHuyo2HioNoQ6WgiSQhR fyOmy46YLO81AAxm9SI9DRguHvH6rM2tJMG7iLR/0sC/puJ5RVu7EqTjTDHocB09IPRF N4pAjxTmK5ts29fUcW91Dss6eFlU2HKwm7Lqn3ngrSLNNTcWVylYfeDq+HxVuHzcpXX6 cMnA==
MIME-Version: 1.0
X-Received: by 10.52.231.231 with SMTP id tj7mr1085860vdc.111.1365824637289; Fri, 12 Apr 2013 20:43:57 -0700 (PDT)
In-Reply-To: <5168ce64$0$29977$c3e8da3$5496439d@news.astraweb.com>
References: <e8878747-fd9d-41d2-9ce1-096a65590f84@googlegroups.com> <asof5aFknkrU1@mid.individual.net> <5363f8ea-f2a6-4a08-a216-18c6666ba698@googlegroups.com> <asoiq9Flkh1U1@mid.individual.net> <51674ffc$0$29977$c3e8da3$5496439d@news.astraweb.com> <pan.2013.04.13.02.33.28.725000@nowhere.com> <5168ce64$0$29977$c3e8da3$5496439d@news.astraweb.com>
Date: Sat, 13 Apr 2013 13:43:57 +1000
Subject: Re: [OT] Lying hard drives [was Re: shutil.copyfile is incomplete (truncated)]
From: Chris Angelico <rosuav@gmail.com>
To: python-list@python.org
Content-Type: text/plain; charset=ISO-8859-1
X-BeenThere: python-list@python.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe: <http://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive: <http://mail.python.org/pipermail/python-list/>
List-Post: <mailto:python-list@python.org>
List-Help: <mailto:python-list-request@python.org?subject=help>
List-Subscribe: <http://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
Newsgroups: comp.lang.python
Message-ID: <mailman.544.1365824646.3114.python-list@python.org>
Lines: 42
NNTP-Posting-Host: 2001:888:2000:d::a6
X-Trace: 1365824646 news.xs4all.nl 2635 [2001:888:2000:d::a6]:34115
X-Complaints-To: abuse@xs4all.nl
Xref: csiph.com comp.lang.python:43499

On Sat, Apr 13, 2013 at 1:17 PM, Steven D'Aprano
<steve+comp.lang.python@pearwood.info> wrote:
> On Sat, 13 Apr 2013 03:33:29 +0100, Nobody wrote:
>> If you want to wait for the data written to be written to the physical
>> disk (in order to obtain specific behaviour with respect to an unclean
>> shutdown), use f.flush() followed by os.fsync(f.fileno()).
>
> If only it were that simple. It has been documented that some disks will
> lie, even when told to sync. When I say "some", I mean *most*. There's
> probably nothing you can do about it, apart from not using that model or
> brand of disk, so you have to just live with the risk.

It's often close to that simple. With most hard disks, you can make
them 100% reliable, but you may have to check some disk parameters (on
Linux, that's just a matter of writing to something in /proc
somewhere, don't remember the details but it's easy to check). The
worst offenders I've met are SSDs...

> USB sticks are especially nasty. I've got quite a few USB thumb drives
> where the "write" light keeps flickering for anything up to five minutes
> after the OS reports that the drive has been unmounted and is safe to
> unplug. I corrupted the data on these quite a few times until I noticed
> the light. And let's not even mention the drives that have no light at
> all...

... but you've met worse.

> But my favourite example of lying hard drives of all time is this:
>
> http://blog.jitbit.com/2011/04/chinese-magic-drive.html
>
> I want one of those!

Awesome! It's the new version of DoubleSpace / DriveSpace!

http://en.wikipedia.org/wiki/DriveSpace

(And its problems, according to that Wikipedia article, actually had
the same root cause - write caching that the user wasn't aware of.
Great.)

ChrisA