Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.user > #194597 > unrolled thread

SSD's and many edits of a single file

Back to article view | Back to linux.debian.user

Contents

  SSD's and many edits of a single file Gene Heskett <gheskett@shentel.net> - 2018-04-09 03:20 +0200
    Re: SSD's and many edits of a single file Abdullah Ramazanoglu <ar018@yahoo.com> - 2018-04-09 03:30 +0200
    Re: SSD's and many edits of a single file Stefan Monnier <monnier@iro.umontreal.ca> - 2018-04-09 04:20 +0200
      Re: SSD's and many edits of a single file Gene Heskett <gheskett@shentel.net> - 2018-04-09 04:50 +0200
        Re: SSD's and many edits of a single file Stefan Monnier <monnier@iro.umontreal.ca> - 2018-04-09 05:20 +0200
          Re: SSD's and many edits of a single file Gene Heskett <gheskett@shentel.net> - 2018-04-09 12:00 +0200
            Re: SSD's and many edits of a single file <tomas@tuxteam.de> - 2018-04-09 13:30 +0200
            Re: SSD's and many edits of a single file Stefan Monnier <monnier@iro.umontreal.ca> - 2018-04-09 15:10 +0200
              Re: SSD's and many edits of a single file Gene Heskett <gheskett@shentel.net> - 2018-04-09 15:40 +0200
                Re: SSD's and many edits of a single file Greg Wooledge <wooledg@eeg.ccf.org> - 2018-04-09 15:50 +0200
                  Re: SSD's and many edits of a single file Stefan Monnier <monnier@iro.umontreal.ca> - 2018-04-09 16:20 +0200
            Re: SSD's and many edits of a single file rhkramer@gmail.com - 2018-04-09 15:50 +0200
              Re: SSD's and many edits of a single file Greg Wooledge <wooledg@eeg.ccf.org> - 2018-04-09 16:00 +0200
                Re: SSD's and many edits of a single file Gene Heskett <gheskett@shentel.net> - 2018-04-09 16:40 +0200
                  Re: SSD's and many edits of a single file Greg Wooledge <wooledg@eeg.ccf.org> - 2018-04-09 16:40 +0200
                  Re: SSD's and many edits of a single file David Christensen <dpchrist@holgerdanske.com> - 2018-04-10 03:30 +0200
                    Re: SSD's and many edits of a single file Gene Heskett <gheskett@shentel.net> - 2018-04-10 06:00 +0200
                      Re: SSD's and many edits of a single file David Christensen <dpchrist@holgerdanske.com> - 2018-04-11 04:30 +0200
                        Re: SSD's and many edits of a single file Gene Heskett <gheskett@shentel.net> - 2018-04-11 09:30 +0200
                          Re: SSD's and many edits of a single file David Christensen <dpchrist@holgerdanske.com> - 2018-04-12 05:50 +0200
                          Re: SSD's and many edits of a single file rhkramer@gmail.com - 2018-04-12 13:50 +0200
                            Re: SSD's and many edits of a single file Gene Heskett <gheskett@shentel.net> - 2018-04-12 14:50 +0200
                              Re: SSD's and many edits of a single file rhkramer@gmail.com - 2018-04-12 16:20 +0200
                              Re: SSD's and many edits of a single file songbird <songbird@anthive.com> - 2018-04-12 17:00 +0200
                                Re: SSD's and many edits of a single file Gene Heskett <gheskett@shentel.net> - 2018-04-13 03:30 +0200
                    Re: SSD's and many edits of a single file <tomas@tuxteam.de> - 2018-04-10 10:00 +0200
                      Re: SSD's and many edits of a single file Gene Heskett <gheskett@shentel.net> - 2018-04-10 13:00 +0200
                        Re: SSD's and many edits of a single file <tomas@tuxteam.de> - 2018-04-10 13:10 +0200
                          Re: SSD's and many edits of a single file Gene Heskett <gheskett@shentel.net> - 2018-04-10 14:20 +0200
                            Re: SSD's and many edits of a single file <tomas@tuxteam.de> - 2018-04-10 15:20 +0200
                            Re: SSD's and many edits of a single file "Thomas Schmitt" <scdbackup@gmx.net> - 2018-04-10 15:20 +0200
                              Re: SSD's and many edits of a single file Stefan Monnier <monnier@iro.umontreal.ca> - 2018-04-10 16:00 +0200
                                Re: SSD's and many edits of a single file "Thomas Schmitt" <scdbackup@gmx.net> - 2018-04-10 16:30 +0200
              Re: SSD's and many edits of a single file Gene Heskett <gheskett@shentel.net> - 2018-04-09 16:20 +0200
                Re: SSD's and many edits of a single file <tomas@tuxteam.de> - 2018-04-09 18:30 +0200
              Re: SSD's and many edits of a single file Abdullah Ramazanoglu <ar018@yahoo.com> - 2018-04-09 20:10 +0200
            Re: SSD's and many edits of a single file Celejar <celejar@gmail.com> - 2018-04-10 16:50 +0200
    Re: SSD's and many edits of a single file songbird <songbird@anthive.com> - 2018-04-09 14:10 +0200
      Re: SSD's and many edits of a single file Gene Heskett <gheskett@shentel.net> - 2018-04-09 16:20 +0200
        Re: SSD's and many edits of a single file songbird <songbird@anthive.com> - 2018-04-09 20:20 +0200

Page 1 of 2  [1] 2  Next page →

#194597 — SSD's and many edits of a single file

Greetings folks;

Updodate Wheezy, realtime kernel because machine is running linuxcnc.

Editor is geany and file is left open in the editor, and reloaded into 
linuxcnc as changes are made to the file, and saved but not closed.

And eventually the updates made to the file are not actually saved, geany 
does not report an error and does say success by changing the filename 
font from red which tells me its been modified, back to black in the 
window top border, but does not actually update the file on the disk, 
and the only fix seems to be a reboot.

Is this a known fault with some SSD's? Or???

-- 
Cheers, Gene Heskett
--
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
Genes Web page <http://geneslinuxbox.net:6309/gene>

[toc] | [next] | [standalone]

#194598

On Sun, 8 Apr 2018 21:10:20 -0400 Gene Heskett said:

> Is this a known fault with some SSD's? Or???

~$ sync

FWIW I add the line below in my /etc/crontab as a standard post-install
procedure:

* * * * * root /bin/sync

The kernel and journalling file system should take care of that
already, so periodic syncing might seem as superfluous. Nevertheless,
this increased the robustness of my filesystems by an order of
magnitude, *perceivedly*. :)

Regards
-- 
Abdullah Ramazanoglu

[toc] | [prev] | [next] | [standalone]

#194599

> And eventually the updates made to the file are not actually saved,

Can you be more precise than "eventually"?
More importantly: what makes you think they're not actually saved?


        Stefan

[toc] | [prev] | [next] | [standalone]

#194601

On Sunday 08 April 2018 22:17:33 Stefan Monnier wrote:

> > And eventually the updates made to the file are not actually saved,
>
> Can you be more precise than "eventually"?

Probably 100+ edits and saves over 4 or 5 hours.

> More importantly: what makes you think they're not actually saved?
>
>
>         Stefan

Going to another shell and cat'ing the file shows the old contents. 
Linuxcnc also has a code display that tracks the execution, and new 
contents is not displayed in that window either.  Takes several hours to 
start failing though. I went out to install some keying screws in some 
brass slugs intended to hold threading taps, started at about 13:30, and 
it was doing the silent failure at 17:30.  Machine uptime was about 47 
days, and linuxcnc had not been shut down and restarted in a week.  I 
have SSR's to kill motor power when disabling linuxcnc, but leave it 
running so that I can restart the motor power without having to go thru 
a lengthy re-homing operation.

This seems to be related more to how busy the editor is rather than the 
uptime of linuxcnc.

Now I've rebooted, and the code won't get much more TLC until the next 
project. So I don't expect any more trouble until I start on the next 
phase of this project, probably at least a week down the log since I 
have about 50 more of this part to finish, taxes to do by the 15nth, and 
a safety rail beside a ramp I just built into the front deck so I can 
wheel the missus around in a wheel chair. We've both reached the golden 
years, and passed them by by 2 decades.

Yeah, I was a geek before the word was invented. :)

-- 
Cheers, Gene Heskett
--
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
Genes Web page <http://geneslinuxbox.net:6309/gene>

[toc] | [prev] | [next] | [standalone]

#194602

>> > And eventually the updates made to the file are not actually saved,
>> Can you be more precise than "eventually"?
> Probably 100+ edits and saves over 4 or 5 hours.
>> More importantly: what makes you think they're not actually saved?
> Going to another shell and cat'ing the file shows the old contents. 

So the actual disk isn't to blame (cat'ing a file that was just saved
won't look at the disk anyway).

Sounds like a bug in your text editor.


        Stefan

[toc] | [prev] | [next] | [standalone]

#194607

On Sunday 08 April 2018 23:13:28 Stefan Monnier wrote:

> >> > And eventually the updates made to the file are not actually
> >> > saved,
> >>
> >> Can you be more precise than "eventually"?
> >
> > Probably 100+ edits and saves over 4 or 5 hours.
> >
> >> More importantly: what makes you think they're not actually saved?
> >
> > Going to another shell and cat'ing the file shows the old contents.
>
> So the actual disk isn't to blame (cat'ing a file that was just saved
> won't look at the disk anyway).

Whats it look at 5+ minutes later?

The file had not been updated yet an hour later. Ext4's journal in my 
understanding has a maximum of 5 minutes to sync?  Its now around 10 
hours later and has not yet been updated with the last two lines I'd 
added.

> Sounds like a bug in your text editor.

Maybe, its geany. Lots of people seem to like gedit, but its saves are 
the cause of important configuration files being written back to disk 
with the line order totally trashed, as if you had thrown it on the 
floor in 512 byte pieces, then picked it back up and reassembled it in 
random order. Then try to recover a 1400 LOC configuration file...

Thats happened using gedit enough, on several different machines here 
that its been expunged from my systems, all of them.

Nano would do what I need, but its half a screen jump scroll leads to 
mistakes on my part because its too easy to loose track of the cursor 
when it does scroll.

What else is a good editor? And lets not start yet another vim vs emacs 
war. kwrite and kate come to mind. I'm running Trinity Desktop, maybe I 
should check them out. Whatever, needs to be able to have several files 
open so I can just click back and forth on the tabs as I usually 
separate the files into what uses which cutting tool.

Advice checked out.

Thanks Stefan.

>
>         Stefan



-- 
Cheers, Gene Heskett
--
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
Genes Web page <http://geneslinuxbox.net:6309/gene>

[toc] | [prev] | [next] | [standalone]

#194615

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Mon, Apr 09, 2018 at 05:53:18AM -0400, Gene Heskett wrote:
> On Sunday 08 April 2018 23:13:28 Stefan Monnier wrote:
> 
> > >> > And eventually the updates made to the file are not actually
> > >> > saved,
> > >>
> > >> Can you be more precise than "eventually"?
> > >
> > > Probably 100+ edits and saves over 4 or 5 hours.
> > >
> > >> More importantly: what makes you think they're not actually saved?
> > >
> > > Going to another shell and cat'ing the file shows the old contents.
> >
> > So the actual disk isn't to blame (cat'ing a file that was just saved
> > won't look at the disk anyway).
> 
> Whats it look at 5+ minutes later?
> 
> The file had not been updated yet an hour later. Ext4's journal in my 
> understanding has a maximum of 5 minutes to sync?  Its now around 10 
> hours later and has not yet been updated with the last two lines I'd 
> added.

If the file system stays mounted, this all is irrelevant: it's the
OS's job to present all applications a consistent view of the
file system. Once the writing application (Geany, or whatever) has
committed its write the modification should be visible, whether
the OS has committed its buffers to the storage or not.

> > Sounds like a bug in your text editor.

Yep, that would be my hunch too.

Cheers
- -- t
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)

iEYEARECAAYFAlrLTkEACgkQBcgs9XrR2kYxKQCdEe0tomEdxB/RbDSbhhlItssk
GPUAnjaWd8Uid0K4/iJsWE9D+hmapd9s
=TeCh
-----END PGP SIGNATURE-----

[toc] | [prev] | [next] | [standalone]

#194618

>> So the actual disk isn't to blame (cat'ing a file that was just saved
>> won't look at the disk anyway).
> Whats it look at 5+ minutes later?

Same difference: if `cat` can't see it, then the change hasn't been
received by the OS at all (even less so by the underlying disk).

> The file had not been updated yet an hour later. Ext4's journal in my 
> understanding has a maximum of 5 minutes to sync?  Its now around 10 
> hours later and has not yet been updated with the last two lines I'd 
> added.

The changes you see in `cat`s output will be transferred to disk
eventually, yes.  But those that aren't visible to `cat` just don't
exist, as far as the OS is concerned.

> What else is a good editor?

Is that a trick question?


        Stefan

[toc] | [prev] | [next] | [standalone]

#194623

On Monday 09 April 2018 09:02:55 Stefan Monnier wrote:

> >> So the actual disk isn't to blame (cat'ing a file that was just
> >> saved won't look at the disk anyway).
> >
> > Whats it look at 5+ minutes later?
>
> Same difference: if `cat` can't see it, then the change hasn't been
> received by the OS at all (even less so by the underlying disk).
>
> > The file had not been updated yet an hour later. Ext4's journal in
> > my understanding has a maximum of 5 minutes to sync?  Its now around
> > 10 hours later and has not yet been updated with the last two lines
> > I'd added.
>
> The changes you see in `cat`s output will be transferred to disk
> eventually, yes.  But those that aren't visible to `cat` just don't
> exist, as far as the OS is concerned.
>
> > What else is a good editor?
>
> Is that a trick question?
>
Twasn't intended to be.

However I did find (I actually read its man page!) an alias setting that 
makes nano do line at a time scrolling and installed it, so I'll try it 
when I get back to that project later today. nano hasn't made a mistake 
yet that I didn't do typeing in the wrong place. A big, rapidly blinking 
BLOCK cursor would help these old eyes find it a lot easier. But in 20 
years thats fallen out of style, dammit.
>
>         Stefan



-- 
Cheers, Gene Heskett
--
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
Genes Web page <http://geneslinuxbox.net:6309/gene>

[toc] | [prev] | [next] | [standalone]

#194625

On Mon, Apr 09, 2018 at 09:32:14AM -0400, Gene Heskett wrote:
> A big, rapidly blinking 
> BLOCK cursor would help these old eyes find it a lot easier. But in 20 
> years thats fallen out of style, dammit.

The cursor is a function of the terminal, not of the text editor
running inside the terminal.

rxvt-unicode (the terminal I use on Debian) has a solid block cursor by
default.  You can make it blink by running "rxvt -bc", or by setting the
"cursorBlink" resource in your X resources.  I wouldn't call it *rapidly*
blinking (looks to be about 1 Hz), but it's blinking.  (I've also decided
that I don't care for it, and won't be using this option, but hey,
that's what *options* are for.)

Other terminals may or may not have similar options.

[toc] | [prev] | [next] | [standalone]

#194630

>> A big, rapidly blinking BLOCK cursor would help these old eyes find
>> it a lot easier.  But in 20  years thats fallen out of style, dammit.

100% idle state is important to reduce power consumption, so blinking
while otherwise idle is to be avoided in general, yes.  But it's OK to
blink when there's other activity.  E.g. in Emacs, after execution of
a command by default the cursor blinks (by default) for upto
`blink-cursor-blinks` after which it stays solid.

> The cursor is a function of the terminal, not of the text editor
> running inside the terminal.

That's only for those editor that run within a text-terminal (and even
in those cases, the editor can decide to control the cursor's blinking.
Emacs currently doesn't, but I use a local patch which does let it do
so, so it obeys `blink-cursor-blinks`, `blink-cursor-interval`, and
`blink-cursor-delay` rather than).


        Stefan

[toc] | [prev] | [next] | [standalone]

#194624

On Monday, April 09, 2018 05:53:18 AM Gene Heskett wrote:
> What else is a good editor? And lets not start yet another vim vs emacs
> war. kwrite and kate come to mind. I'm running Trinity Desktop, maybe I
> should check them out. Whatever, needs to be able to have several files
> open so I can just click back and forth on the tabs as I usually
> separate the files into what uses which cutting tool.

I use kate and kwrite (on Wheezy) for some of my important things (my askSam 
workalike mashup), nedit and some others for other purposes.

kate and kwrite both allow multiple files to be open, but, instead of tabs, you 
can (with an option--maybe on the menu) have an additional panel on the left 
hand side of the screen showing you all the files open in that instance and 
allowing you to click on any one of them to switch to that file.

To your original problem, have you tried going to a command line and throwing 
in a couple =sync=s?  I would try that, maybe after saving in your editor, and 
again maybe after open and / or saving in the cnc program.

[toc] | [prev] | [next] | [standalone]

#194626

On Mon, Apr 09, 2018 at 09:46:07AM -0400, rhkramer@gmail.com wrote:
> To your original problem, have you tried going to a command line and throwing 
> in a couple =sync=s?  I would try that, maybe after saving in your editor, and 
> again maybe after open and / or saving in the cnc program.

As others have explained, the OS (Linux) keeps a cache of file contents
that have been written by applications, but not yet committed to
permanent storage.  If you "save" from within the text editor, then the
saved contents should be immediately visible to other processes reading
the file, regardless of whether it has been synced to disk.  They'll
simply get the cached version.

[toc] | [prev] | [next] | [standalone]

#194634

On Monday 09 April 2018 09:51:37 Greg Wooledge wrote:

> On Mon, Apr 09, 2018 at 09:46:07AM -0400, rhkramer@gmail.com wrote:
> > To your original problem, have you tried going to a command line and
> > throwing in a couple =sync=s?  I would try that, maybe after saving
> > in your editor, and again maybe after open and / or saving in the
> > cnc program.
>
> As others have explained, the OS (Linux) keeps a cache of file
> contents that have been written by applications, but not yet committed
> to permanent storage.  If you "save" from within the text editor, then
> the saved contents should be immediately visible to other processes
> reading the file, regardless of whether it has been synced to disk. 
> They'll simply get the cached version.

Which is not happening after several hours and a hundred or more edits. 
Which is why its so intermittent.

-- 
Cheers, Gene Heskett
--
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
Genes Web page <http://geneslinuxbox.net:6309/gene>

[toc] | [prev] | [next] | [standalone]

#194635

On Mon, Apr 09, 2018 at 10:30:58AM -0400, Gene Heskett wrote:
> On Monday 09 April 2018 09:51:37 Greg Wooledge wrote:
> > As others have explained, the OS (Linux) keeps a cache of file
> > contents that have been written by applications, but not yet committed
> > to permanent storage.  If you "save" from within the text editor, then
> > the saved contents should be immediately visible to other processes
> > reading the file, regardless of whether it has been synced to disk. 
> > They'll simply get the cached version.
> 
> Which is not happening after several hours and a hundred or more edits. 
> Which is why its so intermittent.

This is why some people are guessing that the problem lies in the
text editor -- it hasn't *actually* saved the content (via kernel
write() calls) yet.

Of course there are many other possible explanations.  You might have
thought you saved, but you actually didn't (classic PEBKAC).  You might
be catting the wrong file, different from the one that's being edited.
The editor might have been invoked to edit a temporary file within some
kind of version control environment, with your changes not actually
being submitted to the VCS until the editor terminates.

It's unlikely to be an issue with the actual storage hardware.

[toc] | [prev] | [next] | [standalone]

#194658

On 04/09/18 07:30, Gene Heskett wrote:
> On Monday 09 April 2018 09:51:37 Greg Wooledge wrote:
> 
>> On Mon, Apr 09, 2018 at 09:46:07AM -0400, rhkramer@gmail.com
>> wrote:
>>> To your original problem, have you tried going to a command line
>>> and throwing in a couple =sync=s?  I would try that, maybe after
>>> saving in your editor, and again maybe after open and / or saving
>>> in the cnc program.
>> 
>> As others have explained, the OS (Linux) keeps a cache of file 
>> contents that have been written by applications, but not yet
>> committed to permanent storage.  If you "save" from within the text
>> editor, then the saved contents should be immediately visible to
>> other processes reading the file, regardless of whether it has been
>> synced to disk. They'll simply get the cached version.
> 
> Which is not happening after several hours and a hundred or more
> edits. Which is why its so intermittent.

On 04/09/18 02:53, Gene Heskett wrote:
> Lots of people seem to like gedit, but its saves are the cause of
> important configuration files being written back to disk with the
> line order totally trashed, as if you had thrown it on the floor in
> 512 byte pieces, then picked it back up and reassembled it in random
> order. Then try to recover a 1400 LOC configuration file...
> 
> Thats happened using gedit enough, on several different machines ...

One editor failing would also make me suspect the editor.


But two failing editors would make me suspect some common factor, such 
as a shared library and/or the kernel.


Have you tested your hardware -- power supply, memory, and SSD?  Be sure 
to test the SSD before you touch any cables.  If it fails, re-seat 
and/or replace cables and test again.


Have you tested your SSD hypothesis?  Say, by removing the SSD, cloning 
the SSD to another device, installing the other device, and then testing 
for the bug while running the other device?


Can you reproduce the bug on hardware with ECC memory with no memory 
errors reported?


Can you reproduce the bug on RAID1 with no RAID errors reported?


If you want to make it possible for others to find the bug, I would suggest:

1.  Build a machine with the minimum hardware, software, and 
configuration required to demonstrate the bug.  Document everything.

2.  Write a program or script that invokes the bug every time it runs on 
the demonstration machine.  If a program, include a Makefile.

3.  Post the demonstrator document, program, script, Makefile, etc., to 
the relevant support communities.


David

[toc] | [prev] | [next] | [standalone]

#194663

On Monday 09 April 2018 21:25:34 David Christensen wrote:

> On 04/09/18 07:30, Gene Heskett wrote:
> > On Monday 09 April 2018 09:51:37 Greg Wooledge wrote:
> >> On Mon, Apr 09, 2018 at 09:46:07AM -0400, rhkramer@gmail.com
> >>
> >> wrote:
> >>> To your original problem, have you tried going to a command line
> >>> and throwing in a couple =sync=s?  I would try that, maybe after
> >>> saving in your editor, and again maybe after open and / or saving
> >>> in the cnc program.
> >>
> >> As others have explained, the OS (Linux) keeps a cache of file
> >> contents that have been written by applications, but not yet
> >> committed to permanent storage.  If you "save" from within the text
> >> editor, then the saved contents should be immediately visible to
> >> other processes reading the file, regardless of whether it has been
> >> synced to disk. They'll simply get the cached version.
> >
> > Which is not happening after several hours and a hundred or more
> > edits. Which is why its so intermittent.
>
> On 04/09/18 02:53, Gene Heskett wrote:
> > Lots of people seem to like gedit, but its saves are the cause of
> > important configuration files being written back to disk with the
> > line order totally trashed, as if you had thrown it on the floor in
> > 512 byte pieces, then picked it back up and reassembled it in random
> > order. Then try to recover a 1400 LOC configuration file...
> >
> > Thats happened using gedit enough, on several different machines ...
>
> One editor failing would also make me suspect the editor.
>
>
> But two failing editors would make me suspect some common factor, such
> as a shared library and/or the kernel.
>
I'd buy that if the failures were similar.  They are not.
>
> Have you tested your hardware -- power supply, memory, and SSD?  Be
> sure to test the SSD before you touch any cables.  If it fails,
> re-seat and/or replace cables and test again.

I saw this once when the drive was a 2T piece of spinning rust. Way 
overkill for that machine, but a 1T is being used for amanda, here on 
this machine and its around 90% right now.  So at some 
point "/amandatapes" is going to find another terabyte.
>
> Have you tested your SSD hypothesis?  Say, by removing the SSD,
> cloning the SSD to another device, installing the other device, and
> then testing for the bug while running the other device?

I have a twin to that one, but only one handy sata port. So it would be 
an extended project to clone it to the other drive.  And since I saw it 
on spinning rust too, I'm not inclined to sharpen that finger just yet.

That machine has also ran about 7 cycles of memtest86 with no errors 
since I noticed it the first time.

> Can you reproduce the bug on hardware with ECC memory with no memory
> errors reported?

I don't have anything with ECC memory in it. That usually runs the mobo 
cost up quite a ways.

> Can you reproduce the bug on RAID1 with no RAID errors reported?

That would also demand another handy sata port.  I'll see if I can cobble 
something up if and when it does it again. Possible if it has a back 
panel sata connector I think.
>
> If you want to make it possible for others to find the bug, I would
> suggest:
>
> 1.  Build a machine with the minimum hardware, software, and
> configuration required to demonstrate the bug.  Document everything.
>
> 2.  Write a program or script that invokes the bug every time it runs
> on the demonstration machine.  If a program, include a Makefile.
>
> 3.  Post the demonstrator document, program, script, Makefile, etc.,
> to the relevant support communities.
>
> David

Thanks David.

-- 
Cheers, Gene Heskett
--
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
Genes Web page <http://geneslinuxbox.net:6309/gene>

[toc] | [prev] | [next] | [standalone]

#194706

On 04/09/18 20:58, Gene Heskett wrote:
> On Monday 09 April 2018 21:25:34 David Christensen wrote:
> 
>> On 04/09/18 07:30, Gene Heskett wrote:
>>> On Monday 09 April 2018 09:51:37 Greg Wooledge wrote:
>>>> On Mon, Apr 09, 2018 at 09:46:07AM -0400, rhkramer@gmail.com
>>>>
>>>> wrote:
>>>>> To your original problem, have you tried going to a command line
>>>>> and throwing in a couple =sync=s?  I would try that, maybe after
>>>>> saving in your editor, and again maybe after open and / or saving
>>>>> in the cnc program.
>>>>
>>>> As others have explained, the OS (Linux) keeps a cache of file
>>>> contents that have been written by applications, but not yet
>>>> committed to permanent storage.  If you "save" from within the text
>>>> editor, then the saved contents should be immediately visible to
>>>> other processes reading the file, regardless of whether it has been
>>>> synced to disk. They'll simply get the cached version.
>>>
>>> Which is not happening after several hours and a hundred or more
>>> edits. Which is why its so intermittent.
>>
>> On 04/09/18 02:53, Gene Heskett wrote:
>>> Lots of people seem to like gedit, but its saves are the cause of
>>> important configuration files being written back to disk with the
>>> line order totally trashed, as if you had thrown it on the floor in
>>> 512 byte pieces, then picked it back up and reassembled it in random
>>> order. Then try to recover a 1400 LOC configuration file...
>>>
>>> Thats happened using gedit enough, on several different machines ...
>>
>> One editor failing would also make me suspect the editor.
>>
>>
>> But two failing editors would make me suspect some common factor, such
>> as a shared library and/or the kernel.
>>
> I'd buy that if the failures were similar.  They are not.

They both seem to involve getting blocks from an app to memory and 
blocks from memory to disk (and/or disk to memory).  They could be 
variations on a theme.


Can you reproduce the bugs on a machine with straight-up Debian (e.g. no 
linuxcnc)?


As a safety net, I would suggest a versioning file system (e.g. similar 
to the versioning feature of VMS).  STFW is pretty thin, but copyfs 
might be worth a try:

https://packages.debian.org/search?keywords=copyfs&searchon=names&suite=all&section=all


David

[toc] | [prev] | [next] | [standalone]

#194708

On Tuesday 10 April 2018 22:24:39 David Christensen wrote:

> On 04/09/18 20:58, Gene Heskett wrote:
> > On Monday 09 April 2018 21:25:34 David Christensen wrote:
> >> On 04/09/18 07:30, Gene Heskett wrote:
> >>> On Monday 09 April 2018 09:51:37 Greg Wooledge wrote:
> >>>> On Mon, Apr 09, 2018 at 09:46:07AM -0400, rhkramer@gmail.com
> >>>>
> >>>> wrote:
> >>>>> To your original problem, have you tried going to a command line
> >>>>> and throwing in a couple =sync=s?  I would try that, maybe after
> >>>>> saving in your editor, and again maybe after open and / or
> >>>>> saving in the cnc program.
> >>>>
> >>>> As others have explained, the OS (Linux) keeps a cache of file
> >>>> contents that have been written by applications, but not yet
> >>>> committed to permanent storage.  If you "save" from within the
> >>>> text editor, then the saved contents should be immediately
> >>>> visible to other processes reading the file, regardless of
> >>>> whether it has been synced to disk. They'll simply get the cached
> >>>> version.
> >>>
> >>> Which is not happening after several hours and a hundred or more
> >>> edits. Which is why its so intermittent.
> >>
> >> On 04/09/18 02:53, Gene Heskett wrote:
> >>> Lots of people seem to like gedit, but its saves are the cause of
> >>> important configuration files being written back to disk with the
> >>> line order totally trashed, as if you had thrown it on the floor
> >>> in 512 byte pieces, then picked it back up and reassembled it in
> >>> random order. Then try to recover a 1400 LOC configuration file...
> >>>
> >>> Thats happened using gedit enough, on several different machines
> >>> ...
> >>
> >> One editor failing would also make me suspect the editor.
> >>
> >>
> >> But two failing editors would make me suspect some common factor,
> >> such as a shared library and/or the kernel.
> >
> > I'd buy that if the failures were similar.  They are not.
>
> They both seem to involve getting blocks from an app to memory and
> blocks from memory to disk (and/or disk to memory).  They could be
> variations on a theme.
>
>
> Can you reproduce the bugs on a machine with straight-up Debian (e.g.
> no linuxcnc)?
>
That to me would be rather pointless. The only places where I be doing 
wholesale edits would be for linuxcnc use.
>
> As a safety net, I would suggest a versioning file system (e.g.
> similar to the versioning feature of VMS).  STFW is pretty thin, but
> copyfs might be worth a try:

This "fuse" would be on top of ext4? Or on an SSD that was a copy of this 
one but using fuse as opposed to ext4?

What seems to be lost on you folks not running realtime patched kernels, 
is that when linuxcnc is running, it has total control over the 
hardware, and that linux becomes a client of the realtime system, 
getting what cpu time is left after linuxcnc has finished what it has to 
do to meet the timing constraints needed to run the machine and meet the 
sub-micron positioning accuracy required.

Not your fault of course because to you its effectively real time if you 
do not see a lag between touching a key and the response on screen. But 
a 20ns response time is about 3 orders of magnitude faster than that 
which is realtime to you.

> https://packages.debian.org/search?keywords=copyfs&searchon=names&suit
>e=all&section=all

This link shows 2 versions, but only the older, 1.4 might be usable since 
the newest system I'm running is jessie on an r-pi-3b. Stretch so far is 
not stable on arm64, aka a rock64 yet. Even running an amanda client on 
the arm64 can take it down. Downgrade its install to a jessie derived 
version and its dead stable. And amanda is dead stable on 6 of the 7 
systems with a net cable plugged into it here. 

But this link does not contain a sales pitch describing what it can do 
and why it should be used, a common fault, I think because git doesn't 
seem to have a "box" category to put this stuff into for ready 
presentation when browsing around looking for a solution to a problem. 
This lack of a descriptive sales pitch "box" for this info is, IMNSHO, a 
major failing of the linux genre. There may well be a magic twanger or 8 
out there, but searching the repo's is less than useful in many cases.

All this of course has drifted off-topic, debian moves a bit slow and 
does not actually support the arm64 yet. That I understand is a lack of 
manpower problem. Were I 40 years younger, I might be able to help a wee 
bit. But I'm not, and having had a pulmonary embolism that damned near 
put a ~30~ on my story 3 years back, I can feel the wet ram damage that 
caused.

I think that by changing how I work, so that an edit is done with a fresh 
invocation of the editor each time, that I will not see this problem 
again, so we should really put this thread to bed.

"Magic Twanger" and Froggie indeed, that kiddie tv show is now 68 years 
old. I guess that does date me. 99% of the readers of this message have 
no clue where that phrase came from.

> David 

Thanks David.

-- 
Cheers, Gene Heskett
--
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
Genes Web page <http://geneslinuxbox.net:6309/gene>

[toc] | [prev] | [next] | [standalone]

#194715

On 04/11/18 00:27, Gene Heskett wrote:
> On Tuesday 10 April 2018 22:24:39 David Christensen wrote:
>> Can you reproduce the bugs on a machine with straight-up Debian (e.g.
>> no linuxcnc)?
>>
> That to me would be rather pointless. The only places where I be doing
> wholesale edits would be for linuxcnc use.

If geany and gedit fail on a machine with Debian plus LinuxCNC, but work 
correctly on a machine with Debian alone, that would indicate the bug is 
related to LinuxCNC.


>> As a safety net, I would suggest a versioning file system (e.g.
>> similar to the versioning feature of VMS).  STFW is pretty thin, but
>> copyfs might be worth a try:
> 
> This "fuse" would be on top of ext4? Or on an SSD that was a copy of this
> one but using fuse as opposed to ext4?

Here is information about FUSE:

https://en.wikipedia.org/wiki/Filesystem_in_Userspace


See link below for information about copyfs.


> What seems to be lost on you folks not running realtime patched kernels,
> is that when linuxcnc is running, it has total control over the
> hardware, and that linux becomes a client of the realtime system,
> getting what cpu time is left after linuxcnc has finished what it has to
> do to meet the timing constraints needed to run the machine and meet the
> sub-micron positioning accuracy required.
> 
> Not your fault of course because to you its effectively real time if you
> do not see a lag between touching a key and the response on screen. But
> a 20ns response time is about 3 orders of magnitude faster than that
> which is realtime to you.

I used to work as a embedded systems software engineer back in the 
dot-com days.  My niche was controlling electro-mechanical systems using 
8051, 68xx, and 68xxx micro-controllers.  I have written C and assembly 
language drivers and programs to meet soft and hard real-time 
requirements, including stepper motor sequencing.  I had heard of hard 
real-time kernels that used Linux as their idle loop.  I have since used 
Linux real-time patches for my audio and music hobbies.


>> https://packages.debian.org/search?keywords=copyfs&searchon=names&suit
>> e=all&section=all
> 
> This link shows 2 versions, but only the older, 1.4 might be usable since
> the newest system I'm running is jessie on an r-pi-3b. Stretch so far is
> not stable on arm64, aka a rock64 yet. Even running an amanda client on
> the arm64 can take it down. Downgrade its install to a jessie derived
> version and its dead stable. And amanda is dead stable on 6 of the 7
> systems with a net cable plugged into it here.
> 
> But this link does not contain a sales pitch describing what it can do
> and why it should be used, 

Here is information about copyfs:

https://boklm.eu/copyfs/


> I think that by changing how I work, so that an edit is done with a fresh
> invocation of the editor each time, that I will not see this problem
> again, so we should really put this thread to bed.

I used VMS systems back in the 1980's, and VMS's versioning file system 
was a killer feature that I would love to have on my Linux machines today:

https://mike632t.wordpress.com/2016/03/15/vms-file-operations-for-beginners/


Unfortunately, the semantics of copyfs appear to differ.


> "Magic Twanger" and Froggie indeed, that kiddie tv show is now 68 years
> old. I guess that does date me. 99% of the readers of this message have
> no clue where that phrase came from.

STFW I see Andy's Gang (but it's before my time):

https://www.youtube.com/watch?v=G6a3fck0NBI


> Thanks David.

Thanks for the interesting problem to think about.  :-)


David

[toc] | [prev] | [next] | [standalone]

Page 1 of 2  [1] 2  Next page →

Back to top | Article view | linux.debian.user

csiph-web