Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > linux.kernel > #1599648
| From | Matt Turner <mattst88@gmail.com> |
|---|---|
| Newsgroups | linux.kernel |
| Subject | Re: NFS corruption, fixed by echo 1 > /proc/sys/vm/drop_caches -- next debugging steps? |
| Date | 2017-03-13 18:20 +0100 |
| Message-ID | <tkBNv-3I1-15@gated-at.bofh.it> (permalink) |
| References | <tknhv-1pV-9@gated-at.bofh.it> <tkuM2-717-15@gated-at.bofh.it> |
| Organization | linux.* mail to news gateway |
[Multipart message — attachments visible in raw view] - view raw
On Mon, Mar 13, 2017 at 2:47 AM, James Hogan <james.hogan@imgtec.com> wrote: > On Sun, Mar 12, 2017 at 06:43:47PM -0700, Matt Turner wrote: >> On a Broadcom BCM91250a MIPS system I can reliably trigger NFS >> corruption on the first file read. >> >> To demonstrate, I downloaded five identical copies of the gcc-5.4.0 >> source tarball. On the NFS server, they hash to the same value: >> >> server distfiles # md5sum gcc-5.4.0.tar.bz2* >> 4c626ac2a83ef30dfb9260e6f59c2b30 gcc-5.4.0.tar.bz2 >> 4c626ac2a83ef30dfb9260e6f59c2b30 gcc-5.4.0.tar.bz2.1 >> 4c626ac2a83ef30dfb9260e6f59c2b30 gcc-5.4.0.tar.bz2.2 >> 4c626ac2a83ef30dfb9260e6f59c2b30 gcc-5.4.0.tar.bz2.3 >> 4c626ac2a83ef30dfb9260e6f59c2b30 gcc-5.4.0.tar.bz2.4 >> >> On the MIPS system (the NFS client): >> >> bcm91250a-le distfiles # md5sum gcc-5.4.0.tar.bz2.2 >> 35346975989954df8a8db2b034da610d gcc-5.4.0.tar.bz2.2 >> bcm91250a-le distfiles # md5sum gcc-5.4.0.tar.bz2* >> 4c626ac2a83ef30dfb9260e6f59c2b30 gcc-5.4.0.tar.bz2 >> 4c626ac2a83ef30dfb9260e6f59c2b30 gcc-5.4.0.tar.bz2.1 >> 35346975989954df8a8db2b034da610d gcc-5.4.0.tar.bz2.2 >> 4c626ac2a83ef30dfb9260e6f59c2b30 gcc-5.4.0.tar.bz2.3 >> 4c626ac2a83ef30dfb9260e6f59c2b30 gcc-5.4.0.tar.bz2.4 >> >> The first file read will contain some corruption, and it is persistent until... >> >> bcm91250a-le distfiles # echo 1 > /proc/sys/vm/drop_caches >> bcm91250a-le distfiles # md5sum gcc-5.4.0.tar.bz2* >> 4c626ac2a83ef30dfb9260e6f59c2b30 gcc-5.4.0.tar.bz2 >> 4c626ac2a83ef30dfb9260e6f59c2b30 gcc-5.4.0.tar.bz2.1 >> 4c626ac2a83ef30dfb9260e6f59c2b30 gcc-5.4.0.tar.bz2.2 >> 4c626ac2a83ef30dfb9260e6f59c2b30 gcc-5.4.0.tar.bz2.3 >> 4c626ac2a83ef30dfb9260e6f59c2b30 gcc-5.4.0.tar.bz2.4 >> >> the caches are dropped, at which point it reads back properly. >> >> Note that the corruption is different across reboots, both in the size >> of the corruption and the location. I saw 1900~ and 1400~ byte >> sequences corrupted on separate occasions, which don't correspond to >> the system's 16kB page size. >> >> I've tested kernels from v3.19 to 4.11-rc1+ (master branch from >> today). All exhibit this behavior with differing frequencies. Earlier >> kernels seem to reproduce the issue less often, while more recent >> kernels reliably exhibit the problem every boot. >> >> How can I further debug this? > > It smells a bit like a DMA / caching issue. > > Can you provide a full kernel log. That might provide some information > about caching that might be relevant (e.g. does dcache have aliases?). Thanks for the reply. Please find attached dmesg from a clean boot (which reproduced the problem).
Back to linux.kernel | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread
NFS corruption, fixed by echo 1 > /proc/sys/vm/drop_caches -- next debugging steps? Matt Turner <mattst88@gmail.com> - 2017-03-13 02:50 +0100
Re: NFS corruption, fixed by echo 1 > /proc/sys/vm/drop_caches -- next debugging steps? James Hogan <james.hogan@imgtec.com> - 2017-03-13 10:50 +0100
Re: NFS corruption, fixed by echo 1 > /proc/sys/vm/drop_caches -- next debugging steps? Matt Turner <mattst88@gmail.com> - 2017-03-13 18:20 +0100
Re: NFS corruption, fixed by echo 1 > /proc/sys/vm/drop_caches -- next debugging steps? Ralf Baechle <ralf@linux-mips.org> - 2017-03-15 11:00 +0100
csiph-web