Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.user > #177165 > unrolled thread

"No space left on device" error, but df shows plenty of space

Back to article view | Back to linux.debian.user

Contents

  "No space left on device" error, but df shows plenty of space Kynn Jones <kynnjo@gmail.com> - 2017-01-31 14:10 +0100
    Re: "No space left on device" error, but df shows plenty of space Andy Smith <andy@strugglers.net> - 2017-01-31 14:30 +0100
    Re: "No space left on device" error, but df shows plenty of space Eduardo M KALINOWSKI <eduardo@kalinowski.com.br> - 2017-01-31 14:30 +0100
    Re: "No space left on device" error, but df shows plenty of space Brad Rogers <brad@fineby.me.uk> - 2017-01-31 14:30 +0100
      Re: "No space left on device" error, but df shows plenty of space Kynn Jones <kynnjo@gmail.com> - 2017-02-01 00:50 +0100
        Re: "No space left on device" error, but df shows plenty of space Andy Smith <andy@strugglers.net> - 2017-02-01 13:30 +0100
          Re: "No space left on device" error, but df shows plenty of space Kynn Jones <kynnjo@gmail.com> - 2017-02-01 16:50 +0100
            Re: "No space left on device" error, but df shows plenty of space Andy Smith <andy@strugglers.net> - 2017-02-02 11:10 +0100
              Re: "No space left on device" error, but df shows plenty of space Eike Lantzsch <zp6cge@gmx.net> - 2017-02-02 13:30 +0100
    Re: "No space left on device" error, but df shows plenty of space Tony Baldwin <tonybaldwin@gmx.com> - 2017-01-31 14:30 +0100

#177165 — "No space left on device" error, but df shows plenty of space

[Multipart message — attachments visible in raw view] — view raw

Hi everyone!

When I tried to install `ntp`, there was a "No space left on device" at the
end of `apt-get`'s output, but `df` shows no shortage of space:

    # df
    Filesystem     1K-blocks      Used Available Use% Mounted on
    /dev/sda5      381993164 206410036 156155956  57% /
    udev               10240         0     10240   0% /dev
    tmpfs            6588000      9676   6578324   1% /run
    tmpfs           16469996       108  16469888   1% /dev/shm
    tmpfs               5120         4      5116   1% /run/lock
    tmpfs           16469996         0  16469996   0% /sys/fs/cgroup
    tmpfs            3294000         8   3293992   1% /run/user/1000

According to `df`, the maximum percent usage is 57%.

So I tried `dpgk-reconfigure ntp` (as root), and got the same error:

    # dpkg-reconfigure ntp
    Error: No space left on device
    update-rc.d: warning: start and stop actions are no longer supported;
falling back to defaults
    Error: No space left on device

How can I troubleshoot this problem further?

    # uname -ar
    Linux luna 3.16.0-4-amd64 #1 SMP Debian 3.16.7-ckt20-1+deb8u3
(2016-01-17) x86_64 GNU/Linux

Thanks in advance!

kj

[toc] | [next] | [standalone]

#177169

Hi Kynn,

On Tue, Jan 31, 2017 at 08:01:00AM -0500, Kynn Jones wrote:
>     Filesystem     1K-blocks      Used Available Use% Mounted on
>     /dev/sda5      381993164 206410036 156155956  57% /

[…]

>     # dpkg-reconfigure ntp
>     Error: No space left on device
> 
> How can I troubleshoot this problem further?

Does /dev/sda5 have a btrfs filesystem on it? Type "mount" and look
for the "type" part. If so, these problems are common with btrfs;
see btrfs wiki and/or linux-btrfs mailing list:

    https://btrfs.wiki.kernel.org/index.php/FAQ#Help.21_Btrfs_claims_I.27m_out_of_space.2C_but_it_looks_like_I_should_have_lots_left.21

If not, could be things like deleted files that are still open (see
"lsof" output, look for "(deleted)". Or maybe ran out of inodes. See
"df -i" out to check that.

Cheers,
Andy

-- 
https://bitfolk.com/ -- No-nonsense VPS hosting

[toc] | [prev] | [next] | [standalone]

#177170

On Ter, 31 Jan 2017, Kynn Jones wrote:
> Hi everyone!
>
> When I tried to install `ntp`, there was a "No space left on device" at the
> end of `apt-get`'s output, but `df` shows no shortage of space:

Try 'df -i', you're probably out of inodes.
-- 
Eduardo M KALINOWSKI
eduardo@kalinowski.com.br

[toc] | [prev] | [next] | [standalone]

#177171

[Multipart message — attachments visible in raw view] — view raw

On Tue, 31 Jan 2017 08:01:00 -0500
Kynn Jones <kynnjo@gmail.com> wrote:

Hello Kynn,

>When I tried to install `ntp`, there was a "No space left on device" at
>the end of `apt-get`'s output, but `df` shows no shortage of space:

I've no idea whether this is relevant to your case, but it's possible to
be unable to use disk space if there are insufficient inodes left to
store the relevant directory/file info.  AIUI, a "No space..." error is
the result.

-- 
 Regards  _
         / )           "The blindingly obvious is
        / _)rad        never immediately apparent"
Just stop and take a second
U & Ur Hand - P!nk

[toc] | [prev] | [next] | [standalone]

#177197

[Multipart message — attachments visible in raw view] — view raw

Hi everyone!  Thank you for your suggestions.  They taught some new tricks.

As it happened, I had to reboot the machine for an unrelated (?) reason
(monitor would not wake from sleep).

After the machine rebooted, I was able to run `dpkg-reconfigure ntp`
without error.

Unfortunately, I'll never know what the problem was.

Thank you all for your help.

kj


On Tue, Jan 31, 2017 at 8:28 AM, Brad Rogers <brad@fineby.me.uk> wrote:

> On Tue, 31 Jan 2017 08:01:00 -0500
> Kynn Jones <kynnjo@gmail.com> wrote:
>
> Hello Kynn,
>
> >When I tried to install `ntp`, there was a "No space left on device" at
> >the end of `apt-get`'s output, but `df` shows no shortage of space:
>
> I've no idea whether this is relevant to your case, but it's possible to
> be unable to use disk space if there are insufficient inodes left to
> store the relevant directory/file info.  AIUI, a "No space..." error is
> the result.
>
> --
>  Regards  _
>          / )           "The blindingly obvious is
>         / _)rad        never immediately apparent"
> Just stop and take a second
> U & Ur Hand - P!nk
>

[toc] | [prev] | [next] | [standalone]

#177227

Hi Kynn,

On Tue, Jan 31, 2017 at 06:42:39PM -0500, Kynn Jones wrote:
> After the machine rebooted, I was able to run `dpkg-reconfigure ntp`
> without error.
> 
> Unfortunately, I'll never know what the problem was.

Do you use btrfs?

What does "df -i" report now, after your reboot when things are
working?

Cheers,
Andy

-- 
https://bitfolk.com/ -- No-nonsense VPS hosting

[toc] | [prev] | [next] | [standalone]

#177235

[Multipart message — attachments visible in raw view] — view raw

On Wed, Feb 1, 2017 at 7:24 AM, Andy Smith <andy@strugglers.net> wrote:

> Hi Kynn,
>
> On Tue, Jan 31, 2017 at 06:42:39PM -0500, Kynn Jones wrote:
> > After the machine rebooted, I was able to run `dpkg-reconfigure ntp`
> > without error.
> >
> > Unfortunately, I'll never know what the problem was.
>
> Do you use btrfs?
>

Not that I'm aware of.  (FWIW, if I run `mount | grep -i btrfs` (as root),
I get no output.)

What does "df -i" report now, after your reboot when things are
> working?
>

    # df -i
    Filesystem       Inodes   IUsed    IFree IUse% Mounted on
    /dev/sda5      24264704 1464023 22800681    7% /
    udev            4115384     445  4114939    1% /dev
    tmpfs           4117499     735  4116764    1% /run
    tmpfs           4117499      14  4117485    1% /dev/shm
    tmpfs           4117499       5  4117494    1% /run/lock
    tmpfs           4117499      13  4117486    1% /sys/fs/cgroup
    tmpfs           4117499      10  4117489    1% /run/user/1000

Unfortunately, I don't have the output of `df -i` before I rebooted.

Best,

kj

[toc] | [prev] | [next] | [standalone]

#177281

Hi Kynn,

On Wed, Feb 01, 2017 at 10:43:37AM -0500, Kynn Jones wrote:
> On Wed, Feb 1, 2017 at 7:24 AM, Andy Smith <andy@strugglers.net> wrote:
> > On Tue, Jan 31, 2017 at 06:42:39PM -0500, Kynn Jones wrote:
> > > Unfortunately, I'll never know what the problem was.
> >
> > Do you use btrfs?
> 
> Not that I'm aware of.  (FWIW, if I run `mount | grep -i btrfs` (as root),
> I get no output.)

Okay, so not a btrfs issue.

> > What does "df -i" report now, after your reboot when things are
> > working?
> 
>     # df -i
>     Filesystem       Inodes   IUsed    IFree IUse% Mounted on
>     /dev/sda5      24264704 1464023 22800681    7% /

…and that doesn't show as being anywhere near full, so it most
likely wasn't that either.

Well I suppose it could have been processes holding open deleted
files, though it would have to have been some really big files in
that case, as your filesystem didn't show as being anywhere near
full.

Cheers,
Andy

-- 
https://bitfolk.com/ -- No-nonsense VPS hosting

[toc] | [prev] | [next] | [standalone]

#177286

On Thursday, 2 February 2017 10:05:58 PYST Andy Smith wrote:
> Hi Kynn,
> 
> On Wed, Feb 01, 2017 at 10:43:37AM -0500, Kynn Jones wrote:
> > On Wed, Feb 1, 2017 at 7:24 AM, Andy Smith <andy@strugglers.net> wrote:
> > > On Tue, Jan 31, 2017 at 06:42:39PM -0500, Kynn Jones wrote:
> > > > Unfortunately, I'll never know what the problem was.
> > > 
> > > Do you use btrfs?
> > 
> > Not that I'm aware of.  (FWIW, if I run `mount | grep -i btrfs` (as root),
> > I get no output.)
> 
> Okay, so not a btrfs issue.
> 
> > > What does "df -i" report now, after your reboot when things are
> > > working?
> > > 
> >     # df -i
> >     Filesystem       Inodes   IUsed    IFree IUse% Mounted on
> >     /dev/sda5      24264704 1464023 22800681    7% /
> 
> …and that doesn't show as being anywhere near full, so it most
> likely wasn't that either.
> 
> Well I suppose it could have been processes holding open deleted
> files, though it would have to have been some really big files in
> that case, as your filesystem didn't show as being anywhere near
> full.
> 
> Cheers,
> Andy

let me guess:

facts:
/var is on the same partition as /
there is plenty of space and there are free i-nodes but anyway you got error 
apt-get resulted in "no space left on device"
you rebooted and the error disappeared

=>

/ was mounted as r/o for some reason (unclean filesystem e.g.)

[toc] | [prev] | [next] | [standalone]

#177172

On 01/31/2017 08:01 AM, Kynn Jones wrote:
> Hi everyone!
>
> When I tried to install `ntp`, there was a "No space left on device" at
> the end of `apt-get`'s output, but `df` shows no shortage of space:
>
>     # df
>     Filesystem     1K-blocks      Used Available Use% Mounted on
>     /dev/sda5      381993164 206410036 156155956  57% /
>     udev               10240         0     10240   0% /dev
>     tmpfs            6588000      9676   6578324   1% /run
>     tmpfs           16469996       108  16469888   1% /dev/shm
>     tmpfs               5120         4      5116   1% /run/lock
>     tmpfs           16469996         0  16469996   0% /sys/fs/cgroup
>     tmpfs            3294000         8   3293992   1% /run/user/1000
>
> According to `df`, the maximum percent usage is 57%.
>
> So I tried `dpgk-reconfigure ntp` (as root), and got the same error:
>
>     # dpkg-reconfigure ntp
>     Error: No space left on device
>     update-rc.d: warning: start and stop actions are no longer
> supported; falling back to defaults
>     Error: No space left on device
>
> How can I troubleshoot this problem further?

try doing
$ df -i
This shows inodes.
I once had a problem where I could no longer write to a disk, because 
all of the inodes were occupied due to not properly rotating some 
log(s). Let us know if that reveals anything seemingly significant.

Tony

-- 
http://tonybaldwin.me
all tony, all the time

[toc] | [prev] | [standalone]

Back to top | Article view | linux.debian.user

csiph-web