Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.user > #209500 > unrolled thread

df shows wrong disk size

Back to article view | Back to linux.debian.user

Contents

  df shows wrong disk size Ross Boylan <rossboylan@stanfordalumni.org> - 2019-06-01 19:50 +0200
    Re: df shows wrong disk size Roberto C. Sánchez <roberto@debian.org> - 2019-06-01 20:50 +0200
      Re: df shows wrong disk size Ross Boylan <rossboylan@stanfordalumni.org> - 2019-06-01 23:50 +0200
        Re: df shows wrong disk size Pascal Hambourg <pascal@plouf.fr.eu.org> - 2019-06-02 00:40 +0200
          Re: df shows wrong disk size Ross Boylan <rossboylan@stanfordalumni.org> - 2019-06-02 03:10 +0200
        Re: df shows wrong disk size Henning Follmann <hfollmann@itcfollmann.com> - 2019-06-03 12:50 +0200
          Re: df shows wrong disk size Ross Boylan <rossboylan@stanfordalumni.org> - 2019-06-03 18:20 +0200
            Re: df shows wrong disk size Ross Boylan <rossboylan@stanfordalumni.org> - 2019-06-03 19:40 +0200
              Re: df shows wrong disk size Pascal Hambourg <pascal@plouf.fr.eu.org> - 2019-06-04 07:30 +0200
    Re: df shows wrong disk size Pascal Hambourg <pascal@plouf.fr.eu.org> - 2019-06-01 21:20 +0200
    Re: df shows wrong disk size Gary Dale <gary@extremeground.com> - 2019-06-02 00:50 +0200
      Re: df shows wrong disk size Ross Boylan <rossboylan@stanfordalumni.org> - 2019-06-02 02:40 +0200
        Re: df shows wrong disk size Stefan Monnier <monnier@iro.umontreal.ca> - 2019-06-02 17:20 +0200

#209500 — df shows wrong disk size

df says my volume is 3G, but everything else says it's 4G.  What's
going on and how can I correct it?

This question concerns the total reported space, not the free space.

The volume is an LVM logical volume on buster with an ext4 file
system.  I originally mistakenly created it as 4TB in size.
Then I took it offline, resized the file system to 3G, resized the
logical volume to 4G, and  then auto-resized (that is, ran resize2fs
without specifying an explicit size) the file system to 4G.  When df
showed the size as 3G I thought it might be temporary, but it it
reports the same value after reboot.

(If you're wondering: I resized to 3G first out of concern that the
file system requires a slightly larger "partition" than its own size,
and I didn't want to risk cutting off the end of the file system.)

I thought there might have been a huge amount of reserved space or
journal from the original 4TB size, but the values in dumpe2fs appear
normal to me.

Running buster.

Thanks.
Ross

# df -h /var/local/cache/
Filesystem                  Size  Used Avail Use% Mounted on
/dev/mapper/vgbarley-cache  3.0G  721M  2.1G  26% /var/local/cache
root@barley:~/tempserver/root# lvs vgbarley
  LV      VG       Attr       LSize   Pool Origin Data%  Meta%  Move
Log Cpy%Sync Convert
  cache   vgbarley -wi-ao----   4.00g
## etc
# resize2fs /dev/vgbarley/cache
resize2fs 1.44.5 (15-Dec-2018)
The filesystem is already 1048576 (4k) blocks long.  Nothing to do!
# So both LVM and e2fs utilities see 4G , even thouogh df reports 3G

# somewhat later
# dumpe2fs -h /dev/vgbarley/cache
dumpe2fs 1.44.5 (15-Dec-2018)
Filesystem volume name:   <none>
Last mounted on:          /var/local/cache
Filesystem UUID:          0601d7dc-2efe-46c7-9cac-205a761b70ef
Filesystem magic number:  0xEF53
Filesystem revision #:    1 (dynamic)
Filesystem features:      has_journal ext_attr resize_inode dir_index
filetype needs_recovery extent 64bit flex_bg sparse_super large_file
huge_file dir_nlink extra_isize metadata_csum
Filesystem flags:         signed_directory_hash
Default mount options:    user_xattr acl
Filesystem state:         clean
Errors behavior:          Continue
Filesystem OS type:       Linux
Inode count:              131072
Block count:              1048576
Reserved block count:     52428
Free blocks:              621488
Free inodes:              122857
First block:              0
Block size:               4096
Fragment size:            4096
Group descriptor size:    64
Reserved GDT blocks:      1024
Blocks per group:         32768
Fragments per group:      32768
Inodes per group:         4096
Inode blocks per group:   256
Flex block group size:    16
Filesystem created:       Mon May 27 11:54:50 2019
Last mount time:          Thu May 30 17:06:02 2019
Last write time:          Thu May 30 17:06:02 2019
Mount count:              2
Maximum mount count:      -1
Last checked:             Mon May 27 14:17:18 2019
Check interval:           0 (<none>)
Lifetime writes:          35 GB
Reserved blocks uid:      0 (user root)
Reserved blocks gid:      0 (group root)
First inode:              11
Inode size:              256
Required extra isize:     32
Desired extra isize:      32
Journal inode:            8
Default directory hash:   half_md4
Directory Hash Seed:      24162063-f4a6-4420-b79b-3ad4f9b71ab7
Journal backup:           inode blocks
Checksum type:            crc32c
Checksum:                 0x48ff013b
Journal features:         journal_64bit journal_checksum_v3
Journal size:             1024M
Journal length:           262144
Journal sequence:         0x000005be
Journal start:            1
Journal checksum type:    crc32c
Journal checksum:         0xb7b54059

[toc] | [next] | [standalone]

#209501

On Sat, Jun 01, 2019 at 10:41:20AM -0700, Ross Boylan wrote:
> df says my volume is 3G, but everything else says it's 4G.  What's
> going on and how can I correct it?
> 
> This question concerns the total reported space, not the free space.
> 
> The volume is an LVM logical volume on buster with an ext4 file
> system.  I originally mistakenly created it as 4TB in size.
> Then I took it offline, resized the file system to 3G, resized the
> logical volume to 4G, and  then auto-resized (that is, ran resize2fs
> without specifying an explicit size) the file system to 4G.  When df
> showed the size as 3G I thought it might be temporary, but it it
> reports the same value after reboot.
> 
> (If you're wondering: I resized to 3G first out of concern that the
> file system requires a slightly larger "partition" than its own size,
> and I didn't want to risk cutting off the end of the file system.)
> 
> I thought there might have been a huge amount of reserved space or
> journal from the original 4TB size, but the values in dumpe2fs appear
> normal to me.
> 
> Running buster.
> 
> Thanks.
> Ross
> 
> # df -h /var/local/cache/
> Filesystem                  Size  Used Avail Use% Mounted on
> /dev/mapper/vgbarley-cache  3.0G  721M  2.1G  26% /var/local/cache
> root@barley:~/tempserver/root# lvs vgbarley
>   LV      VG       Attr       LSize   Pool Origin Data%  Meta%  Move
> Log Cpy%Sync Convert
>   cache   vgbarley -wi-ao----   4.00g
> ## etc
> # resize2fs /dev/vgbarley/cache
> resize2fs 1.44.5 (15-Dec-2018)
> The filesystem is already 1048576 (4k) blocks long.  Nothing to do!
> # So both LVM and e2fs utilities see 4G , even thouogh df reports 3G
> 
What is the output of 'df -B4096'?

Regards,

-Roberto

-- 
Roberto C. Sánchez

[toc] | [prev] | [next] | [standalone]

#209508

# df -B4096 /var/local/cache/
Filesystem                 4K-blocks   Used Available Use% Mounted on
/dev/mapper/vgbarley-cache    778160 191713    529923  27% /var/local/cache

# e2fsck -v /dev/vgbarley/cache
e2fsck 1.44.5 (15-Dec-2018)
/dev/vgbarley/cache: clean, 8361/131072 files, 462129/1048576 blocks

I read these as still showing 3G for df but 4G for ext.
Ross

[toc] | [prev] | [next] | [standalone]

#209510

Le 01/06/2019 à 23:46, Ross Boylan a écrit :
> 
> # e2fsck -v /dev/vgbarley/cache
> e2fsck 1.44.5 (15-Dec-2018)
> /dev/vgbarley/cache: clean, 8361/131072 files, 462129/1048576 blocks

You must specify -f for a complete check.

[toc] | [prev] | [next] | [standalone]

#209514

# e2fsck -v -f /dev/vgbarley/cache
e2fsck 1.44.5 (15-Dec-2018)
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information

        8410 inodes used (6.42%, out of 131072)
          47 non-contiguous files (0.6%)
           1 non-contiguous directory (0.0%)
             # of inodes with ind/dind/tind blocks: 0/0/0
             Extent depth histogram: 5628/2
      469120 blocks used (44.74%, out of 1048576)
           0 bad blocks
           1 large file

        2684 regular files
        1583 directories
           0 character device files
           0 block device files
           0 fifos
           0 links
        4134 symbolic links (2772 fast symbolic links)
           0 sockets
------------
        8401 files

I don't see any errors, and it still reports 1M blocks @4k/block -> 4G
Although the blocks in use are very high, 469k -> 1.6G (48%)

In contrast, df reports 770M/3G in use (28%).  So if there's 1G that
is somehow hidden from that total, adding it to both sides gives
roughly
1.8G/4G  which is close to the 1.6G reported in use above.

du -sh says 745M is in use.

Reserved blocks reported by dumpe2fs is only 5% of the block count, so
they can't account for the difference.

e2fs also reports 621488 free blocks, roughly 2.4GB.  That is roughly
consistent with free space reported by df, even though their total
space counts disagree.

Maybe the data management structures, which were originally
appropriate for a filesystem 1000x larger than the one I have now, are
somehow chewing up blocks in some way not directly reported?  But it
seems quite a coincidence that the reported drive size by df matches
the earlier shrunken size of 3G so well.

Ross

[toc] | [prev] | [next] | [standalone]

#209553

On Sat, Jun 01, 2019 at 02:46:00PM -0700, Ross Boylan wrote:
> # df -B4096 /var/local/cache/
> Filesystem                 4K-blocks   Used Available Use% Mounted on
> /dev/mapper/vgbarley-cache    778160 191713    529923  27% /var/local/cache
> 
> # e2fsck -v /dev/vgbarley/cache
> e2fsck 1.44.5 (15-Dec-2018)
> /dev/vgbarley/cache: clean, 8361/131072 files, 462129/1048576 blocks
> 
> I read these as still showing 3G for df but 4G for ext.
> Ross
> 

You cheat ;)
please show that

/dev/mapper/vgbarley-cache == /dev/vgbarley/cache

-H

-- 
Henning Follmann           | hfollmann@itcfollmann.com

[toc] | [prev] | [next] | [standalone]

#209579

# ls -l /dev/mapper/vgbarley-cache /dev/vgbarley/cache
lrwxrwxrwx 1 root root 8 Jun  1 17:26 /dev/mapper/vgbarley-cache -> ../dm-19
lrwxrwxrwx 1 root root 8 Jun  1 17:26 /dev/vgbarley/cache -> ../dm-19

On Mon, Jun 3, 2019 at 2:32 AM Henning Follmann
<hfollmann@itcfollmann.com> wrote:

>
> You cheat ;)
> please show that
>
> /dev/mapper/vgbarley-cache == /dev/vgbarley/cache

# ls -l /dev/mapper/vgbarley-cache /dev/vgbarley/cache
lrwxrwxrwx 1 root root 8 Jun  1 17:26 /dev/mapper/vgbarley-cache -> ../dm-19
lrwxrwxrwx 1 root root 8 Jun  1 17:26 /dev/vgbarley/cache -> ../dm-19

[toc] | [prev] | [next] | [standalone]

#209580

I just noticed the reported journal size is exactly 1G, which would
account for the difference:
Journal size:             1024M
That's assuming the units are bytes; if they are blocks, it's just a
crazy value.

I'll see what the extN experts have to say.
Ross

[toc] | [prev] | [next] | [standalone]

#209596

Le 03/06/2019 à 19:36, Ross Boylan a écrit :
> I just noticed the reported journal size is exactly 1G, which would
> account for the difference:
> Journal size:             1024M
> That's assuming the units are bytes; if they are blocks, it's just a
> crazy value.

Sounds interesting. Here on a ~4 GiB ext4 filesystem the journal size is 
between 32 and 128 MiB. You can try to change it with tune2fs.

[toc] | [prev] | [next] | [standalone]

#209505

Le 01/06/2019 à 19:41, Ross Boylan a écrit :
> df says my volume is 3G, but everything else says it's 4G.  What's
> going on and how can I correct it?

Did you try e2fsck ?

[toc] | [prev] | [next] | [standalone]

#209511

I suggest trying gparted to read the partition table on your drive. 
There may be a problem and gparted is usually pretty good at finding 
partition table errors.


On 2019-06-01 1:41 p.m., Ross Boylan wrote:
> df says my volume is 3G, but everything else says it's 4G.  What's
> going on and how can I correct it?
>
> This question concerns the total reported space, not the free space.
>
> The volume is an LVM logical volume on buster with an ext4 file
> system.  I originally mistakenly created it as 4TB in size.
> Then I took it offline, resized the file system to 3G, resized the
> logical volume to 4G, and  then auto-resized (that is, ran resize2fs
> without specifying an explicit size) the file system to 4G.  When df
> showed the size as 3G I thought it might be temporary, but it it
> reports the same value after reboot.
>
> (If you're wondering: I resized to 3G first out of concern that the
> file system requires a slightly larger "partition" than its own size,
> and I didn't want to risk cutting off the end of the file system.)
>
> I thought there might have been a huge amount of reserved space or
> journal from the original 4TB size, but the values in dumpe2fs appear
> normal to me.
>
> Running buster.
>
> Thanks.
> Ross
>
> # df -h /var/local/cache/
> Filesystem                  Size  Used Avail Use% Mounted on
> /dev/mapper/vgbarley-cache  3.0G  721M  2.1G  26% /var/local/cache
> root@barley:~/tempserver/root# lvs vgbarley
>    LV      VG       Attr       LSize   Pool Origin Data%  Meta%  Move
> Log Cpy%Sync Convert
>    cache   vgbarley -wi-ao----   4.00g
> ## etc
> # resize2fs /dev/vgbarley/cache
> resize2fs 1.44.5 (15-Dec-2018)
> The filesystem is already 1048576 (4k) blocks long.  Nothing to do!
> # So both LVM and e2fs utilities see 4G , even thouogh df reports 3G
>
> # somewhat later
> # dumpe2fs -h /dev/vgbarley/cache
> dumpe2fs 1.44.5 (15-Dec-2018)
> Filesystem volume name:   <none>
> Last mounted on:          /var/local/cache
> Filesystem UUID:          0601d7dc-2efe-46c7-9cac-205a761b70ef
> Filesystem magic number:  0xEF53
> Filesystem revision #:    1 (dynamic)
> Filesystem features:      has_journal ext_attr resize_inode dir_index
> filetype needs_recovery extent 64bit flex_bg sparse_super large_file
> huge_file dir_nlink extra_isize metadata_csum
> Filesystem flags:         signed_directory_hash
> Default mount options:    user_xattr acl
> Filesystem state:         clean
> Errors behavior:          Continue
> Filesystem OS type:       Linux
> Inode count:              131072
> Block count:              1048576
> Reserved block count:     52428
> Free blocks:              621488
> Free inodes:              122857
> First block:              0
> Block size:               4096
> Fragment size:            4096
> Group descriptor size:    64
> Reserved GDT blocks:      1024
> Blocks per group:         32768
> Fragments per group:      32768
> Inodes per group:         4096
> Inode blocks per group:   256
> Flex block group size:    16
> Filesystem created:       Mon May 27 11:54:50 2019
> Last mount time:          Thu May 30 17:06:02 2019
> Last write time:          Thu May 30 17:06:02 2019
> Mount count:              2
> Maximum mount count:      -1
> Last checked:             Mon May 27 14:17:18 2019
> Check interval:           0 (<none>)
> Lifetime writes:          35 GB
> Reserved blocks uid:      0 (user root)
> Reserved blocks gid:      0 (group root)
> First inode:              11
> Inode size:              256
> Required extra isize:     32
> Desired extra isize:      32
> Journal inode:            8
> Default directory hash:   half_md4
> Directory Hash Seed:      24162063-f4a6-4420-b79b-3ad4f9b71ab7
> Journal backup:           inode blocks
> Checksum type:            crc32c
> Checksum:                 0x48ff013b
> Journal features:         journal_64bit journal_checksum_v3
> Journal size:             1024M
> Journal length:           262144
> Journal sequence:         0x000005be
> Journal start:            1
> Journal checksum type:    crc32c
> Journal checksum:         0xb7b54059
>
>

[toc] | [prev] | [next] | [standalone]

#209513

On Sat, Jun 1, 2019 at 3:49 PM Gary Dale <gary@extremeground.com> wrote:
>
> I suggest trying gparted to read the partition table on your drive.
> There may be a problem and gparted is usually pretty good at finding
> partition table errors.
>
Since the file system is sitting on an LVM logical volume, I don't
think the partitioning of the raw disks is directly relevant.

The analogue to the partition size is the logical volume size.  As my
original message showed, LVM does think the volume is 4GB.

If the filesystem and the volume manager both agree on 4GB, I don't
know where df is getting the notion that it's 3GB.  It seems very
likely it's a holdover from when I shrunk the filesystem to 3GB.  I
speculate that when I did the resize2fs that made it 4GB the fact that
I didn't explicitly specify a size meant the code path in resize2fs
didn't reset something that the explicit shrink to 3GB did set.

Ross

[toc] | [prev] | [next] | [standalone]

#209528

> If the filesystem and the volume manager both agree on 4GB, I don't
> know where df is getting the notion that it's 3GB.  It seems very

Sure looks like a bug.  I think reporting it as a bug to the ext234
people is The Right Thing to do.


        Stefan

[toc] | [prev] | [standalone]

Back to top | Article view | linux.debian.user

csiph-web