Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.user > #171153 > unrolled thread

ISO Verif: Is the "default 512 block size" == "I/O sector" size?

Back to article view | Back to linux.debian.user

Contents

  ISO Verif: Is the "default 512 block size" == "I/O sector" size? Andrew F Comly   康大成 <andrew.comly@protonmail.com> - 2016-08-17 04:20 +0200
    Re: ISO Verif: Is the "default 512 block size" == "I/O sector" size? "Thomas Schmitt" <scdbackup@gmx.net> - 2016-08-17 10:50 +0200

#171153 — ISO Verif: Is the "default 512 block size" == "I/O sector" size?

[Multipart message — attachments visible in raw view] — view raw

Thomas,
You said earlier that 512 is the default block size. But I looked up that "bs" also refers to block size.
e.g.
"dd bs=1M count=613"
Analyzing:
bs=1M; 1M = 1Mibibyte;
1Mibibyte = 1048576 bytes; 1048576 bytes / 512 bytes/block = 2048 blocks
Thus
1M = 2048 blocks of 512 bytes/block
613 * 2048 = 1255424.

So it appears that we have two "block sizes". "dd" has its block size, "bs"; and then there is some foreign "other" block size. But after perusing the stats of my sdb drive:
[code]
$ fdisk -l
...
Disk /dev/sdb: 16.4 GB, 16358768640 bytes
255 heads, 63 sectors/track, 1988 cylinders, total 31950720 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x6350107e

Device Boot Start End Blocks Id System
/dev/sdb1 * 0 1255423 627712 0 Empty
/dev/sdb2 8196 8803 304 ef EFI (FAT-12/16/32)
[/code]
it seems that the 512 bytes/block isn''t some foreign "other" block size (not changeable), but is the "I/O" or "sector" size. This seems to inherently be 512 bytes/sector. More specifically from the above stats:
[code]
255 heads, 63 sectors/track, 1988 cylinders, total 31950720 sectors
Units = sectors of 1 * 512 = 512 bytes
[/code]
we can calculate 255 * 63 * 1988 = 31937220, which is really close to the above "31950720", and then
31937220 * 512 = 16351856640, where "16351856640" is really close to the above 16358768640.

Is my hypothesis that the "512 block size" is the "I/O" or "sector" size correct?



Sincerely,
Andrew F Comly





-------- Original Message --------
Subject: Re: [off list] iso verification: results - successful but still bewildered
Local Time: August 15, 2016 11:57 AM
UTC Time: August 15, 2016 11:57 AM
From: scdbackup@gmx.net
To: andrew.comly@protonmail.com

Hi,

> May I organize this info and post

Be invited.
But take care not to produce misleading rumors in this swamp of
fuzzy image ends and various block sizes.

The decisive point with checksumming from CD, DVD, or USB stick is
to process exactly the original size as it was when Debian computed
its checksum lists.
This size can be learned from the size of the ISO image file, if you
have it in reach:

$ ls -l debian-8.5.0-i386-lxde-CD-1.iso
-rw-r--r-- 1 thomas thomas 642777088 Jun 4 16:11 debian-8.5.0-i386-lxde-CD-1.iso

Your number came from dd's count when copying the ISO onto USB stick.
That's equivalent to the "ls -l" way of learning the size.

Or you take it from the inner info of the ISO 9660 filesystem if you
only have some media where it is stored beginning at block 0:

$ /sbin/isosize /dev/sdb
642777088

$ /sbin/isosize /dev/sr0
642777088

These counts are in bytes and would make an awfully slow read run if
used as block count with program "dd":

$ dd if=/dev/sdb bs=1 count=642777088 | sha512sum

We can safely divide it by 2048 and use bs=2048, because 2048 is the
block size which the entrails of the ISO use for addressing data
and counting the filesystem size.

$ expr 642777088 / 2048
313856

$ dd if=/dev/sdb bs=2048 count=313856 | sha512sum

Larger blocksizes are suitable only if they yield no remainder when
they divide the byte size. (Caution, "expr" does not show division
remainders or fractions.)

--------------------------------------------------------------------

If you want, we can explore why the script check_debian_iso did not
work for you and enhance it for the next user with the same issue.
I'd then ask Steve McIntyre to find it some place on the Debian servers
and to change the wiki link.


Have a nice day :)

Thomas

[toc] | [next] | [standalone]

#171162

Hi,

Andrew F Comly wrote:
> You said earlier that 512 is the default block size.

The default of program "dd", to be exacting.

Block size is normally a property of devices and filesystems.
Many devices offer direct random access only at the granularity of
their block size.
E.g. a CD can be read in chunks of 2 KiB. If software needs to address
in finer steps then it has to load full blocks and pick from them
the bytes which are desired.

Filesystems use block addresses to point from name to data. It is
technically desirable that file system block size is an integer multiple
of the device block size. (Multiplication factor 1 is normal.)


> I looked up that "bs" also refers to block size.

Option bs= is the combination of dd options ibs= and obs=. The program
uses these sizes for single read or write operations. (One may experience
surprises if the input does not deliver full blocks or the output demands
larger ones.)
The advise to use large bs= when curbing data reading from a device or file
is merely to improve the read and write speed. Copying many small chunks
can impose substantial overhead.


> So it appears that we have two "block sizes".

In the case of buffered i/o on a USB stick or DVD you may more or less
use any block size which divides without remainder the desired amount of
bytes.
(Giant sizes should be avoided because they bring no improvement while
 filling giant memory buffers.)


> $ fdisk -l

fdisk seems to assume block size 512 with data files.
With devices it obviously inquires their block size.
E.g. with a CD:

  $ /sbin/fdisk -lu /dev/sr4
  ...
  Units: sectors of 1 * 2048 = 2048 bytes


> 255 heads, 63 sectors/track, 1988 cylinders,

The Cylinder/Head/Sector addresses of disk-like devices are quite bogus
nowadays. They are uphold for legacy software (and to confuse programmers
like me).
Regrettably MBR partition tables show these numbers, so one has sometimes
to find the factors sector-per-head and heads-per-cylinder from comparing
Logical Block Addresses with CHS adresses.
(Mathematically this is equivalent to the task of finding those points of
 a 2D-parabola which have integer coordinates.)

255 heads (per cylinder) and 63 sectors/track is a poor guess, of course,
as 16358768640 consists entirely of prime factors 2, 3, and 43.
The numbers 255 and 63 contain prime factors 5, 7, and 17 which spoil
any attempt to match above big number by integer multiplication.

The reason for 255, 63 is simply that they are the largest values
possible for heads-per-cyl and sec-per-head in the CHS address fields
of a MBR partition table.
The range of CHS addressing ends at 8,422,686,720 bytes, because MBR
partition table entries can only record up to 1024 cylinders.

So with your 32 GB stick, CHS is a mere ornament.


> Is my hypothesis that the "512 block size" is the "I/O" or "sector" size
> correct?

Roughly yes. But mainly in the case of fdisk, not so much in the case of dd.
WIth fdisk the number 512 stems either from device properties or from the
default of fdisk, if no device properties can be inquired (as from a data
file).

Unless the input of dd is a device which may deliver less than the block
size in a single read operation, you are free to choose a dd blocksize.
(It can become tricky with pipes as input. Even if you choose a block size
 lower than the pipe buffer size, you imay get less input bytes per block
 than expected.)


Have a nice day :)

Thomas

[toc] | [prev] | [standalone]

Back to top | Article view | linux.debian.user

csiph-web