Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.user > #269001 > unrolled thread

bootable pendrive from zip file

Back to article view | Back to linux.debian.user

Contents

  bootable pendrive from zip file Luis Muñoz Fuente <luis.munoz.edu@juntadeandalucia.es> - 2024-04-22 19:00 +0200
    Re: bootable pendrive from zip file "Thomas Schmitt" <scdbackup@gmx.net> - 2024-04-22 20:00 +0200
      Re: bootable pendrive from zip file Luis Muñoz Fuente <luis.munoz.edu@juntadeandalucia.es> - 2024-04-22 20:10 +0200
        Re: bootable pendrive from zip file "Thomas Schmitt" <scdbackup@gmx.net> - 2024-04-22 20:30 +0200
          Re: bootable pendrive from zip file Luis Muñoz Fuente <luis.munoz.edu@juntadeandalucia.es> - 2024-04-22 21:00 +0200
          Re: bootable pendrive from zip file Luis Muñoz Fuente <luis.munoz.edu@juntadeandalucia.es> - 2024-04-22 21:10 +0200
            Re: bootable pendrive from zip file "Thomas Schmitt" <scdbackup@gmx.net> - 2024-04-22 21:50 +0200
      Re: bootable pendrive from zip file Max Nikulin <manikulin@gmail.com> - 2024-04-23 04:50 +0200
        Re: bootable pendrive from zip file "Thomas Schmitt" <scdbackup@gmx.net> - 2024-04-23 08:30 +0200
          Re: bootable pendrive from zip file Luis Muñoz Fuente <luis.munoz.edu@juntadeandalucia.es> - 2024-04-24 10:50 +0200
          Re: bootable pendrive from zip file Max Nikulin <manikulin@gmail.com> - 2024-04-25 05:30 +0200
            Re: bootable pendrive from zip file "Thomas Schmitt" <scdbackup@gmx.net> - 2024-04-25 09:00 +0200
              Re: bootable pendrive from zip file Max Nikulin <manikulin@gmail.com> - 2024-04-25 13:20 +0200
                Re: bootable pendrive from zip file "Thomas Schmitt" <scdbackup@gmx.net> - 2024-04-25 13:50 +0200

#269001 — bootable pendrive from zip file

Hello:
I recently used clonezilla and followed these instructions:

https://clonezilla.org/liveusb.php#linux-setup

to create a bootable pendrive from a zip file. What I liked about this 
method is that I can continue saving data on the pendrive and if I want 
to delete clonezilla I just have to delete it normally, without the need 
to format.

The usual method of creating a pendrive to install debian is:

https://www.debian.org/releases/stable/amd64/ch04s03.en.html#usb-copy-isohybrid

but this way I can't write more files to the pendrive, unless I format 
the empty space. And when I want to delete the installation from the 
pendrive I have to use fdisk and mkfs.

I have tried to transform the debian-12.5.0-amd64-netinst.iso file into 
zip but the size has gone from 659 MiB to 1.5 GiB, when with clonezilla 
both take up the same size. This increase in size makes it take longer 
to copy the data to the pendrive.
Does anyone know why this happens.
Thanks in advance.

[toc] | [next] | [standalone]

#269002

Hi,

Luis Muñoz Fuente wrote:
> I recently used clonezilla and followed these instructions:
> https://clonezilla.org/liveusb.php#linux-setup

The variation for "uEFI", i assume.

This aims at an undocumented habit of EFI implementations to look in
any FAT filesystem for a \EFI\BOOT directory with a suitable BOOT*.EFI
file and to start it, if found.
(Officially documented is to look in FAT filesystems of partitions with
MBR type 0xEF or GPT type GUID C12A7328-F81F-11D2-BA4B-00A0C93EC93B.)


> I have tried to transform the debian-12.5.0-amd64-netinst.iso file into zip
> but the size has gone from 659 MiB to 1.5 GiB, [...]
> Does anyone know why this happens.

To get an answer you will have to show what you did and where you
measured the resulting size (as ZIP archive file or on the USB stick ?).


But i don't think that intermediate storage as ZIP is needed at all.

The make-bootable-by-copy trick depends on /EFI/BOOT and BOOT*.EFI files
being in the ISO or ZIP archive. A Debian netinst ISO filesystem for amd64
contains an unpacked copy of its EFI boot partition files.
(Others do too, thanks to the relentless proselytization of Pete Batard,
 the developer of program Rufus.)

So just mount the ISO:

  $ sudo mount debian-12.5.0-amd64-netinst.iso /mnt/iso
  mount: /dev/loop0 is write-protected, mounting read-only

and copy its content to the mounted USB stick.
You will perceive about 100 MB increase in size, because Linux does not
represent the hardlinks in the ISO which save some space.

The result is supposed to boot where a Clonezilla stick boots after it
was made by unpacking the ZIP archive.
Another question is how far the programs in a Debian netinst ISO are
prepared to run from a FAT filesystem and to find their files in it.

Your mileage may vary.

------------------------------------------------------------------------
Illustration: The two copies of the \EFI\BOOT directory in a netinst ISO

  $ sudo mount debian-12.2.0-amd64-netinst.iso /mnt/iso
  $ find /mnt/iso/EFI/boot
  /mnt/iso/EFI/boot
  /mnt/iso/EFI/boot/bootia32.efi
  /mnt/iso/EFI/boot/bootx64.efi
  /mnt/iso/EFI/boot/grubia32.efi
  /mnt/iso/EFI/boot/grubx64.efi
  $ mount /mnt/iso/boot/grub/efi.img /mnt/fat
  $ find /mnt/fat/efi/boot | sort
  /mnt/fat/efi/boot
  /mnt/fat/efi/boot/bootia32.efi
  /mnt/fat/efi/boot/bootx64.efi
  /mnt/fat/efi/boot/grubia32.efi
  /mnt/fat/efi/boot/grubx64.efi
  $


Have a nice day :)

Thomas

[toc] | [prev] | [next] | [standalone]

#269003

El 22/4/24 a las 19:49, Thomas Schmitt escribió:
> Hi,

Thanks for the reply. My question is rather why the size increases so 
much. When I take a folder that occupies 5 GiB and with mkisofs I create 
an iso file, it still occupies 5 GiB. And if I later extract the files 
it takes up 5 GiB again, why does extracting the files from the debian 
iso increase the size so much?

[toc] | [prev] | [next] | [standalone]

#269004

Hi,

Luis Muñoz Fuente wrote:
> why does extracting the files from the debian iso increase the
> size so much?

Hard to say if you do not show what you do in particular.


In general an increase of about 120 MB is to be expected because of
expansion of hardlinks:

  $ du debian-12.2.0-amd64-netinst.iso
  643076  debian-12.2.0-amd64-netinst.iso
  $ sudo mount debian-12.2.0-amd64-netinst.iso /mnt/iso
  mount: /dev/loop0 is write-protected, mounting read-only
  $ du -s /mnt/iso
  762497  /mnt/iso

The bulk of duplication is with directory trees /firmware and
/pool/non-free-firmware . Some is with kernels and initrds.


> When I take a folder that occupies 5 GiB and with mkisofs I create an iso
> file, it still occupies 5 GiB.

Not if your disk filesystem supports sparse files and your files contain
substantial areas of unwritten bytes. In that case the ISO will be larger.

> And if I later extract the files it takes up 5 GiB again,

Not if there was a substantial amount of hardlink siblings among your
input files. mkisofs will store them with shared content but Linux will
represent them as independent files.

But an increase of an amd64 netinst ISO from 659 to 1500 MB cannot be
explained by hardlinks alone. Maybe you put it into the ZIP archive
twice ?


Have a nice day :)

Thomas

[toc] | [prev] | [next] | [standalone]

#269005

El 22/4/24 a las 20:25, Thomas Schmitt escribió:
> Hard to say if you do not show what you do in particular.

Yes, sorry.

$ du debian-12.5.0-amd64-netinst.iso
644100	debian-12.5.0-amd64-netinst.iso

# mount debian-12.5.0-amd64-netinst.iso /mnt
mount: /mnt: ATENCIÓN: el dispositivo está protegido contra escritura; 
se monta como sólo lectura.

$ du -s /mnt
764031	/mnt

$ cp -r /mnt/. ~/tmp

$ du -s ~/tmp
767072	/home/usuario/tmp

 From the Files program, right click properties on the tmp folder:
1.576 elementos, 783,1 MB en total

Now I go into the tmp folder, select everything, right click properties:
3.140 elementos, 1,6 GB en total

but with pcmanfm I always see the size of 749.1 MiB (785,448,960 bytes), 
so it seems that Files does not show me the information well.

Thanks

[toc] | [prev] | [next] | [standalone]

#269006

I assume the problem is the debian link, which points to the same directory:
$ ls -l tmp/debian
lrwxrwxrwx 1 user user 1 Apr 22 20:47 tmp/debian -> .
and creates a loop, I guess that's also why if I compress with:
zip -r debian.zip tmp
It never ends but from the graphical environment it does compress well, 
because it will not pay attention to the recursive link.

[toc] | [prev] | [next] | [standalone]

#269007

Hi,

Luis Muñoz Fuente wrote:
> I assume the problem is the debian link, which points to the same directory:
> $ ls -l tmp/debian
> lrwxrwxrwx 1 user user 1 Apr 22 20:47 tmp/debian -> .
> and creates a loop,

That's not a link loop, because "." is not a symbolic link.
But if a tree traversal is instructed to follow symbolic links, then the
traversal becomes endless recursion which is quite equivalent to an
endless loop.

I guess it would work better with zip option --symlinks.
But FAT cannot represent symbolic links. So in the FAT filesystem you will
possibly need at least one layer of duplication to emulate the link.
I.e. a copy of the whole file tree which is accessible via /debian. In
that copy there will probably be no need for another tree under
/debian/debian.


Have a nice day :)

Thomas

[toc] | [prev] | [next] | [standalone]

#269016

On 23/04/2024 00:49, Thomas Schmitt wrote:
> This aims at an undocumented habit of EFI implementations to look in
> any FAT filesystem for a \EFI\BOOT directory with a suitable BOOT*.EFI
> file and to start it, if found.
> (Officially documented is to look in FAT filesystems of partitions with
> MBR type 0xEF or GPT type GUID C12A7328-F81F-11D2-BA4B-00A0C93EC93B.)

Out of curiosity, does the requirement of specific GUID exist for 
removable drives? A USB drive may be formatted without partition table.

I am in doubts if any user will really benefit from UEFI implementations 
strictly following the spec. Making a bootable media is harder for 
regular users, chance of unintentional action is low, it is unlikely an 
obstacle for attackers trying to force users to boot a compromised image.

> So just mount the ISO:
> and copy its content to the mounted USB stick.

7z and bsdtar can extract content of ISO files without mounting images.

[toc] | [prev] | [next] | [standalone]

#269018

Hi,

Max Nikulin wrote:
> Out of curiosity, does the requirement of specific GUID exist for removable
> drives?

It is disputed, whether the specs say that the partitions must be marked
by 0xEF in legacy MBR tables and by C12A7328-F81F-11D2-BA4B-00A0C93EC93B
in GPT.
In practice there seems to be no such demand. A zillion Microsoft users
would complain if the specs were interpreted narrowly.


> A USB drive may be formatted without partition table.

The specs only talk of partitions. Whether the real implementations would
look into the FAT filesystem of an unpartitioned device would have to be
tested.

(Boot firmware is a bitch. The hunchbacked partition layout of Debian ISOs
is still the one which boots on most machines. Other distros decided to
clean up at the cost of losing some old laptops.)


> 7z and bsdtar can extract content of ISO files without mounting images.

But mounting needs no special software and gives you the opportunity
to use many different ways of copying, which may be decisive when the
target is a heavily restricted filesystem like FAT.

(I still wonder which software in the Debian ISO needs the symbolic link
"/debian -> ." and which parts of the file tree are accessed via this
link. Probably one can avoid to duplicate the whole tree under /debian.)


Have a nice day :)

Thomas

[toc] | [prev] | [next] | [standalone]

#269044

El 23/4/24 a las 8:21, Thomas Schmitt escribió:
> (I still wonder which software in the Debian ISO needs the symbolic link
> "/debian -> ." and which parts of the file tree are accessed via this
> link. Probably one can avoid to duplicate the whole tree under /debian.)

Hello:
I have copied the files to the pendrive formatted in fat32 with the command:

cp -a /mnt/. /media/user/USB

and logically it did not copy the symbolic links and I tried installing 
Debian on a computer and it worked perfectly, although I will not use it 
as a method of generating bootable USBs to avoid possible errors that 
are difficult to detect in the future. Clonezilla does not have links 
and that is why it can be copied directly to the pendrive.

Thanks so much

[toc] | [prev] | [next] | [standalone]

#269075

On 23/04/2024 13:21, Thomas Schmitt wrote:
> 
> Max Nikulin wrote:
>> Out of curiosity, does the requirement of specific GUID exist for removable
>> drives?
> 
> It is disputed, whether the specs say that the partitions must be marked
> by 0xEF in legacy MBR tables and by C12A7328-F81F-11D2-BA4B-00A0C93EC93B
> in GPT.

It happened so that I had locally a file with UEFI spec Version 2.3.1, 
Errata C June 27, 2012. I tried to search it for "removable" and for 
"0xef". Since I am not familiar with context of the following snippets, 
my interpretation may be wrong. Later versions may have some updates.

12.3.3 Number and Location of System Partitions
... Further, UEFI implementations may allow the use of conforming FAT 
partitions which do not use the ESP GUID. Partition creators may prevent 
UEFI firmware from examining and using a specific partition by setting 
bit 1 of the Partition Attributes (see 5.3.3) which will exclude the 
partition as a potential ESP.

 From my point of view it is opposed to "must be" for strict partition 
type checks.

I have noticed that FAT12 and FAT16 are allowed for removable media. The 
restriction is that while a fixed media may have multiple ESP, a 
removable one must have single EFI partition.

>> A USB drive may be formatted without partition table.
> 
> The specs only talk of partitions.

12.3.1 System Partition
... For a diskette (floppy) drive, a partition is defined to be the 
entire media.

Some details are in 12.3.4.2 Diskette

USB pen drive is not a diskette, but it increases probability that 
superfloppy formatting style is supported. Of course, singe FAT 
partition is more portable.

>> 7z and bsdtar can extract content of ISO files without mounting images.
> 
> But mounting needs no special software and gives you the opportunity
> to use many different ways of copying, which may be decisive when the
> target is a heavily restricted filesystem like FAT.

I usually have p7zip-full installed to be able to extract files from 
various archives, so I do not consider it as a too special. I did not 
need extra flexibility, with 7z there was no need for extra mount/umount 
commands, just extracting as a regular user was enough. So in some cases 
it is more convenient.

[toc] | [prev] | [next] | [standalone]

#269080

Hi,

i wrote:
> > It is disputed, whether the specs say that the partitions must be marked
> > by 0xEF in legacy MBR tables and by C12A7328-F81F-11D2-BA4B-00A0C93EC93B
> > in GPT.

Max Nikulin wrote:
> It happened so that I had locally a file with UEFI spec Version 2.3.1,
> Errata C June 27, 2012.
> [...]
> "12.3.3 Number and Location of System Partitions
> ... Further, UEFI implementations may allow the use of conforming FAT
> partitions which do not use the ESP GUID."
> [...]
> From my point of view it is opposed to "must be" for strict partition type
> checks.

It is not forbidden, indeed. But it is not guaranteed that it works.
So if the boot success were not covered by tradition, it would be a
case of "your mileage may vary".

Another problem with the statement is that it only talks of GUID and
thus of GPT partitioning, while the specs allow MBR partitioning too.
It needs a bit of semantical stretching to let it cover the whole specs.


> Later versions may have some updates.

The statement is still in UEFI 2.8 as 13.3.3.


> Some details are in 12.3.4.2 Diskette
> USB pen drive is not a diskette, but it increases probability that
> superfloppy formatting style is supported. Of course, singe FAT partition is
> more portable.

If that's a quote from the specs, then it isn't in UEFI 2.8 any more.
If it's a statement by you, then i agree that there is a chance for
unpartitioned USB sticks to work.
You may also see implementations which boot via the El Torito boot
catalog from USB stick and via a partition table from optical media.


Have a nice day :)

Thomas

[toc] | [prev] | [next] | [standalone]

#269082

On 25/04/2024 13:51, Thomas Schmitt wrote:
> Max Nikulin wrote:
>> "12.3.3 Number and Location of System Partitions
>> ... Further, UEFI implementations may allow the use of conforming FAT
>> partitions which do not use the ESP GUID."
> 
> Another problem with the statement is that it only talks of GUID and
> thus of GPT partitioning, while the specs allow MBR partitioning too.

I agree. I have not tried to find anything similar for MBR.

For images provided by distributions, it is important to follow specs to 
minimize chance of failure. For users balance may be different.

I prefer non-destructive approach for making bootable images. It allows 
to add some files. It is even possible to remount media as read-writable 
while a live image is booted. I was not aware that partition type might 
be an issue.

I consider direct byte to byte copy of an image (instead of putting 
files to an earlier formatted drive) as the last resort. A decade ago I 
had to use it when I messed up something with syslinux configuration.

>> USB pen drive is not a diskette, but it increases probability that
>> superfloppy formatting style is supported. Of course, singe FAT partition is
>> more portable.
> 
> If that's a quote from the specs,

Sorry for confusion, certainly this paragraph is my comment.

As to the "debian" symlink, it might be either for apt-cdrom or for 
those who mounts images as local repository. It is just a guess, I have 
no example of a tool that needs the symlink.

[toc] | [prev] | [next] | [standalone]

#269083

Hi,

Max Nikulin wrote:
> I was not aware that partition type might be an issue.

Thanks to the normative power of the facts a "may" in the specs becomes
a reason to return a mainboard with an EFI that chooses to join the
"may not" side.


Have a nice day :)

Thomas

[toc] | [prev] | [standalone]

Back to top | Article view | linux.debian.user

csiph-web