Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.user > #186790 > unrolled thread

Installing the installer

Back to article view | Back to linux.debian.user

Contents

  Installing the installer Richard Owlett <rowlett@cloud85.net> - 2017-09-14 15:50 +0200
    Re: Installing the installer "Thomas Schmitt" <scdbackup@gmx.net> - 2017-09-14 17:00 +0200
      Re: Installing the installer Richard Owlett <rowlett@cloud85.net> - 2017-09-19 16:00 +0200
        Re: Installing the installer "Thomas Schmitt" <scdbackup@gmx.net> - 2017-09-19 17:20 +0200
          Re: Installing the installer Fungi4All <fungilife@protonmail.com> - 2017-09-19 21:10 +0200
            Re: Installing the installer "Thomas Schmitt" <scdbackup@gmx.net> - 2017-09-19 22:20 +0200
        A custom installer - was [Installing the installer] Richard Owlett <rowlett@cloud85.net> - 2017-09-23 17:50 +0200
          Re: A custom installer - was [Installing the installer] Richard Owlett <rowlett@cloud85.net> - 2017-09-24 15:50 +0200
    Re: Installing the installer Kushal Kumaran <kushal@locationd.net> - 2017-09-15 05:10 +0200
    Towards a customized netinst {a rephrasing of "Installing the  installer"} Richard Owlett <rowlett@cloud85.net> - 2017-09-21 15:10 +0200
      Re: Towards a customized netinst "Thomas Schmitt" <scdbackup@gmx.net> - 2017-09-21 16:50 +0200
        Re: Towards a customized netinst Richard Owlett <rowlett@cloud85.net> - 2017-09-22 16:10 +0200

#186790 — Installing the installer

Yes, I know the subject line looks strange. It's intentional.
I wish to do something not quite normal ;/

I wish to "install" to an arbitrary ext2/ext4 partition in such a manner 
that:
   1. Grub2 will recognize it as a legit OS.
   2. if the partition is on a USB flash drive it will boot normally
      on suitable hardware.
   3. all directories and files shall be modifiable.

I had done
     xorriso -osirrox on:auto_chmod_on -overwrite nondir \
       -indev "/home/richard/Downloads/debian-9.1.0-i386-netinst.iso" \
       -extract / /media/richard/netinst1

There were no error messages but Grub2's "update-grub" would not 
recognize it nor would my hardware boot it.

I suspect there something of the boot process that I don't grok.
Guidance please.

[toc] | [next] | [standalone]

#186795

Hi,

Richard Owlett wrote:
> I wish to "install" to an arbitrary ext2/ext4 partition in such a manner
> that:
>   1. Grub2 will recognize it as a legit OS.

Why not install - without quotes - Debian to as few partitions as possible ?
(Can it be forced to touch only one ?)

The system in the ISO can surely be booted somehow up to some point.
But even if it boots completely, it is not prepared to act as general
purpose GNU/Linux. It's job is to install Debian.

A Live-CD ISO would be somewhat more suitable, with some additional
preparations for your demand number 3.


> xorriso -osirrox on ... -extract / /media/richard/netinst1

(This is just one way to copy the directory tree out of the ISO into a disk
 tree. xorriso packs them up and packs them out.)


> Grub2's "update-grub" would not recognize it nor would my hardware boot it.

  https://wiki.archlinux.org/index.php/GRUB#Generate_the_main_configuration_file
says the partition must host the current root filesystem. The page proposes
chroot.


>   2. if the partition is on a USB flash drive it will boot normally
>     on suitable hardware.

Somehow the first bootloader of the drive must know about your partition
and how to boot its system.
So it won't be fully automatic by just dd-ing the partition image into a
partition device. Probably you will have to edit the GRUB configuration,
possibly by editing the configuration sources of the system which manages
the GRUB installation. ("# DO NOT EDIT THIS FILE" ...)


>   3. all directories and files shall be modifiable.

This makes a plain Live-CD ISO unsuitable. One would have to install a
writable filesystem in another partition. I have read about making an
overlay on the read-only ISO filesystem by the writable one to make the ISO
virtually writable.


But actually you want a runnable normal GNU/Linux. So install one and
practice how to transplant it onto USB sticks. It will be interesting
to see how long the copy script becomes in the end.
(Do filesystems have own UUIDs in the partition ? The partition UUID
 is in the partition table, normally outside of the partition.)


Have a nice day :)

Thomas

[toc] | [prev] | [next] | [standalone]

#187010

On 09/14/2017 09:51 AM, Thomas Schmitt wrote:
>
> Richard Owlett wrote:
>> I wish to "install" to an arbitrary ext2/ext4 partition in such
>> a manner
>> that:
>>   1. Grub2 will recognize it as a legit OS.
>
> Why not install - without quotes - Debian to as few partitions
> as possible ?

Because that is NOT my current goal.

> The system in the ISO can surely be booted somehow up to some point.
> But even if it boots completely, it is not prepared to act as general
> purpose GNU/Linux. It's job is to install Debian.

We agree. However, my goal is NOT a "general purpose GNU/Linux".
My goal IS a "thingy/dodad/whatsit" which can perform a single task.
That task being to be indistinguishable, to a human at a terminal, from 
a standard installer residing on a CD/DVD.

>
>> xorriso -osirrox on ... -extract / /media/richard/netinst1
>
> (This is just one way to copy the directory tree out of the ISO into
> a disk tree. xorriso packs them up and packs them out.)

I assumed that using xorriso on both ends would give me a "byte for 
byte" copy. That would imply that whatever was on the deveolper's end 
was not bootable.  ????

>
> [SNIP]
>
> But actually you want a runnable normal GNU/Linux.

No.
I want a "thingy/dodad/whatsit" that will install Debian to another 
location, be it device or partition.

[toc] | [prev] | [next] | [standalone]

#187016

Hi,

Richard Owlett wrote:
> > > xorriso -osirrox on ... -extract / /media/richard/netinst1

i wrote:
> > (This is just one way to copy the directory tree out of the ISO into
> > a disk tree. xorriso packs them up and packs them out.)

> I assumed that using xorriso on both ends would give me a "byte for byte"
> copy.

No, that's the job of "dd" or similar copy programs.


> > >  1. Grub2 will recognize it as a legit OS.

You will have to teach it by configuration file entries like described in
  https://wiki.gentoo.org/wiki/GRUB2/Chainloading
  https://superuser.com/questions/154133/grub-boot-from-iso
I wrote some remarks about that to Ethan Andrews in
  "Re: How do I boot a Debian 9.1.0 amd64 iso from GRUB?"
  https://lists.debian.org/debian-user/2017/09/msg00516.html

Your mileage may vary when the booted kernel expands its realm from
the initial RAM-disk to the ISO filesystem in the partition. It depends
much on the software in the initial RAM-disk and in the ISO whether
this will work.

Pascal Hambourg mentions problems with the initial RAM-disk content in
  https://lists.debian.org/debian-user/2017/09/msg00566.html

Felix Miata pointed to the naked kernels and initial RAM-disk images
  https://lists.debian.org/debian-user/2017/09/msg00568.html
I guess he means things like
  http://ftp.nl.debian.org/debian/dists/testing/main/installer-amd64/current/images/hd-media/
sparsely described in
  http://ftp.nl.debian.org/debian/dists/testing/main/installer-amd64/current/images/MANIFEST


> > >   2. if the partition is on a USB flash drive it will boot normally
> > >      on suitable hardware.

If you do not want to modify the installed GRUB on the hard disk on that
system, then you will need  a GRUB on that USB stick which gets preferred
by the firmware over the GRUB on hard disk, and then does the booting
of the operating system in the ISO.


> > >   3. all directories and files shall be modifiable.

> > But actually you want a runnable normal GNU/Linux.

> I want a "thingy/dodad/whatsit" that will install Debian to another
> location, be it device or partition.

If you do not want to unpack the ISO then you cannot directly modify files.
(I assume ISO 9660 multi-session is not what you intend, but rather
 normal filesystem operations from the running operating system.)

So you would need a separate writable filesystem and overlay it over
the ISO when the operating system is running. But a Debian installation
ISO is not prepared for doing that out of the box, afaik.
So you would have to make your own ISO which has such capabilities.

The hard part is modifying an unpacked Debian installation ISO so that
it can do what you want when it gets started from an ISO in a partition.
Maybe the pieces mentioned by Felix Miata can help.

Packing up such ISO would not be difficult. One would not have to
make it bootable by firmware by MBR or EFI system partition, because your
USB stick's GRUB would be set up to be started by the firmwares.


Question at that point is of course why one would want to have a read-only
filesystem like ISO 9660 in the partition.
The only reason would be if one wants to easily reset the filesystem by
erasing the overlay filesystem, or if one could not get the Debian software
inside the ISO to work from some suitable read-write filesystem.


Have a nice day :)

Thomas

[toc] | [prev] | [next] | [standalone]

#187027

[Multipart message — attachments visible in raw view] — view raw

> From: scdbackup@gmx.net
> To: debian-user@lists.debian.org
>
> Hi,
>
> Richard Owlett wrote:
>> > > xorriso -osirrox on ... -extract / /media/richard/netinst1
>
> i wrote:
>> > (This is just one way to copy the directory tree out of the ISO into
>> > a disk tree. xorriso packs them up and packs them out.)
>
>> I assumed that using xorriso on both ends would give me a "byte for byte"
>> copy.
>
> No, that"s the job of "dd" or similar copy programs.
>
>> > > 1. Grub2 will recognize it as a legit OS.
>
> You will have to teach it by configuration file entries like described in
> https://wiki.gentoo.org/wiki/GRUB2/Chainloading
> https://superuser.com/questions/154133/grub-boot-from-iso
> I wrote some remarks about that to Ethan Andrews in
> "Re: How do I boot a Debian 9.1.0 amd64 iso from GRUB?"
> https://lists.debian.org/debian-user/2017/09/msg00516.html
>
> Your mileage may vary when the booted kernel expands its realm from
> the initial RAM-disk to the ISO filesystem in the partition. It depends
> much on the software in the initial RAM-disk and in the ISO whether
> this will work.
>
> Pascal Hambourg mentions problems with the initial RAM-disk content in
> https://lists.debian.org/debian-user/2017/09/msg00566.html
>
> Felix Miata pointed to the naked kernels and initial RAM-disk images
> https://lists.debian.org/debian-user/2017/09/msg00568.html
> I guess he means things like
> http://ftp.nl.debian.org/debian/dists/testing/main/installer-amd64/current/images/hd-media/
> sparsely described in
> http://ftp.nl.debian.org/debian/dists/testing/main/installer-amd64/current/images/MANIFEST
>
>> > > 2. if the partition is on a USB flash drive it will boot normally
>> > > on suitable hardware.
>
> If you do not want to modify the installed GRUB on the hard disk on that
> system, then you will need a GRUB on that USB stick which gets preferred
> by the firmware over the GRUB on hard disk, and then does the booting
> of the operating system in the ISO.
>
>> > > 3. all directories and files shall be modifiable.
>
>> > But actually you want a runnable normal GNU/Linux.
>
>> I want a "thingy/dodad/whatsit" that will install Debian to another
>> location, be it device or partition.
>
> If you do not want to unpack the ISO then you cannot directly modify files.
> (I assume ISO 9660 multi-session is not what you intend, but rather
> normal filesystem operations from the running operating system.)
>
> So you would need a separate writable filesystem and overlay it over
> the ISO when the operating system is running. But a Debian installation
> ISO is not prepared for doing that out of the box, afaik.
> So you would have to make your own ISO which has such capabilities.
>
> The hard part is modifying an unpacked Debian installation ISO so that
> it can do what you want when it gets started from an ISO in a partition.
> Maybe the pieces mentioned by Felix Miata can help.
>
> Packing up such ISO would not be difficult. One would not have to
> make it bootable by firmware by MBR or EFI system partition, because your
> USB stick"s GRUB would be set up to be started by the firmwares.
>
> Question at that point is of course why one would want to have a read-only
> filesystem like ISO 9660 in the partition.
> The only reason would be if one wants to easily reset the filesystem by
> erasing the overlay filesystem, or if one could not get the Debian software
> inside the ISO to work from some suitable read-write filesystem.
>
> Have a nice day :)
>
> Thomas

It is hard to believe that this conversation can exist in a bubble of the time/space
continuum without the term "refracta" in it.  The most fascinating piece of software
or should I say family of sftw. since the time of space invaders.

Unless I am missing some fact about being unable to port refracta to debian, which
I doubt very much.  What is also absurd is that Thomas is involved in this conversation
while refracta is heavily relying on xorisso.

If I didn't know better I would have thought that Thomas wrote refracta himself.
Have two wonderful days, botho yas.

[toc] | [prev] | [next] | [standalone]

#187031

Hi,

Fungi4All wrote:
> "refracta" [...] The most fascinating piece of software 
> or should I say family of sftw. since the time of space invaders.

Can it boot a Debian installation ISO from a disk partition and let
it behave as if it had booted directly from a whole disk ?


> What is also absurd is that Thomas is involved in this conversation
> while refracta is heavily relying on xorisso.

They must be doing this behind my back. :))


Have a nice day :)

Thomas

[toc] | [prev] | [next] | [standalone]

#187134 — A custom installer - was [Installing the installer]

On 09/19/2017 08:56 AM, Richard Owlett wrote:
> On 09/14/2017 09:51 AM, Thomas Schmitt wrote:
>>
>> Richard Owlett wrote:
>>> I wish to "install" to an arbitrary ext2/ext4 partition in such
>>> a manner
>>> that:
>>>   1. Grub2 will recognize it as a legit OS.
>>
>> Why not install - without quotes - Debian to as few partitions
>> as possible ?

My goal is NOT a "general purpose GNU/Linux".
My goal IS a "thingy/dodad/whatsit" which can perform a single task.
That task would be indistinguishable, to a human at a terminal, from a 
standard installer residing on a CD/DVD. It will install Debian to 
another location, be it device or partition.

Evidently something similar to my goals existed in the early work on 
what became the current installer. Searching for pages describing how 
the boot process worked I found several pages by Joey Hess. Initially I 
found "os-prober" <https://joeyh.name/code/os-prober/> and later
"Joey Hess" <http://joeyh.name/> -> <http://joeyh.name/code/> {the links 
under "Past projects"}

There are also useful links in the Debian Wiki, including:
BootProcess
	<https://wiki.debian.org/BootProcess>
CategoryBootProcess
	<https://wiki.debian.org/CategoryBootProcess>
SystemBootProcess
	<https://wiki.debian.org/SystemBootProcess>
Title Search: "boot process"
	<https://wiki.debian.org/DebianDesktop?action=fullsearch&context=180&value=boot+process&titlesearch=Titles>

I currently have about two dozen links a single reference whose target 
audience was people with goals/questions similar to mine.

At least I now know what I want is possible.

[toc] | [prev] | [next] | [standalone]

#187155 — Re: A custom installer - was [Installing the installer]

On 09/23/2017 10:43 AM, Richard Owlett wrote:
> On 09/19/2017 08:56 AM, Richard Owlett wrote:
>
>[snip]
>
> My goal is NOT a "general purpose GNU/Linux".
> My goal IS a "thingy/dodad/whatsit" which can perform a single task.
> That task would be indistinguishable, to a human at a terminal, from a
> standard installer residing on a CD/DVD. It will install Debian to
> another location, be it device or partition.
>
>[snip]
>
> There are also useful links in the Debian Wiki, including:
> BootProcess
>     <https://wiki.debian.org/BootProcess>
> CategoryBootProcess
>     <https://wiki.debian.org/CategoryBootProcess>
> SystemBootProcess
>     <https://wiki.debian.org/SystemBootProcess>
> Title Search: "boot process"
>     <https://wiki.debian.org/DebianDesktop?action=fullsearch&context=180&value=boot+process&titlesearch=Titles>
>
>

Over on debian-boot@lists.debian.org I asked a related question. Some of 
the links were:
https://www.ibm.com/developerworks/linux/library/l-linuxboot/
    Inside the Linux boot process -- a guided tour from the MBR
    to 1st user-space application
https://en.wikipedia.org/wiki/Linux_startup_process
    The multi-stage initialization process performed during booting
    a Linux installation.
http://wiki.osdev.org/System_Initialization_%28x86%29
    System Initialization (x86) -- the traditional BIOS-based system
    initialization process.
https://wiki.archlinux.org/index.php/Arch_boot_process
    To boot Linux, a boot loader must be installed. The boot loader
    being responsible for loading the kernel and initial ramdisk before
    initiating the boot process.

They will do nicely to update my 1970's preconceptions - think paper 
tape and PDP11's prior to introduction of the RX01/RX02 floppies
and later exposure to an 8085 based embedded system. Now {perhaps ;} my 
questions will be phrased so that they make sense to those used to 
current technology.

[toc] | [prev] | [next] | [standalone]

#186826

Richard Owlett <rowlett@cloud85.net> writes:

> Yes, I know the subject line looks strange. It's intentional.
> I wish to do something not quite normal ;/
>
> I wish to "install" to an arbitrary ext2/ext4 partition in such a
> manner that:
>   1. Grub2 will recognize it as a legit OS.
>   2. if the partition is on a USB flash drive it will boot normally
>      on suitable hardware.
>   3. all directories and files shall be modifiable.
>
> I had done
>     xorriso -osirrox on:auto_chmod_on -overwrite nondir \
>       -indev "/home/richard/Downloads/debian-9.1.0-i386-netinst.iso" \
>       -extract / /media/richard/netinst1
>
> There were no error messages but Grub2's "update-grub" would not
> recognize it nor would my hardware boot it.
>
> I suspect there something of the boot process that I don't grok.
> Guidance please.

You want debootstrap.  See
https://verahill.blogspot.com/2013/03/361-installing-debian-on-usb-stick-from.html,
for example.

-- 
regards,
kushal

[toc] | [prev] | [next] | [standalone]

#187075 — Towards a customized netinst {a rephrasing of "Installing the installer"}

My initial goal is a "thingy" which a human at a terminal would not 
notice any operational difference from netinst.iso on a CD.
This "thingy" would:
   1. *NOT* have any internal assumption of being ISO9660 compliant.
   2. intrinsically assume it will be resident on an ext2 device.
   3. be recognizable and boot-able by Grub2.

On 09/19/2017 10:14 AM, Thomas Schmitt wrote
<https://lists.debian.org/debian-user/2017/09/msg00772.html>:
 > Richard Owlett wrote:
 >>>> xorriso -osirrox on ... -extract / /media/richard/netinst1
 >
 > i wrote:
 >>> (This is just one way to copy the directory tree out of the ISO into
 >>> a disk tree. xorriso packs them up and packs them out.)
 >
 >> I assumed that using xorriso on both ends would give me a
 >> "byte for byte" copy.
 >
 > No, that's the job of "dd" or similar copy programs.
 >

I had been attempting to follow the pattern of "Debian-Installer: How to 
modify an existing CD image - Create copy of the image" 
<https://wiki.debian.org/DebianInstaller/Modify/CD#Create_copy_of_the_image>

Using "tar" instead of "bsdtar" gave an error message that "This does 
not look like a tar archive". That was the reason for attempting to use 
xorriso with the assumption it would give a perfect copy of image used 
by the developers/packagers to create netinst.iso .

Re-reading <https://wiki.debian.org/DebianInstaller/Modify/CD> after 
reading < 
http://ftp.nl.debian.org/debian/dists/testing/main/installer-amd64/current/images/MANIFEST> 
suggested at least one reason I could not boot the resulting partition - 
I needed an appropriate set of initrd/vmlinuz.

[toc] | [prev] | [next] | [standalone]

#187079 — Re: Towards a customized netinst

Hi,

Richard Owlett wrote:
> I had been attempting to follow the pattern of "Debian-Installer: How to
> modify an existing CD image - Create copy of the image"
> <https://wiki.debian.org/DebianInstaller/Modify/CD#Create_copy_of_the_image>

That's about what you did with xorriso. I'd call "unpacking".
An alternative would simply be to mount the ISO and use normal file
copy program. Like:

  sudo mkdir /mnt/iso
  sudo mount -o loop debian-9.1.0-i386-netinst.iso /mnt/iso
  cd $HOME
  cp -a /mnt/iso unpacked_iso

To see the resulting file tree:

  find unpacked_iso | less


> That was the reason for attempting to use xorriso
> with the assumption it would give a perfect copy of image

Unpacking an ISO copies onto your disk the directory tree and the files.
It cannot be perfect with ctime but other file attributes from the ISO
can be copied.
Insofar your assumption about xorriso was right. It's just not what i
would call a "byte for byte copy" and also not the hard part of reaching
your goal.

I could agree on "unpacking nearly perfect file copies from ISO to ext2"
to describe this step.


> I needed an appropriate set of initrd/vmlinuz.

Yes.
I expect it is mostly about the programs, scripts, and configuration
files in the initrd that gets normally booted from the ISO.

Possibly a few programs need adaptions later in the course of starting
the installer and finding a first package repository. But since the system
is prepared to find itself on an USB stick, it should not be too hard to
adapt it to a disk partition.
The way to finding the cause of such a stumblestone can be exhausting,
though.


>   1. *NOT* have any internal assumption of being ISO9660 compliant.
>   2. intrinsically assume it will be resident on an ext2 device.

The only plausible addiction to ISO 9660 would be that other filesystem
drivers might be missing in vmlinuz or initrd. So make sure there
is a driver for ext2.


>   3. be recognizable and boot-able by Grub2.

GRUB does not recognize systems by themselves. It can search devices and
partitions with certain properties, up to the existence of a particular
file in that partition. But that is all defined in grub.cfg and the files
which it includes directly or indirectly by statement "source".

The thing which normally recognizes the ISO as bootable and starts the
boot loader is the firmware of the computer: BIOS or EFI.
But if the ISO is in a partition rather than on a whole storage device,
then the boot loader of the machine's hard disk will be started, because
BIOS and EFI only look at whole devices for bootability.

So an USB stick, which shall boot on uprepared machines, needs to carry
its own boot loader and you will have to teach this boot loader how to
boot the stuff in the ext2 which once was stored in an ISO.
You will have to study 
  /boot/grub/x86_64-efi/grub.cfg
and
  /boot/grub/grub.cfg
in the mounted ISO or in the copied directory tree.
You will have to install GRUB with a suitable configuration onto the USB
stick like you would do with a system hard disk. I.e. the installed GRUB
of the USB stick may not depend on a /boot directory on any other device.

If you copy your ext2 on a partition of a non-removable disk, then you
need to adapt the system's GRUB to boot that partition. The learned
knowledge about grub.cfg should be re-usable in that situation.
But you'd need to find the scripts which write the grub.cfg files of
that system. (Direct editing is heavily discouraged.)


Have a nice day :)

Thomas

[toc] | [prev] | [next] | [standalone]

#187092 — Re: Towards a customized netinst

Thomas Schmitt's post of 09/21/2017 09:47 AM prompted me to rephrase my 
initial post of this sub-thread.


My initial goal is a "thingy" which a human at a terminal would not 
notice any operational difference from netinst.iso on a CD.
This "thingy" would:
   1. *NOT* have any internal assumption of being ISO9660 compliant.
      1a. be recognizable and boot-able by Grub2.
        {Re-reading <https://wiki.debian.org/DebianInstaller/Modify/CD>
         after reading
< 
http://ftp.nl.debian.org/debian/dists/testing/main/installer-amd64/current/images/MANIFEST>
         suggested at least one reason I could not boot the resulting
         partition - I needed an appropriate set of initrd/vmlinuz.

         Browsing some Wiki pages indicates I need to read MORE about the
         boot process and os-prober.}
   2. intrinsically assume it will be resident on an ext2 device.

I had been attempting to follow the pattern of "Debian-Installer: How to 
modify an existing CD image - Create copy of the image" 
<https://wiki.debian.org/DebianInstaller/Modify/CD#Create_copy_of_the_image> 
.

Using "tar" instead of "bsdtar" giving an error message that "This does 
not look like a tar archive" was motivation to use xorriso. As Thomas 
Schmitt has pointed out, there are other means. Using it is a better map 
to to my mental image while preventing my mental image clashing to badly 
with reality.

[toc] | [prev] | [standalone]

Back to top | Article view | linux.debian.user

csiph-web