Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.user > #220932 > unrolled thread

Goal: a specialized inventory of installed packages

Back to article view | Back to linux.debian.user

Contents

  Goal: a specialized inventory of installed packages Richard Owlett <rowlett@cloud85.net> - 2020-04-05 17:50 +0200
    Re: Goal: a specialized inventory of installed packages The Wanderer <wanderer@fastmail.fm> - 2020-04-05 18:00 +0200
      Re: Goal: a specialized inventory of installed packages Richard Owlett <rowlett@cloud85.net> - 2020-04-05 19:00 +0200
        Re: Goal: a specialized inventory of installed packages Andrei POPESCU <andreimpopescu@gmail.com> - 2020-04-06 12:50 +0200
      Re: Goal: a specialized inventory of installed packages Andrei POPESCU <andreimpopescu@gmail.com> - 2020-04-06 12:40 +0200
    Re: Goal: a specialized inventory of installed packages David Wright <deblis@lionunicorn.co.uk> - 2020-04-05 19:10 +0200
      Re: Goal: a specialized inventory of installed packages Richard Owlett <rowlett@cloud85.net> - 2020-04-05 20:20 +0200
        Re: Goal: a specialized inventory of installed packages David Wright <deblis@lionunicorn.co.uk> - 2020-04-06 19:10 +0200
    Re: Goal: a specialized inventory of installed packages Marco Möller <talby@debianlists.mobilxpress.net> - 2020-04-05 20:30 +0200
      Re: Goal: a specialized inventory of installed packages Richard Owlett <rowlett@cloud85.net> - 2020-04-06 04:30 +0200

#220932 — Goal: a specialized inventory of installed packages

I moved from WindowsXP when Squeeze was the current release.
In the first year I did *many* installs from scratch to determine what I 
wanted in a final system (made much use of preseeding).

I currently have a configuration of Stretch that meets most of my needs.
As the installation was performed using the default Debian installer, 
there is a plethora of packages of no interest installed. As some 
important packages are not installed by default, apt and Synaptic were 
used to install them.

I'm setting out to do an _extremely_ custom *minimal* install of Buster.
The desired inventory shall list *ONLY* top level packages.
[ E.G. if gfortran was purposely installed, the ~dozen packages 
installed because they were tagged as depends, recommends, or suggests 
wold *NOT* be listed. ]

Is there a suitable tool? [Synaptic's History menu item is closest I've 
found].

TIA

[toc] | [next] | [standalone]

#220934

[Multipart message — attachments visible in raw view] — view raw

On 2020-04-05 at 11:30, Richard Owlett wrote:

> I moved from WindowsXP when Squeeze was the current release. In the
> first year I did *many* installs from scratch to determine what I 
> wanted in a final system (made much use of preseeding).
> 
> I currently have a configuration of Stretch that meets most of my
> needs. As the installation was performed using the default Debian
> installer, there is a plethora of packages of no interest installed.
> As some important packages are not installed by default, apt and
> Synaptic were used to install them.
> 
> I'm setting out to do an _extremely_ custom *minimal* install of
> Buster. The desired inventory shall list *ONLY* top level packages. [
> E.G. if gfortran was purposely installed, the ~dozen packages 
> installed because they were tagged as depends, recommends, or
> suggests wold *NOT* be listed. ]

This sounds like a job for apt's "manual" vs. "auto" status flag.

'apt-mark showmanual' will list all the packages which are flagged as
being manually installed.

Specifying a package explicitly to 'apt-get install' or similar flags it
as manually installed, if you actually go through and complete the
installation (cancelling the install also cancels the flag). Any
packages which it pulls in as dependencies - whether "Depends:"
"Recommends", or "Suggests:" - get flagged as "auto" instead.

From what I've seen, it looks as if debian-installer also flags some
packages as manually installed, during initial install of the Debian
system. I don't know which ones do and don't get that treatment.

I don't know if that's good enough for you, but it's very likely the
best you're going to get.

-- 
   The Wanderer

The reasonable man adapts himself to the world; the unreasonable one
persists in trying to adapt the world to himself. Therefore all
progress depends on the unreasonable man.         -- George Bernard Shaw

[toc] | [prev] | [next] | [standalone]

#220936

On 04/05/2020 10:52 AM, The Wanderer wrote:
> On 2020-04-05 at 11:30, Richard Owlett wrote:
> 
>> I moved from WindowsXP when Squeeze was the current release. In the
>> first year I did *many* installs from scratch to determine what I
>> wanted in a final system (made much use of preseeding).
>>
>> I currently have a configuration of Stretch that meets most of my
>> needs. As the installation was performed using the default Debian
>> installer, there is a plethora of packages of no interest installed.
>> As some important packages are not installed by default, apt and
>> Synaptic were used to install them.
>>
>> I'm setting out to do an _extremely_ custom *minimal* install of
>> Buster. The desired inventory shall list *ONLY* top level packages. [
>> E.G. if gfortran was purposely installed, the ~dozen packages
>> installed because they were tagged as depends, recommends, or
>> suggests wold *NOT* be listed. ]
> 
> This sounds like a job for apt's "manual" vs. "auto" status flag.
> 
> 'apt-mark showmanual' will list all the packages which are flagged as
> being manually installed.

*YES* THANK YOU!

> 
> Specifying a package explicitly to 'apt-get install' or similar flags it
> as manually installed, if you actually go through and complete the
> installation (cancelling the install also cancels the flag). Any
> packages which it pulls in as dependencies - whether "Depends:"
> "Recommends", or "Suggests:" - get flagged as "auto" instead.
> 
>  From what I've seen, it looks as if debian-installer also flags some
> packages as manually installed, during initial install of the Debian
> system. I don't know which ones do and don't get that treatment.

I did a test run. I think I see the pattern of which packages it marks 
as manual.
E.G. It shows systemd related items as "manual". But for *MY* purposes I 
would class them as "auto". That will not be a problem in practice. I'll 
just do what the installer thinks is an absolute minimal install and run 
'apt-mark showmanual' against it. I'll save that list. Then create a 
script to ignore those lines when 'apt-mark showmanual' is run against 
my actual system.

> 
> I don't know if that's good enough for you, but it's very likely the
> best you're going to get.
> 

It looks like a good match so far. Thanks again.

[toc] | [prev] | [next] | [standalone]

#220980

[Multipart message — attachments visible in raw view] — view raw

On Du, 05 apr 20, 11:59:17, Richard Owlett wrote:
> 
> I did a test run. I think I see the pattern of which packages it marks as
> manual.
> E.G. It shows systemd related items as "manual". But for *MY* purposes I
> would class them as "auto". That will not be a problem in practice. I'll
> just do what the installer thinks is an absolute minimal install and run
> 'apt-mark showmanual' against it. I'll save that list. Then create a script
> to ignore those lines when 'apt-mark showmanual' is run against my actual
> system.

Something I used to do for similar purposes[1] was to mark everything as 
auto installed and then mark as manual the packages I was actually using 
(typically things like mutt, GUI/TUI browser, etc.).

This was done using aptitude's interactive mode (not sure if apt-mark 
even existed at the time), as I could easily see what aptitude wanted to 
keep or remove.

If you intend to also *remove* packages based on this approach do mind 
the settings for "APT::AutoRemove::RecommendsImportant" and 
"APT::AutoRemove::SuggestsImportant".

[1] I stopped doing this because I found the optimization was not worth 
the time spent.

Kind regards,
Andrei
-- 
http://wiki.debian.org/FAQsFromDebianUser

[toc] | [prev] | [next] | [standalone]

#220979

[Multipart message — attachments visible in raw view] — view raw

On Du, 05 apr 20, 11:52:32, The Wanderer wrote:
> 
> From what I've seen, it looks as if debian-installer also flags some
> packages as manually installed, during initial install of the Debian
> system. I don't know which ones do and don't get that treatment.

At least the packages installed during the 'debootstrap' phase. Possibly 
also those installed by tasks (if one is selected) as well as linux 
image(s) and bootloader (grub).

Kind regards,
Andrei
-- 
http://wiki.debian.org/FAQsFromDebianUser

[toc] | [prev] | [next] | [standalone]

#220937

On Sun 05 Apr 2020 at 10:30:41 (-0500), Richard Owlett wrote:

> I currently have a configuration of Stretch that meets most of my needs.

> I'm setting out to do an _extremely_ custom *minimal* install of Buster.
> The desired inventory shall list *ONLY* top level packages.
> [ E.G. if gfortran was purposely installed, the ~dozen packages
> installed because they were tagged as depends, recommends, or suggests
> wold *NOT* be listed. ]
> 
> Is there a suitable tool? [Synaptic's History menu item is closest
> I've found].

I assume that what you're after is a command like
  apt-get install a b c d …
where the list is the shortest that would install the said system.

In which case, I would presume the answer is no.

But it's not too onerous to write a script to parse the status file
(dpkg-query does the grunt work) and determine which packages are
not a [pre-]depends/recommends/suggests of any other. You may or
may not want to prune the resulting list, removing the names of any
packages that the d-i has no option but to install itself.

How kindly apt will take to be presented with such a list of packages
for installation at one sitting, I don't know.

Cheers,
David.

[toc] | [prev] | [next] | [standalone]

#220940

On 04/05/2020 12:00 PM, David Wright wrote:
> On Sun 05 Apr 2020 at 10:30:41 (-0500), Richard Owlett wrote:
> 
>> I currently have a configuration of Stretch that meets most of my needs.
> 
>> I'm setting out to do an _extremely_ custom *minimal* install of Buster.
>> The desired inventory shall list *ONLY* top level packages.
>> [ E.G. if gfortran was purposely installed, the ~dozen packages
>> installed because they were tagged as depends, recommends, or suggests
>> wold *NOT* be listed. ]
>>
>> Is there a suitable tool? [Synaptic's History menu item is closest
>> I've found].
> 
> I assume that what you're after is a command like
>    apt-get install a b c d …
> where the list is the shortest that would install the said system.
> 
> In which case, I would presume the answer is no.
> 
> But it's not too onerous to write a script to parse the status file
> (dpkg-query does the grunt work) and determine which packages are
> not a [pre-]depends/recommends/suggests of any other. You may or
> may not want to prune the resulting list, removing the names of any
> packages that the d-i has no option but to install itself.
> 
> How kindly apt will take to be presented with such a list of packages
> for installation at one sitting, I don't know.
> 
> Cheers,
> David.

I wasn't aiming for tool to automajically recreate an existing system.
I was going more for a tool to allow me to visualize specifying what my 
new system would/should look like.
'apt-mark ...' seems to be aimed at my goals.
Thanks.

[toc] | [prev] | [next] | [standalone]

#221013

On Sun 05 Apr 2020 at 13:17:37 (-0500), Richard Owlett wrote:
> On 04/05/2020 12:00 PM, David Wright wrote:
> > On Sun 05 Apr 2020 at 10:30:41 (-0500), Richard Owlett wrote:
> > 
> > > I currently have a configuration of Stretch that meets most of my needs.
> > 
> > > I'm setting out to do an _extremely_ custom *minimal* install of Buster.
> > > The desired inventory shall list *ONLY* top level packages.
> > > [ E.G. if gfortran was purposely installed, the ~dozen packages
> > > installed because they were tagged as depends, recommends, or suggests
> > > wold *NOT* be listed. ]
> > > 
> > > Is there a suitable tool? [Synaptic's History menu item is closest
> > > I've found].
> > 
> > I assume that what you're after is a command like
> >    apt-get install a b c d …
> > where the list is the shortest that would install the said system.
> > 
> > In which case, I would presume the answer is no.
> > 
> > But it's not too onerous to write a script to parse the status file
> > (dpkg-query does the grunt work) and determine which packages are
> > not a [pre-]depends/recommends/suggests of any other. You may or
> > may not want to prune the resulting list, removing the names of any
> > packages that the d-i has no option but to install itself.
> > 
> > How kindly apt will take to be presented with such a list of packages
> > for installation at one sitting, I don't know.
> 
> I wasn't aiming for tool to automajically recreate an existing system.
> I was going more for a tool to allow me to visualize specifying what
> my new system would/should look like.
> 'apt-mark ...' seems to be aimed at my goals.
> Thanks.

Well, it does appear from your response, "I'll just do what the
installer thinks is an absolute minimal install and run 'apt-mark
showmanual' against it", that you're still enjoying doing installs,
so you can probably iterate towards a solution by by trying your
minimal list and seeing what results (comparing it with your
"reference" installation).

And it does appear from your other response that minimalism is more
important than function, if you exclude Recommends.

But in any case, I was assuming you wanted a reverse tool, to work out
which packages on your system are top-level, in the sense that they
are not dependent on any other package that you've installed.

I've just cannibalised a function that I use for kind of similar
purposes, because with being kept at home it seemed a good time to try
out bash's associative arrays. (I've always dropped into Python in the
past.) You might want to try it out: its default is to include
Recommends, but you can generate a longer list by using strict Depends.

It doesn't handle alternatives ( | is treated as , ) and it ignores
Pre-Depends. It also works ab initio, as if you had to drive the
d-i selection yourself. But one could modify it to merge in a
dpkg-query listing from a system that's untouched post-install.

It uses trash files (as I often do) rather than pipes so that you can
inspect partial results. As I say, it was just to get my hands dirty
with bash associative arrays. Strictly, they're not functionally
required here, but they're meant to be fast to search than a list
of an indexed array. Anyway:

function my-packages-toplevels {
    [ "$1" = "-?" ] && printf '%s\n' "Usage:	$FUNCNAME [Depends-only]
	lists the packages installed on this system that are not dependent (and,
	if lacking any argument, not recommended) by any other package." >&2 && return 1
    local -A Depsrecs
    local Pkg Format=' ${Depends} ${Recommends} \n'
    [ -n "$1" ] && Format=' ${Depends} \n'
    local Unique1="$(mktemp ${Uniquetrash:-/tmp}/$FUNCNAME-"$(date +%s)"-1some-XXXX)"
    local Unique2="$(mktemp ${Uniquetrash:-/tmp}/$FUNCNAME-"$(date +%s)"-2all-XXXX)"
    dpkg-query -W --showformat "$Format" | sed -e 's/([^)]\+)//g;s/,/ /g;s/|/ /g;s/[ ]\+/ /g;s/ /\n/g;' | sort -u | tail -n +2 >| "$Unique1"
    while IFS=$'\n' read -r Pkg; do
	Depsrecs["$Pkg"]="yes"
    done < "$Unique1"
    dpkg-query -W --showformat '${Package}\n' | sort >| "$Unique2"
    while IFS=$'\n' read Pkg; do
	[[ ! -n "${Depsrecs[$Pkg]}" ]] && printf '%s\n' "$Pkg"
    done < "$Unique2"
    return 0
}

Cheers,
David.

[toc] | [prev] | [next] | [standalone]

#220941

Once you have your list of packages in a text file, for each package one 
line, you could apply the list like this:

     apt install $(< mylist.txt)

Consider to first do a simulation run for finding problems in the list:
     -s

Consider to use the following flag in order to not draw in a maybe 
gigantic tail of not really wanted stuff:
     --no-install-recommends

If you from time to time would like to include some package from the 
maybe gigantic tail of recommends, then simply add it individually to 
the list!

I maintain manually the list, which you are about to generate now, and 
have very good experience starting with the following complete command:
     apt install -s --no-install-recommends $(< mylist.txt)

When installing the basic Debian system from the official media,
you may want to do this in mode "expert install", and you may want to 
know that all offered graphical Desktop Environments are full fledged 
installs coming with many (IMHO too much!) applications, while 
afterwards going for a manual install of the desktop environment outside 
of the Debian installer will allow you to go for a fine grained 
"minimal" Desktop Environment installation only coming with the really 
needed functionality and without the applications.
I therefore recommend to have your personally selected packages for the 
wished Desktop Environment (KDE, Gnome, XFCE, etc.) included in your 
personal list, and when arriving at the Debian installer's "tasksel" 
menu do not only deactivate all the listed Desktop Environments, which 
become offered, but even do deactivate the head entry “Debian desktop 
environment” itself, otherwise this would install a full featured GNOME 
desktop although below in the list you have had deactivated GNOME. I 
usually only select here “standard system utilities” and “print server”, 
and then have all the needed packages for "my" Desktop Environment in my 
list.
If following this path, then be aware that for a laptop essential or 
nice to have tools like wpasupplicant or network-manager (and many 
more!) will not become automatically installed for you! Setting up your 
personal list also concerning the Desktop Environment manually will thus 
require some extended effort. Unfortunately, the Debian installer is 
only offering to install full fledged Desktop Environments, but not 
minimal versions of it.

Good luck! Marco.

[toc] | [prev] | [next] | [standalone]

#220954

On 04/05/2020 01:29 PM, Marco Möller wrote:
> Once you have your list of packages in a text file, for each package one 
> line, you could apply the list like this:
> 
>      apt install $(< mylist.txt)

Good to know that is known to work.

> 
> Consider to first do a simulation run for finding problems in the list:
>      -s
> 
> Consider to use the following flag in order to not draw in a maybe 
> gigantic tail of not really wanted stuff:
>      --no-install-recommends

That had been my intention for that reason.
A few years ago someone suggested additional methods to install a 
minimum number of packages. I'll have to search my old emails - possibly 
on a semi-retired machine.

> 
> If you from time to time would like to include some package from the 
> maybe gigantic tail of recommends, then simply add it individually to 
> the list!
> 
> I maintain manually the list, which you are about to generate now, and 
> have very good experience starting with the following complete command:
>      apt install -s --no-install-recommends $(< mylist.txt)
> 
> When installing the basic Debian system from the official media,
> you may want to do this in mode "expert install",

That has been my procedure.

> and you may want to 
> know that all offered graphical Desktop Environments are full fledged 
> installs coming with many (IMHO too much!) applications,

You're preaching to the choir ;/

> while 
> afterwards going for a manual install of the desktop environment outside 
> of the Debian installer will allow you to go for a fine grained 
> "minimal" Desktop Environment installation only coming with the really 
> needed functionality and without the applications.
> I therefore recommend to have your personally selected packages for the 
> wished Desktop Environment (KDE, Gnome, XFCE, etc.) included in your 
> personal list, and when arriving at the Debian installer's "tasksel" 
> menu do not only deactivate all the listed Desktop Environments, which 
> become offered, but even do deactivate the head entry “Debian desktop 
> environment” itself, otherwise this would install a full featured GNOME 
> desktop although below in the list you have had deactivated GNOME. I 
> usually only select here “standard system utilities” and “print server”, 
> and then have all the needed packages for "my" Desktop Environment in my 
> list.

I've a collection of experimental preseed.cfg files aimed at that.

> If following this path, then be aware that for a laptop essential or 
> nice to have tools like wpasupplicant or network-manager (and many 
> more!) will not become automatically installed for you! Setting up your 
> personal list also concerning the Desktop Environment manually will thus 
> require some extended effort. Unfortunately, the Debian installer is 
> only offering to install full fledged Desktop Environments, but not 
> minimal versions of it.

Since I saw a site that would create a custom preseed.cfg in response to 
a series of questions resembling those the standard installer asks. That 
site is obsolete as it predates Squeeze IIRC. In any case I would do it 
in TCL/Tk as a educational coding exercise.

> 
> Good luck! Marco.
> 
> 
> 

Thank you.

[toc] | [prev] | [standalone]

Back to top | Article view | linux.debian.user

csiph-web