Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.user > #176167 > unrolled thread

nosh version 1.30

Back to article view | Back to linux.debian.user

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  nosh version 1.30 Jonathan de Boyne Pollard <J.deBoynePollard-newsgroups@NTLWorld.com> - 2017-01-01 01:20 +0100
    nosh version 1.31 Jonathan de Boyne Pollard <J.deBoynePollard-newsgroups@NTLWorld.com> - 2017-01-14 12:50 +0100
      nosh version 1.32 Jonathan de Boyne Pollard <J.deBoynePollard-newsgroups@NTLWorld.com> - 2017-01-30 10:30 +0100
        nosh version 1.33 Jonathan de Boyne Pollard <j.deboynepollard-newsgroups@ntlworld.com> - 2017-04-09 22:10 +0200

#176167 — nosh version 1.30

The nosh package is now up to version 1.30 .

* http://jdebp.eu./Softwares/nosh/
* 
https://www.freebsd.org/news/status/report-2015-07-2015-09.html#The-nosh-Project
* http://jdebp.info./Softwares/nosh/

service bundles
---------------

As usual, there are more service bundles, including for the UWSGI 
"Emperor" and the new services in FreeBSD/TrueOS 11 such as ypldap. 
There are now services to run Sendmail in the same manner as the 
services that run exim.  Note that this is slightly different to the old 
FreeBSD division of labour.  There are individually controllable 
services for SMTP Submission, SMTP Relay, the Submission queue runner, 
and the Relay queue runner.

doco
----

The Guide has been extended with several new chapters, including a 
gazetteer of interesting directories, a chapter on log file 
post-processing, a chapter on logging security, a chapter on per-user 
service management, and some notes for individual services.  The 
commands list has moved from the blurb into the Guide, too, as it seems 
like something that an administrator might find handy to have available 
when there's no Internet connection.

* http://jdebp.eu./Softwares/nosh/guide.html

service management
------------------

There's now a hardlimit chain-loading command, analogous to softlimit.  
The convert-systemd-services utility now makes use of this and permits 
setting separate hard and soft limits, or only one or the other, with 
settings like LimitOFILE=32:128 and LimitNPROC=:infinity .

There's now a local-reaper chain-loading command, that can turn "local 
reaper" status for the current process on or off.  Have a care when 
using this, per the note on the manual page.  There is a 
LocalReaper=true extension to systemd service units for this.

netlink-datagram-socket-listen is now available on the BSDs for script 
compatibility.  It always aborts with an address family error.

There's a new hangup subcommand of system-control, equivalent to the 
existing -H option to svc .

enhancements to system-control stop/start/reset and single-shot services
------------------------------------------------------------------------

This is the first big item for 1.30 :

The start and stop subcommands of system-control now operate more 
quickly.  Instead of polling once per second, they monitor the 
supervise/status files of each service that is in the process of being 
started and stopped, with kevent().

In addition, system-control now supports the notion of services that 
become ready when their main process has exited, marked with a new flag 
file in the service directory.  convert-systemd-units has been modified 
to convert "oneshot" services to this, instead of to services that put 
all of the run code into the start program.  Thus "oneshot" services 
that are running their actual main programs are reported as "running" by 
svstat, rather than as "starting".

This takes advantage of the extended status information that 
service-manager has been writing to the status file since version 1.28.  
The sharp-eyed may have noticed that in version 1.28 the output of 
"svstat"/"system-control status" gained information about the exit 
statuses of the start, run, restart, and stop programs. This is what 
system-control now uses to detect whether ready-after-run services ran 
before they stopped.  (Detection of ready-after-run services that are 
running with no processes, because they are "remain" services, can be 
and is done with just the daemontools-encore-compatible status information.)

Old-style "oneshot"s will continue to work as before, as of course they 
become ready as soon as the run process is spawned, which is after they 
have run their programs as part of start.

The benefit of this new style, apart from reporting a running service as 
actually "running", which should help with nagios monitoring and the 
like, is that "oneshot" services converted from systemd no longer have 
to be marked as RemainAfterExit=true in order to avoid a dummy "pause" 
process hanging around.  This is the case for old-style "oneshot" 
services.  They have to run something in run, after all, and that 
something has to keep running in order for the service to be considered 
ready and services ordered after it to be unblocked.  A ready-after-run 
service, however, unblocks ordered-after services if it has reached the 
stopped state via a run, thus puts its programs in run, thus doesn't 
have to have a dummy pause process, and can be RemainAfterExit=false 
without adding to the process list.

log file management
-------------------

export-to-rsyslog had a bug that caused it to skip old log files (the 
@nnnnnnnnnnnnnnnnnnnnnnnn.s ones) in catch-up mode.  This has been 
corrected.  There is now a follow-log-directories command that can 
substitute for tail -F .  It knows the actual structure of log 
directories, operates using one or more cursors like export-to-rsyslog 
does, and copes correctly with cyclog/multilog log rotation (which GNU 
tail, at least, apparently has problems with when the timing is 
particularly wrong on a loaded system).

See also http://jdebp.eu./FGA/do-not-use-logrotate.html

build
-----

More warnings are now turned on with clang++ during the build, and a lot 
of the resultant warnings have been eliminated where appropriate.  The 
check for eg++ in preference to g++ is now limited to OpenBSD, where (at 
least on OpenBSD 5.9) eg++ is still ahead of g++ by a wide margin.

Per-user service management
---------------------------

Changes in per-user service management are the second big item for 1.30 :

The per-user service manager instances are now invoked via userenv, so 
all per-user services that you run under nosh service management, D-BUS 
servers or otherwise, will have your own HOME, SHELL, and USER set.  
Several per-user daemon softwares were expecting HOME to be set.

To match what the Desktop Bus people are doing, the dbus socket path for 
the per-user D-BUS broker has changed from 
"/run/user/$USER/dbus/user_bus_socket" to "/run/user/$USER/bus".  In 
theory, this is addressable (in D-BUS speak) as "unix:runtime=yes". In 
practice, there is no version of D-BUS available on stable/release 
FreeBSD, TrueOS, or Debian that understands this address syntax.  So one 
still has to use "unix:path=/run/user/$USER/bus".

The Desktop Bus people and the desktop environments people are also 
switching from per-login D-BUS brokers to per-user D-BUS brokers. The 
nosh toolset has already had this for over a year, since the middle of 
2015.  Each real-person user account has an optional per-user service 
management service (e.g. user-services@fred).  What is new is that 
per-user service bundle areas are now populated with a whole load of 
service bundles for real services, many relating to GUI desktop 
environments, and the per-user D-BUS broker has moved to there, from 
being a system-level service bundle.

The configuration import subsystem creates these new per-user service 
bundles in the home directories of individual real users, under 
~fred/.config/service-bundles/services/ and 
~fred/.config/service-bundles/targets/ (for user fred).  These run 
per-user services for a whole load of things, from GNOME editor and 
emacs through dconf and KDE Notify to urxvtd and GNOME Terminal.

The configuration import subsystem also sets up a bypass for D-BUS's 
broken "bus activation" mechanism, so that instead of attempting to run 
these D-BUS servers directly, the D-BUS broker instead tells the nosh 
per-user service manager to run them.  This takes the form of a 
replacement dbus-daemon-launch-helper, and the per-user D-BUS brokers 
now employ a modified configuration file that invokes it.

There's a full explanation of how this all works in the new chapter on 
demand-starting user-level Desktop Bus services in the nosh Guide.

Notes:
   * For emacs as a per-user service, you must have a very recent emacs 
with its very-late-to-the-party --new-daemon option.
   * GNOME Weather and its interaction with GeoClue2 are only partly 
tested, because the versions of them available for the test platforms 
were attempting to contact a weather service that the U.S. Government 
discontinued in June 2016; and this was hardwired into their code.

[toc] | [next] | [standalone]

#176680 — nosh version 1.31

The nosh package is now up to version 1.31 .

* http://jdebp.eu./Softwares/nosh/

* 
https://www.freebsd.org/news/status/report-2015-07-2015-09.html#The-nosh-Project

* http://jdebp.info./Softwares/nosh/

This release fixes a problem with emergency mode that was introduced by 
accident in 1.29 .  The emergency-login@console service was not properly 
enabled by package installation.  Now it once again is.

There are a number of bug fixes in this release, such as rare corner 
cases in how convert-systemd-units generates arguments to pass to sh, 
what port the nginx server part of Appcafe binds to when not the 
default, the use of setuidgid-fromenv to set more than 1 supplementary 
group ID, and making the Makefile in tinydns@* services work with both 
BSD and GNU make.  Various service bundles that perform 
clean-up-directories actions at bootstrap have been made more difficult 
to accidentally re-trigger after bootstrap.

There is also a fair amount of new features:

* The automatically-generated data for tinydns@* services now 
encompasses all of the reverse lookup domain names for private/local IP 
addresses, so none of the DNS traffic involving such lookups will leak 
out of your machine/organization to the rest of Internet.

* The userenv command has gained the ability to (optionally) set a whole 
lot more environment variables from the capabilities in /etc/login.conf 
and ~/.login_conf .  It now can be used as the 
setup-the-user-environment part of a command chain that is designed to 
perform the setup of an interactive login session. This is particularly 
useful for fixing PCDM, the display manager in TrueOS.

* The pipe command can now arrange to clean up the child process in one 
of two ways.  This is made use of in the dnscache service bundles, and 
dnscache services no longer contain the perpetual zombie process that 
they had in version 1.30 .

* Presets now support wildmat-style character set wildcards. e.g. one 
can now write "ttylogin@vc[0-9]-tty" as a service name pattern.

* If you have been using the --verbose option to the start/stop/reset 
subcommands of system-control, you'll notice that it now colourizes its 
output.  Its output has also been adjusted to more clearly indicate 
blocked services and what they are blocked by.

The big item is that there is now a complete set of simple control 
groups manipulation commands, the pre-supplied service bundles all make 
use of it, and all service bundles created by convert-systemd-units make 
use of it.  (All of this is a no-op on FreeBSD/TrueOS and OpenBSD, of 
course.)

If you've read the Linux doco, you'll know that control groups do not 
require any sort of centralized gatekeeper process, and are a 
decentralized system that can be driven with just the echo command.  In 
practice, using echo is non-trivial.  The move-to-control-group, 
delegate-control-group-to, and set-control-group-knob commands take the 
hassle out of working out exactly what to echo where.  They do all of 
the hard work of determining what the directory name of the current 
control group under /sys/fs/cgroup is, and present a simple system 
allowing one to create and navigate to another control group, delegate 
control over the current control group (and its subgroups) to an 
unprivileged user, and set control group knobs.

The set-control-group-knob utility further illustrates the convenience 
functionality over and above a simple echo command. It can calculate a 
knob setting as a percentage of another number, handle SI and IEEE/IEC 
multiplier suffixes, and translate the device file names that are 
(comparatively) convenient for humans into the literal major and minor 
device numbers that the Linux control groups API actually operates in 
terms of.

There are new chapters in the Guide covering the automatic import of 
FreeBSD 9 and PC-BSD Warden jails, how jailing services on 
FreeBSD/TrueOS works, and limiting services.  The limiting services 
chapter covers both the original Unix resource limits system and Linux 
control groups.

[toc] | [prev] | [next] | [standalone]

#177130 — nosh version 1.32

The nosh package is now up to version 1.32 .

* http://jdebp.eu./Softwares/nosh/

* 
https://www.freebsd.org/news/status/report-2015-07-2015-09.html#The-nosh-Project

* http://jdebp.info./Softwares/nosh/

This release fixes two problems with Gentoo Linux (control group version 
detection and a problem with mounting API filesystems) that we hashed 
out on the Supervision mailing list.  It furthermore contains a change 
to the way that convert-systemd-units generates service bundles that 
fixes problems with control group setup when the service unit defines a 
"slice" for the service or when the service unit is a template. In 
furtherance of that there's a new create-control-group command.

Other things in this release include improvements to the (unpackaged) Z 
Shell command-line completions, which now display option completion 
menus properly; some improvements to the Terminals chapter in the Guide; 
fixes to various service bundles that were using shell reserved words 
and operators such as "for" and "&&" without explicitly invoking the 
shell; additions to userenv for setting DBus and XDG Runtime variables; 
and a fix that prevents "system-control reset" from looping indefinitely 
when run by an unprivileged user such as "messagebus" that lacks access 
to the control/status API.

The major improvement in this release, though, is to console-fb-realizer 
on TrueOS.

FreeBSD gives console-fb-realizer uhid device files to use for input 
devices, which speak the USB HID report protocol and which 
console-fb-realizer has been happy with for a long time.  TrueOS 
provides either ums/ukbd devices, which lack various features because 
they speak the old sysmouse and atkbd protocols, or ugen devices.  There 
are no uhid devices available. console-fb-realizer can now use the ugen 
devices.  Moreover, it will detach the ums/ukbd drivers from the ugen 
devices using the new detach-kernel-usb-driver command, so that there 
aren't two things both attempting to read HID reports.

console-fb-realizer also now correctly sets the keyboard LEDs on both 
FreeBSD and TrueOS.

There have been several minor adjustments to the kernel VT sharing parts 
of console-fb-realizer, preparatory to splitting the program up into 
separate parts for input and output devices, permitting things such as 
multiple keyboards each with its own keyboard map and numlock semantics, 
in a future release.

[toc] | [prev] | [next] | [standalone]

#179938 — nosh version 1.33

The nosh package is now up to version 1.33 .

* http://jdebp.eu./Softwares/nosh/
*
https://www.freebsd.org/news/status/report-2015-07-2015-09.html#The-nosh-Project
* http://jdebp.info./Softwares/nosh/

This has been held back because of work being done by someone else.  I don't
want to steal xyr thunder, so I'll leave the announcement of that work to xem.
 Suffice it to say that it will interest a new group of people.

There are several major improvements in 1.33 .

Packaging
---------

In the version 1.29 announcement I said that the Debian packaging system was
going to be brought into line with the system used for FreeBSD/TrueOS and
OpenBSD.  This is now done.  Debian and the BSDs all now use a similar system
for generating each package manager's package maintenance instructions from an
abstract package description.

==============================================================
=========== IMPORTANT UPGRADE NOTE FOR Debian: ===============
==============================================================

An important consequence of the aforementioned is that the semantics of the
nosh-bundles package have changed. In earlier versions, the various nosh-run-*
packages were how one set services running, except for a small rump set of
services that were set up by the nosh-bundles package.

This is now no longer the case. The nosh-bundles package now presets and starts
no services at all. *All* running of services must be achieved with the
nosh-run-* packages or some other sets of scripts and presets.

To this end, there are now two new packages, nosh-run-debian-desktop-base and
nosh-run-debian-server-base. These parallel the
nosh-run-{freebsd,trueos}-{desktop,server}-base packages already available since
1.29 for FreeBSD/TrueOS. You must install, for a working fully-nosh-managed
system, exactly one of the nosh-run-debian-{desktop,server}-base packages.

If you are running nosh service management under systemd, you can of course run
as many or as few services under the nosh service manager as you care to switch
over from systemd. But if you are running a fully-nosh-managed system these
packages will arrange to run the various fundamentals that one pretty much
cannot do without, such as mounting/unmounting volumes, running
udev/eudev/vdev/mdev, binfmt loading, and initializing the PRNG.

Log service account names
-------------------------

The naming scheme used for the user accounts for dedicated log service users has
changed.  Installing the new nosh-bundles package should automatically rename
all existing log service accounts to use the new scheme.

The new naming scheme is slightly more compact, and copes better with services
that have things like underscores and plus characters (e.g. powerd++) in their
names.

As an ancillary to this, system-control now has an "escape" subcommand which can
be (and indeed is) used in scripts to perform the escaping transformations.

More packages
-------------

There are now four more -shims packages, for commands whose names conflict with
commands from other packages: nosh-kbd-shims, nosh-bsd-shims, nosh-core-shims,
and nosh-execline-shims.

nosh-kbd-shims, for example, contains a chvt shim that is an alias for the (also
new) console-multiplexor-control command; with it, and suitable privileges to
access the virtual terminal's input queue, one can switch between multiplexed
user-space virtual terminals in much the same way as the old chvt command does
with kernel virtual terminals.

The Z Shell command-line completion for the various commands in the toolset
(system-control, svcadm, shutdown, svstat, and so forth), which has been
available to the people building from source for a while, is now also available
as a binary package.

Configuration import
--------------------

ldconfig on TrueOS is now properly handled.  In particular, the external
configuration import subsystem now correctly pulls in and converts all of the
ldconfig directories.  (TrueOS has a lot more things that require ldconfig
support than stock FreeBSD does.)

The configuration import subsystem also now handles instances of Percona server,
alongside MySQL and MariaDB.  Moreover, these are now handled by the same set of
service bundles, which always produce service bundles named mysql@*.  MySQL
version 5.7 or later is now assumed.

The configuration import subsystem now automatically generates OpenVPN service
bundles based upon the current OpenVPN configuration.

=======================
==== CAVE: OpenVPN ====
=======================

The upgrade process attempts to remove the old hardwired openvpn@server and
openvpn@client service bundles.  However, you might encounter remnants of these
service bundles lying around in /var/sv that you will find that you need to
clean up by hand.

GOPHER
------

To accompany the new gopherd server in djbwares 5, there is a gopher6d service
bundle that runs it, serving up the same static files area as http6d, https6d,
and ftp4d do.

The FreeBSD, OpenBSD, and Debian package repositories can now be browsed with
GOPHER.  This is gopherd in action.  On the server side, generating the
index.gopher files is a fairly humdrum exercise in the use of redo (to
regenerate the indexes only when the directory contents change) and printf (to
construct the GOPHER format menus).

UCSPI-UNIX
----------

Two new UCSPI tools have been added to enable UCSPI-UNIX servers to listen on
and accept connections on AF_UNIX sequential packet sockets.  udevd is one such
server, and it is now handed its listening socket at startup rather than
expected to open its own.

[toc] | [prev] | [standalone]

Back to top | Article view | linux.debian.user

csiph-web