Re: Selective rotation of journald logs

From	Mariusz Gronczewski <xani@devrandom.pl>
Newsgroups	linux.debian.user
Subject	Re: Selective rotation of journald logs
Date	2024-02-23 13:50 +0100
Message-ID	<IaDa1-c4Kk-5@gated-at.bofh.it> (permalink)
References	<IaAOR-c3sL-1@gated-at.bofh.it> <IaCdX-c4b5-5@gated-at.bofh.it> <IaCxk-c4xf-13@gated-at.bofh.it>
Organization	Devrandom.pl

Show all headers | View raw

Dnia 2024-02-23, o godz. 13:02:00
Nicolas George <george@nsup.org> napisał(a):

> Mariusz Gronczewski (12024-02-23):
> > That is not a feature systemd's logging have.  
> 
> That is what it seems, but I would like second opinions.
> 
> >						You'd have to make a
> > rsyslogd rule to put it in one directory  
> 
> Thanks, but my question was about systemd's infrastructure. Answers
> about the old systlog/logrotate infrastructure are a waste of time
> since I already know how they work and they are amply documented
> elsewhere.
> 

So to say it short: It is horrid. There is no any sensible indexing or
split between services or anything. Single spamming service can easily
make it rotate out more important but much more rare log entries and
there is nothing you can do about it.

It is also woefully inefficient, with each "line" of log taking few
times more than it does in text or even if thrown into SQLite database.

For example:

[13:37:01]cthulhu:/var/log/journal☠ ls -s -h /tmp/log.sqlite
79M /tmp/log.sqlite
[13:37:12]cthulhu:/var/log/journal☠ sqlite /tmp/log.sqlite 'select count(*) from log'
386351
journalctl |wc -l
381770
[13:37:48]cthulhu:/var/log/journal☠ journalctl |dd of=/dev/zero bs=1M
0+15442 records in
0+15442 records out
63643115 bytes (64 MB, 61 MiB) copied, 5,47791 s, 11,6 MB/s
du -h /var/log/journal/
337M	/var/log/journal/44cf6f547971fc33309d1e9e000002e7
337M	/var/log/journal/

(I've raised a bug 8 years ago about it https://github.com/systemd/systemd/issues/2460 )

To summarize: the ~61MB of resulting text of logs takes around 337MB 
on disk when saved by journald, ~80MB in sqlite (~103MB when I added 
index for time and process). Use the old ways you know, it's a waste 
of time to try to wrangle journald to do what you want.


-- 
Mariusz Gronczewski (XANi) <xani666@gmail.com>
GnuPG: 0xEA8ACE64
https://devrandom.eu

Thread

Selective rotation of journald logs Nicolas George <george@nsup.org> - 2024-02-23 11:20 +0100
  Re: Selective rotation of journald logs Mariusz Gronczewski <xani@devrandom.pl> - 2024-02-23 12:50 +0100
    Re: Selective rotation of journald logs Nicolas George <george@nsup.org> - 2024-02-23 13:10 +0100
      Re: Selective rotation of journald logs Mariusz Gronczewski <xani@devrandom.pl> - 2024-02-23 13:50 +0100
        Re: Selective rotation of journald logs Nicolas George <george@nsup.org> - 2024-02-23 13:50 +0100
          Re: Selective rotation of journald logs Greg Wooledge <greg@wooledge.org> - 2024-02-23 14:00 +0100
            Re: Selective rotation of journald logs Nicolas George <george@nsup.org> - 2024-02-23 14:10 +0100
              Re: Selective rotation of journald logs Greg Wooledge <greg@wooledge.org> - 2024-02-23 14:10 +0100
                Re: Selective rotation of journald logs Nicolas George <george@nsup.org> - 2024-02-23 14:20 +0100
                Re: Selective rotation of journald logs Mariusz Gronczewski <xani@devrandom.pl> - 2024-02-23 14:30 +0100
              Re: Selective rotation of journald logs Mariusz Gronczewski <xani@devrandom.pl> - 2024-02-23 14:20 +0100
                Re: Selective rotation of journald logs Nicolas George <george@nsup.org> - 2024-02-23 14:30 +0100
                Re: Selective rotation of journald logs Mariusz Gronczewski <xani@devrandom.pl> - 2024-02-23 14:40 +0100
          Re: Selective rotation of journald logs Mariusz Gronczewski <xani@devrandom.pl> - 2024-02-23 14:10 +0100
        Journald's qualities (was: Selective rotation of journald logs) Stefan Monnier <monnier@iro.umontreal.ca> - 2024-02-23 16:30 +0100
          Re: Journald's qualities (was: Selective rotation of journald logs) Mariusz Gronczewski <xani@devrandom.pl> - 2024-02-23 18:20 +0100
            Re: Journald's qualities Stefan Monnier <monnier@iro.umontreal.ca> - 2024-02-23 21:40 +0100
              Re: Journald's qualities The Wanderer <wanderer@fastmail.fm> - 2024-02-24 01:20 +0100
                Re: Journald's qualities Jeffrey Walton <noloader@gmail.com> - 2024-02-24 05:10 +0100
                Re: Journald's qualities Mariusz Gronczewski <xani@devrandom.pl> - 2024-02-24 11:00 +0100
                Re: Journald's qualities Stefan Monnier <monnier@iro.umontreal.ca> - 2024-02-24 05:20 +0100
          Re: Journald's qualities (was: Selective rotation of journald logs) Dan Ritter <dsr@randomstring.org> - 2024-02-23 19:00 +0100
            Re: Journald's qualities Stefan Monnier <monnier@iro.umontreal.ca> - 2024-02-23 21:50 +0100
            Re: Journald's qualities (was: Selective rotation of journald logs) Nicholas Geovanis <nickgeovanis@gmail.com> - 2024-02-23 22:10 +0100
              Re: Journald's qualities (was: Selective rotation of journald logs) Mariusz Gronczewski <xani@devrandom.pl> - 2024-02-24 11:10 +0100
            Re: Journald's qualities (was: Selective rotation of journald logs) Mariusz Gronczewski <xani@devrandom.pl> - 2024-02-24 10:40 +0100
              Re: Journald's qualities Anssi Saari <anssi.saari@debian-user.mail.kapsi.fi> - 2024-02-26 12:20 +0100
                Re: Journald's qualities Mariusz Gronczewski <xani@devrandom.pl> - 2024-02-26 12:40 +0100
                Re: Journald's qualities Jeffrey Walton <noloader@gmail.com> - 2024-02-26 17:10 +0100
                Re: Journald's qualities Anssi Saari <anssi.saari@debian-user.mail.kapsi.fi> - 2024-02-27 10:50 +0100
  Re: Selective rotation of journald logs Max Nikulin <manikulin@gmail.com> - 2024-02-23 16:40 +0100
    Re: Selective rotation of journald logs Max Nikulin <manikulin@gmail.com> - 2024-02-25 04:00 +0100
    Re: Selective rotation of journald logs Nicolas George <george@nsup.org> - 2024-02-26 15:50 +0100

csiph-web