Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.user > #274051 > unrolled thread

Popularity contest files in /var/log

Back to article view | Back to linux.debian.user

Contents

  Popularity contest files in /var/log Bruno Schneider <boschneider@gmail.com> - 2024-10-25 21:00 +0200
    Re: Popularity contest files in /var/log Andy Smith <andy@strugglers.net> - 2024-10-25 21:10 +0200
      Re: Popularity contest files in /var/log Michel Verdier <mv524@free.fr> - 2024-10-26 15:50 +0200
      Re: Popularity contest files in /var/log Bruno Schneider <boschneider@gmail.com> - 2024-10-29 15:10 +0100
        Re: Popularity contest files in /var/log Andy Smith <andy@strugglers.net> - 2024-10-29 16:40 +0100
          Re: Popularity contest files in /var/log Bruno Schneider <boschneider@gmail.com> - 2024-10-29 21:30 +0100

#274051 — Popularity contest files in /var/log

I noticed that I have hundreds of files whose names start with
"popularity-contest" in /var/log. They don't even seem to be logs.
They seem to be data that popularity-contest sends away.

Has anyone else noticed this? Is this a bug? It seems they are in the
wrong place or, at least, logrotate should be erasing the old ones.

-- 
Bruno Schneider

[toc] | [next] | [standalone]

#274055

Hi,

On Fri, Oct 25, 2024 at 03:54:58PM -0300, Bruno Schneider wrote:
> I noticed that I have hundreds of files whose names start with
> "popularity-contest" in /var/log. They don't even seem to be logs.
> They seem to be data that popularity-contest sends away.

popularity-contest does log to the file /var/log/popularity-contest,
which is rotated by logrotate.

However, if popularity-contest experiences a problem during submit then
it does leave behind a file named like /var/log/popularity-contest.12345
(and possibly another with .gpg on the end) where "12345" was the
process ID of the running popularity-contest.

So if that's what you see, you may wish to look in to why your
popularity-contest is consistently failing.

I would also agree that the system log directory is probably not the
best place for such files to be left behind, so that's maybe a wishlist
bug also.

Thanks,
Andy

-- 
https://bitfolk.com/ -- No-nonsense VPS hosting

[toc] | [prev] | [next] | [standalone]

#274077

On 2024-10-25, Andy Smith wrote:

> On Fri, Oct 25, 2024 at 03:54:58PM -0300, Bruno Schneider wrote:
>> I noticed that I have hundreds of files whose names start with
>> "popularity-contest" in /var/log. They don't even seem to be logs.
>> They seem to be data that popularity-contest sends away.
>
> popularity-contest does log to the file /var/log/popularity-contest,
> which is rotated by logrotate.

It's not really logs, it's the packages list sent. For me it should be in
something like /var/lib but not in /var/log
It's rotated by the script /etc/cron.daily/popularity-contest using
savelog and not logrotate

> I would also agree that the system log directory is probably not the
> best place for such files to be left behind, so that's maybe a wishlist
> bug also.

I opened a bug for this
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1085020
Feel free to add your point of view

[toc] | [prev] | [next] | [standalone]

#274145

On Fri, Oct 25, 2024 at 4:00 PM Andy Smith wrote:
> However, if popularity-contest experiences a problem during submit then
> it does leave behind a file named like /var/log/popularity-contest.12345
> (and possibly another with .gpg on the end) where "12345" was the
> process ID of the running popularity-contest.

Yes, that's it. However, file names with a single digit also exist,
they seem to have the same content (but not the corresponding file
with the .gpg suffix) and they seem to be rotated.

> So if that's what you see, you may wish to look in to why your
> popularity-contest is consistently failing.

Well, journalctl says nothing about popularity-contest. I would expect
SMTP to servers outside to fail but
/etc/popularity-contest.conf has the option:
     USEHTTP="yes"

I don't see why HTTP submissions would fail. I don't know how else I
could look into why popularity-contest consistently fails.

-- 
Bruno Schneider

[toc] | [prev] | [next] | [standalone]

#274149

Hi Bruno,

On Tue, Oct 29, 2024 at 11:03:04AM -0300, Bruno Schneider wrote:
> I don't see why HTTP submissions would fail.

You can't conceive of how a TCP connection over the Internet to a remote
server managed by someone else could fail? This speaks to a lack of
imagination I'm afraid. 😀

> I don't know how else I could look into why popularity-contest
> consistently fails.

If I were you I'd be looking at how often it fails compared to how often
it is run. Are there files left behind in /var/log for every invocation
for example.

I'd probably also try running it manually to see what happens.

According to /etc/cron.daily/popularity-contest this runs
"/etc/cron.daily/popularity-contest --crond" every day, and that is a
shell script. You can try just running it as root, i.e.

# /etc/cron.daily/popularity-contest --crond

and for even more info you can try running it with "bash -x" i.e.

# bash -x /etc/cron.daily/popularity-contest --crond

I can see in that script that it does log something on failure to HTTP
POST:

# try to post the report through http POST
if [ "$SUBMITURLS" ] && [ "yes" = "$USEHTTP" ]; then
    for URL in $SUBMITURLS ; do
        if setsid $TORIFY /usr/share/popularity-contest/popcon-upload \
            -u $URL -f $POPCON 2>/dev/null ; then
                SUBMITTED=yes
                set_sub
        else
                logger -t popularity-contest "unable to submit report to $URL."
        fi
    done
fi

So it'd be interesting to know if it is logging that or if the failure
is elsewhere.

If failure is intermittent this is going to be a lot more difficult. For
what it's worth, some of my servers have had occasional problems
submitting popcon and I just write that off as Debian popcon server
being down for maintenance or some other connectivity issue mid way.

Thanks,
Andy

-- 
https://bitfolk.com/ -- No-nonsense VPS hosting

[toc] | [prev] | [next] | [standalone]

#274159

On Tue, Oct 29, 2024 at 11:03 AM Bruno Schneider wrote:
> Well, journalctl says nothing about popularity-contest. I would expect
> SMTP to servers outside to fail but

My bad there, journalctl must be run by a privileged user. Doing so, I
can see lots of "unable to submit report to..." messages.

For the record, the command is:
# journalctl -t popularity-contest

On Tue, Oct 29, 2024 at 12:37 PM Andy Smith wrote:
> You can't conceive of how a TCP connection over the Internet to a remote
> server managed by someone else could fail? This speaks to a lack of
> imagination I'm afraid. 😀

Ok, what I really mean is that it shouldn't be failing consistently.

> If I were you I'd be looking at how often it fails compared to how often
> it is run. Are there files left behind in /var/log for every invocation
> for example.

Ok, that is hard to tell. It doesn't log successes, It should submit
data once a week, but for some weeks I can find more than one pair of
files left behind, so I'm guessing it is aware that it failed last
time and tries again. If it fails EVERY time, I guess there should be
more files (instead of mostly one pair per week).

I tried running it a few times by hand. No output on the shell, but
logs say they all failed.

Since I just want it to stop cluttering my /var/log and I don't need
it, I guess I'll just get rid of the whole thing.

-- 
Bruno Schneider

[toc] | [prev] | [standalone]

Back to top | Article view | linux.debian.user

csiph-web