Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.user > #235552 > unrolled thread

Web log analysis

Back to article view | Back to linux.debian.user

Contents

  Web log analysis Richard Hector <richard@walnut.gen.nz> - 2021-05-27 12:00 +0200
    Re: Web log analysis IL Ka <kazakevichilya@gmail.com> - 2021-05-27 14:40 +0200
      Re: Web log analysis Polyna-Maude Racicot-Summerside <debian@polynamaude.com> - 2021-05-27 16:30 +0200
    Re: Web log analysis Polyna-Maude Racicot-Summerside <debian@polynamaude.com> - 2021-05-27 22:00 +0200
      Re: Web log analysis Dan Ritter <dsr@randomstring.org> - 2021-05-27 23:00 +0200
        Re: Web log analysis Alexandre Rossi <niol@zincube.net> - 2021-05-28 14:00 +0200
    Re: Web log analysis Richard Hector <richard@walnut.gen.nz> - 2021-06-23 10:10 +0200

#235552 — Web log analysis

Hi all,

I need to get a handle on what my web servers are doing.

I remembered the name awstats, and installed it, but when I finally got 
it going (I had a typo in my nginx custom format), the output was 
somewhat opaque.

Things I'd like:

Ability to safely provide access to my customers
Ability to show trends over a decent length of time (at least a year)
Reasonable support for custom log formats, and ideally be helpful if I 
get it wrong ("It doesn't match" really wasn't very helpful)
Easy to navigate different views
Graphs would be nice
Not too much load on the web server
Free Software
Included in Debian preferred

Many of my sites are Wordpress sites, so I could choose from the many 
available plugins, but I suspect those will probably only start 
reporting from when they're installed; I have existing logs going back 
anything up to a few years. Generic is probably better.

Any suggestions?

Cheers,
Richard

[toc] | [next] | [standalone]

#235555

[Multipart message — attachments visible in raw view] — view raw

On Thu, May 27, 2021 at 12:55 PM Richard Hector <richard@walnut.gen.nz>
wrote:

> Hi all,
>
> I need to get a handle on what my web servers are doing.
>
There is also a webalizer which is similar to AWStat.

I use google analytics to monitor website. It knows nothing about my logs
of course, but provides lots of information.
https://analytics.google.com/analytics/academy/course/6

[toc] | [prev] | [next] | [standalone]

#235565

[Multipart message — attachments visible in raw view] — view raw

Hi,

On 2021-05-27 8:32 a.m., IL Ka wrote:
> 
> 
> On Thu, May 27, 2021 at 12:55 PM Richard Hector <richard@walnut.gen.nz
> <mailto:richard@walnut.gen.nz>> wrote:
> 
>     Hi all,
> 
>     I need to get a handle on what my web servers are doing.
> 
> There is also a webalizer which is similar to AWStat.
> 
> I use google analytics to monitor website. It knows nothing about my
> logs of course, but provides lots of information.
> https://analytics.google.com/analytics/academy/course/6
> <https://analytics.google.com/analytics/academy/course/6>
> 
We all know what Google Analytics does.
And this exactly not what this guy is looking for. Not looking for a
tool that adds some code to his website and track his customer.

All of the things he's asking for aren't helped by Google Analytics.

This is as useful as using a infrared thermometer to get the temperature
of a router, thinking this way you'll know how many user a logged into
your network because the thing get warmer.
> 
> 
>  

-- 
Polyna-Maude R.-Summerside
-Be smart, Be wise, Support opensource development

[toc] | [prev] | [next] | [standalone]

#235592

[Multipart message — attachments visible in raw view] — view raw

Prometheus

Grafana

Those are two tools used for metrics.

On 2021-05-27 5:55 a.m., Richard Hector wrote:
> Hi all,
> 
> I need to get a handle on what my web servers are doing.
> 
> I remembered the name awstats, and installed it, but when I finally got
> it going (I had a typo in my nginx custom format), the output was
> somewhat opaque.
> 
> Things I'd like:
> 
> Ability to safely provide access to my customers
> Ability to show trends over a decent length of time (at least a year)
> Reasonable support for custom log formats, and ideally be helpful if I
> get it wrong ("It doesn't match" really wasn't very helpful)
> Easy to navigate different views
> Graphs would be nice
> Not too much load on the web server
> Free Software
> Included in Debian preferred
> 
> Many of my sites are Wordpress sites, so I could choose from the many
> available plugins, but I suspect those will probably only start
> reporting from when they're installed; I have existing logs going back
> anything up to a few years. Generic is probably better.
> 
> Any suggestions?
> 
> Cheers,
> Richard
> 

-- 
Polyna-Maude R.-Summerside
-Be smart, Be wise, Support opensource development

[toc] | [prev] | [next] | [standalone]

#235595

Polyna-Maude Racicot-Summerside wrote: 
> 
> Prometheus
> 
> Grafana
> 
> Those are two tools used for metrics.

Prometheus collects specific metrics and sends them and stores
them; Grafana displays them.

In this case, Richard seems to be looking for analysis of the
web server logs he has already collected.

analog
awfful
awstats
goaccess
logstalgia
logswan
visitors
webalizer

are all things that do that and are available in Debian.

I've used awstats, visitors and webalizer; I don't remember any
of them being particularly difficult.

-dsr-

[toc] | [prev] | [next] | [standalone]

#235609

Hi,

> In this case, Richard seems to be looking for analysis of the
> web server logs he has already collected.
> 
> analog
> awfful
> awstats
> goaccess
> logstalgia
> logswan
> visitors
> webalizer

analog, awfful, awstats, visitors and webalizer seem unmaintained.

logstalgia is an Xorg app that displays log stats in retro videogame
fashion.

logswan only outputs JSON so needs to be fed into somtehing else
to produce graphs.

Of those, only goaccess seems to be maintained. It complies with
many of your requirements, although analysing data over long periods
is not straightforward. I use it to produce one static report per month
and it suits my basic needs.

Cheers,

Alex

[toc] | [prev] | [next] | [standalone]

#236337

On 27/05/21 9:55 pm, Richard Hector wrote:
> Hi all,
> 
> I need to get a handle on what my web servers are doing.

Apologies for my lack of response.

Thanks for all of the useful and interesting replies.

I'll look into this further later; in the meantime I think I solved my 
immediate needs with tools like grep :-)

Cheers,
Richard

[toc] | [prev] | [standalone]

Back to top | Article view | linux.debian.user

csiph-web