Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.user > #175716 > unrolled thread

best practices for fighting spam with Debian?

Back to article view | Back to linux.debian.user

Contents

  best practices for fighting spam with Debian? Daniel Pocock <daniel@pocock.pro> - 2016-12-15 09:20 +0100
    Re: best practices for fighting spam with Debian? Sven Hartge <sven@svenhartge.de> - 2016-12-15 10:50 +0100
      Re: best practices for fighting spam with Debian? Brian <ad44@cityscape.co.uk> - 2016-12-15 21:40 +0100
        Re: best practices for fighting spam with Debian? Sven Hartge <sven@svenhartge.de> - 2016-12-15 21:50 +0100
          Re: best practices for fighting spam with Debian? Daniel Pocock <daniel@pocock.pro> - 2016-12-16 08:00 +0100
            Re: best practices for fighting spam with Debian? <tomas@tuxteam.de> - 2016-12-16 10:00 +0100
              Re: best practices for fighting spam with Debian? Sven Hartge <sven@svenhartge.de> - 2016-12-16 10:20 +0100
                Re: best practices for fighting spam with Debian? <tomas@tuxteam.de> - 2016-12-16 11:00 +0100
                  Re: best practices for fighting spam with Debian? Lisi Reisz <lisi.reisz@gmail.com> - 2016-12-16 22:00 +0100
                    Re: best practices for fighting spam with Debian? Lisi Reisz <lisi.reisz@gmail.com> - 2016-12-16 22:30 +0100
                      Re: best practices for fighting spam with Debian? <tomas@tuxteam.de> - 2016-12-16 22:50 +0100
                        Re: best practices for fighting spam with Debian? Brian <ad44@cityscape.co.uk> - 2016-12-16 23:50 +0100
                          Re: best practices for fighting spam with Debian? Lisi Reisz <lisi.reisz@gmail.com> - 2016-12-17 00:50 +0100
                    Re: best practices for fighting spam with Debian? Joe <joe@jretrading.com> - 2016-12-16 22:30 +0100
                  Re: best practices for fighting spam with Debian? Richard Owlett <rowlett@cloud85.net> - 2016-12-17 13:10 +0100
            Re: best practices for fighting spam with Debian? Sven Hartge <sven@svenhartge.de> - 2016-12-16 10:10 +0100
      Re: best practices for fighting spam with Debian? Vincent Lefevre <vincent@vinc17.net> - 2016-12-16 13:10 +0100
        Re: best practices for fighting spam with Debian? Sven Hartge <sven@svenhartge.de> - 2016-12-16 13:20 +0100
        Re: best practices for fighting spam with Debian? Joe <joe@jretrading.com> - 2016-12-16 19:00 +0100
          Re: best practices for fighting spam with Debian? Vincent Lefevre <vincent@vinc17.net> - 2016-12-19 13:10 +0100
            Re: best practices for fighting spam with Debian? Joe <joe@jretrading.com> - 2016-12-19 18:40 +0100
              Re: best practices for fighting spam with Debian? Vincent Lefevre <vincent@vinc17.net> - 2016-12-20 01:30 +0100
    Re: best practices for fighting spam with Debian? Glenn English <ghe2001@gmail.com> - 2016-12-15 20:30 +0100
      Re: best practices for fighting spam with Debian? Sven Hartge <sven@svenhartge.de> - 2016-12-15 22:00 +0100
    Re: best practices for fighting spam with Debian? terryc <terryc@woa.com.au> - 2016-12-16 03:40 +0100

Page 1 of 2  [1] 2  Next page →

#175716 — best practices for fighting spam with Debian?

Over time, I've noticed that the quantity of spam getting through my
Debian mail server has been increasing.

What steps are people taking to minimize the spam problem, especially if
you are using Debian as both mail server and desktop?

The wiki contains details about SpamAssassin[1] and examples of
anti-spam configurations for various mailers such as Postfix[2].  Are
there any new developments in anti-spam that are not covered there?
Could anybody provide updates to those pages?  What would professional
mail server admins consider best practice today?

One part of the problem I observe personally comes from having multiple
email addresses.  Those spammers who are not already on some blacklist
that I use are able to deliver copies of the same spam to each of my
addresses and so I often see 3 to 5 copies of a particular spamming in
one day.  Presumably I could limit these duplicates by disabling some of
the email addresses.

Are there any mechanisms in place that allow members of the free
software community and other power users to identify spammings in
progress and have them blacklisted more quickly?  Many of the blacklist
sites don't reveal how people can contribute to them.  When I continue
getting multiple copies of a particular spam throughout the day I feel
this was a lost opportunity to blacklist somebody.

How do people deal with MailChimp spam?  Almost every other day I find
some company I haven't had any dealings with for years has uploaded all
their contacts, including myself, to MailChimp and started sending some
newsletter.  MailChimp seems to encourage this "softer" form of spamming
as a legitimate business model and it happens so regularly that it is
very tempting to blacklist the whole company.

I've also observed there are a few companies in the travel industry who
forcefully re-subscribe victims to their spam campaigns every time a new
booking is made, even if the user already unsubscribed.  Even without
the newsletters, a single flight or hotel booking can generate well over
10 emails with advertising pushed under headings like "important
information about your flight" or "reminder to leave a review".  So far
the only effective means I've found for dealing with this is using
procmail to put them into a folder for travel, I never look there unless
I deliberately need to find some previous message from one of these
companies.

Regards,

Daniel


1. https://wiki.debian.org/DebianSpamAssassin
2. https://wiki.debian.org/Postfix#anti-spam:_smtp_restrictions

[toc] | [next] | [standalone]

#175718

Daniel Pocock <daniel@pocock.pro> wrote:

> Could anybody provide updates to those pages?  What would professional
> mail server admins consider best practice today?

Problem is: the more (drastic) anti-SPAM measures everyone take, the
more broken e-mail as a whole becomes.

I noticed in the last two years it became quite difficult to send mails
to the larger freemail providers like Hotmail, Gmail or GMX/Web.de (from
Germany) because of reputation based blacklisting of whole hosting
providers and ISPs.

And then there is the problem with bot nets, connection from many
different IPs and thus subverting any rate limiting approach.

That said:

At work we use a multi-layered approach, using Exim4 as MTA.

At first we check if the HELO tries to fake the IP of the MX or the
hostname of the MX or any of our local domains. If this is the case, the
connection is rejected immediately, because this is a sure sign of
forgery and SPAM.

Then we check if the sending IP is in multiple RBLs.

Next is a check if the sending IP resolves correctly and the hostname
resolves back to this IP.

Then we have our own internal reputation system based on the
"spamminess" of an ASN. It is simple percentage calculation of ham/spam over
a sliding window of 28 days. The result of the overall spam/ham-check at
the end feeds back into this.

- At this point it is decided if the connection is to be greylisted. If
  any of the above criteria are hit, greylisting is activated.

After this, ClamAV and Sophos AV are run. ClamAV detects more than just
virus/trojans these days and rejects many phishing attempts.

Finally Spamassassin is run, using the official rules and some
additional local tweaks (for example using the iXHash plugin
https://sourceforge.net/projects/ixhash/ and
https://wiki.apache.org/spamassassin/iXhash and
(in German) http://www.heise.de/ix/nixspam/ from the German Computer
Magazine iX).

Its spam-score and the results of our other checks are tallied together
and if either Spamassassins spam-score or the combination of our own
score and the Spamassassins spam-score are over a certain threshold the
mail is rejected.

This seems all very complicated (it is), but because of the environment
I work in (University) it is very important for us (and our users) to
have more control over which email is rejected, trying to minimize false
positives as much as possible while also trying to detect as much SPAM
as possible. A very fine balance, to say the least.

> Are there any mechanisms in place that allow members of the free
> software community and other power users to identify spammings in
> progress and have them blacklisted more quickly?  Many of the
> blacklist sites don't reveal how people can contribute to them.  When
> I continue getting multiple copies of a particular spam throughout the
> day I feel this was a lost opportunity to blacklist somebody.

There is the commercial DCC clearing house project
https://www.dcc-servers.net/dcc/ which kind of does this: detecting spam
runs by collecting fuzzy checksums on many many servers. In addition
pyzor and razor2 and similar.

> How do people deal with MailChimp spam?  Almost every other day I find
> some company I haven't had any dealings with for years has uploaded all
> their contacts, including myself, to MailChimp and started sending some
> newsletter.  MailChimp seems to encourage this "softer" form of spamming
> as a legitimate business model and it happens so regularly that it is
> very tempting to blacklist the whole company.

I personally blacklisted the whole company. You might also want to add
the following rule to your local.cf for spamassassin:

,----
| # Interspire Email Marketer
| header          __IME_XMailer_LID       X-Mailer-LID =~ /\d+/
| header          __IME_XMailer_RecptId   X-Mailer-RecptId =~ /\d+/
| header          __IME_XMailer_SID       X-Mailer-SID =~ /\d+/
| header          __IME_XMailer_Sent_By   X-Mailer-Sent-By =~ /\d+/
| meta            IME_XMailer_SPAM        (__IME_XMailer_LID && __IME_XMailer_RecptId && __IME_XMailer_SID && __IME_XMailer_Sent_By)
| score           IME_XMailer_SPAM        4.0
`----

This company and their software was the bane of my existence until I
added the above rules. Now: precious silence.

Grüße,
S°

-- 
Sigmentation fault. Core dumped.

[toc] | [prev] | [next] | [standalone]

#175728

On Thu 15 Dec 2016 at 10:41:58 +0100, Sven Hartge wrote:

[..Snip...]

> This seems all very complicated (it is), but because of the environment
> I work in (University) it is very important for us (and our users) to
> have more control over which email is rejected, trying to minimize false
> positives as much as possible while also trying to detect as much SPAM
> as possible. A very fine balance, to say the least.

It is (as you say) complicated but

1. Why is it important for *your university*?

2. Why is it important for your users?

The two are are not the same. For a start, users have no control over
your mail server. Secondly, users might want to receive every mail they
are sent, irrespective of what *you* think of its spamminess. After all,
it is their mail, not yours. There is no "fine balance" as far as a
user is concerned. Either they get the mail or they don't.

Thirdly, if you want to say "we control what mail we accept", that's
fine. But don't make it appear that you are doing a user a favour by
rejecting mail they do not mind getting. You may very well try to
eliminate false positives but that is not the same as allowing a user
to receive all the mail which is sent to him.

Users can control what mail they receive, of course. Why not leave it
up to them to decide?

-- 
Brian.

[toc] | [prev] | [next] | [standalone]

#175729

Brian <ad44@cityscape.co.uk> wrote:
> On Thu 15 Dec 2016 at 10:41:58 +0100, Sven Hartge wrote:

> [..Snip...]

>> This seems all very complicated (it is), but because of the environment
>> I work in (University) it is very important for us (and our users) to
>> have more control over which email is rejected, trying to minimize false
>> positives as much as possible while also trying to detect as much SPAM
>> as possible. A very fine balance, to say the least.

> It is (as you say) complicated but

> 1. Why is it important for *your university*?
> 2. Why is it important for your users?

Our users (mostly professors) can sometimes be a bit ... particular
concerning mail reception and possible false positives. 

This is why we built the whole filtering stuff mostly ourselves instead
of "outsourcing" it to a ready-made appliance. This would of course been
a much quicker path but you lose control of the inner workings this way.

Also the whole internal spam filtering harness has been built over the
last 15 years and at the beginning such filtering appliances where not
even a thing.

> The two are are not the same. For a start, users have no control over
> your mail server. Secondly, users might want to receive every mail they
> are sent, irrespective of what *you* think of its spamminess. After all,
> it is their mail, not yours. There is no "fine balance" as far as a
> user is concerned. Either they get the mail or they don't.

> Users can control what mail they receive, of course. Why not leave it
> up to them to decide?

This is why the filter is configurable by the user and the user may also
disable it completely, if s/he so choses. (Few do and many of those who
chose to disable it reenable it after a short while.)

There is a fine balance between protecting users from trojans, phishing
and virus infections on one side and allowing all legitimate mail on the
other side.

The filter setup for my own private mail server is much stricter than
that of my workplace, because I am the only user and can, for example,
reject whole countries because I personally only ever get spam from
there (looking at you China, Argentina and Brazil). This is of course
nothing I could enforce at work, obviously.

Grüße,
Sven.

-- 
Sigmentation fault. Core dumped.

[toc] | [prev] | [next] | [standalone]

#175737

On 15/12/16 21:47, Sven Hartge wrote:
> Brian <ad44@cityscape.co.uk> wrote:
>> On Thu 15 Dec 2016 at 10:41:58 +0100, Sven Hartge wrote:
> 
>> [..Snip...]
> 
>>> This seems all very complicated (it is), but because of the environment
>>> I work in (University) it is very important for us (and our users) to
>>> have more control over which email is rejected, trying to minimize false
>>> positives as much as possible while also trying to detect as much SPAM
>>> as possible. A very fine balance, to say the least.
> 
>> It is (as you say) complicated but
> 
>> 1. Why is it important for *your university*?
>> 2. Why is it important for your users?
> 
> Our users (mostly professors) can sometimes be a bit ... particular
> concerning mail reception and possible false positives. 
> 
> This is why we built the whole filtering stuff mostly ourselves instead
> of "outsourcing" it to a ready-made appliance. This would of course been
> a much quicker path but you lose control of the inner workings this way.
> 
> Also the whole internal spam filtering harness has been built over the
> last 15 years and at the beginning such filtering appliances where not
> even a thing.
> 
>> The two are are not the same. For a start, users have no control over
>> your mail server. Secondly, users might want to receive every mail they
>> are sent, irrespective of what *you* think of its spamminess. After all,
>> it is their mail, not yours. There is no "fine balance" as far as a
>> user is concerned. Either they get the mail or they don't.
> 
>> Users can control what mail they receive, of course. Why not leave it
>> up to them to decide?
> 
> This is why the filter is configurable by the user and the user may also
> disable it completely, if s/he so choses. (Few do and many of those who
> chose to disable it reenable it after a short while.)
> 
> There is a fine balance between protecting users from trojans, phishing
> and virus infections on one side and allowing all legitimate mail on the
> other side.
> 

A key point to consider for any filtering is the user experience when a
legitimate message is not delivered.

This includes the following:

- does the sender know their mail was not delivered and do they get a
reasonable explanation?  I've heard there are spam filters who give a
"user doesn't exist" error which is somewhat disrespectful to genuine
senders.  If the messages end up in a junk folder the sender has no
idea.  Personally I think it is better to graylist or reject messages
with a basic explanation like "Your message could not be delivered
because of local policy"

- does the recipient have to wade through a junk mail folder looking for
things that went there by mistake?  In this case, I feel the spam
problem hasn't really been solved at all, the user still has to do the
work sifting through spam to find the ham and it is still a painful
overhead for them.

In some countries and certain industries a lot of companies have simply
started using cloud solutions from some of the large vendors.  They tend
to get all the messages from companies in their industry on the same
platform, but I noticed that one of them was incorrectly putting mail
that had come from one of my Debian mail servers into their junk folder.
 Their IT manager spent a lot of time chasing the vendor to find out why
this was happening and they never got any valid answer.  It was
frustrating for them and for people sending the mail.  The vendor would
just say that they have to keep their policies secret.

Regards,

Daniel

[toc] | [prev] | [next] | [standalone]

#175739

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Fri, Dec 16, 2016 at 07:51:45AM +0100, Daniel Pocock wrote:

[...]

> - does the sender know their mail was not delivered and do they get a
> reasonable explanation?  I've heard there are spam filters who give a
> "user doesn't exist" error which is somewhat disrespectful to genuine
> senders. [...]

I wish this were so. The really Big Ones (have seen that with
outlook.com, possibly with gmail.com), either just drop silently
the mail (!) or deliver it to the end user's spam box (which
combined with the (reasonable!) recommendation to not click
on dubious mail amounts to the same).

Thus mail gets not delivered *without any notification to the
sender*.

This has the potential to kill mail (which for the Big Ones might
be a Good Thing: mail is too decentral and too hard to monetarize).

regards
- -- t
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)

iEYEARECAAYFAlhTrOoACgkQBcgs9XrR2kYcBgCfb3PjslZBSJaqBGr+U7VTB1AM
Zq4An038E/Ed5H+F4aa8kS8xC7VhR9gF
=Zhzs
-----END PGP SIGNATURE-----

[toc] | [prev] | [next] | [standalone]

#175741

tomas@tuxteam.de wrote:
> On Fri, Dec 16, 2016 at 07:51:45AM +0100, Daniel Pocock wrote:

>> - does the sender know their mail was not delivered and do they get a
>> reasonable explanation?  I've heard there are spam filters who give a
>> "user doesn't exist" error which is somewhat disrespectful to genuine
>> senders. [...]

> I wish this were so. The really Big Ones (have seen that with
> outlook.com, possibly with gmail.com), either just drop silently
> the mail (!) or deliver it to the end user's spam box (which
> combined with the (reasonable!) recommendation to not click
> on dubious mail amounts to the same).

> Thus mail gets not delivered *without any notification to the
> sender*.

At least in the case of delivery to the SPAM box the mail has been
actually delivered to the user.

The first one is really the problem, where the mail just vanishes and
both the sender and the receiver have no idea where it went.

> This has the potential to kill mail (which for the Big Ones might
> be a Good Thing: mail is too decentral and too hard to monetarize).

10 years ago, SPAM was told to kill eMail. Now it's the over-the-top
anti-SPAM solutions doing the same.

Grüße,
Sven.

-- 
Sigmentation fault. Core dumped.

[toc] | [prev] | [next] | [standalone]

#175743

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Fri, Dec 16, 2016 at 10:16:03AM +0100, Sven Hartge wrote:
> tomas@tuxteam.de wrote:
> > On Fri, Dec 16, 2016 at 07:51:45AM +0100, Daniel Pocock wrote:
> 
> >> - does the sender know their mail was not delivered and do they get a
> >> reasonable explanation?  I've heard there are spam filters who give a
> >> "user doesn't exist" error which is somewhat disrespectful to genuine
> >> senders. [...]
> 
> > I wish this were so [...]

> > Thus mail gets not delivered *without any notification to the
> > sender*.
> 
> At least in the case of delivery to the SPAM box the mail has been
> actually delivered to the user.

"Technically", yes.

> The first one is really the problem, where the mail just vanishes and
> both the sender and the receiver have no idea where it went.

My point is, that from the user's perspective (at both ends) those
are more or less equivalent (perhaps the second is even worse,
because the provider can just "wash his hands").

> > This has the potential to kill mail (which for the Big Ones might
> > be a Good Thing: mail is too decentral and too hard to monetarize).
> 
> 10 years ago, SPAM was told to kill eMail. Now it's the over-the-top
> anti-SPAM solutions doing the same.

The difference being that in the first case there wasn't a monetary
incentive to kill mail (on the contrary, a slight incentive to not
kill it) and now there is.

Sorry, having my Black Day today :-/

regards
- -- tomás

> Sigmentation fault. Core dumped.

I like this one: made me smile :-)
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)

iEYEARECAAYFAlhTuswACgkQBcgs9XrR2kbS3gCeNDBhuT5jwaVKrEZ9VOonyTkX
RcgAn2zMG6OvtpAT/69kMsG6ZikOPhDf
=Vq9E
-----END PGP SIGNATURE-----

[toc] | [prev] | [next] | [standalone]

#175754

On Friday 16 December 2016 09:58:36 tomas@tuxteam.de wrote:
> > At least in the case of delivery to the SPAM box the mail has been
> > actually delivered to the user.
>
> "Technically", yes.

No, not just technically.  Actually.  The recipient then has the choice 
whether to ignore it or not.
>
> > The first one is really the problem, where the mail just vanishes and
> > both the sender and the receiver have no idea where it went.
>
> My point is, that from the user's perspective (at both ends) those
> are more or less equivalent (perhaps the second is even worse,
> because the provider can just "wash his hands")

No, they are not.  If it is delivered to the spam folder it is there and can 
be rescued if the recipient wishes.  If it is just dropped the recipient does 
not know anything about it, and may do nothing when it might have been 
important to do something.

Many users use PASSWORD and 123123 as passwords.  That does not render 
passwords pointless.  The fact that some people do not know enough to use 
Spam folders differently from deleted mail is not a reason why the rest of us 
should have our email dropped.

Lisi

[toc] | [prev] | [next] | [standalone]

#175755

On Friday 16 December 2016 21:19:45 Joe wrote:
> On Fri, 16 Dec 2016 20:55:32 +0000
>
> Lisi Reisz <lisi.reisz@gmail.com> wrote:
> > The fact that some people do not know
> > enough to use Spam folders differently from deleted mail is not a
> > reason why the rest of us should have our email dropped.
>
> The reason why your mail gets dropped is that it cannot accurately be
> identified as not-spam, as I'm sure you know.

You have over-cut.  Here is what you cut, which is highly relevant:

"On Friday 16 December 2016 09:58:36 tomas@tuxteam.de wrote:
> > At least in the case of delivery to the SPAM box the mail has been
> > actually delivered to the user.
>
> "Technically", yes.

No, not just technically.  Actually.  The recipient then has the choice 
whether to ignore it or not."

Delivering to a Spam folder, and dropping mail, are simply not the same thing, 
however hard it may be to distinguish the correct route for any one piece of 
mail.

Lisi.
>
> If it really is spam, then the Reply-To header is forged, and the last
> thing that should be done is to send the message 'back', *quoting* *it*
> *in* *full*, to a third party who had nothing to do with it, and who has
> now been sent spam from a presumably legitimate mail server.
>
> The problem is not to do with the silent dropping of spam, which is the
> correct behaviour for a mail server which has been foolish enough to
> accept it in the first place, but to identify it with 100% accuracy and
> thereby to prevent the dropping of genuine email. Solve that and you
> can name your price...
>
> The next best thing is to identify spam at the SMTP transmission stage
> and to refuse it, which *does* tell the sender of mis-identified
> legitimate email that it has not been delivered.

[toc] | [prev] | [next] | [standalone]

#175757

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Fri, Dec 16, 2016 at 09:31:06PM +0000, Lisi Reisz wrote:
> On Friday 16 December 2016 21:19:45 Joe wrote:
> > On Fri, 16 Dec 2016 20:55:32 +0000
> >
> > Lisi Reisz <lisi.reisz@gmail.com> wrote:

[...]

> You have over-cut.  Here is what you cut, which is highly relevant:
> 
> "On Friday 16 December 2016 09:58:36 tomas@tuxteam.de wrote:
> > > At least in the case of delivery to the SPAM box the mail has been
> > > actually delivered to the user.
> >
> > "Technically", yes.
> 
> No, not just technically.  Actually.  The recipient then has the choice 
> whether to ignore it or not."

Let's just agree to differ on that.

regards
- -- t
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)

iEYEARECAAYFAlhUXzEACgkQBcgs9XrR2kYexACeKNH/Dm9fs2oQflhEXP6MY/jk
pX8Ani5PJGcAI1nwtaZswrVJPY372xHD
=XuwD
-----END PGP SIGNATURE-----

[toc] | [prev] | [next] | [standalone]

#175758

On Fri 16 Dec 2016 at 22:40:01 +0100, tomas@tuxteam.de wrote:

> On Fri, Dec 16, 2016 at 09:31:06PM +0000, Lisi Reisz wrote:
> > On Friday 16 December 2016 21:19:45 Joe wrote:
> > > On Fri, 16 Dec 2016 20:55:32 +0000
> > >
> > > Lisi Reisz <lisi.reisz@gmail.com> wrote:
> 
> [...]
> 
> > You have over-cut.  Here is what you cut, which is highly relevant:
> > 
> > "On Friday 16 December 2016 09:58:36 tomas@tuxteam.de wrote:
> > > > At least in the case of delivery to the SPAM box the mail has been
> > > > actually delivered to the user.
> > >
> > > "Technically", yes.
> > 
> > No, not just technically.  Actually.  The recipient then has the choice 
> > whether to ignore it or not."
> 
> Let's just agree to differ on that.

No, let's just say your "technically" means exactly the same as Lisi's
"actually". Then there is no need to differ and everybody is happy. :)

-- 
Brian.

[toc] | [prev] | [next] | [standalone]

#175759

On Friday 16 December 2016 22:40:22 Brian wrote:
> On Fri 16 Dec 2016 at 22:40:01 +0100, tomas@tuxteam.de wrote:
> > On Fri, Dec 16, 2016 at 09:31:06PM +0000, Lisi Reisz wrote:
> > > On Friday 16 December 2016 21:19:45 Joe wrote:
> > > > On Fri, 16 Dec 2016 20:55:32 +0000
> > > >
> > > > Lisi Reisz <lisi.reisz@gmail.com> wrote:
> >
> > [...]
> >
> > > You have over-cut.  Here is what you cut, which is highly relevant:
> > >
> > > "On Friday 16 December 2016 09:58:36 tomas@tuxteam.de wrote:
> > > > > At least in the case of delivery to the SPAM box the mail has been
> > > > > actually delivered to the user.
> > > >
> > > > "Technically", yes.
> > >
> > > No, not just technically.  Actually.  The recipient then has the choice
> > > whether to ignore it or not."
> >
> > Let's just agree to differ on that.
>
> No, let's just say your "technically" means exactly the same as Lisi's
> "actually". Then there is no need to differ and everybody is happy. :)

"When I use a word," Humpty Dumpty said, in a rather a scornful tone, "it 
means just what I choose it to mean-neither more nor less."
"The question is," said Alice, "whether you can make words mean so many 
different things."
"The question is," said Humpty Dumpty, "which is to be master that's all."

:-)
Lisi

[toc] | [prev] | [next] | [standalone]

#175756

On Fri, 16 Dec 2016 20:55:32 +0000
Lisi Reisz <lisi.reisz@gmail.com> wrote:


> The fact that some people do not know
> enough to use Spam folders differently from deleted mail is not a
> reason why the rest of us should have our email dropped.
> 

The reason why your mail gets dropped is that it cannot accurately be
identified as not-spam, as I'm sure you know.

If it really is spam, then the Reply-To header is forged, and the last
thing that should be done is to send the message 'back', *quoting* *it*
*in* *full*, to a third party who had nothing to do with it, and who has
now been sent spam from a presumably legitimate mail server.

The problem is not to do with the silent dropping of spam, which is the
correct behaviour for a mail server which has been foolish enough to
accept it in the first place, but to identify it with 100% accuracy and
thereby to prevent the dropping of genuine email. Solve that and you
can name your price...

The next best thing is to identify spam at the SMTP transmission stage
and to refuse it, which *does* tell the sender of mis-identified
legitimate email that it has not been delivered.

-- 
Joe

[toc] | [prev] | [next] | [standalone]

#175770

On 12/16/2016 3:58 AM, tomas@tuxteam.de wrote:
> [snip]
>
>>> This has the potential to kill mail (which for the Big Ones might
>>> be a Good Thing: mail is too decentral and too hard to monetarize).
>>
>> 10 years ago, SPAM was told to kill eMail. Now it's the over-the-top
>> anti-SPAM solutions doing the same.
>
> The difference being that in the first case there wasn't a monetary
> incentive to kill mail (on the contrary, a slight incentive to not
> kill it) and now there is.
>

There is definitely a market for a good paid email delivery 
service - I'm a happy customer thereof. I am not sure of how well 
it would scale up for a potential mass market provider.

They are a small company servicing the geographically local 
business community. I was a retail customer >20 years ago and 
enjoyed the benefits of their growth as a service provider. I 
avoid the so called "free" email and news-servers as they _cost_ 
too much.

[toc] | [prev] | [next] | [standalone]

#175740

Daniel Pocock <daniel@pocock.pro> wrote:

> A key point to consider for any filtering is the user experience when a
> legitimate message is not delivered.

> This includes the following:

> - does the sender know their mail was not delivered and do they get a
> reasonable explanation?  I've heard there are spam filters who give a
> "user doesn't exist" error which is somewhat disrespectful to genuine
> senders.

I can understand the reason for this: Don't give the spammers any
indication if an address is valid. But I have yet to see any solid
evidence that it makes a difference if one responds with a "500 Your
was classified as SPAM." or "500 User does not exist".

Far more "damage" is done by clicking on any (so called) unsubscribe
links in SPAM mails.


> If the messages end up in a junk folder the sender has no idea.
> Personally I think it is better to graylist or reject messages with a
> basic explanation like "Your message could not be delivered because of
> local policy"

Exactly.

> - does the recipient have to wade through a junk mail folder looking
> for things that went there by mistake?  In this case, I feel the spam
> problem hasn't really been solved at all, the user still has to do the
> work sifting through spam to find the ham and it is still a painful
> overhead for them.

Some people want to operate this way. 

> In some countries and certain industries a lot of companies have
> simply started using cloud solutions from some of the large vendors.
> They tend to get all the messages from companies in their industry on
> the same platform, but I noticed that one of them was incorrectly
> putting mail that had come from one of my Debian mail servers into
> their junk folder.  Their IT manager spent a lot of time chasing the
> vendor to find out why this was happening and they never got any valid
> answer.  It was frustrating for them and for people sending the mail.
> The vendor would just say that they have to keep their policies
> secret.

Also exactly the reason why we $here decided to do the heavy lifting
ourselves instead of buying a ready-made appliance.

S°

-- 
Sigmentation fault. Core dumped.

[toc] | [prev] | [next] | [standalone]

#175747

On 2016-12-15 10:41:58 +0100, Sven Hartge wrote:
> Next is a check if the sending IP resolves correctly and the hostname
> resolves back to this IP.
  ^^^^^^^^^^^^^^^^^^^^^^^^
I wonder whether this rule should really be used. A hostname may resolve
to multiple IP's, and for some reasons, the original IP may not be in
the list (e.g. if truncated).

-- 
Vincent Lefèvre <vincent@vinc17.net> - Web: <https://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <https://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

[toc] | [prev] | [next] | [standalone]

#175748

Vincent Lefevre <vincent@vinc17.net> wrote:
> On 2016-12-15 10:41:58 +0100, Sven Hartge wrote:

>> Next is a check if the sending IP resolves correctly and the hostname
>> resolves back to this IP.
>   ^^^^^^^^^^^^^^^^^^^^^^^^

> I wonder whether this rule should really be used. A hostname may
> resolve to multiple IP's, and for some reasons, the original IP may
> not be in the list (e.g. if truncated).

This is why this should not be a hard and final filter criterion but
just a hint you might want to look at this email a bit more in-depth.

S°

-- 
Sigmentation fault. Core dumped.

[toc] | [prev] | [next] | [standalone]

#175750

On Fri, 16 Dec 2016 13:05:35 +0100
Vincent Lefevre <vincent@vinc17.net> wrote:

> On 2016-12-15 10:41:58 +0100, Sven Hartge wrote:
> > Next is a check if the sending IP resolves correctly and the
> > hostname resolves back to this IP.  
>   ^^^^^^^^^^^^^^^^^^^^^^^^
> I wonder whether this rule should really be used. A hostname may
> resolve to multiple IP's, and for some reasons, the original IP may
> not be in the list (e.g. if truncated).
> 

It is therefore an important matter that the email admin ensures that
the PTR record *always* resolves to a hostname which resolves back,
which can if necessary be achieved by creating a fictitious A record for
just this purpose. It is not necessary for this hostname to match either
HELO or domain MX, though it's worth doing if that isn't difficult.

This check used to be pretty much a guarantee of a business IP address,
but unfortunately many home ISP accounts now have complementary A-PTR
records, making spam detection much harder. But even today, this DNS
check is my second most useful anti-spam measure, the first being to
accept email only to named account holders.

-- 
Joe

[toc] | [prev] | [next] | [standalone]

#175823

On 2016-12-16 17:58:26 +0000, Joe wrote:
> It is therefore an important matter that the email admin ensures that
> the PTR record *always* resolves to a hostname which resolves back,
> which can if necessary be achieved by creating a fictitious A record for
> just this purpose. It is not necessary for this hostname to match either
> HELO or domain MX, though it's worth doing if that isn't difficult.

But you need to hope that the nameserver is compliant and
in particular, won't truncate the list of IP addresses as
with: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=834098

> This check used to be pretty much a guarantee of a business IP address,
> but unfortunately many home ISP accounts now have complementary A-PTR
> records, making spam detection much harder.

Their users may need to send mail after all.

-- 
Vincent Lefèvre <vincent@vinc17.net> - Web: <https://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <https://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

[toc] | [prev] | [next] | [standalone]

Page 1 of 2  [1] 2  Next page →

Back to top | Article view | linux.debian.user

csiph-web