Fighting spam with amavis

Recently I set up a spam-filter for my Kolab 3.0 server system. There are plenty of how-tos out there, so I only highlight some bits of my setup. Since I’m running Kolab on Ubuntu Server 12.04 (always use LTS for servers!), I started off installing the package for the clamav and spamassassin wrapper amavis and configured it according Combating Spam with a preseeded a (non-sql) bayes databayes and did some testing.

Instead of /etc/amavis/amavisd.conf, I used /etc/amavis/conf.d/40-kolab to setup my config (conf.d seems to be the default for ubuntu), which you can check out on pastebin.

Besides the bayes database I use network checks  pyzor razor2 and dcc, that detect bulk spam and others by using checksum databases. Because of it’s license dcc is not included in most linux distribution and has to be compiled from source. For instructions, read here. Pyzor and razor can be installed and setup easily. To actually use them you have to enable the plugins in the /etc/spamassassin/*.pre files and to set the use_plugin parameter to 1 in local.cf. Check out my spamassassin configuration on pastebin.

If you use a firewall you have to allow some things for the network requests to work, namely the UDP ports 6277 and 24441 for dcc and pyzor respectively and TCP port 2307 for razor2 for INPUT.

Site-wide spam trap

I experience the bayes database, though preseeded, isn’t good at classifying spam and ham, so we have to have some test-phase, in which you should be able to manually sort spam and ham. In my setup amavis is configured to reinject mail that reaches the spam cutoff level (e.g. 7) to be dropped, but to foreward a copy of that mail to the quarantine folder. This folder should have only post permissions for anyone (really the “anyone”-identifier!) to enable cyrus to post mail there.

In a private setup (and only there!) you can think about setting up the shared/Spam@domain.com as writable for everyone, meaning that users are able to read and delete (potential) spam of other users. In such a case you could create the trap as shared folder in kolab-webadmin and set the write permissions for all your users e.g.

sudo kolab sam shared/Spam@domain.com lrswipted anyone

or with a bash for-loop if you want it not for everyone

for user in a b c d e; do
    sudo kolab sam shared/Spam@domain.com lrswipted $user;
done

You should not even consider to do this unless you are in a really trustworthy environment.

Per-user spam traps

A more convincing design would be to have a global spam trap without reading permissions for ordinary users and a spam folder in every users INBOX. For this, setup a global spam trap in the shared namespace, e.g.

sudo kolab cm  shared/Spam@domain.com
sudo kolab sam shared/Spam@domain.com anyone p

You can also handle this folder as a shared folder created by kolab-webadmin (with address Spam@domain.com, nice ;)) but then I had problems removing permissions, so I canceled that. For the personal spam folders, add something like

autocreate_folders = {
          'Spam': {
              },
    }

to /etc/kolab/kolab.conf to have it created for new users. Also roundcubemail has a section

$rcmail_config['kolab_folders_mail_junkemail'] = 'Spam';

in /etc/roundcubemail/kolab_folders.inc.php. After all folders are created (also do it for existing users!) you have to tell amavis that it should pass spam to that folder. This is done using three directives in our amavis-config 40-kolab

# pass spam to local recipients to user+Spam@domain.com
$recipient_delimiter = '+';
$addr_extension_spam = 'Spam';
$final_spam_destiny = D_PASS;

Make sure that those parameters are not overwritten later on. The addr_extension_spam only works for local domains, but kolab does pull this information from ldap, so no additional amavis configuration is needed for this purpose.

Now restart amavis and send some testmail. If classified as spam, it should go to shared/Spam@domain.com and to user/user.name/Spam@domain.com.

Regularly train your bayes-filter

It is crucial to train the filter, letting it know about your ham and spam. I use a little shell script to achieve this which is run regularly by cron. You can check it out on pastebin. Of course you have to adjust the spool path and the search keywords according to your system. The cron directive for this looks like that

# SPAM AND HAM
# every day learn about new spam and ham
28  11  *  *  *  /usr/local/bin/sa-learn-cronjob
# amavis cronjob
34 11 * * *  /usr/bin/sudo -u amavis /usr/sbin/amavisd-new-cronjob sa-sync
# dcc cronjob
37 11 * * *  /usr/bin/cron-dccd

I use the cronjob for dcc (/usr/bin/cron-dccd) as recommended here.

Spamassassin roundcube plugin

Each user gets different spam, so it is useful if users are able to setup own rules and such. There is a nice roundcube plugin which is unfortunately not default in kolab. Get it and extract it to /usr/share/roundcubemail/plugins/. To configure and enable it, you have to link the config file like

sudo ln -s /usr/share/roundcubemail/plugins/sauserprefs/config.inc.php /etc/roundcubemail/sauserprefs.inc.php

The plugin needs a mysql table which holds spamassassin user preferences. Create this table in roundcube’s mysql database by issuing

CREATE TABLE userpref (
  username varchar(100) NOT NULL default '',
  preference varchar(30) NOT NULL default '',
  value varchar(100) NOT NULL default '',
  prefid int(11) NOT NULL auto_increment,
  PRIMARY KEY  (prefid),
  KEY username (username)
) ENGINE=MyISAM;

Note, that TYPE=MyISAM is not supported anymore in new versions of mysql. For more information see wiki.apache.org/spamassassin. Afterwards setup the database access and maybe more settings in sauserprefs.inc.php and this section to spamassassins local.cf to pull user preferences from the created table. Also here you have to adjust database name, user and password to your system.

# Spamassassin for Roundcubemail
# http://www.tehinterweb.co.uk/roundcube/#pisauserprefs
user_scores_dsn DBI:mysql:ROUNDCUBEMAILDBNAME:localhost:3306
user_scores_sql_password ROUNCUBEMAILPASSWORD
user_scores_sql_username ROUNDCUBEMAILDBUSERNAME
user_scores_sql_custom_query SELECT preference, value FROM _TABLE_ WHERE username = _USERNAME_ OR
username = '$GLOBAL' OR username = CONCAT('%',_DOMAIN_) ORDER BY username ASC

Lastly enable the plugin by adding its name to your plugins array in config.inc.php

$config['plugins'] = array('sauserprefs',...);

I hope I covered the setup in such detail that you can follow it without much efford. If not, just comment or e-mail me🙂

About

Peter Pan. Kann fliegen mit Feenstaub.

Tagged with: , , , ,
Posted in Linux, Technik
4 comments on “Fighting spam with amavis
  1. Stefan says:

    Thank you for this walktrough. It helped me a lot.

    For my multidomain Setup on Debian, I had to do the following additional tasks to get delivery work to the quarantine user and to the spam Folder of each Mailbox:

    In /etc/postfix/transport for each Domain set
    shared@example.com lmtp:unix:/var/lib/imap/socket/lmtp
    to enable delivery to a shared Folder in that Domain at all.

    In /etc/postfix/ldap/virtual_alias_maps_sharedfolders*.cf set
    result_format = shared+shared/%s
    to get mails delivered to the shared Folder.
    Note, the first shared coresponds to the value of postuser in /etc/imapd.conf

    In /etc/postfix/virtual_alias_maps_manual.cf set
    spam@example.com shared+shared/spam@example.ext
    to be able to let Spammer write to a more convenient mail address (spam@example.com)
    Execute postmap /etc/postfix/virtual_alias_maps_manual.cf for the changes to take effect.
    Restart postfix (Service postfix restart)

    In 40-kolab set $spam_quarantine_to = ‘shared+shared/spam@example.com’
    to have spam quarantined to the shared Folder

    For Delivery of Spam to the users Spam Folder I had to set permissions for each Folder like
    kolab sam user/username/Spam@example.com anyone p
    otherwise the Spam marked mails went to the inbox instead of the spam Folder.

    If someone has a good idea on how to set the permission of the users Spam Folder on Mailbox creation with kolab-webadmin, I would very appretiate to hear about it.

  2. laclaro says:

    I have to add, that the sauserprefs plugin does not work with amavis just like that. You have to configure amavis accept sql userprefs by spamassassin. You have to use the “sa_userconf_maps” and “sa_username_maps” in you amavis configuration which are poorly documented. I did not get it running until now.

  3. […] time ago I blogged about fighting spam with amavis for the Kolab community. Now the story continues by means of the roundcube integration with […]

  4. Yell Ow says:

    Thanx for the How-To. And thx Stefan for the add-on. Helped a lot.🙂

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Posts by topic…
…by month
Have a look at…


%d bloggers like this: