Rspamd + Dovecot + ISPConfig: Automatic Spam/Ham Training
How to make Rspamd train spam/ham automatically.
If an email is moved to the IMAP Junk folder, Rspamd should automatically learn it as spam. If the email is moved out of the Junk folder again, it should automatically learned again as ham by Rspamd. The more users do this, the faster the spam filter is trained because the settings apply server-wide.
A correctly installed ISPConfig-server with Rspamd on Ubuntu according to the Perfect Server Setup is required for this tutorial.
Tested with Ubuntu 20 and Debian 12.
Rspamd manual learning
Spam
Example of using a user’s Junk folder for Rspamd spam training (unwanted mails)
rspamc learn_spam /var/vmail/domain.xx/username/Maildir/.Junk/cur
Results for file: cur/1638796375.M239907P2872704.ispconfig.serv.er,S=4305,W=4397:2,Sb (0.035 seconds)
error = "<[email protected]> has been already learned as spam, ignore it";
filename = "cur/1638796375.M239907P2872704.ispconfig.serv.er,S=4305,W=4397:2,Sb";
scan_time = 0.036000;
Results for file: cur/1638786629.M743027P2830658.ispconfig.serv.er,S=57675,W=58742:2,ST (0.055 seconds)
success = true;
filename = "cur/1638786629.M743027P2830658.ispconfig.serv.er,S=57675,W=58742:2,ST";
scan_time = 0.056000;Ham
Example of using a user’s INBOX for Rspamd ham training (harmless mails)
rspamc learn_ham /var/vmail/domain.xx/username/Maildir/cur
Automatic Rspamd training with Dovecot
Configure Dovecot for ISPConfig
Don’t edit /etc/dovecot.conf directly because it will be overwritten after each ISPConfig update. Edit /usr/local/ispconfig/server/conf-custom/install/dovecot_custom.conf.master instead and do a Force Update. This will reconfigure /etc/dovecot/dovecot.conf and restart Dovecot.
vi /usr/local/ispconfig/server/conf-custom/install/dovecot_custom.conf.master
protocol imap {
mail_plugins = imap_sieve
}
plugin {
sieve_plugins = sieve_imapsieve sieve_extprograms
# From elsewhere to Spam folder
imapsieve_mailbox1_name = Junk
imapsieve_mailbox1_causes = COPY
imapsieve_mailbox1_before = file:/etc/dovecot/rspamd/rspamd-learn-spam.sieve
# From Spam folder to elsewhere
imapsieve_mailbox2_name = *
imapsieve_mailbox2_from = Junk
imapsieve_mailbox2_causes = COPY
imapsieve_mailbox2_before = file:/etc/dovecot/rspamd/rspamd-learn-ham.sieve
sieve_pipe_bin_dir = /etc/dovecot/rspamd
sieve_global_extensions = +vnd.dovecot.pipe +vnd.dovecot.environment
}Write dovecot.conf and restart Dovecot
ispconfig_update.sh --force
.
.
.
The following local config override templates were found, be sure to incorporate upstream changes if needed:
/usr/local/ispconfig/server/conf-custom/install/dovecot_custom.conf.master
.
.
.
Restarting services ...
Update finished.Check Dovecot configuration
dovecot -n
.
.
.
plugin {
imapsieve_mailbox1_before = file:/etc/dovecot/rspamd/rspamd-learn-spam.sieve
imapsieve_mailbox1_causes = COPY
imapsieve_mailbox1_name = Junk
imapsieve_mailbox2_before = file:/etc/dovecot/rspamd/rspamd-learn-ham.sieve
imapsieve_mailbox2_causes = COPY
imapsieve_mailbox2_from = Junk
imapsieve_mailbox2_name = *
quota = dict:user::file:/var/vmail/%d/%n/.quotausage
quota_status_nouser = DUNNO
quota_status_overquota = 552 5.2.2 Mailbox is full
quota_status_success = DUNNO
sieve = /var/vmail/%d/%n/.sieve
sieve_after = /var/vmail/%d/%n/.ispconfig.sieve
sieve_before = /var/vmail/%d/%n/.ispconfig-before.sieve
sieve_global_extensions = +vnd.dovecot.pipe +vnd.dovecot.environment
sieve_max_actions = 100
sieve_max_redirects = 25
sieve_max_script_size = 2M
sieve_pipe_bin_dir = /etc/dovecot/rspamd
sieve_plugins = sieve_imapsieve sieve_extprograms
}
.
.
.
protocol imap {
auth_verbose = yes
mail_plugins = imap_sieve quota imap_quota listescape
namespace {
inbox = yes
location =
prefix =
separator = /
type = private
}
}
.
.
.Ports used for Dovecot and Rspamd
egrep "993/tcp|4190/tcp|11332/tcp|11333/tcp|11334/tcp" /etc/services
imaps 993/tcp # IMAP over SSL
sieve 4190/tcp # ManageSieve Protocol
rspamd-proxy 11332/tcp
rspamd-worker 11333/tcp
rspamd-controller 11334/tcp
Check running ports
netstat -unplet | egrep "dovecot|rspamd"
tcp 0 0 0.0.0.0:110 0.0.0.0:* LISTEN 0 19727752 2946685/dovecot
tcp 0 0 0.0.0.0:143 0.0.0.0:* LISTEN 0 19727789 2946685/dovecot
tcp 0 0 0.0.0.0:4190 0.0.0.0:* LISTEN 0 19727736 2946685/dovecot
tcp 0 0 0.0.0.0:993 0.0.0.0:* LISTEN 0 19727791 2946685/dovecot
tcp 0 0 0.0.0.0:995 0.0.0.0:* LISTEN 0 19727754 2946685/dovecot
tcp 0 0 127.0.0.1:11332 0.0.0.0:* LISTEN 121 19729074 2946677/rspamd: mai
tcp 0 0 127.0.0.1:11333 0.0.0.0:* LISTEN 121 19729084 2946677/rspamd: mai
tcp 0 0 127.0.0.1:11334 0.0.0.0:* LISTEN 121 19729079 2946677/rspamd: mai
tcp6 0 0 :::110 :::* LISTEN 0 19727753 2946685/dovecot
tcp6 0 0 :::143 :::* LISTEN 0 19727790 2946685/dovecot
tcp6 0 0 :::4190 :::* LISTEN 0 19727737 2946685/dovecot
tcp6 0 0 :::993 :::* LISTEN 0 19727792 2946685/dovecot
tcp6 0 0 :::995 :::* LISTEN 0 19727755 2946685/dovecotCreate script folder
mkdir -p /etc/dovecot/rspamd
Create sieve scripts
Spam
vi /etc/dovecot/rspamd/rspamd-learn-spam.sieve
require ["vnd.dovecot.pipe", "copy", "imapsieve", "environment", "variables"];
if environment :matches "imap.user" "*" {
set "username" "${1}";
}
pipe :copy "rspamd-learn-spam.sh" [ "${username}" ];Ham
vi /etc/dovecot/rspamd/rspamd-learn-ham.sieve
require ["vnd.dovecot.pipe", "copy", "imapsieve", "environment", "variables"];
if environment :matches "imap.mailbox" "*" {
set "mailbox" "${1}";
}
if string "${mailbox}" "Trash" {
stop;
}
if environment :matches "imap.user" "*" {
set "username" "${1}";
}
pipe :copy "rspamd-learn-ham.sh" [ "${username}" ];Compile sieve scripts
sievec /etc/dovecot/rspamd/rspamd-learn-ham.sieve sievec /etc/dovecot/rspamd/rspamd-learn-spam.sieve
Compiled sieve scripts were created now in /etc/dovecot/rspamd
find /etc/dovecot/rspamd|grep svbin$
/etc/dovecot/rspamd/rspamd-learn-ham.svbin
/etc/dovecot/rspamd/rspamd-learn-spam.svbinSet permissions of sieve scripts
chmod u=rw,go= /etc/dovecot/rspamd/rspamd-learn-{spam,ham}.{sieve,svbin}
chown vmail:vmail /etc/dovecot/rspamd/rspamd-learn-{spam,ham}.{sieve,svbin}Create shell scripts
Spam
vi /etc/dovecot/rspamd/rspamd-learn-spam.sh
#!/bin/sh
exec /usr/bin/rspamc -h localhost:11334 learn_spamHam
vi /etc/dovecot/rspamd/rspamd-learn-ham.sh
#!/bin/sh
exec /usr/bin/rspamc -h localhost:11334 learn_hamSet permissions of shell scripts
chmod u=rwx,go= /etc/dovecot/rspamd/rspamd-learn-{spam,ham}.sh
chown vmail:vmail /etc/dovecot/rspamd/rspamd-learn-{spam,ham}.shCheck permissions of scripts
ls -al /etc/dovecot/rspamd
total 32
drwxr-xr-x 2 root root 4096 Dec 6 20:52 .
drwxr-xr-x 5 root root 4096 Dec 6 19:17 ..
-rwx------ 1 vmail vmail 60 Dec 6 20:48 rspamd-learn-ham.sh
-rw------- 1 vmail vmail 314 Dec 6 19:59 rspamd-learn-ham.sieve
-rw------- 1 vmail vmail 452 Dec 6 20:07 rspamd-learn-ham.svbin
-rwx------ 1 vmail vmail 61 Dec 6 20:49 rspamd-learn-spam.sh
-rw------- 1 vmail vmail 199 Dec 6 19:58 rspamd-learn-spam.sieve
-rw------- 1 vmail vmail 354 Dec 6 20:05 rspamd-learn-spam.svbinRestart services
service dovecot restart service rspamd restart
Finished!
From now on, any email that is moved to the Junk folder will be automatically learned as spam by Rspamd. If the message is moved out of the Junk folder again, Rspamd will automatically learn it as ham again.
Rspamd logfile
While moving an email from or to the Junk folder
tail -f /var/log/rspamd/rspamd.log|grep controller
2021-12-06 21:06:39 #2970301(controller) <ccf5c6>; csession; rspamd_controller_check_password: allow unauthorized connection from a trusted IP 127.0.0.1
2021-12-06 21:06:39 #2970301(controller) <ccf5c6>; csession; rspamd_message_parse: loaded message; id: <[email protected]>; queue-id: <undef>; size: 4397; checksum: <e33773d1266d71b688d27d4b2da4d60e>
2021-12-06 21:06:39 #2970301(controller) <ccf5c6>; csession; rspamd_mime_part_detect_language: detected part language: en
2021-12-06 21:06:39 #2970301(controller) <ccf5c6>; csession; rspamd_controller_learn_fin_task: <127.0.0.1> learned message as ham: [email protected]
2021-12-06 21:08:12 #2970301(controller) <a983e5>; csession; rspamd_controller_check_password: allow unauthorized connection from a trusted IP 127.0.0.1
2021-12-06 21:08:12 #2970301(controller) <a983e5>; csession; rspamd_message_parse: loaded message; id: <[email protected]>; queue-id: <undef>; size: 4397; checksum: <e33773d1266d71b688d27d4b2da4d60e>
2021-12-06 21:08:12 #2970301(controller) <a983e5>; csession; rspamd_mime_part_detect_language: detected part language: en
2021-12-06 21:08:12 #2970301(controller) <a983e5>; csession; rspamd_controller_learn_fin_task: <127.0.0.1> learned message as spam: [email protected]Rspamd GUI
The number of emails learned is constantly increasing.
Rspamd configuration
ISPConfig GUI
Email → Spamfilter-Policy → Normal → Rspamd
