Simscan: Difference between revisions

From QmailToaster
Jump to navigation Jump to search
No edit summary
No edit summary
 
(10 intermediate revisions by the same user not shown)
Line 1: Line 1:
[[Configuration#Simscan|Back]]<br>
== Scanning ==
== Scanning ==


Line 16: Line 17:
  This tells simscan for domain.tld to scan with Dspam, Rspam, ClamAV, Spamassassin,...<br>
  This tells simscan for domain.tld to scan with Dspam, Rspam, ClamAV, Spamassassin,...<br>
  Upon edit of simcontrol the file must be put into proper format for use, run
  Upon edit of simcontrol the file must be put into proper format for use, run
  # qmailctl cdb
  # qmailctl cdb<br>
Note: simscan automatically runs dspam client (/usr/bin/dspamc). So the dspam server must be running.
      In order to run dspam (/usr/bin/dspam) instead set the environment variable DSPAM_BIN in either 
      /etc/tcprules.d/tcp.smtp or in the smtp run script.


== Ramdisk ==
== Ramdisk ==
Line 37: Line 41:
== Filtering ==
== Filtering ==


FEDORAREPO=
  Install Dspam, Rspam, SA userprefs<br>
  Install Dspam, Rspam, SA userprefs
  '''# Fedora EL8'''<br>
 
  cat >> /etc/yum.repos.d/fedoramirrors << EOF
  # Begin Fedora Install CentOS 8 ***** ONLY *****
https://d2lzkl7pfhq30w.cloudfront.net/pub/archive/fedora/linux/releases/28/Everything/x86_64/os/
  rel=`grep "release 8" /etc/*-release`
http://mirror.math.princeton.edu/pub/fedora-archive/fedora/linux/releases/28/Everything/x86_64/os/
  if [[ ! -z $rel ]]
http://pubmirror1.math.uh.edu/fedora-buffet/archive/fedora/linux/releases/28/Everything/x86_64/os/
  then
https://pubmirror2.math.uh.edu/fedora-buffet/archive/fedora/linux/releases/28/Everything/x86_64/os/
  rel=8
http://mirrors.kernel.org/fedora-buffet/archive/fedora/linux/releases/28/Everything/x86_64/os/
  FEDORAREPO=--enablerepo=fedora
https://dl.fedoraproject.org/pub/archive/fedora/linux/releases/28/Everything/x86_64/os/
  sites=( https://d2lzkl7pfhq30w.cloudfront.net/pub/archive/fedora/linux/releases/28/Everything/x86_64/os/  
EOF<br>
  http://mirror.math.princeton.edu/pub/fedora-archive/fedora/linux/releases/28/Everything/x86_64/os/  
cat >> /etc/yum.repos.d/fedora28.repo << EOF
  http://pubmirror1.math.uh.edu/fedora-buffet/archive/fedora/linux/releases/28/Everything/x86_64/os/  
  [fedora]
  https://pubmirror2.math.uh.edu/fedora-buffet/archive/fedora/linux/releases/28/Everything/x86_64/os/  
  name=Fedora 28
  http://mirrors.kernel.org/fedora-buffet/archive/fedora/linux/releases/28/Everything/x86_64/os/  
mirrorlist=file:///etc/yum.repos.d/fedoramirrors
  https://dl.fedoraproject.org/pub/archive/fedora/linux/releases/28/Everything/x86_64/os/ )
enabled=0
  printf '%s\n%s\n%s\n%s\n%s\n%s\n' '[fedora]' 'name=Fedora 28' 'mirrorlist=file:///etc/yum.repos.d/fedoramirrors' \
gpgcheck=0
        'enabled=0' 'gpgcheck=0' 'priority=100' > /etc/yum.repos.d/fedora28.repo
priority=100
  printf '%s\n%s\n%s\n%s\n%s\n%s\n' "${sites[0]}" "${sites[1]}" "${sites[2]}" "{$sites[3]}" "${sites[4]}" "${sites[5]}" \
EOF<br>
        > /etc/yum.repos.d/fedoramirrors
'''# Fedora EL9'''<br>
  else
cat >> /etc/yum.repos.d/fedoramirrors << EOF
    rel=7
https://d2lzkl7pfhq30w.cloudfront.net/pub/archive/fedora/linux/releases/34/Everything/x86_64/os/
  fi
http://mirror.math.princeton.edu/pub/fedora-archive/fedora/linux/releases/34/Everything/x86_64/os/
  # End Fedora Install CentOS 8 ***** ONLY *****
http://pubmirror1.math.uh.edu/fedora-buffet/archive/fedora/linux/releases/34/Everything/x86_64/os/
https://pubmirror2.math.uh.edu/fedora-buffet/archive/fedora/linux/releases/34/Everything/x86_64/os/
http://mirrors.kernel.org/fedora-buffet/archive/fedora/linux/releases/34/Everything/x86_64/os/
https://dl.fedoraproject.org/pub/archive/fedora/linux/releases/34/Everything/x86_64/os/
EOF<br>
cat >> /etc/yum.repos.d/fedora34.repo << EOF
[fedora]
name=Fedora 34
mirrorlist=file:///etc/yum.repos.d/fedoramirrors
enabled=0
gpgcheck=0
priority=100
EOF<br>
'''# Fedora EL10'''<br>
cat >> /etc/yum.repos.d/fedora41.repo << EOF
[fedora41]
name=Fedora 41 RPM - x86_64
baseurl=https://dl.fedoraproject.org/pub/fedora/linux/releases/41/Everything/\$basearch/os/
enabled=0
gpgcheck=0
priority=100
[fedora41-source]
name=Fedora 41 SRPM - x86_64 Source
baseurl=https://dl.fedoraproject.org/pub/fedora/linux/releases/41/Everything/source/tree/
  enabled=0
gpgcheck=0
  priority=100
  EOF


  # Begin DSpam DB Install, Skip if already installed
  # Begin DSpam DB Install, Skip if already installed

Latest revision as of 15:13, 21 August 2025

Back

Scanning

In /etc/tcprules.d/tcp.smtp set environment variable QMAILQUEUE to simscan binary 
  ...,QMAILQUEUE="/var/qmail/bin/simscan",...
Rebuild tcp.smtp
# qmailctl cdb

Control

Simscan's control file is /var/qmail/control/simcontrol
It's format looks like thus:
:clam=yes,spam=yes,spam_hits=9.5,spam_passthru=yes,attach=.vbs:.lnk:.scr:.wsh:.hta:.pif This tells simscan for all domains to scan with ClamAV, Spamassassin, filter attachments, queueing all mail after filtering
Consider the following format:
domain.tld:dspam=yes,rspam=yes,clam=yes,spam=yes,spam_passthru=yes,... This tells simscan for domain.tld to scan with Dspam, Rspam, ClamAV, Spamassassin,...
Upon edit of simcontrol the file must be put into proper format for use, run # qmailctl cdb
Note: simscan automatically runs dspam client (/usr/bin/dspamc). So the dspam server must be running. In order to run dspam (/usr/bin/dspam) instead set the environment variable DSPAM_BIN in either /etc/tcprules.d/tcp.smtp or in the smtp run script.

Ramdisk

Use htop or other program to determine available memory for ramdisk (size=)
 # clamgid=$(test -z "$(cat /etc/passwd | grep ^clamav)" && id -g clamscan 2>/dev/null || id -g clamav 2>/dev/null) && echo $clamgid
 # qmailctl stop
 # mount -t tmpfs -o size=1024m,nodev,noexec,noatime,uid=$clamgid,gid=0,mode=0750 myramdisk /var/qmail/simscan
 # ls -ld /var/qmail/simscan
    drwxr-x--- 2 clamscan root 6 Aug 17 13:28 /var/qmail/simscan
 # qmailctl start
 # df -h /var/qmail/simscan
    Filesystem      Size  Used Avail Use% Mounted on
    myramdisk       1.0G     0  1.0G   0% /var/qmail/simscan
 # cp /etc/fstab /etc/fstab.bak
 # echo "myramdisk /var/qmail/simscan tmpfs size=1024m,nodev,noexec,noatime,uid=$clamgid,gid=0,mode=0750 0 0" >> /etc/fstab
 Make sure /etc/fstab file is okay before a reboot
 # cat /etc/fstab

Filtering

Install Dspam, Rspam, SA userprefs
# Fedora EL8
cat >> /etc/yum.repos.d/fedoramirrors << EOF https://d2lzkl7pfhq30w.cloudfront.net/pub/archive/fedora/linux/releases/28/Everything/x86_64/os/ http://mirror.math.princeton.edu/pub/fedora-archive/fedora/linux/releases/28/Everything/x86_64/os/ http://pubmirror1.math.uh.edu/fedora-buffet/archive/fedora/linux/releases/28/Everything/x86_64/os/ https://pubmirror2.math.uh.edu/fedora-buffet/archive/fedora/linux/releases/28/Everything/x86_64/os/ http://mirrors.kernel.org/fedora-buffet/archive/fedora/linux/releases/28/Everything/x86_64/os/ https://dl.fedoraproject.org/pub/archive/fedora/linux/releases/28/Everything/x86_64/os/ EOF
cat >> /etc/yum.repos.d/fedora28.repo << EOF [fedora] name=Fedora 28 mirrorlist=file:///etc/yum.repos.d/fedoramirrors enabled=0 gpgcheck=0 priority=100 EOF
# Fedora EL9
cat >> /etc/yum.repos.d/fedoramirrors << EOF https://d2lzkl7pfhq30w.cloudfront.net/pub/archive/fedora/linux/releases/34/Everything/x86_64/os/ http://mirror.math.princeton.edu/pub/fedora-archive/fedora/linux/releases/34/Everything/x86_64/os/ http://pubmirror1.math.uh.edu/fedora-buffet/archive/fedora/linux/releases/34/Everything/x86_64/os/ https://pubmirror2.math.uh.edu/fedora-buffet/archive/fedora/linux/releases/34/Everything/x86_64/os/ http://mirrors.kernel.org/fedora-buffet/archive/fedora/linux/releases/34/Everything/x86_64/os/ https://dl.fedoraproject.org/pub/archive/fedora/linux/releases/34/Everything/x86_64/os/ EOF
cat >> /etc/yum.repos.d/fedora34.repo << EOF [fedora] name=Fedora 34 mirrorlist=file:///etc/yum.repos.d/fedoramirrors enabled=0 gpgcheck=0 priority=100 EOF
# Fedora EL10
cat >> /etc/yum.repos.d/fedora41.repo << EOF [fedora41] name=Fedora 41 RPM - x86_64 baseurl=https://dl.fedoraproject.org/pub/fedora/linux/releases/41/Everything/\$basearch/os/ enabled=0 gpgcheck=0 priority=100 [fedora41-source] name=Fedora 41 SRPM - x86_64 Source baseurl=https://dl.fedoraproject.org/pub/fedora/linux/releases/41/Everything/source/tree/ enabled=0 gpgcheck=0 priority=100 EOF
# Begin DSpam DB Install, Skip if already installed
wget https://raw.githubusercontent.com/qmtoaster/dspam/master/dspamdb.sql
if [ "$?" != "0" ]; then
  echo "Error downloading dspam db: ($?), exiting..."
  exit 1
fi
MYSQLPW=
# Get DB password for administrator and check validity.
if [ -z "$MYSQLPW" ]; then
   read -s -p "Enter MySQL/MariaDB admin password to create dspam database: " MYSQLPW
fi
credfile=~/sql.cnf
echo -e "[client]\nuser=root\npassword='$MYSQLPW'\nhost=localhost" > $credfile
mysqladmin --defaults-extra-file=$credfile status > /dev/null 2>&1
if [ "$?" != "0" ]; then
  echo "Bad MySQL/MariaDB administrator password or MySQL/MariaDB is not running. Exiting..."
  exit 1
fi
echo ""
echo "Dropping Dspam database if it exists already..."
mysql --defaults-extra-file=$credfile -e "use dspam" &> /dev/null
[ "$?" = "0" ] && mysqldump --defaults-extra-file=$credfile dspam > dspam.sql \
              && mysql --defaults-extra-file=$credfile -e "drop database dspam" \
              && echo "dspam db saved to dspam.sql and dropped..."
# Create dspam with correct permissions
echo "Creating Dspam database..."
mysqladmin --defaults-extra-file=$credfile reload
mysqladmin --defaults-extra-file=$credfile refresh
mysqladmin --defaults-extra-file=$credfile create dspam
mysqladmin --defaults-extra-file=$credfile reload
mysqladmin --defaults-extra-file=$credfile refresh
echo "Adding dspam users and privileges..."
mysql --defaults-extra-file=$credfile -e "CREATE USER dspam@localhost IDENTIFIED BY 'p4ssw3rd'"
mysql --defaults-extra-file=$credfile -e "GRANT ALL PRIVILEGES ON dspam.* TO dspam@localhost"
mysqladmin --defaults-extra-file=$credfile reload
mysqladmin --defaults-extra-file=$credfile refresh
echo "Done with dspam database..."
mysql --defaults-extra-file=$credfile dspam < dspamdb.sql
mysqladmin --defaults-extra-file=$credfile reload
mysqladmin --defaults-extra-file=$credfile refresh
# End DSpam DB install


# Rspam Installation:
wget https://rspamd.com/rpm-stable/centos-$rel/rspamd.repo -O /etc/yum.repos.d/rspamd.repo
dnf $FEDORAREPO --enablerepo=qmt-devel install dspam dspam-libs dspam-client dspam-mysql dspam-web rspamd
systemctl enable --now dspam
systemctl status dspam
systemctl enable --now rspamd
systemctl status rspamd
# Update Simscan
dnf --enablerepo=qmt-devel update simscan
# Up qmail limits
sed -i 's/softlimit -m.*\\/softlimit -m 256000000 \\/' /var/qmail/supervise/smtp/run
qmailctl stop
qmailctl start


# SA user prefs
# Create SA DB and load table into MySQL
cat >> ./sadb.sql << EOF
CREATE TABLE userpref (
 username varchar(100) NOT NULL default ,
 preference varchar(50) NOT NULL default ,
 value varchar(255) NOT NULL default ,
 prefid int(11) NOT NULL auto_increment,
 ts timestamp NOT NULL DEFAULT current_timestamp() ON UPDATE current_timestamp(),
 PRIMARY KEY  (prefid),
 KEY username (username)
) ENGINE=InnoDB;
EOF
mysqladmin --defaults-extra-file=$credfile create spamassassin
mysql --defaults-extra-file=$credfile -e "CREATE USER spamassassin@localhost IDENTIFIED BY 'p4ssw3rd'"
mysql --defaults-extra-file=$credfile -e "GRANT ALL PRIVILEGES ON spamassassin.* TO spamassassin@localhost"
mysqladmin --defaults-extra-file=$credfile reload
mysqladmin --defaults-extra-file=$credfile refresh
mysql --defaults-extra-file=$credfile spamassassin < sadb.sql
# Read SA SQL DB
cat >> /etc/mail/spamassassin/sql.cf << EOF
user_scores_dsn                  DBI:mysql:spamassassin:localhost:3306
user_scores_sql_password         p4ssw3rd
user_scores_sql_username         spamassassin
user_scores_sql_custom_query     SELECT preference, value FROM _TABLE_ WHERE username = _USERNAME_ OR username = '$GLOBAL' OR username = 
CONCAT('%',_DOMAIN_) ORDER BY username ASC
EOF
echo "include sql.cf" >> /etc/mail/spamassassin/local.cf
# Add Per User Settings in SA SQL DB (Yours may differ)
echo "INSERT INTO userpref (username,preference,value) VALUES ('\$GLOBAL','required_hits','5.0');" | mysql -u root -p spamassassin
echo "INSERT INTO userpref (username,preference,value) VALUES ('user@dom.com','required_hits','7.0');" | mysql -u root -p spamassassin
echo "INSERT INTO userpref (username,preference,value) VALUES ('globalspam','required_hits','5.0');" | mysql -u root -p spamassassin
echo "select * from userpref" | mysql -u root -p spamassassin
# Tell SA Daemon To Use SQL DB
sed -i 's/SPAMDOPTIONS=".*"/SPAMDOPTIONS="--create-prefs -m10 -q -x -u clamscan"/' /etc/sysconfig/spamassassin
# Bayesian Filtering
mkdir /etc/mail/spamassassin/.spamassassin
chown -R clamscan:clamscan /etc/mail/spamassassin/.spamassassin
cat >> /etc/mail/spamassassin/bayes.cf << EOF
use_bayes 1
use_bayes_rules 1
bayes_auto_learn 1
bayes_path /etc/mail/spamassassin/.spamassassin/bayes
bayes_auto_learn_threshold_spam 6.0
bayes_file_mode 0775
EOF
echo "include bayes.cf" >> /etc/mail/spamassassin/local.cf
systemctl restart spamassassin


# Tell simscan, append to simcontrol, reload cdb
dspam=yes,rspam=yes,regex
optional: dspamuser=globalspam (Otherwise the first envelope 'rcpt to' address is used)
optional: spamuser=globalspam (Add to userprefs table)
optional: remove spam_hits=* (Otherwise userprefs will not be used)
qmailctl cdb
# This is my configuration:
:clam=yes,spam=yes,rspam=yes,dspam=yes,spam_passthru=yes,attach=.exe:.pif:.scr