Subject: [fwtk-users] Update on Spam Filtering

From: Ted Keller <[email-masked]>
Date: Sun, 10 Mar 2002 09:25:47 -0500 (EST)

Frequently, I've seen a number of messages slip throught the spamfiltering
routings that I've included in the smap routines.  It took me a while -
but I finally caught one of those messages - and determined that the
message body was actually base64 encoded - thank you very much.  It took
me a while to catch these because our internal exchange system would
decode these into normal text messages - when they would bounce, the
encodeing was lost.  So it was important to catch one of these (actually
serveral) on the way in - before it got to a down-stream mailer.  That
allowed me to inspect the message - and have a few beers as I decided how
to approach this little encoding problem.

To fix these - and being very lazy - I grabbed the base64 routines from
the apache code.  They had done all the work - debugged it - and make a
very stable package for base64 decoding.  Then I included just enough code
to provide basic mime-type recognition - and bingo - I got messages back
into ascii that were able to be processed by the content filters.

What has this done - well - I've increased my hit rate by some 25% based
on total messages captured.  Tipically, I filter out some 40,000 messages
weekly.  With this addition, last week (and I didn't have it in all week)
I exceeded 50,000 messages - and my postmaster mailbox was reduced down to
near zero!

Keep in mind - this doesn't do virus checking - and it only looks at
base64 encodeing.  There may be others out there that I haven't hit yet.
I use commercial products for the virus protection stuff and they do their
job very well.

I will be packaging this up shortly if any of you are interested.  Just
drop me a line and I will tell you where to get it.  The package will also
include the specific apache modules - direct from the apache source kits.
My belief is this is in line with their licensing agreements since the
copyrights are present in their routines.  They've done a good job - they
should get the credit.

Let me know if your are interested.

Keith - this should be quality enough to post on the fwtk.org site.  Would
like to have an offline conversation to verify it stays inline with the
fwtk licensing as well.

ted keller

---
You are currently subscribed to fwtk-users as: [email-masked]
To unsubscribe send a blank email to [email-masked]

---

From: "Don" <[email-masked]>
Date: Mon, 11 Mar 2002 09:22:38 -0500

Hi Ted,

As before, please let me know when your update is ready.

Thanks,
Don

----- Original Message ----- 
From: "Ted Keller" <[email-masked]>
To: "fwtk-users" <[email-masked]>
Sent: Sunday, March 10, 2002 9:25 AM
Subject: [fwtk-users] Update on Spam Filtering

---

---

From: Boringer, Heiko <[email-masked]>
Date: Sat, 16 Mar 2002 18:05:40 +0100

Hello Ted,

where can i get the latest smap-release. My old one from fwtk 2.1 will not
run on my Kernel 2.4.17. Smap -d shows a loop an delivers no mails.

Please send me the link to the latest smap.tar.
kind regards
Heiko Boringer

-----Ursprungliche Nachricht-----
Von: Ted Keller [mailto:[email-masked]]
Gesendet: Sonntag, 10. Marz 2002 15:26
An: fwtk-users
Betreff: [fwtk-users] Update on Spam Filtering

ted keller

---
You are currently subscribed to fwtk-users as: [email-masked]
To unsubscribe send a blank email to [email-masked]

---

---

From: Rick Murphy <[email-masked]>
Date: Sat, 16 Mar 2002 19:58:14 -0500

>run on my Kernel 2.4.17. Smap -d shows a loop an delivers no mails.

Not speaking for Ted, but a smap patch isn't likely to help you..
Mail is delivered by sendmail, run from smapd. Changing smap will probably 
not fix your problem.

Is smapd running? Are the incoming mail messages being written to smapd's 
queue directory? What error messages are being logged by smap/smapd when a 
message is processed?
         -Rick

---