Subject: [fwtk-users] Smap Message: peer dropped connection

From: Paul Dunphy <[email-masked]>
Date: Wed, 17 Oct 2001 16:24:55 -0400

Hi,

I wonder if you might be able to help me: I am running FWTK 2.1 with the 
Yao smap patches. Several times a day, I get messages like the following in 
my syslog:

smap[4228]: xxx.xxx.xxx.xxx host address lookup failed (where 
xxx.xxx.xxx.xxx is the IP address of the sender)
smap[4228]: connect host=unknown/xxx.xxx.xxx.xxx

and, about 20 minutes later:

smap[4228]: peer dropped connection: Success

... and no message is received. In all cases, the FWTK box is unable to 
resolve the IP address of the sender, even though I am 99% certain that our 
DNS stuff is set up correctly (I can resolve most IP addresses without 
problems). Could this be a DNS problem at the sender's end? Must smap be 
able to resolve the sender's IP address before it will accept a message?

Any help you can provide would be greatly appreciated...

Thanks,
Paul

---------------------------------------------------------------------
Paul T. Dunphy, P.Eng.
Systems Administrator/Research Engineer
Ryerson University
Toronto, Ontario, CANADA
E-mail: [email-masked]

---
You are currently subscribed to fwtk-users as: [email-masked]
To unsubscribe send a blank email to [email-masked]

---

From: "Alvaro Munoz-Aycuens Mtnz." <[email-masked]>
Date: Thu, 18 Oct 2001 00:25:41 +0200

Is just the smtp client that doesn't issue a QUIT command at the end, 
example of this broken behaviour is in mozilla mail client.

Hope that helps, regards

Alvaro Munoz

Paul Dunphy wrote:

> Yao smap patches. Several times a day, I get messages like the following 
> in my syslog:
> 
> resolve the IP address of the sender, even though I am 99% certain that 
> our DNS stuff is set up correctly (I can resolve most IP addresses 
> without problems). Could this be a DNS problem at the sender's end? Must 
> smap be able to resolve the sender's IP address before it will accept a 
> message?
> 
> You are currently subscribed to fwtk-users as: [email-masked]

---

---

From: Ted Keller <[email-masked]>
Date: Wed, 17 Oct 2001 18:42:20 -0400 (EDT)

Paul,

smap does not require you to resolve the sender address.  Manytimes -
remote smtp servers do not resolve.  Your DNS is probably fine.

This looks like one of two things.

1. the link (probably theirs) is not very dependable - and drops.

2. their server simply does not follow the smtp protocol - and
particullary one of the following
	a. no trailing . on the data part of the message
	b. the remote server does not fall back to rfc821 smtp protocol.

I suspect it is (b) above.

Check out the remote server

(telnet xxx.xxx.xxx.xxx 25) and see what answers up.

Several versions of Microsoft exchange do not fall back to rfc821
standards.  Even exchange 5.5 without the appropriate patch levels do not
fall back (I have the technet artical somewhere - I can provide it if you
need it).  There are other systems out there as well that fail the same
way.

Hope this helps.

ted keller

On Wed, 17 Oct 2001, Paul Dunphy wrote:

> You are currently subscribed to fwtk-users as: [email-masked]

---

---

From: Rick Murphy <[email-masked]>
Date: Wed, 17 Oct 2001 19:51:57 -0400

>smap[4228]: peer dropped connection: Success

That's caused by having a long spam-blocking list applied to a host that 
doesn't have a good reverse lookup, particularly when the attempt to look 
them up takes a long time. (An immediate negative response isn't a problem 
- the problem happens when a host takes 60 seconds to time out.)

What happens is each "smap: spam ..." line in the netperm-table is passed 
to hostmatch with the caller's IP address. If the spam block pattern isn't 
an IP address, hostmatch has to turn the IP address back into a hostname; 
that has to be done for every spam block entry. It's pretty apparent what 
can happen - if you have a host that takes 30 seconds to time out on a 
reverse lookup and you have 30 spam-block hostnames, you're going to take 
15 minutes to verify if you're going to let them mail to you.

A host that's got this problem will never be able to send you mail unless 
you remove all the spam block entries in your netperm-table.

I fixed this in my version of smap by making a new function called 
"hostmatch2" that gets passed both the hostname and the host's IP address; 
it then calls hostmatch with the appropriate arguments to stop the timeouts.

My patch (relative to FWTK 2.1) is attached.
         -Rick
---
You are currently subscribed to fwtk-users as: [email-masked]
To unsubscribe send a blank email to [email-masked]

---

From: Paul Dunphy <[email-masked]>
Date: Mon, 22 Oct 2001 09:11:10 -0400

Thanks for all your help, folks. The problem appears to have been what Rick 
Murphy suggested; if I shorten the list of blocked spam domains, everything 
seems to work fine. I'll try applying the patch as soon as I get time.

Cheers...
Paul

---------------------------------------------------------------------
Paul T. Dunphy, P.Eng.
Systems Administrator/Research Engineer
Centre for the Study of Commercial Activity
Ryerson University
Toronto, Ontario, CANADA
Phone: (416) 979-5000 ext. 7537
Fax: (416) 979-5378
E-mail: [email-masked]

---

---

From: Rick Murphy <[email-masked]>
Date: Tue, 23 Oct 2001 22:25:46 -0400

At 09:11 AM 10/22/01 -0400, Paul Dunphy wrote:
>Thanks for all your help, folks. The problem appears to have been what 
>Rick Murphy suggested; if I shorten the list of blocked spam domains, 
>everything seems to work fine. I'll try applying the patch as soon as I 
>get time.

Just FYI, there are several things that I've added to smap since Joe Yao's 
patch.

First, "verify-reverse". Any "from" address that has a domain that matches 
one of these entries must be sent from a host with a reverse DNS matching 
that entry.
For example, I have:
smap:   verify-reverse msn.com yahoo.com hotmail.com aol.com lycos.com
That means that anyone trying to send with "mail from: <[email-masked]>" 
must be sent from a host that's in the hotmail.com domain. This is a real 
spamkiller - 75% of the junk mail I get forges hotmail or yahoo.

Second, block-list. These entries are used to configure DNS blocking lists.
An example:
smap:   block-list "MAPS RBL" block .blackholes.mail-abuse.org "%s/%s was found
in the MAPS RBL (see http://mail-abuse.org/rbl)"
(all on one line, of course). This defines a block list called "MAPS RBL", 
which blocks mail from any host on the RBL. The 
".blackholes.mail-abuse.org" suffix is added to the host address and looked 
up in the DNS blocking list.
The "block" argument can also be "mark" - in this case, the mail isn't 
rejected, it's marked with an "X-RBL-Warning:" header line that you can 
test using procmail.

The lookup can match a particular return value - for example, the OsiruSoft 
blocking list (relays.osirusoft.com) returns 127.0.0.2 for an open relay; 
you block that by adding the match address after the relay list name:
smap:   block-list "ORL" block .relays.osirusoft.com:127.0.0.2 "%s/%s was found
in the Osirusoft relays list (see http://relays.osirusoft.com/)"

Finally, there's a "whitelist" entry - any host that matches these is 
ignored when checking for spam blocks, reverse checks, etc. This allows you 
to permit mail from an important correspondent that happens to be on one of 
your blocking lists.
         -Rick

---

---

From: "Todd Williams" <[email-masked]>
Date: Wed, 24 Oct 2001 13:48:36 -0400

Hi Rick,

Are your smap patches readily available for download by any chance?  The
reverse lookup especially sounds like a great idea.

Thanks,

Todd

-----Original Message-----
From: Rick Murphy [mailto:[email-masked]]
Sent: Tuesday, October 23, 2001 10:26 PM
To: fwtk-users
Subject: [fwtk-users] Re: Smap Message: peer dropped connection

smap:   block-list "MAPS RBL" block .blackholes.mail-abuse.org "%s/%s was
found

smap:   block-list "ORL" block .relays.osirusoft.com:127.0.0.2 "%s/%s was
found

---
You are currently subscribed to fwtk-users as: [email-masked]
To unsubscribe send a blank email to [email-masked]

---

---

From: Ted Keller <[email-masked]>
Date: Wed, 24 Oct 2001 22:19:47 -0400 (EDT)

Hmm,  This is a really neat idea.  Would like a copy of the verify-reverse
patches.

Seems again that we should look at integrating the three versions of smap
together.  Would like to come up with the "next" version somehow.

tek

---

---

From: Rick Murphy <[email-masked]>
Date: Thu, 25 Oct 2001 06:24:53 -0400

>together.  Would like to come up with the "next" version somehow.

I posted the patch in an earlier message and can resend it if you're 
interested.
I tried merging the various smap versions together but it's a lot of work 
and I don't have a lot of time right now. You're welcome to give it a shot. :-)
         -Rick

---

---

From: Ted Keller <[email-masked]>
Date: Thu, 25 Oct 2001 08:42:37 -0400 (EDT)

Rick,

I would appreciate a resend.  Would like to integrate it into my version -
maybe I'll even get around to integrating your RBL support as well.  I
think with that, I could offer a fairly complete integrated version of all
the major varients.

ted keller

---

---

From: "Todd Williams" <[email-masked]>
Date: Tue, 30 Oct 2001 14:37:02 -0500

Hi,

I've been play testing (on a non-production box attached to the internet -
scary, eh?) some of the code in the patch provided by RMurphy and the
original JSDY patches, and came up with a couple of questions:

1.  Is it the normal response of the verify-reverse and associated routines
to just drop a DNS-spoofed connection without any error (return) codes
displayed to the spoofer?  It seems to me this would make the spoofer's mail
queue continue to attempt connections for days until the message drops dead,
rather than error out due to a 550 return code immediately, and never
return?  Perhaps there's something more to this than I'm seeing.

2.  In my testing with the spam code, I've noticed something odd once the
"spam" has been rejected and dumped into a secure file.  I realize what
causes it - the file is no longer where smap expects it to be.  Are we
missing a close statement somewhere?  Is this normal behavior, and if so,
could it present any problems?
				SMAP  - 451 System problem
		/var/log/messages - fwtksyserr: cannot fprintf: Bad file descriptor

3.  Also in testing the spam code, after smap rejects a message with a 554
Unacceptable address/domain and moves it into a secure file, upon issuing
the "QUIT" command to smap, I received an error message:
				SMAP  - 451 System problem
		/var/log/messages	- fwtksyserr: cannot fprintf: Interrupted system call

Everything else seems to be working okay.

Thanks for your help in advance.

Todd Williams

-----Original Message-----
From: Rick Murphy [mailto:[email-masked]]
Sent: Thursday, October 25, 2001 6:25 AM
To: fwtk-users
Subject: [fwtk-users] Re: Smap Message: peer dropped connection

I posted the patch in an earlier message and can resend it if you're
interested.
I tried merging the various smap versions together but it's a lot of work
and I don't have a lot of time right now. You're welcome to give it a shot.
:-)
         -Rick

---

---

---

From: Rick Murphy <[email-masked]>
Date: Tue, 30 Oct 2001 17:24:19 -0500

>return?  Perhaps there's something more to this than I'm seeing.

Here's what I see:

Escape character is '^]'.
220 fw.itm-inst.com ESMTP/smap Ready; NO UCE Tue, 30 Oct 01 17:03:03 -0500
helo there
250 (there) pleased to meet you.
mail from:<[email-masked]>
250 <[email-masked]>... Sender Ok
rcpt to:<[email-masked]>
550 From address blocked - attempted spoof detected
quit
221 Closing connection
Connection closed by foreign host.

So, it's not immediately disconnecting.

>                 /var/log/messages - fwtksyserr: cannot fprintf: Bad file 

I don't use the "max-email" feature, so there's likely some problem there.
Try editing line 1790 to read:

                                rejecting = 1;
                         } else                          <-- edit here

                         /*
                         ** Read the body, copy from stdin to vout,
                         ** return 0 for good, otherwise various error
                         ** values.
                         */

>                 /var/log/messages       - fwtksyserr: cannot fprintf: 

This is probably the same problem as above.
I'll have a look at what's going on.
         -Rick

---

---

From: "Todd Williams" <[email-masked]>
Date: Wed, 31 Oct 2001 10:42:52 -0500

Hey folks,

Good news.  The "else" statement was an easy fix for the bad file descriptor
problem.

It would appear that the max-email and max-dirent (along with the associated
"secure" file handling stuff) seems to alter the behavior of smap
significantly as far as the verify-reverse, and possibly other parts of the
code go.

This example is with the max-email and max-dirent entries configured in
netperm-table:
(The names and numbers have been changed to protect the innocent.)

$ telnet thistestdomain.com 25
Trying thistestdomain.com...
Connected to thistestdomain.com.
Escape character is '^]'.
220 thistestdomain.com ESMTP/smap Ready; NO UCE Tue, 30 Oct 01
20:21:53 -0500
helo there
250 (there) pleased to meet you.
mail from:<[email-masked]>
250 <[email-masked]>... Sender Ok
rcpt to:<[email-masked]>
250 <[email-masked]> recipient Ok
DATA
354 Enter mail, end with "." on a line by itself
Testing 123
<INSERT SINGLE PERIOD HERE>
554 Unacceptable address/domain: <[email-masked]> [yahoo.com]/
Connection closed by foreign host.

When I disable the max-email and max-dirent entries in netperm-table, and
perform the same test, I see what you saw:   550 From address blocked -
attempted spoof detected

It does look like there might be problem(s) with the max-email/max-dirent
code and how it interacts with the rest of the code and patches.

Thanks for all your help with this,

Todd Williams

-----Original Message-----
From: Rick Murphy [mailto:[email-masked]]
Sent: Tuesday, October 30, 2001 5:24 PM
To: fwtk-users
Subject: [fwtk-users] Re: Smap Message: peer dropped connection

Here's what I see:

                                rejecting = 1;
                         } else                          <-- edit here

                         /*

---

---

From: Joseph S D Yao <[email-masked]>
Date: Wed, 31 Oct 2001 13:17:57 -0500

Caveat: I have not looked at your specific code yet [with Rick's
changes].  I did notice that in some cases the original 'smap' sent an
inappropriate error code - sometimes while the sender was still
sending!  The latter behaviour is guaranteed to make the remote machine
try again.  I changed at least the maxbytes code to read the WHOLE
message [and not save it] before sending an appropriate error code
[something like, "E-go 'way, ya bother me"].

There may be similar problems elsewhere in the code.  While I tried to
review it all for security problems, I did not have enough time to test
all code paths.

-- 
Joe Yao				[email-masked] - Joseph S. D. Yao
OSIS Center Systems Support					EMT-B
-----------------------------------------------------------------------
   This message is not an official statement of OSIS Center policies.

---