Subject: queue filling up... I/O errors

From: [email-masked]
Date: Thu, 19 Apr 2001 19:33:00 -0000 (GMT)

[To be removed from this list send the message "unsubscribe fwtk-users" in the
BODY of a mail message to [email-masked].]

Hello everybody,

I am encountering pretty heavy problems with smap / smapd /sendmail.
While my config generally works flawlessly, messages to a couple of
hosts seem to time out during data transfer. I can easily reach the
relevant mail exchangers and since it's clearly not a length-based
problem, I don't think my ip-filter config is the culprit. If this were
the case, it should work during the next re-send, which it doesn't. 

A lot of these mails go out to yahoo and from the list archive I could
there have already been problems reported with them. Unfortunately it's
not yahoo exclusively, neither can I lock it down to one certain MTA
smap doesn't like talking to. 

Mysteriously, some mails are being sent without any problems while
others land in the queue because of I/O errors or timeouts or
connection resets. This seems to happen especially with forwarded
messages, but then again, not exclusively. 

I am totally clueless. 

>From the sendmail homepage I could tell there were certain
incompatibilities reported - I couldn't find any specific for OpenBSD
2.8 and the sendmail version that comes shipped with it (is there a
simple way to find out which version it is?). 

Is this maybe a platform-related problem? Anyone else using OpenBSD
2.8??

I'll attach my queue output to illustrate the issue. Maybe some of you
have problem with certain of the listed hosts as well. 

This is really giving me a bad headache, so I'm very grateful for every
helpful hint. FWTK version's 2.1, of course, smap has the Yao-patch
applied. 

Cheers, Jan

                /var/spool/mqueue (28 requests)
----Q-ID---- --Size-- -----Q-Time-----
------------Sender/Recipient------------
f3JEIu417172*   20484 Thu Apr 19 16:18 <[email-masked]>
                 (I/O error)
                                       <[email-masked]>
f3JEluw31730   104950 Thu Apr 19 16:47 <[email-masked]>
                 (I/O error)
                                       <[email-masked]>
f3JD1OA03796   100026 Thu Apr 19 15:01 <[email-masked]>
                 (Warning: could not send message for past 4 hours)
                                       <[email-masked]>
f3JBdqj18237*    1634 Thu Apr 19 13:39 <[email-masked]>
                 (<[email-masked]>... reply: read error from
mail.atkon.)
                                       <[email-masked]>
f3JBSMJ13484     1361 Thu Apr 19 13:28 <[email-masked]>
                 (Deferred: 451 Timeout)
                                       <[email-masked]>
f3JBZqe31960     1387 Thu Apr 19 13:35 <[email-masked]>
                 (I/O error: Input/output error)
                                       <[email-masked]>
f3J7JWx06867      700 Thu Apr 19 09:19 <[email-masked]>
                 (Deferred: Connection timed out with
mx2.mail.yahoo.com.)
                                       <[email-masked]>
f3J7obb25545    14271 Thu Apr 19 09:50 MAILER-DAEMON
      8BITMIME   (Deferred: Connection refused by gs3.serverdienst.de.)
                                       <[email-masked]>
f3J701129967     1359 Thu Apr 19 09:00 <[email-masked]>
                 (Deferred: Connection timed out with
mx2.mail.yahoo.com.)
                                       <[email-masked]>
f3J6sVa30689     1385 Thu Apr 19 08:54 <[email-masked]>
                 (I/O error: Input/output error)
                                       <[email-masked]>
f3J23Qx08220    17448 Thu Apr 19 04:03 MAILER-DAEMON
                 (I/O error)
                                       <[email-masked]>
f3IGTCe27668    13212 Wed Apr 18 18:29 MAILER-DAEMON
                 (host map: lookup (Advisor.net): deferred)
                                       <[email-masked]>
f3IEHdD14321    20811 Wed Apr 18 16:17 <[email-masked]>
                 (I/O error)
                                       <[email-masked]>
f3ID37T00078*    1379 Wed Apr 18 15:03 <[email-masked]>
                 (Deferred: Connection reset by mail.snafu.de.)
                                       <[email-masked]>
f3IBQ5v00150   220116 Wed Apr 18 13:26 <[email-masked]>
      8BITMIME   (I/O error)
                                      
<[email-masked]>
f3I8B1J07959     1067 Wed Apr 18 10:11 <[email-masked]>
                 (<[email-masked]>... reply: read error from
velma.s)
                                       <[email-masked]>
f3I710800841   645910 Wed Apr 18 09:01 <[email-masked]>
                 (I/O error)
                                       <[email-masked]>
f3HFXeP15290     1067 Tue Apr 17 17:33 <[email-masked]>
                 (Deferred: Connection reset by velma.seeglasgow.com.)
                                       <[email-masked]>
f3HD6Zu11150     1305 Tue Apr 17 15:06 <[email-masked]>
      8BITMIME   (<[email-masked]>... reply: read error from
mail.b)
                                       <[email-masked]>
f3HCnZS29113*    1528 Tue Apr 17 14:49 <[email-masked]>
                 (<[email-masked]>... reply: read error from
mail.bigfoot.)
                                       <[email-masked]>
f3HBlYb06640      721 Tue Apr 17 13:47 <[email-masked]>
                 (Deferred: 451 Timeout)
                                       <[email-masked]>
f3H9ssR29880     1410 Tue Apr 17 11:54 <[email-masked]>
                 (<[email-masked]>... reply: read error from
mx2.ma)
                                       <[email-masked]>
f3H8tNt02532   218604 Tue Apr 17 10:55 <[email-masked]>
      8BITMIME   (I/O error)
                                      
<[email-masked]>
f3H7bL902824      720 Tue Apr 17 09:37 <[email-masked]>
                 (Deferred: Connection timed out with
mx2.mail.yahoo.com.)
                                       <[email-masked]>
f3H6roa11291   642351 Tue Apr 17 08:53 <[email-masked]>
                 (I/O error)
                                       <[email-masked]>
f3H6toA12584   642555 Tue Apr 17 08:55 <[email-masked]>
                 (I/O error)
                                       <[email-masked]>
f3G8Ur316379      977 Mon Apr 16 10:30 <[email-masked]>
      8BITMIME   (Deferred: Connection timed out with
mx2.mail.yahoo.com.)
                                       <[email-masked]>
f3FJo4x17545*  900736 Sun Apr 15 21:50 <>
                 (I/O error)
                                       <[email-masked]>

-- 
Radio HUNDERT,6 Medien GmbH Berlin
- EDV -
[email-masked]

---

From: Ted Keller <[email-masked]>
Date: Fri, 20 Apr 2001 09:47:44 -0400 (EDT)

Note - this is not a smap/smapd issue.  Sendmail is the MTA that attempts
do to the delivery.  Focus your research in the sendmail area.

You may want to start by doing a survey on the remote system - determing
what their standard mailer is (telnet remotehost 25 usually does the
trick).

Next - determine if you ever deliver mail to them - Is this a hard
problem - or one of those hardtosovle it sometimes works type of problem.

Pick one - contact them and see if they can shed any light (oh! my disk
filled up).

ted keller

---

From: [email-masked]
Date: Fri, 20 Apr 2001 16:08:11 -0000 (GMT)

> do to the delivery.  Focus your research in the sendmail area.

Already did, I just wanted to reassure. 
 
> what their standard mailer is (telnet remotehost 25 usually does the

I know, I know, and I already did. Unfortunately, the MTA were largely
different. I remember qmail and exim, but there were also others which
I hadn't heard of before (some NT stuff, I assume). So it can't really
be an incompatibility issue with one certain MTA...

> problem - or one of those hardtosovle it sometimes works type of

That's exactly it. If only it were I couldn't ever deliver mail to them!
Oddly enough, I thought it might some kind of encoding issue, since a
few mails from Windows users bounced... I then sent a message from my
XFmail to one of the attempted adresses and had it delivered
flawlessly. Then again, I tried to forward a message to the same adress
and I got an I/O error. 

It's all mixed up - plain text, HTML (ouch),
7bit clean, 8bit MIME, I can't really make out any common remarks of
the bounced messages. 
 
> Pick one - contact them and see if they can shed any light (oh! my

I already did - Deutscher Bundestag, the german House of Parliament,
that is. The guy I talked to told me I was the only one reporting
problems and that it might be a firewall issue, but the firewall
adminstrator was not present at the time. Argh. 

I might move on in the queue, just like sendmail would :-%

Thanks, however. 

Jan

---

From: Ted Keller <[email-masked]>
Date: Fri, 20 Apr 2001 11:15:27 -0400 (EDT)

Well I have another theory - just a theory here.....

I went and looked in my sendmail queues. I also have a few messages that
list I/O errors for their non delivery stats.  But - I noted one from my
firewall to my internal mailhub with an I/O error.  In looking at the
internal mail hub - I didn't see any logs for this message - the firewall
listed the status as an I/O error.  Hmmmm.  These systems are on the same
local net - they sit side by side.  The wire is clean.

So here's the theory.

I run the Trend Micro Virus scanner on the internal system.  It accepts
the inbound connections - not sendmail. My suscpisions are that if it gets
busy - it may partially accept the connection - then drop it - leaving the
sending mta in an errored conndition. The sendind MTA then queues it for
later transmission.

I need to look further here to see if this holds any water at all...

Any other thoughts?

tek

On Fri, 20 Apr 2001 [email-masked] wrote:

> Still, I wasn't 100% sure. As I do have split DNS with two MXers, smap
> accepts the mail from the internal mailhost first, so I was concerned
> that smap might have done something nasty (tm) to the messages ;o))
>
> It'S pretty clear this was probably not the case.
>
> Cheers, Jan
> --

---

From: [email-masked]
Date: Fri, 20 Apr 2001 16:21:28 -0000 (GMT)

Cheers, Jan

---

From: [email-masked]
Date: Fri, 20 Apr 2001 16:21:28 -0000 (GMT)

Cheers, Jan

---

From: Rick Murphy <[email-masked]>
Date: Fri, 20 Apr 2001 14:38:36 -0400

>f3JEluw31730   104950 Thu Apr 19 16:47 <[email-masked]>

As someone has already pointed out, these are sendmail problems, not FWTK.
But.. it's possibly related to your firewall. Do you have anything 
listening on the IDENT port? Some hosts will perform an unceremonious 
disconnect when they either can't resolve your IP to a hostname or if they 
can't connect to your IDENT port. Sometimes that error won't occur until 
after you say HELO, so you really have to go through the entire SMTP 
exchange to tell.

Running the sendmail queue in verbose mode (sendmail -v -q) can sometimes 
provide hints, as well.
         -Rick

---

From: Brion Leary <[email-masked]>
Date: Fri, 20 Apr 2001 18:39:05 -0400

I think you're getting closer.  I made mods to smap to
limit the number of children,  After accept, but before
fork, it checks number of running children, if too
many it sleeps until one dies.  Looking through my logs
I believe that if the wait for a free child slot is too
long the connection with the sender is dropped - times
out.  My guess is the sending MTA would report this as
an I/O error.

- Brion

---

From: Scott Campbell <[email-masked]>
Date: Fri, 20 Apr 2001 11:48:21 -0700 (PDT)

> It'S pretty clear this was probably not the case.

I was poking around the sendmail.org faq and found they were mentioning to
check your root.cache/named.boot to make sure it is up to date.  If one of
the entries isn't then it could lead to intermitant time-out problems.
Don't know if that would lead to this particular error but it was in the
faq as related -
http://www.sendmail.org/faq/section3.html#3.10
for some other possible leads.

My firewall probably isn't as busy as yours since I haven't seen these
yet.

Scott E. Campbell
_______________________________
Computer Operations
Greater Victoria Public Library
Victoria BC CANADA

[email-masked]

---

From: [email-masked]
Date: Fri, 20 Apr 2001 16:21:28 -0000 (GMT)

Cheers, Jan