IF THIS PAGE IS HELPFUL TO YOU, PLEASE JUST WRITE A SHORT COMMENT AND LET ME KNOW! (WHY OR WHY NOT)  I'd be HAPPY to help out with your problems!

What: Exchange 2003 (or any backend) with Microsoft IIS SMTP box as front relay/antivirus/antispam gateway

How: All the mail that comes in to your IIS SMTP 6.0 (Windows 2003 with SP1) box, destined for your internal mail server just sits there.  Outbound mail appears to be fine, most of the time, but it fails sometimes too, giving a DNS error in your event log.  Connection failures happen all over the map. Mail doesn't get delivered until you reboot your box, or perform IISReset on the SMTP relay.

Error:

Event Type: Warning
Event Source: smtpsvc
Event Category: None
Event ID: 4006
Description:
Message delivery to the host 'x.x.x.x' failed while delivering to the remote domain 'domain' for the following reason: The semaphore timeout period has expired.

My input

I've been dealing with Exchange and IIS SMTP boxes for several years now, and these are the things I've done at my sites to keep my mail flowing.  I've blogged about it plenty before, with my 'annoyed' comments on how the server hangs =).

We're talking GFI's MailEssentials v11, MailSecurity v8.1 (don't use me if you want your mail to flow reliably!), and Sybari Antigen for Microsoft SMTP 7.5 SR3 or greater on Windows 2003 with or without SP1. (Sidenote: Windows 2003 R2 should be exciting, but more on this some other time!)

This is what I've done:

1. Uninstall Symantec 9.0 Corporate/Enterprise or greater on your IIS SMTP box.  The transparent SMTP filter/scanner activates itself randomly and starts interfering.  Antivirus is not necessary here, but if you MUST, use CA's eTrust and set it to IGNORE your INETPUB\MAILROOT (or whatever your SMTP processing directory is!).  I can't stress this point enough.
2. GFI MailSecurity causes headaches.  Uninstall IIS SMTP altogether once you've removed the event sinks from GFI, et al., and reinstall IIS SMTP.  ONLY install MailEssentials and Sybari Antigen. MailSecurity does lots of weird things, as of version 8.1.
3. Go into your IIS SMTP server properties.  Change the following settings:
   1. General Tab
      1. Do not limit the number of connections
      2. Timeout (minutes) = 2
   2. Messages Tab
      1. Do not limit message size (You can change this, but don't go too low!)
      2. Limit Session Size = 102400 KB
      3. Do not limit number of messages per connection
      4. Limit number of recipients to 500
   3. Delivery
      1. Outbound Connections... Timeout, 2 minutes
4. Make sure you also specific an external DNS server in your internal Exchange SMTP box, and on this IIS SMTP relay.  Internal DNS servers are not good enough, especially Microsoft ones.  Give it DNS servers (notice the plural!) from your ISP.  Route mail, where possible using [192.168.1.1] (for example) as each domain's 'remote server'.  Keep the square brackets in there to keep DNS problems out.
5. If you continue to experience external mail delivery issues, contact your ISP for a smarthost.  Just fire everything to the smarthost and let it route, since your IP blocks can be blacklisted.  smtp.isp.net is less likely to be blocked, and it becomes some else's problem.  My Exchange box has alternate routes out, so if the smarthost fails, it kicks over to the local delivery route (direct delivery) via my IIS SMTP box.  Experiment, and see what works for you.
6. ISA's SMTP Application filter is your friend.  As I've blogged before, it picks up invalid HTML tags and stuff being sent to the raw SMTP port by certain scanning utilities/libraries.  Its good to keep as a robust filter on the connections your relay server accepts.
7. Cisco PIX boxes with the mail protection feature are evil.  Turn it off.
8. The only Exchange IS level antivirus that I've used, and rely on on production systems to process over 10000 emails a day is F-Secure's product for Microsoft Exchange.  v6.40 appears to be the latest that I can confirm that works well.  GFI's products in VS mode slow the server down horribly, the last I tried.  Keep in mind that Microsoft's own Exchange servers have NO IS level antivirus because they've deemed them all to be "unreliable" and "unstable".  So keep your gateways doing the work =).
9. If you have an IDS, keep an eye on it.  It might be interfering.
10. Clear out your badmail folders monthly.  Keep a monitor on HDD space on the relay box to prevent directories from ballooning and taking down your mail.

As far as antivirus/antispam products go, here are my recommendations:

1. Backend Exchange 2003 = F-Secure for Exchange, or no protection at all.  Microsoft doesn't use it, so you don't have to either =)
2. IIS 6.0 SMTP relay = GFI MailEssentials for spam, and Sybari Antigen for Microsoft SMTP 7.5 SR3 or greater.

I have yet to use/test/see a 3rd party SMTP gateway that processes mails reliably.  They all appear to have buffer overflow exploits, or other issues, or have incredibly annoying activation requirements.  Keep it simple.  I like to install "corporate editions" that just work.  Activation and other hassles on my mail software is ridiculous.