I seem to encounter this issue quite often and felt this topic warrants a dedicated blog post. The basic point of this post is to explain that you cannot have more than one default gateway on separate NICS on a multihomed server! Well, technically you actually can, but things won’t work correctly. Now I am not saying that you cannot have multiple Default Gateways on a specific NIC as this is quite possible as Windows will assign metrics so one Default Gateway is given priority over another which provides redundancy. What I am saying is that you cannot have a Default Gateway on one NIC and then assign a Default Gateway on another NIC.
Any time I have seen Multihomed Servers (OCS Edge, Exchange Edge, ISA, Etc.) malfunctioning, the first thing I’ll do is a ROUTE PRINT. Quite often, I’ll see several lines that display:
That instantly tells me that multiple Default Gateways are assigned. You should only be seeing one line with 0.0.0.0. The entire point of a Default Gateway is it’s the last resort on where to send a packet. Now with that in mind, does it make any sense to have multiple last resorts? No!
So please, put the Default Gateway on only one NIC. For OCS, I typically put it on the Access Edge NIC. For Exchange Edge/ISA, I put it on the Internet Facing NIC. Ok, so you may be thinking, well my external router doesn’t allow RDP traffic… How am I going to manage my box from the inside since the RDP packets will be blocked at the external firewall? What I always do on an Edge Server (and you should also be doing this on any multi-homes DMZ/Edge Server including ISA), is create static routes so any internal traffic will go to your internal network from your internal NIC. It’s essentially creating a fake Default Gateway for only specific subnets (your internal subnets) set on your Internal NIC.
So let’s say you’re setting up an OCS Edge Server and it has 4 NICs:
Access Edge – 10.10.10.100 (DMZ Subnet) – Default Gateway Assigned here
Web Conferencing Edge – 10.10.10.101/24 (DMZ Subnet)
Audio / Video Edge – 10.10.10.102/24 (DMZ Subnet)
Internal NIC – 192.168.200.100/24 (Internal Network)
So how can we get all internal traffic to go out directly through the Internal NIC even though the Default Gateway is assigned to the Access Edge? As stated before, we’ll create a static route. So let’s say your internal router is 192.168.200.1, we’ll create a static route using the following syntax
route add 192.168.200.0 mask 255.255.255.0 192.168.200.1 -p
So for anything destined to the 192.168.200.x network (due to mask being 255.255.255.0 it will route to the default gateway of 192.168.200.1. And Windows is smart enough to see that 192.168.200.1 is on the same subnet as your 192.168.200.100 NIC and assign that as the interface it should send it out of. Problem solved!
Now what if you have a bunch of internal subnets that have similar address ranges? Simple! Supernet your internal networks!
route add 192.168.0.0 mask 255.255.0.0 192.168.200.1 -p
This supernet basically says anything that’s 192.168.x.x (only uses 1st 2 octets since you’re using a mask of 255.255.0.0 otherwise known as /16), send it to the 192.168.200.1 gateway. And again, Windows is smart enough to see that 192.168.200.1 is on the same subnet as your 192.168.200.100 NIC and assign that as the interface it should send out of. So if you have a 192.168.200.x, a 192.168.199.x, or a 192.168.198.x network, all those packets will route to the 192.168.200.1 router which will then send the packet to the appropriate subnet. Problem solved!
And the -p stands for persistent. It means that the static route will survive a reboot.
All the above applies to ISA as well. Let’s say you’re doing LDAPS authentication which uses port 636. Your external router may not allow 636. So by creating the static route to your internal network, the LDAPS traffic won’t be going through your external router and be blocked. It instead will go through your internal router which would most likely be allowing it as Internal Routers are more relaxed in their restrictions.
One thing to take into consideration is that if you are in an environment where the Default Gateways are assigned to all NICs and you modify your server to be properly configured with a Default Gateway on one NIC, make sure that any services such as remote backup on your server are allowed to access over the internet over the ports required for these services or things such as remote backup will start failing.