Tuesday, 8 September 2009

Group Policy Processing Errors on ISA Server and Fun with Large ICMP Packets

I recently came across an interesting problem on an ISA Server deployment which I thought might be useful to share.

In this particular deployment, the ISA Server array members were located in a DMZ behind an existing back firewall. Consequently, the back firewall separated ISA Server from internal infrastructure like Active Directory Domain Controllers, Exchange servers etc.

Please Note: Placement of ISA Server in an existing DMZ is a far from ideal deployment scenario, but unfortunately common when using ISA Server as a reverse proxy solution. In an ideal scenario, the ISA Server array members would be placed in parallel to the existing back firewall, or possibly dual-homed directly between the existing DMZ and the internal network. However, that is a discussion for another day ;)

In order to use the Kerberos Constrained Delegation (KCD) feature of ISA Server, it was necessary for these array members to be members of the internal Active Directory domain; the same domain as the published Exchange servers in fact. You can read more here on KCD here and here.

To support the necessary communications for Active Directory domain membership, the back firewall was configured to permit the necessary protocols from the array members to the Active Directory domain controllers. See my previous blog article here for details on the protocols required. 

I tested my web publishing rules and everything was working as expected from an ISA Server perspective. Time to go home? No, not quite :(

However, what did happen was that I started receiving Userenv application event log errors, specifically:

Event ID: 1054
Source: Userenv
Type: Error
Description:
Windows cannot obtain the domain controller name for your computer network. (The specified domain either does not exist or could not be contacted). Group Policy processing aborted.

At first I assumed that the back firewall had been configured incorrectly, but after validating the configuration using Ping and PortQry, I realised that this was not the case.

After a bit of investigation I came across the following Microsoft KB article which describes how to enable debug logging for Userenv:

How to enable user environment debug logging in retail builds of Windows (KB221833)

From looking at the userenv.log file generated using the above KB article, I could see the following errors when using the gpupdate /force command:

PingComputer: Second send failed with 11010

So, this started me thinking about Ping and the use of ICMP between the array members and the Active Directory Domain Controllers. If you refer back to my previous blog article here you will see that PING (ICMP Request -Type 8) is included within the required protocol list. Therefore, ICMP should not be blocked by firewall policy processing on the back firewall.

I confirmed this was not the case using a simple Ping command (which uses a default packet size of 32 bytes) from the array members to one of the Domain Controllers:

Ping <DC IP Address>

Result: Reply from <DC IP Address>: bytes=32 …

Therefore, we had successful ping replies.

More investigation led me to the follow Microsoft KB article:

How to troubleshoot Group Policy object processing failures that occur across multiple forests (KB910206)

Looking at Scenario 2 in this KB article seemed to pinpoint my exact symptoms. In particular, the following text intrigued me:

Group Policy processing stops if large fragmented ICMP packets are not allowed on the network because of slow link detection. When this occurs, Microsoft Windows XP does not detect a fast link or a slow link. Instead Windows XP fails out of Group Policy processing. Only a generic USERENV 1054 event is logged in the Application event log. This problem affects both computer and user policies.

This got me thinking that maybe the back firewall wasn’t blocking ICMP traffic, but was in fact blocking (or dropping) large or fragmented ICMP packets. A quick test using the –l Ping switch with a packet size of 2048 bytes (the default value used by Group Policy processing) confirmed that ICMP was in fact being blocked when using 2048 byte packet sizes:

Ping <DC IP Address> –l 2048

Result: Request timed out.

Lowering the packet size to 1024 made no difference, still blocked:

Ping <DC IP Address> –l 1024

Result: Request timed out.

Finally, lowering the packet size to 500 bytes resulted in successful replies:

Ping <DC IP Address> –l 500

Result: Reply from <DC IP Address>: bytes=500 …

The only real conclusion I could draw from this process of elimination was that the back firewall was configured to block ICMP packets above a certain size or fragmented ICMP packets; most likely packets above 512k. I assumed this was a security feature or some form of DOS protection on the back firewall, and a reasonable measure.

Disabling slow link detection as described in the above KB article (KB910206) didn’t appear to help though and I was still receiving event log errors. I don’t give up easily though!

So, I looked for a solution to reduce the actual ICMP packet sizes used in Group Policy processing. A bit more investigation resulted in the following Microsoft KB article:

Group Policies may not apply because of network ICMP policies (KB816045)

After creating the PingBufferSize registry key and setting this to a value of 500, I used the gpupdate /update command again, and this time I received a successful application event log entry of:

Event ID: 1704
Source: SceCli
Type: Information
Description:
Security policy in the Group policy objects has been applied successfully.

From that point on, the Userenv errors stopped occurring and Group Policy was applied successfully…case closed I think ;)

The only thing left to do was to apply the registry change to all array members to ensure consistent Group Policy process across the entire array.

As normal, the problem was not actually caused by ISA Server, but by the surrounding network infrastructure. It might have been nice if the back firewall administrator had told me about this ICMP limitation from day one, but they likely didn't even know :)

Hope this helps someone else with a similar problem…

No comments:

Post a Comment