Tuesday, 26 August 2008

Enabling Network Load Balancing (NLB) Multicast Mode with ISA Server 2006 Enterprise Edition

I have seen a few questions on the ISA forums with regard to enabling support for Network Load Balancing (NLB) Multicast support on ISA Server 2006 Enterprise Edition. Although this seems like a relatively simple feature, there seems to be a bit of confusion about how best to enable the feature, what elements are actually required and where to find all of the associated information for the complete process.

Based upon my understanding and a recent need to enable this feature for a customer, I thought it may be useful to provide an overview of the procedure and try to encapsulate all of the required information in one single blog entry. An high-level overview of the procedure is provided below:

  • Step 1: Disable Network Load Balancing Integration on the array (only necessary if NLB integration is already enabled)
  • Step 2: Run RemoveAllNLBSetting.cmd on each array member, which is available as part of NLBclear.exe tool from here (only necessary if NLB integration is already enabled)
  • Step 3: Ensure KB942639 or Service Pack 1 are installed on all Configuration Storage Servers and ISA array members (note the correct order of installation)
  • Step 4: Run the KB938550 script on the primary Configuration Storage Server and configure the required NLB mode as necessary.
  • Step 5: Enable Network Load Balancing Integration on the array.
In order to enable the multicast feature on an array, all ISA Servers need to be running the hotfix included in KB942639 or have installed Service Pack 1. Until this update, ISA Server was only supported in unicast mode, which is not always an ideal mode for certain network environments. A better understanding of the different NLB modes can be found in the NLB FAQ along with the dependences and implications of enabling multicast mode in routed network environments.

However, installation of the updated ISA binaries is not enough, as you also need to extend the Active Directory Application Mode (ADAM) schema to support the new feature attributes. Luckily this process is fully automated in the scripts provided by Microsoft in KB938550. These same scripts are also used to enable the required NLB mode as required; namely Unicast (the default mode), Multicast or Multicast with IGMP. For convenience, a copy of the necessary scripts can also be found here as from what I can tell, these scripts are not included with Service Pack 1 and have to be obtained separately from Microsoft.

If you have multiple Configuration Storage Servers (CSS) in your ISA Enterprise, before running the script (called KB938550.wsf) it is necessary to determine which CSS is holding the Schema Master FSMO role. In general, this will be the first CSS installed into the ISA Enterprise, but this may not always be the case if the environment has been changed or subject to some form of disaster recovery. In order to determine which server holds this FSMO role, Jim Harrison has written a great script available here which does the job perfectly! Even if the environment is quite standard, it is probably worth using this script anyhow, just to ensure you are on the applying the changes to the Schema Master (see update section at end of the article for more information).

Once you have determined the correct CSS server, you can then follow the instructions included in KB938550 to extend the ADAM schema and enable the required NLB mode. Note that the script will need to be run on each array if you have multiple arrays and multiple ISA Servers. The basic syntax for this script is:

cscript kb938550.wsf /array:<ArrayName> /nlb:<NLB Mode> /net1:<ISA Network 1> /net2:<ISA Network 2> ... /netX:<ISA Network X>

where net1, net2 ... netX represents the ISA Server networks that require reconfiguration.

For a standard two-NIC ISA Server array, the example command line syntax would be:

cscript kb938550.wsf /array:MyArray /nlb:multicast /net1:Internal /net2:External

An example of the correct syntax and the associated output is shown below:

A similar output can be seen below for enabling 'Multicast with IGMP' mode, which also shows the script extending the Configuration Storage schema:

In the event that you need to return our example two-NIC ISA Server back to the default unicast NLB mode, the following syntax would be used:

cscript kb938550.wsf /array:MyArray /nlb:unicast /net1:Internal /net2:External

An important part of the process to consider, especially for distributed deployments, is to wait for the changes to fully replicate between CSS roles throughout the environment before continuing onto subsequent steps in the procedure.

Please Note: It is also interesting to note that the use of NLB multicast mode still does not appear to allow ISA Server to support vendor based NIC teaming, contrary to information provided by most hardware vendors. From my own experience, this appears to be the case with HP NIC teaming software/drivers and likely with other vendors too. I think this is more to do with the ISA Server specific implementation of NLB, as opposed to NLB itself; however, I cannot be totally sure of this.

As discussed within the NLB FAQ, the use of multicast NLB will probably require the use of static ARP entries on routers and other Layer 3 devices throughout the network environment to map the NLB cluster MAC address to each of the NLB virtual IP addresses, and hence provide correct NLB load balancing and failover functionality. This is a final important step in the overall high availability ISA Server configuration and is often missed by many ISA Server firewall admins that are not familiar with NLB.

On Cisco network devices, I believe the correct syntax for adding static ARP entries is:

arp <NLB Virtual IP Address 1> <NLB Cluster MAC Address> arpa
arp <NLB Virtual IP Address 2> <NLB Cluster MAC Address> arpa
arp <NLB Virtual IP Address X> <NLB Cluster MAC Address> arpa

So, I hope this blog entry has made things a little clearer, and if nothing else provides all the necessary information in one place for easy access.

UPDATE 05/01/09

I have been informed that the latest version of the kb938550.wsf script automatically determines which CSS holds the FSMO Master role, in addition to the other schema changes - this is handy, so thanks Microsoft!


  1. Very useful blog. Thanks for sharing this priceless information.

  2. The latest version of the kb938550.wsf script automatically determines which CSS holds the FSMO Master role.

  3. Hi,

    great blog, thanks!

    However I am still not quite certain as to where I should configure the static ARP entries. Are these only needed on the router/default gw that is directly behind/in-front-of the NLB cluster? Or do I also need to configure this on my Blade switches?

    Best regards,
    Enrico Klein

  4. >>n-rico,

    Thanks :)

    I think the entries are only needed on the L3 devices that are closest to the NLB enabled interfaces.

    More info provide here: http://www.internetworkpro.org/

    This seems imply we are only concerned with the "local router" and talks about the NLB "default gwateway device".

    Good luck!



  5. Hi Jason,

    thanks for your reply. I was thinking about the same thing. Thanks for confirming. I'll give it a try tomorrow, I'm going to convert our ISA unicast array to multicast, so wish me luck :)


  6. How do you find out the max address of the Mulitcast VIP? In MS NLB it is easy, it is on the config screen shown in gray.

  7. >> Robert.

    Nothing in the ISA GUI, so same as for Native NLB.

    All VIPs will use the same cluster MAC address.

  8. To find out the NLB MAC address being used on a specific network, run the following command:

    wlbs ip2mac $network_name

  9. Okay so in a configuration where you have

    Internet <=> External Router <=> TMG 2010 <=> Internal Router <=> Internal Network

    Would you have to put the Multicast MAC for the External Network on the External Router and the Multicast MAC for the Internal Network on the Internal Router if they all are set to NLB with Multicast?

  10. Hi Ryan,

    Unless you are using Mutlicast with IGMP, then yes, I believe you will need to define static ARP entries on both routers.

    This assumes that you have NLB VIPS on both internal and external TMG interfaces...