Tài liệu Module 5: Clustering pptx

Resource Dependencies In an Exchange 2000 cluster, we need to create a new Cluster Group to house the Exchange Virtual Server.. Cluster Service Account Permissions Related articles/bugs:

Upgrading an Exchange Virtual Server to

Module 5: Clustering

Information in this document, including URL and other Internet Web site references, is subject to change without notice Unless otherwise noted, the example companies, organizations, products, domain names, e-mail addresses, logos, people, places, and events depicted herein are fictitious, and no association with any real company, organization, product, domain name, e-mail address, logo, person, place or event is intended or should be inferred Complying with all applicable copyright laws is the responsibility of the user Without limiting the rights under copyright, no part of this document may be reproduced, stored in or introduced into a retrieval system, or transmitted in any form or by any means (electronic, mechanical, photocopying, recording, or otherwise), or for any purpose, without the express written permission of Microsoft Corporation Microsoft may have patents, patent applications, trademarks, copyrights, or other intellectual property rights covering subject matter in this document Except as expressly provided in any written license agreement from Microsoft, the furnishing of this document does not give you any license to these patents, trademarks, copyrights, or other intellectual property

 2003 Microsoft Corporation All rights reserved

Microsoft, MS-DOS, Windows, Windows NT, Active Directory, ActiveX, Excel, Exchange Server 5.5, Exchange 2000 Server, Exchange Server 2003, Internet Explorer, Internet Information Server, Wordare either registered trademarks or trademarks of Microsoft Corporation in the United States and/or other countries

The names of actual companies and products mentioned herein (Groupwise, Lotus cc:Mail, Lotus Notes) may be the trademarks of their respective owners

Resource Dependencies

In an Exchange 2000 cluster, we need to create a new Cluster Group to house the Exchange Virtual Server In order to successfully create a System Attendant Resource, we must first have a physical disk resource, an IP address, and a Network Name in that group

When we create the System Attendant resource, the other Exchange resources will be automatically created During the creation process, a dependency tree will be created The dependency tree is shown below

The Information Store resource has five dependencies: SMTP, HTTP, POP, IMAP and Microsoft Search service The message transfer agent (MTA) and Routing Engine resources are directly dependant on the System Attendant In the event of a failover, all resources that have a dependency must go offline before the resource that it is dependant on them can attempt to go offline

In the scenario above the SMTP, HTTP, IMAP4, POP3 and Microsoft Search service must successfully go offline (or fail) before the Information Store resource can attempt to go offline The MTA and Routing Engine resources can attempt to go offline immediately, as they do not have any resources that are dependant on them

Traditionally in Exchange 2000 clusters, the SMTP and the Information Store resources took the longest amount of time to go offline/come online This could

be attributed to large SMTP queues or mounting/dismounting large databases This obviously will lead to longer failover times as the Information Store resource has to wait for the SMTP resource to go offline before it can attempt to

go offline/come online

Exchange 2000

Resource Dependency

Tree

In Exchange Server 2003, the resource-dependant tree has been altered so that all Exchange 2003 cluster resources are now directly dependant on the System Attendant resource

Here we see that all the Exchange related resources are now directly dependant

on the System Attendant This effectively means that the SMTP (and other protocol resources) can now be brought online/go offline in parallel with the store This makes for faster failovers of the Exchange Virtual Server

During the creation of the Exchange Virtual Server process, the correct dependencies will be set

The POP3 and IMAP4 resources are not created by default If they are created manually, then you will need to set a dependency on the System Attendant (this is mandatory)

During an upgrade of an Exchange 2000 Exchange Virtual Server, the resource dependencies will be changed to the new Exchange 2003 resource dependency

tree From the “Exchange Server Setup Progress.log” file we can see these

changes being made Open the log file and search for

ScUpgradeResourceDependencies Here we will see each resource being

[08:36:54] Entering ScUpgradeResourceDependencies [08:36:54] Checking dependencies of resource 'SMTP Virtual Server Instance - (EVS-01)'

[08:36:54] Entering ScChangeResourceDependency [08:36:54] About to change resource dependency for resource 'SMTP Virtual Server Instance - (EVS-01)'

[08:36:54] Leaving ScChangeResourceDependency

You will see the above entries for all Exchange resources that are upgraded to Exchange 2003

Cluster Service Account Permissions

Related articles/bugs:

„ 329702.KB.EN-US

In order to successfully create, delete or modify an Exchange 2000 Exchange Virtual Server, the Windows 2000 cluster service account required “Exchange Full Administrator” permissions at the organization level if it was the first Exchange Virtual Server in the org If it was not the first Exchange Virtual Server in the org then it required Exchange Full Administrator on the Admin

Group that it was being installed into

The Exchange Virtual Server creation process (shown above) can be broken down as follows:

1 User DOMAIN\Administrator logs in to one of the Nodes and starts Cluster Administrator (cluadmin.exe) The process cluadmin.exe runs as the currently logged in user (DOMAIN\Administrator) The Administrator then attempts to create a new Exchange System Attendant Excluadmin.dll will gather information from Active Directory in order to create the System Attendant (e.g Org name and Administrative Group name etc) The user DOMAIN\Administrator needs permissions to read from the configuration partition of the Active Directory

2 When excluadmin.dll has collected the necessary information, it will then pass the information to exres.dll Exres.dll is the Exchange resource dll Exres.dll runs in the Resource Monitor process, which runs in the context of the Cluster Service Account

3 Exres.dll will then load exsetdata.dll in order to create the objects in Active Directory Exsetdata.dll also runs in the Resource Monitor process

4 Exsetdata.dll will then create the necessary objects in the Active Directory

As Exsetdata.dll runs in the context of the Cluster Service Account, this account will require Full Exchange Administrator permissions in order to

create the objects successfully

Permission

requirements in

Exchange 2000

In Exchange 2003 the permissions have changed in order to remove this requirement Any person or application that runs as the Windows 2000 cluster service account essentially has the ability to destroy an Exchange 2000 organization

The Exchange 2003 permissions requirements are as follows:

In the Exchange 2003 the Exchange Virtual Server creation process can be broken down as follows:

1 The user DOMAIN\Administrator logs in to a Node in the cluster and starts Cluster Administrator (cluadmin.exe) This process runs in the context of DOMAIN\Administrator The Administrator then attempts to create a new Exchange System Attendant resource Excluadmin.dll will gather

information from Active Directory in order to create the System Attendant (e.g Org name and Administrative Group name etc) The user

DOMAIN\Administrator will need to permissions to read from Active Directory for this operation to be successful

2 When excluadmin.dll has collected the necessary information, it will then load Exsetdata.dll directly Exsetdata.dll runs in the same process as Excluadmin.dll (DOMAIN\Administrator)

3 Exsetdata.dll will then create the objects in Active Directory As exsetdata.dll runs in the context of DOMAIN\Administrator, it is this account that requires the Exchange Full Administrator permissions to the configuration partition of Active Directory

Permissions

requirements in

Exchange 2003

After an Exchange 2000 Exchange Virtual Server has been successfully upgraded to Exchange 2003 the cluster service account for that cluster can

be removed from the organization and/or Administrative Group objects’ permissions using the delegate control wizard Remember that if that account is used by other Exchange 2000 clusters, then you will have to leave the permissions in place until they have been upgraded to Exchange

2003

Windows 2000 Cluster Service Account:

„ Local Administrator on each Node in the cluster

„ Exchange Full Administrator on org object if other Exchange 2000 clusters remain in org

Windows 2003 Cluster Service Account

„ Local Administrator on each Node

„ No permissions required on org

Permissions required

quick check:

MsExchange_NodeState

MsExchange_NodeState is a registry value that is set during the installation of the Exchange 2003 binary files on a cluster Node It can be seen in the following screen shot:

In this screen shot we can see that Nodes 1 and 2 have the value set and Nodes

3 and 4 do not

In Exchange 2000 we did not check if a node in the cluster had the Exchange binary files installed This would cause problems when calculating if

Active/Active or Active/Passive was allowed in the cluster

For example, If we have a three-Node cluster but we have only installed Exchange 2000 on two of the Nodes, we should be allowed to have an

Active/Active or Active/Passive cluster, as only two of the Nodes can actually host an Exchange Virtual Server Unfortunately exres.dll (The Exchange cluster resource dll) does not see that one of the Nodes does not have the Exchange

2000 binaries and therefore should not be counted when calculating if Active/Active or Active/Passive is allowed In this scenario if we created two Exchange Virtual Servers and then tried to bring both of them online on one Node, one of the Exchange Virtual Servers would fail to come online

In Exchange 2003 we now write this value to the registry on each Node in order

to tell us that the Node is Exchange enabled (i.e it can be added as a possible owner of an Exchange 2003 resource)

It is also used to determine the maximum number of Exchange Virtual Servers that can be created in the cluster

For example if we have six Nodes in the cluster but only four of them have the MSExchange_NodeState set to 1, then we can create a maximum of three Exchange Virtual Servers

The maximum is three in this example, as we always enforce at least one passive node in cluster with greater than two Nodes

The registry value is written to the following location:

HKLM\Cluster\Nodes\<Node_ID>\Parameters\MSExchange_NodeState REG_DWORD=1

The Node_ID value is set when the node is added as a member of the cluster The first node in a cluster will always have a Node ID of 1

When the value is set to 1, then the node is Exchange 2003-enabled and can be set as a possible owner of Exchange 2003 resources in the cluster

If this value is set to 0 or does not exist, then the Node will not be allowed as a possible owner for Exchange 2003 resources

If you attempt to add a cluster node as a possible owner of an Exchange 2003 resource you will receive the following error:

Another registry entry MSExchange_CurrentBuild is used to track the version

of Exchange 2003 that is installed on the Node For example, if we were to upgrade the binary files on Node 1 to a higher value than Node 2, then the MSExchange_CurrentBuild versions would be different on the two nodes

Note

From the Exchange Server Setup Progress log we can see Setup writing these attributes:

[02:25:13] Entering CAtomClusterServer::ScSetExchangeStateOnCluster [02:25:13] Entering CAtomClusterServer::ScSetNodeProperty [02:25:13] Setting DWORD MSExchange_NodeState=1 on node 'NODE1'

[02:25:13] Setting DWORD MSExchange_CurrentBuild=452526080 on node 'NODE1'

[02:25:13] Leaving CAtomClusterServer::ScSetNodeProperty [02:25:13] Leaving

CAtomClusterServer::ScSetExchangeStateOnCluster [02:25:13] Entering

CAtomClusterServer::ScEnableNodeAsPossibleOwer [02:25:13] Leaving

CAtomClusterServer::ScEnableNodeAsPossibleOwer

This can also be seen from the cluster log:

00000550.00000308::2003/04/20-00:25:13.497 INFO [DM] Setting value of MSExchange_NodeState for key Nodes\1\Parameters to 0x00000001

00000550.00000308::2003/04/20-00:25:13.497 INFO [DM] Setting value of MSExchange_CurrentBuild for key Nodes\1\Parameters to 0x1af90000

DNS registration/Kerberos

Related articles:

„ Article 235529 Windows 2000 SP3 added support for Kerberos authentication against clustered virtual servers Prior to Windows 2000 SP3, all authentication against clustered virtual servers was NTLM or NTLM version 2 (NTLMv2) Prior to Windows

2000 SP3, a clustered virtual server did not have a corresponding Active

Directory computer object When the RequireKerberos property was set to 1

and the Network Name resource was restarted an Active Directory computer object would be created

Support for another property RequireDNS was also added in SP3 This would

mean that the clustered virtual server would have to successfully register its own host record (A-record) in DNS in order to come online

Both of these properties are private properties of a Network Name resource and can be set to either 0 or 1 RequireKerberos and RequireDNS have defaults of 1 and 0, respectively Setting either of these values to 1 is not supported for an Exchange 2000 Virtual Server An Exchange 2000 Network Name resource required that these properties be set to 0

Therefore all authentication against Exchange 2000 Virtual Servers is NTLM or NTLM version 2

In Exchange 2003 we now enforce the use of Kerberos authentication This is done automatically by the setup process for non-clustered servers For clusters, these private properties are set during the creation of the virtual server To view

the properties in Windows 2000 we need to use the command line cluster.exe

tool as follows:

Cluster res “my EVS Network Name” /priv

Windows 2000 SP3

In Windows 2000 this can only be set by using the command line tool cluster.exe In the screenshot above, the cluster.exe command has already been used to change the RequireDNS property to a value to “1”

In Windows 2003 Server these properties are changeable from the GUI of cluster administrator

As mentioned earlier, there is no need to set these manually as the setup process will automatically set requireKerberos to 1 and requireDNS to 0 If, after starting these resources with their defaults, you change requireKerberos to 0 (i.e uncheck it from the UI,) then the Network Name resource will fail to come online Changing the DNS requirement does not affect how resources come online unless you are using static DNS, where all records need to be entered by- hand

From the Exchange Server Setup Progress.log we can see Exchange Virtual Server-creation setting these values:

[07:23:58] Entering ScEnableKerberosAndDNSOnEVS [07:23:58] Entering ScBindToEVS

[07:23:58] Binding to cluster'node02.exchange.org' [07:23:58] Binding to EVS 'EVS01'

[07:23:58] Leaving ScBindToEVS [07:23:58] Searching for network name resource 'EVS01' [07:23:58] Resource 'EVS01 Network Name' owns network name 'EVS01'

[07:23:58] Entering ScEnableKerberosAndDNSOnResource [07:23:58] Entering ScGetKerberosAndDNSFromNetworkNameResource [07:23:58] RequireKerberos=0

[07:23:58] RequireDNS=0 [07:23:58] Leaving ScGetKerberosAndDNSFromNetworkNameResource [07:23:59] Setting properties to enable Kerberos on resource 'EVS01 Network Name'

[07:23:59] Entering ScSetKerberosAndDNSPropertiesOnResource [07:23:59] Setting property 'RequireKerberos'='1'

[07:23:59] Leaving ScSetKerberosAndDNSPropertiesOnResource [07:23:59] Bringing resource 'EVS01 Network Name' online [07:24:39] Network name resource 'EVS01 Network Name' successfully enabled for Kerberos

[07:24:39] Entering BringDependentResourcesOnline [07:24:39] Bringing online resources dependent on network name resource

[07:24:39] Leaving BringDependentResourcesOnline [07:24:39] Leaving ScEnableKerberosAndDNSOnResource

[07:24:39] Leaving ScEnableKerberosAndDNSOnEVS

The requirement for DNS registration to succeed was removed from Exchange

2003 in build 6917 This was due to the fact that the Network Name resource would fail to come online if the DNS service in use did not support dynamic updates

If a dynamic update DNS service is in use, then the RequireDNS can be set to

1 for an Exchange 2003 Network Name resource

If a non dynamic update DNS service is in place then it is essential to make sure that the correct A-records exist for the Exchange Virtual Server Network Names resource If this is missing you may run into transport/routing issues

The requireKerberos setting will now create an Active Directory object for the

Exchange Virtual Server In order for this to succeed, the Cluster Service account will need to have the “Add workstations to the Domain” right It should

be a domain account and will therefore have this right by default

A detailed description of the Network Name resources in Windows 2003 can be

obtained in article 302389

AntiAffinityClassNames

AntiAffinityClassNames is a new feature of Windows 2003 clustering It gives

us the ability to assign a node as a hot spare for a particular cluster group in a cluster of three or more Nodes

AntiAffinityClassNames is a public property of cluster groups in Windows

2003 that contains an arbitrary string of characters

AntiAffinityClassNames is not available in Windows 2000 clusters

If there is a failure of a resource in a Windows 2003 cluster group (and the resource is configured to affect the group), the failover manager will check the value of the AntiAffinityClassNames attribute It will then look for a Node in the cluster that does not host a cluster group with that AntiAffinityClassNames value If there is such a node in the cluster, then it is considered to be the best possible destination for the cluster group

Note

In the image above we can see a four-Node cluster There are three Exchange Virtual Servers and the AntiAffinityClassNames property is set to “Microsoft Exchange Virtual Server”

If EVS01 was to fail, then failover manager would try and find a node in the cluster that did not own a cluster group with this property set to “Microsoft Exchange Virtual Server”

In this case EVS01 would fail over to node CLU-NODE04

This, of course, assumes that node CLU-NODE04 is set as a possible owner of the resources that make up EVS01 (which it should be by default)

Note

To check the value of AntiAffinityClassNames use the cluster.exe from the command prompt

Cluster group “my Group” /prop

The value is blank by default

To set the property on a cluster group use the following syntax:

Cluster group “my Group” /prop AntiAffinityClassNames=”Some_Text_String”

The value of AntiAffinityClassNames is not visible from the GUI It can only

be seen using the cluster.exe command line tool

When one creates an Exchange 2003 virtual server on a Windows 2003 cluster

this attribute will be automatically set to “Microsoft Exchange Virtual

Server” If you are seeing it set to some other string then it has probably been

changed manually and should be changed back to the default setting

Using Cluster Administrator, we can manually move a group to another Node

in the cluster When we have a cluster with more than two Nodes, we will have

a new choice called Best Possible

In the above image we can see that there are three Nodes in the cluster By

selecting Best Possible, then the cluster group containing the Exchange Virtual

Server will be moved to a Node that does not currently host a group with the

AntiAffinityClassNames set to “Microsoft Exchange Virtual Server”

It is important to note that the AntiAffinityClassNames property is set on the cluster group If we were to create a new cluster group and then move all of the Exchange 2003 cluster resources into this group then the attributes that were set

on the original cluster group do not follow the resources

The attributes that are affected in this scenario are AntiAffinityClassNames and MSExchange_VirtualServerNames

It is unsupported to move the Exchange 2003 cluster resources between cluster groups If you are not seeing these properties set on the Exchange 2003 cluster group then the resources may have been moved manually

In this scenario it is necessary to recreate the attributes using the command line cluster.exe utility

Related articles:

Windows 2000 DataCenter Server:

„ 279802 – The default behaviour when you do a move group operation on a cluster

„ 197047 – Failover/Failback policies on Microsoft Cluster Server

Windows 2003 Enterprise and DataCenter:

„ 301114 – Proper usage of the Preferred Owner List on a cluster Server (not finished)

„ 299631 – Failover behaviour on clusters of three or more nodes

„ 296799 – How to configure Windows Clustering Groups for hot spare support

Mount Point Drives

Mount Point Drives have been available since Windows 2000 They were, however, not supported for use in Windows 2000 clusters This essentially limited the drive letter assignments in a Windows 2000 cluster to a maximum

of 26 This number included local drive assignments on each cluster node For example, a cluster node may have local physical disks A: (Floppy Drive), C: (Local Disk) and D: (Local Disk) This would effectively leave 23 drives that could be assigned to the cluster As each node in the cluster may be a possible owner of each of these 23 disk resources, this would limit the entire cluster to 23 disk resources

As the supported number of Nodes in clusters increases, this limitation begins

to seriously hamper the ability to build enterprise class solutions

In Windows 2003 Server, support for mount point drives has been added in Windows clusters

Support for Mount Point Drives does not exist in Windows 2000

A mount point drive is an entire volume that is mounted inside another folder that is hosted by another volume

The steps to create a mount point drive available for cluster use are as follows:

1 A new unformatted disk will be available in Disk Manager Make sure that

it is a Basic Volume Right-click it and choose new partition

2 Choose Primary partition and click Next

3 Set the size for the partition and click Next

4 Instead of assigning a Drive Letter to the disk, we are going to mount it inside a folder on another volume The volume must be a disk that is visible

to the cluster (i.e not a local disk)

The mount point volume will also be hosted in the same cluster group as the hosting volume

5 In this scenario we are going to use Disk R: which is a disk in our cluster

6 I have created a new folder on R:\ called Mount which will host the new volume

7 Give the volume a label and then format it using NTFS It must be formatted with NTFS

8 Click Next to complete the New Partition Wizard

9 We can now see our Mount Point Drive in Disk Manager

10 To avoid confusion, Mount Point drives appear as physical disks rather than folders when viewed in Explorer If you were to remove the Mount Point Drive then the folder R:\Mount would remain on R:\ as a normal folder

11 Now we have to create the cluster resource for the Mount Point Drive

The Mount Point Drive resource must be in the same cluster group as our hosting disk R:\

Using Cluster Administrator, locate the correct Cluster Group and then

create a new resource Give the resource a name and choose Physical Disk

for the resource type Click Next

Note

12 Set the possible owners for the resource In this scenario we only have one Node

13 Set a dependency on R:\ NOTE: It is essential that we do this Cluster

Administrator will not prompt you to do this By setting a dependency on R:\ we ensure that the resource for the Mount Point Drive is taken offline when the R:\ resource is taken offline

14 Now choose the correct Mount Point Volume that we created earlier If you are unsure, then use Disk Manager to locate the correct disk number

15 After clicking “Finish,” the Mount Point Drive resource will now appear in Cluster Administrator

16 The Mount Point Drive properties can then be seen in the registry:

A few rules of thumb regarding Mount Point Drives:

1 The partition must be mounted inside a disk/volume that is available to the cluster, i.e Shared Storage

2 The cluster resource for the Mount Point Drive must exist in the same cluster group as the disk that is hosting it

3 The cluster resource for the Mount Point Drive must have a dependency on the resource for the disk that is hosting it

4 The System Attendant cluster resource must have a dependency on all disk resources including Mount Point Drive resources (assuming they are being used for Exchange purposes)

Creating an Exchange Virtual Server

Once the Exchange 2003 binaries have been installed on the cluster Node we can now create an Exchange Virtual Server It is a good idea to make sure that all the required Nodes in the cluster have the Exchange 2003 binary file installed before proceeding with the Exchange Virtual Server creation

The Exchange Virtual Server creation process is much the same as Exchange

2000

First we need to create a cluster group to house the Exchange Virtual Server Within this group we must have at least one physical disk resource, at least one

IP address resource, and a network name resource

The network name resource must have a dependency on all the IP address resources in the cluster group (assuming that all the IP address resources will be used for Exchange 2003 purposes)

When we have these resources in the group, we can begin the creation of the System Attendant resource Once the System Attendant Resource has been created all the other Exchange 2003 cluster resources will be created automatically

Right-click on the Exchange 2003 cluster group and choose New Resource and then choose Microsoft Exchange System Attendant Give it an identifiable name and click Next

Add the Nodes that will be possible owners of the System Attendant Resource These Nodes will also be added as possible owners of all the other Exchange resources that are automatically created A Node must be specified as a possible owner of a resource in order for us to failover to that Node

If you attempt to add a Node that does not have the MSExchange_NodeState value set to 1 then you will receive the following error:

The specified Node will not be able to be a possible owner of the resource until the Exchange 2003 binary files have been installed on the Node (which in turn will add the MSExchange_NodeState value in the registry)

The MSExchange_NodeState value is used to determine which Nodes can be possible owners of an Exchange Virtual Server

Please refer to the MSExchange_NodeState section for a more detailed description of this

Note

Now we have to set the dependencies on the System Attendant Resource You must set a dependency on the Network Name resource and all disk resources that you plan to use for Exchange 2003 related use This includes all Mount Point Disks which will contain Exchange 2003 data

If more than one Administrative Group exists, choose the Administrative Group that we will install the Exchange Virtual Server into

Only mixed Administrative Groups or pure Exchange 2000/2003 Administrative Groups will be displayed Pure Exchange 5.5 sites will not be displayed, as the first Exchange 2003 Server in a Exchange 5.5 site cannot be a cluster

If this is not the first Exchange Virtual Server in the cluster, then this choice will be automatically set to the Admin Group of the other Exchange Virtual Server(s) All Exchange Virtual Servers in a cluster must be part of the same Admin Group

Note

Within the chosen Administrative Group we now have to choose a Routing Group that the Exchange Virtual Server will be a member of, if more than one Routing Group exists This can be changed afterwards if an incorrect Routing Group is chosen

Now we have to choose a path for the Exchange Virtual Server We must specify a path on a disk that we specified in the dependencies section earlier

We can specify a location on a Mount Point disk

Note