Ever since Exchange Server 5.5, Microsoft
has offered the option to use Windows Clustering to create a highly
available Exchange Mailbox environment. In a typical shared-storage
cluster environment there are two server nodes available, both running
Exchange Server, and both servers are connected to a shared storage
solution. In the early days, this shared storage was built on a shared
SCSI bus and later on, SANs with a Fiber Channel or iSCSI network
connection were used. The important part was the shared storage where
the Exchange Server databases were located.
At any given point in time only one server node is the "owner" of this shared data, and it is this server node that is providing the client services; this server node is also known as the active node. The other node was not able to access this data, and was therefore the passive node.
A private network between the two server nodes is used for
intra-cluster communications, such as a heartbeat signal, allowing both
nodes to determine the state of the cluster, and if other nodes are
still alive.
In addition to the two nodes, an "Exchange Virtual Server"
was created as a cluster resource (note that this has nothing to do
with virtual machines!). This is the resource that (Outlook) clients
connect to in order to get access to their mailboxes. When the active
node fails, the passive node takes over the Exchange Virtual Server,
which then continues to run. Although users will notice a short downtime
during the fail-over, it is an otherwise seamless experience, and no
action is needed from an end-user perspective.
Although this solution offers redundancy, there's
still a single point of failure: the shared database of the Exchange
server. In a typical environment this database is stored on a SAN, and
by its nature a SAN is a highly available environment. But when
something does happen to the database, a logical failure for example, the database is unavailable for both nodes, resulting in total unavailability.
1. Exchange database replication
Microsoft offered a new solution in Exchange Server
2007 to create highly available Exchange environments: database
replication. When using database replication, a copy of a database was
created, resulting in database redundancy. This technology was available
in three flavors:
Local Continuous Replication (LCR) – a copy of the database is created on the same server.
Cluster Continuous Replication (CCR) – a copy of the database is created on another node in a Windows failover cluster (there can only be two nodes in a CCR cluster).
Stand-by Continuous Replication (SCR)
– this came with Exchange Server 2007 SP1. A copy of a database is
created on any other Exchange server (i.e. not necessarily in the
cluster). This is not meant as a high availability solution, but more as
a disaster recovery solution.
This is how database replication works in a CCR clustered environment:
Exchange Server 2007 is installed on a Windows Server
2003 or Windows Server 2008 Fail-over cluster. There's no shared
storage in use within the cluster, but each node has its own storage.
This can be either on a SAN (fibre channel or iSCSI) or Direct Attached
Storage (DAS) – i.e. local physical disks.
As mentioned earlier, the active node in the cluster
is servicing client requests, and Exchange Server uses the standard
database technology with a database, log files, and a checkpoint file.
When Exchange Server is finished with a log file, the log file is sent
immediately to the passive node of the cluster. This can either be via a
normal network connection or via a dedicated replication network.
The passive node receives the log file and checks it
for errors. If none are found, the data in the log file is relayed into
the passive copy of the database. This is an asynchronous process,
meaning the passive copy is always a couple of log files behind the
active copy, and so information is "missing" in the passive copy.
In this environment, all
messages are sent via a Hub Transport Server, even internal messages.
The Hub Transport Server keeps track of these messages in a CCR
environment, and can therefore send missing information (which the
passive node actually requests) to the passive copy of the cluster in
case of a cluster fail-over. This is called the "Transport Dumpster" in a Hub Transport Server.
This kind of replication works very well; a lot of
System Administrators are using CCR replication and are very satisfied
with it. There are a couple of drawbacks, though:
An Exchange Server 2007 CCR environment is
running on Windows Server 2003 or Windows Server 2008 clustering. For
many Exchange administrators this brings a lot of additional complexity
to the environment.
Windows Server 2003
clustering in a multi-subnet environment is nearly impossible, although
this has improved (but is still not perfect) in Windows Server 2008
failover-clustering.
Site Resilience is not seamless.
CCR clustering is only possible in a two node environment.
All three kinds of replication (LCR, CCR and SCR) are managed differently.
To overcome these issues, Microsoft has dramatically
improved the replication technology, and reduced the administrative
overhead at the same time. This is achieved by completely hiding the
cluster components behind the implementation of Exchange Server 2010.
The cluster components are still there, but the administration is
completely done with the Exchange Management Console or the Exchange
Management Shell.
2. Database Availability Group and Continuous Replication
In Exchange Server 2010, Microsoft introduces the
concept of a Database Availability Group or DAG, which is a logical unit
of Exchange Server 2010 Mailbox Servers. All Mailbox Servers within a
DAG can replicate databases to each other, and a single DAG can hold up
to 16 Mailbox Servers and up to 16 copies of a database. The idea of
multiple copies of a database in one Exchange organization is called Exchange Mobility; one database exists on multiple servers, each instance of which is 100% identical and thus has the same GUID.
With a DAG in place, clients connect to an Active
Database, which is the database where all data is initially stored.
Also, new SMTP messages that arrive, either from outside or inside the
organization, are stored in this database first. When the Exchange
Server has finished processing information in the database's log file,
the file is replicated to other servers (you can assign which servers
should have a copy of the database). The log file is inspected upon
receipt and, if everything is all right, the information contained in
the log file is dropped into the local copy of the database.
In Exchange Server 2010, all clients connect to the
Client Access Server, including all MAPI clients like Microsoft Outlook.
Supported Outlook clients in Exchange Server 2010 include Outlook 2003,
Outlook 2007 and Outlook 2010. So, the Outlook client connects to the
Client Access Server which, in turn, connects to the mailbox in the
Active Copy of the database, as you can see in Figure 6.
Unfortunately, this is only true for Mailbox Databases. When an Outlook
client needs to access a Public Folder Database, the client still
accesses the Mailbox Server directly.
When the active copy of a database or its server
fails, one of the passive copies of the database becomes active. The
order of fail-over is configurable during the configuration of database
copies. The Client Access Server automatically notices the fail-over,
and starts using the new Active Database. Since the Outlook client is
connected to the Client Access Server and not directly to the database, a
database fail-over is fully transparent. Messages like "The connection
to the server was lost" and "The connection to the server is restored"
simply do not appear any more.
When building a highly available Mailbox Server
environment in a DAG, there's no need to build a fail-over cluster in
advance, as additional Mailbox Servers can be added to the DAG on the
fly. However, for the DAG to function properly, some fail-over
clustering components are still used, but these are installed during
DAG's configuration. All Management of the DAG and the database copies
is performed via the Exchange Management Console or the Exchange
Management Shell; the Windows Cluster Manager is no longer used.
NOTE
The Database
Availability Group with Database Copies is the only high availability
technology used in Exchange Server 2010. Older technologies like SCR,
CCR and SCR are no longer available. The traditional Single Copy Cluster
(SCC) with shared storage is also no longer supported.
Configuration of a Database Availability Group is no
longer limited to a server holding just the Mailbox Server Role. It is
possible to create a two-server situation with the Hub Transport, Client
Access and Mailbox Server role on both servers, and then create a
Database Availability Group and configure Database Copies. However, it
isn't a High Availability configuration for the Client Access or Hub
Transport servers unless you've put load balancers in front of them,
since it's not possible to use the default Windows Network Load
Balancing (NLB) in combination with the fail-over clustering components.
Regardless, this is a great improvement for smaller deployments of
Exchange Server 2010 where high availability is still required.
2.1 Active Manager
In Exchange Server 2007, Cluster Continuous
Replication uses the cluster resource management model to install and
manage the High Availability solution. Initially, the Windows cluster is
built and then Exchange setup is run in clustered mode, registering the
EXRES.DLL in the failover-cluster, and the Clustered Mailbox Server
(CMS) was created. For a High Available Exchange Server 2007 environment
it is always necessary to build a fail-over cluster in advance, even if it's just a one-node cluster!
The cluster components are now hidden in Exchange Server 2010, and a new component named the Active Manager
has been introduced. The Active Manager replaces the resource model and
fail-over management features offered in previous versions of Exchange
Server.
The fail-over clustering components have not been
completely removed, though, and some of them are actually still used. If
you open the Fail-over Cluster Manager in Administrative Tools, you'll
find the DAG, cluster networks, etc. Do not
try to manage the DAG using the Failover Cluster Manager, as this is
not supported. The only way to manage the DAG is using the Exchange
Management Console or the Exchange Management Shell!
The Active Manager runs on all Mailbox Servers that
are members of a DAG, and there are two roles; the Primary Active
Manager (PAM) and the Standby Active Manager (SAM). The PAM is running
on the Mailbox Server that also holds the cluster quorum, and this is
the server that decides which databases are active and which databases
are passive in a DAG. The SAM is responsible for determining server or
database failures (the PAM does this on its own server for its own local
databases) and, if detected, communicates with the PAM to initiate a
failover.
The replication service monitors the health
of the mounted databases in a DAG, and monitors the ESE engine for any
I/O issues or failures. If anything goes wrong here, the replication
service immediately contacts the Active Manager. In the case of a
failover, the Active Manager determines which database should become the
Active Copy of the database (depending on the fail-over order you've
specified during configuration).
