BizTalk Server 2009 : What the Orchestration Engine Provides

1. What an Orchestration Is

BizTalk orchestrations are used to visually model workflows and provide the primary mechanism to implement business process automation within a solution. They are the equivalent of the good-old flowcharts programmers used to detail algorithms in functional specifications before sequence diagrams and object-centric design. Orchestrations are the most powerful tool within the BizTalk Server toolbox, as they allow for the rapid development and deployment of complex processes that in many circumstances can be implemented with little to no coding. They are created within Visual Studio and are compiled into .NET assemblies that are deployed to the Global Assembly Cache and registered in the BizTalk Management Database.

Just like in any subroutine, you can declare and use variables within an orchestration. Orchestrations started or called programmatically or through a caller orchestration may also be passed parameters. In those aspects, an orchestration is no different from a procedure or function. Orchestrations may also receive and send messages, due to the integration of the orchestration engine, known as the XLANG engine, and the BizTalk messaging subservice. The orchestration engine's constructs in the Orchestration toolbox allow developers to construct, modify, and transform messages that an orchestration sends and receives. Developers can add C#-like expressions^[]—XLANG expressions—to an orchestration flow to perform complex logic or call external assemblies. Orchestration developers may declare transaction scopes within an orchestration and define compensation blocks that will be implicitly called to roll back the transaction if an exception occurs.

^[] XLANG expressions constitute the subset of C# expressions that can be used within the orchestration. The XLANG engine performs a fair amount of housecleaning and state management after each expression statement. A simple C#-like statement in an Expression shape results in several other operations performed by the XLANG engine.

2. What the Orchestration Engine Provides

The BizTalk orchestration engine, the XLANG engine, consists of a set of SQL Server stored procedures, jobs that run on the BizTalk Messagebox database—msgbox DB—and Management Database as well as a set of managed assemblies that run within BizTalk host instances. The XLANG engine is the maestro that schedules, manages, and monitors orchestration execution across host instances. It manages orchestration instantiation, execution, termination, and migration across host instances using a predefined amount of system resources, like memory and threads. For the engine to be able to perform its functions, it needs to be able to interrupt the executions of orchestration instances regularly. Instead of resorting to a language that gets interpreted at runtime to run on top of the engine, visual orchestration constructs (shapes) that form the flow of an orchestration are compiled into a set of calls to the XLANG APIs in a managed assembly. This allows the engine to control the scheduling and execution of orchestration instances without the performance hit associated with interpreted languages. In essence, the XLANG engine and orchestration instances are running side by side in the BizTalk host instance.

The integration of the XLANG engine and the rest of the BizTalk Server components, as illustrated in Figure 7-1, helps provide the basic services required for building and running reliable enterprise-grade orchestrations:

• The ability to scope transaction and designate compensation blocks and exception handlers.

• Support for atomic or long-running transactional scopes. Atomic scopes mean that the engine will cease to dehydrate^[] the running orchestration instance until it exits the atomic scope. Long-running scopes mean that the execution time of each step in the scope is undetermined or very long. The engine persists the instance's state on each operation that crosses the instance boundary, like sending messages, and eventually dehydrates the orchestration instance if it is idle and meeting the dehydration thresholds.^[]

  ^[] The dehydration of a running orchestration instance is the act of persisting the current orchestration instance state to the BizTalk Messagebox database and releasing all transient system resources, memory, and threads back to the resource pools. Rehydration of a dehydrated orchestration instance is the act of loading the persisted orchestration instance state into memory and consuming resources from the pools to continue execution of that instance at its last persisted state.

• Fault tolerance. Leveraging state persistence of orchestration instances, the BizTalk orchestration engine can recover from a failure by recycling or restarting a failed host and resuming execution of previously running instances from the last persisted state. If the orchestration is deployed on a BizTalk Server Group running multiple host instances on different servers, the engine may also resume such instances on a different host instance running on a different server. This ensures absolute resilience in the case of a catastrophic failure of a server.

• Scalability and load balancing. The power of orchestration instance state persistence allows the engine to dehydrate an orchestration instance on one host instance and resume its execution on another host instance on a different server if the original host instance it was running on is overloaded. It can also balance the load across the running host instances on multiple servers.

• Activity tracking and monitoring. If designed by the Orchestration Designer, an orchestration can log and track activity execution that later can be monitored and reviewed through the Business Activity Monitor.

• Integration with the Business Rule Engine and firing business policies from within orchestrations.

• Integration with the BizTalk Messaging Engine and sending and receiving messages from within orchestrations.

• Leveraging XSLT maps and performing message transformation from within orchestrations.

• Leveraging the BizTalk Messaging Engine to allow for correlation of sent and received messages to ensure their delivery to the proper orchestration instance for the implementation of complex execution flows that span multiple requests.

In short, the BizTalk orchestration engine allows for the implementation and deployment of resilient transactional business processes that are scalable, fault tolerant, and able to send and receive messages or execute business rules.

Figure 1. Where the orchestration engine fits in BizTalk Server