Some BizTalk artifacts lend themselves
nicely to particular patterns of solutions, while their use could be
devastating to the performance of a different solution. The following
sections describe some of these solution patterns and the pitfalls that
you as a BizTalk developer need to watch out for.
1. Batch Processing and Legacy Systems
Legacy mainframe systems excel in what they were
designed to do best, batch processing. With the emergence of online
applications and progressive automation of internal processes, the load
onto these legacy systems is exponentially increasing. Instead of
handling a handful of requests concurrently, they are now expected to
handle hundreds and thousands of concurrent requests as business goes
online and customers perform actions resulting in transactions executing
on these back-end legacy systems. EAI platforms such as BizTalk
facilitate the integration of newly developed interactive systems with
legacy back ends such as those batch-processing mainframes. Third-party
or custom adapters could be used to communicate with the mainframes, and
transformation maps could be used to translate data back and forth.
BizTalk-based solutions developed to integrate with the mainframe will
leverage the Messagebox as a queue to throttle and batch transaction
requests to the mainframes to ensure their ability to handle the
incoming load of requests.
Such a solution works perfectly fine, but should be
stress and load tested to ensure that the queuing and throttling does
not affect the overall performance of the solution. With BizTalk 2006
and 2009 auto-throttling, queuing is not a big problem, but with
previous versions of BizTalk, queuing leads to the growth of the spool
table in the Messagebox, which affects the overall performance of the
Messagebox and consequently the BizTalk server.
2. Interactive Orchestrations (the Request/Response Approach)
The response time in interactive orchestrations such
as workflows exposed as web services could be critical to the
correctness of the complete solutions. For example, a call from an ASP.
NET web app to an orchestration exposed as a web service might be
expected to return within 2 seconds or less so that the web application
can complete its response to the client within the required timeout
period. To ensure that BizTalk is not holding incoming web service
requests from being submitted to the proper orchestration instance as
soon as they arrive, the batch size should be set to 1 in the
adm_serviceclass table in the Management Database for the web service
receive host. To minimize the time spent by the engine performing its
plumbing functions, you should use atomic transaction scopes whenever
possible to minimize the number of persistence points in the
orchestration.
If tuning is not enough to improve the response time,
you should consider redesigning the interaction with the orchestration
to support polling. For example, the web application will call a one-way
web method as soon as the data needed to pass to the orchestration is
available to that web application, passing in a parameter that the
orchestration will use for correlation. The web application would then
call a two-way request-response web method, right before the web
application really needs the response from the orchestration instance,
passing in the same parameter used for correlation and receiving the
expected response from the orchestration. This request/poll method
allows the orchestration instance extra time to execute and formulate
the response while the web application is still executing and promptly
responding to client requests.
3. Calling an Assembly or Sending a Message to a Web Service
Complex logic in a process is usually wrapped in an
external assembly called by the orchestration at different points in its
execution. The called logic in those external assemblies will execute
on the same thread assigned to the orchestration instance by the engine.
The engine cannot dehydrate the thread while it is executing within the
external assembly. Also, if a failure occurs and the server is
restarted, the orchestration instance will continue execution from the
last persistence point, meaning that it will issue the same call to the
external assembly again, resulting in the execution of the same logic
twice. Logic in those external assemblies should not be affecting state
in permanent data stores.
In the rare circumstances that logic in those
external assemblies does affect state in permanent data stores, those
assemblies should be wrapped in protective transactions that prevent
those permanent data stores from being altered until the transaction
fully executes successfully. Those assemblies should then be exposed as
web services and called through the messaging infrastructure to ensure
that they are called once and only once.
For complex logic in external assemblies called by
orchestrations that are expected to take a considerable amount of time
to execute or pause in idle state for an event to occur and complete,
execution should also be isolated and exposed as a web service. This
ensures that the orchestration instance calling that logic can dehydrate
while waiting for a response to come back from the web service and
relinquish its resources instead of holding them while waiting for the
response.
4. Error Handling and Suspended Instances
It's Murphy's Law that in a production environment
errors are bound to occur. BizTalk solutions not designed to handle and
recover from errors are going to suffer a performance degradation that
might lead to an eventual catastrophic failure. You might be thinking,
"Impossible, I have fault tolerance built into every part of the
solution," but consider the following scenario. A BizTalk orchestration
running within your application is issuing solicit/response calls to an
external web service. One of the services that you are calling happens
to fail intermittently and throws an exception. What will happen to your
orchestration? If you are not properly handling exceptions in your
orchestration, it is going to fail with an unhandled exception, and you
will end with a suspended-unresumable instance on your hands. What is
worse is that you just had a business transaction fail, and the only way
to retry it is to have the logic to recover from it and retry it all
from the beginning. So, lesson number 1: handle exceptions, including
those raised by external assemblies, expressions, or the messaging
subsystem caused by external systems.
OK, so now you have learned your lesson and are
handling exceptions throughout the orchestration. What about those
suspended-resumable message instances being accumulated in the queue
every time the called web service throws an exception? If not handled,
those are going to keep on piling up in the queues and spool and
eventually negatively affect the overall system performance. So, lesson
number 2: configure and handle error reports.
As the BizTalk Server 2009 documentation (Microsoft, 2009) states, "At
each point along the pathway that a message follows through the BizTalk
Server messaging subsystem, failures can occur in the BizTalk Server
infrastructure and in custom pipeline components, orchestrations, and so
forth. If you have specified error reporting for the port through which
a message is entering or will leave, BizTalk Server publishes an error
report message derived from the failing message. The error report
message is routed to the subscribing routing destination, such as a send
port or orchestration; all previously promoted properties are demoted
and selected properties related to the specific messaging failure are
promoted to the message context."
BizTalk Server does not suspend the message when
failed message routing is enabled, it routes the message instead. Failed
message routing can be enabled on both receive and send ports. Any
failure that occurs in adapter processing, pipeline processing, mapping,
or message routing results in an error message if routing for failed
messages is enabled. Also, when a messaging error occurs while an
orchestration is receiving from a receive port or sending to a send
port, the resulting error message is associated with the messaging ports
to which the orchestration is bound. The error message is a clone of
the original message. When a failed message is generated, BizTalk Server
promotes error-report-related message context properties and demotes
regular message context properties before publishing the failed message.
If failed message routing is not enabled, messages that fail are simply
suspended.
"Error messages are delivered to orchestrations or
send ports that have subscribed to receive them. A subscription
typically selects an error message based on the name of the port in
which the messaging error occurred (either a send or a receive port). A
subscription might also filter on other properties promoted to the
error's message context (for example, Inbound-TransportLocation or
FailureCode)" (Microsoft, "BizTalk Server 2009 Documentation," 2009).
In short, to ensure the sanity of your business
transactions and healthy operation of the system, handle exceptions and
configure your application to handle error reports to recover from
errors and prevent suspended message instances.
5. Orchestration Engine Configuration
The orchestration engine uses an XML file called
BTSNTSvc.exe.config to determine certain behaviors. A service reads this
configuration information once, when it is started. Any changes to it
will not be picked up unless the service is stopped and restarted.
See the examples that follow for different nodes and potential values.
5.1. Example: All Validations On
The following configuration example illustrates how to enable assembly, schema, correlation, and logging validation.
<?xml version="1.0" ?>
<configuration>
<configSections>
<section
name="xlangs" type="Microsoft.XLANGs
.BizTalk.CrossProcess.XmlSerializationConfigurationSectionHandler,
Microsoft.XLANGs.BizTalk.CrossProcess" />
</configSections>
<runtime>
<assemblyBinding xmlns="urn:schemas-microsoft-com:asm.v1">
<probing privatePath="BizTalk Assemblies;Developer Tools;
Tracking" />
</assemblyBinding>
</runtime>
<xlangs>
<Configuration>
<Debugging
ValidateAssemblies="true"
ExtendedLogging="false"
/>
</Configuration>
</xlangs>
</configuration>
5.2. Example: Assembly Validation Only
The following configuration example illustrates how to enable assembly validation only.
<?xml version="1.0" ?>
<configuration>
<configSections>
<section
name="xlangs"
type="Microsoft.XLANGs.BizTalk
.CrossProcess.XmlSerializationConfiguration
SectionHandler, Microsoft.XLANGs.BizTalk.CrossProcess"
/>
</configSections>
<runtime>
<assemblyBinding xmlns="urn:schemas-microsoft-com:asm.v1">
<probing privatePath="BizTalk Assemblies;Developer Tools;
Tracking" />
</assemblyBinding>
</runtime>
<xlangs>
<Configuration>
<Debugging
ValidateAssemblies="true"
ValidateSchemas="true"
ValidateCorrelations="true"
ExtendedLogging="true"
/>
</Configuration>
</xlangs>
</configuration>
5.3. Example: Dehydration
The following configuration example illustrates how to configure dehydration settings.
<?xml version="1.0" ?>
<configuration>
<configSections>
<section name="xlangs" type= "Microsoft.XLANGs
.BizTalk.CrossProcess.XmlSerializationConfigurationSectionHandler,
Microsoft.XLANGs.BizTalk.CrossProcess" />
</configSections>
<runtime>
<assemblyBinding xmlns="urn:schemas-microsoft-com:asm.v1">
<probing privatePath="BizTalk Assemblies;Developer Tools;
Tracking" />
</assemblyBinding>
</runtime>
<xlangs>
<Configuration>
<!--
MaxThreshold: the maximal time, in seconds,
that a dehydratable orchestration
is retained in memory before being dehydrated.
MinThreshold: the minimum time, in seconds, that a
dehydratable orchestration is retained in memory before
it is considered for dehydration.
ConstantThreshold: the dynamic threshold usually
fluctuates between the min and max values specified.
However, you can make the threshold
a fixed value by setting this. A value of −1
tells the engine not to use a constant threshold.
-->
<Dehydration MaxThreshold="1800" MinThreshold="1"
ConstantThreshold="−1">
<!--
Currently, virtual memory can become a bottleneck
on 32-bit machines due to unmanaged heap fragmentation,
so you should throttle by this resource as well.
You should reconfigure if /3GB is set.
Optimal and maximal usage are in MB.
-->
<VirtualMemoryThrottlingCriteria OptimalUsage="900"
MaximalUsage="1300" IsActive="true" />
<!--
This is a useful criterion for throttling, but
appropriate values depend on whether the box is being
shared among servers. If the machine has a lot of RAM
and is not being shared with other functions,
then these values can be significantly increased.
Optimal and maximal usage are in MB.
-->
<PrivateMemoryThrottlingCriteria OptimalUsage="50"
MaximalUsage="350" IsActive="true" />
</Dehydration>
</Configuration>
</xlangs>
</configuration>
5.4. Example: AppDomain Configuration
Assemblies are assigned to named domains using
assignment rules (more about which appears in the comments within the
code). If no rule is specified for some assembly, the assembly will be
assigned to an ad hoc domain. The number of such assigned assemblies per
ad hoc domain is determined by the value of AssembliesPerDomain.
<?xml version="1.0" ?>
<configuration>
<configSections>
<section name="xlangs"
type="Microsoft.XLANGs.BizTalk.CrossProcess.XmlSerialization
ConfigurationSectionHandler, Microsoft.XLANGs.BizTalk.CrossProcess" />
</configSections>
<runtime>
<assemblyBinding xmlns="urn:schemas-microsoft-com:asm.v1">
<probing privatePath="BizTalk Assemblies;Developer Tools;Tracking" />
</assemblyBinding>
</runtime>
<xlangs>
<Configuration>
<!--
<!--
AppDomain configuration.
Assemblies are assigned to named domains using assignment. If no 
<BaseSetup>
<ApplicationBase>c:\myAppBase</ApplicationBase>
<ConfigurationFile>c:\myAppBase\myConfig.config</ConfigurationFile>
<DynamicBase>DynamicBase_0</DynamicBase>
<DisallowPublisherPolicy>true</DisallowPublisherPolicy>
<ApplicationName>ApplicationName_0</ApplicationName>
<PrivateBinPath>PrivateBinPath_0</PrivateBinPath>
<PrivateBinPathProbe>PrivateBinPathProbe_0</PrivateBinPathProbe>
<ShadowCopyDirectories>ShadowCopyDirectories_0</ShadowCopyDirectories>
<ShadowCopyFiles>ShadowCopyFiles_0</ShadowCopyFiles>
<CachePath>CachePath_0</CachePath>
<LicenseFile>LicenseFile_0</LicenseFile>
<LoaderOptimization>NotSpecified</LoaderOptimization>
</BaseSetup>
</DefaultSpec>
<!--
- <!--
In this section the user may specify named configurations for specific app domains,
identified by their "friendly name". The format of any app-domain spec is identical
to that of the default app-domain spec.
-->-->
<AppDomainSpecs>
<AppDomainSpec Name="MyDomain1" SecondsIdleBeforeShutdown=
<ExactAssignmentRules>
<!-- An exact assembly rule specifies a strong assembly name and an app domain name.
If the strong assembly name equals the rule's assembly name, it is assigned to
the corresponding app domain.-->
<ExactAssignmentRule AssemblyName="BTSAssembly1,
Version=1.0.0.0, Culture=neutral, PublicKeyToken=9c7731c5584592ad
AssemblyName_0" AppDomainName="MyDomain1" />
AppDomainName_1" />
<ExactAssignmentRule AssemblyName=
"BTSAssembly2, Version=1.0.0.0, Culture=neutral,
PublicKeyToken=9c7731c5584592ad AssemblyName_0"
AppDomainName="AppDomainName_1" />
<ExactAssignmentRule AssemblyName=
"AssemblyName_0" AppDomainName="AppDomainName_1" />
</ExactAssignmentRules>
<PatternAssignmentRules>
<!-- A pattern assignment rule specifies a regular expression
and an app domain name. If the strong assembly name matches the
expression, it is assigned to the corresponding app domain.
This allows version-independent assignment, assignment by
public key token, or assignment by the custom assembly key.
-->
<!--
Assign all assemblies with name BTSAssembly3,
regardless of version and public key,
to the MyDomain1 app domain.
-->
<PatternAssignmentRule AssemblyNamePattern=
" BTSAssembly3, Version=\d.\d.\d.\d, Culture=neutral,
PublicKeyToken=.{16}"AssemblyNamePattern_0"
AppDomainName="AppDomainName_1" />
AppDomainName="MyDomain1" />
<PatternAssignmentRule
AssemblyNamePattern="AssemblyNamePattern_0"
AppDomainName="AppDomainName_1" />
</PatternAssignmentRules>
</AppDomains>
</Configuration>
</xlangs>
</configuration>
In addition to BizTalk-specific configuration
information, the BTSNTSvc.exe.config file is also the place where .NET
application components that run in the context of an orchestration, an
adapter, or a pipeline obtain their configuration information at runtime
using the standard .NET <appSettings> tag under the <configuration>
tag. Because BizTalk already provides a mechanism for custom adapters
and pipeline components to obtain configuration information, the <appSettings> tag in the BTSNTSvc.exe.config file would most likely be used by custom .NET components called from within an orchestration. For example:
<appSettings>
<add key="configParamName" value="configParamValue" />
</appSettings>
