IT tutorials
 
Applications Server
 

BizTalk Server 2009 : What the Maestro Needs to Know: Advanced Orchestration Concepts - Pitfalls of Orchestration Development

5/24/2013 7:29:58 PM
- How To Install Windows Server 2012 On VirtualBox
- How To Bypass Torrent Connection Blocking By Your ISP
- How To Install Actual Facebook App On Kindle Fire

Some BizTalk artifacts lend themselves nicely to particular patterns of solutions, while their use could be devastating to the performance of a different solution. The following sections describe some of these solution patterns and the pitfalls that you as a BizTalk developer need to watch out for.

1. Batch Processing and Legacy Systems

Legacy mainframe systems excel in what they were designed to do best, batch processing. With the emergence of online applications and progressive automation of internal processes, the load onto these legacy systems is exponentially increasing. Instead of handling a handful of requests concurrently, they are now expected to handle hundreds and thousands of concurrent requests as business goes online and customers perform actions resulting in transactions executing on these back-end legacy systems. EAI platforms such as BizTalk facilitate the integration of newly developed interactive systems with legacy back ends such as those batch-processing mainframes. Third-party or custom adapters could be used to communicate with the mainframes, and transformation maps could be used to translate data back and forth. BizTalk-based solutions developed to integrate with the mainframe will leverage the Messagebox as a queue to throttle and batch transaction requests to the mainframes to ensure their ability to handle the incoming load of requests.

Such a solution works perfectly fine, but should be stress and load tested to ensure that the queuing and throttling does not affect the overall performance of the solution. With BizTalk 2006 and 2009 auto-throttling, queuing is not a big problem, but with previous versions of BizTalk, queuing leads to the growth of the spool table in the Messagebox, which affects the overall performance of the Messagebox and consequently the BizTalk server.

2. Interactive Orchestrations (the Request/Response Approach)

The response time in interactive orchestrations such as workflows exposed as web services could be critical to the correctness of the complete solutions. For example, a call from an ASP. NET web app to an orchestration exposed as a web service might be expected to return within 2 seconds or less so that the web application can complete its response to the client within the required timeout period. To ensure that BizTalk is not holding incoming web service requests from being submitted to the proper orchestration instance as soon as they arrive, the batch size should be set to 1 in the adm_serviceclass table in the Management Database for the web service receive host. To minimize the time spent by the engine performing its plumbing functions, you should use atomic transaction scopes whenever possible to minimize the number of persistence points in the orchestration.

If tuning is not enough to improve the response time, you should consider redesigning the interaction with the orchestration to support polling. For example, the web application will call a one-way web method as soon as the data needed to pass to the orchestration is available to that web application, passing in a parameter that the orchestration will use for correlation. The web application would then call a two-way request-response web method, right before the web application really needs the response from the orchestration instance, passing in the same parameter used for correlation and receiving the expected response from the orchestration. This request/poll method allows the orchestration instance extra time to execute and formulate the response while the web application is still executing and promptly responding to client requests.

3. Calling an Assembly or Sending a Message to a Web Service

Complex logic in a process is usually wrapped in an external assembly called by the orchestration at different points in its execution. The called logic in those external assemblies will execute on the same thread assigned to the orchestration instance by the engine. The engine cannot dehydrate the thread while it is executing within the external assembly. Also, if a failure occurs and the server is restarted, the orchestration instance will continue execution from the last persistence point, meaning that it will issue the same call to the external assembly again, resulting in the execution of the same logic twice. Logic in those external assemblies should not be affecting state in permanent data stores.

In the rare circumstances that logic in those external assemblies does affect state in permanent data stores, those assemblies should be wrapped in protective transactions that prevent those permanent data stores from being altered until the transaction fully executes successfully. Those assemblies should then be exposed as web services and called through the messaging infrastructure to ensure that they are called once and only once.

For complex logic in external assemblies called by orchestrations that are expected to take a considerable amount of time to execute or pause in idle state for an event to occur and complete, execution should also be isolated and exposed as a web service. This ensures that the orchestration instance calling that logic can dehydrate while waiting for a response to come back from the web service and relinquish its resources instead of holding them while waiting for the response.

4. Error Handling and Suspended Instances

It's Murphy's Law that in a production environment errors are bound to occur. BizTalk solutions not designed to handle and recover from errors are going to suffer a performance degradation that might lead to an eventual catastrophic failure. You might be thinking, "Impossible, I have fault tolerance built into every part of the solution," but consider the following scenario. A BizTalk orchestration running within your application is issuing solicit/response calls to an external web service. One of the services that you are calling happens to fail intermittently and throws an exception. What will happen to your orchestration? If you are not properly handling exceptions in your orchestration, it is going to fail with an unhandled exception, and you will end with a suspended-unresumable instance on your hands. What is worse is that you just had a business transaction fail, and the only way to retry it is to have the logic to recover from it and retry it all from the beginning. So, lesson number 1: handle exceptions, including those raised by external assemblies, expressions, or the messaging subsystem caused by external systems.

OK, so now you have learned your lesson and are handling exceptions throughout the orchestration. What about those suspended-resumable message instances being accumulated in the queue every time the called web service throws an exception? If not handled, those are going to keep on piling up in the queues and spool and eventually negatively affect the overall system performance. So, lesson number 2: configure and handle error reports.[] As the BizTalk Server 2009 documentation (Microsoft, 2009) states, "At each point along the pathway that a message follows through the BizTalk Server messaging subsystem, failures can occur in the BizTalk Server infrastructure and in custom pipeline components, orchestrations, and so forth. If you have specified error reporting for the port through which a message is entering or will leave, BizTalk Server publishes an error report message derived from the failing message. The error report message is routed to the subscribing routing destination, such as a send port or orchestration; all previously promoted properties are demoted and selected properties related to the specific messaging failure are promoted to the message context."[]

[] In BizTalk 2004, the messaging subsystem publishes negative acknowledgments (NACKs) whenever a message fails and its instance is suspended. Orchestrations should be able to subscribe to those NACKs and perform the required recovery and consequent cleanup of those suspended instances. The handling of those NACKs and cleanup of suspended messages is essential to ensure the system's overall healthy operation as well as the sanity of the business transaction.

[] "An error message is a clone of the original failed message, with all previously promoted properties demoted and with a set of error-specific properties promoted to the message context. . . . Previously promoted properties are demoted to avoid unintended delivery to subscribers not designated to receive the error message. The error message is published for distribution to subscribers (orchestrations, send ports, and send port groups)" (Microsoft, "BizTalk Server 2009 Documentation").

BizTalk Server does not suspend the message when failed message routing is enabled, it routes the message instead. Failed message routing can be enabled on both receive and send ports. Any failure that occurs in adapter processing, pipeline processing, mapping, or message routing results in an error message if routing for failed messages is enabled. Also, when a messaging error occurs while an orchestration is receiving from a receive port or sending to a send port, the resulting error message is associated with the messaging ports to which the orchestration is bound. The error message is a clone of the original message. When a failed message is generated, BizTalk Server promotes error-report-related message context properties and demotes regular message context properties before publishing the failed message. If failed message routing is not enabled, messages that fail are simply suspended.

"Error messages are delivered to orchestrations or send ports that have subscribed to receive them. A subscription typically selects an error message based on the name of the port in which the messaging error occurred (either a send or a receive port). A subscription might also filter on other properties promoted to the error's message context (for example, Inbound-TransportLocation or FailureCode)" (Microsoft, "BizTalk Server 2009 Documentation," 2009).

In short, to ensure the sanity of your business transactions and healthy operation of the system, handle exceptions and configure your application to handle error reports to recover from errors and prevent suspended message instances.

5. Orchestration Engine Configuration

The orchestration engine uses an XML file called BTSNTSvc.exe.config to determine certain behaviors. A service reads this configuration information once, when it is started. Any changes to it will not be picked up unless the service is stopped and restarted.

See the examples that follow for different nodes and potential values.

5.1. Example: All Validations On

The following configuration example illustrates how to enable assembly, schema, correlation, and logging validation.

<?xml version="1.0" ?>
<configuration>
       <configSections>
             <section
                    name="xlangs" type="Microsoft.XLANGs 
.BizTalk.CrossProcess.XmlSerializationConfigurationSectionHandler,
Microsoft.XLANGs.BizTalk.CrossProcess" /> </configSections> <runtime> <assemblyBinding xmlns="urn:schemas-microsoft-com:asm.v1"> <probing privatePath="BizTalk Assemblies;Developer Tools;
Tracking" /> </assemblyBinding> </runtime> <xlangs> <Configuration> <Debugging ValidateAssemblies="true" ExtendedLogging="false" /> </Configuration> </xlangs> </configuration>

5.2. Example: Assembly Validation Only

The following configuration example illustrates how to enable assembly validation only.

<?xml version="1.0" ?>
<configuration>
       <configSections>
              <section
                     name="xlangs"
                     type="Microsoft.XLANGs.BizTalk 
.CrossProcess.XmlSerializationConfiguration
SectionHandler, Microsoft.XLANGs.BizTalk.CrossProcess" /> </configSections>

<runtime>
              <assemblyBinding xmlns="urn:schemas-microsoft-com:asm.v1">
                     <probing privatePath="BizTalk Assemblies;Developer Tools; 
Tracking" /> </assemblyBinding> </runtime> <xlangs> <Configuration> <Debugging ValidateAssemblies="true" ValidateSchemas="true" ValidateCorrelations="true" ExtendedLogging="true" /> </Configuration> </xlangs> </configuration>

5.3. Example: Dehydration

The following configuration example illustrates how to configure dehydration settings.

<?xml version="1.0" ?>
<configuration>
       <configSections>
              <section name="xlangs" type= "Microsoft.XLANGs 
.BizTalk.CrossProcess.XmlSerializationConfigurationSectionHandler,
Microsoft.XLANGs.BizTalk.CrossProcess" /> </configSections> <runtime> <assemblyBinding xmlns="urn:schemas-microsoft-com:asm.v1"> <probing privatePath="BizTalk Assemblies;Developer Tools;
Tracking" /> </assemblyBinding> </runtime> <xlangs> <Configuration> <!-- MaxThreshold: the maximal time, in seconds, that a dehydratable orchestration is retained in memory before being dehydrated. MinThreshold: the minimum time, in seconds, that a dehydratable orchestration is retained in memory before it is considered for dehydration. ConstantThreshold: the dynamic threshold usually fluctuates between the min and max values specified.

However, you can make the threshold
                     a fixed value by setting this. A value of −1
                     tells the engine not to use a constant threshold.
                     -->
                     <Dehydration MaxThreshold="1800" MinThreshold="1"
                       ConstantThreshold="−1">
                            <!--
                            Currently, virtual memory can become a bottleneck
                            on 32-bit machines due to unmanaged heap fragmentation,
                            so you should throttle by this resource as well.
                            You should reconfigure if /3GB is set.
                            Optimal and maximal usage are in MB.
                            -->
                            <VirtualMemoryThrottlingCriteria OptimalUsage="900"
                              MaximalUsage="1300" IsActive="true" />
                            <!--
                             This is a useful criterion for throttling, but
                             appropriate values depend on whether the box is being
                             shared among servers. If the machine has a lot of RAM
                             and is not being shared with other functions,
                             then these values can be significantly increased.
                             Optimal and maximal usage are in MB.
                             -->
                             <PrivateMemoryThrottlingCriteria OptimalUsage="50"
                               MaximalUsage="350" IsActive="true" />
                      </Dehydration>
               </Configuration>
       </xlangs>
</configuration>

					  

5.4. Example: AppDomain Configuration

Assemblies are assigned to named domains using assignment rules (more about which appears in the comments within the code). If no rule is specified for some assembly, the assembly will be assigned to an ad hoc domain. The number of such assigned assemblies per ad hoc domain is determined by the value of AssembliesPerDomain.

<?xml version="1.0" ?>
<configuration>
    <configSections>
        <section name="xlangs"
          type="Microsoft.XLANGs.BizTalk.CrossProcess.XmlSerialization 
ConfigurationSectionHandler, Microsoft.XLANGs.BizTalk.CrossProcess" /> </configSections>

<runtime>
        <assemblyBinding xmlns="urn:schemas-microsoft-com:asm.v1">
            <probing privatePath="BizTalk Assemblies;Developer Tools;Tracking" />
        </assemblyBinding>
    </runtime>
    <xlangs>
        <Configuration>
            <!--
                <!--
  AppDomain configuration.
                 Assemblies are assigned to named domains using assignment. If no 
rule is specified for some assembly, the assembly will be assigned to an ad hoc domain. The number of such assigned assemblies per ad hoc domain is determined by the value of AssembliesPerDomain. -->--> <AppDomains AssembliesPerDomain="10"> <!-- <!-- In this section, the user may specify default configuration for any app domain created that does not have a named configuration associated with it (see AppDomainSpecs later in this example). SecondsEmptyBeforeShutdown is the number of seconds that an app domain is empty (that is, it does not contain any orchestrations) before being unloaded. Specify −1 to signal that an app domain should never unload, even when empty. Similarly, SecondsIdleBeforeShutdown is the number of seconds that an app domain is idle (that is, it contains only dehydratable orchestrations) before being unloaded.Specify −1 to signal that an app domain should never unload when idle but not empty. When an idle but nonempty domain is shut down, all of the contained instances are dehydrated first. --> --> <DefaultSpec SecondsIdleBeforeShutdown="1200" SecondsEmptyBeforeShutdown="1800"> <!-- <!-- BaseSetup is a serialized System.AppDomainSetup object. This is passed as is to AppDomain.CreateAppDomain() and can be used to influence assembly search path, etc. --> -->

<BaseSetup>
   <ApplicationBase>c:\myAppBase</ApplicationBase>
   <ConfigurationFile>c:\myAppBase\myConfig.config</ConfigurationFile>
   <DynamicBase>DynamicBase_0</DynamicBase>
   <DisallowPublisherPolicy>true</DisallowPublisherPolicy>
   <ApplicationName>ApplicationName_0</ApplicationName>
   <PrivateBinPath>PrivateBinPath_0</PrivateBinPath>
   <PrivateBinPathProbe>PrivateBinPathProbe_0</PrivateBinPathProbe>
   <ShadowCopyDirectories>ShadowCopyDirectories_0</ShadowCopyDirectories>
   <ShadowCopyFiles>ShadowCopyFiles_0</ShadowCopyFiles>
   <CachePath>CachePath_0</CachePath>
   <LicenseFile>LicenseFile_0</LicenseFile>
   <LoaderOptimization>NotSpecified</LoaderOptimization>
  </BaseSetup>
  </DefaultSpec>
                  <!--
                      - <!--
  In this section the user may specify named configurations for specific app domains,
  identified by their "friendly name". The format of any app-domain spec is identical
  to that of the default app-domain spec.
                  -->-->
                  <AppDomainSpecs>
                      <AppDomainSpec Name="MyDomain1" SecondsIdleBeforeShutdown= 
"−1" SecondsEmptyBeforeShutdown="12000"> <BaseSetup> <PrivateBinPath>c:\PathForAppDomain1</PrivateBinPath> <PrivateBinPath>PrivateBinPath_0</PrivateBinPath> <PrivateBinPathProbe>PrivateBinPathProbe_0</PrivateBinPathProbe> </BaseSetup> </AppDomainSpec> <AppDomainSpec Name="MyFrequentlyUnloadingDomainMyTrashyDomain" SecondsIdleBeforeShutdown="60" SecondsEmptyBeforeShutdown= "60" /> </AppDomainSpecs> <!-- The PatternAssignmentRules and ExactAssignmentRules control assignment of assemblies to app domains. When a messag arrives, the name of its corresponding orchestration's assembly is determined. Then, the assembly is assigned an app domain name. The rules guide this assignment. Exact rules are consulted first, in their order of definition, and then the pattern rules. The first match is used. If no match is found, the assembly will be assigned to an ad hoc domain. The configuration and number of assemblies per ad hoc domain is controlled by the AssembliesPerDomain attribute and the DefaultSpec section. -->

<ExactAssignmentRules>

<!-- An exact assembly rule specifies a strong assembly name and an app domain name.
 If the strong assembly name equals the rule's assembly name, it is assigned to
 the corresponding app domain.-->
                    <ExactAssignmentRule AssemblyName="BTSAssembly1, 
Version=1.0.0.0, Culture=neutral, PublicKeyToken=9c7731c5584592ad
AssemblyName_0" AppDomainName="MyDomain1" /> AppDomainName_1" /> <ExactAssignmentRule AssemblyName= "BTSAssembly2, Version=1.0.0.0, Culture=neutral,
PublicKeyToken=9c7731c5584592ad AssemblyName_0" AppDomainName="AppDomainName_1" /> <ExactAssignmentRule AssemblyName= "AssemblyName_0" AppDomainName="AppDomainName_1" /> </ExactAssignmentRules> <PatternAssignmentRules> <!-- A pattern assignment rule specifies a regular expression and an app domain name. If the strong assembly name matches the expression, it is assigned to the corresponding app domain. This allows version-independent assignment, assignment by public key token, or assignment by the custom assembly key. --> <!-- Assign all assemblies with name BTSAssembly3, regardless of version and public key, to the MyDomain1 app domain. --> <PatternAssignmentRule AssemblyNamePattern=
" BTSAssembly3, Version=\d.\d.\d.\d, Culture=neutral,
PublicKeyToken=.{16}"AssemblyNamePattern_0" AppDomainName="AppDomainName_1" /> AppDomainName="MyDomain1" /> <PatternAssignmentRule AssemblyNamePattern="AssemblyNamePattern_0" AppDomainName="AppDomainName_1" /> </PatternAssignmentRules> </AppDomains> </Configuration> </xlangs> </configuration>

In addition to BizTalk-specific configuration information, the BTSNTSvc.exe.config file is also the place where .NET application components that run in the context of an orchestration, an adapter, or a pipeline obtain their configuration information at runtime using the standard .NET <appSettings> tag under the <configuration> tag. Because BizTalk already provides a mechanism for custom adapters and pipeline components to obtain configuration information, the <appSettings> tag in the BTSNTSvc.exe.config file would most likely be used by custom .NET components called from within an orchestration.[] For example:

[] A separate custom config file may be specified per app domain using the <ConfigurationFile/> key to hold the required configuration and app settings for custom application components.

<appSettings>
<add key="configParamName" value="configParamValue" />
</appSettings>
 
Others
 
- BizTalk Server 2009 : What the Maestro Needs to Know: Advanced Orchestration Concepts - Dynamic Message Transformations
- BizTalk Server 2009 : What the Maestro Needs to Know: Advanced Orchestration Concepts - Correlation
- Installing Exchange Server 2007 : Planning an Active Directory Infrastructure
- Installing Exchange Server 2007 : Understanding the Prerequisites for Exchange Server 2007, Understanding Active Directory Requirements for Exchange Server 2007
- Installing Configuration Manager 2007 : Multisite Configuration (part 3) - Troubleshooting Secondary Site Installation
- Installing Configuration Manager 2007 : Multisite Configuration (part 2) - Configuring Senders, Attaching to Parent, Installing Secondary Sites
- Installing Configuration Manager 2007 : Multisite Configuration (part 1) - Configuring Addresses
- Improving Dynamics GP with Hacks : Using Comments without needing a comment name
- Improving Dynamics GP with Hacks : Building custom help with Window notes
- SharePoint 2010 : Service Applications - Setting up the Managed Metadata Service
 
25 Inspiring Game of Thrones Quotes
 
Top 10
 
- Microsoft Visio 2013 : Adding Structure to Your Diagrams - Finding containers and lists in Visio (part 2) - Wireframes,Legends
- Microsoft Visio 2013 : Adding Structure to Your Diagrams - Finding containers and lists in Visio (part 1) - Swimlanes
- Microsoft Visio 2013 : Adding Structure to Your Diagrams - Formatting and sizing lists
- Microsoft Visio 2013 : Adding Structure to Your Diagrams - Adding shapes to lists
- Microsoft Visio 2013 : Adding Structure to Your Diagrams - Sizing containers
- Microsoft Access 2010 : Control Properties and Why to Use Them (part 3) - The Other Properties of a Control
- Microsoft Access 2010 : Control Properties and Why to Use Them (part 2) - The Data Properties of a Control
- Microsoft Access 2010 : Control Properties and Why to Use Them (part 1) - The Format Properties of a Control
- Microsoft Access 2010 : Form Properties and Why Should You Use Them - Working with the Properties Window
- Microsoft Visio 2013 : Using the Organization Chart Wizard with new data
programming4us programming4us
 
Popular tags
 
Video Tutorail Microsoft Access Microsoft Excel Microsoft OneNote Microsoft PowerPoint Microsoft Project Microsoft Visio Microsoft Word Active Directory Biztalk Exchange Server Microsoft LynC Server Microsoft Dynamic Sharepoint Sql Server Windows Server 2008 Windows Server 2012 Windows 7 Windows 8 Adobe Indesign Adobe Flash Professional Dreamweaver Adobe Illustrator Adobe After Effects Adobe Photoshop Adobe Fireworks Adobe Flash Catalyst Corel Painter X CorelDRAW X5 CorelDraw 10 QuarkXPress 8 windows Phone 7 windows Phone 8 BlackBerry Android Ipad Iphone iOS