30 Useful Biztalk Server Tips PDF
30 Useful Biztalk Server Tips PDF
30 Useful Biztalk Server Tips PDF
BIZTALK
SERVER
TIPS
Ricardo Torre
contents
2
Efficient
Host
Separation
Choose
robust
adapters
Monitor
performance
SLAs with
BAM
12
BAM
lightweight
tracking
Minimize
persistence
points
Performance
Testing is key
10
13
Review
the
performance
of your
environment
regularly
Take advantage
of Multi MessageBox
Configuration
Package
all your
resources
into MSI
Build your
custom pipelines
with components using
default properties
Cluster
host
instances
with
adapter
like
FTP or
POP3
14
15
Health
Check
with
Message
Box
Viewer
11
Create
security
boundaries
Use the
Business Rule
Engine to implement
Business Logic
Split big
messages in the
receive pipeline
16
Use
Visual Studio for
functional and load testing
18
Cluster
the Master
Secret Server
21
Plan
your
Backup and
DR strategy
19
Purge
BAM Data
22
Use
Webgarden
for IIS
scalability
24
27
Use fast
disks for your
BizTalk Database subsystem
20
23
25
Maintain you
Message Box efficiently
Implement a
Enterprise
Service
Bus using the
ESB Toolkit
17
28
30
Use SSO
as the application
configuration store
Configure and
activate the DTA
Purge and Archive Job
26
Consider
Test 100%
to prefer
of your code
XSLT in maps
Avoid
Orchestrations
when possible
29
Develop
adapter
using WCF
Take advantage of
Multi MessageBox Configuration
TWEET
SHARE
POST
When configuring hosts always consider the best practices for host separation, start with the separation of artifacts per functionality (receive, send,
orchestration and tracking) then consider isolation (per application, business
unit, etc), then for performance reasons and finally for reliability reasons. A
good Host separation policy will help you scale your environment, ensure stability and security.
Generally it is recommended to start any BizTalk environment with at least 4
hosts, for Receiving, Orchestrations, Sending and Tracking. This follows the
first principle of separation of functionality which will be easier to configure
scalability and easier to control the behaviour such as stopping all receives to
enable an environment drain. Another important host separation technique is
to have a dedicated tracking host, since tracking is a low priority sub-service
if you have tracking in a very busy host it might start accumulating tracking
information on the BizTalk Message Box.
Isolating artifacts with different security requirements is another reason for
host separation, you can assign different Windows Groups, Users and certificates to each host creating security boundaries which will reduce the risk of
running them in the same environment as other artifacts which increases the
return on investment you have done in BizTalk.
You can also separate hosts for security and reliability reasons, since each host
has its own set of dedicated resources such as database tables and windows
services you can separate artifacts or applications that have high performance
requirements that will benefit from these additional resources. If you have
some unreliable artifact you can also use host separation for reducing the
impact of such component (Custom Adapter, Pipeline, etc.) since if that particular host instance crashes it wont affect anything else .
You can read more on this topic here.
SHARE
POST
SHARE
POST
SHARE
POST
Run Message Box Viewer (MBV) daily to evaluate the health state of your BizTalk environment, this tool brings the knowledge of Microsoft Product Group,
Support and Field Engineers into a single report. You can schedule MBV from
either MBV itself or via BizTalk360 to run every day and be notified if any new
critical or noncritical errors surface.
Microsoft recommends that you run Message Box Viewer every day as part of
the daily operational checklist (see: Checklist: Performing Daily Maintenance
Checks) for checking the health of everything that is BizTalk related such as
the databases, configuration, best practices, registry settings, service packs
and cumulative updates, runtime problems, adapter problems, etc. The likelihood of what you want to check to be done by MBV is very high.
The reports generated are also in a big degree a documentation of your BizTalk environment and can be used to review the BizTalk configuration and
identify message flows within the system. In case problems are identified that
can be solved by the BizTalk Terminator, MBV will generate a configuration file
to load in the BizTalk Terminator and allow you to run the solutions needed
for your BizTalk system.
SHARE
POST
SHARE
POST
SHARE
POST
Include every artifact in your BizTalk Application as a MSI resource for a fully
automated application installation, you can include anything you need including script files that are executed during install and uninstall. This will give you
a single installation package with minimal manual intervention which reduces
the likely would of introducing problems.
TWEET
SHARE
POST
BAM can be easily used to calculate performance SLAs and provide this information to business owners, you can then enable BAM Alerting and use that
for monitoring your performance SLAs. If you also use the BAM API you will
be able to extend this SLA monitoring capability outside the boundaries of
BizTalk.
By defining an activity in BAM that collects at least two milestones you will be
able to calculate the duration between them using a calculated field in BAM.
Once this is done you now have access to the KPI you need to measure and
you can create a query in BAM to identify instances where the duration was
bigger than your defined threshold. You can also use BAM Alerts to receive a
notification every time your query returns new results.
Read more on this topic BAM to Implement Service and SLA Monitoring
TWEET
SHARE
POST
10
Build your custom pipelines with components using their default properties
and then use runtime bindings for adjusting them for multiple purposes. This
will reduce the amount of artifacts you need in your solution and will increase
the code re-usage making it easier to test and deploy.
When developing BizTalk applications it is common that developers will create
multiple pipelines just for addressing variations in the configuration of pipeline components. BizTalk allows you to change this properties through configuration and that can be imported and exported with bindings. This in some
cases can mean hundreds of artifacts that you save from creating and deploying; this will also simplify testing your applications.
A very common example of this is the flat file disassembler where you need to
identify the schema to which the flat file being received should be parsed to.
TWEET
SHARE
POST
11
Lock down your BizTalk system by using different security context when accessing sensitive information, you can achieve this by running your hosts and
host instances with different Windows Groups and Users. Reducing the surface of attack with this approach will make your environment less prone to
attacks.
Security is a very important topic when integrating with highly critical applications inside and across organizations. Planning and implementing security
is a demanding task. In BizTalk you will have to assign permissions to receive,
send and query against your most important applications giving the BizTalk
environment a considerable amount of power and at the same time risk of
getting compromised.
An approach recommended is to limit the permissions needed by BizTalk to
just exactly where they are needed, so if my receive host need to be able to
pool a particular SQL Table I should create a BizTalk user account and corresponding Windows Group to assign to the Host and Host Instance that needs
that privilege while at the same time only running artifacts in the same host
that share the same level of permissions. Read more the BizTalk Security
SHARE
POST
12
Collect performance counters on a regular basis and analyse them with PAL
for the best performance insight of a BizTalk environment. Collect performance data during your utilization peak time and compare the results with a
baseline to be able to plan for the right capacity in your system.
Monitoring the performance of your BizTalk System is key for a stable and
predictable behaviour. Since the amount of counters available in a BizTalk
System is vast it is hard to know which ones to pay more attention or understanding the correlations of their values. This is where PAL will help you by
providing a list of counters recommended to collect and then parses all these
counters and evaluates them based on the knowledge of experienced BizTalk
engineers and generates alerts when thresholds are being violated. A comprehensive report is generated after running this tool.
Not only will you benefit from the collective knowledge for BizTalk but also
from other software in the stack such as Windows, IIS, SQL Server and more.
Start using it Performance Analysis of Logs (PAL) Tool.
TWEET
SHARE
POST
13
Use Microsoft Clustering for adapters that required a single host instance running at a time like FTP or POP3 to avoid duplicate messages while providing
high availability. Failure to implement this will result in either having to deal
with duplicate messages or implementing manual processes for performing
failover.
Due to the nature of some protocols it is impossible to guarantee high availability, scalability and failover. In this cases we normally see customers disabling one of the host instances to prevent duplicate messages. This only
provides manual failover and what should be done is to cluster the hosts
where this adapters run, Windows Clustering will guarantee that at least on
host instance is running and that no more than one is working.
Read more on Improving Fault Tolerance in BizTalk Server by Using a Windows
Server Cluster
TWEET
SHARE
POST
14
Use Business Rule Engine to implement business logic that is modular, reusable and simple. It will allow you to operate on information contained in .NET
objects, database tables and XML Documents. BRE also enables developers to
create and maintain applications with minimal effort.
The Business Rule Engine can be a good way of to modularize the constantly
changing business logic that is often developed inside Orchestrations, making
changes a more smooth process since updating a policy to the latest version
doesnt require any downtime at all. With some training even non BizTalk developers or tech savvy non developers can create and test rules.
Business Rule Engine is also a high performance engine to evaluate business
rules even when compared with WF Rules, read more on this topic WF Rules
and MS BRE Comparing Performance
TWEET
SHARE
POST
15
If you have a big message consider using envelop schemas and the default
pipelines to split the message at the entrance point in BizTalk for best performance and high resource utilization. Using this method also doesnt require
the creation of custom pipelines or very expensive splitting in the orchestrations.
TWEET
SHARE
POST
16
Use Visual Studio test capabilities for functional testing and load testing, you
can simulate real load of your environment while at the same time monitoring
performance and behavior of your environment during the test in real time.
When load testing start by doing short duration tests first and then long duration reliability tests.
Here are few great links
Using Visual Studio To Generate Load Against a Two-Way Request-Response
WCF Receive Location
Load Testing BizTalk Server Solutions with Visual Studio
TWEET
SHARE
POST
17
SQL Server is vital for any BizTalk environment, ensure maximum performance
for the most demanding databases with a fast disk subsystem, consider using
RAID 1+0 and SSDs for a high performance environment. In the majority of
the BizTalk environments this is the first physical bottleneck you will face.
When doing load testing you will find that after you configure you environment to avoid any logical bottlenecks you will finally find a physical one. Normally the very first physical bottleneck you see is the IO disk subsystem performance on the SQL Server, this makes investing in the disk subsystem the
most important scaling you can do on an environment. Betting on a highly
available RAID system with fast disks is very important.
Read more on Monitoring and Reducing Database I/O Contention
TWEET
SHARE
POST
18
Cluster SSO Master Secret Server using Windows Clustering to guarantee this
component is also highly available, you have to do this to achieve full HA. The
best candidate servers on which you can cluster the Master Secret Server is
the SQL Server cluster where BizTalk is installed because you can reuse this
cluster for this low weight service.
If you have not clustered the Master Secret Server there is no high availability
in the BizTalk environment, in this cases the first machine you have installed is
the Master Secret Server and if it becomes unavailable the whole BizTalk environment will not work since it depends on the Master Secret server to read
configuration from the SSO database.
If you want to install the SSO cluster on the BizTalk machines you can only do
it if you also cluster all the hosts in the same Cluster application which means
you will have an active passive BizTalk runtime environment. Read more on
High-Availability SSO Installation Options
TWEET
SHARE
POST
19
Use the out of the box SSIS packages for purging old data from BAM ensuring
only relevant information is available and that the Performance of the BAMPrimaryImport is the best. You can create SQL Jobs to run this packages which
are automatically generated when you deploy your BAM Activities.
TWEET
SHARE
POST
20
For BizTalk Application settings use SSO for secure, scalable and highly available store. This is more secure than the usually used BizTalk configuration file
and is more maintainable since you will have a single place to update your
BizTalk Applications configuration and much more easily you can automate
the deployment of these settings.
Here is a YouTube video showcasing Using SSO for Application Configuration
TWEET
SHARE
POST
21
Configure and activate the BizTalk Backup Job to generate the only supported
BizTalk backup files and ensure you are able to restore them by planning and
testing a disaster recovery plan. The success criteria for a Disaster Recovery
plan happens only when you test your scenario.
The only supported way of backing up the BizTalk databases is by using the
out of the box Backup BizTalk Server Job combined the BizTalk Log Shipping
for automatic restoration. It is recommended that you plan the recovery of all
BizTalk server components including the databases, configuration and applications; you should also consider the recovery point objective (RPO) and the
recovery time objective (RTO) when considering about investing in your disaster recovery plan.
Read more on this topic here.
TWEET
SHARE
POST
22
When IIS becomes a bottleneck you can use webgarden for IIS scalability, this
will create multiple hosting processes in the server and IIS will manage the
balancing of load between them. Optimizing your web layer like this is very
simple and highly reliable when compared to changing other internal IIS engine settings.
If you have the need to process a large amount of two way receive port connections through IIS in a BizTalk Application it is likely you will exhaust some
threading capabilities in the IIS engine and hit a logical bottleneck, if you have
access to an IIS expert you can possible reconfigure the engine to overcome
that limitation, or instead using the very scalable and easy to use web garden
capability.
Read On Web Gardens, ASP.NET, and IIS 6.0
TWEET
SHARE
POST
23
Configure and activate the DTA Purge And Archive Job to maintain a healthy
BizTalkDTADb database, just keep the tracking information you need and delete from the live tracking database a soon as possible for the best experience
when searching or understanding what happened to your business processes
in BizTalk.
A big BizTalkDTADb will result in frequent timeouts and a generally bad experience when working with tracking data in BizTalk, the best way to handle
this problem is by configuring and activating the DTA Purge and Archive Job.
If you do need the tracking information and you need to have a smaller database you can use the archiving capabilities for making older tracking information available for querying.
See How to Configure the DTA Purge and Archive Job
TWEET
SHARE
POST
24
Dont let your BizTalk Message Box database grow significantly by accumulating suspended messages, put in place a monitoring mechanism to identify
and resolve suspended messages as soon as possible. When you find a database integrity problem that is causing your databases to grow use the BizTalk
Terminator to resolve this.
Maintaining the health of your BizTalk databases is vital for any BizTalk environment, specially when we are talking about the BizTalkMsgBoxDb, there are
several reasons for the databases to grow like suspended messages that you
should avoid and resolved quickly when they happen. You can also have some
integrity problems identified by using the out of the box Monitor Job or by
running a tool like the Message Box Viewer, in this cases you can run the BizTalk Terminator which is a tool provided by Microsoft to address these issues.
Read about the BizTalk Terminator
TWEET
SHARE
POST
25
Consider to prefer
XSLT in maps
When developing maps consider using XSLT as the preferred language for
describing your transformations by either using Custom XSLT, Scripting Functoids or even when developing Custom Functoids. This will result in maps that
are faster to run and easier to debug.
Using the out of the box functoids is sometimes hard or impossible to implement the necessary transformations leaving developers a lot of options on
how to overcome those limitations. Using custom XSLT as a full map, a inline
XSLT scripting functoid or developing a functoid that outputs XSLT is in the
majority of the times the best option due to the end result being just on single XSLT that needs to be compiled and run just like any other simple map
without any need for external components.
TWEET
SHARE
POST
26
TWEET
SHARE
POST
27
Implement a Enterprise
Service Bus using the ESB Toolkit
Implement a Enterprise
Service Bus using the ESB Toolkit
TWEET
SHARE
POST
28
Avoid Orchestrations
when possible
Use static routing, content based routing or itineraries to avoid using Orchestrations and use routing of failed messages for advance error handling since
messaging doesnt provide a rich error handling capability. This approach will
give you the high performance of messaging and the power of the Orchestrations when necessary.
When a high volume of messages is going to be processed in BizTalk every
cost is important, activating an Orchestration has a significant impact due to
the additional Message Box hop and the Orchestration execution. In a lot of
scenarios you cannot avoid Orchestrations, but in many cases you can avoid
them by making use of the messaging capabilities such as routing or transformations in the ports. Another interesting aspect of this is that if something
goes bad with simple messaging and you need the power of the Orchestrations you can still use them when combined with Routing of Failed Messages.
TWEET
SHARE
POST
29
When developing new adapters create a Custom WCF Channel or use the
WCF LOB SDK as a reference starting point, this will allow you to create a scalable and easy to host adapter that can be used across other .NET solutions.
This level of flexibility will make the adapter more likely to be reused somewhere else besides the BizTalk world and so being more valuable than simply
building it with the BizTalk Adapter framework.
If you have to decide to implement a BizTalk Adapter you will want to make it
so that it can be as relevant as possible and choosing the framework you will
use is an important task. The simple fact that developing adapters for WCF
makes them reusable should be a compelling enough reason to decide.
You can understand the Differences Between the WCF LOB Adapter SDK and
the BizTalk Server Adapter Framework here.
TWEET
SHARE
POST
30
Use Event Tracking for Windows (ETW) for a high performance tracking and
debugging of your BizTalk Applications. Since this method uses some high
performance structures within Windows to host tracking the impact is near
zero if you are not consuming the output and highly valuable if you are troubleshooting a problem.
You can access to an implementation of this type of tracing on Best Practices
for Instrumenting High Performance BizTalk Solutions
TWEET
SHARE
POST
www.BizTalk360.com