BizTalk Server Optimization & Maintenance

Download this post as Word | PDF

I have engaged with BizTalk Server 2006 R2 application development, maintenance & performance tuning processes during my past projects. I have learnt that when it comes to the production server maintenance, it is very important to proactively monitor and optimize the BizTalk & SQL Server in a timely manner.
There are many numbers of articles available under the BizTalk Server optimization. Following are the summarizations of some key steps that you need to be followed.
Mainly, I am categorizing it as:

• Infrastructure Support
• BizTalk Application Server Maintenance
• BizTalk SQL Server Maintenance

Infrastructure Support

Having a good Infrastructure is very important due to its dependencies that are directly affecting your BizTalk Server performance.

• Have servers with very good performance by considering CPU, DISKS (SANs), Memory (> 8 GB), Sufficient Network bandwidth, Etc.
• Use 64-Bit Operating System and also BizTalk Server, SQL Server 64-Bit Enterprise Editions.
• There are some significant performance differences between 32 Bit & 64 Bit.
• For example the 64-bit virtual address space is exponentially larger than the 32-bit virtual address space. Moving to 64-bit will effectively resolve most of the BizTalk Memory Leaks issues.

• Your BizTalk Server should be implemented in a Multi Hosted environment at least with 2/3 Servers by dividing Send, Receive, Process Host Instances respectively.
• You should need to think about the I/O Contingency for heavy disk read/write processes and allocate separate disks such as SANs to isolate heavy I/O processes.
• Maintain separate Backup Servers.
• Customize your Anti Virus, File System Scan processes efficiently by excluding your BizTalk, SQL heavy read/write folders.
• Keep your O/S & SWs up-to-date.
• Implement monitors such as SCOM in order to proactively monitor the health of BizTalk Server and SQL Server.

BizTalk Application Server Maintenance

• Your BizTalk Server edition must be updated with latest Service Packs, Cumulative Updates & Patches.
• Verify the MSDTC setting and connectivity between your servers.
• DTC Ping Tool:- http://support.microsoft.com/kb/918331
• http://blogs.msdn.com/b/distributedservices/archive/2009/02/07/the-hidden-tool-msdtc-transaction-tracing.aspx
• http://support.microsoft.com/kb/899115
• http://technet.microsoft.com/en-us/library/cc733136(WS.10).aspx
• http://www.dotnetbase.co.uk/post/2011/09/02/Enabling-and-Reading-MSDTC-trace-files.aspx

• Create separate host instances & handlers for Send, Receive & Process. (If is it in the multi-hosted environment, create them respectively in separate servers). Also create a dedicated host for tracking purposes.
• Disable tracking for orchestrations, send ports, receive ports and pipelines as it not required.

Note: - Tracking may cause the potential performance overhead for BizTalk Server.

• Setup Performance Counters in the BizTalk Servers as required.(track memory, CPU, etc)
• Run BizTalkMsgBox Viewer Tool periodically and analyze the report to identify possible bottlenecks & suggestions.
• Download:- http://blogs.technet.com/b/jpierauc/archive/2007/12/18/msgboxviewer.aspx
• http://support.biztalk360.com/customer/portal/articles/141339-configuring-message-box-viewer

• Configure and monitor your BizTalk Host Throttling values properly.
• http://msdn.microsoft.com/en-us/library/aa559893.aspx
• http://msdn.microsoft.com/en-us/library/aa578302.aspx

• Make sure the Housekeeping processes/scripts are configured and scheduled properly in order to delete old files periodically.
• Set “Ordered Delivery” property true if you have a WCF Send port to gain significant performance.
• http://geekswithblogs.net/LeonidGaneline/archive/2012/01/11/biztalk-ordered-delivery.aspx
• http://msdn.microsoft.com/en-us/library/aa559637.aspx
• This may resolve transaction failures due to parallel execution of Orchestrations.
• Ex:- When your BizTalk WCF Send port is calling a WCF Service instance to update data to SQL the database.
• http://msdn.microsoft.com/en-us/library/dd203050(v=bts.10).aspx

BizTalk SQL Server Maintenance

Consider the following performance guidelines I gathered from Microsoft articles and I have added some additional details regarding configuring Microsoft SQL Server™ with BizTalk Server 2006:

• Whenever possible, use a fast disk subsystem with SQL Server. Use a redundant array of independent disks type 10 (RAID10/0+1) or a storage area network (SAN) with backup power supply.
• Isolate each MessageBox database on a separate server from the BizTalk Tracking database (BizTalkDTADb). For smaller deployments if CPU resources are available, it might be sufficient to isolate the MessageBox database on a separate physical disk from the BizTalk Tracking database.
• The primary MessageBox database could be the bottleneck due to CPU processor saturation or latency from disk operations (average disk queue length). If CPU processing is the bottleneck, add CPU processors to the primary MessageBbox. If not, try to disable publishing on the master MessageBox database. This way the master MessageBox database can more efficiently handle routing of messages to the other MessageBox databases
• If disk operations are the bottleneck, move the BizTalk Tracking database to a dedicated SQL Server computer and/or dedicated disk. If CPU processing and disk operations on the primary MessageBox database are not the bottleneck, you can create new MessageBox databases on the same SQL Server computer to leverage your existing hardware.
• Follow SQL Server best practices to isolate the transaction and data log files for the MessageBox and BizTalk Tracking databases onto separate physical disks.
• Allocate sufficient storage space for the data and log files. Otherwise SQL Server will automatically consume all of the available space on the disks where the log files are kept. The initial size of the log files depends on the specific requirements in your scenario. Estimate the average file size in your deployment based on testing results, and expand the storage space before implementing your solution.
• Allocate sufficient storage space for high-disk-use databases, such as the MessageBox, Health and Activity Tracking (HAT), and Business Activity Monitoring (BAM). If your solution uses the BizTalk Framework messaging protocol, allocate sufficient storage space for the BizTalk Configuration database (BizTalkMgmtDb).
• Configure BizTalk SQL Jobs properly. There are almost 12 SQL Jobs available for BizTalk. If they were not configured them properly or not running, there can be some severe BizTalk performance issues. Ex:- BizTalk Backup Jobs, DTA Purge and Archive Jobs
• http://support.microsoft.com/kb/919776

• Depending on business needs, such as data retention periods, and the volume of data processed in your scenario, configure the Archive/Purge jobs on the HAT-Tracking database such that the BizTalk Tracking database does not grow too large. The growth of this database can degrade performance because reaching the full capacity of the database imposes a limit on the rate of data insertion. This is especially true when especially when one BizTalk Tracking database supports multiple MessageBox databases.

Use following extensive values for the “DTA Purge and Archive (BizTalkDTADb)” job for better performance

declare @dtLastBackup datetime

set @dtLastBackup = GetUTCDate()

exec dtasp_PurgeTrackingDatabase 1, 0, 1, @dtLastBackup

• Scale up the servers hosting the MessageBox and BizTalk Tracking databases if they are the bottleneck. You can scale up the hardware by adding CPUs, adding memory, upgrading to faster CPUs, and using high-speed dedicated disks.
• Splitting the TempDB files across multiple files may resolve performance issues related to I/O operations. As a general guideline, create one file data file per processor and use the same size for all files created.
• Change the database auto-grow settings to a fixed value such as 100-150MB. By default the database growth is configured to 10%, which can lead to delays when growing larger databases.
• SQL Server memory should be set to a fixed value by setting both Min Server Memory and Max Server Memory to the same value. In general, allocate 75% of physical memory to SQL Server and leave 25% for the rest of the operating system and any applications. If this is a dedicated SQL Server, you can decrease the amount reserved for the operating system to a minimum of 1GB.

• Make sure the BizTalk Database backup jobs are configured properly.