Installing BizTalk on cloned Virtual Machines

Subtitled: The dangers of virtualization and how I stepped in deep DTC doo

I was doing a BizTalk installation this week.  I’ve done hundreds of them and they normally well.  There are almost always problems, but I’ve seen most of them so I know how to work around them.  Common issues are DTC or security related.  I’ve worked through tons of those.  The SQL install went well.  I was using Windows Server 2008 R2 Enterprise all around and SQL 2008 Server 2008 R2 Standard.  I was trying to install BizTalk 2010 Standard.  Both servers, SQL and BizTalk, were VMs.  When I tried to configure BizTalk to blew up on the BizTalk Group.  Digging into the error log I found this gem buried in the trace:

AdminLib GetBTSMessage: hrErr=80070002; Msg=The system cannot find the file specified.;

The install would also fail to delete the BizTalkMgmtDb, which was a hassle as I had to manually delete it every time. 

Now this isn’t my first rodeo so I already know MSDTC problems are extremely common in distributed BizTalk installations.  I broke out DTCPing and it passed fine.  I also saw this blog explaining exactly the same issue:

http://serverchronicle.blogspot.com/2011/02/adminlib-getbtsmessage-hrerr80070002.html

This too pointed to a DTC issue, but the DTCPing passed.  I tried all the possibilities listed in

Troubleshooting Problems with MSDTC but still had no luck.  Because I wasn’t using a domain admin account (a common request from customers) I had pre-provisioned all the users and groups.  I decided I’d break our subinacls.exe to try to explicitly grant Network Service the access needed for DTC.  To my surprise I received an access denied error when running this. 

I eventually decided try the install with a domain admin account.  That also failed, with the same error.  Now I was getting frustrated.  I reran DTCPing and noticed a warning in the output:

WARNING: the CID values for both test machines are the same

This told me a little something I had overlooked.  I asked the administrator who provisioned the boxes if they’d been sysprepped and if they were from the same image.  The answer to the second question was yes, the answer to the first was no; these machines had not been sysprepped.  I wasn’t sure how important this was because Mark Russinovich had pointed out three years ago that NewSID wasn’t really necessary anymore.  But now I had something concrete.  SIDs might be able to be duplicated, but I didn’t know about CIDs, which are Computer IDs. 

On the advice of another senior architect at West Monroe Partners I uninstalled MSDTC from the command line: msdtc –uninstall.  I also deleted most of the MSDTC registry keys I could find.  I then rebooted and reinstalled MSDTC.  Like magic, the configuration worked.  Amazing!

More messaging like Rabbits

After playing around some with RabbitMQ I decided I’d really let BizTalk message like rabbits.  I modified the file send port so that it sent back to the same queue instead of to the file location.  This creates a feedback loop so a message will keep propagating.  Then to make it more “rabbit like” I added a send port group that had the filter: BTS.ReceivePortName == RMQI_Q_Receive Or BTS.ReceivePortName == RMQI_Receive_Start

This way starting with the first message received from the file drop; every message receive results in two sends to the queue (which just feeds back and results in another doubling of messages). 

Two things became readily apparent.  BizTalk scales well and works fast and RabbitMQ will certainly keep pace.  In moments I had thousands of messages flying through my system. 

Checking the Queues section of the RabbitMQ management portal I was easily able to see the volume flowing through.  I almost think this could be a good benchmarking tool for BizTalk, but the BizTalk Benchmark Wizard is already so good I don’t really think I need a RabbitMQ based one. 

Messaging like Rabbits

I have been all kinds of excited about AMQP lately and yesterday’s news from David Ingham about AMQP on Service Bus got me pretty excited.  Obviously I’m always excited about BizTalk and my work lately has pushed me far heavier into the WCF-Adapters than I had previously gone (including using WCF 4.5 features from within BizTalk 2010). 

Eager to try some AMQP out and not willing to wait I decided I’d give RabbitMQ a try.  I knew they had a WCF binding for their implementation and decided it would probably be a good place to start.  This way I could placate my “open source” critics and also get some good AMQP fun going on Windows… quickly (I’m told AMQP works on Linux too). 

The setup was pretty easy for the server aspect, but the client part was not so much.  Leonid Ganeline’s awesome TechNet article on this helped me very much: http://social.technet.microsoft.com/wiki/contents/articles/7401.biztalk-and-rabbitmq.aspx.

I knew I had to GAC the assemblies and register the binding in machine.config, but I couldn’t find the assemblies anywhere (neither could Leonid), so I ended up building them locally (though you can strong name assemblies after they’re compiled I instead just added a key and strong named them).  I also changed the target to .NET 4.0.  You need to compile RabbitMQ.Client and RabbitMQ.ServiceModel.

I followed a very similar pattern with a file drop that Leonid did and it worked great.  I’m playing around with the graphical tools (management console) for RabbitMQ and I still don’t really know what many of these defaults are, but I do plan to work on it a bit more. 

Windows Azure 1.6/1.5 SDK Service Bus Authentication issue: 40102: Missing Authorization Token

I’ve been playing  a lot with the Azure 1.6 SDK and have been enjoying it quite a bit.  I’m in the process of modifying some of the examples in BizTalk 2010 Patterns to use Azure and ran into an interesting issue that seems to have only affected Service Bus.  I was trying to use queues as a destination and kept receiving this error:

The adapter failed to transmit message going to send port “OP_File_Send_CRM_SalesOrder” with URL “sb://XXXXXX.servicebus.windows.net/salesorders”. It will be retransmitted after the retry interval specified for this Send Port. Details:”System.UnauthorizedAccessException: 40102: Missing Authorization Token.TrackingId:8ac56df4-4094-495b-b302-ddf3c842efe7_6,TimeStamp:3/7/2012 6:38:19 PM —> System.ServiceModel.FaultException: 40102: Missing Authorization Token.TrackingId:8ac56df4-4094-495b-b302-ddf3c842efe7_6,TimeStamp:3/7/2012 6:38:19 PM

I assumed that I must have been using ACS incorrectly and decided to try the owner token.  Still no luck – though I did learn a lot about claims in ACS.  I also tried many permutations switching between transportClientEndpointBehavior settings for ClientCredentials and TokenProvider.

After far too long I remembered something from last week’s MVP Summit.  There had been discussion about versions of the SDK in use and the process of changing to another.  Another MVP said something about having to cut over.  I’m sure this is common knowledge to many (and probably in the SDK readme / docs), but I thought, well maybe I need to uninstall the old SDK.  Before I did this I thought I’d run the RelayConfigurationInstall and uninstall all the previous 1.5 bits that had been on my system (I vaguely remember running this ages ago when I installed 1.5).  I opened a command line to C:\Program Files\Windows Azure AppFabric SDK\V1.5\Assemblies and ran:

RelayConfigurationInstaller.exe /u

I then uninstalled the 1.5 SDK.  Finally I ran the RelayConfigurationInstall from the 1.6 SDK (located at C:\Program Files\Windows Azure SDK\v1.6\ServiceBus\bin).

RelayConfigurationInstaller.exe /i

Amazingly everything worked after that with no other changes.  I was set!

Favorite “New BizTalk Feature” and yet another reason BizTalk rocks

As most know by now the Visual Studio 11 Beta came out last week.  I have been eagerly trying out new features since I installed it at the MVP Summit.  The WCF features had me the most excited so far.  I have been discussing using UDP from BizTalk (via WCF) for some time with friends and coworkers.

I know there has been a “guidance” on this for some time, but many clients don’t like a guidance, they want something supported and in their platform.  Today my friend and coworker Madhav tied it all together.  He blogged about it here:

http://maddcoder.wordpress.com/2012/03/05/udp-broadcast-using-biztalk-server-2010-19/

It’s a great post and it opens a new world of opportunities for many WCF (and BizTalk) developers).  Most impressively to me is that BizTalk 2010 just works with the new bindings that are a part of WCF 4.5.  I think this is a great testament to the platform and the decision five years ago to move towards WCF for adapters.  It was awesome to see this today!

Another class for BizTalk 2010 Patterns

I just taught chapters 4-8 to a client and had a great time doing it.  I did find some minor corrections with the book, mostly namespaces, but I will have a write up soon.  The class went fairly well considering I let everyone use their own laptop (a change from my normal strategy of supplying a VM).  This did cause some issues, which is prompting me to create an installation and preparation guide, but it worked out fairly well. 

Thank you to all who attended and as always, feedback is welcome!