BizTalk Summit 2014 London

This week I had the pleasure to speak at the BizTalk Summit 2013 in London which was sponsored by BizTalk 360.  I have to say it was the best BizTalk event I’ve ever been to.  My presentation is posted here but the slides aren’t much without the presenter… or the videos showing how I implement BAM on an order processing solution with zero code – fortunately there is a video to be posted by BizTalk 360 if you’re interested in seeing the message.  The sample solution I use, with all artifacts, is located at https://danrosanova.wordpress.com/rwb.

The talk was about Real World Business Activity Monitoring and the message was well received.  In short I drive home that we owe it to our customers (i.e. business people) to provide BAM so they can see what’s happening in the way they are comfortable – which is normally Excel or Reporting Services.  I took away two strong points from giving this talk.

First – more BizTalk shops use BAM than I had ever imagined.  Its use is fairly limited in the US, sadly, and some markets, but clearly many people use it.  About half the audience said they use it in production – I expected 1-2%.  This is encouraging as BAM is really worth doing.  It’s easy to implement and delivers high value at both technical and business levels.

Second I learned that there is still a lot of interest in building these sorts of dashboards and that many people were eager to give BAM a try given how easy my presentation makes it look.  There is also a lot of interest in BizTalk – which I was really glad to see.  It is a great platform and used correctly has a strong place in many organizations.  I will be blogging a lot about BAM in the coming weeks and about Big Data, which brings me to my final take away.

Big Data is a big topic and it’s in the news a lot.  My next four presentations all focus on Big Data and Advanced Analytics and I’m going to dedicate some time to bringing these together with BizTalk and with BAM.  My next talk at the Informs Conference on Business Analytics and Operations Research on April 1 in Boston is focused on Situational Awareness with Big Data tools.  It is very much like BAM with Big Data with even more dimensions.

Advertisements

Configuring the AS2 Runtime on BizTalk 2013 – one small caveat

I’ve been doing a lot of BizTalk lately, even by my standards, and a lot of BizTalk installations as well.  I’ve blogged and written (in BizTalk 2010 Patterns) about how important it is to change the default configurations that come out of the BizTalk Configuration Wizard.  The most important item is to create separate BizTalk hosts for processing.  I normally do at least four: send, receive, processing (orchestration), and tracking.  In keeping with this I also create handlers for all the adapters for the send and receive hosts.  I also remove the handlers for the default BizTalkServerApplication host. 

 

This has led me a small but important detail.  Although BizTalk easily accommodates configuring further features later (like EDI or BAM), I did run into one issue.  The AS2 Runtime configuration creates a SQL receive location (Classic SQL not the WCF kind) and it expects, nay demands, the default host to have a receive handler for the SQL Adapter.  This wasn’t that hard to track down, but could cause headaches for others so I thought I would share. 

Installing BizTalk on cloned Virtual Machines

Subtitled: The dangers of virtualization and how I stepped in deep DTC doo

I was doing a BizTalk installation this week.  I’ve done hundreds of them and they normally well.  There are almost always problems, but I’ve seen most of them so I know how to work around them.  Common issues are DTC or security related.  I’ve worked through tons of those.  The SQL install went well.  I was using Windows Server 2008 R2 Enterprise all around and SQL 2008 Server 2008 R2 Standard.  I was trying to install BizTalk 2010 Standard.  Both servers, SQL and BizTalk, were VMs.  When I tried to configure BizTalk to blew up on the BizTalk Group.  Digging into the error log I found this gem buried in the trace:

AdminLib GetBTSMessage: hrErr=80070002; Msg=The system cannot find the file specified.;

The install would also fail to delete the BizTalkMgmtDb, which was a hassle as I had to manually delete it every time. 

Now this isn’t my first rodeo so I already know MSDTC problems are extremely common in distributed BizTalk installations.  I broke out DTCPing and it passed fine.  I also saw this blog explaining exactly the same issue:

http://serverchronicle.blogspot.com/2011/02/adminlib-getbtsmessage-hrerr80070002.html

This too pointed to a DTC issue, but the DTCPing passed.  I tried all the possibilities listed in

Troubleshooting Problems with MSDTC but still had no luck.  Because I wasn’t using a domain admin account (a common request from customers) I had pre-provisioned all the users and groups.  I decided I’d break our subinacls.exe to try to explicitly grant Network Service the access needed for DTC.  To my surprise I received an access denied error when running this. 

I eventually decided try the install with a domain admin account.  That also failed, with the same error.  Now I was getting frustrated.  I reran DTCPing and noticed a warning in the output:

WARNING: the CID values for both test machines are the same

This told me a little something I had overlooked.  I asked the administrator who provisioned the boxes if they’d been sysprepped and if they were from the same image.  The answer to the second question was yes, the answer to the first was no; these machines had not been sysprepped.  I wasn’t sure how important this was because Mark Russinovich had pointed out three years ago that NewSID wasn’t really necessary anymore.  But now I had something concrete.  SIDs might be able to be duplicated, but I didn’t know about CIDs, which are Computer IDs. 

On the advice of another senior architect at West Monroe Partners I uninstalled MSDTC from the command line: msdtc –uninstall.  I also deleted most of the MSDTC registry keys I could find.  I then rebooted and reinstalled MSDTC.  Like magic, the configuration worked.  Amazing!

More messaging like Rabbits

After playing around some with RabbitMQ I decided I’d really let BizTalk message like rabbits.  I modified the file send port so that it sent back to the same queue instead of to the file location.  This creates a feedback loop so a message will keep propagating.  Then to make it more “rabbit like” I added a send port group that had the filter: BTS.ReceivePortName == RMQI_Q_Receive Or BTS.ReceivePortName == RMQI_Receive_Start

This way starting with the first message received from the file drop; every message receive results in two sends to the queue (which just feeds back and results in another doubling of messages). 

Two things became readily apparent.  BizTalk scales well and works fast and RabbitMQ will certainly keep pace.  In moments I had thousands of messages flying through my system. 

Checking the Queues section of the RabbitMQ management portal I was easily able to see the volume flowing through.  I almost think this could be a good benchmarking tool for BizTalk, but the BizTalk Benchmark Wizard is already so good I don’t really think I need a RabbitMQ based one. 

Messaging like Rabbits

I have been all kinds of excited about AMQP lately and yesterday’s news from David Ingham about AMQP on Service Bus got me pretty excited.  Obviously I’m always excited about BizTalk and my work lately has pushed me far heavier into the WCF-Adapters than I had previously gone (including using WCF 4.5 features from within BizTalk 2010). 

Eager to try some AMQP out and not willing to wait I decided I’d give RabbitMQ a try.  I knew they had a WCF binding for their implementation and decided it would probably be a good place to start.  This way I could placate my “open source” critics and also get some good AMQP fun going on Windows… quickly (I’m told AMQP works on Linux too). 

The setup was pretty easy for the server aspect, but the client part was not so much.  Leonid Ganeline’s awesome TechNet article on this helped me very much: http://social.technet.microsoft.com/wiki/contents/articles/7401.biztalk-and-rabbitmq.aspx.

I knew I had to GAC the assemblies and register the binding in machine.config, but I couldn’t find the assemblies anywhere (neither could Leonid), so I ended up building them locally (though you can strong name assemblies after they’re compiled I instead just added a key and strong named them).  I also changed the target to .NET 4.0.  You need to compile RabbitMQ.Client and RabbitMQ.ServiceModel.

I followed a very similar pattern with a file drop that Leonid did and it worked great.  I’m playing around with the graphical tools (management console) for RabbitMQ and I still don’t really know what many of these defaults are, but I do plan to work on it a bit more. 

Windows Azure 1.6/1.5 SDK Service Bus Authentication issue: 40102: Missing Authorization Token

I’ve been playing  a lot with the Azure 1.6 SDK and have been enjoying it quite a bit.  I’m in the process of modifying some of the examples in BizTalk 2010 Patterns to use Azure and ran into an interesting issue that seems to have only affected Service Bus.  I was trying to use queues as a destination and kept receiving this error:

The adapter failed to transmit message going to send port “OP_File_Send_CRM_SalesOrder” with URL “sb://XXXXXX.servicebus.windows.net/salesorders”. It will be retransmitted after the retry interval specified for this Send Port. Details:”System.UnauthorizedAccessException: 40102: Missing Authorization Token.TrackingId:8ac56df4-4094-495b-b302-ddf3c842efe7_6,TimeStamp:3/7/2012 6:38:19 PM —> System.ServiceModel.FaultException: 40102: Missing Authorization Token.TrackingId:8ac56df4-4094-495b-b302-ddf3c842efe7_6,TimeStamp:3/7/2012 6:38:19 PM

I assumed that I must have been using ACS incorrectly and decided to try the owner token.  Still no luck – though I did learn a lot about claims in ACS.  I also tried many permutations switching between transportClientEndpointBehavior settings for ClientCredentials and TokenProvider.

After far too long I remembered something from last week’s MVP Summit.  There had been discussion about versions of the SDK in use and the process of changing to another.  Another MVP said something about having to cut over.  I’m sure this is common knowledge to many (and probably in the SDK readme / docs), but I thought, well maybe I need to uninstall the old SDK.  Before I did this I thought I’d run the RelayConfigurationInstall and uninstall all the previous 1.5 bits that had been on my system (I vaguely remember running this ages ago when I installed 1.5).  I opened a command line to C:\Program Files\Windows Azure AppFabric SDK\V1.5\Assemblies and ran:

RelayConfigurationInstaller.exe /u

I then uninstalled the 1.5 SDK.  Finally I ran the RelayConfigurationInstall from the 1.6 SDK (located at C:\Program Files\Windows Azure SDK\v1.6\ServiceBus\bin).

RelayConfigurationInstaller.exe /i

Amazingly everything worked after that with no other changes.  I was set!

Favorite “New BizTalk Feature” and yet another reason BizTalk rocks

As most know by now the Visual Studio 11 Beta came out last week.  I have been eagerly trying out new features since I installed it at the MVP Summit.  The WCF features had me the most excited so far.  I have been discussing using UDP from BizTalk (via WCF) for some time with friends and coworkers.

I know there has been a “guidance” on this for some time, but many clients don’t like a guidance, they want something supported and in their platform.  Today my friend and coworker Madhav tied it all together.  He blogged about it here:

http://maddcoder.wordpress.com/2012/03/05/udp-broadcast-using-biztalk-server-2010-19/

It’s a great post and it opens a new world of opportunities for many WCF (and BizTalk) developers).  Most impressively to me is that BizTalk 2010 just works with the new bindings that are a part of WCF 4.5.  I think this is a great testament to the platform and the decision five years ago to move towards WCF for adapters.  It was awesome to see this today!