Apache Storm on Windows

In a release in February Apache Storm community added Windows platform support for Storm 0.9.1

I for one have been very excited to see this.  The Hortonworks distribution of Hadoop (HDP) is the only one that runs on both Windows and Linux and this gives a lot more choice to traditional enterprise clients.  I’ve been working with HDP for about a year and a half now and really like the experience – both on Linux and Windows. 

Storm is a very exciting development in real time data processing using a Hadoop cluster.  This is useful for running models that you’ve created by more traditional batch processing and map reduce within Hadoop.  Storm uses a simple spout and bolt topology for processing tuples of information at scale and in real time.  More information can be found at the storm site: http://storm.incubator.apache.org/

I am now wondering if this technology, now running on Windows, will make it into the Windows Azure HDInsight service.  I certainly don’t have any inside information on this, but I’d be interested to see it. 

Windows Azure 1.6/1.5 SDK Service Bus Authentication issue: 40102: Missing Authorization Token

I’ve been playing  a lot with the Azure 1.6 SDK and have been enjoying it quite a bit.  I’m in the process of modifying some of the examples in BizTalk 2010 Patterns to use Azure and ran into an interesting issue that seems to have only affected Service Bus.  I was trying to use queues as a destination and kept receiving this error:

The adapter failed to transmit message going to send port “OP_File_Send_CRM_SalesOrder” with URL “sb://XXXXXX.servicebus.windows.net/salesorders”. It will be retransmitted after the retry interval specified for this Send Port. Details:”System.UnauthorizedAccessException: 40102: Missing Authorization Token.TrackingId:8ac56df4-4094-495b-b302-ddf3c842efe7_6,TimeStamp:3/7/2012 6:38:19 PM —> System.ServiceModel.FaultException: 40102: Missing Authorization Token.TrackingId:8ac56df4-4094-495b-b302-ddf3c842efe7_6,TimeStamp:3/7/2012 6:38:19 PM

I assumed that I must have been using ACS incorrectly and decided to try the owner token.  Still no luck – though I did learn a lot about claims in ACS.  I also tried many permutations switching between transportClientEndpointBehavior settings for ClientCredentials and TokenProvider.

After far too long I remembered something from last week’s MVP Summit.  There had been discussion about versions of the SDK in use and the process of changing to another.  Another MVP said something about having to cut over.  I’m sure this is common knowledge to many (and probably in the SDK readme / docs), but I thought, well maybe I need to uninstall the old SDK.  Before I did this I thought I’d run the RelayConfigurationInstall and uninstall all the previous 1.5 bits that had been on my system (I vaguely remember running this ages ago when I installed 1.5).  I opened a command line to C:\Program Files\Windows Azure AppFabric SDK\V1.5\Assemblies and ran:

RelayConfigurationInstaller.exe /u

I then uninstalled the 1.5 SDK.  Finally I ran the RelayConfigurationInstall from the 1.6 SDK (located at C:\Program Files\Windows Azure SDK\v1.6\ServiceBus\bin).

RelayConfigurationInstaller.exe /i

Amazingly everything worked after that with no other changes.  I was set!