It Runs Hourly…

These may be the most damning three words in the contemporary technology landscape.  They frustrate and aggravate me every time I run into them.  Why do I find these words so offensive?  It’s because they harken back to a dark time in computing’s past where mainframes ruled the landscape and fast was overnight.  In this dark past software and business processes were subservient to technology. 

Sadly despite the fact that the technological reason for these types of designs has completely disappeared they are still rampant in many organizations.  The barrier is no longer technology it is imagination; or a lack thereof.  Many people who came of age in those darker computing times still see the world as one big batch waiting to be processed.  Perhaps it is my deep experiences in lean environments and teams, but to me waiting equates to waste and waste should be eliminated. 

It goes beyond that though.  The batch is actually a very complicated construct.  It does not work the way our business processes likely would have; there is no single piece flow.  It’s a batch.  From a processing standpoint there are much deeper issues such as conflict resolution, duplicate detection, and the header and footer overhead implicit in the batch.  Worse yet, the batch is often single threaded.  Thanks to advances in parallel computing (and cloud computing) the batch is no longer the high volume champion it used to be. 

The real world, the one we all live and work in, is not batch based.  It is real-time and it is event based.  If the fire alarm goes off I leave the building I don’t wait for the fire department to come gather everyone.  I buy things online or in stores as I’m moving I don’t just note them and submit a request for everything I’d like to buy at the end of the day.  The hourly process is an attempt to turn the nightly batch, the workhorse of the 1970s, into something more modern.  It’s every bit as complicated and by today’s standards not much faster.  Would you want your credit card authorization to take up to an hour?  No.  How about your lunch order?  Absolutely not. 

Lately I have run into quite a few situations with BizTalk clients where the scheduled task adapter has been used extensively.  I’ve always found it interesting that people want to take a real-time event based platform and turn it into a batch processor.  Some I think don’t realize a service window can often perform the same thing more reliably.  I believe there is a reason that a scheduled task adapter was never included with BizTalk.  Perhaps it’s designers had the same aversion to batch processing that I do. 

It is time to break free of the batch – join the real-time revolution and model your systems and services after the world we live in today – not the mainframe from yesterday.


About danrosanova
I am a Principal Program Manager for Messaging at Microsoft and product owner for Azure Messaging: Service Bus, Relay, and Event Hubs. I have a long history in distributed computing on a variety of platforms and have focused on large scale messaging and middleware implementations from inception to implementation. I was a five time Microsoft MVP before joining Microsoft and author of the book Microsoft BizTalk Server 2010 Patterns.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: