It Runs Hourly…

These may be the most damning three words in the contemporary technology landscape.  They frustrate and aggravate me every time I run into them.  Why do I find these words so offensive?  It’s because they harken back to a dark time in computing’s past where mainframes ruled the landscape and fast was overnight.  In this dark past software and business processes were subservient to technology. 

Sadly despite the fact that the technological reason for these types of designs has completely disappeared they are still rampant in many organizations.  The barrier is no longer technology it is imagination; or a lack thereof.  Many people who came of age in those darker computing times still see the world as one big batch waiting to be processed.  Perhaps it is my deep experiences in lean environments and teams, but to me waiting equates to waste and waste should be eliminated. 

It goes beyond that though.  The batch is actually a very complicated construct.  It does not work the way our business processes likely would have; there is no single piece flow.  It’s a batch.  From a processing standpoint there are much deeper issues such as conflict resolution, duplicate detection, and the header and footer overhead implicit in the batch.  Worse yet, the batch is often single threaded.  Thanks to advances in parallel computing (and cloud computing) the batch is no longer the high volume champion it used to be. 

The real world, the one we all live and work in, is not batch based.  It is real-time and it is event based.  If the fire alarm goes off I leave the building I don’t wait for the fire department to come gather everyone.  I buy things online or in stores as I’m moving I don’t just note them and submit a request for everything I’d like to buy at the end of the day.  The hourly process is an attempt to turn the nightly batch, the workhorse of the 1970s, into something more modern.  It’s every bit as complicated and by today’s standards not much faster.  Would you want your credit card authorization to take up to an hour?  No.  How about your lunch order?  Absolutely not. 

Lately I have run into quite a few situations with BizTalk clients where the scheduled task adapter has been used extensively.  I’ve always found it interesting that people want to take a real-time event based platform and turn it into a batch processor.  Some I think don’t realize a service window can often perform the same thing more reliably.  I believe there is a reason that a scheduled task adapter was never included with BizTalk.  Perhaps it’s designers had the same aversion to batch processing that I do. 

It is time to break free of the batch – join the real-time revolution and model your systems and services after the world we live in today – not the mainframe from yesterday.


Three weeks until Service Technology Symposium

Three weeks from today I will deliver the presentation High Performance Computing in the Cloud at the Service Technology Symposium in London, UK.  The talk will provide a brief into to HPC discuss different flavors of HPC.  Then cover current cloud based HPC options and experiences.  Although the presentation is vendor neutral specific examples on the two largest cloud platforms will be discussed to provide pricing and experience details.  HPC has never been so accessible and I hope you will join me to see how it can help any organization.