Is Amazon DynamoDB the look of the future?

Amazon is now offering a key/value data store that relies on Solid state disc for storage. DynamoDB is the name, and it is intended to complement S3 as a lower-latency store. It’s higher cost, but offers better performance for those customers that need it.

Two things on this.

  1. The HighScalability blog calls Amazon’s use of SSD as a “radical step.”  That view may become antiquated rapidly.The one outlier in the datacenter of today is the use of spinning mechanical platters as a basis to store data. Think about that. There’s one kind of moving part in a datacenter – the disk.  It consumes much of the power and causes much of the heat. We will see SSD replace mag disk as a mainstream storage technology, sooner than most of us think.  Companies like Pure Storage will lead the way but EMC and the other big guys are not waiting to get beaten.Depending on manufacturing ramp-up, this could happen in 3-5 years. It won’t be radical. The presence of spinning platters in a datacenter will be quaint in 8 years.
  2. The exposure of the underlying storage mechanism to the developer is a distraction.  I don’t want to have to choose my data programming model based on my latency requirements.  I don’t want to know, or care, that the storage relies on an SSD. That Amazon exposes it today is a temporary condition, I think. The use of the lower-latency store ought to be dynamically determined by the cloud platform itself, based on provisioning configuration provided by the application owner.  Amazon will just fold this into its generic store. Other cloud platform providers will follow.The flexibility and intelligence allowed in provisioning the store is the kind of thing that will provide the key differentiation among cloud platforms.

What does it mean? Microsoft upping efforts on Infrastructure-as-a-service

Wired is reporting a rumor that Microsoft will soon launch a new Infrastructure-as-a-service offering to compete with Amazon EC2, in June.

What Does it Mean?

I have no idea whether the “rumor” is true, or even what it really means. I speculate that the bottom line is that we’ll be able to upload arbitrary VHDs to Azure. Right now Microsoft allows people to upload VHDs that run Windows Server 2008.  With this change they may support “anything”.  Because it’s a virtual hard drive, and the creator of that hard drive has full control over what goes into it, that means an Azure customer will be able to provision VMs in the Microsoft cloud that run any OS, including Linux. This would also represent a departure from the stateless model that Windows Azure currently supports for the VM role. It means that VHDs running in the Windows Azure cloud will be able to save local state across stop/restart.

Should we be Surprised?

Is this revolutionary?  Windows Azure already offers compute nodes; it’s beta today but it’s there, and billable.  So there is some degree of Infrastructure-as-a-service capability today.

For my purposes “infrastructure as a service”  implies raw compute and storage, which is something like Amazon’s EC2 and S3. A “platform as a service” walks up the stack a little, and offers some additional facilities for use in applications. This might include application management and monitoring, enhancements to the storage model or service, messaging, access control, and so on. All of those are general-purpose things, usable in a large variety of applications, and we’d say they are “higher level” than storage and compute. In fact those services are built upon the compute+storage baseline.

For generations in the software business, Microsoft has been a major provider of platforms. With its launch in 1990, Windows itself was arguably the first broadly adopted “application platform”.  Since the early 90’s, specialization and evolution have resulted in an proliferation of platforms in the industry – we have client platforms, server platforms (expanding to include the Hypervisor), web platforms (IIS+ASP.NET, Apache+PHP), data platforms, mobile platforms and so on. And beyond app platforms, since Dynamics Microsoft has also beein in the business of offering applications as well, and it’s here we see the fractal nature of the space.  The applications can act as platforms for a particular set of extensions.  In any case, it’s clear that Microsoft has offerings in all those spaces, and more.

Beneath the applications like Dynamics, and beneath the traditional application platforms like Windows + SQL Server + IIS + .NET, Microsoft has continued to deliver the foundational infrastructure, specifically to enable other alternative platforms. Oracle RDBMS and Tomcat running on Windows is a great example of what I mean here. Sure, Microsoft would like to entice customers to adopt the entirety of their higher-level platforms, but the company is willing to make money by supplying lower-level infrastructure for alternative platforms.

Considering that history, the rumor that Microsoft is “upping efforts on infrastructure as a service” should not be surprising.  Microsoft has long provided offerings at many levels of “the stack”.  I expect that customers have clearly told Microsoft they want to run VHDs, just like on EC2, and Microsoft is responding to that.  Not everyone will want this; most people who want this will also want higher-level services.  I still believe strongly in the value of higher-level cloud-based platforms.

Platform differentiation in the Age of Clouds

It used to be that differentiation in server platforms was dominated by the hardware. There were real, though fluctuating and short-lived, performance differences between Sun’s Sparc, HP’s PA-RISC, IBM’s RIOS and Intel’s x86. But for the moment, the industry has found an answer to the hardware question; servers use x64.

With standard high volume servers, the next dominant factor for differentiation was on the application programming model.  We had a parade of players like CORBA, COM, Java, EJB, J2EE, .NET. More recently we have PHP, node.js, Ruby, and Python. The competition in that space has not settled on a single, decisive winner, and in my judgment, that is not likely to happen. Multiple viable options will remain, and the options that enjoy relatively more success do share some common attributes: ease of programming (eg, building an ASPNET service or a PHP page) is favored over raw performance (building an ISAPI or an Apache module in C/C++).  Also, flexibility of the model (JSP/Tomcat/RESTlets) is favored over more heavily prescriptive metaphors (J2EE). I expect the many options in server platform space to continue; the low-cost to develop and extend these platform alternatives means there is no natural economic value of convergence, as there was in server hardware where the R&D costs are relatively high.

Every option in the space will offer its own unique combination of strengths, and enterprises will choose among them. One company might prefer strong support for running REST services, while another might prefer the application metaphor  of Ruby on Rails.  Competition will continue.

But programmer-oriented features will not be the key differentiator in the world of cloud-hosted platforms. Instead, I expect to see operational and deployment issues to dominate.

  • How Reliable is the service?
  • How difficult is it to provision a new batch of servers?
  • How flexible is the hosting model? Sometimes I want raw VMs, sometimes I want higher-level abstractions. I might want to manage a “farm” of servers at a time, or even better, I might want to manage my application without regard for how many VMs back it.
  • How extensive are the complementary services, like access control, messaging, data, analysis, and so on.
  • What kind of operational data do I get out of that farm of servers? I want to see usage statistics and patterns of user activity.

It won’t be ease of development that wins the day.

Amazon has been very disruptive with its AWS, and Microsoft is warming to the competition. This is all good news for the industry. It means more choices, better options, and lower costs, all of which promotes innovation.