The Holy Grail of Scalability

You will hear this from all cloud hosting providers these days, all the time. Scalable, scale on demand, burstable. But what does that really mean and do you really need it? The answers are less obvious than you could think. Of course it does sound cool, but it may be a good idea to look under the hood and check what is inside, and who can benefit from it, and why.

The cloud is basically a way to glue together a farm of servers, even a datacenter or even many datacenters into something which looks and acts as a single machine, or single server you can access via one domain in your web browser. It is glued together using some advanced software, network protocols, fast connections and other magic we will not even try to understand here, but the result is that it can be seen as something which smart people called a cloud.

When there are no visible barriers, like those you normally see when you are looking at your desktop computer, your phone and your laptop, when it is everything like just one device powering your website, you don’t need to worry if your site will be able to handle huge traffic when it will happen, because you know that there are so many computers glued together behind the scenes, ready to serve you, so they will handle it, no problem.

Now, stop for a moment and look at this cool thing called cloud, again. Do you need to support millions of users, like, say, Amazon online store, or iCloud service? Or maybe you have at least a few thousands of users who are using your online storage or backup service? Or maybe you are running some very popular forum for gamers or some blogging platform with thousands of active blogs? Also, can we safely assume that you don’t plan to start a next Twitter or Gmail clone, at least not later this week?

The point is that most people look for cloud hosting only because others are good in selling you something you don’t need and will never be able to use. The point is that the cloud is a way to create and manage services which by definition must escape the boundaries of single machine to work, they need to scale easily to exist. The point is that the cloud is a cool solution for your host – not necessarily for you. It is not you who benefit from it the most, no matter how hard you will be persuaded that it is you and your web site who will benefit. If the cloud works for you and your website or service must use the cloud infrastructure to function, then you are not really a cloud user, you are a provider of cloud based service. This is really not the same thing.

But the other part of this story is that while in reality, near 99% of Drupal sites will never need to escape the boundaries of single machine, the cloud allows hosting providers to offer you a virtual units of resources, with ability to add more units on demand, so theoretically you can scale the resources and your budget up and down, to match your real needs, so you don’t need to worry if that small VPS you have used so far will not crash under the unexpected load, and on the other side, you don’t need to worry that you are paying far too much for that big dedicated machine while using its power maybe twice a year, during events specific to your audience.

However, the size of the site, its audience and expected traffic is not the only reason companies do the cloud based hosting. Look at Amazon EC2 and S3 service, for example. Their main goal is not the high performance, it is an infinite scalability at as low cost as possible. The low cost is the key, and the reason why Amazon has built its cloud – using their own words – “from inexpensive commodity hardware components”. They don’t use high-end servers you can find elsewhere, because then they couldn’t meet the low price requirement. And because they use mostly cheap stuff, they must use technology to accomodate the fact that the cheap components will fail and will fail often. So, a bit ironically, they must use High Availability (part of that magical glue we have mentioned earlier) not only to guarantee scalability for your site. They need the redundancy not only to use the redundancy as a selling point. They need it to protect themselves from the high risk of hardware failures. But of course they will sell you this as an affordable and scalable, cool service everyone wants.

So, let’s admit it – the scalable cloud hosting offers you some really, really nice comfort – only the power you need, at as low cost as possible, when you need it, while leaving all the hard work on the cloud provider side. Sounds perfect, right?

However, does it need to be the cloud, with its magical glue inside we will not even try to understand, to offer you this level of comfort? Does it need to be so complex and fragile behind the scenes, with its famous dominos effects, SAN mysteriously lost connections, not so isolated noisy neighbors and other esoterics?

We will try to look through the marketing buzzwords and discover the real stuff under the hood. It is not to suggest what you need or not, it is to question popular beliefs, so you could find and use the tools you really need and/or prefer. The perfect Holy Grail of Scalability, perfect for you.

Acquia.comGetPantheon.comOmega8.cc

!We post new articles daily (Monday-Friday), so you may want to subscribe to this feed
in your favourite RSS reader to follow up and post your comments on Twitter or App.net.
Core Conceptions « Previous | Next » The Speed Matters