Our good friends at Amazon arrive on the scene. First they offered a stream of services which were interesting, but never really got your attention. But this last announcement. That one changes everything.

And it does change everything. Unfortunately people are thinking of the service in terms of their current ideas of scaling infrastructure. And I have a sneaking suspicion that it doesn’t work that way

The most obvious difference is bandwidth. You pay a definite premium on bandwidth when you use amazons web services, and EC2 is no exception. However one more sinister, sneaky, and diabolical quirk lies in the way. You see bandwidth is something that you may well be willing to pay a premium form. And its also mostly circumventable with judicious use of static content hosting, compression, caching, and the like. But permanence is something that most people take for granted. And permanence is exactly what EC2 lacks. This is the caveat which will start to change how people think about operational resources. And probably for the better.

Before I go into my current theories regarding the overall usage of Amazons web services to their fullest potential… Which will be part 3… Lets talk about how Amazon views all the things you’ve been taking for granted differently.

First let me state how the normal data-center works: It works via units. Most OPS departments consider the data-center operational unit to be one server. But most data-centers never reach the size and complexity of those attained by Amazon, Google, Etc. And they have to think outside several very limiting boxes. Consider the planet as an analogy. It wasn’t too long ago, in geological time frames, when the idea of running out of space (or any other resource) was seemingly absurd. However fast forward a thousand or two years to the present day, and we’re looking to tackle some very difficult problems head on: dealing with finite resources and a growing population. Just as its a certainty that, eventually, man will outgrow earth. So will any company outgrow their data-center in two or three distinct phases

Phase 1: We just need servers. At this point it’s basically a just get *something* game. You have *very* limited resources… and you get whatever you can. Right now is NOT the time to be worrying about how t spend money, but how to make it!

Phase 2: After several iterations of Phase 1… The diversity of our data-center is just unmanageable! At this point you realize that you need uniformity in the data-center to keep scaling… and hopefully you can afford it. Start phasing out the old patchwork network of randomly configured machines with shiny new identical machines. Enter a state of bliss for a good while

Phase 3: We’ve just realized that we cannot depend on unified hardware for that long. Our supply of this configuration of machine just ran out! The idea of throwing out 200 servers so that you can buy 500 more (for a gain of 300 servers) for the sake of this idea of uniformity makes you sick. And purchasing a thousand so that they can sit in a warehouse for use later is ridiculous as well… You’ll be running on 2 year old new servers in 2 years! Thats daft! There has to be a better way!

And there is. At this point the big boys stop thinking of a data-center in terms of machines. Break the machine into smaller units. Processor gigahertz, memory megabytes, processor gigabytes. Now lump all those together into a stack. And you have ?Ghz + ?Mb + ?Gb distributed throughout your data-center. But you still don’t have the complete solution. The last piece falls into place with virtualization. What this last piece of the puzzle allows you to do is to divvy out those individual resources as you see fit, and on the fly.

And thats what Amazon has done. They’ve pooled their processor power into one giant stack. They’ve pooled their memory into another. Their hard disk space into another. And they found that in this way they can hand out slices of those stacks on demand. And that they can also reclaim those slices on demand. So if you want to take advantage of the infrastructure that amazon is providing: stop thinking that a machine is a specific thing and only that thing

You’re in the world of on-demand. Lets start thinking in terms of what that means and how to use it to your advantage!

And I’ll begin that thought process in part 3