Author: apokalyptik

apokalyptikhttp://blog.apokalyptik.com/

All this talk about Amazon EC2 and bandwidth

Let me share some perspective on what bandwidth numbers actually mean… Because talking about 1000Gb is a term that doesnt actually connect with anything in people minds

1000Gb/month means that on a 31 day month you are sending out (numbers rated as 1000 equals 1k because thats prevalent)

  • 373Kbytes per second…
  • 22Mbytes per minute
  • 1.3Gbytes per hour
  • 32Gbytes per day

or the ability to

  • broadcast a 256bkit audio stream to 11.5 users around the clock
    (remember 8 bits == 1 byte)
  • send out over 1,500 CDROM images
  • send out over 200 DVD images (single sided)
  • send out almost 286,000 MP3’s (3.5Mb each)

In otherwords… its a lot of bandwidth… assuming i’m not so tired that my numbers are flawed (which is very possible)

so… at EC2 to send out 5KB/sec for the same month would cost $2.68. 50KB @ $26.80. 500KB @ … well you get the idea.

One last comparison before I head off to bed… At EC2, for $160/month (comparable to a mid-level hosting provider), you get about 365Gb xfer per month… or 136KB/sec for the entire month (or 4 256Kbit audio streams, 547 CDs, 54 DVDs, 101,714 mp3’s, or one 365,000,000 letter text file ;))

5 Excellent Uses for Amazon EC2

The Test/Development Environment

One of those things which is always difficult is putting together the right test environment. Traditionally purchasing a seperate set of hardware, , a seperate datacenter, A developers “local copy,” or something as cheesy as different vhosts on the same machine have been the only real answers. Most of those are grossly inneficient as well as expensive. At the best they’re “ok” (because you arent going to put any *real* power in this arena when you have mission critical applications to deploy) and at worst they’re downright dangerous (how many people do you know have never accidentally edited the wrong file in the wrong folder?)

Enter Amazon, wisking you away to a magical place in which its possible to throw as much horsepower as neccesary at a test environment. Where its possible to wipe and reload the entire thing in a *very* short period of time. And where you only have to pay for it when it’s being used. With Amazon EC2 there is no longer any reason that a web app company cannot afford a truly useful test/development environment

Offsite Backups

Do you worry that something is going to happen to your data? Is the only backup copy of your working code on one of those CDR’s sitting in the “server closet”… was it taken more than 3 months ago? If you answered yes to any of these questions Amazon EC2 is right up your alley. With a comple of public keys, and rsync you can put the power of secure backups to work for you! But wait, theres more! Once the backup has completed schedule a shutdown command, and only pay for what you need! Pretend you have 100Gb of data to backup… it changes at 5% per dat, and takes only 4 hours to run the backup remotely… If you setup a once per month full copy, and preform incrimental backups daily you’re looking at in the neighborhood of $130 per month. Beat that with a tape drive… And you dont outgrow the EC2+S3 combo in 3 months of prolific Word Document writing like you would that last tape drive 🙂

Version Control

Closely coupled with the development environment Version control is the heart and soul of any source based application, and theres no reason not to put this on something like an EC2 server. The bandwidth usage is minimal. The gauranteed backed up aspect is a huge load off the restless mind, and the next time you have to run reports on the last three hundred and seventy five thousand revisions… it doesnt bring your real (or test) environment screetching to a rather annoying halt!

Trouble Ticketing

You took the time to setup a nice trouble ticketing system for both your customers and your employees. It’s customized to the teeth. It rocks. It rolls. It just went down with the rest of your infrstructure just now when you blew a circuit breaker. Or did it? Not if you moved this piece of mission critical (and resource light) infrastructure to EC2. You’ll basically be able to run this puppy at the cost of $.10 per hour because the traffic itll cost you will be… what… $3 bucks a month? Change. And the ability to still communicate with all your customers during a major outage? priceless

Monitoring and Alerting

For roughly the same reasons as noted above using EC2 for monitoring your mission critical applications is the right thing to do. Not only do you have a third party verification f availability you have an off network source for things like throughput and responsiveness. If you’re monitor sends SMS through your mail server… and your mail server just died… you wont hear a peep out of your phone until someone brings it back up. And then when it *is* backup not only do you catch flack for not noticing… but all holy hell breaks loose on your phone as 400 SMS messages come in from l;ast nights backlog. Do yourself (and your organization) a favor… EC2 + Nagios + Mon = peace of mind (throw in cacti and you have answers to the dreaded question: “so how is our performance lately anyhow?”). Plus if you use something like UltraDNS which offers a programatic API for their services you can use amazon as the trigger for moving from one ser of servers to the other. Wonderful!

Some comparisons

Lets make some sample comparisons based on 1 year assumptions

High bandwidth (1 Year: 10 servers, 5Tb xfer per month)

  • Low end host – $12,000
  • Mid end host – $21,600
  • Amazon EC2 – $128,880

Low bandwidth (1 year: 10 servers, 200Gb xfer per month)

  • Low end host – $12,000
  • Mid end host – $21,600
  • Amazon EC2 – $8,920

Mixed solution

  • (
  • 8 servers @ amazon for processing (100Gb/mo),
  • 2 servers at a hosting co for bandwidth (2Tb/mo)
  • )
  • EC2 + Low end: $9524 (vs $12,000 hosted, and $9,280 at EC2)
  • EC2 + Mid end: $11,684 (vs $22,800 hosted, and $9,280 at EC2)

Eating Amazon EC2 for lunch :D

At lunch Feedster, and Technorati are neck and neck (well for this past 4 hour stint) with 5 EC2 entries each. Feedster has 3 of the 5 not being junk… Technorati has 5 of 5 not being junk… Of course nothing really new to report on those posts…

However 12f does make an interesting point in his article Innovative Amazon.Com Web Services

Let’s take a hypothetical case – a client that wants a 5 year hosting deal for an ERP solution. They will need servers, storage, and support.

* 20 CPUs
* 1 Terabyte of Storage
* 200 GB of Bandwidth consumption per month

Over 5 years, using Amazon’s infrastructure, your cost will be approximately $100,000. This is going to be far, far less than any comparable quote you would receive from a traditional hosting service.

This is going to save me a lot of headache, and some cash

Seeing as I just had a 75 ft network cable chewed through which connects my office (upstairs) to the internet connection (cable modem down stairs)… and that I just happen to have bought (a long while back) a second openWRT-able router this article: OpenWRT wifi bridge is going to save me headache, time, and money.

First off… buying premade network cable… isnt cheap… and second buying wireless bridges is also not cheap (in the neighborhood of $80 to $160) expecially which are capable of 54Mb/sec…

My last wireless bridge used for this purpose was an old 802.11b jobber which was sorely inadequate… it would choke on the number of connections I have a tendency to hape open… Throughput was *TERRIBLE*… Even though my internet connection is only 6M (8M now) the 11M bridge just could *NOT* cut the cake… And it ran me $80 at the time I bought it…

Heres to hoping that OpenWRT is just as good at bridging as it is at routing

I’ll keep you updated… perhaps I’ll do this tonight (I dont have to do it immediately because I happened to have an old 100Ft cable which is not in use any longer, since my wife dropped her desktop in favor of my old laptop)

I don’t think I agree

Blackfriars’ Marketings post: Amazon.com’s EC2 hosting service: technology diworsification mentions the following:

Amazon.com has fallen into a strategic trap. Instead of investigating more and better ways to serve its retail customers, it has decided to focus on being a technology company instead, presumably because technology companies often command higher valuations. This EC2 service (even the name is geeky) will distract the company’s energy when it should be gearing up for a tough fourth quarter selling season. Amazon survived the dot com bust because people could understand its business and its competitive advantages in retail. Moves like EC2 will make that understanding much harder now.

I agree that if Amazon starts advertising these services on the amazon.com storefront a lot of people will be confused about it… But I dont see Amazon doing that. If Amazon has gotten their infrastructure refined to the point that offering these services to themselves, internally, has not only become easy but trivial then theres really no reason that they cant make a market out of it. If you do something, and you do it well, theres no reason it shouldnt make you money.

I’d go so far as to argue that this move by amazon is probably one of the most gratifying things that most NetOps guys have seen in a long, long, time. Historically the network operation geeks are the atlases upon whose backs the web 2.0 coders tend to stand. And like mythical Atlas holding the world by himself they tend to go unoticed as more high profile names take credit for the entire job… To me, being an ops person at heart, it’s been immensly gratifying to see a company like amazon look at their Ops guys and say "hey, these guys are valuable… we really need these guys… and they can help our company both directly and indirectly"

If you’ve got it… Sell it 😉