We have SQS, we have S3, and we have EC2, so what next from the Amazon AWS team?.

There is really only one piece of the puzzle missing… And its a piece that has a lot of people griping. I have a strong hunch that Amazon is working on the problem, because I have a strong hunch that it is (or was) one of their major hurdles. And that problem is the database service.

How do you provide an easy to use interface to relational lookup-able storage? How do you make it universal? How do you make it secure? How do you make it FAST?

The first 3 questions are all answerable in roughly the same way: Make it a service, and let the service handle the interface, security, and universality. They’ve sucessfully applied the web-service to messaging, storage, and cpu power, theres no reason that this wouldnt be the final piece to the jigsaw puzzle. The last question carries with it the greatest problem, though. Allowing people to store data and run queries without the innevitable tanking of the server process would be a challange, to say the least (artificial intelligence is no match for human stupidity, after all).

But thats besides the point. If you break down into two components: anchors and tags — that is something is data or something is data about the data. provide a schema that works without collision problems, and – more importantly works both ways (finding tags related to an anchor, AND finding anchors relating to a tag) you cover probably 90% of peoples needs in one fell swoop.

I’ve been thinking a lot about how to do this, lately, as I’ve been drowning in a sea of data myself which is easy to manage in one direction but difficult in the other while keeping the size of the whole thing down.

Not only would that provide Amazon with the ability to have its finger in basically every new technological cookie jar BUT would provide huge massive gigantic enormous amounts of datas on what people really think about things. It would be an exceptional win for amazon, I think, and could indeed be leveraged to a huge advantage in the marketplace market. Because, as netflix has shown us recently, reliably finding things which relate to other things is *big* business.