I've been thinking along these lines for some time. Getting databases to work with ec2/s3 is too much work. A different approach has to be taken.

The thing I'm concerned about is deleting/updating entries in each bucket and the performance of retrieving data. Databases have the advantage that with decent SQL you can limit the quantity of data retrieved. Having buckets full of 'stuff' means that you will probably be returning more information than you need. With some thought maybe you can reduce this with decent hashing?

As your probably aware, s3 doesn't allow modifications to objects in buckets, so if I understand you correctly, the whole bucket would have to be initially retrieved (if it wasn't in cache) and serialized back to s3 once it had been updated.

Also you would also have to do a linear search through the bucket to get to the exact place the information that you want to modify/delete is.

Have you given any thought of the best way to store this information?

Thanks again for these amazon writeups… its really changing the way I think about databases and scalability!