I mentioned a while back that I was going to be playing with the S3 Infinidisk Product. What I found in my testing was that this product is not prime time ready. There was a nasty bug which caused data to be lost if the mv command was used. The scripts themselves were unintuitive. They required fancy-pants nohupping or screening to use long term. Oh, and a database definitely will not work on top of this FS. It seems obvious in retrospect, but I wanted to be sure. InnoDB wont even build its initial files much less operated on the FS. To top it all off, My pre-sales support question was never even so much as acknowledged.
No, I think I’ll be leaving this product alone for now and sticking with clever uses of s3sync and s3cmd, thanks.
My name is Bill Donahue, president of Openfount. First of all, thanks for using S3InfiDisk. If you provide your email address, I'll check to see why your presales question wasn't answered.
However, there are some misconceptions in your other comments. S3InfiDisk is still in beta. It is being used heavily by many customers, but there may by still be lingering bugs with some operations. We will check on the mv problem, and will resolve the issue immediately. All of our customers are happy, particularly with the speed with which we address any problems.
The comment about no databases working on top of S3InfiDisk is incorrect. S3InfiDisk uses FUSE. FUSE does not support a small subset of file operations that some applications need. For instance, FUSE does not support locking or certain mmap operations. InnoDB probably uses these operations, hence it will not work under any file system built with FUSE. To generalize from InnoDB to all databases is wrong. InnoDB is the exception, not the rule.
We have customers using rsync with S3InfiDisk in large environments. And since it's the real rsync, you get the entire rsync, not just what a vendor is willing to support in reinventing the wheel. Note that mv won't work with s3sync by definition – it's a much narrower product. Using a FUSE file sytem, like S3InfiDisk, is clearly an advantage in most cases.
I’m unable to reproduce any mv problem. All my tests pass. Are you sure you’re using the latest version? There have been several updates to S3InfiDisk, and you need to make sure you’re using the latest one. You can download it at:
http://s3.amazonaws.com/openfount/openfount-fuse-s3.zip
If you’re using the prebuilt EC2 instance, after you download it, copy it to /opt/s3infidisk and unzip it there.
Can you send an email to [email protected] so that we can process this properly? It’s difficult to do this through a blog reply.