Hosting and accessing the HDF5 file on Amazon EC2, S3 and / or EBS

Question

I want to host a fairly large HDF5 file on the Internet, and I see Amazon S3 and / or EBS as a possible place to store it, and perhaps use an EC2 instance or two for processing in the background. I am looking for practical ideas on how this could be done or was done before and, if possible, a link to an actual example. I think this can be done in the same way as if someone was accessing a flat file in some other built-in data storage, but would like to avoid reusing the wheel if there is information available.

Answer 1

you can simply mount your S3 locally or on EC2 using FUSE:

http://code.google.com/p/s3fs/wiki/FuseOverAmazon

at this point, you can use all the usual libraries to access HDF5.