I am building a system for analyzing a large amount of financial data on securities prices. The big problem with this is determining which storage method to use for the data, that the data will be in 10 of terabytes. There will be many requests for data, such as obtaining averages, calculating standard deviations and amounts filtered by several columns, such as price, time, volume, etc. Joining statements are optional, but it would be nice to have.
Right now, I am looking at the community of monetdb and greenplum communities for evaluation. So far, they seem wonderful, but for more complex functions, some of them are needed in some of these releases (using multiple servers, insert / update statements, etc.).
What solutions would you use for this situation, and the advantages that it provides in comparison with the alternatives? Profitability is a major plus. If I have to pay for a data warehouse solution, I would do it, but I would prefer to avoid it and, if possible, take an open source / community route.
source
share