Key Value Vaults Against RDBMs or "Cloud" DB (SDB)

Question

I am comfortable in the MySQL space by creating several applications over the past few years and then constantly improving aspects of performance and scalability. I also have some experience with memcached to speed up applications on frequently asked result sets. And I recently introduced Amazon SDB as my primary “database” for an e-commerce experiment.

To simplify, the quick excuse that I thought out for using the SDB service was that using a database structure without a schema would allow me to focus on the logical problem of my project and quickly accumulate content in my data warehouse, that is, don’t worry about setting up and normalization of all possible permutations of product attributes before distribution; just start downloading the products and the SDB will just remember everything that is available.

Now that I have managed to go through the first few iterations of my project, and I need to set up simple interfaces for the data, I’m running to the problems that I took for granted when working with MySQL. Example: grouping in select statements and constraint syntax for the query "items from 50 to 100". The benefit of lightness that I gained using the free SDB schema architecture, I lost the success of querying / looping a result set with just over 1800 elements.

Now I'm reading about projects like Tokyo Cabinet that extend the concept of key value storage in memory to provide pseudo-relational functionality at ridiculously faster speeds (I read somewhere 14x).

My question is: Are there some rudimentary recommendations or heuristics that I, as an application developer / developer, can go through to evaluate which BB technology is most suitable at each stage of my project.

Ex: At the prototyping stage, where logical / technical unknown applications make the data structure fluid: use the SDB. At a more mature stage, when end users are prioritized, use traditional tools where you don’t need to spend time editing, grouping or paginating.

Practical experience using these tools would be greatly appreciated.

Thank you SO!

Shahib R.

Answer 1

New solutions are not silver bullets.

Compared to traditional DBMSs, these systems improve some aspects (scalability, availability or simplicity) by using trade-off of other aspects (reduced query capability, possible consistency, terrible performance for certain operations).

Think that this is not a replacement for a traditional database, but they are specialized tools for a known specific need.

Take Amazon Simple DB, for example, SDB is basically a huge spreadsheet, if that's what your data looks like, then it probably works well, and its excellent scalability and simplicity will save you a lot of time and money.

If your system requires very structured and complex queries, but you insist on one of these wonderful new solutions, you will soon find yourself in the middle of reprogramming an amateurish, poorly designed RDBMS, with all its inherent problems.

In this regard, if you don’t know if they meet your needs, I think it’s actually better to do the first few iterations in a traditional DBMS, because they give you maximum flexibility and capabilities, especially when deploying on a single server and at moderate load. (see CAP Theorem ).

Once you have an idea of ​​how your data will look and how it will be used, you can meet your needs with an alternative solution.

If you need the simplicity of a cloud solution, but you need a relational database, you can check: Amazon Relational Database Service

Answer 2

The problems that you discover are why RDBMS experts are looking at some alternative yellow-eye systems. Yes, alternative systems cope with certain requirements very quickly, but as soon as you want to do something else with the same data, fleetest suddenly becomes lagging. In contrast, an RDBMS typically controls higher amplitude variations; it may not be as fast as the fastest for the specialized workload, which fleetest is micro-optimized for processing, but it rarely worsens so fast when it has to deal with other requests.