How should I handle final consistency in SimpleDB, especially regarding unit testing?

Question

We are building a web application on top of the Amazon web services stack, and I still love it.

We also make full use of test-based development, and it also seems fantastic.

I just hope someone can help me with the problem I am having that is related to a possible Amazon SimpleDB sequence.

The best example of a problem is the unit test, which adds the user and then verifies that the user has been added successfully by making a call to retrieve this newly added user.

I could easily continue and just write tests for this, and everything could work fine, but I know about “possible consistency” and that when I make a call to retrieve the user, the user may not actually be added. Obviously, if the fetch select function is called and the user is not logged in, it will return false or fail.

What I would like to know is the best way to handle this? I saw suggestions for creating a function that sleeps for 5 seconds between requests and attempts 10 times. I also saw exponentially biased solutions. What is the best solution?

Answer 1

I recommend not using the actual SimpleDB service to test your own code. You will test your code + SimpleDB client + network + SimpleDB. What you need is a mock SimpleDB client to run unit tests. This way you only check the code that needs to be tested. Test-based development does not require you to test if the database works in unit tests for your code.

If you are testing your own SimpleDB client code, you can use the SimpleDB mock service or something like M / DB which is a SimpleDB clone that you can run locally.

But this leads to a bigger problem, because SimpleDB provides consistency in the sequence, rather than constant read-write. Your code will absolutely have to deal with the fact that a newly added item will not be immediately returned from a get request or request.

I have no reason to think that your code cannot handle this. I'm just saying that, as a rule, when you encounter problems such as tests, it indicates problems that need to be considered when testing code. You may find that you want either a general level of caching between your application code and SimpleDB, or you may need a session cache that can ensure "your record" consistency.

Answer 2

So, are you testing a database or code that goes against this? If you are testing code that goes against the database, then you should have tests that expect the code to add the user and then not return the user until, say, some random amount of time has passed. Obviously, this is most easily configured using a fake database, which tracks the amount of time since the last request and returns only the expected value after the specified time since the expiration of the initial request. You would have similar tests in which the fake is configured to never return a value, tune to return it immediately, etc. A tool that you fake to track interactions, and you can configure in your test how you want your code to behave so that your code follows the expected behavior (for example, polling at least twice).

Answer 3

First, as @Mocky stated, mistakenly getting a real database into unit tests is wrong. You can do this in integration tests, but they will work because all your test tests pass.

As for unit tests that are aware of a possible sequence, here are a few ideas:

• check that the user returns from simpledb after writing it (without the ec effect) to set the user to access the test identity

• check that the user does not return from simpledb after writing it (ec effect) to set the user to access to accept the error

• check that errors from ec are considered receive user + accept error test error is being processed

I was thinking about using a memcache layer (or elastic cache) to provide a cached user, so when users write to simplified, they are also sent to the cache. And when simpledb errors occur, the cache is checked side by side before the code decides that there really is no such user.

(I'd really like to hear what you think about this last paragraph)