Create a scalable ASP.NET MVC web application

Question

I am currently creating an ASP.NET MVC web application in C #.

I want to make sure that this application is built so that it can scale in the future without the need for serious re-factoring.

I am very interested in using some kind of queue to publish any entries in my database and have a process that asynchronously polls this queue for the update. After this data has been sent back to the database, the client must then be updated with new information. An example here is that the process of writing data back to the database can take a short time based on business rules that are executed on the server.

My question is what would be the best way to handle updating from a client \ browser perspective.

I am thinking about sending data back to the server and adding it to the queue and immediately sending a response to the client, and then polling at some frequency to get the updated data. Any best practices or patterns in this case would be appreciated.

Also, in terms of reading data from a database, would you suggest using any specific methods or you could read directly from db enough, given my scenario.

Update I thought I would post an update about this, as it had been a while. We have actually finished using Windows Azure, but the solution is applicable to other platforms.

What we have finished is using the Windows Azure queue to send messages / commands. This is a very fast process and returns immediately. Then we have a working role that processes these messages in another thread. This allows us to minimize any db records / updates to web roles in theory, allowing us to scale more easily.

We process informing the user by email or even quietly depending on the type of data we are dealing with.

Answer 1

Not sure if this helps, but why don't you have an automatic page refresh every 30 seconds, for example. This is sometimes how news feeds work on sports sites, saying that the page will be updated every x minutes.

 <meta http-equiv = "refresh" content = "120; url = index.aspx">

Answer 2

Why not let the user manually query the status of the request? Here's how a typical e-commerce application is implemented. When you buy something online, the order is sent to the queue for full completion. After presenting it, the user is presented with a "Thank you for your order" page and a link in which they can check the status of the order. The user can view the link at any time to check the status; there is no need for an automatic polling mechanism.

Is your script different from this?

Answer 3

Sorry, I may have misunderstood my previous answer. I talked about the “queue” as something stored in an SQL database, but it seems that when reading a message you are probably talking about a separate component of the message queue, such as MSMQ or JMS?

I would never put a message queue in the interface, between SQL and the SQL server. Queues are useful for time scaling, which is suitable for backend components where deviations in processing time are acceptable (for example, order fulfillment) ... when working with users, this variance is usually unacceptable.

Answer 4

While I don't know if I agree with the logic of why, I know that something like jQuery will make your life a lot easier. I would suggest creating a RESTful web API that consumes your client code. For example, do you want to publish a new order in the system and respond to the customer? Make a message at www.mystore.com/order/create and return the new URI to access the order (i.e., order #) as the URI (www.mystore.com/order/1234). This response is then stored in client code, and the jQuery call is configured to poll to respond or stop polling on error.

For further reading, check out this Wikipedia article on the concept of REST.

In addition, you can consider Reactive Extensions for.NET and check the RxJS subproject, which has some pretty smooth ways with the polling problem, without forcing you to write the polling code yourself. Fun stuff to play!

Answer 5

Perhaps you can add a pending transaction area to the user interface. When you queue a transaction, add it to the user's pending transactions list.

When it completes, show that the pending transactions list of the user the next time they request a new page.

You can make the completed transaction stay indicated until the user clicks on it or for a certain period of time.