How much application session data can you spend?

Question

I currently have an application that gets more than 20,000 users daily, and basically they look at one data table. This data table is filled with approximately 20 rows, but is retrieved from the "datatable" in db with 200,000-600,000 records of information in the table. Edit: These 20 lines are “dynamic” and change if the user enters any information through a text field.

Currently, I also store user data along with profile data.

Currently, I am making about 4 callbacks each time a datatable is displayed, and I cannot get it before 1 call.

Question: I was wondering if I can actually fill out the state of the application every 5 seconds with 200,000-600,000 rows of data and will it really speed up the system? Edit: to do with dynamic lines that a user or any other user enters, the content needs to be updated frequently.

Question 2: How much can I store in the application cache and still slip away?

Edit:. With over 20,000 users accessing these 200,000 lines, I will need to cache all of them, or at least, I think, for best practices. When a user comes to my site, this is one of the main pages that they look at, and is likely to return 2-5 times per visit.

Edit: The user sees a unique set of 20 lines, which may differ from any other 20 lines that users see. This is a VERY dynamic site that can be updated several different lines approximately once per second.

Edit: If it is stored in session state, it will only speed up the number of times a person views the page. Not for all applications, because a person could view the page only once, and then leave.

Answer 1

Technically, I believe that what you want to do is possible, but I would not recommend it. Before you go this route, you need to consider several factors.

• Do you have equipment to support it? If you do not have memory for this configuration, and you have to redo the pages, then you are likely to lose most of the benefits associated with caching it in memory. If you use a process state server, then the system has overhead for serialization.

• How do you plan to search for something in this number of rows? The database server handles a lot of search and sorting for you backstage. There are some pretty complicated algorithms that they use that you lose if you cache data on a web server.

There is no real hard rule regarding when something happens faster in the database, and not in memory. It depends on how the application is configured and how the data is stored.

Answer 2

You say that basically they look at one table, and this table extends from 200 to 600K lines. How often is this table pulled? Is this a homepage scenario where users mostly view the first page of data? Why cache all 200K lines, why not cache the first 20?

Answer 3

Are you sure you want to keep this in session state? I would prefer Application State if they use the same database, so only one data set will be stored in memory.

I think the memory limit is controlled by IIS. There are restrictions on Maximum Virtual Memory and Maximum Used Memory . Remember to check data availability.

Check this out: Configuring ASP.NET Applications in Workflow Isolation Mode (IIS 6.0)

Answer 4

Can you clarify for me - you say that the user receives data from 20 records that are unique to this user and is the result of querying the 600K table? Are entries static to the user?

If there are only 20 entries that remain static when they are associated with the user, can you create serialized objects that can be passed to the user on request? That is, put them in a state in which they are ready to go, so you do not need to hit DB.