I am creating a website to display products and product categories. Data comes from an external service as a 500 KB file. A website is ASP.NET, C #.
XML is structured as a list of categories. Within each category there may be some products and / or some other categories.
Obviously, we cannot call this external service to get a large XML file for each page request, so we call it every few hours and cache it. I need to do things like this:
- Display the product category menu on the left side of the page.
- Show all products in the selected category
- Show extended information about one product
My question is as follows:
Firstly, on the page displaying all products in the "DVD" category, let's say I do the following (when loading the page):
XDocument allCategories = Cache["CategoriesXml"]; // loop through the XML and find the DVD category // Get all products under it, then display them
Bringing XML categories into a local variable (remember, this is 500k), is there a server leak? Remember that I will have to do this every time the page loads. At the same time, there may be thousands of people who look at different pages. If a thousand people download the same page in a few seconds, will I have a thousand copies of this XML file hanging in memory? Or will the garbage collector handle all this for me?
Is it better to quote directly over the cached item, or is it less productive (and / or bad practice)?
Secondly, I said that I am caching the whole XML file. I get products or categories from this XML by looping them (I use LINQ over XML). Would it be better to create a category type and a product type, put them in arrays and cache? Then loop through category objects and arrays, rather than XDocument? What will be more effective?
Thirdly, what would you say is the best practice in terms of how to architect this system. Say I have a data access layer, a business object layer, and a web application. Where should I put a link to an external service to get XML? On which layer should I cache it? Does this application even have a level of data access in the sense that part of the DAL is performed by some other system? Currently, my DAL is only available for access to our database, and it’s not good for me to post a link to the webservice there, but maybe not? Is it wrong to use caching in the business layer (i.e. interferes with unit tests, etc.)? I looked at the middle layer between the website and the business layer, exclusively for caching - is this a good or bad idea?
I’ve already completed most of this site, in fact - I just look back and wonder if I did it in the best way, so I want to compare your suggestions with what I actually did, and I hope I can come back and improve it.
Thanks!