Extreme Thread Safe Collection

Question

I have a ConcurrentBag in .Net 4.5, in which I store about 4,000 rows from a database. I keep DTO.

My entire application relies on this. I have functions that return the whole list, as well as functions that return a single item. So many places in my code, I make LINQ queries in collections, etc.

I pushed all this to production, in a place that receives significant traffic, and immediately a 100% processor. I used the iis diagnostic tool and, of course, there were another 50 threads waiting for the ConcurrentBag at the dead end.

The documentation says that this collection is thread safe, but either it’s wrong, or the performance of this collection is not good, therefore it is not indirect for threads.

This collection, unfortunately, is not readable. If one of the functions that scans the ID returns null, it will go to the web service and add it.

I also converted it to ConcurrentDictionary and had the same problem. Locks for days in the .Values ​​property.

What is the fastest and most reliable solution in the most extreme scenarios?

private ConcurrentBag<Students> _students; public static ConcurrentBag<DestinyHash> GetStudents() { if (_students == null) { _students = new ConcurrentBag<Students>(); } return _students; } public static Student GetStudentByID(int id) { if (GetStudents().Any(x => x.id == id)) { return ... } _students.Add(getStudentFromDb(id)); return... } 

An example of use is placed throughout the application.

 Helper.GetStudents().FirstOrDefault(x => x.name == "foo" && x.status == "bar"); Helper.GetStudentByID(50); 

Answer 1

The simple answer is that you are using the wrong container.

ConcurrentBag not universal. It is intended to be used more as a pool of reusable objects, which you could (as a rule, as a last step) reduce to a single non-competitive value. One such problem for which it can be used is to summarize the list at the same time.

If your main use of ConcurrentBag disconnected from adding / removing, and you often list the collection, then you are using it incorrectly.

If you post more code, you will get more targeted help. Concurrency is one of those areas where understanding the problem is very important to ensure that the solution is implemented.

Edit:

ConcurrentDictionary will work for what you do. The trick is that you do not want to use ConcurrentDictionary.Values - this will lock the dictionary and copy its contents. If you just use its IEnumerable<T> interface, everything will be fine. For instance:

 private ConcurrentDictionary<int,Student> _students; public static IEnumerable<Student> GetStudents() { return _students.Select(x => x.Value); } public static Student GetStudentByID(int id) { Student s; if(_students.TryGetValue(id, out s)) return s; s = getStudentFromDb(id); _students[id] = s; return s; } 

Answer 2

msdn: All public and secure ConcurrentBag members are thread safe and can be used from multiple threads at the same time. However, members accessed through one of the interfaces implemented by ConcurrentBag, including extension methods, are not guaranteed to be thread safe and can be synchronized by the caller.