Card performance issue

Question

If someone can help me, I am completely out of touch.

So, I have this code (this is a very simplified version of my code):

while(readNewFile()) { while(getNewStructFromFile()) { unsigned long starttime = GetTickCount(); customerData.fillFromBinaryData(structPointer); cout<< GetTickCount() - starttime; aMap.insert(pair<int,string>(customerData.phoneNumber,"")); } // Ouptut all data aMap.clear(); } 

Basically, it just reads the entries from the binary file. customerData get data and populate its variables with data from it. Then it inserts the phone number into the card (for debugging, I really just insert the int and the empty string).

The problem is that after a short time this program becomes very slow; if I comment on the insertion of the card, the program works fine without problems with a constant runtime for each file. If I use the card insert, after several files the program runs very slowly again (from 8 to 10 seconds to 1 minute or more). But debugging with GetTickCount() , it shows me that the delay occurs in customerData.fillFromBinaryData (first 0 ms, and then it jumps to 30-40 ms (to fill class variables)). But if I comment on this simple insertion of the map, there is no delay in filling the object with data! Where is the logic in this? Can anyone give me a hint, I have no ideas. Sorry if this question is not very good.

I tried different types of cards, but again, this shows me that the delay is not related to inserting a card.

Change / Possible Solution:

If anyone has similar problems, I installed VS2015 and the delay using the cards disappeared! I'm not sure how this is connected, but Hooray!

Answer 1

It may happen that if the card is very large, you will have a problem with memory management, and fillFromBinaryData will need some memory allocation, which will now be slower. Due to memory fragmentation maybe?

I would suggest trying some libraries for this particular purpose. However, I forgot what they are called. I just know that there is one available from Google, "jemalloc" or something similar.

The main point of a custom memory pool is that you can allocate memory once as a large group and use it only in your application area using custom allocators.

Another thing is maybe to stop using the card and use an unordered card instead. Change the time complexity to insert from O (logn) to O (1) with the perfect hash function, as there is a phone number for you.

• It is called jemalloc and it is not from Google. :)